Python Improved

A better Python .tmLanguage syntax highlighting definition for Sublime Text and TextMate. It includes support for both Python 2 and Python 3, and unlike any other Python syntax definition now fully supports Unicode identifiers anywhere in your code! It also provides its own improved regex syntax definition for inline highlighting of raw string literals.

Inspired by:

• the original TextMate and Sublime Text Python.tmLanguage files
• facelessuser's Better Python
• Peter Varo's Python 3 syntax definition

as well as a number of my own changes to make things more consistent and understandable. For customized syntax highlighting taking advantage of all the new scopes, use PythonImproved with the Neon Color Scheme, or modify your own favorite color scheme with the scopes below.

Installation and Use

If you haven't already, install Package Control, then select Python Improved from the Package Control: Install Package option in the Command Palette. To use PythonImproved as your default Python syntax, open a .py file, then select View -> Syntax -> Open all with current extension as... -> PythonImproved.

While I haven't yet tried to install PythonImproved with TextMate, I can't think of a good reason as to why it wouldn't work. You could try putting it in the same directory as the standard Python.tmbundle package, in the Syntaxes subdirectory. Then, just pick PythonImproved from the syntax menu.

New/Changed Scopes

If you prefer to modify your own color scheme, here is a list of new/modified scopes, along with some examples. It's not perfectly complete, but it's a start.

• support.ipython.in and support.ipython.out: IPython In [1]:/Out [1]: fields — designed for use with SublimeREPL. The cell number can be themed with a different color using support.ipython.cell-number.
• constant.numeric.integer.(long).binary.python: binary literals 0b00101010, 0b00101010L
• keyword.control.import.python now contains import, from, and as
• support.type.exception.python now matches any identifier that ends with Exception or Error, not just the built-in ones like IndentationError or RuntimeException, allowing for the highlighting of custom exceptions such as those included in third-party modules.
• Function annotation support for Python 3, thanks to @facelessuser. New scopes added: punctuation.separator.annotation.python, punctuation.separator.annotation.result.python, punctuation.definition.parameters-group.begin.python, and punctuation.definition.parameters-group.end.python.
• You can now have comments in multi-line function definitions:

python def myfunc(self, # gotta have self param1="value", # values are cool param2=True, # or False, whatever *args, # I'm here for an argument **kwargs): # you never know

• New scopes for bytes, unicode, and raw/regex strings, thanks to @simonzack: string.quoted.(single|double).(block|single-line).(bytes|bytes-raw|bytes-raw-regex).python
• Also from @simonzack, highlighting of self|cls in parameter strings: variable.parameter.function.(keyword|language)
• comment.line.note.python is a comment line that contains (BUG|FIXME|TODO|XXX) at the beginning. comment.line.note.notation.python matches the actual word itself, so you can differentially highlight the word and the whole line:

• constant.other.allcaps.python captures variable names that are in all caps (OPENING_PORT, for example), assuming the convention that these are generally treated as constants in the code. Matches CONSTANT, class.CONSTANT and the CONSTANT part of CLASS.CONSTANT, but not CLASS.function(), class.FUNCTION(), or FUNCTION().
• Fixed the octal integers so the Python 3-style 0o123 is matched as well as the old-style 0123
• Built-in functions like any(), dict(), len(), raw_input(), etc. now have their arguments highlighted just like any other function. Many thanks to @facelessuser for the regex, and @FichteFoll for valuable discussion. For those working with Python 2, print is still a standalone keyword (as are assert and del).
• support.function.magic and support.function.builtin have now been split in two — name and call, so that __init__ (support.function.magic.name.python), for example, can be themed differently than __init__() (support.function.magic.call.python).
• Relatedly, magic function names (and calls), also known as the "dunder" methods for being surrounded by double underscores, have been collated from the 2.7 and 3.5 Data Model docs and cleaned up so that as much as possible is included there, but outdated or incorrect things are not. The same is true of the magic variables (support.variable.magic).
• support.type now contains only what's defined in https://docs.python.org/X/library/functions.html and stdtypes.html (where X is 2 or 3) where the item is a class. They are highlighted as such only if not followed by an opening parenthesis — if it is, it's highlighted as support.function.builtin.call. This addresses #16.
• Defined escaped characters (like \n, \', \\, etc.) are now individually named as constant.character.escape.*, where * is newline, single-quote, backslash, etc.
• And probably some more stuff I forgot about...

Notes

• To facilitate hacking, I'm also including my .YAML-tmLanguage file in the repo, which I use for my day-to-day work (I really hate debugging regexes embedded in XML). Install PackageDev for syntax highlighting, and tools for converting between YAML, JSON, and XML/Plist formats. Neon of course has great coloring for the .YAML-tmLanguage format, and especially the regexes :)
• Speaking of which, for raw/regex strings, regexes will be scoped according to the accompanying Regular Expressions (Python Improved).tmLanguage file, instead of the builtin Python regular expressions definition. If you're using Neon for syntax highlighting (or any color scheme that highlights regexes), use a lowercase r to denote your string as containing a regex (i.e., r"\b(?i:(0[o]?[0-7]+))"). However, if you're just using a raw string literal to, for example, define a Windows path and you don't want regex highlighing for all the back slashes and whatnot, use an uppercase R (R"C:\Users\MattDMo"). Python can't tell the difference, but it will look nicer in your editor.
• All Django-related stuff has been removed. If you want it back, just dig through the repo's history and you can find it. It was just too distracting.
• I removed the SQL-related stuff from the string definitions, because 1) somebody complained, and 2) like Django, it was distracting. It didn't cover all of SQL, only highlighted some keywords, and just wasn't worth it.
• Unicode escapes should now appear correctly in all strings, as with Python 3 all strings are Unicode. I think I got it right, if you think otherwise just let me know.
• I've begun working on correctly highlighting all the various elements of the new-style string formatting mini-language, but I haven't applied it to the most recent release while I work out the kinks. Feel free to join the discussion.
• Now that the ST3 public beta supports .sublime-syntax files, I'm going to begin transitioning PI over to that format. If you'd like to contribute, chime in on this issue. One major advantage will be fixing this bug with raw string literals.

Issues

If you have questions, concerns, or suggested improvements, I'd love to hear from you! Feel free to open an issue or send a pull request and I'll get back to you as soon as I can. You can also email me at mattdmo@mattdmo.com or find me on Twitter @MattDMo.

License

© 2013-2022 Matt Morrison mattdmo@mattdmo.com.

This is free software. It is licensed under the MIT License. Feel free to use this in your own work. However, if you modify and/or redistribute it, please attribute me in some way, and it would be great if you distribute your work under this or a similar license, but it's not required.. A shout-out or a beer would be appreciated.

Support

Issues

New feature wish list for 3.0

opened on 2019-10-22 18:36:01 by MattDMo

PythonImproved 3.0 work has officially started... with this tracking issue. Gotta start somewhere.

• [ ] Move completely to .sublime-syntax (unofficial docs). The .YAML-tmLanguage files can be renamed .sublime-syntax without issues, or translated to .tmLanguage PLIST format with PackageDev.
• [ ] Make more and better use of meta patterns, variables, stacks, and other .sublime-syntax enhancements for readability/maintainability
• [ ] Make sure tests are comprehensive
• [ ] Clean up repo
• [ ] Python 3.6:
• [ ] F-strings, including = specifier from 3.8
• [ ] Variable annotations
• [ ] Underscores in numeric literals
• [ ] async, await in generators and comprehensions, are reserved keywords as of 3.7
• [ ] Python 3.7:
• [ ] New builtin breakpoint()
• [ ] Python 3.8:
• [ ] := walrus operator for assignment expressions
• [ ] Positional-only parameters

HTML output

opened on 2017-11-13 18:36:26 by ghandic

Hi,

I'm trying to make a parser that will create a html div of some code to put into blogs and for it to look like sublime's formatting. I manually made mine on my own website with some basic regex searches, I'm wondering if I could tap into the code from this to get a parser? If theres something out there already thats easy to modify then please tell me I'm wasting my time! Heres and example on my recent blog

http://www.andrewchallis.co.uk/category/shiny/docker-creating-a-portable-image-recognition-app-with-tensorflow-and-shiny/

It's just created with lots of span's and some css

[Fixed] #70

opened on 2017-03-28 08:32:58 by Kristinita

1. Behavior before pull request

See #70.

2. Behavior after pull request

Thanks.

[Bug] Scopes for first letters of words

opened on 2017-03-06 11:53:28 by Kristinita

1. Summary

Sometimes I get for first letters of words another scopes.

2. Settings

For example, I edit file color_theme.py, lines 13—26.

3. Expected behavior

The same scopes for e and tree in etree word, p and arsers in parsers word and so on.

4. Actual behavior

In etree word I have a scope source.python for e letter and source.python meta.identifier.python for tree letters and so on.

5. Environment

Operating system and version: Windows 10 Enterprise LTSB 64-bit EN Sublime Text: Build 3126

Thanks.

How to set background highlight for certain strings? (YAML/Sublime Text 3 themes)

opened on 2017-01-31 14:28:25 by MrVocabulary

Hi! I have created a custom theme and syntax highlighting for linguistic purposes that work very well — they change text color according to rather complex regexes.

However, I want to change the background color for given strings — any two or more spaces, to be exact. How to do it?

I am talking about such effect: http://imgur.com/a/gwftx

Parenthesis not matching with multi-line lambda logic statement

opened on 2016-10-27 21:44:22 by kamakazikamikaze

I have the following (ugly) statement

Python arrays = sorted(filter( lambda x: not any(y for y in ['NOMON', 'NEW'] if y in x) and any(y for y in ['-AN', '-P2P'] if y in x), [x['hostname'].encode().upper() for x in r.json()]))

Matching parenthesis do not highlight properly in PythonImproved, but work just fine with Python.

With the cursor at the end:

Cursor in the middle:

Cursor at the lambda:

PythonImproved does not treat any lines after this statement as though they were parameters to the filter or sorted methods, so it appears to be an aesthetic issue. (Code works fine.)

Releases

Parsimonious Wildebeest 2017-12-21 03:38:53

got rid of meta.identifier

Ostentatious Lawnmower 2016-07-24 21:32:15

Just changed Regular Expressions (Python Improved).tmLanguage's suffix to .hidden-tmLanguage so it doesn't show up in the Syntax menus, allowing for more convenient selection of the syntax.

Viable Tardigrade 2016-06-26 19:03:41

See messages/2.2.0.txt for changes.

self snafu 2015-12-07 18:42:33

Hopefully this is the last bugfix for self/cls highlighting

Fashionable Stallion 2015-12-05 19:22:00

This is a quick bugfix release for the 2.1 series. cls(arg=val, ...) now has proper highlighting for the function arguments. Some highlighting regressions which resulted from trying to address the above in 2.1.0 were fixed as well.

Philosophical Emu 2015-12-05 03:35:05

Lots of bug fixes in this version, as expected after a new major release. I suppose one of the most user-facing changes is the reversion back to the old style of highlighting raw/regex string literals. r"blah" will still be scoped as before, with regex highlighting (if any is present). However, you now have the option of declining regex highlighting by using a capital R (R"blah") instead. Python doesn't care if the R is in caps or not, it simply treats the string as a raw string literal.

Other changes include fixing magic functions (again) so that function calls are properly scoped as support.function.magic.call while magic function definitions are support.function.magic.name. If you use Neon the highlighting is still identical, for now. I also fixed an edge case where using cls as a function call broke highlighting of the arguments (self can now be used as well, obviously assuming __call__ has been implemented in your class). I also got rid of some unnecessary(?) code that wasn't, as far as I could tell, actually doing anything. If something breaks, please tell me. I've also added tests/test_code.py which is just a bunch of random snippets for testing. Finally, I've made some changes to the README, mostly updating (or adding) URLS, as well as detailing all of the above.