Configure Eclipse PyDev for debugging Python 3.x with honest Unicode and Cyrillic

One day, on a cold winter evening, I suddenly remembered that I had not shared a simple secret how to achieve simple Python happiness beyond the limits of an ASCII reservation in such a wonderful-free IDE like Eclipse with the PyDev plugin. And happiness with debugging and honest Unicode, which means the following: if you call your variable Cyrillic, you can see its value by setting a breakpoint, write a couple of lines of text in Russian and nothing will fall off.
Yes, dear reader, Eclipse PyDev is not very friendly with characters outside of 0x7F and debugging likes to fall off every time you try to read the value of a Cyrillic variable. Yes, what's there, just pointing the mouse at a Unicode character leads to fatal consequences when debugging code written in Python 3.x (UTF-8). If your file system settings are different from UTF-8, congratulations, you cannot even run your script. I mean exactly what, for example under Windows, your wonderful script with a single word in Russian will simply disable PyDev.
Maybe I overdid it, exaggerating, do not worry, we can fix it ourselves, just by reading this small instruction. As a reward, we will receive a free development tool, quite convenient, fantastically flexible in settings and improvements, up to development in several languages, with the built-in versioning tool.
Under the cut instructions and indecent size images.

We are preparing the spaceport

Our future universal spaceship called Eclipse is downloaded without Python development tools, for this there is an Eclipse PyDev extension, we will install it immediately after downloading. From the download page, select the version of Eclipse you like. I would recommend “Eclipse IDE for C / C ++ Developers”, after all, Python works very well with extensions in C / C ++, and the code itself is perfectly integrated into C ++ thanks to the Boost.Python library.
When the download is complete, unzip the eclipse folder to any location on your computer that is consistent with your feng shui.
Just run eclipse \ eclipse, most likely it will just start, but if you have never used the JDK before, it will request it for work, it will not start without it. The JDK is the Java Development Kit, downloaded from the Oracle website, from this download page .
When you start, Eclipse asks where to save all the projects to it, point it to your Workspace (remember this word, you need to look for it in the settings menu). Finally, after booting, this Java child will show you your Welcome, don't be scared, just close it.
Somehow this will look like this at the end of our excursion:



In the meantime, let's start with the most important, with the fact that the fuss is with encoding. The most important thing is to make sure that your default workplace is set to UTF-8 encoding.
Go to the menu Window => Preferences select there General => Workspace at the very bottom of the page you need to specify the " Text file encoding " parameter, if it is not set in UTF-8 by default, write this string value in the Other field, as shown in the picture:
')


So we have everything to start developing the application. Oh no, not everything, Python is still nothing to write and debug. Need a PyDev plugin.

Getting ready for landing on Python-3

Installing PyDev is probably the easiest part of the job of setting up a workplace.
Go to the " Help => Eclipse Marketplace ... " menu and type PyDev in the search bar and click the [ Install ] button in the found plugin



After that, we agree with everything, we are waiting for the download and installation, then we will be offered to restart Eclipse, and why restart, they are asked.
After installation, we are invited to play hide and seek with the installed environment - Perspective (Perspective) for development in Python, I will tell you where to find it, at the same time I will switch to the PyDev perspective. In the upper right corner there is a row of small buttons for switching perspectives, usually there is something like C ++, Debug, Java, SVN and a little scarlet plus sign to the left of them, so you need to click on it to add PyDev perspective.



In the Open Perspective window that opens, select the Python development environment — freshly installed PyDev.
We have to switch to the PyDev perspective right after the selection. Some changes should occur in the menu and in the main window, which you may not notice right away, but they switch the creation of new projects and files and the debugging process to the Python development mode.
Almost everything. We are ready to set up and launch our first Python 3 project in Eclipse PyDev.

First project

Well, let's create it, our first project.
Menu File => New => PyDev Project, fill in the name, choose the version of Python 3.x and now the most important thing: you need to specify where the Python interpreter is located, for this you need to click on the link "Please configure an interpreter ..."



Everything, we can click Finish.

Our first script

It's time to try to run our script, and at the same time find out what this article is for.
The “File => New => PyDev module” menu, or simply right-click on the “New => PyDev module” project, give our new script a name and click Finish.
The script file will automatically be created in the encoding specified in the Workspace settings that we set at the very beginning. In Python 3.x, all code is standard UTF-8 encoded.
To check Unicode support, we write this script:

 = "  !.." print() 

It's time to debug it!
Zhmakayem on the green beetle (De-Bug) or simply F11. Choose Python Run (this is clearly not a unit test). Go…
... We arrived! ..



Hehe, but for free!
We look into the file where we fell and see a banal jamb with the encoding in _pydev_execfile.py
Open it by reference in the Eclipse console frame.
The very first line in the file seems to hint that for Python 3.x here it’s not just a horse that was not lying, even the pony didn’t fit.
This is what our wonderful file looks like:

 #We must redefine it in Py3k if it's not already there def execfile(file, glob=None, loc=None): if glob is None: import sys glob = sys._getframe().f_back.f_globals if loc is None: loc = glob stream = open(file, 'rb') try: encoding = None #Get encoding! for _i in range(2): line = stream.readline() #Should not raise an exception even if there are no more contents #Must be a comment line if line.strip().startswith(b'#'): #Don't import re if there's no chance that there's an encoding in the line if b'coding' in line: import re p = re.search(br"coding[:=]\s*([-\w.]+)", line) if p: try: encoding = p.group(1).decode('ascii') break except: encoding = None finally: stream.close() if encoding: stream = open(file, encoding=encoding) else: stream = open(file) try: contents = stream.read() finally: stream.close() exec(compile(contents+"\n", file, 'exec'), glob, loc) #execute the script 

With a critical eye, we look around the battlefield and find several jambs:
1. For Python 3.x encoding, there will always be None, which is fatal in Windows where the file system settings are far from UTF-8, and indeed it is wrong to require the first line to specify the encoding from a file with Python code 3, which is always UTF-8.
It is ruled by the banal check sys.version_info [0]> = 3
2. For files starting with BOM, the first character is not truncated, and compile is very sensitive to everything extra, it simply does not understand the characters at the beginning of the script.
Without going into subtleties, you can quickly fix everything for example like this:

 def execfile(file, glob=None, loc=None): import sys if glob is None: glob = sys._getframe().f_back.f_globals if loc is None: loc = glob if sys.version_info[0] >= 3: encoding = 'utf-8' else: stream = open(file, 'rb') try: encoding = None #Get encoding! for _i in range(2): line = stream.readline() #Should not raise an exception even if there are no more contents #Must be a comment line if line.strip().startswith(b'#'): #Don't import re if there's no chance that there's an encoding in the line if b'coding' in line: import re p = re.search(br"coding[:=]\s*([-\w.]+)", line) if p: try: encoding = p.group(1).decode('ascii') break except: encoding = None finally: stream.close() if encoding: stream = open(file, encoding=encoding) else: stream = open(file) try: contents = stream.read() if sys.version_info[0] >= 3 and contents.startswith('\uFEFF'): contents = contents[1:] #Remove BOM finally: stream.close() exec(compile(contents+"\n", file, 'exec'), glob, loc) #execute the script 

After this little fix, everything is just fine with us. It remains to tell about debugging and a small parameter, without which our starship will not take off beyond the execution of the code.

Debug mode and Unicode

Put a breakpoint on the second line, we will look at the contents of the variable word - this is done by double clicking on the area to the left of the desired line of code.



We start the debugging mode on F11 or by clicking on the green bug in the toolbar. We get up at breakpoint.
(By the way, lyrical digression, when switching to a Debug perspective, take a couple of minutes, drag everything unnecessary down.)
Now we try to do the following: hover the mouse over the variable word . After all, you probably wonder what's inside? ..
... oh, they fell again! .. This time, completely mysteriously - the script's work was completed, but the process of debugging itself died in torment.
But for free! ..
Although everything is pretty simple here - go to eclipse.ini next to the executable module and add the line there:

 -Dfile.encoding=UTF-8 

Save the yinch, restart Eclipse for the changes to take effect.
Everything.
You can now debug as much as you want!
Call the variables even Cyrillic, even in Hindi, call in Russian classes and methods, whatever!
I'm not saying that this is right, I just give you this opportunity, or rather, I’m returning what was taken away by the voracious free Eclipse PyDev.

useful links

I decided to give a link to the archive with the Eclipse PyDev configured for development in Python 3.x: eclipse.7z (~ 171 MB)
After downloading, you will still have to do the following:
1) - migrate the Workspace folder
2) - configure in Window => Prefereces => General => Workspace encoding UTF-8
3) - open the PyDev perspective (initially hidden) through the Open Perspective
4) - when creating the first project, specify the location of the Python interpreter

Conclusion

You probably ask me why I didn’t immediately, in the first paragraph, download a ready-made archive with patched Eclipse with PyDev.
I will answer that I decided to share the experience of an uncomplicated, but very useful solution, giving an impressive PROFIT in development in Python 3.x.
It costs you nothing to go through the steps and get a great development tool, and at the same time understand some subtleties in the work of Python 3 and the difference with the old and not always good Python 2.x.
Remember: you have no reason to limit yourself to ASCII and generally limit yourself to anything.
Good luck in your development!