Text editors, whose main task is to display a monospaced font (for example, a code), should, as the name implies, show characters of the same width.

But there is a nuance

In Unicode, there are characters that are not allowed to be seen. A text editor can simply render a text with such a symbol, or it can take some action to make it noticeable.

Who are they?

Word joiner , U + 2060

I replaced zero-width no-break space (U + FEFF), because U + FEFF was used to encode BOM (byte-order mark, several bytes at the beginning of the file, indicating its encoding and byte order). This symbol prohibits line breaks where it occurs.

Zero-width no-break space, U + FEFF

Obsolete character, replaced by word joiner, used for the same purpose.

Zero-width joiner , U + 200D

Used in Indian and Arabic fonts to combine characters that would not be combined without it.

Zero-width non-joiner , U + 200C

In faces with ligatures, you can insert it between letters so that the ligature is not:

It is even found on keyboards:

Zero-width space , U + 200B

It is used when you need to mark the boundary of words without inserting a space. This text will be carried by the words:

Word Word Word Word Word Word Word Word Word Word Word Word Word Word Word word word word word

And this one is not:

WordWordWordWordWordWordWordWordWordWordWordWordWordWordWordWordWordWordWordWordWordWord

Invisible Operators : function application U + 2061 , invisible times U + 2062 , invisible separator U + 2063

"Invisible Operators" added to Unicode 3.2. Needed to denote mathematical operations in expressions.

For example, this entry: A _ij
It can mean either the index (i, j) in a two-dimensional array, or the index i * j in a one-dimensional array. To eliminate ambiguity, you can use either the Invisible times or the Invisible separator, so that it is clear what was meant.

Similarly, f (x + y) is either a multiplication or a function.

Visually, they should not be different, but some parsers will be able to understand what was meant.

Mongolian vowel separator, U + 180E

From the name it is clear what he is for. This symbol has repeatedly caused problems . Very well described in this answer .

What it looks like

Of course, the display depends not only on the editor, but also on the font, let's look at rendering the text, without changing the settings of the editors.

Atom, Sublime, VSCode, Xamarin Studio, Xcode, Notepad ++:

Cat does not show them:

But if you run it with the -A option in linux or -v at macOS, then almost all the characters are visible (thanks for the help in the comments):

 cat -v invisibles.txt U+2060 foo?M-^A?bar WORD JOINER U+2061 foo?M-^A?bar FUNCTION APPLICATION U+2062 foo?M-^A?bar INVISIBLE TIMES U+2063 foo?M-^A?bar INVISIBLE SEPARATOR U+180E foo?M-^Nbar MONGOLIAN VOWEL SEPARATOR U+200B foo?M-^@M-^Kbar ZERO WIDTH SPACE U+200C foo?M-^@?M-^@?M-^@M-^Lbar ZERO WIDTH NON-JOINER U+200D foo?M-^@M-^Mbar ZERO WIDTH JOINER U+FEFF foobar ZERO WIDTH NO-BREAK SPACE 

Vim also does not report on some characters, even with the set list setting enabled, but less does better:

Web

GitHub, so these characters are shown in pull request ah and diff ah:

One of the popular code editors, CodeMirror:

In the same CodeMirror used by jsbin, in IE, some of the characters are visible:

ACE guesses that there is a bjaka, and says that something is unclean here, but what exactly it shows is not always:

Code editors and diff tools

Editors on IntelliJ platform:

Different code comparison tools for macOS (P4Merge, FileMerge, KDiff3):

KDiff3, attempt counted, but this is not enough.

SourceTree: does not handle text at all, bad:

Tortoise, too, is almost nothing:

git diff : well done, showed everything, also highlighted (although, in fact, made it less). Just fine, for diff tools, this is a role model:

Anguish: brainfuck, which is not

Someone made the Anguish programming language using only invisible characters. It is based on the brainfuck, but uses not the punctuation , but the characters we talked about above. There is even a Perl interpreter and usage examples .

Exploitation

Bad code, so be it, you can make a bookmark quite simply:

 function f() { //   ,    return 'access_denined'; } let code = f(); if (code === 'access_denied') { return 401; } 

What to do

Write a clean code,% username%. Follow the best practices, they came up with not just like that, but in order to keep fewer things in your head, including noticing such things in a timely manner. I saw a magic line, a strange or unverified default case, something else: there is time - do not be lazy, rewrite as it should. Conduct a code review, see what you commit to your turnip, maintain a good coverage. Remember that the line can be not only what is visible on the screen, check in the hex editor if a suspicion arose.

In general, the probability of implementing a backdoor through an invisible symbol, of course, is, but no more than yes: you can easily find it, and you can insert a bookmark into the govnokod using other methods.

Read

• Unicode Demystified A Practical Programmer's Guide to the Encoding Standard, by Richard Gillam (you know where to look) is a good book about unicode, much is told, including about such characters