If you plant a thousand monkeys for a thousand typewriters, then for a thousand years they will write a terminal emulator. - instead of an epigraph.

Sorry false start, this is not me, this is an android smartbook.

When we started the cloud, the first problem was “how do we get the console”. The standard XCP mechanism implies that the console is drawn using VNCTerm, and those who want to see it must first go to XenAPI, get the console session-id there, go to the console port, transfer the session id, get the RFB wrapped in HTTP, deploy HTTP, remove RFB (aka VNC), give it to a local VNC renderer (VNC client or java applet with the same functionality). At the same time, the console was closed (the session was broken) with each restart of the virtual machine. She was torn even when migrating a virtual machine. In other words, it was a technology that meant "looked with one eye, repaired ssh / iptables and forgot." Uncomfortable, slow, difficult. I did not want to roll out such a product.
')
And I climbed into the wilds of serial-howto, console-howto and a few more terrible documents telling how to properly configure interrupts on the ISA card for multi-cards, as well as the specifics of setting linux-2.2 to work with them. At the same time, the console device was studied in zena (an attentive reader could even notice exactly when I more or less figured out this issue - I wrote a brief overview on what was happening with the console on Habré).

After that, the thought came: you need to write your own, because there is no ready-made good else's

At first we wanted to take at least ready-made components and make our own of them. I still remember that wonderful scheme in which we planned to save anyterm output to the database, do double tunneling of the serial port using UDP ... It looked, to put it mildly, unsightly.

Then I got the idea to cut out anyterm. To do this, it was necessary to see how the terminalki work. It was very funny and instructive (anyone can read the PuTTY source code). The main problem with this study was that they draw a lot on the screen. Right in the process of processing input. It was difficult to separate the specifics of DC from, in fact, what is a console.

After a while, we came up with the idea of ​​“we need our own terminal emulator”.
The task seemed relatively simple, until we touched the abyss, called "escape codes and terminal types ...".

Typewriters

So, in the beginning was the typewriter. At some point, there was a desire to combine the telegraph with a typewriter. This is how the teletype came about.
Of course, the engineers who created the teletype, there was no reason to do everything from scratch. They simply attached codes to each typewriter key. After some battles in the style of MS VS Netscape, the html5 standard was created for the codes for these machines, I mean teletypes. If my memory doesn’t change my memory, then it is ASCII, where all the key combinations that are typical for an American typewriter are provided. Including the BELL code, which, by the way, should not do BEEP at all, but do a zink, because the typewriters had a bell, not a speaker.



In the future, when mainframes appeared, they obviously output information to printers. Which were either teletypes, or printing devices similar to them.

When someone guessed to attach a monitor and keyboard to the mainframe, he naturally tried to ensure compatibility with existing printers. Then, when manufacturers began to make new models of terminals, they tried to provide support for at least the basic set of functionality of previous versions ... This chain dragged on for a very long time vt52 (incompatible with the subsequent ones) - and glorious vt100, vt102, vt120, vt200, vt220, vt320, vt420, vt520 ...

TTY emulation terminal emulator, typewriter emulator

Then, when writing Linux, to be able to run programs (like vim, emacs, or mc), there was nothing left to do to support the terminal emulator ... In the same way, by the way, UNIX and its clones went: solaris, FreeBSD, hp- ux, etc.

For the graphics mode, we wrote our own terminal emulators - the most advanced of them are xterm and rxvt (perfect - this means supporting everything, and not the most convenient / beautiful). By the way, the most common ones - gnome-terminal and konsole - support far from everything and far from everything that is supported, processed correctly (those who wish can run vttest on their favorite terminals themselves and look at this disgrace).

All these terminal emulators did the same thing - they emulated the operation of a terminal that emulates the terminals of previous generations that emulate a teletype, which emulate a typewriter.

Typewriters never die!

Living examples:

For example, all self-respecting terminals have the option to issue BELL 8 characters to the end of the line. This function is useless on the terminal - but very useful for a typewriter.

Why do we have separate carriage return codes (guess what kind of carriage the carriage is talking about) and "line feed"? Of course, because on typewriters these were separate actions (on the most advanced typewriters there was a lever combining CR / LF in itself).

A more subtle and non-obvious example. Why doesn't the Backspace character delete the character on the left, but only move the cursor, and delete delete the character on the right, and even shift the entire line?

Obviously: there was a backspace button on typewriters. And, since a typewriter can only move the carriage back and forth, but cannot delete characters, the backspace does not delete the character either. And there was no delete on typewriters - and there was no reason to do backward compatibility.

Dot matrix printers also left their indelible impression in Linux. The pager program can emulate a dot matrix printer. When, when outputting to a matrix printer, they wanted to make the text bold or underline, the carriage was shifted to the left and the text was printed again (or underscores were printed over the existing text). If you feed such text to the pager (pg) program, it will show it appropriately (it will emulate a dot matrix printer). However, I otveksya from the console.

Another thing that greatly influenced the modern console was how the picture was displayed on the screen. This affected the text mode CGA (which, in turn, had a lot of influence on Linux). To understand how this happened, we need to remember how primitive text screens displayed the text image:

Two areas were stored in memory — attributes and characters (in some implementations through one, in some - as two separate areas of memory). Color attributes were used as binary switches for supplying voltage from the R, G, B channels. The sequence of bits (the result of scanning a line of bits from the character of the font selected by the index (code) of the output symbol) switched between two signal variants — the background color and the tone color.

The “brightness” attribute that appeared later only added additional voltage to the final signal. The same explains why the “bright black” for the console, it is not deep black, but only gray. Well, quite unexpected is the fact that bright black in bright black is quite distinguishable, it looks like dark sulfur in almost black.

Another funny feature is the interpretation of the bold attribute - since the first terminals had only one font in the range, they couldn’t show bold (as opposed to dot matrix printers) - and the solution was to show bold as a bright color tone (characters).

The internal structure of the terminal has identified many other interesting points. For example, where should we go if the line and column numbers are passed to us outside of the allowed values? Obviously - the last allowed value. And if very large? Of course, we have coordinates in int8 / int16, so: it’s quite natural to do a ring overflow.

Another great thing that came to the 21st century from the 19th century is the tap stops. When typewriters began to improve, there was a problem of typing text in several columns. The solution was elegant - there was a long row of pins behind the typewriter, each about the width of a symbol. The pin could be in good condition or depressed. A special lever was used to push the pin into the current position. Another lever released the carriage (which was pulled by the spring) so that the carriage stopped at the given tabtop. Entering the text in this case looked like this: “enter text, press tab”, enter the following text. The most advanced typewriters, like Bashkiria-M, had buttons “2”, “3”, which allowed to skip several tabs. How this was implemented in mechanics - I do not know. In addition to the installation of tabstop (they, obviously, could be done on almost every column), there was a tab-reset lever that simply “squeezed” all the pins. And of course, all terminals, starting with vt102 and ending with linux-3.0rc2, support the esc-code for “unlocking tabs on typewriter tabs”.

our work

When we wrote the terminal a lot was not clear, for example, what are the tabstop after a reset? (Readers brought up in msdos incorrectly answer "8"). The answer is to look in typewriters. What are the tops left after pressing all the pins? Of course, no. Until you make them with your hands, nothing will appear. That is why the reset command, the first copyrights in which they were delivered before I was born, during the reset of the terminal sends the code “reset the topstop”, and then walks through the screen in 8-gap portions and arranges the so desired topstop every 8 characters. Although different types of terminals may imagine a different size of “tabs by default”.

In the course of our work on the terminal emulator, it is this “iron logic” that became clear only with memories of typewriters, terminals with 8kb of onboard memory, hardware signal sweep circuits, etc. Delivering the greatest torment for programmers. Actually, for a programmer, bobry is the author of our terminal emulation library.

Many generations of terminals, their emulators, etc. led to several standards of varying degrees of blurriness and the iron rule of the terminal emulator writer mimic legacy. No one has a complete documentation on the terminal code (even the vt102 manual, a huge Talmud, still has quite a few muddy places).

The situation with the documentation on the terminals can be described something like this: we have excellent documentation on the terminals and their codes, most of which are written in C and may contradict each other.

Frankly, I was not mentally prepared for such an endless nightmare of Legacy, when we took the terminal. That is, I theoretically knew that in Linux, the last maintainer sent a nafig terminal emulator code and left it unmainained, but I did not expect such an abyss.

About two months passed between the first working prototype and the release - and these two months all the time it seemed to us that for the most part it works, only trifles remain.

Actually, the main culprit of the celebration, the main and main author of the library is bobry , who courageously fought the abyss of Legacy, C and longjumps in the xterm code (under my accompaniment of trolling on compability with typewriters and amazing stories about the nature of tapstop, grandma "Bashiriya-3M" still came in handy, not for nothing that I was messing with her in the hungry and cold 90s around the years 90-93 ...). I can seriously say that he is currently one of the few programmers in the country (especially young programmers who haven’t found PL / I in the product), who knows well about how the Linux console works, knows esc codes and other horrors secret terminal emulation cuisine ... By the way, the name ncurses is a library for rendering images using esc codes, generally speaking, comes from English to curse - to express gratitude for a convenient and understandable thing.

However, I praise us again:

At the moment, our library is one of the best, and certainly the best among those that do not draw right away on the screen. By the way, this is the main feature of our library - it allows you to operate with the image of the terminal in isolation from the way it is drawn. This is what allowed us, for example, to easily and naturally write to the terminal a message stating that the virtual machine was forcibly completed (if the user kills the machine forcibly). The virtual machine itself is already stopped at this point - and we, with cunning sapy, simply append the completion message to it. Regular games with vncterm will not allow this.

In the next part I will tell you more about this library, in addition, there will be another drama of modern humanity - browsers and their understanding of what a “pressed button” is and how we made paste working in browsers.