Virtual creatures and their habitats: the past and present TTY in Linux

Ubuntu is integrated into Windows 10 Redstone, Visual Studio 2017 has gained support for developing for Linux - even Microsoft is losing ground in favor of the growing number of Torvalds supporters, but you still don't know the secrets of the virtual terminal in modern distributions?

Do you want to fix this space and open the source code? TTY, MASTER, SLAVE, N_TTY, VT, PTS, PTMX ... A heap of concepts, virtual devices and random magic? It all adds up to a rather logical picture, if you remember how it all began ...

1. START FROM SCRATCH & KEEP CALM

TTY: PALEOSIC

We stepped straight into the thirties of the 20th century and found ourselves in the still very young Teletype Corporation. Directly in front of us is He-S-Who-All-Started — a teletype, which is a “direct-printing telegraph,” which transmits text messages between two subscribers.

Subscriber A dials characters on the keyboard that are converted into electrical signals. By the most ordinary cable, the signals “run” on the teletype of subscriber B and are already printed there on the most ordinary paper. If the signal is duplex, then we are very lucky, and subscriber B can immediately write his answer; if not, then he will need to first connect the second wire for feedback.
')

Here, at Teletype Corporation, they still don’t know what the future is in store for their product, and they certainly don’t suspect that the TTY will outlive the TTY abbreviation. We will not spoil the intrigue for them, let's go further.

TTY: MEZOZA

Forty years have passed, we are in the laboratory of Digital Equipment Corporation, admiring the first mini-computer (interactive!) PDP-1. For the input and output of information, as well as to ensure user interaction, the already familiar teletype is connected to it.

The fact is that the leading engineering minds decided not to reinvent the wheel and adapt the already existing cheap and accessible mechanism for new needs. The teletype was directly connected to the computer (and not to another teletype, as it was before) and they called this case a console . The operator performing the input sees how the characters typed are instantly printed on paper, but this happens without the participation of the OS - thanks to the typewriter principle stored in the console .

TTY: PALEOGEN

We find ourselves at the very beginning of the 80s, this time at Bell Laboratories. One of the most important releases of the “early” UNIX version 7 for PDP-11 has just been released here. The features of this release are as follows: the user-entered command is now displayed on the ECHO principle (the character typed on the keyboard first enters the accumulation buffer and only then does the OS send instructions to print this character), simple editing options for the commands entered are supported (you can “erase” the character or whole line, move the caret), there is a separation of modes:

• raw mode (line editing is not performed; control sequences are recognized as normal characters; the entered character is immediately transferred to the process);
• cooked mode (special characters are recognized and the stop and interrupt signals are generated for the process; the finished line is transferred to the process only after pressing the Return key).

Quite an expected question: how can you "erase" what has already been printed teletype? To visually perform Unix Version 7 editing operations, you can print certain characters: for example, @ - erase the entire line, # - erase the last character. That is, if our teletype typed ld @ lk # s, and the operator pressed Return, then the ls command went to be executed. This is not TTY LINE DISCIPLINE (it will be discussed further), but a big step forward in terms of processing input at the OS level.

By the way, for these 20 years Digital Equipment Corporation not only developed the aforementioned PDP-11, but also thought about how to improve the teletype: the so-called smart terminals appeared.

We look to the right: this is the VT100, one of the first terminals able to work in love and harmony with the PDP-11 and supported by Unix Version 7.

At the OS level, both the console and the smart terminal are now perceived as character devices that are connected via the UART interface, which converts the asynchronous data stream into a sequence of characters. For the OS, they are basically identical, the only difference is whether to erase the characters on the terminal screen or print the face character using a teletype.


The left shows a general scheme of interaction between the computer and the console (or smart terminal). This scheme has one drawback, which is not at all pleasing to the operator of the PDP-11 console: one session (or session) is associated with one console, in which the user can start several processes in the background, but only one at a time will be active on one TTY one.

And our poor operator is forced in the literal sense of the word to move from one console to another, if he suddenly comes up with a few sessions.

Tty: neogene

We found ourselves in the editorial office of PC MAGAZINE, we are considering the latest issue of January 13, 1987. One of the reversals actively convinces us not to spare money on a PC with UNIX System V. What are the arguments? In particular - grep, awk, sort, split, cut, paste, vi, ed - word processing has clearly stepped forward. And the most interesting: terminal emulators are now at our service! Thanks to virtual consoles, it is already possible to run as many as four sessions without the need to connect more and more new physical teletypes.

In addition, the life of monochrome terminals can now be colored: it supports CGA, Hercules, EGA graphics. The poor-man operator can breathe easy, the teletype (as well as the smart terminal) in the form of an iron beast threatens him only in nightmares.

We will only advise the operator: never go into the / dev directory - after all, there are several ttyX waiting for him and remind you that the same good old teletype lives under the hood of the virtual console.

TTY: ANTHROPOGEN

At the previous step, we were convinced: the console was made virtual (there will not be a solid edition of PC MAGAZINE to lie). What does it mean - the entire I / O mechanism is rewritten and completely changed? Then why is the virtual device still ttyX? It's simple: the virtual console is emulated as the most physical, and the place of the UART driver is probably taken by someone else. The changed scheme will be discussed in detail a little further.

There is only one step to the finish line, but we cannot even skip it out of respect for Linus Torvalds. Now is the year 1993, and we finally have the good fortune to review the source code for Linux 0.95. Why this particular release? It is there that the TTY-abstraction has already formed, which is closest to what we have in the latest distributions: three distinct layers formed ( TTYX - TTY_LINE_DISCIPLINE - TTY_DRIVER ).

In addition, Linux 1.0 will be released just a year later, where the window interface provided by the XFree86 project will appear. From this point on, virtual terminals will be added to the virtual console , which the user (in almost unlimited quantities) will be able to run without leaving the graphical shell ... However, before plunging into the subtleties and details of the improved io-magic, let's return to our present to Ubuntu 16.04.

2. STOP BEAT AROUND THE BUSH & LOOK INSIDE

VIRTUAL CREATIVES AND PLACES OF THEIR HABITATION

Only some devices in the / dev / directory are used daily: / dev / sdaX, / dev / mem, / dev / zero, / dev / random ... But there are several groups of devices that do not often attract our attention, but more than deserve it. These devices are / ttyX, / vcsX, / vcsaX, as well as / ptmx and / pts / X. In fact, they will be discussed further.



And our first object is a virtual console. Each such object has at least a sacred number identifier and a totem animal virtual device / tty file, of which 64 are found in the virtual forest of the / dev directory.

Check if we have the opportunity to talk with them. We execute Ctrl-Alt-FX (or chvt X, where X is the console number, for example, Ctrl-Alt-F1) and notice that X can be equal to 1, 2 ... 6. At the same time, we open a virtual console when we first start we are prompted to enter a username and password and create a new session for us. If X is 7, then we return to our native graphics cores and understand that / tty7 is associated with XServer. Go ahead. Eight, nine, ten and so on up to 63 - do not give signs of life.

The fact is that in Linux there is a macro MAX_NR_CONSOLES (64), which defines the maximum allowable number of virtual consoles, which are represented by 64 virtual device files / dev / ttyX. However, the last word is left behind the parameter ACTIVE_CONSOLES (/ etc / default / console-setup), and this parameter defaults to six.

Initialization of consoles occurs in several stages. First, the kernel, after receiving control from Grub, during the initialization of the subsystems, calls the “console_init” function, which creates the primary console, the “boot console,” designed to display debug information. This console performs the output of characters in the most primitive way: through “putchar”, which directly accesses the BIOS, initializing and filling in the biosregs structure, and outputting a character to the console using the 0x10 interrupt.

Later, during the execution of “fs-initcall” and “console-initcall”, virtual devices and structures are created under 6 full-fledged virtual consoles - “real console”. The activation of these consoles is performed by the first / sbin / init process launched by the kernel, which launches the getty program, which reads the /etc/init/console.conf and /etc/init/tilX.conf configuration files and subsequently displays the contents of the welcome file etc / to the console issue and launch login. Next, XServer initiates console activation on dev / tty7, on which the graphical shell runs.



However, we still have questions. What an unknown object / dev / tty0? And if each / dev / ttyX is a virtual console device, then why do we need / dev / console and / dev / tty? For the answer, go to tty1 (by pressing Ctrl-Alt-F1) and run the following script in the background:

sleep 10 echo “tty0” > /dev/tty0 echo “tty” > /dev/tty echo “console” > /dev/console 

Then go to, say, tty4 and wait a few seconds. After the expiry we see the following picture:


The distribution of roles becomes clear: / dev / tty0 = / dev / console = current console, i.e. both are always associated with the console that we currently see in front of us, and / dev / tty “remembers” the console with which the process started. Therefore, while our process was running, / dev / tty0 and / dev / console were defined for it in the course of the play, depending on the current active console, but / dev / tty remained unchanged.

Do not rush to leave the directory / dev. There are a little more than a dozen of curious objects: / dev / vcsX (virtual console screen) and / dev / vcsaX (virtual console screen with attributes). Another experience: move to tty5 and leave some traces of your stay, then go to any other console (even if its number 3), do “cat” on / dev / vcs5 and see exactly the state of console 5, in which we left her a few seconds ago. In this case, respectively, / dev / vcs3 and / dev / vcs (as well as / dev / vcsa) belong to console 3, on which we are at the moment.

We understand that / dev / vcsX is nothing more than a pool of memory, a device of virtual memory of the console, which allows us to move without loss between instances of tty. Paired with it is / dev / vcsaX, which provides basic information about the screen state: colors, various attributes (eg flicker), current cursor position, screen configuration (number of rows and columns). Summing up the scheme:

TOP VIEWS BETTER

Now we’ll stop with the study of tty zoology for a while and move on to the tty-abstraction itself, of which our virtual devices are a part. Let's look at the general structure of the tty-complex and select three components:

1. / dev / ttyX is a virtual console device in the file system that took the place of the UART driver and with which we are already familiar. At the same level, the devices / dev / vcsX and / dev / vcsaX are located, communication with them is carried out directly through / dev / ttyX.
2. TTY Line Discipline is a driver that makes ECHO a recruited command and allows us to edit it. Also, the driver of this layer generates signals when typing control sequences (^ C, ^ Z, etc.). By default, N_TTY reigns here, but this module can be replaced, for example, with its driver - let's experiment with it a little later;
3. TTY driver is a driver that provides a set of methods for initializing and opening the console, as well as methods that process input / output operations, suspend the console when switching and restarting its work, and, of course, provides for “transferring” the command received from the user to the active process.

Remember the complaints of the operator PDP-11? He did not like to waste time on transitions from one physical console to another. Now the situation is as follows: we have 7 virtual consoles by default, and in front of them - an office chair on wheels (of course, also virtual). When we switch from one console to another, the operating system moves our chair to the desired tty, and together with the chair "switches" to it and the complex of physical io-devices: the monitor now has the state of our new console, it also receives keyboard input and etc.

At the same time, the processes from the first tty continue to work: they read commands from the file of their virtual console, write to this file, but - since they are torn off the “chair” - they do not receive any events (the same ^ C and ^ Z) and - because the physical devices “left” along with the “chair” - they can only accumulate their “output” in the buffer in order to send it to the monitor as soon as the “chair” returns.

TRAVELING TO THE CENTER OF THE EARTH

Yes, from above everything looks quite presentable. But you,%% username, probably want to see how the three-level interaction of tty-components is implemented directly in the code? For the answer, you will have to descend from heaven to earth, even better to say - underground, into the depths of the Ubuntu source code (we will work with the kernel of version 4.4).

Let's make a preemptive move - let's see through which structures the tty-abstraction is linked into a single whole.

First, it is “tty_struct”, which has a field “tty_ldisc” (this is the structure of the 2nd layer driver methods), a field “tty_driver” (this is the 3rd layer driver) and immediately “tty_operations” (this is the structure of the driver methods 3rd layer, for the sake of convenience, rendered directly to the "tty_struct").


That is, “tty_struct” provides access to the TTY_LINE_DISCIPLINE and TTY_DRIVER layers. Got access to it - 2/3 stack of tty-abstraction, consider, in front of us. Now we need to understand how the transition from the virtual device files to this very structure is carried out. The answer is simple: the “tty_file_private” structure just has a “tty_struct” type field. Therefore, referring to the virtual device file on the 1st level, we can easily access the levels higher.

While the puzzle is taking shape, but this is not enough for us. Let us run the kernel (using qemu and cgdb) and consider the backtrace of the output (echo) of a single character entered by the user from the keyboard:


So, we are at level I of the tty stack. The “write” system call occurs, which is processed by the “tty_write” function on our tty. It is passed a pointer to the structure of the virtual device file and a buffer with a symbol. In the tty_write function, the tty_struct instance is received from the file. When accepting the game, “tty_struct” first of all causes the TTY_LINE_DISCIPLINE driver - “tty_ldisc”, the default location of which is N_TTY. The first level is passed!

N_TTY accepts the baton: in turn, calls the “n_tty_write” method, and then transfers the buffer to the “output_process_block” function, which, having made sure that we have entered no carriage positioning symbol, asks “tty_struct” to call “tty_driver”. That's right, we go to level III.

“Tty_struct” successfully plays the role of an intermediary, and now - the “con_write” method of the tty driver named “console_driver” is already running. A driver of the third level would be happy to do his job, but he is alone, many consoles - which one should you work with? The tty_struct comes to the rescue again and hands the driver the necessary instance of the vc structure (it is responsible for the state of its specific console and contains its keyboard, screen settings, as well as a set of graphical display methods).

"Console_driver" locks the console and calls "vc_data" to finally execute the echo character. “Vc_data” realizes with horror: they turned to her not for the sake of the question of the state of health of the console entrusted to her, but for the sake of action. This means only one thing: it's time to call for help the “consw” methods, which in our case represents VGA (with a different kernel configuration, this could be, for example, a framebuffer). And for sure - VGA quickly takes on the case, hides the cursor, prints the character, scrolls the screen if necessary or switches to a new line, moves and displays the cursor. “Vc” exhales: “console_driver” accepted the job and unlocked the console. We can breathe out and we, because all three levels have been successfully completed, each component has completed its mission.

3. KEEP AN EYE ON VIRTUAL TERMINAL

But that's not all: it's time to get acquainted with representatives of another class of inhabitants / dev - with terminal emulators . These are the same xterm or gnome-terminal, which we run from a console equipped with a graphical shell, using, for example, Ctrl-Alt-T or Ctrl-Shift-T.

They live in a separate aviary / dev / pts (= pseudo-terminal slave) and are files with numbers 0, 1, 2, etc. Perform ps in the current terminal and see - we are on / dev / pts / 1. Press Alt + 5 - we move to our fourth in order of opening the terminal, the virtual device file is / dev / pts / 20. At any terminal we are met by bash, with each terminal is associated with its many processes. No surprises yet.



But note: / dev / pts / X are created dynamically, run on the same console / dev / tty7, do not require login to start, and the most interesting is that there is no “chair” rule: we can open several virtual terminals and simultaneously observe how work is done on each one of them. Once again,% username% may give rise to doubts: does the principle of tty-abstraction persist here and how does the device fit into this principle — the slave, which is probably controlled by a certain device — the master?

The new object does not keep you waiting: in a single copy, the ptmx file is in the same / dev directory (In fact, it may not be the only one: if more than one console with a graphical shell is running). By mana, when opening / dev / ptmx, the subordinate part of the / dev / pts / X pseudo-terminal is created, which is associated with its leading part “ptm” (accessed via the file descriptor, but the real file is not created). Then "ptm" is passed to the functions grantpt and unlockpt, and after all this you can open directly / dev / pts / X, which will behave exactly like a virtual console (except for the features described above).

For us in the idea key of tty-abstraction, this means the following: when a user wants to start a terminal emulator , XServer requests / dev / ptmx to create a virtual device / dev / pts / X. The powerful “multiplexer” / dev / ptmx kindly does this, assigns the device file to the terminal instance and ... / dev / pts / X takes the place of / dev / ttyX, it is assigned the TTY_LINE_DISCIPLINE layer driver, it is gently received by TTY_DRIVER. The stack above / dev / pts / X takes on the usual form. The task of studying the terminal emulator mechanism smoothly reduces to the previous history with a virtual console , but its detailed study requires a separate article (which is included in the plans for the future!).

4. LET'S PLAY WITH TTY LINE DISCIPLINE

For a moment, remember the tty- "Paleozoic": there was a time when the TTY_LINE_DISCIPLINE layer was not a separate layer and did not have a full-fledged modern functionality. Let's try to estimate the weight of the changes that have occurred since then.

First, make sure that we are really dealing with N_TTY:


Everything is relative, therefore we act cardinally and with the help of stty we disable all the useful features of N_TTY:


The result clearly demonstrates the area of ​​responsibility of N_TTY, without which the output is not formatted, input is not displayed. Moreover, if we open a new terminal, then we will be convinced of the integrity and integrity of its LINE_DISCIPLINE. The resulting effect makes me think: we know exactly which component handles all our input, we can modify it for each virtual terminal separately and, I remember, we heard that this component can be replaced by loading your own module.

Unfortunately, N_TTY itself is part of the kernel. Therefore, we take as a basis the other drivers of the LINE_DISCIPLINE layer, provided in Linux and loaded as modules. In their image and likeness, we modify the source code file n_tty.c:

1. Add a module shipping function, in which the tty_register_ldisc function is called, which “acquaints” the kernel with our personal line of discipline. In this function, the first parameter is its unique identifier, and the second is a pointer to the structure with driver methods.
2. Add the “unloading” function of the module, in which the tty_unregister_ldisc function is called, respectively.
3. In the tty_ldisc_ops structure itself, we will give a new name for the driver.
4. We will make sure that our module “learns” the functions it needs from the tty_io.c file (it does not please us with the EXPORT SYMBOL macros, which forces us to either finish all the required functions manually or link together with tty_io.c).

Now we add some functionality that distinguishes our line of discipline from the original. Remember that it is TTY_LINE_DISCIPLINE that processes service sequences, so it’s a sin not to conjure in this field. To do this, open the function "n_tty_receive_char_special", in which TTY_LINE_DISCIPLINE checks whether the entered characters are not special and, when it is found, sends the appropriate signal. For example, we swap the signals generated for Ctrl + Z and Ctrl + C:


After that, we receive from our modified file the kernel module our_ldisc.ko directly. Download it, make sure the download is successful. Verify that "our_modyfied_ldisc" actually registered as TTY_LINE_DISCIPLINE. Open the terminal and see the number of pts. After that, we assign our driver responsible for the / dev / pts / X TTY_LINE_DISCIPLINE layer:


Let's set up a new line of discipline with the help of the “stty echo cooked” command - now the terminal is working in our usual mode. Run the test program with an eternal cycle and compare the effect of Ctrl + Z and Ctrl + C:


We have achieved the desired: generation of signals is redefined at the driver level of the TTY_LINE_DISCIPLINE layer individually for one terminal emulator! There is a field for fantasy work: from magic tricks with processing utility sequences to a custom command filter.

FINALLY

Now for you,% username%, secrets of virtual consoles and terminal emulators are no longer secrets, random magic is not magic, but a technology that has traveled a considerable way to create a flexible tty subsystem, and a teletype is not an artifact of antiquity, but an invention (by the way , our, domestic), without the descendants of which the modern computer somehow does not want to be presented.

We love to tell fascinating stories. Do you want to listen to them live? Come to the "confrontation" NeoQUEST-2017 , there you will find many interesting reports : from "hardware" to cryptography! Admission is free when registering on the site.