Cyrillic programming can improve performance
Hi, Habrahabr. As you know, technical English is the language of the world of information technology. Basic documentation, all programming standards are in English. Among others, the main ASCII code page and the portable character set include 26 Latin characters with which there are no problems when using different encodings. Historically, this has been due to the international level of English and US leadership in the field of information technology. This circumstance allows to achieve maximum compatibility of technology in the era of the Internet and globalization. In this article I do not set a goal to change standards, but just want to show an alternative approach to IT in Russian.
Initially, I simply wanted to build an ASCII-like 7-bit symbol table, including the entire Russian alphabet, 10 numbers and all punctuation marks from the POSIX standard character set, just for the sake of sporting interest. In the process, more and more it became clear that such a table is much more convenient than ASCII because of the precise definition of its subsections. I know that there is Unicode, but here the possibility of single-byte encoding is considered. In the process of creating additional benefits were found, the table was completely rewritten many times, control characters appeared, then one of the final results is presented.
')
What are the advantages? We will analyze in parts.
1. Obviously, the letters in full composition occupy the entire second half of the table. There was not enough space just for the letter E, it’s not critical, everyone has long been accustomed to her special status as a “distant relative”, although I personally would have removed from the list a solid sign for Kommersant, it is unclear why the linguists left instead of the more necessary E. therefore, as in ISO 8859-5 and Win-1251, it is allocated a place separate from the alphabet, the upper and lower case in the neighboring cells. Without this good letter, the Russian alphabet consists of 32 letters, adding a second register, we get 64 characters, or 2 ^ 6. Thus, one bit can determine whether a character is a letter and whether the middle register (about which later) applies to it. As it is necessary to the present letter number, it goes in alphabetical order, at the same time, first all lower case, then all capital letters. Why not the other way around, I will say later, the main thing now is that, thanks to the power of two, the borders of both alphabets coincide with the borders of the rows of the hexadecimal table, which is simply wonderful. The binary index of each letter in the array is equal to the number of the last 5 bits on the code page, the register is determined by the sixth, and the literal nature of the character is the seventh.
2. In the first half, the set of non-letters is also divided in two, twice: the same addressing is valid here, and an additional register is added, I called it “middle”. The top line is occupied by the control characters, here they are written in Russian, because the presence of Latin in the Orthodox Russian picture, contrary to all the rules, looked clumsy. Their appointment is not important, I am not a master assembler and all the more machine commands. Since the creation of ASCII, technological progress has gone far ahead, and most of the 70s control characters are no longer relevant. With this encoding, teletypes and many switches cannot work, but to use a computer with a modern architecture, 16 commands are quite enough if you need more, at the end a dedicated “UPR” command is specially assigned, upon receipt of which the device will perceive the next byte differently.
If it is short: - \ 0, - \ a, - \ b, TAB - \ t, - \ n, - \ v, UND - \ f, WHO - \ r, ONCE - space, KOH - signal on completion, SRT - stop command, BEP - upper case, shift, FIC - lock register, caps lock, AL - middle register, alternative letter assignment, alt, OTM - cancel, esc, UPR - control, start of command, ctrl. The teams selected are not the best, in particular, there is no necessary DEL command — delete, a process response request, but the NRM solves the problem. In addition, this article does not consider the details of the processor. It is important that the entire set of control characters is located in the first line and complies with the POSIX standard. For control commands, the registers do not work, the first 4 zero bits at the beginning disable any switching of the rest.
3. On the second line are two registers. The first, upper, changes the value of the third bit on the left from 0 to 1 and converts the reading to the fourth line. The second one, the middle one, changes the third and fourth bits and translates into the third one. You can change the second line from the fourth, then the “middle” register will be called rather the “bottom” one, and the switching will be done in one substitution. But I arranged the lines as arranged for better readability. For her, by the way, the left half of 3 and 4 lines is occupied by arithmetic signs, and the right half is punctuation. If someone decided to look at the table with special care, already noticed useful symbols ¬ (logical NOT, a very necessary thing) and ¤ (currency sign, for financial documentation) missing in ASCII, and also the native letter E, hidden without a certain key and two backup familiarity.
The order of characters in a row also matters: one column - one key. That is, the key coding of the keyboard is fully correlated with the table. Yes, the mechanical keyboard layout is normal here, but the functional is also its own, the fact is that the QWERTY standard was designed to maximize print slowdown. In the 1870s, there was no method of blind typing, and the Remington 1 typewriter had already gained commercial success. Its predecessors did not have a single layout, and when working on them, there was often a clutch of levers with each other. Christopher Scholes constantly experimented, trying to load his little fingers as much as possible and slow down the speed of typing text, thereby preventing grip. Often used letters and punctuation marks were difficult to access. Since then, the realities have changed, modern keyboards do not suffer from mechanical diseases, and the standard has remained, after all, relearn typing on a new layout is the same as putting the piano keys in reverse order. However, alternatives still exist, for example, the layout Dvorak. Russian layout was originally designed, based on the ten-finger method, we were more fortunate. But still, the punctuation marks partially inherited QWERTY, and the comma was in upper case at all. In the keypad encoding presented in the article, the numeric keys, and therefore the signs of arithmetic and punctuation, go from left to right. This takes into account the frequency of their use.
Returning to open questions, which is why capital letters follow after capital letters - the BEP team puts us two lines lower for all lines except the first. Since the addressing of characters 3 and 4 lines occurs through the registers of the second line, it is impossible for them to re-use the register by conventional methods (the shift or alt key is either pressed or not, increasing the pressure of the fingers from above is useless). The perception of the rule also plays a big role: if the third bit on the left is 0, then there is no register, 1 means upper case is turned on, small letters are replaced by large letters.
Thanks to these rules, the table can be restored from memory literally after the first reading, and the properties of a new character falling into the stream are determined when reading the first bits, which can be used to significantly increase performance. It is unlikely that this code page will find any use, programmers around the world have long been accustomed to English, and for ordinary users there is Unicode, American corporations are working on fundamental programming and PC architecture. This article is purely informal, although a smart interpreter of high-level PL (and low too) in Russian can still be created. And for even greater performance, to the detriment of readability, I present the second variant of the table, where the middle register is determined by one fourth from the left bit, and both registers are turned on by zero, not by ones.
Initially, I simply wanted to build an ASCII-like 7-bit symbol table, including the entire Russian alphabet, 10 numbers and all punctuation marks from the POSIX standard character set, just for the sake of sporting interest. In the process, more and more it became clear that such a table is much more convenient than ASCII because of the precise definition of its subsections. I know that there is Unicode, but here the possibility of single-byte encoding is considered. In the process of creating additional benefits were found, the table was completely rewritten many times, control characters appeared, then one of the final results is presented.
')
What are the advantages? We will analyze in parts.
1. Obviously, the letters in full composition occupy the entire second half of the table. There was not enough space just for the letter E, it’s not critical, everyone has long been accustomed to her special status as a “distant relative”, although I personally would have removed from the list a solid sign for Kommersant, it is unclear why the linguists left instead of the more necessary E. therefore, as in ISO 8859-5 and Win-1251, it is allocated a place separate from the alphabet, the upper and lower case in the neighboring cells. Without this good letter, the Russian alphabet consists of 32 letters, adding a second register, we get 64 characters, or 2 ^ 6. Thus, one bit can determine whether a character is a letter and whether the middle register (about which later) applies to it. As it is necessary to the present letter number, it goes in alphabetical order, at the same time, first all lower case, then all capital letters. Why not the other way around, I will say later, the main thing now is that, thanks to the power of two, the borders of both alphabets coincide with the borders of the rows of the hexadecimal table, which is simply wonderful. The binary index of each letter in the array is equal to the number of the last 5 bits on the code page, the register is determined by the sixth, and the literal nature of the character is the seventh.
2. In the first half, the set of non-letters is also divided in two, twice: the same addressing is valid here, and an additional register is added, I called it “middle”. The top line is occupied by the control characters, here they are written in Russian, because the presence of Latin in the Orthodox Russian picture, contrary to all the rules, looked clumsy. Their appointment is not important, I am not a master assembler and all the more machine commands. Since the creation of ASCII, technological progress has gone far ahead, and most of the 70s control characters are no longer relevant. With this encoding, teletypes and many switches cannot work, but to use a computer with a modern architecture, 16 commands are quite enough if you need more, at the end a dedicated “UPR” command is specially assigned, upon receipt of which the device will perceive the next byte differently.
If it is short: - \ 0, - \ a, - \ b, TAB - \ t, - \ n, - \ v, UND - \ f, WHO - \ r, ONCE - space, KOH - signal on completion, SRT - stop command, BEP - upper case, shift, FIC - lock register, caps lock, AL - middle register, alternative letter assignment, alt, OTM - cancel, esc, UPR - control, start of command, ctrl. The teams selected are not the best, in particular, there is no necessary DEL command — delete, a process response request, but the NRM solves the problem. In addition, this article does not consider the details of the processor. It is important that the entire set of control characters is located in the first line and complies with the POSIX standard. For control commands, the registers do not work, the first 4 zero bits at the beginning disable any switching of the rest.
3. On the second line are two registers. The first, upper, changes the value of the third bit on the left from 0 to 1 and converts the reading to the fourth line. The second one, the middle one, changes the third and fourth bits and translates into the third one. You can change the second line from the fourth, then the “middle” register will be called rather the “bottom” one, and the switching will be done in one substitution. But I arranged the lines as arranged for better readability. For her, by the way, the left half of 3 and 4 lines is occupied by arithmetic signs, and the right half is punctuation. If someone decided to look at the table with special care, already noticed useful symbols ¬ (logical NOT, a very necessary thing) and ¤ (currency sign, for financial documentation) missing in ASCII, and also the native letter E, hidden without a certain key and two backup familiarity.
The order of characters in a row also matters: one column - one key. That is, the key coding of the keyboard is fully correlated with the table. Yes, the mechanical keyboard layout is normal here, but the functional is also its own, the fact is that the QWERTY standard was designed to maximize print slowdown. In the 1870s, there was no method of blind typing, and the Remington 1 typewriter had already gained commercial success. Its predecessors did not have a single layout, and when working on them, there was often a clutch of levers with each other. Christopher Scholes constantly experimented, trying to load his little fingers as much as possible and slow down the speed of typing text, thereby preventing grip. Often used letters and punctuation marks were difficult to access. Since then, the realities have changed, modern keyboards do not suffer from mechanical diseases, and the standard has remained, after all, relearn typing on a new layout is the same as putting the piano keys in reverse order. However, alternatives still exist, for example, the layout Dvorak. Russian layout was originally designed, based on the ten-finger method, we were more fortunate. But still, the punctuation marks partially inherited QWERTY, and the comma was in upper case at all. In the keypad encoding presented in the article, the numeric keys, and therefore the signs of arithmetic and punctuation, go from left to right. This takes into account the frequency of their use.
Returning to open questions, which is why capital letters follow after capital letters - the BEP team puts us two lines lower for all lines except the first. Since the addressing of characters 3 and 4 lines occurs through the registers of the second line, it is impossible for them to re-use the register by conventional methods (the shift or alt key is either pressed or not, increasing the pressure of the fingers from above is useless). The perception of the rule also plays a big role: if the third bit on the left is 0, then there is no register, 1 means upper case is turned on, small letters are replaced by large letters.
Thanks to these rules, the table can be restored from memory literally after the first reading, and the properties of a new character falling into the stream are determined when reading the first bits, which can be used to significantly increase performance. It is unlikely that this code page will find any use, programmers around the world have long been accustomed to English, and for ordinary users there is Unicode, American corporations are working on fundamental programming and PC architecture. This article is purely informal, although a smart interpreter of high-level PL (and low too) in Russian can still be created. And for even greater performance, to the detriment of readability, I present the second variant of the table, where the middle register is determined by one fourth from the left bit, and both registers are turned on by zero, not by ones.
Source: https://habr.com/ru/post/349562/
All Articles