When working with many programs, the use of various combinations of hot keys is the key to high performance and user convenience. For quite complex programs, we memorize dozens of specific key combinations for various actions. This allows you to focus on the actual work, rather than wandering around the multi-storey menu in search of the necessary item.

Therefore, the ideal memory for a combination of hot keys - muscle. As long as the combination is in muscle memory, it is not necessary during the work to switch attention to the performance of the corresponding action — it is done on automatism.

This is good, but this user input interface has problems.

Let's look at them on the example of work in the familiar to the majority of inhabitants of the habr environment - integrated development environment (IDE).
')
This is a complex software system that has many aspects of use (writing code, navigating through code, refactoring, compiling and executing, debugging, working with a version control system, etc.).

Each of these aspects has its own set of hot keys that are not used in other aspects (when I write code, I do not need to remember which key combination opens the window for viewing local variables in the debugger).

The first problem is that occasionally used combinations are forgotten.

The first problem is that if an action is done irregularly, then the corresponding key combination is likely to be forgotten by the next time it is used. At the same time, at the moment when this combination becomes necessary again, it can be used not once or twice, but rather actively for some time.

Example: let's say you decide to conduct a large-scale refactroing of your software system.

The main actions that will need to be done are making various semi-automated changes to the code (renaming characters, extracting pieces of code into separate functions, changing function signatures, etc.). In addition, you will often have to jump from place to place on the project, look at the various class diagrams, call hierarchies, etc.

This set of actions can be quite different from the one used in the usual writing of code, therefore, hot key combinations for such actions can be washed away from the head over time, since major refactorings are done irregularly.

To combat this problem, some programmers make various cheat-sheets with lists of commands that are useful in rare cases. When such a case occurs, you have to refer to the crib (or google).

The second problem is the complexity of migration between systems.

If for any reason you had to change the IDE, the hotkeys of the old development environment learned at the level of muscle memory can be a serious barrier. We'll have to relearn, and this is not a very pleasant and unproductive activity.

In addition, the old combinations in the new system can have completely different values, which is especially annoying.

The third problem is the sharing of multiple systems.

Everything can be really bad here. The same actions in different systems may have different combinations of hot keys. In this case, you have to constantly keep in your head a set of combinations for each system. In addition, you need to mentally monitor all the time that the system is used right now. The key thing here is “to keep in mind” - you can forget about muscle memory, which will lead to a drop in performance.

Suppose you are debugging a client-server system, where the server is written in C ++ in Visual Studio, and the client is an application in java in IDEA or eclipse.

The server forms a packet with data, sends them to the client, which somehow processes them. First we need to figure out whether the server side is generating packages for sending. Then, after sending the data by the server, it will be necessary in the debugger on the client side to watch how the received data is processed. Moreover, server communication with the client may not be limited to forwarding a single packet. In this case, you have to jump back and forth between debuggers more than once. The main actions that will be done by you in this case are the commands for step-by-step execution of the program (step over, step into, step out). Conceptually, these actions are the same, but in each of the development environments they have their own functional keys:

	VS	eclipse	IDEA
step into	F11	F5	F7
step over	F10	F6	F8
step out	Shift + F11	F7	Shift + F8

In this case, for example, 'step into' in eclipse is done with the F5 key, which in VS is responsible for the 'resume execution', which can knock down the entire debugging process by negligence.

In such situations, I often noticed for myself that I start debugging almost exclusively with a mouse — this is much slower, but I know for sure what kind of action and in what environment I am doing now.

purpose

Creating a keyboard input emulation system that simplifies the use of programs that have a large set of actions initiated by using hot key combinations.

System requirements:

• No need for the user to remember the actual values ​​of the hot key combinations.
• Ease of sending user commands to the application. The interface should allow organizing teams in such a way that they can be easily and quickly found and executed.
• Easy system setup - the user should be able to adjust the interface to fit his needs, limit only to relevant commands at this particular moment.
• The ability to easily reuse command configurations between similar applications.
• The ability to automatically control that the target window for a command is active at the moment of command execution (to avoid situations such as the problem described above, the F5 key in eclipse and visual studio debuggers).
• Cross-platform application - I wanted to abstract not only from the actual combinations of hot keys, but also from the operating system.

The main idea of ​​the system is that a server is started on the computer, which is responsible for generating keyboard commands. He works in the background and sends commands to the window that is currently active. The android device that is used for user input / output is connected to this server. All commands are initiated by pressing the buttons displayed on the android device. When this happens, the server sends the appropriate key combination to the active window on its side.

Since the matching key combination for this action occurs automatically on the server, it will be possible to define several rules for such matching. This allows you to work consistently with different programs (if the sets of actions produced in them are similar).

Server configuration

For this system to work, you must have the following information:

• Data about the environments in which we work and for which we want to emulate hot keys.
• Data on hotkey combinations. It is necessary for each action to specify a combination of hot keys that initiates it in each of the environments.
• The configuration of the elements on the client device: the location, type and behavior of the buttons that will be used to send commands.

Data on hotkeys in different environments

The data from the first and second points are stored in a simple table (text-based csv file with tabs as separators), which looks like this:



In it, each line represents all the information about a particular team. The first four columns are service. They store data that is used by the program to build the user interface:

1. command_id - a unique string representation of the command in the system
2. command_category - a string representation of the category to which the command belongs
3. command_note - a short description of the command that will be written on its button in the user interface
4. command_description - a more detailed description of the command.

All other columns of the table - information about the combinations of hot keys in various environments. The format of the combinations is described in the documentation of the library that I use to emulate the keyboard. The format description can be found on the library documentation website .

Building user interface

The second text configuration file is used to write the user interface. The interface consists of a set of pages. The pages are buttons, each of which corresponds to a command.

0. simplest option

Here is an example of the simplest configuration of the page and how it will look on the client device:


In the configuration of the interface, command identifiers are simply written - one per line. This translates into such an interface for the user:

1. Several buttons in a row, the gaps between the buttons

If in the configuration you specify several elements separated by semicolons in one line, then the corresponding buttons will be placed in one row. Between the buttons, you can also make empty spaces (for this, it is enough not to specify the command ID in the appropriate place).

The following configuration illustrates these things:


The interface in which it is broadcast to the user:

2. Reuse of buttons

There are cases when some action is not defined for one of the environments, but I don’t want the place to be empty. Perhaps there is some other similar enough action that can be placed in the same place in this case.

To do this, you simply need to write in order identifiers separated by commas in the appropriate place. The system automatically scans the list and selects the first of the options that will be active in this environment:


The appearance of the interface in various environments:


Notice how the 'restart' button in the VS environment turns into a 'stop debug' environment for eclipse, since the combination for 'restart debug' is not defined for eclipse.

3. Options

Many teams in fact are whole groups of commands that are similar in meaning, so you want to combine them together in the client UI, but you don’t want them to take up too much space. For example, you can give commands like 'build something': 'build solution', 'build project', 'compile file'. For such groups of commands, you can define a special page that will automatically appear and disappear when you click the group button. This special page differs from the standard only in the title - its body is the same.


An example of working in the UI:


This approach is very convenient with commands like '[do a certain action] with [something]', where exactly one of several options is chosen for [something].

All options for the command's scope are transferred to an additional page and automatically displayed when needed.

Option pages can also be embedded one into another, forming a deeper and more extensive structure.

Binding to a specific input window

In order not to keep track of what window is now active (so that the simulated keyboard input does not go to anyone), you can ask the program to control it. To do this, the system starts the 'stickEnvToWindow' option.

In this case, when selecting a new environment, the user will be shown the following query:


The user will have to make the window he needs active and inform the program about it by clicking the button at the bottom of the screen of the client device. After that, whenever the user requests the execution of a command, the server will check whether the correct window is in focus. If this is not the case, the execution of the command is suspended, and the following stub is displayed to the user:


In this case, the user must either return the focus to the required window and request the execution of the command again, or cancel the command:

Keyboard Automation

Naturally, this approach can be used not only for hotkeys. Any frequently repeated keyboard operation can be automated using this program.

For myself, I regularly use such things:

Insert words or phrases in another language

Often when typing Russian text I have to switch to the English layout for one word, or even a couple of characters. The Russian layout does not recognize the existence of the following characters: '<> [] {} $ # @ & ^ ~, some of which are needed regularly. In addition, in the Russian text there can be repeated terms in English, and, if it is html, then also tags.

Simulation of typing in a different layout can be implemented simply by adding commands to switch the keyboard layout before and after the actual typing of inserted text.

For example, to print the <html> tag, you need to use the following combination: '+ {COMMA} html + {PERIOD}'. It will work correctly if the English layout is enabled. If the Russian layout is enabled, then you need to add a hotkey to switch the layout to the beginning and end of the combination. In my case, this is '% {SHIFT}' (Alt + Shift). Thus, if the Russian keyboard layout is enabled in the text editor, then the following sequence of characters: '% {SHIFT} + {COMMA} html + {PERIOD}% {SHIFT}' will print <html> there.

Work example:

In-place conversion of the highlighted word.

Suppose I need to wrap some variables in a macro or function call in a large number of places in the code. For example, one of the functions needs to change the type of the parameter, so in all places where this function is called, you need to convert what was transferred there to the new type.

If such changes need to be made at least a few dozen, then the idea of ​​automating it becomes very attractive.

Steps to do in this case:

1. select a variable by double clicking
2. simulate the cut command (Ctrl + x)
3. we simulate typing - the name of the wrapping function and the breaking parenthesis
4. simulate the command "paste" (Ctrl + v)
5. close the bracket

Steps 2-5 are automated by the following combination: '^ {X} my + function + name + 9 ^ {V} +0'. Work example:


In addition, it is possible that the form of these calls may differ slightly in different places of the code (it is possible that somewhere you need to specify the namespace of the called function, pass additional parameters, etc.). All these variations are easy to implement and have on hand.

Conclusion

The system is still raw - it’s rather a prototype, not a final version. But, as it seems to me, already in its current state, it is able to simplify and increase the efficiency of working with hot keys.

I did not touch upon the technical details of the implementation in this article in order not to inflate it. If there is interest, it will be possible to make an article dedicated to this.

The implementation code is on my githaba . There is also some more formal documentation about the formats of configuration files and command line arguments.

If you do not want to download the source code and compile the code, there is a version compiled for windows (you will need to install Microsoft Visual C ++ 2015 Redistributable to start).

For self-compiling the source code, you need C ++ 14 - a compatible compiler and boost (using boost :: asio and boost :: program_options). The documentation has a description of how to build a project using gcc and Visual Studio.

I would be happy to hear criticism about the implementation, as well as suggestions for what can be added to the system. Besides, I don’t really like the format of the UI description configuration file. If anyone has any ideas, how can you better imagine such an interface (preferably in plain text format), please comment.

What else would you like to add in this system:

• Ability to execute several commands in one request (this will seriously improve the functionality of the simulation of typing).
• Automatic tracking and switching keyboard layout of the active window.
• Improving the functionality of working with configuration files, adding new types of configs.
• Refactoring program code.
• Improving the system of messages to the user about the problems in the configuration files (now the messages are not always informative).
• Bug fixes simulation of some key combinations (see below).

All links in one place:

• Project on githaba , exe for windows.
• Additional repository with command configuration files for various tasks. There you can also find examples from this article: link .
• Libraries used in the project: boost , Robot , tau (my article about it).
• Microsoft Visual C ++ 2015 Redistributable - you need to run the compiled version.
• Client application for android .

PS

Problems and non-obviousness that should be known before use:

1. Only UTF-8 encoding is supported. If the program feeds the config in a different encoding, it can curse a rather non-obvious message. Please consider this and save the files in the correct encoding.
   
   
2. In windows, there is a problem with simulating several combinations of the Shift + 'smth' format. It appears for the PgUp, PgDn, Home, End, Insert, Delete keys and arrows. The combination of the type '+ {INSERT}', which should simulate Shift + Insert, in fact, simulates just pressing Insert.
   
   
3. In the configuration examples, many hotkey combinations were not tested for performance. I just google a few lists and merge them into a common file. Therefore, errors are possible there. I would appreciate reports of problems found in configs, as well as pull requests.