Octodon: going deeper underground
In this essay, I would like to slightly move away from the history of the Oktodon keyboard and talk about the real problems that we encountered at different stages of prototype development. So it turns out that you never know how difficult the docking of the high-level API (in our case, the Android API) and self-made hardware will be. Pitfalls lurk literally at every turn, and you can never even approximately estimate how time-consuming this or that task. Considering, in addition, that time is limited - there is always some exhibition or presentation on the nose, to which the prototype should already work, solutions have to be found by all possible means.
photo by Patrick Pleul / AFP / Getty Images
... and the result is rarely elegant.
I participate in the creation of Octodon as an Android developer, electronics engineer and radio installer. It so happened that despite the fact that in the past few years my main activity is high-level development on .NET, my childhood was made happy by the ZX Spectrum computer and, accordingly, the assembler for the Z80 processor. As a result, I began to understand too early how the computer works at the level of signals and timing diagrams, and this could not fail to impose a print - I still really love low-level development. And when I was offered to participate in the project of creating a fundamentally new keyboard, a “piece of iron,” I simply could not refuse.
')
Hacks (and not to call otherwise), about which I will now tell, I am not at all proud. But they allowed the project to grow and develop, allowed to have not a bare idea, but a working implementation capable of attracting stakeholders, future users and investors. At this stage of development, all this code is no longer used, do not worry :)
It is possible that in order to create a more complete picture of what we are doing at all, it will be interesting for you to read the previous post written by our leader AlexLysenko : Keyboard Oktodon in search of the Right Click
Our first prototype used Bluetooth sharing. It would seem that there could not be a more convenient technology for connecting the keyboard and the phone, and there was confidence that everything would work out of the box. But this did not happen.
The first problem arose from the fact that, starting with version 2.3, the support of BT-keyboards in the system appeared in Android. As soon as the system detected a “paired” keyboard, it immediately captured it and began to use it as an input tool. Since we were going to do the same, we inevitably would have printed two characters, one for us and the system one, with one click of the Octodon joystick. It was necessary to somehow muff an overly talkative Android. The phone was “rooted”, and digging into the source code of the kernel and system libraries began in order to find the most convenient place to ignore the keyboard connection. That place was the library libui.
In the open_device function, a dirty hack of the form was implemented:
As you can see, we just don’t let us open our device. Now the OS has stopped taking away the keyboard from us, and it has become possible to communicate with it using the Bluetooth functions of the Android API. And here the problem was again waiting: the BT socket for receiving data could not be opened at all - the standard createRfcommSocketToServiceRecord function for this fell with an error. Annals of history did not preserve details, but the problem took a lot of time. As a result, it came to the reverse-engineering of products from (then still) Market, which allowed working with this BT-keyboard on “old” Androids who did not know how to do it themselves. And here, behind the barricades of obfuscation, an excellent thing was found in one of the programs:
Call a private type constructor! The surprise was even greater when the submitted piece of code took and immediately earned. Understand what, how and why there was no time, so the code was left right in this form. This is one of the drawbacks of low-level development - it is often necessary to accept the fact that part of the code works “magically”: no one knows why it works, and no one knows why a more normal solution refuses to work ... Fortunately, we quickly left this Bluetooth keyboards, having made only one prototype with its use, therefore, we managed to get rid of such “magic” soon.
As soon as it became clear that the use of ready-made devices severely limits us, it was decided to develop our own. The main smartphone on which we focused, by this time was the Samsung Galaxy S2. He had a normally and stable USB host, to which we were going to connect. As the controller was taken Teensy 1.0. It turned out not to be at the right time that its capabilities were not enough for us, and that something should have been taken from a more recent version, 2.0. The difference was in the libraries for creating HID-compatible devices. If 1.0 could only pretend to be a HID keyboard or a mouse, then on 2.0 it was possible to implement a much more versatile Raw HID device using our own protocol. But what to do, first had to be content with what was. When emulating the HID-keyboard, the problem described above arose again - when connected, the system itself captured it as an input device, and the characters sent were printed twice - by us and the OS.
The solution described above was no longer suitable - we decided that we would have had enough of the changes to the system. If we want to create a device that can be released to the market, then it should work on any Galaxy S2, there should be no need for re-flashing.
In the process of experimenting with a conventional USB keyboard connected to the S2, it turned out that for some reason the smartphone does not respond to keystrokes in the digital part of the keyboard. This was a great help, because our Teensy could somehow encode the deviations of the Octodon joysticks into numeric key presses. For each change in the state of the joysticks, we began to send parcels of 3 bytes:
Thus, simply pressing the first joystick to the left gave us the “00+” sequence, releasing - “00-”. Without releasing one joystick, you can reject the second one - in this way you can encode simultaneous presses of several buttons, which we really needed for the Shift mode.
However, there was another task to somehow get the transferred scancodes. The system API ignored them, so I had to go deeper. To our great happiness, Android is * nix, and that’s all - the file. The required file was in / dev / input and, depending on the location of the stars in the sky, either event7 or event8 was called. Only root had access to it, so I had to rant the device, but that was all - I didn’t need a flashing. The use of root in Android itself is an interesting task, since there is no way to get the corresponding rights through the API (or we could not find it). Therefore, we do everything right through the terminal:
The prototype, which worked in this way, existed for a long time, went through many exhibitions, and is still alive today. Over time, the code became more sophisticated, and the names of the device files instead of the hardcode began to be set via the config file.
Well, then came the golden age. For Galaxy S2 came out the 4th Android, and the ordered Teensy 2.0 came to us. The docking of the device developed on the basis of the Raw HID controller with the USB Host API that appeared in the new OS went so smoothly that it didn’t fit into the framework of this essay, which was intended to tell about problems. An old dream came true, Octodon could work on the “stock” smartphone without having to do anything with it. We were able to produce small-scale samples that just work, and they do not need to be brought to mind with a file. But beautiful solutions without crutches are a topic for a separate article, especially since the Habr pages didn’t tell about USB Host API, it seems, never. So to be continued!
photo by Patrick Pleul / AFP / Getty Images
... and the result is rarely elegant.
I participate in the creation of Octodon as an Android developer, electronics engineer and radio installer. It so happened that despite the fact that in the past few years my main activity is high-level development on .NET, my childhood was made happy by the ZX Spectrum computer and, accordingly, the assembler for the Z80 processor. As a result, I began to understand too early how the computer works at the level of signals and timing diagrams, and this could not fail to impose a print - I still really love low-level development. And when I was offered to participate in the project of creating a fundamentally new keyboard, a “piece of iron,” I simply could not refuse.
')
Hacks (and not to call otherwise), about which I will now tell, I am not at all proud. But they allowed the project to grow and develop, allowed to have not a bare idea, but a working implementation capable of attracting stakeholders, future users and investors. At this stage of development, all this code is no longer used, do not worry :)
It is possible that in order to create a more complete picture of what we are doing at all, it will be interesting for you to read the previous post written by our leader AlexLysenko : Keyboard Oktodon in search of the Right Click
Prototype # 1 - Bluetooth
Our first prototype used Bluetooth sharing. It would seem that there could not be a more convenient technology for connecting the keyboard and the phone, and there was confidence that everything would work out of the box. But this did not happen.
The first problem arose from the fact that, starting with version 2.3, the support of BT-keyboards in the system appeared in Android. As soon as the system detected a “paired” keyboard, it immediately captured it and began to use it as an input tool. Since we were going to do the same, we inevitably would have printed two characters, one for us and the system one, with one click of the Octodon joystick. It was necessary to somehow muff an overly talkative Android. The phone was “rooted”, and digging into the source code of the kernel and system libraries began in order to find the most convenient place to ignore the keyboard connection. That place was the library libui.
In the open_device function, a dirty hack of the form was implemented:
if ((id.vendor == 0x05AC && id.product == 0x0239)) // VID PID { LOGI("Ignoring device %04x %04x", id.vendor, id.product); close(fd); fd = -1; return -1; } As you can see, we just don’t let us open our device. Now the OS has stopped taking away the keyboard from us, and it has become possible to communicate with it using the Bluetooth functions of the Android API. And here the problem was again waiting: the BT socket for receiving data could not be opened at all - the standard createRfcommSocketToServiceRecord function for this fell with an error. Annals of history did not preserve details, but the problem took a lot of time. As a result, it came to the reverse-engineering of products from (then still) Market, which allowed working with this BT-keyboard on “old” Androids who did not know how to do it themselves. And here, behind the barricades of obfuscation, an excellent thing was found in one of the programs:
private static BluetoothSocket createBluetoothSocket(int type, int fd, boolean auth, boolean encrypt, String address, int port) { try { Class[] parameterTypes = new Class[] { Integer.TYPE, // type Integer.TYPE, // fd Boolean.TYPE, // auth Boolean.TYPE, // encrypt String.class, // address Integer.TYPE // port }; Constructor<BluetoothSocket> constructor = BluetoothSocket.class.getDeclaredConstructor(parameterTypes); constructor.setAccessible(true); Object[] parameters = new Object[] { Integer.valueOf(type), Integer.valueOf(fd), Boolean.valueOf(auth), Boolean.valueOf(encrypt), address, Integer.valueOf(port), }; return (BluetoothSocket)constructor.newInstance(parameters); } catch (Exception ex) { return null; } } BluetoothSocket intrSocket = createBluetoothSocket(3 /* TYPE_L2CAP */, -1, false, false, "DC:2C:26:A0:C2:50" /* MAC-address */, 0x13 /* INTR port */); Call a private type constructor! The surprise was even greater when the submitted piece of code took and immediately earned. Understand what, how and why there was no time, so the code was left right in this form. This is one of the drawbacks of low-level development - it is often necessary to accept the fact that part of the code works “magically”: no one knows why it works, and no one knows why a more normal solution refuses to work ... Fortunately, we quickly left this Bluetooth keyboards, having made only one prototype with its use, therefore, we managed to get rid of such “magic” soon.
Prototype # 2 - USB HID keyboard
As soon as it became clear that the use of ready-made devices severely limits us, it was decided to develop our own. The main smartphone on which we focused, by this time was the Samsung Galaxy S2. He had a normally and stable USB host, to which we were going to connect. As the controller was taken Teensy 1.0. It turned out not to be at the right time that its capabilities were not enough for us, and that something should have been taken from a more recent version, 2.0. The difference was in the libraries for creating HID-compatible devices. If 1.0 could only pretend to be a HID keyboard or a mouse, then on 2.0 it was possible to implement a much more versatile Raw HID device using our own protocol. But what to do, first had to be content with what was. When emulating the HID-keyboard, the problem described above arose again - when connected, the system itself captured it as an input device, and the characters sent were printed twice - by us and the OS.
The solution described above was no longer suitable - we decided that we would have had enough of the changes to the system. If we want to create a device that can be released to the market, then it should work on any Galaxy S2, there should be no need for re-flashing.
In the process of experimenting with a conventional USB keyboard connected to the S2, it turned out that for some reason the smartphone does not respond to keystrokes in the digital part of the keyboard. This was a great help, because our Teensy could somehow encode the deviations of the Octodon joysticks into numeric key presses. For each change in the state of the joysticks, we began to send parcels of 3 bytes:
- Scancode “Num 0” ... ”Num 9” is the number of the joystick, the status of which is passed.
- Scancode “Num 0” ... ”Num 4” is the direction of deflection of the joystick. 0 - to the left, 1 - down, ..., 4 - central pressing.
- “Num +” if it was pressed or “Num -” if released.
Thus, simply pressing the first joystick to the left gave us the “00+” sequence, releasing - “00-”. Without releasing one joystick, you can reject the second one - in this way you can encode simultaneous presses of several buttons, which we really needed for the Shift mode.
However, there was another task to somehow get the transferred scancodes. The system API ignored them, so I had to go deeper. To our great happiness, Android is * nix, and that’s all - the file. The required file was in / dev / input and, depending on the location of the stars in the sky, either event7 or event8 was called. Only root had access to it, so I had to rant the device, but that was all - I didn’t need a flashing. The use of root in Android itself is an interesting task, since there is no way to get the corresponding rights through the API (or we could not find it). Therefore, we do everything right through the terminal:
Process root = Runtime.getRuntime().exec("su"); // su DataOutputStream shellStream = new DataOutputStream(root.getOutputStream()); shellStream.writeBytes("cat /dev/input/event7 & && cat /dev/input/event8 &"); // , , InputStream inputStream = root.getInputStream(); InputStream errorStream = root.getErrorStream(); // 16- byte[] buf = new byte[16]; if (inputStream.available() > 15) { int data = inputStream.read(buf, 0, 16); } The prototype, which worked in this way, existed for a long time, went through many exhibitions, and is still alive today. Over time, the code became more sophisticated, and the names of the device files instead of the hardcode began to be set via the config file.
Well, then came the golden age. For Galaxy S2 came out the 4th Android, and the ordered Teensy 2.0 came to us. The docking of the device developed on the basis of the Raw HID controller with the USB Host API that appeared in the new OS went so smoothly that it didn’t fit into the framework of this essay, which was intended to tell about problems. An old dream came true, Octodon could work on the “stock” smartphone without having to do anything with it. We were able to produce small-scale samples that just work, and they do not need to be brought to mind with a file. But beautiful solutions without crutches are a topic for a separate article, especially since the Habr pages didn’t tell about USB Host API, it seems, never. So to be continued!
Source: https://habr.com/ru/post/212941/
All Articles