Manage homemade iron over the air using Open Sound Control
In this article I will try to tell you how you can remotely control your self-made device connected to the network from a phone or tablet on iOS and Android. Anyone familiar with the topic at this point has already decided that it will be about the next web interface to your Arduino and mbedas - if I hurry to interrupt your thoughts, it will not. The way that I want to talk about is quick, cheap, has ready feedback, convenient controls and has visibility, which will be envied by the most polished web interface.
So that the material does not seem to be cut off from real life, I will show how we did this trick 4 months ago with our Lightpack. This situation is slightly different from the spherical scenario of using the Open Sound Control protocol (further OSC) in a vacuum, but nevertheless is a good example of how it can be effectively used in rapid prototyping.
We had an open-source USB backlight device , which is controlled by cross-platform software and has an API for managing from external programs or plug-ins on the side of this software. We wanted to control this device over the network from our tablets and smartphones. Turn it on and off, change settings profiles, control brightness, adjust the backlight for each individual channel, and maybe even launch other plug-ins.
')
In your case, the situation may be different, not complicated by an intermediary in the form of a full-fledged PC and some kind of API. For example, a device on the ARM / AVR periodically connecting to the network, or at least an Arduino with an ethernet shield that 24/7 logs for you the power consumption behind the dashboard at the entrance and sends the data straight to the Google document.
The first thing that comes to mind to solve this problem is the web interface. It would seem universal: you can access from any platform. But we did not want to force the web server to constantly spin, no matter how microscopic it might be. Yes, we didn’t find good web-interface designers that would have suitable controls, we didn’t want to draw our own and understood that the result can only work seriously on the desktop and ordinary MK will not master it if in the future we have to manage standalone -device.
However, if you can connect to the device via the network (in our case, you can send commands to the API directly via Telnet) you could write a full-fledged application for the mobile platform that would just send message packets, and the server parsed them and thus control the device . Despite the fact that in the evening we collected a small prototype for Android , which exploited this method, it still remained “expensive” even in comparison with the web interface.
As time went on, the task was recorded somewhere on the subcortex and didn’t get out until I accidentally stumbled upon the TouchOSC mobile app, which (almost) works equally well on iOS and Android.
After the demo videos and screenshots, it was simply impossible to pass by without trying.)
In short, the TouchOSC application ( AppStore , Google Play ) implements the OSC and MIDI protocol interfaces on your mobile devices, which all specialized synthesizers, sound editors, DJ programs, recording studios, etc., are used to working with. TouchOSC makes it possible not to use Only the predefined layouts for the controls (layouts) of the slider-tab buttons, but also very simple and fast to draw your own in a special editor. All this stuff works through wi-fi, but it can also work by wire if there are special adapters.
A typical layout from the preset looks like this. Level of difficulty: "Koreans", which, however, does not prevent you from repeating it in the layout editor.
Surface analysis showed that this application is not the only one and, if desired, for iOS you can use iOSC, OSCemote, Mrmr OSC controller, Remokon for OSC (for Android, I will not even list them - there are even more). The same analysis showed that the protocol itself is used in a large number of programs and devices, and the most important thing for us is that libraries and wrappers with its implementation exist for many programming languages ​​and platforms (for example, options for Arduino and mbed ). An incomplete list is on the official site.
We have an opportunity to apply the new, interesting, at first glance, toolkit in an atypical way. Ahead was a free day off, so with the words “call priyant” timsat unfolded Google on one half of the monitor and Notepad ++ on the second.
Open Sound Control is a batch protocol for transferring data between multimedia devices (primarily electronic musical instruments / synthesizers and PCs), which has recently been increasingly used as a substitute for MIDI. The main features of the protocol work remains through the network, but there are other features . The latest version of the specification is available on the website of the Center for Audio Technology at Berkeley University (CNMAT).
As I already mentioned, today the typical use of the protocol is to provide an interface between input devices (synthesizers, tablets, game controllers, etc.) and specialized programs such as Super Collider, Traktor, REAPER, etc. OSC out of the box is supported by stunning devices like Lemur and Monome .
Working over the network, the protocol allows data exchange in both directions, which allows for feedback from the device. If you take a closer look at the screenshots, then some of the control buttons have an additional indicator that can be lit after the command sent earlier was executed by the server in reality. Those. Your tablet can reflect the real state of the device you are controlling (target). If you get the hang of it, you can even draw graphics, but this is for men who know a lot about perversions.
Here is an example of messages exchanged between the OSC server and the client:
In this case, we have the exact control ID and / 1 / fader2 0.497120916843 means that the second tab on the first tab (yes, there you can use several tabs within the same layout), the value 0.49 came to the server, etc. The Z-message from this example can be enabled at will and I will tell about it later.
In the example with Lightpack, it was convenient for us to send profile names to the tablet and switch between them. In addition, when starting a session, the positions of all controls on the tablet are automatically aligned with the actual settings of the target device.
To work with the Lightpack API, we use our own wrapper written in Python. Therefore, and also because the OSC-library satisfying us existed for Python, we decided to write a small script engine, which was a connector between the API and the network device that sent the OSC message. The main requirements for such an implementation:
Although, if you dig deeper with the tunnels on your router, then the second restriction can be bypassed and managed to drink for your hamster from the office at the other end of the city.
For those who, like me, prefer pictures, I’ve got an interaction scheme:
In our case, the exchange between the connector and the API is implemented via sockets (this is a feature of the API itself), while, as seen in the demo, everything works pretty quickly. Of course, in the equation with the hamster on Arduino, right behind the router will be the device itself with the center of logic in the form of a microcontroller with firmware which already includes libraries for working with the network and OSC.
In fact, for implementation, we had to deal only with the process of initialization of the exchange, feedback and conversion of the received / transmitted values. The lion's share of the code is taken by a simple mapping. Those. after you draw the layout you will have on your hands a list of addresses of controls (buttons, sliders, shortcuts, etc.), each of which must be compared with the API function. Everything.
Honestly, I do not know what you can write more about. In terms of programming, the task is primitive. The source code of this “connector” as usual, lies in our repository and is accessible to all.
By the way, some pretty convenient additional features are implemented in TouchOSC: First, it is possible to use the / ping message that the client and server exchange to maintain the connection even when the data itself does not run. Secondly, the so-called z-message is supported - a special message that tells the server that the user at the client has finally finished lifting his finger on the screen (the moment the input is finished). We did not use this feature, but I do not exclude that there are situations in which it will be extremely useful. Thirdly, like most similar applications, TouchOSC can send data to the server from the accelerometer built into the mobile device. Those. You can use your smartphone or tablet as a tricky input device for your network homemade products (twist the camera on servos, etc.).
Everything about the interaction with the application TouchOSC runs smoothly. Even the loading of own layouts is carried out “through the air” with the push of a single button. The only noticeable drawback so far is the impossibility of loading your own layout for the Android version. In a correspondence with the author, he managed to convince me that “already about to prepare a release”.
The library for Python was not well documented, so we initialized the data exchange process longer than actually programmed the layout behavior.
In general, the process of creating such a machine is extremely intuitive and will not take much time. That is why I would advise using OSC for rapid prototyping instead of full-fledged client-server stacks with web interfaces. Especially if you are going to present this prototype at an event, or, say, to a customer, a more vivid implementation variant accompanied by wow! -How-you-is-missed-in-eppstor I simply don’t know.
So again: If you need to quickly and cheaply implement an easy way to visually control a homemade network device with feedback, then the OSC protocol is an interesting choice.
So that the material does not seem to be cut off from real life, I will show how we did this trick 4 months ago with our Lightpack. This situation is slightly different from the spherical scenario of using the Open Sound Control protocol (further OSC) in a vacuum, but nevertheless is a good example of how it can be effectively used in rapid prototyping.
Task
We had an open-source USB backlight device , which is controlled by cross-platform software and has an API for managing from external programs or plug-ins on the side of this software. We wanted to control this device over the network from our tablets and smartphones. Turn it on and off, change settings profiles, control brightness, adjust the backlight for each individual channel, and maybe even launch other plug-ins.
')
In your case, the situation may be different, not complicated by an intermediary in the form of a full-fledged PC and some kind of API. For example, a device on the ARM / AVR periodically connecting to the network, or at least an Arduino with an ethernet shield that 24/7 logs for you the power consumption behind the dashboard at the entrance and sends the data straight to the Google document.
The first thing that comes to mind to solve this problem is the web interface. It would seem universal: you can access from any platform. But we did not want to force the web server to constantly spin, no matter how microscopic it might be. Yes, we didn’t find good web-interface designers that would have suitable controls, we didn’t want to draw our own and understood that the result can only work seriously on the desktop and ordinary MK will not master it if in the future we have to manage standalone -device.
However, if you can connect to the device via the network (in our case, you can send commands to the API directly via Telnet) you could write a full-fledged application for the mobile platform that would just send message packets, and the server parsed them and thus control the device . Despite the fact that in the evening we collected a small prototype for Android , which exploited this method, it still remained “expensive” even in comparison with the web interface.
As time went on, the task was recorded somewhere on the subcortex and didn’t get out until I accidentally stumbled upon the TouchOSC mobile app, which (almost) works equally well on iOS and Android.
After the demo videos and screenshots, it was simply impossible to pass by without trying.)
Idea
In short, the TouchOSC application ( AppStore , Google Play ) implements the OSC and MIDI protocol interfaces on your mobile devices, which all specialized synthesizers, sound editors, DJ programs, recording studios, etc., are used to working with. TouchOSC makes it possible not to use Only the predefined layouts for the controls (layouts) of the slider-tab buttons, but also very simple and fast to draw your own in a special editor. All this stuff works through wi-fi, but it can also work by wire if there are special adapters.
A typical layout from the preset looks like this. Level of difficulty: "Koreans", which, however, does not prevent you from repeating it in the layout editor.
Surface analysis showed that this application is not the only one and, if desired, for iOS you can use iOSC, OSCemote, Mrmr OSC controller, Remokon for OSC (for Android, I will not even list them - there are even more). The same analysis showed that the protocol itself is used in a large number of programs and devices, and the most important thing for us is that libraries and wrappers with its implementation exist for many programming languages ​​and platforms (for example, options for Arduino and mbed ). An incomplete list is on the official site.
We have an opportunity to apply the new, interesting, at first glance, toolkit in an atypical way. Ahead was a free day off, so with the words “call priyant” timsat unfolded Google on one half of the monitor and Notepad ++ on the second.
Protocol
Open Sound Control is a batch protocol for transferring data between multimedia devices (primarily electronic musical instruments / synthesizers and PCs), which has recently been increasingly used as a substitute for MIDI. The main features of the protocol work remains through the network, but there are other features . The latest version of the specification is available on the website of the Center for Audio Technology at Berkeley University (CNMAT).
As I already mentioned, today the typical use of the protocol is to provide an interface between input devices (synthesizers, tablets, game controllers, etc.) and specialized programs such as Super Collider, Traktor, REAPER, etc. OSC out of the box is supported by stunning devices like Lemur and Monome .
Working over the network, the protocol allows data exchange in both directions, which allows for feedback from the device. If you take a closer look at the screenshots, then some of the control buttons have an additional indicator that can be lit after the command sent earlier was executed by the server in reality. Those. Your tablet can reflect the real state of the device you are controlling (target). If you get the hang of it, you can even draw graphics, but this is for men who know a lot about perversions.
Here is an example of messages exchanged between the OSC server and the client:
/1/fader2 0.497120916843 /1/fader2 /z /1/rotary1 0.579584240913 /1/rotary1 0.605171322823 /1/rotary1 /z In this case, we have the exact control ID and / 1 / fader2 0.497120916843 means that the second tab on the first tab (yes, there you can use several tabs within the same layout), the value 0.49 came to the server, etc. The Z-message from this example can be enabled at will and I will tell about it later.
In the example with Lightpack, it was convenient for us to send profile names to the tablet and switch between them. In addition, when starting a session, the positions of all controls on the tablet are automatically aligned with the actual settings of the target device.
Implementation
To work with the Lightpack API, we use our own wrapper written in Python. Therefore, and also because the OSC-library satisfying us existed for Python, we decided to write a small script engine, which was a connector between the API and the network device that sent the OSC message. The main requirements for such an implementation:
- Availability of client / server ports (you can customize your own in TouchOSC)
- Client and server must be on the same network.
Although, if you dig deeper with the tunnels on your router, then the second restriction can be bypassed and managed to drink for your hamster from the office at the other end of the city.
For those who, like me, prefer pictures, I’ve got an interaction scheme:
In our case, the exchange between the connector and the API is implemented via sockets (this is a feature of the API itself), while, as seen in the demo, everything works pretty quickly. Of course, in the equation with the hamster on Arduino, right behind the router will be the device itself with the center of logic in the form of a microcontroller with firmware which already includes libraries for working with the network and OSC.
In fact, for implementation, we had to deal only with the process of initialization of the exchange, feedback and conversion of the received / transmitted values. The lion's share of the code is taken by a simple mapping. Those. after you draw the layout you will have on your hands a list of addresses of controls (buttons, sliders, shortcuts, etc.), each of which must be compared with the API function. Everything.
Honestly, I do not know what you can write more about. In terms of programming, the task is primitive. The source code of this “connector” as usual, lies in our repository and is accessible to all.
By the way, some pretty convenient additional features are implemented in TouchOSC: First, it is possible to use the / ping message that the client and server exchange to maintain the connection even when the data itself does not run. Secondly, the so-called z-message is supported - a special message that tells the server that the user at the client has finally finished lifting his finger on the screen (the moment the input is finished). We did not use this feature, but I do not exclude that there are situations in which it will be extremely useful. Thirdly, like most similar applications, TouchOSC can send data to the server from the accelerometer built into the mobile device. Those. You can use your smartphone or tablet as a tricky input device for your network homemade products (twist the camera on servos, etc.).
Underwater rocks
Everything about the interaction with the application TouchOSC runs smoothly. Even the loading of own layouts is carried out “through the air” with the push of a single button. The only noticeable drawback so far is the impossibility of loading your own layout for the Android version. In a correspondence with the author, he managed to convince me that “already about to prepare a release”.
The library for Python was not well documented, so we initialized the data exchange process longer than actually programmed the layout behavior.
In general, the process of creating such a machine is extremely intuitive and will not take much time. That is why I would advise using OSC for rapid prototyping instead of full-fledged client-server stacks with web interfaces. Especially if you are going to present this prototype at an event, or, say, to a customer, a more vivid implementation variant accompanied by wow! -How-you-is-missed-in-eppstor I simply don’t know.
So again: If you need to quickly and cheaply implement an easy way to visually control a homemade network device with feedback, then the OSC protocol is an interesting choice.
Source: https://habr.com/ru/post/149838/
All Articles