BYOD in a container: we virtualize Android. Part two
I continue the story about the technology of Android virtualization, made in Parallels Labs by a group of students of the Department of MIT of the Academic University of St. Petersburg as part of the master's work. In yesterday's article, we looked at the general concept of Android virtualization, which allows it. The task of today's post is to deal with the virtualization of telephony, sound and user input systems.
Audio and telephony were virtualized in user space. The general idea of ​​such virtualization is to implement a proxy (server and client) at the level of a hardware independent interface in the Android stack. The proxy server is placed in a separate container with the original software stack of the operating system for performing client requests on physical devices. A proxy client is placed in each container with an Android user.
')
A typical scenario for using an audio device when running multiple Android would be listening to music in one of the operating systems and playing the sound of a game running in another. The user probably would like to hear the sound of all running Android, but in fact the physical sound device is not designed for this. When you try to use it, the OS reports an error and sometimes cannot play a sound until the next reboot. Audio unit virtualization should provide the ability to simultaneously listen to the sound of all Android.
You also need to implement interaction with the telephony service so that, during a telephone conversation, the audio device records and transmits the voice of the user to the cellular network and reproduces the voice of the interlocutor received from the cellular network.
AudioHardwareInterface interface is a replaceable hardware independent interface to the sound device, as we are told about by the Android Platform Developer's Guide. In addition, it is the lowest-level and simplest hardware independent audio interface. In fact, it is a stream for recording sound to play. In addition to the audio stream played by Android, in its implementation you can get other useful information.
It would be possible to perform a substitution at a higher level. For example, substitute part of the Android audio framework. But the analysis showed that the audio framework is much more complicated than the AudioHardwareInterface. Therefore, in the case of audio device virtualization, the proxy client is an implementation of AudioHardwareInterface.
The main responsibility of the proxy client is to send the Android audio stream and other useful information to the proxy server, which will play all Android audio streams. Audio streams are played in uncompressed form, so transferring them using copying to a proxy server can result in noticeable overhead. We avoid them by using inter-container shared memory to transfer audio streams. Information and control messages are transmitted using the mechanism of intercontainer messaging.
The main task of the proxy server is to mix the audio streams received from the proxy clients and play the resulting audio stream on the physical audio device. To ensure maximum portability of the proxy server, sound mixing and playback should be carried out independently of the device. The Android audio framework interfaces are hardware independent. In addition, in the audio framework already laid the possibility of mixing audio streams. Android’s audio framework does not provide access to its mixer from the outside, so using it requires either embedding it into the audio framework or using the mixer implicitly. The architecture of our sound virtualization solution is shown in the figure.
The simplest solution that allows you to mix and play audio in Android is to use audio classes from the Android SDK. In our solution, the audio stream of each user is reproduced by a separate object of the AudioTrack class from the Android SDK. Using classes allows the Android audio framework to mix all the audio streams that are played back and sends the resulting audio stream to a device’s speaker or headphone output.
It should be noted that the classes from the Android SDK are written in Java, and their use requires the launch of the Dalvik JVM. Running Dalvik in the case of Android means launching the entire OS due to the use of a mechanism like bootstrapping. Here we have a problem: running Android takes 200-250 MB of RAM and consumes computing resources. This is an inadmissible overhead for the task of reproducing and mixing audio streams, but in this paper we managed to avoid them. In fact, we are not using Java classes from the Android SDK, but their native backend written in C ++. That is why the launch of Dalvik, which means that the launch of a full Android container is not required.
Fully native applications, such as a proxy server, cannot be registered in the Android rights monitoring system. At the same time, the services accessed by the proxy server silently ignored requests due to lack of rights. Everything was decided by removing the rights checks in these services. We did this peace of mind trick, since this deletion does not affect security. In the container with the proxy server, foreign applications are never launched, it is completely under the control of virtualization technology.
The user probably wants the sound volume of different Android to be adjusted. It turned out that in Android 2.x there is no abstraction of a hardware mixer, i.e. mixing is done through the CPU. Since the mixing of audio streams inside each Android is completely independent (there is no shared shared hardware mixer), the overall volume level of each Android can be configured in Android itself. The proxy server simply does not make changes to the sound levels of each OS, and the user adjusts the sound level of each Android in their user interface, as if there was no virtualization on the phone.
Mobile devices typically have multiple audio output devices. For example, a large speaker, handset, headphones, bluetooth headset. Launched OSs may simultaneously require playback of their audio streams on different output devices. In some cases, their requirements will be incompatible. For example, it does not make sense to play one sound on a large speaker, and the other in headphones or a tube. It will also be strange if during a call in the handset not only the interlocutor's voice is heard, but also the music playing before the call.
Therefore, a policy of routing audio streams to various output devices was developed, which implements the user's “least surprise” strategy. As part of this strategy, in particular, it is sometimes required to simulate the playback of Android sound. To do this, the proxy client is switched to a dummy playback mode, in which synchronous functions are blocked for the duration of the audio stream being played, so Android thinks that it played the sound, although in fact it was not sent to the proxy server for playback.
In the proxy client, which is an implementation of AudioHardwareInterface, we can get information about whether the phone is currently on the phone or not. If the conversation is on, then you need to set up the audio subsystem for recording and reproducing voice and interacting with the telephony service. In fact, all you have to do is to translate the AudioHardwareInterface implementation supplied with the device into the call mode by calling the setMode method (MODE_IN_CALL) and set the sound routing to the handset, otherwise the interlocutor’s voice will be played by the current audio output device. Thus, the device provider is responsible for implementing the interaction with the telephony service.
The most interesting and difficult was the technology of dispatching telephone calls between containers. The software control stack for accessing mobile networks on Android contains the following components:
1. API com.android.internal.telephony package.
2. The rild daemon, multiplexing user application requests in mobile network access equipment.
3. Proprietary library of access to equipment.
4. GSM modem driver.
The interface between components (2) and (3) is documented and is called the Radio Interface Layer, or RIL. It is described in the development / pdk / docs / porting / telephony.jd file.
For RIL virtualization, I had to solve the following problems:
Proprietary RIL implementations in various “Androids” are trying to re-initialize the equipment;
It is not clear which of the “Androids” should receive incoming calls and SMS;
It is not clear how to manage calls and SMS from from different "Androids".
It should also be noted here that the implementation of the GSM modem drivers differs significantly on different devices, so the existence of a universal solution for virtualizing this device in the kernel is impossible. In addition, the proprietary RIL library uses an undocumented protocol to interact with a GSM modem, which is an obstacle to its virtualization, even for a specific smartphone model.
The specific implementation of the RIL library on the phone is unknown, but its interface is known. We have created a level of abstraction in the form of a virtual RIL proxy and RIL client. On the client side, the rild daemon instead of the original RIL library loads the RIL client implemented by us. RIL-client, intercepting calls to the interface, transmits calls through the mechanism of inter-container interaction RIL-proxy, which in turn makes the main interface virtualization, and uses the original RIL library to access telephony.
The server part is implemented as a single-threaded application that loads the proprietary implementation of RIL and receives messages from proxy clients running in user containers using the inter-container interaction mechanism. In the OnRequestComplete handler, we do not know the type of request, but the method of packaging the response of a proprietary RIL implementation depends on it, so when a request arrives, the server remembers the token match the request number in order to call the response marshaller in this handler. The RIL proxy server should handle the situation when clients receive requests with identical tokens, so when a new request arrives, the proxy server looks at the list of request tokens processed by the proprietary RIL library, and if it turns out, that the request with such a token is now being processed, the server generates a new token for the incoming request. Thus, the proxy server for all incoming requests stores their real and dummy tokens.
For each type of requests coming to OnRequest, a processing policy is defined. Currently three policies are defined:
For each type of asynchronous notification, routing policies are also defined:
Android uses the evdev subsystem to collect kernel input events. For each process that opens the / dev / input / eventX file, this driver creates a queue of events from the input subsystem kernel.
Thus, incoming input events can be delivered to all containers. The evdev_event () function is an input event manager, so in order to prevent the delivery of input messages to inactive containers, this function has been modified so that all incoming input events are added only to the process queues of the active container.
After we managed to launch our solution on smartphones, we did a series of experiments in order to quantify the characteristics of our solution. I must say that we did not work on optimization - this work is still to be done.
The main goals of testing were:
Testing was conducted on a Samsung Galaxy S II smartphone using the following smartphone usage scenarios:
1. Simple (turn on the phone and do nothing with it);
2. Playing music with backlight on;
3. Angry Birds (where can I go without them?) And simultaneous playing of music.
Test scripts were executed on three configurations:
1. Original CyanogenMod 7 for Samsung Galaxy S II (1);
2. One container with CyanogenMod 7 (2);
3. Two containers with CyanogenMod 7: music was played in one container, the other was playing (3).
In each test run, the following parameters were measured:
1. The amount of free memory and the size of the cache file system (for information / proc / meminfo);
2. Battery charge (according to / sys / class / power_supply / battery / capacity.
Each test run lasted 30 minutes, measurements were made with a frequency of 2 c. Here's what we got:
It may seem strange that memory costs in the case of containers are less than in real environments. The exact cause has not been established (we have not tried to figure out this issue so far), but the same effect is also observed in the Cells project, which they described in their article .
Testing showed the acceptability of the developed solutions and demonstrated insignificant consumption of the memory and the smartphone's battery by the developed technology except for the case (the last row of the table), in which the increase in energy consumption by the two containers almost doubled compared to the unmodified Android. Currently, work is underway to clarify the reasons for this behavior.
As can be seen from the values ​​of memory consumption, when launching the second container, it increases by more than 2 times, which is an obstacle to launching several containers on one smartphone: experiments show that Google Nexus S, which has 380 MB, cannot start more than one container without activation of the paging file.
Around the mobile devices now generates a large part of the technological news. The master's (or graduate, as usual) project of a group of students showed the viability of virtualization technology on mobile devices and gave an idea of ​​how this technology can be used to create products.
Some interesting facts about the creation of our solution:
If something important is left behind the scenes, ask questions in the comments. I will try to answer them.
Audio and telephony were virtualized in user space. The general idea of ​​such virtualization is to implement a proxy (server and client) at the level of a hardware independent interface in the Android stack. The proxy server is placed in a separate container with the original software stack of the operating system for performing client requests on physical devices. A proxy client is placed in each container with an Android user.
')
Audio device
A typical scenario for using an audio device when running multiple Android would be listening to music in one of the operating systems and playing the sound of a game running in another. The user probably would like to hear the sound of all running Android, but in fact the physical sound device is not designed for this. When you try to use it, the OS reports an error and sometimes cannot play a sound until the next reboot. Audio unit virtualization should provide the ability to simultaneously listen to the sound of all Android.
You also need to implement interaction with the telephony service so that, during a telephone conversation, the audio device records and transmits the voice of the user to the cellular network and reproduces the voice of the interlocutor received from the cellular network.
Proxy client
AudioHardwareInterface interface is a replaceable hardware independent interface to the sound device, as we are told about by the Android Platform Developer's Guide. In addition, it is the lowest-level and simplest hardware independent audio interface. In fact, it is a stream for recording sound to play. In addition to the audio stream played by Android, in its implementation you can get other useful information.
It would be possible to perform a substitution at a higher level. For example, substitute part of the Android audio framework. But the analysis showed that the audio framework is much more complicated than the AudioHardwareInterface. Therefore, in the case of audio device virtualization, the proxy client is an implementation of AudioHardwareInterface.
The main responsibility of the proxy client is to send the Android audio stream and other useful information to the proxy server, which will play all Android audio streams. Audio streams are played in uncompressed form, so transferring them using copying to a proxy server can result in noticeable overhead. We avoid them by using inter-container shared memory to transfer audio streams. Information and control messages are transmitted using the mechanism of intercontainer messaging.
Proxy server
The main task of the proxy server is to mix the audio streams received from the proxy clients and play the resulting audio stream on the physical audio device. To ensure maximum portability of the proxy server, sound mixing and playback should be carried out independently of the device. The Android audio framework interfaces are hardware independent. In addition, in the audio framework already laid the possibility of mixing audio streams. Android’s audio framework does not provide access to its mixer from the outside, so using it requires either embedding it into the audio framework or using the mixer implicitly. The architecture of our sound virtualization solution is shown in the figure.
The simplest solution that allows you to mix and play audio in Android is to use audio classes from the Android SDK. In our solution, the audio stream of each user is reproduced by a separate object of the AudioTrack class from the Android SDK. Using classes allows the Android audio framework to mix all the audio streams that are played back and sends the resulting audio stream to a device’s speaker or headphone output.
It should be noted that the classes from the Android SDK are written in Java, and their use requires the launch of the Dalvik JVM. Running Dalvik in the case of Android means launching the entire OS due to the use of a mechanism like bootstrapping. Here we have a problem: running Android takes 200-250 MB of RAM and consumes computing resources. This is an inadmissible overhead for the task of reproducing and mixing audio streams, but in this paper we managed to avoid them. In fact, we are not using Java classes from the Android SDK, but their native backend written in C ++. That is why the launch of Dalvik, which means that the launch of a full Android container is not required.
Fully native applications, such as a proxy server, cannot be registered in the Android rights monitoring system. At the same time, the services accessed by the proxy server silently ignored requests due to lack of rights. Everything was decided by removing the rights checks in these services. We did this peace of mind trick, since this deletion does not affect security. In the container with the proxy server, foreign applications are never launched, it is completely under the control of virtualization technology.
Sound level control
The user probably wants the sound volume of different Android to be adjusted. It turned out that in Android 2.x there is no abstraction of a hardware mixer, i.e. mixing is done through the CPU. Since the mixing of audio streams inside each Android is completely independent (there is no shared shared hardware mixer), the overall volume level of each Android can be configured in Android itself. The proxy server simply does not make changes to the sound levels of each OS, and the user adjusts the sound level of each Android in their user interface, as if there was no virtualization on the phone.
Sound output devices
Mobile devices typically have multiple audio output devices. For example, a large speaker, handset, headphones, bluetooth headset. Launched OSs may simultaneously require playback of their audio streams on different output devices. In some cases, their requirements will be incompatible. For example, it does not make sense to play one sound on a large speaker, and the other in headphones or a tube. It will also be strange if during a call in the handset not only the interlocutor's voice is heard, but also the music playing before the call.
Therefore, a policy of routing audio streams to various output devices was developed, which implements the user's “least surprise” strategy. As part of this strategy, in particular, it is sometimes required to simulate the playback of Android sound. To do this, the proxy client is switched to a dummy playback mode, in which synchronous functions are blocked for the duration of the audio stream being played, so Android thinks that it played the sound, although in fact it was not sent to the proxy server for playback.
Interaction with telephony service
In the proxy client, which is an implementation of AudioHardwareInterface, we can get information about whether the phone is currently on the phone or not. If the conversation is on, then you need to set up the audio subsystem for recording and reproducing voice and interacting with the telephony service. In fact, all you have to do is to translate the AudioHardwareInterface implementation supplied with the device into the call mode by calling the setMode method (MODE_IN_CALL) and set the sound routing to the handset, otherwise the interlocutor’s voice will be played by the current audio output device. Thus, the device provider is responsible for implementing the interaction with the telephony service.
Telephony
The most interesting and difficult was the technology of dispatching telephone calls between containers. The software control stack for accessing mobile networks on Android contains the following components:
1. API com.android.internal.telephony package.
2. The rild daemon, multiplexing user application requests in mobile network access equipment.
3. Proprietary library of access to equipment.
4. GSM modem driver.
The interface between components (2) and (3) is documented and is called the Radio Interface Layer, or RIL. It is described in the development / pdk / docs / porting / telephony.jd file.
For RIL virtualization, I had to solve the following problems:
Proprietary RIL implementations in various “Androids” are trying to re-initialize the equipment;
It is not clear which of the “Androids” should receive incoming calls and SMS;
It is not clear how to manage calls and SMS from from different "Androids".
It should also be noted here that the implementation of the GSM modem drivers differs significantly on different devices, so the existence of a universal solution for virtualizing this device in the kernel is impossible. In addition, the proprietary RIL library uses an undocumented protocol to interact with a GSM modem, which is an obstacle to its virtualization, even for a specific smartphone model.
Client part
The specific implementation of the RIL library on the phone is unknown, but its interface is known. We have created a level of abstraction in the form of a virtual RIL proxy and RIL client. On the client side, the rild daemon instead of the original RIL library loads the RIL client implemented by us. RIL-client, intercepting calls to the interface, transmits calls through the mechanism of inter-container interaction RIL-proxy, which in turn makes the main interface virtualization, and uses the original RIL library to access telephony.
Server part
The server part is implemented as a single-threaded application that loads the proprietary implementation of RIL and receives messages from proxy clients running in user containers using the inter-container interaction mechanism. In the OnRequestComplete handler, we do not know the type of request, but the method of packaging the response of a proprietary RIL implementation depends on it, so when a request arrives, the server remembers the token match the request number in order to call the response marshaller in this handler. The RIL proxy server should handle the situation when clients receive requests with identical tokens, so when a new request arrives, the proxy server looks at the list of request tokens processed by the proprietary RIL library, and if it turns out, that the request with such a token is now being processed, the server generates a new token for the incoming request. Thus, the proxy server for all incoming requests stores their real and dummy tokens.
Request Routing Policy
For each type of requests coming to OnRequest, a processing policy is defined. Currently three policies are defined:
- Unconditional blocking of the request. The policy is set for all requests that have never been encountered during the development of a proxy server.
- Unconditional request transfer to a proprietary library. The policy applies to all requests that do not change the state of the proprietary library and GSM modem.
- Request transfer from active container to proprietary library. Used for requests related to sending SMS and making calls.
For each type of asynchronous notification, routing policies are also defined:
- Unconditional blocking notification.
- Routing to the active container. The policy is set for notifications related to the acceptance of SMS and calls.
- Routing to all containers. For service notifications "iron" - for example, about a change in the signal level of the cellular network
Input subsystem
Android uses the evdev subsystem to collect kernel input events. For each process that opens the / dev / input / eventX file, this driver creates a queue of events from the input subsystem kernel.
Thus, incoming input events can be delivered to all containers. The evdev_event () function is an input event manager, so in order to prevent the delivery of input messages to inactive containers, this function has been modified so that all incoming input events are added only to the process queues of the active container.
Something about testing
After we managed to launch our solution on smartphones, we did a series of experiments in order to quantify the characteristics of our solution. I must say that we did not work on optimization - this work is still to be done.
The main goals of testing were:
- demonstration of the readiness of the developed technology;
- measuring memory consumption on virtualized phones;
- definition of energy consumption of the developed technology.
Testing was conducted on a Samsung Galaxy S II smartphone using the following smartphone usage scenarios:
1. Simple (turn on the phone and do nothing with it);
2. Playing music with backlight on;
3. Angry Birds (where can I go without them?) And simultaneous playing of music.
Test scripts were executed on three configurations:
1. Original CyanogenMod 7 for Samsung Galaxy S II (1);
2. One container with CyanogenMod 7 (2);
3. Two containers with CyanogenMod 7: music was played in one container, the other was playing (3).
In each test run, the following parameters were measured:
1. The amount of free memory and the size of the cache file system (for information / proc / meminfo);
2. Battery charge (according to / sys / class / power_supply / battery / capacity.
Each test run lasted 30 minutes, measurements were made with a frequency of 2 c. Here's what we got:
It may seem strange that memory costs in the case of containers are less than in real environments. The exact cause has not been established (we have not tried to figure out this issue so far), but the same effect is also observed in the Cells project, which they described in their article .
Testing showed the acceptability of the developed solutions and demonstrated insignificant consumption of the memory and the smartphone's battery by the developed technology except for the case (the last row of the table), in which the increase in energy consumption by the two containers almost doubled compared to the unmodified Android. Currently, work is underway to clarify the reasons for this behavior.
As can be seen from the values ​​of memory consumption, when launching the second container, it increases by more than 2 times, which is an obstacle to launching several containers on one smartphone: experiments show that Google Nexus S, which has 380 MB, cannot start more than one container without activation of the paging file.
Conclusion
Around the mobile devices now generates a large part of the technological news. The master's (or graduate, as usual) project of a group of students showed the viability of virtualization technology on mobile devices and gave an idea of ​​how this technology can be used to create products.
Some interesting facts about the creation of our solution:
- There are three main developers of the project, two of whom were students of the Academic University in St. Petersburg at the time of their work, and one was in charge of a graduation project;
- The project lasted 11 months;
- The project was assisted by engineers from the Moscow office of Parallels;
- The project involved three Android devices.
If something important is left behind the scenes, ask questions in the comments. I will try to answer them.
Source: https://habr.com/ru/post/175071/
All Articles