Using Android device as a thin UI for C ++ programs
I want to share with the community a project that I have been quietly doing over the past few months.
There are often situations when you want to manage your program from a phone or tablet, but it is impractical (or not possible) to write a separate application for this on the phone (too much effort for this project, no development experience for android, etc.) ). This situation periodically arose in me, and in the end I decided to deal with this problem once and for all. The result is a system, the architecture and use of which are described in this article.
Creating a system that allows you to implement a plug-in UI based on android devices in C ++ programs. At the same time, we wanted to minimize the dependencies of custom C ++ code on third-party libraries, as well as to abstract it from the data transfer protocol. The system should consist of two parts: C ++ libraries and android applications.
The system has a client-server architecture, where clients are android devices, the server is a user program. Communication between them is carried out using TCP / IP sockets. To implement the communication protocol, the TAU library was created.
')
The main tasks for which the library is responsible:
The library consists of the following namespaces:
We'll start the demonstration of using the library with the simplest example, in which a welcome message will be displayed on the screen. Here is what the custom code will look like in our case:
The main class that contains all user logic that interacts with the client device ( MyEventsDispatcher ) must be inherited from tau :: util :: BasicEventsDispatcher . It overrides 2 methods from the base class: onClientConnected () and packetReceived_clientDeviceInfo () . The first is called when the client connects. The second method will be executed when the client device information comes after the connection (the first packet after connection is sent by the client).
In our case, the first method is trivial - it only displays an informational message on the console. In the second method, the server sends a layout to the client — data on which interface should be displayed on the client.
All the code responsible for transferring data over the network is in main (). In this case, boost :: asio is used to implement the communication. In the tau :: util namespace there are corresponding abstractions, which makes this example as compact as possible. Using boost is optional - any implementation of TCP / IP sockets can be fairly easily used with a library.
As an example, we will use g ++ for compilation. In our case, the command will be as follows:
As you can see, the compiler is passed several additional parameters:
This set of options is the most common version of the assembly, which will allow the library to be included in any project with minimal effort.
You can get rid of them all if you want. If you use the library within the project, you do not need to specify -I $ LIBRARY_LOCATION and -D TAU_HEADERONLY . For compilers that are compatible with C ++ 11, the -D TAU_CPP_03_COMPATIBILITY option is not needed. Dependence on boost :: asio has only the network part, which can be easily rewritten without dependencies.
After compiling and running, the server starts listening on port 12345.
We start the client on the phone, create a connection and connect to it to display a message. This is how it will look like (I started the server on a remote computer via PuTTY, and the client was started in the emulator):
This example does not provide for the transmission and receipt of additional notifications between the client and the server, so let's move on to the following example.
In this example, we will add several different elements in our server, learn how to receive notifications from them, change their state and switch pages.
The server code will look like this:
All changes from the previous example were made in the class MyEventsDispatcher . The following event handler methods from the client have been added:
In addition, the layout sent to the client upon connection has changed accordingly.
Since we have a demo, the code in the handlers is as simple as possible - dumping event information into the console, as well as sending various commands to the client.
All commands will be sent to the client from the handler for pressing the packetReceived_buttonClick () buttons (of course, this is not necessarily done there, but it’s easier and clearer).
Each command corresponds to a packet transmitted from the server to the client. Formation and sending of these packets occurs when calling special methods defined in BasicEventsDispatcher :
Here is how this example works:
As you can see, for each action on the client device, the corresponding code is executed on the server. When the values of the user input elements change, the server receives a notification about the new value of the variable in this element. In the button press handler, various packages are sent from the server to the client (the type of package depends on which button was pressed). This example also shows how page switching works. Breaking the layout into pages allows you to group elements according to their functions. On the client screen, only one of the pages is always displayed, which reduces the load on the interface.
The last example today is the partial implementation of one of the tasks for which I started this project. This is the simplest keyboard input emulator for windows (uses the SendInput () winapi function).
The code for this example is on my githaba . I will not give it here - it does not demonstrate anything new in the use of the library compared to the second example. Here I will give only a demo of his work:
The code of this example is easy to expand to more complex tasks of emulating keyboard input.
Instead of a conclusion, I want to appeal to the community. Do you need a similar system? Do I invent another bicycle? What needs to be described in more detail? In which direction do you need to develop further?
The following development paths occur to me now (they are almost completely orthogonal, so I would like to prioritize):
In addition, I will be glad to hear criticism about the architecture and implementation of the library in the state in which it is now.
References:
Foreword
There are often situations when you want to manage your program from a phone or tablet, but it is impractical (or not possible) to write a separate application for this on the phone (too much effort for this project, no development experience for android, etc.) ). This situation periodically arose in me, and in the end I decided to deal with this problem once and for all. The result is a system, the architecture and use of which are described in this article.
purpose
Creating a system that allows you to implement a plug-in UI based on android devices in C ++ programs. At the same time, we wanted to minimize the dependencies of custom C ++ code on third-party libraries, as well as to abstract it from the data transfer protocol. The system should consist of two parts: C ++ libraries and android applications.
System architecture
The system has a client-server architecture, where clients are android devices, the server is a user program. Communication between them is carried out using TCP / IP sockets. To implement the communication protocol, the TAU library was created.
')
The main tasks for which the library is responsible:
- generation and processing of data transmitted between the server and the client
- transfer control to user code to react to various events that occurred on the client (UI event handlers)
- creating and maintaining a connection between the server and the client
- generation of necessary data structures for describing the configuration of UI elements displayed on clients
The library consists of the following namespaces:
- tau :: communications_handling - is responsible for the formation of packets, the parsing of data received from the client, the call of handlers in user code. Everything that happens between the moments of connecting and disconnecting the client is controlled by the class code from this namespace.
- tau :: layout_generation - contains functionality that allows you to create json-structures that describe the location and behavior of elements of the user interface. This data is then sent to the client and it displays the corresponding UI.
- tau :: util - contains various auxiliary functionality that is not necessary for the library to be used in a custom C ++ project, but often turns out to be useful. The classes in this namespace are the only ones that can use third-party libraries or non-standard compiler extensions. Therefore, here are the classes responsible for working with TCP / IP sockets - this is platform-specific code. Now there are two implementations of network communication: based on boost :: asio and C ++ / CLI. The removal of the implementation of all network calls outside of tau :: communications_handling allows the user of the library to write the entire network portion if desired.
- tau :: common - contains classes used from other parts of the library (they are derived here so that there are no dependencies between tau :: layout_generation and tau :: communications_handling)
Using. Example # 1 - hello, world
We'll start the demonstration of using the library with the simplest example, in which a welcome message will be displayed on the screen. Here is what the custom code will look like in our case:
Hidden text
#include <tau/layout_generation/layout_info.h> #include <tau/util/basic_events_dispatcher.h> #include <tau/util/boost_asio_server.h> class MyEventsDispatcher : public tau::util::BasicEventsDispatcher { public: MyEventsDispatcher( tau::communications_handling::OutgiongPacketsGenerator & outgoingGeneratorToUse): tau::util::BasicEventsDispatcher(outgoingGeneratorToUse) {}; virtual void packetReceived_requestProcessingError( std::string const & layoutID, std::string const & additionalData) { std::cout << "Error received from client:\nLayouID: " << layoutID << "\nError: " << additionalData << "\n"; } virtual void onClientConnected( tau::communications_handling::ClientConnectionInfo const & connectionInfo) { std::cout << "Client connected: remoteAddr: " << connectionInfo.getRemoteAddrDump() << ", localAddr : " << connectionInfo.getLocalAddrDump() << "\n"; } void packetReceived_clientDeviceInfo( tau::communications_handling::ClientDeviceInfo const & info) { using namespace tau::layout_generation; std::cout << "Received client information packet\n"; std::string greetingMessage = "Hello, habrahabr!"; sendPacket_resetLayout(LayoutInfo().pushLayoutPage( LayoutPage(tau::common::LayoutPageID("FIRST_LAYOUT_PAGE"), LabelElement(greetingMessage))).getJson()); } }; int main(int argc, char ** argv) { boost::asio::io_service io_service; short port = 12345; tau::util::SimpleBoostAsioServer<MyEventsDispatcher>::type s(io_service, port); std::cout << "Starting server on port " << port << "...\n"; s.start(); std::cout << "Calling io_service.run()\n"; io_service.run(); return 0; } The main class that contains all user logic that interacts with the client device ( MyEventsDispatcher ) must be inherited from tau :: util :: BasicEventsDispatcher . It overrides 2 methods from the base class: onClientConnected () and packetReceived_clientDeviceInfo () . The first is called when the client connects. The second method will be executed when the client device information comes after the connection (the first packet after connection is sent by the client).
In our case, the first method is trivial - it only displays an informational message on the console. In the second method, the server sends a layout to the client — data on which interface should be displayed on the client.
All the code responsible for transferring data over the network is in main (). In this case, boost :: asio is used to implement the communication. In the tau :: util namespace there are corresponding abstractions, which makes this example as compact as possible. Using boost is optional - any implementation of TCP / IP sockets can be fairly easily used with a library.
Compilation
As an example, we will use g ++ for compilation. In our case, the command will be as follows:
g++ -lboost_system -pthread -lboost_thread -D TAU_HEADERONLY -D TAU_CPP_03_COMPATIBILITY -I $LIBRARY_LOCATION main.cpp -o demo As you can see, the compiler is passed several additional parameters:
- include path to library sources (option -I $ LIBRARY_LOCATION )
- additional libraries needed for boost :: asio (options -lboost_system -pthread -lboost_thread )
- declare additional macros to indicate how we compile the library in our project ( -D TAU_HEADERONLY -D TAU_CPP_03_COMPATIBILITY )
This set of options is the most common version of the assembly, which will allow the library to be included in any project with minimal effort.
You can get rid of them all if you want. If you use the library within the project, you do not need to specify -I $ LIBRARY_LOCATION and -D TAU_HEADERONLY . For compilers that are compatible with C ++ 11, the -D TAU_CPP_03_COMPATIBILITY option is not needed. Dependence on boost :: asio has only the network part, which can be easily rewritten without dependencies.
After compiling and running, the server starts listening on port 12345.
We start the client on the phone, create a connection and connect to it to display a message. This is how it will look like (I started the server on a remote computer via PuTTY, and the client was started in the emulator):
Creating a server connection
This example does not provide for the transmission and receipt of additional notifications between the client and the server, so let's move on to the following example.
Example 2 - a more detailed demonstration of the system's capabilities
In this example, we will add several different elements in our server, learn how to receive notifications from them, change their state and switch pages.
The server code will look like this:
Hidden text
#include <tau/layout_generation/layout_info.h> #include <tau/util/basic_events_dispatcher.h> #include <tau/util/boost_asio_server.h> namespace { std::string const INITIAL_TEXT_VALUE("initial text"); tau::common::LayoutID const LAYOUT_ID("SAMPLE_LAYOUT_ID"); tau::common::LayoutPageID const LAYOUT_PAGE1_ID("LAYOUT_PAGE_1"); tau::common::LayoutPageID const LAYOUT_PAGE2_ID("LAYOUT_PAGE_2"); tau::common::ElementID const BUTTON_WITH_NOTE_TO_REPLACE_ID("BUTTON_WITH_NOTE_TO_REPLACE"); tau::common::ElementID const BUTTON_TO_RESET_VALUES_ID("BUTTON_TO_RESET_NOTES"); tau::common::ElementID const BUTTON_TO_PAGE_1_ID("BUTTON_TO_PG1"); tau::common::ElementID const BUTTON_TO_PAGE_2_ID("BUTTON_TO_PG2"); tau::common::ElementID const BUTTON_1_ID("BUTTON_1"); tau::common::ElementID const BUTTON_2_ID("BUTTON_2"); tau::common::ElementID const BUTTON_3_ID("BUTTON_3"); tau::common::ElementID const BUTTON_4_ID("BUTTON_4"); tau::common::ElementID const TEXT_INPUT_ID("TEXT_INPUT"); tau::common::ElementID const BOOL_INPUT_ID("BOOL_INPUT"); tau::common::ElementID const LABEL_ON_PAGE2_ID("LABEL_ON_PAGE2"); }; class MyEventsDispatcher : public tau::util::BasicEventsDispatcher { public: MyEventsDispatcher( tau::communications_handling::OutgiongPacketsGenerator & outgoingGeneratorToUse): tau::util::BasicEventsDispatcher(outgoingGeneratorToUse) {}; virtual void packetReceived_requestProcessingError( std::string const & layoutID, std::string const & additionalData) { std::cout << "Error received from client:\nLayouID: " << layoutID << "\nError: " << additionalData << "\n"; } virtual void onClientConnected( tau::communications_handling::ClientConnectionInfo const & connectionInfo) { std::cout << "Client connected: remoteAddr: " << connectionInfo.getRemoteAddrDump() << ", localAddr : " << connectionInfo.getLocalAddrDump() << "\n"; } virtual void packetReceived_clientDeviceInfo( tau::communications_handling::ClientDeviceInfo const & info) { using namespace tau::layout_generation; std::cout << "Received client information packet\n"; LayoutInfo resultLayout; resultLayout.pushLayoutPage(LayoutPage(LAYOUT_PAGE1_ID, EvenlySplitLayoutElementsContainer(true) .push(EvenlySplitLayoutElementsContainer(false) .push(BooleanInputLayoutElement(true).note(INITIAL_TEXT_VALUE).ID(BOOL_INPUT_ID)) .push(ButtonLayoutElement().note(INITIAL_TEXT_VALUE) .ID(BUTTON_WITH_NOTE_TO_REPLACE_ID))) .push(TextInputLayoutElement().ID(TEXT_INPUT_ID).initialValue(INITIAL_TEXT_VALUE)) .push(EmptySpace()) .push(EmptySpace()) .push(EmptySpace()) .push(EvenlySplitLayoutElementsContainer(false) .push(ButtonLayoutElement().note("reset notes").ID(BUTTON_TO_RESET_VALUES_ID)) .push(EmptySpace()) .push(ButtonLayoutElement().note("go to page 2").ID(BUTTON_TO_PAGE_2_ID) .switchToAnotherLayoutPageOnClick(LAYOUT_PAGE2_ID)) ) ) ); resultLayout.pushLayoutPage(LayoutPage(LAYOUT_PAGE2_ID, EvenlySplitLayoutElementsContainer(true) .push(EvenlySplitLayoutElementsContainer(false) .push(ButtonLayoutElement().note("1").ID(BUTTON_1_ID)) .push(ButtonLayoutElement().note("2").ID(BUTTON_2_ID))) .push(EvenlySplitLayoutElementsContainer(false) .push(ButtonLayoutElement().note("3").ID(BUTTON_3_ID)) .push(ButtonLayoutElement().note("4").ID(BUTTON_4_ID))) .push(EvenlySplitLayoutElementsContainer(true) .push(LabelElement("").ID(LABEL_ON_PAGE2_ID)) .push(ButtonLayoutElement().note("back to page 1").ID(BUTTON_TO_PAGE_1_ID))) )); resultLayout.setStartLayoutPage(LAYOUT_PAGE1_ID); sendPacket_resetLayout(resultLayout.getJson()); } virtual void packetReceived_buttonClick( tau::common::ElementID const & buttonID) { std::cout << "event: buttonClick, id=" << buttonID << "\n"; if (buttonID == BUTTON_TO_RESET_VALUES_ID) { sendPacket_updateTextValue(TEXT_INPUT_ID, INITIAL_TEXT_VALUE); } else if (buttonID == BUTTON_TO_PAGE_1_ID) { sendPacket_changeShownLayoutPage(LAYOUT_PAGE1_ID); } else if (buttonID == BUTTON_1_ID) { sendPacket_changeElementNote(LABEL_ON_PAGE2_ID, "Button 1 pressed"); } else if (buttonID == BUTTON_2_ID) { sendPacket_changeElementNote(LABEL_ON_PAGE2_ID, "Button 2 pressed"); } else if (buttonID == BUTTON_3_ID) { sendPacket_changeElementNote(LABEL_ON_PAGE2_ID, "Button 3 pressed"); } else if (buttonID == BUTTON_4_ID) { sendPacket_changeElementNote(LABEL_ON_PAGE2_ID, "Button 4 pressed"); } } virtual void packetReceived_layoutPageSwitched( tau::common::LayoutPageID const & newActiveLayoutPageID) { std::cout << "event: layoutPageSwitch, id=" << newActiveLayoutPageID << "\n"; } virtual void packetReceived_boolValueUpdate( tau::common::ElementID const & inputBoxID, bool new_value, bool is_automatic_update) { std::cout << "event: boolValueUpdate, id=" << inputBoxID << ", value=" << new_value << "\n"; } virtual void packetReceived_textValueUpdate( tau::common::ElementID const & inputBoxID, std::string const & new_value, bool is_automatic_update) { std::cout << "event: textValueUpdate, id=" << inputBoxID << ",\n\tvalue=" << new_value << "\n"; sendPacket_changeElementNote(BOOL_INPUT_ID, new_value); sendPacket_changeElementNote(BUTTON_WITH_NOTE_TO_REPLACE_ID, new_value); } }; int main(int argc, char ** argv) { boost::asio::io_service io_service; short port = 12345; tau::util::SimpleBoostAsioServer<MyEventsDispatcher>::type s(io_service, port); std::cout << "Starting server on port " << port << "...\n"; s.start(); std::cout << "Calling io_service.run()\n"; io_service.run(); return 0; } All changes from the previous example were made in the class MyEventsDispatcher . The following event handler methods from the client have been added:
- handler of event of pressing of the button packetReceived_buttonClick . The button ID is passed to the method as a parameter.
- handlers of packets that transmit variable values from client to server: packetReceived_boolValueUpdate, packetReceived_intValueUpdate, packetReceived_floatPointValueUpdate, packetReceived_textValueUpdate
- handler of event of change of the displayed page with packetReceived_layoutPageSwitched elements
In addition, the layout sent to the client upon connection has changed accordingly.
Since we have a demo, the code in the handlers is as simple as possible - dumping event information into the console, as well as sending various commands to the client.
All commands will be sent to the client from the handler for pressing the packetReceived_buttonClick () buttons (of course, this is not necessarily done there, but it’s easier and clearer).
Each command corresponds to a packet transmitted from the server to the client. Formation and sending of these packets occurs when calling special methods defined in BasicEventsDispatcher :
- sendPacket_resetLayout () - replacement of the entire layout
- sendPacket_requestValue () - request for the value of a variable in one of the inputs
- sendPacket_updateBooleanValue (), sendPacket_updateIntValue (), sendPacket_updateFloatPointValue, sendPacket_updateTextValue () - change the value of variables in inputs
- sendPacket_changeElementNote () - change any read-only text (text on buttons, checkboxes, labels)
- sendPacket_changeShownLayoutPage () - switching to another page with elements
- sendPacket_changeElementEnabledState () - switching the active status of items (inactive items are displayed, but they cannot be interacted with)
Here is how this example works:
Demonstration of working with UI elements
As you can see, for each action on the client device, the corresponding code is executed on the server. When the values of the user input elements change, the server receives a notification about the new value of the variable in this element. In the button press handler, various packages are sent from the server to the client (the type of package depends on which button was pressed). This example also shows how page switching works. Breaking the layout into pages allows you to group elements according to their functions. On the client screen, only one of the pages is always displayed, which reduces the load on the interface.
Example 3 - something useful
The last example today is the partial implementation of one of the tasks for which I started this project. This is the simplest keyboard input emulator for windows (uses the SendInput () winapi function).
The code for this example is on my githaba . I will not give it here - it does not demonstrate anything new in the use of the library compared to the second example. Here I will give only a demo of his work:
Keyboard Emulation
The code of this example is easy to expand to more complex tasks of emulating keyboard input.
Epilogue
Instead of a conclusion, I want to appeal to the community. Do you need a similar system? Do I invent another bicycle? What needs to be described in more detail? In which direction do you need to develop further?
The following development paths occur to me now (they are almost completely orthogonal, so I would like to prioritize):
- adding client applications for other platforms (IOS, PC)
- expansion of data transmission protocol functionality (heartbeat packets for connection monitoring, communication control commands)
- adding new UI elements (drop-down boxes, images, etc)
- deeper customization of the appearance of the elements on the client (colors, fonts, styles)
- support for more specific client device functions (notifications, sensors, volume buttons, etc)
- adding server libraries for other programming languages
In addition, I will be glad to hear criticism about the architecture and implementation of the library in the state in which it is now.
References:
Source: https://habr.com/ru/post/309882/
All Articles