Geekly Articles each Day
📜 ⬆️ ⬇️

Message broker for ZMQ-based service architecture - or developer’s leisure



A strong wind blew into the side of the ship. Small splashes and raindrops made the slightly unshaven face squint under the glasses. It was not just cold: the cold penetrated everywhere. Under the jacket, pants. His hands were numb from him and the blood froze. But the sailor knew that somewhere there behind the cape there is a quiet island where you can wait out bad weather.
The shore was met by an exhausted crew with the noise of trees and the whisper of reeds. People knew that they had only a day to rest, wash and continue the struggle with the elements.


')
But at this very moment of silence and unity with nature, devoid of tasks for survival, the brain of one of the participants, a programmer for life, began to look for problems with redoubled force. It was the brain of the author and he gave birth to the idea ...

Foreword



Made just-for-fun in 5 days aboard a ship with an average speed of 7 knots.

Introduction



The professional happiness of a programmer is quite simple - to write interesting tasks in your favorite language and get paid for it (preferably not small, although there is always little money). Such desires led to the birth of a whole approach in the form of stand-alone applications and the exchange process between them: SOA (in particular, SOAP / WSDL / XML-RPC / JSON-RPC, etc.), REST, micro-service architecture. The point is that following the precepts of Unix, a separate functionality is allocated to applications, and the data exchange between them is specified separately.
One of my hobbies is related to the work of a distributed network of small modules: smart home, computing system and other similar tasks. For communication between them, it is convenient to use a central message broker. Typical solution: RabbitMQ , Redis, ActiveMQ and other similar solutions. From the monsters of the industry can be noted IBM Broker, IBM MQ , Tibco .
But I didn't like something about them.

"Fatal flaws" of existing solutions




The choice fell on our own implementation of a certain universal, fast, low-resource, simple protocol message router.

Components



Communication layer in the form of ZeroMQ for easy messaging.
There are difficulties with programming language. JVM-based, Python, Ruby dismissed because of virtual machines, and therefore excessive consumption of resources. I wanted Rust, but after the start of implementation I realized that I had to wait for the further stabilization of the standard library. Go - it would be great if there was a native zmq implementation. As a result, C ++ / C.

Primary implementation



Broker consists of:


The connection point to the broker is a socket of type ROUTER. When receiving data from a socket REQ (from clients), a message is generated containing a unique client code. Next, a package is created for the request to the service, consisting of the service name, customer number and other data. Schematically, this can be represented as:



The code can be viewed on GitHub .

Secondary implementation



The implementation seemed very simple. It was necessary to invent and solve some more problems.

How to use multiple instances of services with the same name?


Valid for load balancing and fault tolerance.
The fact is that if the same connection name is used in the DEALER-to-ROUTER mode, then only the last connection will be used.
The solution is a separate load balancer for each service connecting to the broker and creating a separate connection point in the DEALER-to-DEALER mode. In this case, for services, it is enough to change the connection address to the balancer instead of the broker.

Options
 USAGE: 

    waha-proxy-balance [-b <zmq bind>] [-v] -n <name> [-u <zmq bind>] [-]
                        [--version] [-h]


 Where: 

    -b <zmq bind>, --backend <zmq bind>
      Backend binding for proxy

    -v, --verbose
      Show much more output

    -n <name>, --name <name>
      (required) Balancing service name

    -u <zmq bind>, --url <zmq bind>
      Broker url

    -, --ignore_rest
      Ignores the rest of the labeled arguments following this flag.

    --version
      Displays version information and exits.

    -h, --help
      Displays usage information and exits.


    Broker balance module



The code here is on github

How to provide access to services via HTTP?



Considering the massive love of the people for the HTTP interface, which, given the ability to work even through a smart toaster, is not without meaning, you need to do it right: json / plain output, support for both GET and POST requests, and JSONP for every fireman.
With the help of Poco libraries, it turned out to be implemented as a separate daemon.



Options
 USAGE: 

    waha-proxy-http [-t <ms>] [--threads <int>] [-p <int>] [-v] [-u <zmq
                     bind>] [-] [--version] [-h]


 Where: 

    -t <ms>, --timeout <ms>
      Request timeout without `timeout` param.  -1 - infinity

    --threads <int>
      HTTP max threads

    -p <int>, --port <int>
      HTTP binding port

    -v, --verbose
      Show much more output

    -u <zmq bind>, --url <zmq bind>
      Broker url

    -, --ignore_rest
      Ignores the rest of the labeled arguments following this flag.

    --version
      Displays version information and exits.

    -h, --help
      Displays usage information and exits.


    Broker HTTP proxy (REST like) module



The code here is on github

Why poco?
Why not boost / libevent or something? Because Poco is a very thin wrapper over system calls, without too much overweight, is collected with minimal effort and excellent documentation. And it just is much clearer to me than the boost.


How to make a good administrator and just a person who is reluctant to program?



We make a daemon that calls any other program, sends the message fields to it as arguments and interprets the output as an answer. Simplified such a CGI.

Options
 USAGE: 

    waha-service-script [-s <script>] [--no-stdout] [-e] [-r] -n <name>
                         [-v] [-u <zmq bind>] [-] [--version] [-h] <string>
                         ...


 Where: 

    -s <script>, --script <script>
      Script path for 'script' mode

    --no-stdout
      Do not use stdout as message

    -e, --stderr
      Append script stderr output

    -r, --ret-code
      Append script return code as last field in message

    -n <name>, --name <name>
      (required) Balancing service name

    -v, --verbose
      Show much more output

    -u <zmq bind>, --url <zmq bind>
      Broker url

    -, --ignore_rest
      Ignores the rest of the labeled arguments following this flag.

    --version
      Displays version information and exits.

    -h, --help
      Displays usage information and exits.

    <string> (accepted multiple times)
      Predefined args for script


    Broker script service



The code here is on github

How to do even better?



Make access to all services through the console. With plain / hex / base64 / json input and output.

Options
 USAGE: 

    waha-cli [-t <ms>] -n <name> [-v] [-u <zmq bind>] [-p] [-e <plain
              | base64 | hex>] [--out-sep <char>] [-o <empty | plain | delimited
              | json>] [-d <plain | base64 | hex>] [--in-sep <char>] [-i <args
              | plain | delimited | json>] [-] [--version] [-h] <string> ...


 Where: 

    -t <ms>, --timeout <ms>
      Request timeout.  -1 - infinity

    -n <name>, --name <name>
      (required) Remote service name

    -v, --verbose
      Show much more output

    -u <zmq bind>, --url <zmq bind>
      Broker url

    -p, --pretty
      Pretty JSON output for 'json'

    -e <plain | base64 | hex>, --encoder <plain | base64 | hex>
      Output encoder for 'delimited' output

    --out-sep <char>
      Output separator

    -o <empty | plain | delimited | json>, --output <empty | plain | delimited | json>
      Output mode in CLI mode

    -d <plain | base64 | hex>, --decoder <plain | base64 | hex>
      Input decoder for 'delimited' input

    --in-sep <char>
      Input separator

    -i <args | plain | delimited | json>, --input <args | plain | delimited | json>
      Input mode in CLI mode

    -, --ignore_rest
      Ignores the rest of the labeled arguments following this flag.

    --version
      Displays version information and exits.

    -h, --help
      Displays usage information and exits.

    <string> (accepted multiple times)
      Args in request message for for 'args' input


    ZMQ broker console client interface



The code here is github

Build everything



GIT + CMake + make

 git clone https://github.com/reddec/waha.git && cd waha && mkdir build && cd build
 cmake ../ -DCMAKE_BUILD_TYPE = Release
 make && make package


The Debian package will be in waha - *. Deb, for the rest of waha - *. Zip

Usage example



Authorization of users with a large read load and a small write.



Run broker (default port is 10000)
 $ waha-broker


Run balancer for reading (port 10001 by default)
 $ waha-proxy-balance -n system.user.check


Run httpasswd to check login / password (run as many times as you want to distribute the load). Displays only the exit code. Everything that comes after - is passed as arguments to the script.
 $ waha-service-script -u tcp: //127.0.0.1: 10001 -n system.user.check -s htpasswd --ret-code --no-stdout - -bv users.passwd


Run the script to add users (without balancing) along with the output
 $ waha-service-script -n system.user.add -s htpasswd --ret-code --stderr - -b users.passwd


Starting the HTTP interface (default port 9001)

 $ waha-proxy-http


sample request

 http://127.0.0.1:9001/system.user.check?args=["admin","admin "]
 http://127.0.0.1:9001/system.user.check?args=["admin","admin""&callback=func123


Using the command interface



Output without formatting:
  $ waha-cli -n system.user.check admin admin

JSON output:
  $ waha-cli -o json -n system.user.check admin admin


Total



Modular, working message broker, written during a vacation for recreation and entertainment by a programmer just for fun. So that the code does not disappear, put it in public access.
If anyone needs this - please consult.

Source: https://habr.com/ru/post/262977/

More articles:


All Articles