AMQP in Russian
Today, quite a bit of information about the AMQP (Advanced Message Queueing Protocol) protocol and its application, especially in Russian. In general, it is a wonderful, already widely supported open protocol for transferring messages between system components with low latency and at high speed. In this case, the semantics of messaging is customized to the needs of a particular project. Such solutions existed before, but this is the first standard for which there are a large number of free implementations.
The basic idea is that individual subsystems (or independent applications) can exchange messages in an arbitrary manner through an AMQP broker that performs routing, it is possible to guarantee delivery, distribution of data streams, and subscription to the desired types of messages. As classical examples, financial applications are usually cited, for example, related to the delivery of information about securities rates to consumers in real time, it is also possible RPC-interaction of two subsystems that have no connection with each other (interaction through a common AMQP protocol) and so further and the like.
Today, the topic of real-time information delivery is extremely relevant (just recall at least Twitter, Google Wave). Here, messaging systems can serve as an internal data exchange mechanism that ensures the delivery of data (data changes) to customers.
')
My goal today is not to talk about how to write applications for AMQP. I just want to tell you a little about the fact that this is not at all scary, not very difficult, and it really works, although the standard is still in development, new versions of the protocol, brokers, etc. come out. But this is already quite production-quality. I'll tell you only basic tips to help “enter” the protocol.
For a start, a small collection of links (mostly in English): what is messaging in general and why AMQP is like that ( Messaging in general and AMQP design ); comparison of different messaging implementations, in particular those based on AMQP ( Message Queue Comparison ); Thrift-enabled AMQP client library for Twisted Framework (Python) ( Thrift, AMQP in Twisted ); Red Hat's guide on what is messaging and how to work with AMQP describes their “boxed” product based on AMQP, but is also suitable for any AMQP brokers ( AMQP Programming Tutorial for C ++, Java, and Python ); there is a lot of documentation, descriptions of architectural solutions on the ZeroMQ website, which is not exactly an AMQP broker, but a common architecture, implementation details are of particular interest; Duncan McGregor's review article on txAMQP and AMQP in general ( A Simfonia on Messaging with txAMQP , II , III ).
Next you need to select the AMQP broker that you will use. When choosing, you should consider the server’s actual characteristics: speed, reliability, ease of installation and support, but also look carefully at the broker’s version of the AMQP protocol — it must match the version of the client’s AMQP library. From brokers, I would advise RabbitMQ , written in Erlang, and Qpid , C ++ (AMQP 0-10) and Java (0-8, 0-9) versions.
The AMQP protocol itself is quite interesting: at the lowest level, the coding format of the data into a binary form for transmission over a TCP connection is defined, the format for transferring RPC requests between the server and the client is above. The semantics of working with messages, creating queues, etc. described in the XML specification, which essentially sets the RPC interface of the server and the client (examples of such XML files for versions 0-8 and 0-10 ). This XML is the latest and final protocol specification. Moreover, protocol versions 0-8 and 0-10 differ so much that it is hardly possible to support them simultaneously in one program. Even more interesting, sometimes these spec files for different AMQP brokers, formally supporting the same version of the protocol, are so different that they are not interchangeable. But these are rather minor technical problems.
AMQP is based on three concepts:
At AMQP, I was attracted by an efficient binary protocol, protocol focusing on minimal latency and configuration flexibility. I used it to send messages to all servers of the cluster (fanout exchange) and to organize analogues of “mailboxes” to which messages addressed to specific clients (direct exchange) are delivered. To receive messages, there is no need to poll the server, it is enough to subscribe to messages from the queue, and the server will transmit them at the moment they appear.
As the client library, I chose the txAMQP library for Twisted Framework (Python). In general, everything works, but somewhere you need small “finishing touches” and “spin-ups”, which I plan to publish on a launchpad. In AMQP and around AMQP there is a lot of interesting and promising. For example, the RabbitMQ broker can scale and work in a single cluster. I think this is a very useful and promising technology.
The basic idea is that individual subsystems (or independent applications) can exchange messages in an arbitrary manner through an AMQP broker that performs routing, it is possible to guarantee delivery, distribution of data streams, and subscription to the desired types of messages. As classical examples, financial applications are usually cited, for example, related to the delivery of information about securities rates to consumers in real time, it is also possible RPC-interaction of two subsystems that have no connection with each other (interaction through a common AMQP protocol) and so further and the like.
Today, the topic of real-time information delivery is extremely relevant (just recall at least Twitter, Google Wave). Here, messaging systems can serve as an internal data exchange mechanism that ensures the delivery of data (data changes) to customers.
')
My goal today is not to talk about how to write applications for AMQP. I just want to tell you a little about the fact that this is not at all scary, not very difficult, and it really works, although the standard is still in development, new versions of the protocol, brokers, etc. come out. But this is already quite production-quality. I'll tell you only basic tips to help “enter” the protocol.
For a start, a small collection of links (mostly in English): what is messaging in general and why AMQP is like that ( Messaging in general and AMQP design ); comparison of different messaging implementations, in particular those based on AMQP ( Message Queue Comparison ); Thrift-enabled AMQP client library for Twisted Framework (Python) ( Thrift, AMQP in Twisted ); Red Hat's guide on what is messaging and how to work with AMQP describes their “boxed” product based on AMQP, but is also suitable for any AMQP brokers ( AMQP Programming Tutorial for C ++, Java, and Python ); there is a lot of documentation, descriptions of architectural solutions on the ZeroMQ website, which is not exactly an AMQP broker, but a common architecture, implementation details are of particular interest; Duncan McGregor's review article on txAMQP and AMQP in general ( A Simfonia on Messaging with txAMQP , II , III ).
Next you need to select the AMQP broker that you will use. When choosing, you should consider the server’s actual characteristics: speed, reliability, ease of installation and support, but also look carefully at the broker’s version of the AMQP protocol — it must match the version of the client’s AMQP library. From brokers, I would advise RabbitMQ , written in Erlang, and Qpid , C ++ (AMQP 0-10) and Java (0-8, 0-9) versions.
The AMQP protocol itself is quite interesting: at the lowest level, the coding format of the data into a binary form for transmission over a TCP connection is defined, the format for transferring RPC requests between the server and the client is above. The semantics of working with messages, creating queues, etc. described in the XML specification, which essentially sets the RPC interface of the server and the client (examples of such XML files for versions 0-8 and 0-10 ). This XML is the latest and final protocol specification. Moreover, protocol versions 0-8 and 0-10 differ so much that it is hardly possible to support them simultaneously in one program. Even more interesting, sometimes these spec files for different AMQP brokers, formally supporting the same version of the protocol, are so different that they are not interchangeable. But these are rather minor technical problems.
AMQP is based on three concepts:
- Message (message) is a unit of transmitted data, its main part (content) is not interpreted by the server in any way, structured headers can be attached to the message.
- Exchange point - messages are sent to it. The point of exchange distributes messages in one or more queues. However, messages are not stored at the exchange point. Exchange points are of three types: fanout - the message is transmitted to all queues attached to it; direct - the message is transferred to the queue with the same name as the routing key (the routing key is specified when sending the message); topic is something between fanout and exchange, the message is sent to the queues for which the mask matches the routing key, for example,
app.notification.sms.*- all messages sent with the keys beginning withapp.notification.smswill be delivered to theapp.notification.sms. - Queue — messages are stored here until collected by the client. The client always picks up messages from one or more queues.
At AMQP, I was attracted by an efficient binary protocol, protocol focusing on minimal latency and configuration flexibility. I used it to send messages to all servers of the cluster (fanout exchange) and to organize analogues of “mailboxes” to which messages addressed to specific clients (direct exchange) are delivered. To receive messages, there is no need to poll the server, it is enough to subscribe to messages from the queue, and the server will transmit them at the moment they appear.
As the client library, I chose the txAMQP library for Twisted Framework (Python). In general, everything works, but somewhere you need small “finishing touches” and “spin-ups”, which I plan to publish on a launchpad. In AMQP and around AMQP there is a lot of interesting and promising. For example, the RabbitMQ broker can scale and work in a single cluster. I think this is a very useful and promising technology.
Source: https://habr.com/ru/post/62502/
All Articles