Overview of MQ (Messages queue) for use in projects on PHP. Part 1

In parallel to the main work, I am thinking about the implementation of the architecture for game projects in the “background” (let me remind you that the main area of ​​my interests and work is the creation of online browser games). Lately, I am increasingly returning to the idea that it would be interesting to implement the main game server based on message queues (MQ or Messages queue). That is, the engine of such a game will be a set of components that will communicate with each other through asynchronous messages, and each component can be either a message generator or a subscriber, that is, execute other messages.

This approach, as I understand it, is widely used in the Java world, there is a Java Message Service ( JMS ) standard for this and message brokers are used and the Enterprise service bus ( ESB ) architecture is based on this, for example, Apache ServiceMix . But for us, this is still a high sphere of large projects, and in the specifics of the web and web-based applications, I would like to consider whether something like this can be done, but with less cost, and to provide application resiliency, load balancing and asynchronous processing. And of course, it is very desirable that this be implemented in PHP as the main language of implementation of all server components.

And so, once again - MQ, this is the architecture and middleware, which is engaged in collecting, storing and routing (distributing) messages between components. I do not pretend to be complete, and, quite possibly, do not take into account many nuances, so do not consider my definitions as an axiom, it is best to read additional literature if you want to understand MQ architecture more deeply (for example, these articles: [ 1 ], [ 2 ], [ 3 ]) and definition in Wikipedia - Message queue
')
For now, let's look very briefly, at a glance, which systems for implementing message systems are there, and, most importantly, which of them can be considered as the basis for the specifics of my project. And although I thought to consider only PHP implementation, it turned out that it was necessary to “dig deeper”, so we will touch on systems in other languages, but with the ability to interact with PHP applications.

Apache ActiveMQ is an open implementation of Message Broker and Enterprise Integration Patterns (if very briefly now, an extension for implementing additional processing according to the rules). This project, in my opinion, of all open, the most powerful and developing, version 5.1 recently released. It implements many standards and provides all the features needed for Enterprise-level solutions, it is included in the Apache Java technology stack. What interested me is the possibility of cross-language messaging, which means that customers can be implemented in any language. For Java, C, C ++, C # platforms , this is the OpenWire library , which implements the Wire protocol for native access to MQ, for other languages, including, interestingly, PHP, is Stomp - an implementation of libraries for different scripting languages, which turns messages into JMS. By the way, if you need to provide secure communication and messaging, you can use SSL.

MQS (Minimalist Queue Services) is a project, so to speak, from the other end. This small system written in Perl organizes a message queue system using the XML-RPC protocol, the messages themselves can be stored in any database or in files. Unfortunately, the project seems to be abandoned, since the latest news on the site is dated April 2005, and the current version is 0.0.14.

Spread is another implementation of the message queue, this time in C ++, and there are versions for various platforms, including Win32, Linux, BSD and MacOS. The current version is 4.0. The system is distributed and focused on high-performance systems, where there are a lot of customers and, accordingly, communications. Declared support, in the latest 4.0 version, technology Virtual Synchrony . What is interesting - the delivery immediately includes binary versions for several platforms, as well as built-in interfaces for some languages ​​- C / C ++, Java, Perl, Python, Ruby. It is strange that the fourth P - PHP is not among them, but there is an extension in the PECL that implements the entire Spread API. The current version 2.1 and quite new, it means the project is developing. There are also implementations for other languages, including alternatives to built-in interfaces, so look at the list here , there is an extension even for MS Excel. Among the interesting projects is the mod_log_spread Apache module, which allows you to collect access logs from several web servers.

RabbitMQ is a high-performance platform written on Erlang, based on the Open Telecom Platform, which means it is a very reliable and scaled system, often used in telecommunication applications and other similar systems. There is an interface only for Java and C ++. The system supports the standard AMQP ( Open Standard for Messaging Middleware ). The system is interesting, only Erlang would know, although something tells me that by designing the entire server module on this platform, we would get a lot of "buns", in particular, the most popular Jabber server ejabberd was written on this platform, which apply in online gaming projects.

Beanstalkd is also an interesting project created as part of the development of one of the applications for the social network Facebook, which is used by about 10 million people (an application, not a network). It is a specialized server for storing and processing job queues, using data storage in memory to ensure speed, but at the expense of fault tolerance. This project is very similar to MemcacheQ already described by us , and the developers themselves are grateful to the creators of memcached for the principles of architecture and protocol. The system is designed to create an asynchronous queue of custom task handlers that do not require an immediate response, for example, sending letters, background processing, etc. There are client APIs for various languages, including Erlang, OCaml, Perl, PHP, Python, Ruby. For PHP, the library is located here and still has version 0.11, development itself started just a couple of months ago (judging by the project registration). You can read a great overview in more detail : The Beanstalk Messaging Queue Server is written in C, which provides high performance, but the project’s specificity in terms of storing all data in RAM is not suitable for those areas where maximum resiliency is urgently needed, even at the cost of additional software and costs to store data in the database.

That's all for now. What about PHP, you ask? The solutions described are based on third-party platforms, but they have libraries for communicating with PHP applications. But there are also native, PHP solutions of this kind of applications, of course, not complete analogs, here the specificity of the PHP environment is already affected, but sufficient for many applications. I will tell you about such decisions in the continuation of this material.