phpDaemon - asynchronous application framework
Today we will talk about phpDaemon - an asynchronous modular daemon framework that takes on I / O (libevent) processing and other low-level tasks inherent in demons. It makes it easy to write correct network applications with blackjack and whores.
From the box are the servers FastCGI, HTTP, CGI, FlashPolicy, Telnet, WebSocket (!) - yes, this is themagic pendal from Google. And clients mysql, memcached, mongodb ... And much more, a complete list under the cut. Working with the network is really easy. A medium programmer can write, for example, an IRC bot in a matter of hours.
As a visual example, I implemented this chat on phpDaemon + WebSocket + MongoDB + jQuery . It clearly demonstrates the advantages of this technology: instant message delivery, minimal data overhead, high performance, and application scaling horizontally. Sources of this chat (currently 17 KB). Please note, the chat was tested and works in Chrome, FF, IE6 +, Iron, Safari.
The scope of phpDaemon is very wide both in Web development and beyond. With it, it is good to do real-time multiplayer games (for example, Flash), services with instant interaction, chat rooms, IM gates ... It is also good to use the necessary utility in the household, such as a flashpolicy server.
The very first real application of the project in production is the instant delivery of personal messages: there was an invisible swf file (Flash) of several kilobytes on the site’s pages, it connected to the server and waited for commands. As soon as the user received messages, Flash sent a command to the page (Javascript), which in turn immediately displayed a pop-up icon and clicked. My colleagues and I were amazed at the speed of the reaction, the sound from a nearby computer was heard even before the sender had time to tear off a finger from the left mouse button.
And quite recently, a server has been implemented on request that provides an API to Asterisk (for IP telephony), it perfectly holds the load.
Applications in phpDaemon contain only processing logic, and all low-level calls occur automatically. The project is written entirely in PHP, and you probably already wondered about performance. I / O occurs through libevent, a well-proven library that is used in memcached and in other well-known projects. At the same time, the speed of executing instructions in PHP is very high, it surpasses in this many other languages of its group. In the usual synchronous scripts for all, the lion’s share of time is spent on launching the environment and blocking during I / O, for example, when querying the database, the regular script will not continue until the response is received. There are no such locks in phpDaemon: they sent a request, hung up the callback function if necessary, and went about their business.
phpDaemon is one master process and many workflows, each of which is an application execution environment. The application is an AppInstance successor class that describes its reaction to various events. When running workflows, they create instances of application classes and run the init () method. In the init () method, an application can bind a socket, connect somewhere, or open a local descriptor (one does not exclude the other). The application can also interact with other applications in the workflow, declaring the necessary parameters in them: for example, register the route in WebSocketServer.
After it zabindilo a socket, at receipt of suitable connections the event onAccepted (connId, addr) will be caused. An example of this method from WebSocketServer:
The server providing the crossdomain policy on port 843 weighs only 1.67 kb - FlashPolicy.php . Meanwhile, its clear advantage over many analogs is obvious: it cannot be put in opening a dozen or another telnet sessions. Many popular implementations of flashpolicyd DoS-open even a single session, which does not send anything, since they are synchronized processing, and the process is waiting for the weather by the sea.
')
For processing HTTP requests (including FastCGI), there is a separate entity — Request-derived classes. This entity has input parameters (get, post, cookie, server ...) and states - run / sleep / dead. Requests hang in queues, and the dispatcher calls them in order. If the request does not need to be interrupted, it can do all the work and return the exit code so that the dispatcher removes it from the queue. If the request performs operations that require waiting (for example, it makes a request to MongoDB), then it needs to do $ this-> sleep (30) in order to fall asleep a maximum of 30 per second. And in the callback function to the MongoDB request, it is enough to specify $ request-> wakeup () to immediately get rid of the dream. Then the dispatcher will contact him immediately. The request will be able to continue its execution, having a response from MongoDB. If the answer for some reason is not received, the request may display "We're sorry, try again shortly later." To complete the request in the run () method, either return 1 or $ this-> terminate () is called.
After the request headers are received, the server applications are accepted (boxed - FastCGI, HTTP) contact the appResolver, which determines to whom the application receives the request and calls the beginRequest method on the application. And he, in turn, creates and returns an instance of the class successor Request. Then the request object is put in a queue. An application can optionally add to the queue its own "requests" using the pushRequest method, this is necessary to call certain code at specified intervals (for example, this is done in MongoNode to poll the cursor).
POST-data (and multipart), Uploads, etc. ( page-example ) are fully supported by requests, moreover, there is no longer any need to manually process the UCS-2 encoding in requests (% uFFFF) - this happens automatically.
X-Sendfile (recording the response to the web server's request to a file) and Request-Body-File (reading the body part of the request from the file) are supported. You can start the execution of the request before the body is completely accepted, and you can program the processing of uploading (for example, throwing it right into memcached, and not into a temporary file on disk).
I foresee requests for specific figures showing performance gains, but it is incorrect to compare synchronous and asynchronous frameworks. It is clear that the latter is faster. However, the difference will depend directly on the application, namely, on the number and time of locks in the synchronous version. I will add only the subjective benchmark of the example page ... Requests per second: 4784.80 [# / sec] (mean).
At the moment, the latest stable version of libevent (1.4.10-stable) is used, libevent 2.0 will be released soon, this will allow worker processes to accept connections via epoll_wait and will bring an additional performance gain over the current version. As soon as possible.
Built-in control script:
Workflows are supported by the commands suid, sgid, chroot. They are set at the configuration level. Built-in dynamic MPM (Multi-Process Manager) determines the workload of workflows and launches new ones as part of the settings. Using the API, the MPM algorithm can be overridden in the configuration file. In developing the project, emphasis is placed on both the extensibility and interchangeability of components, and on ease of use.
The following modules are currently included in the distribution:
In addition, bundled applications are examples.
We will be glad to see your modules in the distribution kit!
License - LGPL. The project is relatively fresh, somewhat damp, but stable, and is used in production. Development is underway. In the absence of force majeure, fixes go from several minutes to two days after the report.
Web: http://github.com/kakserpom/phpdaemon
Group: http://groups.google.com/group/phpdaemon
IRC: irc.freenode.org #phpdaemon
E-Mail to me personally - kak.serpom.po.yaitsam@gmail.com
Thanks for attention!
PS If you like this kind of architecture - support is possible, even in principle, writing modules to order.
From the box are the servers FastCGI, HTTP, CGI, FlashPolicy, Telnet, WebSocket (!) - yes, this is the
As a visual example, I implemented this chat on phpDaemon + WebSocket + MongoDB + jQuery . It clearly demonstrates the advantages of this technology: instant message delivery, minimal data overhead, high performance, and application scaling horizontally. Sources of this chat (currently 17 KB). Please note, the chat was tested and works in Chrome, FF, IE6 +, Iron, Safari.
Where is it better to apply?
The scope of phpDaemon is very wide both in Web development and beyond. With it, it is good to do real-time multiplayer games (for example, Flash), services with instant interaction, chat rooms, IM gates ... It is also good to use the necessary utility in the household, such as a flashpolicy server.
The very first real application of the project in production is the instant delivery of personal messages: there was an invisible swf file (Flash) of several kilobytes on the site’s pages, it connected to the server and waited for commands. As soon as the user received messages, Flash sent a command to the page (Javascript), which in turn immediately displayed a pop-up icon and clicked. My colleagues and I were amazed at the speed of the reaction, the sound from a nearby computer was heard even before the sender had time to tear off a finger from the left mouse button.
And quite recently, a server has been implemented on request that provides an API to Asterisk (for IP telephony), it perfectly holds the load.
Architecture and features
Applications in phpDaemon contain only processing logic, and all low-level calls occur automatically. The project is written entirely in PHP, and you probably already wondered about performance. I / O occurs through libevent, a well-proven library that is used in memcached and in other well-known projects. At the same time, the speed of executing instructions in PHP is very high, it surpasses in this many other languages of its group. In the usual synchronous scripts for all, the lion’s share of time is spent on launching the environment and blocking during I / O, for example, when querying the database, the regular script will not continue until the response is received. There are no such locks in phpDaemon: they sent a request, hung up the callback function if necessary, and went about their business.
phpDaemon is one master process and many workflows, each of which is an application execution environment. The application is an AppInstance successor class that describes its reaction to various events. When running workflows, they create instances of application classes and run the init () method. In the init () method, an application can bind a socket, connect somewhere, or open a local descriptor (one does not exclude the other). The application can also interact with other applications in the workflow, declaring the necessary parameters in them: for example, register the route in WebSocketServer.
After it zabindilo a socket, at receipt of suitable connections the event onAccepted (connId, addr) will be caused. An example of this method from WebSocketServer:
public function onAccepted($connId,$addr)
{
$ this ->sessions[$connId] = new WebSocketSession($connId,$ this );
$ this ->sessions[$connId]->clientAddr = $addr;
}
* This source code was highlighted with Source Code Highlighter .The server providing the crossdomain policy on port 843 weighs only 1.67 kb - FlashPolicy.php . Meanwhile, its clear advantage over many analogs is obvious: it cannot be put in opening a dozen or another telnet sessions. Many popular implementations of flashpolicyd DoS-open even a single session, which does not send anything, since they are synchronized processing, and the process is waiting for the weather by the sea.
')
For processing HTTP requests (including FastCGI), there is a separate entity — Request-derived classes. This entity has input parameters (get, post, cookie, server ...) and states - run / sleep / dead. Requests hang in queues, and the dispatcher calls them in order. If the request does not need to be interrupted, it can do all the work and return the exit code so that the dispatcher removes it from the queue. If the request performs operations that require waiting (for example, it makes a request to MongoDB), then it needs to do $ this-> sleep (30) in order to fall asleep a maximum of 30 per second. And in the callback function to the MongoDB request, it is enough to specify $ request-> wakeup () to immediately get rid of the dream. Then the dispatcher will contact him immediately. The request will be able to continue its execution, having a response from MongoDB. If the answer for some reason is not received, the request may display "We're sorry, try again shortly later." To complete the request in the run () method, either return 1 or $ this-> terminate () is called.
After the request headers are received, the server applications are accepted (boxed - FastCGI, HTTP) contact the appResolver, which determines to whom the application receives the request and calls the beginRequest method on the application. And he, in turn, creates and returns an instance of the class successor Request. Then the request object is put in a queue. An application can optionally add to the queue its own "requests" using the pushRequest method, this is necessary to call certain code at specified intervals (for example, this is done in MongoNode to poll the cursor).
POST-data (and multipart), Uploads, etc. ( page-example ) are fully supported by requests, moreover, there is no longer any need to manually process the UCS-2 encoding in requests (% uFFFF) - this happens automatically.
X-Sendfile (recording the response to the web server's request to a file) and Request-Body-File (reading the body part of the request from the file) are supported. You can start the execution of the request before the body is completely accepted, and you can program the processing of uploading (for example, throwing it right into memcached, and not into a temporary file on disk).
I foresee requests for specific figures showing performance gains, but it is incorrect to compare synchronous and asynchronous frameworks. It is clear that the latter is faster. However, the difference will depend directly on the application, namely, on the number and time of locks in the synchronous version. I will add only the subjective benchmark of the example page ... Requests per second: 4784.80 [# / sec] (mean).
At the moment, the latest stable version of libevent (1.4.10-stable) is used, libevent 2.0 will be released soon, this will allow worker processes to accept connections via epoll_wait and will bring an additional performance gain over the current version. As soon as possible.
Manageability, administration, configuration
Built-in control script:
# phpd
usage: phpd (start | (hard) stop | update | reload | (hard) restart | fullstatus | status | configtest | help) ...
# phpd fullstatus
[STATUS] phpDaemon 0.2 is running (/var/run/phpdaemon.pid).
State of workers:
Total: 1
Idle: 1
Busy: 0
Shutdown: 1
Pre-init: 0
Wait-init: 0
Init: 0
Workflows are supported by the commands suid, sgid, chroot. They are set at the configuration level. Built-in dynamic MPM (Multi-Process Manager) determines the workload of workflows and launches new ones as part of the settings. Using the API, the MPM algorithm can be overridden in the configuration file. In developing the project, emphasis is placed on both the extensibility and interchangeability of components, and on ease of use.
Scabbard
The following modules are currently included in the distribution:
- FastCGI - allows you to connect to applications using the FastCGI protocol.
- HTTP - allows you to connect to applications via HTTP.
- CGI - allows you to run regular CGI applications via FastCGI, HTTP and any other transports.
- Flashpolicy - distributes the crossdomain policy for Flash on port 843.
- WebSocketServer - serves WebSocket sessions.
- MongoClient - MongoDB driver.
- MySQLClient - MySQL driver (you can also connect to SphinxQL through it).
- MySQLProxy - proxy MySQL-server.
- MongoNode is an implementation of the MongoDB slave node that allows you to hang events on changes to objects in the database.
- MemcacheClient - Memcache driver.
- LockServer is a distributed lock service.
- LockClient is a client for its own lock service.
- TelnetHoneypot is the simplest telnet server.
- RTEPServer / RTEPClient is an implementation of the Real-Time Events Protocol that allows clients to subscribe to and announce events.
- BitTorrentTracker - BitTorrent tracker using MongoDB.
- DebugConsole - a-la telnet interactive debug console server that allows you to run code in a running process.
In addition, bundled applications are examples.
We will be glad to see your modules in the distribution kit!
Rams in the cost, the refrigerator in the house
License - LGPL. The project is relatively fresh, somewhat damp, but stable, and is used in production. Development is underway. In the absence of force majeure, fixes go from several minutes to two days after the report.
Web: http://github.com/kakserpom/phpdaemon
Group: http://groups.google.com/group/phpdaemon
IRC: irc.freenode.org #phpdaemon
E-Mail to me personally - kak.serpom.po.yaitsam@gmail.com
Thanks for attention!
PS If you like this kind of architecture - support is possible, even in principle, writing modules to order.
Source: https://habr.com/ru/post/79377/
All Articles