An example of building a non-blocking web application
Recently, I have seen a couple of Habratopics ( one , two ) describing the use of non-blocking sockets and event-oriented programming on the web. I want to share my experience creating a web application on this technology.
Recently, I wanted to create my own service for checking invisibility of ICQ numbers . The checking algorithm is old and well-known, but still working - sending a specially crafted service message and analyzing the server response. It was necessary to keep several permanent connections to the ICQ server, as well as to have a web interface for verification requests. The obvious solution is to create a daemon that creates multiple threads for ICQ connections, and somehow receives commands from a web application that uses several workflow processes (or on a preforked architecture) to be able to handle http requests from several clients. But I decided to learn a new technology for myself and make an application that supports several connections and responds to customers using just one stream.
Perl has many implementations of event loops - frameworks for creating event-oriented applications. These are modules that provide an interface for registering handlers for various events (timer triggering, receiving a signal, appearing readable data in a socket), as well as a function after which the program execution is blocked and event processing begins. When an event occurs, the callback specified at registration occurs. Many event loops have the ability to specify a backend for notification of events, for example, high-performance kqueue and epoll.
There are many modules on CPAN that use certain event loops. Sometimes it happens that there are all the necessary modules to solve the problem, but they use different event loops with incompatible interfaces. What to do in such cases?
AnyEvent - DBI for event-based programming. AnyEvent provides an interface that allows you to change the used event loop to another at any time. Also, AnyEvent allows you to use together modules that use different event loops. That is why I wrote the ICQ protocol implementation using AnyEvent.
')
The primary task is to connect to the ICQ message server and implement the verification algorithm. The three modules on the CPAN were rather old and were executed on blocking sockets, so they did not suit me. After all, any blocking inside the callback will cause blocking the processing of all other events! But still, for simplicity, I first implemented the implementation on blocking sockets using numerous ready-made solutions in other languages and these modules, and then I began to redo everything on AnyEvent. Below is the code for receiving one ICQ-FLAP package:
The rest of the logic, lying in the
Great, the implementation of the ICQ protocol and the verification algorithm is ready, now we need a web interface. There is a great FastCGI protocol for connecting web applications to a web server, and on CPAN I found two of its asynchronous implementations for EV and IO :: Async . Chose EV because of its speed . Next, a simple url-dispatcher on attributes was made and a simple template engine Text :: MicroMason was bolted on - everything, the mini-framework for creating asynchronous web applications is ready.
Text :: MicroMason stores templates in compiled form in memory, which has a remarkable effect on performance, but what if you need to change the template? Do not stop the demon, breaking connections of all ICQ-clients? AnyEvent and EV provide the ability to set handlers on signals, this can be used.
Now when SIGUSR1 is received, the config will be reloaded, all old ICQ clients will be deleted and new ones will be created, and upon receipt of SIGUSR2 the templates will be reloaded. As in the case of the timer, you must save the value returned by EV :: signal / AnyEvent-> signal.
Recently, I wanted to create my own service for checking invisibility of ICQ numbers . The checking algorithm is old and well-known, but still working - sending a specially crafted service message and analyzing the server response. It was necessary to keep several permanent connections to the ICQ server, as well as to have a web interface for verification requests. The obvious solution is to create a daemon that creates multiple threads for ICQ connections, and somehow receives commands from a web application that uses several workflow processes (or on a preforked architecture) to be able to handle http requests from several clients. But I decided to learn a new technology for myself and make an application that supports several connections and responds to customers using just one stream.
Event loops
Perl has many implementations of event loops - frameworks for creating event-oriented applications. These are modules that provide an interface for registering handlers for various events (timer triggering, receiving a signal, appearing readable data in a socket), as well as a function after which the program execution is blocked and event processing begins. When an event occurs, the callback specified at registration occurs. Many event loops have the ability to specify a backend for notification of events, for example, high-performance kqueue and epoll.
There are many modules on CPAN that use certain event loops. Sometimes it happens that there are all the necessary modules to solve the problem, but they use different event loops with incompatible interfaces. What to do in such cases?
AnyEvent
AnyEvent - DBI for event-based programming. AnyEvent provides an interface that allows you to change the used event loop to another at any time. Also, AnyEvent allows you to use together modules that use different event loops. That is why I wrote the ICQ protocol implementation using AnyEvent.
')
ICQ protocol
The primary task is to connect to the ICQ message server and implement the verification algorithm. The three modules on the CPAN were rather old and were executed on blocking sockets, so they did not suit me. After all, any blocking inside the callback will cause blocking the processing of all other events! But still, for simplicity, I first implemented the implementation on blocking sockets using numerous ready-made solutions in other languages and these modules, and then I began to redo everything on AnyEvent. Below is the code for receiving one ICQ-FLAP package:
# <br/>
sub recv { <br/>
my ( $self ) = @_ ; <br/>
<br/>
# 6 - FLAP <br/>
sysread $self -> socket , my $data , 6 ; <br/>
# <br/>
my $length = unpack ( 'n' , substr ( $data , - 2 ) ) ; <br/>
# - <br/>
my $channel = unpack ( 'C' , substr ( $data , 1 , 1 ) ) ; <br/>
# , <br/>
sysread $self -> socket , $data , $length ; <br/>
# OC::ICQ::FLAP <br/>
return new OC :: ICQ :: FLAP ( $channel , $data ) ; <br/>
} <br/>
<br/>
# <br/>
sub connect { <br/>
my ( $self , $host , $port ) = @_ ; <br/>
<br/>
# AnyEvent::Handle <br/>
$self -> { io } = new AnyEvent :: Handle ( <br/>
connect => [ $host , $port ] , <br/>
# <br/>
on_error => sub { $self -> on_error ( @_ ) } , <br/>
on_disconnect => sub { $self -> on_disconnect ( @_ ) } , <br/>
) ; <br/>
<br/>
# , , 6 - FLAP <br/>
$self -> { io } -> push_read ( chunk => 6 , sub { $self -> on_read_header ( @_ ) } ) ; <br/>
} <br/>
<br/>
# , FLAP <br/>
sub on_read_header { <br/>
my ( $self , $io , $header ) = @_ ; <br/>
<br/>
# - OC::ICQ::FLAP <br/>
$self -> { flap_channel } = unpack ( 'C' , substr ( $header , 1 , 1 ) ) ; <br/>
# , <br/>
$io -> push_read ( chunk => unpack ( 'n' , substr ( $header , - 2 ) ) , sub { $self -> on_read_data ( @_ ) } ) ; <br/>
} <br/>
<br/>
# , <br/>
sub on_read_data { <br/>
my ( $self , $io , $data ) = @_ ; <br/>
<br/>
# , OC::ICQ::FLAP, <br/>
$self -> _process_flap ( new OC :: ICQ :: FLAP ( $self -> { flap_channel } , $data ) ) ; <br/>
# FLAP <br/>
$io -> push_read ( chunk => 6 , sub { $self -> on_read_header ( @_ ) } ) ; <br/>
}The rest of the logic, lying in the
_process_flap , almost did not have to redo it. To maintain the connection, you need to send empty FLAP on channel 5 every 2 minutes. To do this, use the AnyEvent-provided timer function:# , 2 <br/>
$self -> { keepalive_timer } = AnyEvent -> timer ( after => 120 , interval => 120 , cb => sub { $self -> send_keepalive } ) ; <br/>
<br/>
sub send_keepalive { <br/>
my ( $self ) = @_ ; <br/>
<br/>
if ( $self -> { state } == ONLINE ) { <br/>
$self -> send ( new OC :: ICQ :: FLAP ( 5 , '' ) ) ; <br/>
} <br/>
} <br/>
<br/>
# . $self->{keepalive_timer} <br/>
delete $self -> { keepalive_timer } ;Web interface
Great, the implementation of the ICQ protocol and the verification algorithm is ready, now we need a web interface. There is a great FastCGI protocol for connecting web applications to a web server, and on CPAN I found two of its asynchronous implementations for EV and IO :: Async . Chose EV because of its speed . Next, a simple url-dispatcher on attributes was made and a simple template engine Text :: MicroMason was bolted on - everything, the mini-framework for creating asynchronous web applications is ready.
Text :: MicroMason stores templates in compiled form in memory, which has a remarkable effect on performance, but what if you need to change the template? Do not stop the demon, breaking connections of all ICQ-clients? AnyEvent and EV provide the ability to set handlers on signals, this can be used.
my $sigusr1_watcher = EV :: signal ( 'USR1' , \&restart ) unless $^O =~ /MSWin32/ ; <br/>
my $sigusr2_watcher = EV :: signal ( 'USR2' , \&load_templates ) unless $^O =~ /MSWin32/ ; <br/>
Now when SIGUSR1 is received, the config will be reloaded, all old ICQ clients will be deleted and new ones will be created, and upon receipt of SIGUSR2 the templates will be reloaded. As in the case of the timer, you must save the value returned by EV :: signal / AnyEvent-> signal.
Source: https://habr.com/ru/post/69890/
All Articles