Preamble

Our team is developing a small, easy-to-use, embedded, asynchronous HTTP server for modern C ++ called RESTinio . We started to do it because we needed exactly asynchronous processing of incoming HTTP requests, but there was nothing ready for us to like. As life shows, asynchronous processing of HTTP requests in C ++ applications is needed not only by us. Recently, the developers from one company got in touch with the question of whether it was possible to somehow make friends with asynchronous processing of incoming requests in RESTinio with issuing asynchronous outgoing requests via libcurl .

As the situation was clarified, we found that this company was faced with the conditions that we faced, and because of which we started developing RESTinio. The bottom line is that an application written in C ++ accepts an incoming HTTP request. During the processing of the request, the application needs to contact a third-party server. This server can respond for quite some time. Say 10 seconds (although 10 seconds is still good). If you make a synchronous request to a third-party server, then the working thread on which the HTTP request is executed is blocked. This begins to limit the number of parallel requests that the application can handle.

The solution is to allow the application to asynchronously process all requests: both incoming and outgoing. Then, on a limited pool of working threads (or even generally on one single working thread), it will be possible to process simultaneously tens of thousands of requests, even if the processing time of a single request is tens of seconds.

The trick was that the application for outgoing HTTP requests was already used by libcurl. But in the form of curl_easy , i.e. all requests were executed synchronously. We were asked, is it possible to combine RESTinio and curl_multi? The question for us turned out to be interesting, because Before libcurl in the form of curl_multi it was not necessary to apply. Therefore, it was interesting to immerse yourself in this topic.
')
Plunged. Got a lot of impressions. We decided to share with readers. Maybe someone will be interested in how to live with curl_multi. For, as practice has shown, it is possible to live. But carefully ...;) What we will discuss in a small series of articles based on the experience of implementing a simple simulation of the situation described above with a slowly responding third-party service.

Required disclaimers

In order to prevent a useless and unconstructive flame in the comments (like what happened with the previous article ), I want to make a few warnings:

• first, the following discussion will be about C ++. If you do not like C ++, if you think that C ++ does not belong in the modern world in general and in similar tasks in particular, then this article is not for you. And we have no goal to convince someone that C ++ is good and should be used in such tasks. We only talk about how to solve a similar problem in C ++ if you suddenly had to do it in C ++. Also, we will not argue about why this may be needed and why in real life one cannot simply take and rewrite existing C ++ code into something else;
• secondly, in C ++ there is no generally accepted code convention, therefore any claims from the followers of camelCase, PascalCase, Camel_With_Underscores_Case or even UPPER_CASE will not be perceived. We tried to bring the code in a more or less similar to K & R style, so that it looked familiar to the largest number of readers. For our "corporate" design style of C ++ code does not exactly accept everything. However, if the appearance of the code violates your aesthetic feelings and you are ready to comment in the comments your weighty "fi" about this, then think about it, that's what: there is always someone who does not like the style you use. Is always. Regardless of what style you use;
• thirdly, the code shown by us in no way pretends to be a model of quality and reliability. This is not a production code. What you will see is a quick-and-dirty prototype, which was made on the knee just a day and another day was spent on combing the resulting code a little bit and providing it with explanatory comments. So claims like “yes, who writes like that” or “you should beat your hands for such govnokod” are not accepted, because we express them to ourselves;)

In general, if you do not like one of the above conditions, then we apologize for the time taken away. Further reading does not make sense. Well, if these warnings do not frighten you, then make yourself comfortable. We hope that you will be interested.

What is the essence of the developed imitation?

For demonstration purposes, we have made several applications using RESTinio and libcurl. The simplest of these is a third-party, slow-response simulator server, called delay_server. To start the simulation, you need to run delay_server with the necessary set of parameters (address, port, desired delay times for responses).

Also in the simulation includes several "fronts", called bridge_server_ *. It is the bridge_server that receives requests from the user and redirects requests to delay_server. It is assumed that the user starts delay_server first, then one of the bridge_servers, after which it starts to fire the bridge_server in a convenient way. For example, through curl / wget or utilities like ab / wrk.

The imitation includes three bridge_server implementations:

• bridge_server_1. Very simple option in which only two working threads are used. On one RESTinio it processes incoming HTTP requests, and on the second, outgoing HTTP requests are executed using curl_multi_perform . This implementation will be discussed in the second part of the series;
• bridge_server_1_pipe. More complex option bridge_server_1. There are also two working threads, but an additional pipe is used to send notifications from the RESTinio thread to the libcurl thread. Initially, we did not plan to describe this implementation, but if someone has an interest, we can consider bridge_server_1_pipe in details in an additional article;
• bridge_server_2. A more complex option that uses a pool of worker threads. Moreover, this pool serves both RESTinio and libcurl (using curl_multi_socket_action ). This implementation will be discussed in the final part of the series.

Let's start this series with a description of the implementation of delay_server. The benefit is the most simple and perhaps the most understandable part. Implementations of bridge_server will be much more hardcore.

delay_server

What does delay_server do?

delay_server accepts HTTP GET requests for URLs like / YYYY / MM / DD, where YYYY, MM and DD are numeric values. All other requests delay_server responds with the code 404.

If an HTTP GET request is sent to the URL of the form / YYYY / MM / DD, then the delay_server pauses and then responds with a small text with the greeting “Hello, World” and the value of the paused. For example, if you run delay_server with parameters:

  delay_server -a localhost -p 4040 -m 1500 -M 4000 

those. he will listen on localhost: 4040 and pause for responses between 1.5s and 4.0s. If then execute:

  curl -4 http: // localhost: 4040/2018/02/22 

then we get:

  Hello world!
 Pause: 2347ms. 

Well, or you can turn on tracing what is happening. For a server, this is:

  delayed_server -a localhost -p 4040 -m 1500 -M 4000 -t 

For curl, this is:

  curl -4 -v http: // localhost: 4040/2018/02/22 

For delay_server, we will see something like:

  [2018-02-22 16: 47: 54.441] TRACE: starting server on 127.0.0.1:4040
 [2018-02-22 16: 47: 54.441] INFO: init accept # 0
 [2018-02-22 16: 47: 54.441] INFO: server started on 127.0.0.1:4040
 [2018-02-22 16: 47: 57.040] TRACE: accept connection from 127.0.0.1天8468 on socket # 0
 [2018-02-22 16: 47: 57.041] TRACE: [connection: 1] start connection with 127.0.0.1天8468
 [2018-02-22 16: 47: 57.041] TRACE: [connection: 1] start waiting for request
 [2018-02-22 16: 47: 57.041] TRACE: [connection: 1] continue reading request
 [2018-02-22 16: 47: 57.041] TRACE: [connection: 1] received 88 bytes
 [2018-02-22 16: 47: 57.041] TRACE: [connection: 1] request received (# 0): GET / 2018/02/22
 [2018-02-22 16: 47: 59.401] TRACE: [connection: 1] append response (# 0), flags: {final_parts, connection_keepalive}, bufs count: 2
 [2018-02-22 16: 47: 59.401] TRACE: [connection: 1] sending resp data, buf count: 2
 [2018-02-22 16: 47: 59.402] TRACE: [connection: 1] outgoing data was sent: 206 bytes
 [2018-02-22 16: 47: 59.402] TRACE: [connection: 1] should keep alive
 [2018-02-22 16: 47: 59.402] TRACE: [connection: 1] start waiting for request
 [2018-02-22 16: 47: 59.402] TRACE: [connection: 1] continue reading request
 [2018-02-22 16: 47: 59.403] TRACE: [connection: 1] EOF and no request, close connection
 [2018-02-22 16: 47: 59.403] TRACE: [connection: 1] close
 [2018-02-22 16: 47: 59.403] TRACE: [connection: 1] destructor called 

and for curl:

 * Trying 127.0.0.1 ... * TCP_NODELAY set * Connected to localhost (127.0.0.1) port 4040 (# 0)> GET / 2018/02/22 HTTP / 1.1> Host: localhost: 4040> User-Agent: curl / 7.58.0> Accept: * / *> <HTTP / 1.1 200 OK <Connection: keep-alive <Content-Length: 28 <Server: RESTinio hello world server <Date: Thu, 22 Feb 2018 13:47:59 GMT < Content-Type: text / plain;  charset = utf-8 <Hello world!  Pause: 2360ms.  * Connection # 0 to host localhost left intact 

How does delay_server do this?

Delay_server is a simple single-threaded C ++ application. A built-in HTTP server is launched on the main thread, which pulls the callback assigned by the user when receiving a request for a suitable URL. This callback creates an Asio-shny timer and cocks the created timer to a randomly selected pause (the pause is chosen so as to fall within the limits specified when starting the delay_server). Then callback returns control to the HTTP server, which allows the server to accept and process the next request. When a coded callback timer is triggered, a response to a previously received HTTP request is generated and sent.

Parsing the implementation of delay_server

Main () function

Analysis of the implementation of delay_server will begin immediately with the function main (), gradually explaining what is happening inside and outside main () - a.

So, the main () code looks like this:

int main(int argc, char ** argv) { try { const auto cfg = parse_cmd_line_args(argc, argv); if(cfg.help_requested_) return 1; //    io_context  ,      //     . restinio::asio_ns::io_context ioctx; //          . pauses_generator_t generator{cfg.config_.min_pause_, cfg.config_.max_pause_}; //    ,    //    ,       // (    ). auto actual_handler = [&ioctx, &generator](auto req, auto /*params*/) { return handler(ioctx, generator, std::move(req)); }; //     ,   //    . if(cfg.config_.tracing_) { run_server<traceable_server_traits_t>( ioctx, cfg.config_, std::move(actual_handler)); } else { //   ,    . run_server<non_traceable_server_traits_t>( ioctx, cfg.config_, std::move(actual_handler)); } // ,     . } catch( const std::exception & ex ) { std::cerr << "Error: " << ex.what() << std::endl; return 2; } return 0; } 

What's going on here?

First, we parse the command line arguments and get an object with a configuration for delay_server.

Secondly, we create several objects that we need:

• an asio :: io_context instance that will be used both for processing HTTP server IO operations and for timers that will be entered into the incoming HTTP request handler;
• random delay generator, which is needed just so that the HTTP server is slow to respond to requests;
• A lambda function stored in the variable actual_handler, which will be the same callback called by the HTTP server for incoming HTTP requests. This callback should have a specific format. But the handler () function, which performs the actual processing of requests and is discussed below, has a different format and requires additional arguments. Here is the lambda function and captures the necessary handler () arguments, exposing the signature that RESTinio requires.

Thirdly, we are launching an HTTP server. But the launch is done taking into account whether the user wants to see the server’s tracing or not. Here comes into the small patterned magic that we actively use in RESTinio and about which we have already talked a little about earlier .

Here, in fact, the whole delay_server :)

But the devil, as usual, in the details. Therefore, let's go further, consider what hides behind these simple actions.

Configuration and analysis of the command line

The delay_server uses a very simple structure to describe the server configuration:

 // ,   . struct config_t { // ,       . std::string address_{"localhost"}; // ,    . std::uint16_t port_{8090}; //      . milliseconds min_pause_{4000}; //      . milliseconds max_pause_{6000}; //    ? bool tracing_{false}; }; 

Analysis of the command line is pretty voluminous, so we will not dive into it. But anyone can look under the spoiler to make an impression about what is happening.

Random Delay Generator

Everything is simple here, in principle, there is nothing to discuss. So just the code. In order to be.

Handler () function

One of the key elements of the implementation of delay_server is a small handler () function, inside which the processing of incoming HTTP requests takes place. Here is all the code for this function:

 //   . restinio::request_handling_status_t handler( restinio::asio_ns::io_context & ioctx, pauses_generator_t & generator, restinio::request_handle_t req) { //      (   ). const auto pause = generator.next(); //     Asio-. auto timer = std::make_shared<restinio::asio_ns::steady_timer>(ioctx); timer->expires_after(pause); timer->async_wait([timer, req, pause](const auto & ec) { if(!ec) { //   ,   . req->create_response() .append_header(restinio::http_field::server, "RESTinio hello world server") .append_header_date_field() .append_header(restinio::http_field::content_type, "text/plain; charset=utf-8") .set_body( fmt::format("Hello world!\nPause: {}ms.\n", pause.count())) .done(); } } ); // ,         - //   . return restinio::request_accepted(); } 

This function (via lambda created in main () - e) is called each time an HTTP server receives an incoming GET request to the desired URL. The incoming HTTP request itself is passed in the req parameter of the type restinio :: request_handle_t.

This very restinio :: request_handle_t is a smart pointer to an object with the contents of an HTTP request. That allows you to save the value of req and use it later. This is one of the cornerstones in RESTinio asynchrony: RESTinio jerks a user-supplied callback and sends an instance of request_handle_t to this callback. The user can either immediately generate an HTTP response inside the callback (and then it will be a trivial synchronous processing), or he can save the req on his own or transfer the req to some other thread. Then return control RESTinio. And form the answer later, when the appropriate time comes for that.

In this case, an instance of asio :: steady_timer is created and req is stored in the lambda function, which is passed to async_wait for the timer. Accordingly, the HTTP request object is retained until the timer is triggered.

A very important point in handler () - e is the value returned by it. By the return value, RESTinio understands whether the user has taken responsibility for forming the response to the request or not. In this case, the request_accepted value is returned, which means that the user has promised RESTinio to generate a response to an incoming HTTP request later.

But if handler () returned, say, request_rejected (), then RESTinio would have finished processing the request and would have answered the user with code 501.

So, handler () is called when an incoming HTTP request arrives at the desired URL (why this is discussed below). In handler, the delay for the response is calculated. Then the timer is created and cocked. When the timer runs, a response to the request will be generated. Well, the handler () promises RESTinio to form a response to the request by returning request_accepted.

That's all. Little trifle: fmtlib is used to form the response body. In principle, one could do without it here. But, first of all, we really like fmtlib and we use fmtlib at an opportunity. And, secondly, we still needed fmtlib in bridge_server, so there was no point in abandoning it in delay_server.

Function run_server ()

The run_server () function is responsible for setting up and running an HTTP server. It determines which requests the HTTP server will handle and how the HTTP server will respond to all other requests.

Also in run_server () it is determined where the HTTP server will work. For the case of delay_server, this will be the main thread of the application.

Let's first take a look at the run_server () code, and then consider a few important points that we haven’t talked about yet.

So, here is the code:

 template<typename Server_Traits, typename Handler> void run_server( restinio::asio_ns::io_context & ioctx, const config_t & config, Handler && handler) { //      express-. auto router = std::make_unique<express_router_t>(); //   URL   . router->http_get( R"(/:year(\d{4})/:month(\d{2})/:day(\d{2}))", std::forward<Handler>(handler)); //      404. router->non_matched_request_handler([](auto req) { return req->create_response(404, "Not found") .append_header_date_field() .connection_close() .done(); }); restinio::run(ioctx, restinio::on_this_thread<Server_Traits>() .address(config.address_) .port(config.port_) .handle_request_timeout(config.max_pause_) .request_handler(std::move(router))); } 

What is happening in it and why is this happening exactly?

First, for delay_server, functionality similar to the expressjs query routing system will be used. In RESTinio, this is called the Express router .

You need to create an instance of the object that is responsible for routing requests based on regular expressions. After that, in this object you need to put a list of routes and set each route its handler. What we are doing. Create a handler:

 auto router = std::make_unique<express_router_t>(); 

And we indicate the route that interests us:

 router->http_get( R"(/:year(\d{4})/:month(\d{2})/:day(\d{2}))", std::forward<Handler>(handler)); 

After that, we also set a handler for all other requests. Which will simply answer code 404:

 router->non_matched_request_handler([](auto req) { return req->create_response(404, "Not found") .append_header_date_field() .connection_close() .done(); }); 

This completes the preparation of the Express router we need.

Secondly, when calling run (), we specify that the HTTP server should use the specified io_context and should work on the same thread on which the run () call was made. Plus, the server sets the parameters from the configuration (because the IP address and port, the maximum allowable time for processing requests and the handler itself):

 restinio::run(ioctx, restinio::on_this_thread<Server_Traits>() .address(config.address_) .port(config.port_) .handle_request_timeout(config.max_pause_) .request_handler(std::move(router))); 

Here the use of on_this_thread makes RESTinio run the HTTP server in the context of the same thread.

Why is run_server () a template?

The run_server () function is a template function that depends on two parameters:

 template<typename Server_Traits, typename Handler> void run_server( restinio::asio_ns::io_context & ioctx, const config_t & config, Handler && handler); 

In order to explain why this is so, let's start with the second template parameter - Handle.

Inside main () we create the actual request handler as a lambda function. The real type of this lambda is known only by the compiler. Therefore, in order to pass the lambda handler to run_server (), we need the template parameter Handle. With its help, the compiler will display the required type of the argument handler in run_server ().

But with the Server_Traits parameter, the situation is a bit more complicated. The fact is that the HTTP server in RESTinio needs to set a set of properties that will determine various aspects of server behavior and implementations. For example, whether the server will be adapted to work in multi-threaded mode. Will the server log its operations, etc.? All this is set by the template Traits parameter for the restinio :: http_server_t class. In this example, this class is not visible, because an instance of http_server_t is created inside run (). But all the same Traits should be set. Just the template Server_Traits parameter of the run_server () function and sets the Traits for http_server_t.

We in delay_server needed to define two different types of Traits:

 //    express-router.     //   . using express_router_t = restinio::router::express_router_t<>; //          http-. //    ,     . struct non_traceable_server_traits_t : public restinio::default_single_thread_traits_t { using request_handler_t = express_router_t; }; //    ,    . struct traceable_server_traits_t : public restinio::default_single_thread_traits_t { using request_handler_t = express_router_t; using logger_t = restinio::single_threaded_ostream_logger_t; }; 

The first type, non_traceable_server_traits_t, is used when the server does not need to log its actions. The second type, traceable_server_traits_t, is used when logging should be.

Accordingly, inside the main () function, depending on the presence or absence of the "-t" key, the run_server () function is called either with non_traceable_server_traits_t, or with traceable_server_traits_t:

 //     ,   //    . if(cfg.config_.tracing_) { run_server<traceable_server_traits_t>( ioctx, cfg.config_, std::move(actual_handler)); } else { //   ,    . run_server<non_traceable_server_traits_t>( ioctx, cfg.config_, std::move(actual_handler)); } 

So assigning the right properties to an HTTP server is another reason why run_server () is a template function.

The Traits for restinio :: http_server_t topic is covered in more detail in our previous article on RESTinio .

Conclusion of the first part

That is, in fact, all that could be told about the implementation of delay_server on the basis of RESTinio. We hope that the material described was clear. If not, we will be happy to answer questions in the comments.

In subsequent articles, we will talk about the integration examples of RESTinio and curl_multi, examining the implementation of bridge_server_1 and bridge_server_2. There, the parts that belong specifically to RESTinio will be no longer and no more complicated than what we have shown in this article. And the bulk of the code and the main complexity will result from curl_multi. But that's another story ...

To be continued .