SObjectizer: the problem of overloading agents and means of dealing with it

In previous articles we have mentioned the problem of overloading agents several times. What it is? What does it threaten with? How to deal with it? We will talk about all this today.

The problem of overloading agents arises when some agent sends more messages than it manages to process. As a result, message queues are constantly increasing in size. Growing queues consume memory. Memory consumption slows down the application. Due to the slowdown, the problem agent starts processing messages longer, which increases the growth rate of message queues. Which contributes to faster memory consumption. Which leads to even greater slowdown of the application. That leads to even slower work of the problem agent ... As a result, the application slowly and sadly degrades to complete inoperability.

The problem is aggravated by the fact that the interaction through asynchronous messages and the use of the fire-and-forget approach directly provoke overloads (fire-and-forget is when agent A receives the incoming message M1, processes it, and sends the outgoing message M2 to agent B not caring about the consequences).

Indeed, when send is responsible only for putting a message in the receiving agent's queue, the sender does not block on the send, even if the recipient does not cope with its message flow. This creates the prerequisites for the message senders not to care about whether the recipient is able to cope with the load.

Unfortunately, there is no universal recipe for dealing with overloads, suitable for all situations. Somewhere you can just lose new messages addressed to an overloaded agent. Somewhere this is unacceptable, but you can throw away the oldest messages that have been queued for too long and have ceased to be relevant. Somewhere you can’t lose messages at all, but you can switch to another way of handling messages when overloaded. For example, if an agent is responsible for resizing photos to be displayed on a Web page, then when overloaded, the agent may switch to a coarser resizing algorithm: the quality of photos will fall, but the queue of photos for processing will be resolved faster.

Therefore, a good tool for protection against overloads should be sharpened for a specific task. Because of this, for a long time in SObjectizer there were no ready-made mechanisms available to the user out of the box. The user solved his problems himself.

One of the most practical ways to protect against overload, in our experience, was an approach using two agents: collector and performer. Agent-collector collects incoming messages to be processed in some suitable container. The performer agent processes the messages that the collector agent has collected.

The trick is that the collector and performer agents are bound to different working contexts. Therefore, if the performer agent starts to slow down, this does not affect the work of the collector. As a rule, the collector agent processes its events very quickly: processing usually consists in saving a new message to a separate queue. If this individual queue is full, then the collector agent can immediately return a negative response to a new message. Or throw some old message from the queue. Also, the collector agent may periodically check the time the messages stay in this particular queue: if a message has been waiting for processing for too long and has ceased to be up to date, then the collector agent throws it out of the queue (possibly sending some negative response to the initiator of this message).

The performer agent, as a rule, processes its incoming messages much longer than the collector agent, which is logical, since The responsibility for the actual applied work lies with the performer. The performer agent itself requests the next batch of messages from the collector agent for processing.

In the simplest case, the performer agent completes the processing of the current message and sends to the collector agent a request to issue the next message to be processed. After receiving this request, the agent-collector issues the first message from its turn to the performer agent.

The composition of SObjectizer includes several examples demonstrating various variations on the topic of collectors and performers. Descriptions of these examples can be found in the project’s Wiki ( # 1 , # 2 , # 3 , # 4 ).

As we gained experience using SObjectizer and based on the results of numerous discussions, we came to the conclusion that in spite of the opportunity to make with our own hands protection schemes against overloads of varying degrees of complexity and efficiency, in SObjectizer itself we would also like to have some kind of basic mechanism. Let not advanced, but available directly "out of the box", which is especially in demand with rapid prototyping.

Limits for messages (so-called message limits ) have become such a mechanism. The agent can specify how many instances of messages of a particular type it permits to save in the message queue. And what to do with the "extra" copies. Exceeding the specified limit is an overload and the SObjectizer responds to it in one of the following ways:

• throws out a new message as if it was not there;
• forwards the message to another mbox. This reaction is performed under the assumption that another recipient will be able to process the extra message;
• transform an extra message into another type of message and send a new message to some mbox. This response may allow, for example, to immediately send a negative response to the sender of the message;
• terminates the application by calling std :: abort (). This option is suitable for cases where the load exceeds all conceivable limits and there is almost no chance of recovery, therefore it is better to stop and restart than continue to accumulate the message in the queues.

Let's take a look at how message limits can help cope with overloads (sample source can be found in this repository ).

Suppose we need to process requests for resizing images to a given size. If there are more than 10 such requests, then we need to process new requests using a different algorithm, faster, but less accurate. If this does not help either, then we will log the extra requests in a special way and send an empty picture of the resulting size to the initiator.

Create three agents. The first agent will perform normal image processing. Unnecessary images that he is unable to quickly resolve will be sent to the second agent via message_limit.

//  ,     . class accurate_resizer final : public agent_t { public : accurate_resizer( context_t ctx, mbox_t overload_mbox ) //          // .       // ,      . : agent_t( ctx //     resize_request //       // . + limit_then_redirect< resize_request >( //    10 ... 10, //     mbox. [overload_mbox]() { return overload_mbox; } ) ) {...} ... }; 

The second agent performs faster, but more crude image processing. Therefore, his limit on the number of messages in the queue will be higher. Unnecessary messages redirect to a third agent:

 //  ,     . class inaccurate_resizer final : public agent_t { public : inaccurate_resizer( context_t ctx, mbox_t overload_mbox ) : agent_t( ctx + limit_then_redirect< resize_request >( //    20 ... 20, //     mbox. [overload_mbox]() { return overload_mbox; } ) ) {...} ... }; 

The third agent instead of resizing generates an empty picture. Those. when everything is bad and the load turned out to be very high, it is better to leave empty places instead of pictures than to fall into complete stupor. However, the generation of empty images does not happen instantly, so you can expect that the third agent will have some reasonable limit for pending requests. But if this limit is exceeded, then, perhaps, it is better to bang the entire application through std :: abort, so that after the restart it can start work again. It is possible that the brakes with the processing of the request flow are caused by some incorrect behavior of the application, and the restart will allow you to start anew from scratch. Therefore, the third agent stupidly forces to call std :: abort when the limit is exceeded. Primitive, but extremely effective:

 //  ,      , //        . class empty_image_maker final : public agent_t { public : empty_image_maker( context_t ctx ) : agent_t( ctx //    50- . //   -  ,  -  //   ,       // . + limit_then_abort< resize_request >( 50 ) ) {...} ... }; 

And all together, agents can be created, for example, in the following way:

 //        . //  mbox,       . mbox_t make_resizers( environment_t & env ) { mbox_t resizer_mbox; //        (..   //   ),      // active_obj. env.introduce_coop( disp::active_obj::create_private_disp( env )->binder(), [&resizer_mbox]( coop_t & coop ) { //     , ..    // mbox-     "" . auto third = coop.make_agent< empty_image_maker >(); auto second = coop.make_agent< inaccurate_resizer >( third->so_direct_mbox() ); auto first = coop.make_agent< accurate_resizer >( second->so_direct_mbox() ); //   ,       //   .        //  . resizer_mbox = first->so_direct_mbox(); } ); return resizer_mbox; } 

In general, the message limits mechanism makes it possible in simple cases to do without developing custom collectors and performers. Although they cannot completely replace them (for example, the message limits do not allow to automatically discard the oldest messages from the queue - this is due to the organization of queues of requests from dispatchers).

So, we will try to summarize briefly. If your application consists of agents that interact exclusively through asynchronous messages, and you like to use the fire-and-forget approach, then overloading agents is almost guaranteed. Ideally, to protect your agents, you should use something like pairs of collector-performer agents whose behavior is tailored to your task. But, if you do not need the ideal solution, but rather “cheap and cheerful”, then limits will come to the rescue for messages that SObjectizer provides out of the box.

Ps. Overloading is possible not only for the actors / agents within the Actor Model, but also when using CSP channels. Therefore, in SObjectizer, analogs of CSP-shny channels, message chains , also contain means of protection against overload, somewhat similar to message limits.