On the third day, a new version of SObjectizer became available : 5.6.0 . Its main feature is the rejection of compatibility with the previous stable branch 5.5, which has progressively developed over four and a half years.

The basic principles of SObjectizer-5 work remain the same. Messages, agents, collaborations, and dispatchers are still with us. But something seriously changed, something was thrown away altogether. So just take SO-5.6.0 and recompile your code will not work. Something needs to be rewritten. Something may have to be redesigned.

Why did we care about compatibility for several years, and then decided to take everything and break it? And what broke the most thoroughly?

I will try to tell about it in this article.

Why do I need to break something?

It's just that simple.

SObjectizer-5.5 during its development has absorbed so much of all different and diverse, which was not originally planned, and as a result, he had too many crutches and props inside. With each new version, adding something new to SO-5.5 became harder and harder. And in the end, the question "Why do we need all this?" There was no suitable answer.

So the first reason is the over-complication of the SObjectizer's giblets.

The second reason is that we are stupidly tired of focusing on the old C ++ compilers. Branch 5.5 started in 2014, when we had, if I'm not mistaken, gcc-4.8 and MSVS2013. And at this level, we have so far continued to keep the bar on requirements for the level of support for the C ++ standard.

Initially, we had this, let's say, "selfish interest." In addition, for some time we considered the low quality requirements for supporting the C ++ standard as our “competitive advantage”.

But time passes, selfish interest is over. Some benefits from such a "competitive advantage" is not visible. Maybe they would be if we worked at all with C ++ 98, then a bloody enterprise might be interested in us. But a bloody enterprise in such as we are not interested in principle. Therefore, it was decided to stop limiting yourself and take something fresher. So we took the freshest of the stable at the moment: C ++ 17.

Obviously, not everyone will like this solution; after all, for many, C ++ 17 is now an unattainable cutting edge, which is still very, very far away.

Nevertheless, we decided on such a risk. All the same, the process of promoting SObjectizer is not going fast, so that when SObjectizer becomes more or less widely demanded, C ++ 17 will no longer be a front edge. Rather, it will be treated the same way as it is now in C ++ 11.

In general, instead of continuing to build crutches using a subset of C ++ 11, we decided to seriously rework the insides of SObjectizer using C ++ 17. To build a base on the basis of which SObjectizer can progressively develop over the next four or five years.

What seriously changed in SObjectizer-5.6?

And now let's take a quick look at some of the most striking changes.

Agent cooperatives no longer have string names

Problem

From the very beginning, SObjectizer-5 demanded that each co-operation should have its own unique string name. This feature was inherited by the fifth SObjectizer in a legacy from the previous, fourth SObjectizer.

Accordingly, SObjectizer had to keep the names of registered cooperatives. Check their uniqueness at registration. Search co-operation by name for deregistration, etc., etc.

Since the very first versions in SObjectzer-5 a simple scheme has been used: a single dictionary of registered cooperatives protected by mutex. When registering a cooperation, mutex is captured, the uniqueness of the name of the cooperation, the presence of a parent, etc. is checked After performing the checks, the dictionary is modified, after which the mutex is released. This means that if registration / deregistration of several cooperations simultaneously began at once, then at some moments they will be suspended and wait until one of the operations finishes work with the cooperation dictionary. Because of this, operations with cooperatives did not scale well.

I wanted to get rid of all this in order to improve the situation with the speed of registration of cooperatives.

Decision

Considered two main ways to solve this problem.

First, the preservation of string names, but changing the way the dictionary is stored so that the cooperation registration operation can scale. For example, dictionary sharding, i.e. splitting it into several pieces, each of which would be protected by its mutex.

Secondly, the complete rejection of string names and the use of some identifiers assigned by SObjectizer.

In the results, we chose the second method and completely abandoned the naming of cooperatives. Now in SObjectizer there is such a thing as coop_handle , i.e. some handle whose contents are hidden from the user, but which can be compared with std::weak_ptr .

SObjectizer returns coop_handle when registering a cooperation:

 auto coop = env.make_coop(); ... //    . auto coop_id = env.register_coop(std::move(coop)); // . //   coop_id    . 

This handle should be used when deregistering cooperation:

 auto coop = env.make_coop(); ... //    . auto coop_id = env.register_coop(std::move(coop)); // . //   coop_id    . ... // - . // ,     . //       . env.deregister_coop(coop_id, ...); 

Also, this handle should be used when setting up the parent-child relationship:

 //   . auto parent = env.make_coop(); ... //  parent . auto parent_id = env.register_coop(std::move(parent)); //  . ... //      ,    . auto child = env.make_coop(parent_id); ... 

The storage structure for cooperations within the SObjectizer Environment has also changed dramatically. If up to version 5.5 inclusive it was one common dictionary, then now each cooperation is a repository of links to subsidiary cooperations. Those. cooperatives form a tree with a root in a special root co-operation hidden from the user.

Such a structure allows scaling of the register_coop and deregister_coop much better: mutual blocking of parallel operations occurs only if they both belong to the same parent cooperation. For clarity, here is the result of launching a special benchmark that measures the performance of cooperation operations on my old Ubuntu 16.04 and GCC-7.3 laptop:

 _test.bench.so_5.parallel_parent_child -r 4 -l 7 -s 5 Configuration: roots: 4, levels: 7, level-size: 5 parallel_parent_child: 15.69s 488280 488280 488280 488280 Total: 1953120 

Those. version 5.6.0 managed almost 2M cooperations in ~ 15.5 seconds.

But version 5.5.24.4, the last of the 5.5 branch at the moment:

 _test.bench.so_5.parallel_parent_child -r 4 -l 7 -s 5 Configuration: roots: 4, levels: 7, level-size: 5 parallel_parent_child: 46.856s 488280 488280 488280 488280 Total: 1953120 

The same scenario, but the result is three times worse.

Only one type of dispatcher is left.

Dispatchers are one of the cornerstones of SObjectizer. It is the dispatchers who determine where and how agents will process their messages. So, without the idea of ​​controllers, there probably wouldn't be a SObjectizer.

However, the dispatchers themselves evolved, evolved and evolved to the point that it was not difficult for us to make a new dispatcher for SObjectizer-5.5 even for ourselves. But very troublesome. However, let's go in order.

Initially, all dispatchers that are needed by the application could only be created during the start of a SObjectizer:

 so_5::launch( []( so_5::environment_t & env ) { /* -   */ }, //    SObjectizer-. []( so_5::environment_params_t & params ) { p.add_named_dispatcher("active_obj", so_5::disp::active_obj::create_disp()); p.add_named_dispatcher("shutdowner", so_5::disp::active_obj::create_disp()); p.add_named_dispatcher("groups", so_5::disp::active_group::create_disp()); ... } ); 

I didn’t create the necessary dispatcher before the start - everything is to blame, nothing can be changed.

It is clear that this is inconvenient and as the use of SObjectizer scenarios has expanded, this problem has to be solved. Therefore, the add_dispatcher_if_not_exists method add_dispatcher_if_not_exists , which checked for the presence of a dispatcher and, if not, allowed to create a new instance:

 so_5::launch( []( so_5::environment_t & env ) { ... // - . //     . env.add_dispatcher_if_not_exists( "extra_dispatcher", []{ return so_5::disp::active_obj::create_disp(); } ); }, //    SObjectizer-. []( so_5::environment_params_t & params ) {...} ); 

Such dispatchers were called public. Public controllers had unique names. And with the help of these names, agents were assigned to dispatchers:

 so_5::launch( []( so_5::environment_t & env ) { ... // - . //     . env.add_dispatcher_if_not_exists( "extra_dispatcher", []{ return so_5::disp::active_obj::create_disp(); } ); //         //    . auto coop = env.create_coop( "ping_pong", //     extra_dispatcher. so_5::disp::active_obj::create_disp_binder( "extra_dispatcher" ) ); coop->make_agent< a_pinger_t >(...); coop->make_agent< a_ponger_t >(...); ... }, //    SObjectizer-. []( so_5::environment_params_t & params ) {...} ); 

But public controllers had one unpleasant feature. They began to work immediately after their addition to the SObjectizer Environment and continued to work until the SObjectizer Environment completed its work.

Again, over time, this began to interfere. It was necessary to make it so that it was possible to add dispatchers as needed and so that dispatchers that became unnecessary were automatically removed.

So there were "private" dispatchers. These dispatchers did not have names and lived until references to them existed. It was possible to create private dispatchers at any time after launching the SObjectizer Environment, they were automatically destroyed.

In general, private dispatchers proved to be a very successful link in the evolution of dispatchers, but working with them was very different from working with public dispatchers:

 so_5::launch( []( so_5::environment_t & env ) { ... // - . //     . auto disp = so_5::disp::active_obj::create_private_disp(env); //         //    . auto coop = env.create_coop( "ping_pong", //      . disp->binder() ); coop->make_agent< a_pinger_t >(...); coop->make_agent< a_ponger_t >(...); ... }, //    SObjectizer-. []( so_5::environment_params_t & params ) {...} ); 

Even more private and public controllers differed in implementation. Therefore, in order not to duplicate the code and not to write separately a public and separately private dispatcher of the same type, it was necessary to use rather complex constructions with patterns and inheritance.

As a result, I was tired of accompanying all this diversity, and in SObjectizer-5.6 only one type of dispatcher remained. In fact, it is an analogue of private dispatchers. But without the explicit mention of the word "private". So now the fragment shown above will be written as:

 so_5::launch( []( so_5::environment_t & env ) { ... // - . //     . auto disp = so_5::disp::active_obj::make_dispatcher(env); //         //    . auto coop = env.create_coop( "ping_pong", //      . disp.binder() ); coop->make_agent< a_pinger_t >(...); coop->make_agent< a_ponger_t >(...); ... }, //    SObjectizer-. []( so_5::environment_params_t & params ) {...} ); 

Only the free functions send, send_delayed and send_periodic remain

The development of the API for sending messages in SObjectizer is, in general, probably the most vivid example of how SObjectizer has changed as the support for C ++ 11 has improved in the compilers available to us.

At first the messages were sent as follows:

 mbox->deliver_message(new my_message(...)); 

Or, if you follow the "recommendations of the best dog breeders" (c):

 std::unique_ptr<my_message> msg(new my_message(...)); mbox->deliver_message(std::move(msg)); 

However, then we got at our disposal compilers with variadic templates support, and send functions appeared. It became possible to write like this:

 send<my_message>(target, ...); 

True, it took some more time to send_to_agent from a simple send whole family, including send_to_agent , send_delayed_to_agent , etc. And then this family would shrink to the familiar set of send , send_delayed and send_periodic .

But, despite the fact that the send-functions family was formed quite a long time ago and has been the recommended way of sending messages for several years, the old methods, like deliver_message , schedule_timer and single_timer , were still available to the user.

But in version 5.6.0 in the public API SObjectizer only the free functions send , send_delayed and send_periodic . Everything else has either been removed altogether or transferred to internal SObjectizer namespaces.

So in SObjectizer-5.6, the interface for sending messages has finally become what it should have been if we had compilers with normal support for C ++ 11 from the very beginning. Well, plus the fact that if we had the experience of using this very normal C ++ 11.

Single format send_delayed and send_periodic

With the functions send_delayed and send_periodic in previous versions of SObjectizer, this is also the case.

To work with the timer you need to have access to the SObjectizer Environment. Inside the agent there is a link to the SObjectizer Environment. And inside mchain, there is such a link. But inside the mbox, it was not there. Therefore, if the deferred message was sent to the agent or to mchain, then the send_delayed call was:

 send_delayed<my_message>(target_agent, pause, ...); send_delayed<my_message>(target_mchain, pause, ...); 

For the mbox case, you also had to take a link to the SObjectizer Environment from somewhere else:

 send_delayed<my_message>(this->so_environment(), target_mbox, pause, ...); 

Such a feature of send_delayed and send_periodic was a fine splinter. Which is not that much interfered, but pretty annoyed. And all because initially we did not store in the mbox a link to the SObjectizer Environment.

Violation of compatibility with previous versions was a good reason to get rid of this splinter.

Now you can find out from mbox, for which SObjectizer Environment it was created. And this made it possible to use the uniform format send_delayed and send_periodic for any type of recipient of the timer message:

 send_delayed<my_message>(target_agent, pause, ...); send_delayed<my_message>(target_mchain, pause, ...); send_delayed<my_message>(target_mbox, pause, ...); 

In the literal sense, "a trifle, but nice."

No more ad-hoc agents

As the saying goes, "Every accident has a name, a middle name and a surname." In the case of ad-hoc agents, this is my first, middle and last name :(

The bottom line is this. When we began to talk about SObjectizer-5 in public, we heard a lot of reproaches about the verbosity of the SObjectizer code of examples. And to me personally, this verbosity seemed like a serious problem that needs to be seriously addressed.

One of the sources of verbosity is the need to inherit agents from the special base type agent_t . And from this, it would seem, can not escape. Or not?

So ad-hoc agents appeared, i.e. agents, for the determination of which there was no need to write out a separate class, it was enough just to set the reaction to messages in the form of lambda functions. For example, a classic example of ping-pong on ad-hoc agents could be written like this:

 auto pinger = coop->define_agent(); auto ponger = coop->define_agent(); pinger .on_start( [ponger]{ so_5::send< msg_ping >( ponger ); } ) .event< msg_pong >( pinger, [ponger]{ so_5::send< msg_ping >( ponger ); } ); ponger .event< msg_ping >( ponger, [pinger]{ so_5::send< msg_pong >( pinger ); } ); 

Those. no classes Just call define_agent() from the cooperation and get some object-agent that you can subscribe to incoming messages.

So in SObjectizer-5, the division into ordinary and ad-hoc agents appeared.

Which did not bring any visible bonuses, only extra labor costs to support such a division. And over time, it became clear that ad-hoc agents are like a suitcase without a handle: it's hard to carry and hard to quit. But when working on SObjectizer-5.6, it was decided to quit after all.

At the same time, another lesson was learned, perhaps even more important: in any public discussion of the tool on the Internet there will be a huge number of people who care what kind of tool it is, what it is for, why it is like it is supposed to be used, etc. It is just important for them to express their strong opinion. In the Russian-speaking segment of the Internet, in addition to this, it is still very important to convey to the developers of the tool how much they are stupid and uneducated, and how much the result of their work is not needed.

Therefore, you should be very careful what you say. And you can listen (and only cautiously) to what is said here in this way: "I tried to do it on your tool like this and I don’t like how much code came out here." Even here such wishes should be treated very carefully: "I would take your development if you would have been easier here and here."

Unfortunately, the skill of "filtering" what was said by "well-wishers" on the Internet five years ago was much less than I have now. Hence, such a specific experiment as ad-hoc agents in SObjectizer.

SObjectizer-5.6 no longer supports synchronous agent interaction.

The topic of synchronous interaction between agents is very old and sick.

It began in the days of SObjectizer-4. And in SObjectizer-5 continued. Until finally, the so-called service requests Which initially, it must be admitted, were scary like death. But then I managed to give them a more or less decent look .

But it turned out to be the very case when the first pancake came out lumpy: (

Inside SObjectizer, I had to implement the delivery and processing of regular messages in one way, and the delivery and processing of synchronous requests in another way. It is especially sad that these features had to be reckoned with, including when implementing your own mboxes.

And after the functionality of envelope messages was added to SObjectizer, it became necessary to look even more carefully and more carefully over the differences between ordinary messages and synchronous requests.

In general, with synchronous requests, when accompanying / developing a SObjectizer, there was too much of a headache. So much so that at first there appeared a specific desire to get rid of these very synchronous requests . And then this desire was realized.

And in SObjectizer-5.6, agents can interact again only through asynchronous messages.

And since at times something like a synchronous interaction is still needed, support for this type of interaction was brought into the accompanying project so5extra :

 //    "-". using my_request_reply = so_5::extra::sync::request_reply_t<my_request, my_reply>; ... //  ,    . class request_handler final : public so_5::agent_t { ... //  .      . void on_request(typename my_request_reply::request_mhood_t cmd) { ... //  . //      cmd->request(). //   . cmd->make_reply(...); //      my_reply. } ... void so_define_agent() override { //       . so_subscribe_self().event(&request_handler::on_request); } }; ... //     . so_5::mbox_t handler_mbox = ...; //        15s. //    ,    . my_reply reply = my_request_reply::ask_value(handler_mbox, 15s, ...); //       my_request. 

Those. Now working with synchronous requests is fundamentally different in that the request handler does not return a value from the handler method, as it was before. Instead, the make_reply method is make_reply .

The new implementation is good in that both the request and the response are sent inside the SObjectizer, as well as ordinary asynchronous messages. Essentially, make_reply is a slightly more specific implementation of send .

And, importantly, the new implementation allowed us to obtain functionality that was previously unattainable:

• synchronous requests (i.e. request_reply_t<Request, Reply> objects) can now be saved and / or forwarded to other handlers. What makes it possible to implement various load balancing schemes;
• You can make the response to the request come in the usual mbox of the agent initiating the request. And the initiating agent will process the response in the usual way, like any other message;
• You can send several requests to different recipients at once, and then sort the answers from them in the order they are received:

 using first_dialog = so_5::extra::sync::request_reply_t<first_request, first_reply>; using second_dialog = so_5::extra::sync::request_reply_t<second_request, second_reply>; //         . auto reply_ch = create_mchain(env); //     . first_dialog::initiate_with_custom_reply_to( one_service, reply_ch, so_5::extra::sync::do_not_close_reply_chain, ...); second_dialog::initiate_with_custom_reply_to( another_service, reply_ch, so_5::extra::sync::do_not_close_reply_chain, ...); //    . receive(from(reply_ch).handle_n(2).empty_timeout(15s), [](typename first_dialog::reply_mhood_t cmd) {...}, [](typename second_dialog::reply_mhood_t cmd) {...}); 

So, it can be said that with synchronous interaction in SObjectizer, the following happened:

• for a long time it was not for ideological reasons;
• then it was added and it turned out that sometimes this interaction is useful;
• but experience has shown that the first implementation is not very successful;
• the old implementation was thrown away altogether, and a new implementation was proposed instead.

Made work on their own mistakes, in general.

Conclusion

This article was, quite briefly, told about several changes in SObjectizer-5.6.0 and the reasons behind these changes.

A more complete list of changes can be found here .

In conclusion, I would like to offer those who have not tried SObjectizer to take and try. And to share with us our feelings: what you liked, what you didn’t like, what you didn’t have enough.

We carefully listen to all constructive comments / suggestions. Moreover, in recent years, only what was needed has been included in SObjectizer. So, if you don’t tell us what you would like to have in SObjectizer, this will not appear. And if you tell me, then who knows ...;)

The project now lives and develops here . And for those who are used to using only GitHub, there is a GitHub mirror . This mirror is completely new, so you can ignore the absence of stars.

Ps. For news related to SObjectizer, you can follow this Google group . There you can also raise issues related to SObjectizer.