Balancing 70 thousand requests per second on HighLoad ++

Library of reports

This is not just an article - it is a whole library of reports about the internal structure of various large and high-loaded projects. All these reports were heard at the conferences HighLoad ++ and RIT ++ over the past few years.



And further:

Section "Architecture"

The key section of the HighLoad ++ developer of highly loaded systems developers conference, which will be held in two weeks in Moscow, is, of course, Architecture. The reports in this section are changing - if three or four years ago we listened to general words about how many servers are on Facebook and where files are stored on VK, now such reports no longer pass to the Program. Now is the time for details, microservices, detailed analyzes of one or another architectural pattern.

So, architectural patterns. At this conference we will analyze in detail a couple of them, and the first one is, of course, microservices.
')
A single application is built as a set of small microservices, each of which works in its own process, is as independent as possible from other microservices, implements, as a rule, one business function and communicates with the others using lightweight mechanisms, the same http.

Anton Reznikov and Vladimir Perepelitsa will tell not only about the microservice architecture of the Cloud@Mail.ru, but also the specific implementation of the pattern on the Tarantool NoSQL database. Yes, Tarantool is another NoSQL database, but still it is a full-fledged application server. Applications located next to the data!



Denis Ivanov (2Gis) will continue the topic in the report “The path from the monolith in PHP to microservices on Scala” . I just want to exclaim: “Denis, stop what you are doing, why ?!”. And Denis answers this question simply - 6 nodes with appendices instead of 18. The answer is worthy, we hope to hear the details at the conference.

Another pattern often used in high-load projects is queues. The topic is covered by Pavel Filonov (Positive Technologies) in the report “101 ways of preparing RabbitMQ and a little about pipeline-architecture” .

The report discusses options for using the RabbitMQ messaging system as middleware for building a pipeline architecture. The issues of performance and scaling of both stateless and statefull filters are considered.

The report is similar to the training, but the topic is painfully good!


The next balancing report from Yuri Nasretdinov from Badoo is a serious bid to win. Taking this opportunity, we tried to ask Yuri a few questions in order to shed light on one of the most exciting reports expected at the conference:

- Yura, how did you approach the load level of 70k requests per second?

We approached this level of load very smoothly, for almost 10 years now. In recent years, we have also started to increase the number of users on mobile devices, which also caused an increase in the number of requests per second - our mobile application sends more small requests to the server than the website. I would like to note that 70k requests per second fall on PHP-FPM, the total number of HTTP requests to our site is up to 250k per second.

- What, besides eggs, do you need to keep such loads?

In general, there is no problem in servicing 250,000 HTTP requests per second. You can handle such a load with the help of the banal nginx and DNS round-robin. We use “hardware” solutions for processing incoming traffic - GTM and LTM from F5. They allow you to process all requests on a single IP address, that is, without using DNS round-robin. At the input router, the rules are set according to which requests fall on a particular cluster, the machine weights are set if necessary, and this piece of hardware does the rest for you (or nginx, which we use to proxy requests to the mobile cluster).

Roughly speaking, distributing requests across servers is not an extremely complex task, and for PHP, scaling is simply to add servers to the backend with the appropriate weights.

It is much more difficult not only to keep such a load, but also to create an architecture for storing user data that could store and deliver hundreds of terabytes and even petabytes of photos and text data. There were many reports about our architecture, for example, a master class from Alexey Rybak at DevConf in 2012 or my report about the architecture of photo storage in 2015. Unfortunately, it is not possible to talk briefly about the architecture in the interview, so I would recommend to look at our presentations for details.

- How is this bandwidth of the architecture achieved, what are some interesting places?

Again, this bandwidth is not something outstanding, and the same LTM handles the load even without additional backups on our part.

If we talk about the storage architecture of users, then everything is a little more interesting. We store the data of each user on one server, that is, we shard the data by users, and not by ID or other synthetic values. This allows us to make quite complex samples when working with user data, use JOIN and complex sorting. At the same time, users are not “nailed to nails” to their server, but can move around the cluster if necessary. This is achieved due to the fact that we store the user's “spot_id” and “place_id” (identifiers from which you can determine the server name and the table name on the server) in a central service called the “authorizer”. Requests are made to this service every time you need to establish a connection to the user's database. The service accounts for about 40k read requests per second, and they are serviced by the HandlerSocket protocol with one MySQL instance. This service is probably the most heavily loaded internal service that serves one server, and at the moment we have a margin of performance at least 2 times. Even if we rest on the scalability of MySQL + handlersocket, you can always try, for example, Tarantool for this task - this daemon can issue 100k requests / s per _ core_ (compare with 100k requests to _server_ in the case of MySQL + handlersocket).

- And a few words about your report on HighLoad ++

In the report, I will talk about a system that puts weights for servers on our balancers (LTM for web load and nginx for mobile cluster). The system is needed in order to achieve a much more even distribution of the load across the cluster than it turns out to be done with “hands”. With the growth of load and the number of requests, we began to require an increasing number of servers, and in addition to adding new servers, we decided to spend some effort on ensuring a more even load on existing ones. Before we started doing something, the spread between CPU usage on the most loaded server and on the most free one was about 20-30%, and after that - only 2.5%. On our volumes, this allowed us to save up to hundreds of servers, and it was certainly worth it.

I will talk about the existing approaches to the automatic distribution of weights for the machines, about what approaches we tried, and on what we established as a result. The solution we have written will be laid out in open-source so that you can use it in your project if necessary.

And finally, a classic report on the architecture of a large project, albeit a very large, largest ad service in RuNet - Avito.ru. Mikhail Tyurin, Avito Chief System Architect, will tell you about "Where do your ads live?"

There are no equal high-loaded projects, the devil is in the details, in the nuances of the work of a particular function, the storage and use of a particular data block. That is why I begin my course of lectures on high-loaded systems at MIPT with a story and a demonstration of the importance of the analytical part of working on a high-loaded project. And that is why I recommend to go to the report of Michael - the largest classifier in Europe, 600 million ads - how does it all work?

See you at the conference!

PS Tutorial

By the way, for several weeks now we have been testing a new textbook on the design of high-load systems. This is a series of letters built on the lectures of the best specialists in Russia. Somewhere we decipher lectures, somewhere we provide a video, but the following is important - each of the materials has been checked by us, it is stamped on the quality of the HighLoad ++ conference.

If interested - sign up, it's free: http://highload.guide/

And finally : For the users of Habrakhabr, the conference offers a special discount of 15%, all you need to do is use the code " IAmHabr " when booking tickets.