Overview of WhatsApp Toolkit

In March 2014, the chief engineer of WhatsApp, Rick Reed, talked about optimizing the company's infrastructure, which was required due to the growing number of users. We studied Rick's materials, other presentations on the topic and publications prepared on the basis of conversations with company representatives, and gave a brief overview of the main tools of WhatsApp.

You can also familiarize yourself with the recording of Rick’s speech and slides from his presentation.
')


Before turning to the description of the WhatsApp architecture, let us try to understand why Facebook paid $ 19 billion for a regular instant messenger. The main thing that Facebook needed was that nearly billions of active users of WhatsApp, which still continues to grow.

Analyst Benedict Evans (Benedict Evans), talking about the trillionth market of mobile applications, gives the following fact: "Users of mobile devices around the world send more than 20 billion SMS messages every day, which is comparable to the number of messages passing through the WhatsApp application." At the moment , more than 1 billion people are actively using the messenger, provided that only a few dozen developers are working in the company.

WhatsApp employees do not particularly cover technical solutions, but in September 2015 in San Jose, California, the developer of the company Jamshid Mahdavi gave an interview, during which he briefly spoke about the technologies used. Part of the secret is that the company builds its service using the Erlang programming language.



Erlang originates in the 80s of the last century. Engineers from Ericsson, a hardware and software company for telecommunications companies, have developed a language that is suitable for high-speed telephone networks.

“Instead of creating a language and then deciding what to do with it, they immediately identified a specific task for which it was intended,” said Francesco Cesarini, the guru of the Erlang language. - He solved the problem of scalability and reliability. At that time, telephone networks were the only system that should have these two properties. ”

Moreover, Erlang offers a hot- code replacement, when the application can run both the old and the new version of the code at the same time. It turns out that the software on Erlang can be upgraded without downtime.

WhatsApp uses a modified messaging and presence protocol for XMPP (formerly known as Jabber) with SSL and TLS encryption to send messages. SSL and TLS are widely used cryptographic protocols that provide secure data transmission over the Internet. In addition to WhatsApp protocol is used in web browsers and when working with e-mail.



XMPP architecture

Until the user launches the application, all incoming messages are lined up and stored in the server’s memory. As soon as the message gets to the user's device, it is erased from the server’s memory, and the sender receives confirmation of its receipt. One of the StackOverflow participants described the messaging process:

“Alice decides to send a message to Bob. Alice's smartphone establishes a connection to the WhatsApp server, which, in turn, determines whose device it is. Then Alice sends a TCP message: "For Bob: A giant monster attacks the Golden Gate Bridge." One of the WhatsApp frontend servers deserializes this message and delivers it to the actor Alice.

Actor Alice decides to serialize the message and save it in a file called Alice's Sent Messages. Then the actor Alice decides to transfer it to the actor Bob in the following form: "Message1 from Alice: A giant monster attacks the Golden Gate Bridge." Actor Alice sends a message and waits for delivery confirmation from actor Bob.

Actor Bob, having received the message, saves it to a file called “Inbox for Bob” and sends a notification of receipt of the letter. He then checks if Bob's smartphone is connected to the server. If so, then actor Bob sends a message over TCP.

Bob, seeing the message, decides to answer: "For Alice: Let's build giant robots and fight with him." Now this message is received by the actor Bob and repeats all the steps described above, so that the idea of ​​saving humanity comes to Alice. ”

The multi-user distributed system Mnesia DB, which uses about 2 TB of RAM and stores about 18 billion records, is used as a DBMS. It is written in Erlang. From the point of view of request processing speed, a rather efficient integration is obtained. Photos, audio and video attached to the message are uploaded to the YAWS web server, and links to them are sent to the recipient.

Using Erlang brought incredible scalability. The initial WhatsApp load was 200,000 simultaneous connections to the server. A little later, this figure increased significantly and reached 2.8 million compounds. All this magnificence is processed by approximately 550 servers, of which 150 are chat servers supporting up to 1 million smartphones. About 250 more machines are multimedia servers. The nodes responsible for the operation of the databases have 512 GB of RAM, and the usual computing nodes have 64 GB. You can learn more about the subtleties and characteristics of the scaling process from High Scalability from this post.

In January 2015, WhatsApp launched the web version of its popular mobile app. This is quite an interesting point, because the company has never stored user messages on its server - all information was stored on smartphones. Let's see what tools the developers used.

Biplav Saraf (Biplav Saraf) writes in his blog that the main frameworks were React.js - a framework for creating interfaces from Facebook, Underscore.js - a JavaScript library that implements additional functionality for working with arrays, objects and functions, and Velocity.js - engine for working with animation.

WhatsApp Web is an add-on to a WhatsApp account on your phone, and therefore it connects to a mobile device to synchronize messages. It turns out that to use the web client, the phone must be turned on, because it is directly involved in the transfer of messages.

Biplav wrote a plugin for Chrome to see what happens when you open the web client and see the QR code.



He found out that the information about the current client, OS and browser is sent first and the session ID is transmitted. Biplav believes that the third frame in the list is a TTL frame. When you scan a QR code, the mobile client connects to the specified channel using the WebSocket protocol. WebSocket is a full-duplex protocol over TCP connection designed for real-time messaging between a browser and a web server. The web client receives the following frames:



The selected frame is of the most interest because it contains data transmitted by the mobile client to the web client. It turns out that it even contains information about the battery status of the smartphone and whether the mobile device is charging.

findings

WhatsApp has proven that the size of the company does not matter. A team of several dozen developers was able to create a popular instant messenger using inexpensive hardware and open source software. Of course, a certain role was played by the fact that the company was in the right place at the right time and offered its product to the right customer, but you shouldn’t disregard the fact that developers set realistic goals for themselves and achieved their realization.

When Mahdavi was asked about how the company managed to achieve such a resounding success, he replies that you just need to concentrate on your goal. “Do not be distracted by third-party tasks, other technologies or unnecessary office activities, such as meetings,” Mahdavi said. WhatsApp employees almost never hold meetings. Of course, there are only a few dozen workers, but they achieved everything in just this quantity.

PS Of course, we also share our own experience - most recently we talked about how we upgraded the project site in 1cloud and developed some user functions.