Mail.Ru Agent in the browser: approaches to state synchronization
One of the most notable trends in the modern Internet is the weakening of the role of “classic” desktop messengers, which until recently reigned supremely on users' computers, and the increasing role of mobile and web clients. The latter have gained particular popularity due to the new generation of postal services, as well as social networks.
Mail.Ru, which in 2008 launched the first version of Mail.Ru Agent for the web (hereinafter - simply Agent), did not stand aside.
')
However, if the user may not even think about the difference between the standalone application and the web client, then for the developer the difference is obvious. But the difficulties and problems that have to be solved by changing the philosophy of a separate program to interact with the browser may be less transparent.
The first "enemy" of the developer in this case ... the user himself. Instead of intently chatting with his friends on a social network page or email, he clicks the links and walks through the pages. So - you need to remember the full state of the client and restore it on the next page loaded.
And as soon as this problem is resolved, the user remembers that his browser supports tabs, and is taken to open in new tabs, for example, letters. In this case, we need to synchronize the state of the client with which the user is interacting at the moment, with all open copies of the client - so that in any tab the client looks the same.
In general, the developer has a whole complex of problems of saving / changing and synchronizing the client’s state.
The history of the development of the Agent largely coincides with the history of overcoming the listed problems by the developers. In the process, we encountered many expected and unexpected difficulties that we had to either overcome or bypass. I will try to describe in general terms how it looked in practice.
However, first we define the basic concepts that we often use in our working group. To understand this article, it’s enough to remember only four terms: this is a state , an event , a router and a client .
A state is a set of properties of the Agent that determine its current appearance (presentation). These properties include a list of open dialogs, signs of the current dialog, the presence of unread messages in open dialogs, the state of Agent pseudo-windows (minimized / maximized), typing notification, etc.
The state can be changed by events that contain information about what has changed in the state. An event can be triggered both by a server (for example, when someone wrote a message to a user) or by clients.
A client is a program that runs in the user's browser and displays the user interface according to its state. In addition, the client can generate events — for example, when a user opens or closes a dialog, maximizes or minimizes a pseudo-window.
Finally, a router is a kind of “black box” to which all clients connect, and who always “knows” the state of the last Agent. The main task of the router is to receive events from the server, as well as the client with which the user is currently interacting, and send them to the other connected clients. Thus, the router provides state synchronization between all copies of the client.
NB: In general, from the point of view of network terminology, it would be more correct to call our router a hub, but historically it happened. :)
Stage I Server sync.
The first solution we tried was server synchronization, that is, “real” dedicated servers acted as routers. The advantages seemed obvious: no matter from which browser or computer a connection is made, the status of all clients turns out to be synchronized. The simplicity of the client's design was also very tempting - it turned out to be really “subtle”, because its task was only in user interaction and in drawing state changes.
However, almost immediately revealed two bottlenecks:
As a result, the Agent more or less coped only with chat in one or two dialogs on a fast network connection and provided that the user is moderately active. As soon as the user began to communicate with a large number of contacts and to do it quickly, synchronization “collapsed” under the load of the network overhead projector, especially with a large number of open windows - because the same events had to be delivered separately to each window. In addition to the low response speed, this approach created a high load on the routers, so we rather quickly abandoned it.
Stage II. Transfer synchronization to the client.
The obvious answer to server synchronization questions was the transfer of this task to the client computer. Here and the computing power that is not shared with other users, and the absence of a long "shoulder" inherent in the system "browser1 - server - browser2".
As a solution, Adobe Flash was chosen, or rather its Flash Local Connection class, designed specifically for event dispatching. In this architecture, the first running instance of the client acted as a router. Each new client copy checked whether the router was already created, connected to it and started sending and receiving states from it, and not directly from the server. In fact, it turned out "thin client with thick elements." :)
This implementation brought relief compared to server synchronization, but rather quickly we encountered other difficulties - this time, unfortunately, “buried” inside Flash Player itself.
To begin with, it turned out that the internal storage of Flash Player does not allow for frequently writing and reading data and storing their large volumes, which was a typical scenario with intensive use of the Agent.
However, the biggest problems started after the next update of Flash Player, when for unknown reasons, the speed of Local Connection dropped sharply, and the signal from one tab to another literally "lost in the wires." Combined with the not-so-high-speed performance of the Flash Player as a whole, this often led to significant out-of-sync.
Stage III. HTML 5 client synchronization.
The output of HTML 5 and the implementation of its support in most modern browsers have given new tools to web application developers. The best news for us was the appearance of our own local storage of the browser. Now, without the use of external technologies, it became possible to store locally about 5 MB of data and provide quick access to them to each client instance. In fact, this is the same Local Connection, only much faster and more reliable, as well as without restrictions on reading and writing.
In practice, this allows us to abandon the router and store a general state that all instances of the client “see” and easily change (of course, this only works within one browser). However, along with new opportunities, new difficulties have arisen. For example, to synchronize states between different domains that host pages with an Agent, an iframe is created that is subject to the same origin policy rule. To get around this limitation, I had to use the EasyXDM library and the postMessage transport. Interestingly, this library allows cross-domain message exchange with sender-recipient verification.
Now this synchronization method is the main one, although in older browsers that do not support local storage, we still have to use state synchronization through Flash Player.
And in the end I will tell...
The appearance of the Web Agent is currently not quite in line with our ideas of beauty, but we are actively working on it and very soon we will be ready to officially present the new design. Wait for announcements!
Ilya Naumov,
Project Manager Mail.Ru Agent
Mail.Ru, which in 2008 launched the first version of Mail.Ru Agent for the web (hereinafter - simply Agent), did not stand aside.
')
However, if the user may not even think about the difference between the standalone application and the web client, then for the developer the difference is obvious. But the difficulties and problems that have to be solved by changing the philosophy of a separate program to interact with the browser may be less transparent.
The first "enemy" of the developer in this case ... the user himself. Instead of intently chatting with his friends on a social network page or email, he clicks the links and walks through the pages. So - you need to remember the full state of the client and restore it on the next page loaded.
And as soon as this problem is resolved, the user remembers that his browser supports tabs, and is taken to open in new tabs, for example, letters. In this case, we need to synchronize the state of the client with which the user is interacting at the moment, with all open copies of the client - so that in any tab the client looks the same.
In general, the developer has a whole complex of problems of saving / changing and synchronizing the client’s state.
The history of the development of the Agent largely coincides with the history of overcoming the listed problems by the developers. In the process, we encountered many expected and unexpected difficulties that we had to either overcome or bypass. I will try to describe in general terms how it looked in practice.
However, first we define the basic concepts that we often use in our working group. To understand this article, it’s enough to remember only four terms: this is a state , an event , a router and a client .
A state is a set of properties of the Agent that determine its current appearance (presentation). These properties include a list of open dialogs, signs of the current dialog, the presence of unread messages in open dialogs, the state of Agent pseudo-windows (minimized / maximized), typing notification, etc.
The state can be changed by events that contain information about what has changed in the state. An event can be triggered both by a server (for example, when someone wrote a message to a user) or by clients.
A client is a program that runs in the user's browser and displays the user interface according to its state. In addition, the client can generate events — for example, when a user opens or closes a dialog, maximizes or minimizes a pseudo-window.
Finally, a router is a kind of “black box” to which all clients connect, and who always “knows” the state of the last Agent. The main task of the router is to receive events from the server, as well as the client with which the user is currently interacting, and send them to the other connected clients. Thus, the router provides state synchronization between all copies of the client.
NB: In general, from the point of view of network terminology, it would be more correct to call our router a hub, but historically it happened. :)
Stage I Server sync.
The first solution we tried was server synchronization, that is, “real” dedicated servers acted as routers. The advantages seemed obvious: no matter from which browser or computer a connection is made, the status of all clients turns out to be synchronized. The simplicity of the client's design was also very tempting - it turned out to be really “subtle”, because its task was only in user interaction and in drawing state changes.
However, almost immediately revealed two bottlenecks:
- server delays and overwrite states;
- network latency.
As a result, the Agent more or less coped only with chat in one or two dialogs on a fast network connection and provided that the user is moderately active. As soon as the user began to communicate with a large number of contacts and to do it quickly, synchronization “collapsed” under the load of the network overhead projector, especially with a large number of open windows - because the same events had to be delivered separately to each window. In addition to the low response speed, this approach created a high load on the routers, so we rather quickly abandoned it.
Stage II. Transfer synchronization to the client.
The obvious answer to server synchronization questions was the transfer of this task to the client computer. Here and the computing power that is not shared with other users, and the absence of a long "shoulder" inherent in the system "browser1 - server - browser2".
As a solution, Adobe Flash was chosen, or rather its Flash Local Connection class, designed specifically for event dispatching. In this architecture, the first running instance of the client acted as a router. Each new client copy checked whether the router was already created, connected to it and started sending and receiving states from it, and not directly from the server. In fact, it turned out "thin client with thick elements." :)
This implementation brought relief compared to server synchronization, but rather quickly we encountered other difficulties - this time, unfortunately, “buried” inside Flash Player itself.
To begin with, it turned out that the internal storage of Flash Player does not allow for frequently writing and reading data and storing their large volumes, which was a typical scenario with intensive use of the Agent.
However, the biggest problems started after the next update of Flash Player, when for unknown reasons, the speed of Local Connection dropped sharply, and the signal from one tab to another literally "lost in the wires." Combined with the not-so-high-speed performance of the Flash Player as a whole, this often led to significant out-of-sync.
Stage III. HTML 5 client synchronization.
The output of HTML 5 and the implementation of its support in most modern browsers have given new tools to web application developers. The best news for us was the appearance of our own local storage of the browser. Now, without the use of external technologies, it became possible to store locally about 5 MB of data and provide quick access to them to each client instance. In fact, this is the same Local Connection, only much faster and more reliable, as well as without restrictions on reading and writing.
In practice, this allows us to abandon the router and store a general state that all instances of the client “see” and easily change (of course, this only works within one browser). However, along with new opportunities, new difficulties have arisen. For example, to synchronize states between different domains that host pages with an Agent, an iframe is created that is subject to the same origin policy rule. To get around this limitation, I had to use the EasyXDM library and the postMessage transport. Interestingly, this library allows cross-domain message exchange with sender-recipient verification.
Now this synchronization method is the main one, although in older browsers that do not support local storage, we still have to use state synchronization through Flash Player.
And in the end I will tell...
The appearance of the Web Agent is currently not quite in line with our ideas of beauty, but we are actively working on it and very soon we will be ready to officially present the new design. Wait for announcements!
Ilya Naumov,
Project Manager Mail.Ru Agent
Source: https://habr.com/ru/post/128699/
All Articles