How Wargaming Common Menu Works
Good day!
I want to share with the community development experience JS-widget interproject navigation. It is a module that connects to most sites of the Wargaming universe ( Portals , Wiki , WarGag , etc.).
Its main task is to provide the user with convenient navigation between different services of the same subject. But it also performs a number of other functions, for example, it displays personal notifications, brief dossiers on user profiles in each of our games, and much more.
')
There are quite a lot of sites on which the menu is embedded, and they all have a different design and layout. So historically, the sites are built on different frameworks and at different times, use different libraries and sometimes do not look at each other at all.
The menu should be at the very top of the page, that is, in fact, be displayed first, so that the user does not notice that the menu is a separate component.
Our sites should be removed the need to monitor the release of new projects, changing the addresses of existing ones. Updating menu items should occur without the participation of sites.
The menu should be able to show the current user data on his account - the presence / absence of profiles in different projects, a brief current statistics for each of the projects.
Also, through the menu, the user must have access to private notifications, while the flow of notifications for each user can be quite intensive.
In essence, the project consists of two independent applications - the frontend and backend, which are deployed separately and exist independently.
The frontend is a JS application, a set of static files, and the backend is a JSON-API with access to data using a special token. This scheme was chosen mainly because it is necessary to withstand a decent load (total for all sites) and to ensure operation without downtime with regular updates to the new version, with minimal consequences in the event of a “fall”, since this would mean partial inoperability. almost all of our public web services.
The availability of the frontend is provided by a multi-level caching scheme: the user's browser is a CDN server, the origin server is a spare origin server. The CDN server works as a caching proxy. Origin servers are located in different data centers, and at the CDN configuration level, alternate fallback is configured for them.
Cache disability occurs using the CDN provider purge API command, and the browser cache is managed using the URL GET parameters.
Sites connect the menu by simply adding a script loader to the page and determining the place where it should be drawn:
Then the menu begins to live its own life - it loads up the necessary components, refers to the backend for the necessary data, etc. For example, dropdowns with games, notifications and user data are loaded and rendered only when they are needed.
Since for the site connecting the menu, the script loader is a third-party script that can generally take longer to load than the local statics of the project, it is possible to connect the loader asynchronously (with the async attribute) - so that site loading is not blocked, and the fact of successful loading using callback.
Consumer sites do not participate in menu releases, which means that there is no way to change the src loader to reset the browser cache, so HTTP headers are used:
With them, the browser every time when accessing the file of the loader sends a HEAD request to the server, and if the server responds with 304 Not Modified, the file is taken from the cache.
Frontend rolls out on prod servers assembled package. Grunt builds, he sticks together and minifies source files, compiles scss to css, assembles icons into sprites (SVG and PNG separately), generates predefined link sets for the menu. Also in dev-mode, it is possible to start the project “by itself” on express'e with backend emulation.
All icons are drawn in SVG vector format and compressed into one sprite by the Grunt plugin dr-svg-sprites . This allows you to not worry about a separate copy of the increased size for the retina and wins the file size. In addition, for old IE, this plugin generates a PNG sprite, which is very convenient and saves us from headaches and a lot of bugs.
To configure the menu to the needs of a particular site, data-attributes are used, which the site developer determines when the downloader is embedded.
To the case when the site at the time of embedding the loader itself does not know the values ​​of all the parameters, there is an alternative way to transfer the parameters - to write the corresponding cookies:
And there is a JS-API through which you can do the same asynchronously.
"Configuration" to display the menu on the site is selected based on these parameters. The initial configs store the structure of our resources in a normalized form. At the assembly stage, specific sets of links are compiled for each combination (excluding the impossible ones). Formed sets are uploaded to the site and used to render the template.
The menu can save the user's personal settings regardless of the site where it is connected without the participation of the backend. To save user preferences, Local Storage is used, or Cookies in case the first one is unavailable. Since the menu has its own domain, it allows you to save the settings cross-domain, that is, you can work as if all the menu sites being opened are on the same domain. To achieve this effect, a static HTML file is used, which is loaded in a frame. This frame is on your domain and stores information in LS or in cookies, and PostMessage API is used for data exchange.
The menu can send desktop notifications to the user (if supported by the browser). They are used to report new unread messages when the tab with the site is inactive or the browser is minimized. Since the user can have several tabs open at the same time with different sites, but everyone has a menu and everyone knows how to send a notification, we taught different instances of the application to communicate with each other so that they can agree on who will send the notification. This is done using the same frame that stores data in Local Storage or Cookies, and provides access to them through the JS API.
The menu scaling is adapted for mobile devices, and automatically changes the view on the pre-set breakpoint. As a template engine, we use the John Resig solution , slightly modified for our needs. It is modest in terms of functionality, but it allows using the same templates in JS and on the backend in Jinja2 and does not require heavy libraries to be loaded.
For most of the links, the designers had to make a non-standard underscore (the underline line should be below the standard and appear smoothly). We did it this way:
We first decided to animate the width of the underscore, but it looked too irregular. Then there was the option of approaching the underscore from below, but in the end we decided to dwell on the simple appearance of transparency.
Another design feature was the different styling of the elements in the Games menu, depending on the number of these very elements.
Because There may be a different number of games on the menu in different regions, I had to immediately write all the options in the code. We did it in pure CSS. About the cunning method of counting elements have already been written on Habré here and here .
In short, using the pseudo-selector : nth-last-child (n), we find out if we have at least n elements. How to check that we have exactly n elements? Just add the pseudo-selector : first-child (or nth-child (1) ).
Thus we will select the first element from n . The remaining elements can be selected using the selector ~ .
For example, this is how we stylize a list with six nested elements:
If the user of the site is currently logged in, then the menu contacts the backend for data on this user. Such calls are periodically repeated to update the UI.
The backend is built on Twisted workers, each of which is responsible for a specific functionality; for example, there is a separate server for issuing a user profile, a separate server for notifications, and so on.
However, publicly accessible servers are the tip of the iceberg. All the main work takes place in Twisted-demons, which process events arriving through AMQP queues and store the results of work on different databases.
Their main task is to collect the necessary data on internal web components.
For quick access to data from the outside, a Redis-cache is organized, which is filled and updated by the same workers. Reading from this cache comes directly from Nginx, using the HttpRedis module with fallback to the Python server.
Redis used to be the main data repository. He was completely satisfied because he had good performance and allowed not to worry about the obsolescence of sessions. And data access was instant: all data was in memory. However, the amount of data inexorably grew, and soon we began to face the problem of lack of RAM. For Redis to work correctly, it is necessary that at least as much memory is available as it is already in use, since the storage dump is periodically dumped onto the disk, for which the process is being forked.
We decided to use the MySQL database with the TokuDB engine as a permanent storage (TokuDB was chosen because of good data compression and the ability to quickly change the structure of the tables) and Redis for storing the session key. Redis is also used as cache storage.
If the user has javascript disabled in the browser, the backend can secure and render the menu on the server. Then the mapping takes place in the frame. For rendering, I use the same templates and styles as in JS.
The stated requirements are implemented, but the decision asks for the possibility of more flexible customization. Initially, we calculated that it would be possible to put all the consumer sites in a clear structure, but in fact consumers are very different. We will go in the direction of reducing the participation of menu developers to customize it, allowing each consumer to do it themselves.
Also in the plans are a number of interesting features that using the Common Menu can be implemented and delivered to the user simultaneously on all web services in a short time.
I want to share with the community development experience JS-widget interproject navigation. It is a module that connects to most sites of the Wargaming universe ( Portals , Wiki , WarGag , etc.).
Its main task is to provide the user with convenient navigation between different services of the same subject. But it also performs a number of other functions, for example, it displays personal notifications, brief dossiers on user profiles in each of our games, and much more.
')
First, a little about the requirements.
There are quite a lot of sites on which the menu is embedded, and they all have a different design and layout. So historically, the sites are built on different frameworks and at different times, use different libraries and sometimes do not look at each other at all.
The menu should be at the very top of the page, that is, in fact, be displayed first, so that the user does not notice that the menu is a separate component.
Our sites should be removed the need to monitor the release of new projects, changing the addresses of existing ones. Updating menu items should occur without the participation of sites.
The menu should be able to show the current user data on his account - the presence / absence of profiles in different projects, a brief current statistics for each of the projects.
Also, through the menu, the user must have access to private notifications, while the flow of notifications for each user can be quite intensive.
Now about the implementation
In essence, the project consists of two independent applications - the frontend and backend, which are deployed separately and exist independently.
The frontend is a JS application, a set of static files, and the backend is a JSON-API with access to data using a special token. This scheme was chosen mainly because it is necessary to withstand a decent load (total for all sites) and to ensure operation without downtime with regular updates to the new version, with minimal consequences in the event of a “fall”, since this would mean partial inoperability. almost all of our public web services.
CDN
The availability of the frontend is provided by a multi-level caching scheme: the user's browser is a CDN server, the origin server is a spare origin server. The CDN server works as a caching proxy. Origin servers are located in different data centers, and at the CDN configuration level, alternate fallback is configured for them.
Cache disability occurs using the CDN provider purge API command, and the browser cache is managed using the URL GET parameters.
Connection
Sites connect the menu by simply adding a script loader to the page and determining the place where it should be drawn:
<script id="common_menu_loader" type="text/javascript" charset="utf-8" data-language="en" src="http://menu.com/loader.min.js"></script> <div id="common_menu"></div> Then the menu begins to live its own life - it loads up the necessary components, refers to the backend for the necessary data, etc. For example, dropdowns with games, notifications and user data are loaded and rendered only when they are needed.
Since for the site connecting the menu, the script loader is a third-party script that can generally take longer to load than the local statics of the project, it is possible to connect the loader asynchronously (with the async attribute) - so that site loading is not blocked, and the fact of successful loading using callback.
Consumer sites do not participate in menu releases, which means that there is no way to change the src loader to reset the browser cache, so HTTP headers are used:
location / { expires 7d; } location = /loader.min.js { add_header Cache-Control "no-cache, must-revalidate"; } With them, the browser every time when accessing the file of the loader sends a HEAD request to the server, and if the server responds with 304 Not Modified, the file is taken from the cache.
Assembly
Frontend rolls out on prod servers assembled package. Grunt builds, he sticks together and minifies source files, compiles scss to css, assembles icons into sprites (SVG and PNG separately), generates predefined link sets for the menu. Also in dev-mode, it is possible to start the project “by itself” on express'e with backend emulation.
All icons are drawn in SVG vector format and compressed into one sprite by the Grunt plugin dr-svg-sprites . This allows you to not worry about a separate copy of the increased size for the retina and wins the file size. In addition, for old IE, this plugin generates a PNG sprite, which is very convenient and saves us from headaches and a lot of bugs.
Configuration
To configure the menu to the needs of a particular site, data-attributes are used, which the site developer determines when the downloader is embedded.
<script id="common_menu_loader" type="text/javascript" charset="utf-8" data-notifications_enabled="1" data-chat_enabled="0" data-intro_tooltips_enabled="1" src="http://menu.com/loader.min.js"></script> To the case when the site at the time of embedding the loader itself does not know the values ​​of all the parameters, there is an alternative way to transfer the parameters - to write the corresponding cookies:
cm.options.notifications_enabled="1" cm.options.chat_enabled="0" cm.options.intro_tooltips_enabled="1" And there is a JS-API through which you can do the same asynchronously.
WG.CommonMenu.update({ notifications_enabled: 1, chat_enabled: 0, intro_tooltips_enabled: 1 }); Storage settings
"Configuration" to display the menu on the site is selected based on these parameters. The initial configs store the structure of our resources in a normalized form. At the assembly stage, specific sets of links are compiled for each combination (excluding the impossible ones). Formed sets are uploaded to the site and used to render the template.
The menu can save the user's personal settings regardless of the site where it is connected without the participation of the backend. To save user preferences, Local Storage is used, or Cookies in case the first one is unavailable. Since the menu has its own domain, it allows you to save the settings cross-domain, that is, you can work as if all the menu sites being opened are on the same domain. To achieve this effect, a static HTML file is used, which is loaded in a frame. This frame is on your domain and stores information in LS or in cookies, and PostMessage API is used for data exchange.
Notifications
The menu can send desktop notifications to the user (if supported by the browser). They are used to report new unread messages when the tab with the site is inactive or the browser is minimized. Since the user can have several tabs open at the same time with different sites, but everyone has a menu and everyone knows how to send a notification, we taught different instances of the application to communicate with each other so that they can agree on who will send the notification. This is done using the same frame that stores data in Local Storage or Cookies, and provides access to them through the JS API.
Layout
The menu scaling is adapted for mobile devices, and automatically changes the view on the pre-set breakpoint. As a template engine, we use the John Resig solution , slightly modified for our needs. It is modest in terms of functionality, but it allows using the same templates in JS and on the backend in Jinja2 and does not require heavy libraries to be loaded.
For most of the links, the designers had to make a non-standard underscore (the underline line should be below the standard and appear smoothly). We did it this way:
.link:after { content: ""; border-bottom: 0 solid; position: absolute; top: 50%; left: 0; width: 100%; margin-top: 8px; opacity: 0; transition: .3s ease opacity; } .link:hover:after { opacity: .8; border-bottom-width: 1px; // IE8 hack } We first decided to animate the width of the underscore, but it looked too irregular. Then there was the option of approaching the underscore from below, but in the end we decided to dwell on the simple appearance of transparency.
Another design feature was the different styling of the elements in the Games menu, depending on the number of these very elements.
Because There may be a different number of games on the menu in different regions, I had to immediately write all the options in the code. We did it in pure CSS. About the cunning method of counting elements have already been written on Habré here and here .
In short, using the pseudo-selector : nth-last-child (n), we find out if we have at least n elements. How to check that we have exactly n elements? Just add the pseudo-selector : first-child (or nth-child (1) ).
Thus we will select the first element from n . The remaining elements can be selected using the selector ~ .
For example, this is how we stylize a list with six nested elements:
li:nth-child(1):nth-last-child(6), li:nth-child(1):nth-last-child(6) ~ li { ... } User profile
If the user of the site is currently logged in, then the menu contacts the backend for data on this user. Such calls are periodically repeated to update the UI.
The backend is built on Twisted workers, each of which is responsible for a specific functionality; for example, there is a separate server for issuing a user profile, a separate server for notifications, and so on.
However, publicly accessible servers are the tip of the iceberg. All the main work takes place in Twisted-demons, which process events arriving through AMQP queues and store the results of work on different databases.
Their main task is to collect the necessary data on internal web components.
Data storage
For quick access to data from the outside, a Redis-cache is organized, which is filled and updated by the same workers. Reading from this cache comes directly from Nginx, using the HttpRedis module with fallback to the Python server.
Redis used to be the main data repository. He was completely satisfied because he had good performance and allowed not to worry about the obsolescence of sessions. And data access was instant: all data was in memory. However, the amount of data inexorably grew, and soon we began to face the problem of lack of RAM. For Redis to work correctly, it is necessary that at least as much memory is available as it is already in use, since the storage dump is periodically dumped onto the disk, for which the process is being forked.
We decided to use the MySQL database with the TokuDB engine as a permanent storage (TokuDB was chosen because of good data compression and the ability to quickly change the structure of the tables) and Redis for storing the session key. Redis is also used as cache storage.
JS disabled
If the user has javascript disabled in the browser, the backend can secure and render the menu on the server. Then the mapping takes place in the frame. For rendering, I use the same templates and styles as in JS.
Finally
The stated requirements are implemented, but the decision asks for the possibility of more flexible customization. Initially, we calculated that it would be possible to put all the consumer sites in a clear structure, but in fact consumers are very different. We will go in the direction of reducing the participation of menu developers to customize it, allowing each consumer to do it themselves.
Also in the plans are a number of interesting features that using the Common Menu can be implemented and delivered to the user simultaneously on all web services in a short time.
Source: https://habr.com/ru/post/259687/
All Articles