Experiment with the search results page
The search results page is one of the most popular pages on Yandex. It is loaded about 130 million times a day. This, with an average page size of 25KB, gives us 3TB of traffic per day.
Despite the seeming simplicity, what this page consists of is a huge work of a large number of people and a lot of complex technologies.
Developing interfaces, we usually follow the evolutionary path, changing the page in stages. We check our solutions, introducing them to a small percentage of users — we conduct experiments. We are not satisfied with small changes: we want to develop a product by building a new technological platform on which we will implement our projects in the future.
')
Today we begin the experiment with a new page of search results. And for this we have chosen our platform for testing the search on the world Internet - yandex.com . The first thing you notice when you see a new page is the difference between its design and the current version of the search results page on yandex.ru. But this experiment is primarily not about the design, but about the implementation of significant technological changes. Of course, we are also working on the design and are planning to test new solutions, but the construction of a new one begins with a high-quality foundation. We want to tell exactly how we built it: what decisions were made and what technologies were used.
Template Development
Even before the first picture layouts appeared, we needed to decide on the approach to development. There were two options: to make up a static page and then cut it into templates or develop it directly into real data and write real templates.
In the general case, especially if one team is engaged in the layout and development of templates, the second approach is preferable. It ensures that the project is ready for release (or at least for testing) at any time. That is, at the very beginning of development, a “raw”, but already functioning, prototype of the project arises, which can be “touched”. As a result, we chose this option.
The time spent on the development and testing of the first prototype was greatly helped by our project assembly mechanism based on the BEM ( Block-Element-Modifier ) methodology. It allows you to connect external libraries of blocks, so we assembled a substantial part of the interface from ready-made blocks that already exist on other Yandex services and are located in different internal git and svn repositories.
JavaScript everywhere
Under the hood of the new version of the search results page - JavaScript templates. They work faster than the template engine for the Perl TT2 language, which we used before, and it is easier and easier to write them. Now JS has two levels of templating, client scripts, utilities and assembly technologies, a daemon builder.
At the moment, YUICompressor, which requires Java on the server, is used to compress JS- and CSS-code, so in the long run we want to switch to JS-obfuscators UglifyJS and CSSO - they are faster and less demanding of resources.
The use of the same language at all technological levels, starting with the browser and ending with the assembly utilities, greatly simplifies the life of many participants in the process. Developers can configure (and optionally add) tools to build, write code for client and server equally easily. Deep knowledge of a large number of different technologies is no longer required; it is good enough to know the basic ones: HTML, CSS, JS. Thanks to this, it became easier to connect new people to the team: JavaScript is known to all front-end developers in Yandex, and server specificity can be mastered very quickly.
AJAX, History API
Quite a long time ago, it became “fashionable” to use the AJAX technique to update the data on the page without a full reload. The advantages of this technique are quite obvious: it is not necessary to load the entire page (markup, styles, scripts, pictures), and initialize the scripts each time the data is updated. This greatly speeds up the process of delivering new data to the user. The difference in download speed on slow channels is radical.
However, knowing all the advantages of AJAX, we were in no hurry to apply this approach in Yandex. Search. We wanted to offer our users not just technology, but a holistic product with a redesigned interface. Therefore, the new version of the search results page will require the widespread use of AJAX and History API.
(We talked about our implementation in detail on June 2, 2012 in Minsk on Ya.Subbotnik)
CSS
Initially, we didn’t have a goal to use the full capabilities of CSS3, but almost all of them turned out to be necessary to solve our problems. Arrows menu on the transformations, the shadows of the blocks, pseudo-selectors first-child / last-child instead of modifier classes - all this is in the new interface. At the same time, for old browsers, degradation is provided: instead of shadows - simple single-pixel outlines, instead of arrows in the menu - regular rectangles, svg-gradients in IE9 and solid background fill for IE8 and older.
Another important decision was the use of absolute units of measure to specify font sizes. Almost all of our services now use the indication of sizes in relative percentages or <em> tags, but in the new project we decided to abandon them in favor of pixels. This decision has several arguments. First, the proportion of browsers that cannot scale fonts in pixels is vanishingly small. Secondly, dimensions in absolute units exclude the influence of the context, and layout with absolutely independent blocks is one of the basic principles of the BEM methodology. And thirdly, the search results page should work fine on touch devices. The screen resolution on them is often known in advance and strictly fixed, and only its orientation can change.
Depla
Almost all of our projects store their static files on a static cluster - yandex.st. These are several dozens of cars located in different cities and countries that can give data very quickly.
There are several reasons for using a static cluster:
- no need to generate files before sending - everything is ready and lies on the disk;
- to transfer ready-made files rather easy web server;
- does not require high performance machines;
- the static is cached forever, and the invalidation is carried out either by changing the version of the package, or by changing the hash sum;
- service cookies are not sent to the yandex.st domain, this reduces the amount of traffic;
- “pass-through” files (used on several services, for example jquery.min.js or the Yandex logo) do not need to be loaded every time when switching from service to service.
To put the service files on a static cluster, you need to build a deb package. The package is assembled either manually or with a special script. After assembling the new package, the application for the display is sent to the Conductor - our internal tool that allows laying out the package on hundreds of machines without the participation of people. This is very important, given the number of Yandex services and the frequency of their updates.
A static cluster is a very complex system that solves many problems of optimizing the speed of the browser at the client and delivering files to the user. At the same time, it is very simple to use it: in order to start a new project in Conductor, it is enough to fill out a small application form and receive confirmation from the responsible administrator, after which the developer of the service completely controls the layout process. The time required to prepare the statics of a new service for deployment is literally reduced to a couple of hours of one specialist, and the new version of the package is laid out and deployed in a few seconds.
Updating the search results page will continue: both on yandex.com and on other versions of Yandex, from Turkish to Russian. We have a lot of experiments in our plans, including with interfaces.
Mikhail Troshev ,
Search Engine Interface Development Team Leader
Despite the seeming simplicity, what this page consists of is a huge work of a large number of people and a lot of complex technologies.
Developing interfaces, we usually follow the evolutionary path, changing the page in stages. We check our solutions, introducing them to a small percentage of users — we conduct experiments. We are not satisfied with small changes: we want to develop a product by building a new technological platform on which we will implement our projects in the future.
')
Today we begin the experiment with a new page of search results. And for this we have chosen our platform for testing the search on the world Internet - yandex.com . The first thing you notice when you see a new page is the difference between its design and the current version of the search results page on yandex.ru. But this experiment is primarily not about the design, but about the implementation of significant technological changes. Of course, we are also working on the design and are planning to test new solutions, but the construction of a new one begins with a high-quality foundation. We want to tell exactly how we built it: what decisions were made and what technologies were used.
Template Development
Even before the first picture layouts appeared, we needed to decide on the approach to development. There were two options: to make up a static page and then cut it into templates or develop it directly into real data and write real templates.
In the general case, especially if one team is engaged in the layout and development of templates, the second approach is preferable. It ensures that the project is ready for release (or at least for testing) at any time. That is, at the very beginning of development, a “raw”, but already functioning, prototype of the project arises, which can be “touched”. As a result, we chose this option.
The time spent on the development and testing of the first prototype was greatly helped by our project assembly mechanism based on the BEM ( Block-Element-Modifier ) methodology. It allows you to connect external libraries of blocks, so we assembled a substantial part of the interface from ready-made blocks that already exist on other Yandex services and are located in different internal git and svn repositories.
JavaScript everywhere
Under the hood of the new version of the search results page - JavaScript templates. They work faster than the template engine for the Perl TT2 language, which we used before, and it is easier and easier to write them. Now JS has two levels of templating, client scripts, utilities and assembly technologies, a daemon builder.
At the moment, YUICompressor, which requires Java on the server, is used to compress JS- and CSS-code, so in the long run we want to switch to JS-obfuscators UglifyJS and CSSO - they are faster and less demanding of resources.
The use of the same language at all technological levels, starting with the browser and ending with the assembly utilities, greatly simplifies the life of many participants in the process. Developers can configure (and optionally add) tools to build, write code for client and server equally easily. Deep knowledge of a large number of different technologies is no longer required; it is good enough to know the basic ones: HTML, CSS, JS. Thanks to this, it became easier to connect new people to the team: JavaScript is known to all front-end developers in Yandex, and server specificity can be mastered very quickly.
AJAX, History API
Quite a long time ago, it became “fashionable” to use the AJAX technique to update the data on the page without a full reload. The advantages of this technique are quite obvious: it is not necessary to load the entire page (markup, styles, scripts, pictures), and initialize the scripts each time the data is updated. This greatly speeds up the process of delivering new data to the user. The difference in download speed on slow channels is radical.
However, knowing all the advantages of AJAX, we were in no hurry to apply this approach in Yandex. Search. We wanted to offer our users not just technology, but a holistic product with a redesigned interface. Therefore, the new version of the search results page will require the widespread use of AJAX and History API.
(We talked about our implementation in detail on June 2, 2012 in Minsk on Ya.Subbotnik)
CSS
Initially, we didn’t have a goal to use the full capabilities of CSS3, but almost all of them turned out to be necessary to solve our problems. Arrows menu on the transformations, the shadows of the blocks, pseudo-selectors first-child / last-child instead of modifier classes - all this is in the new interface. At the same time, for old browsers, degradation is provided: instead of shadows - simple single-pixel outlines, instead of arrows in the menu - regular rectangles, svg-gradients in IE9 and solid background fill for IE8 and older.
Another important decision was the use of absolute units of measure to specify font sizes. Almost all of our services now use the indication of sizes in relative percentages or <em> tags, but in the new project we decided to abandon them in favor of pixels. This decision has several arguments. First, the proportion of browsers that cannot scale fonts in pixels is vanishingly small. Secondly, dimensions in absolute units exclude the influence of the context, and layout with absolutely independent blocks is one of the basic principles of the BEM methodology. And thirdly, the search results page should work fine on touch devices. The screen resolution on them is often known in advance and strictly fixed, and only its orientation can change.
Depla
Almost all of our projects store their static files on a static cluster - yandex.st. These are several dozens of cars located in different cities and countries that can give data very quickly.
There are several reasons for using a static cluster:
- no need to generate files before sending - everything is ready and lies on the disk;
- to transfer ready-made files rather easy web server;
- does not require high performance machines;
- the static is cached forever, and the invalidation is carried out either by changing the version of the package, or by changing the hash sum;
- service cookies are not sent to the yandex.st domain, this reduces the amount of traffic;
- “pass-through” files (used on several services, for example jquery.min.js or the Yandex logo) do not need to be loaded every time when switching from service to service.
To put the service files on a static cluster, you need to build a deb package. The package is assembled either manually or with a special script. After assembling the new package, the application for the display is sent to the Conductor - our internal tool that allows laying out the package on hundreds of machines without the participation of people. This is very important, given the number of Yandex services and the frequency of their updates.
A static cluster is a very complex system that solves many problems of optimizing the speed of the browser at the client and delivering files to the user. At the same time, it is very simple to use it: in order to start a new project in Conductor, it is enough to fill out a small application form and receive confirmation from the responsible administrator, after which the developer of the service completely controls the layout process. The time required to prepare the statics of a new service for deployment is literally reduced to a couple of hours of one specialist, and the new version of the package is laid out and deployed in a few seconds.
Updating the search results page will continue: both on yandex.com and on other versions of Yandex, from Turkish to Russian. We have a lot of experiments in our plans, including with interfaces.
Mikhail Troshev ,
Search Engine Interface Development Team Leader
Source: https://habr.com/ru/post/149327/
All Articles