View Search Engine Watch on Yandex search engine, its growth and development. The author of the article is Andy Atkins-Kruger (Search Engine Watch). Translated with the support of Marketing Drive.ComScore
published August data showing a significant increase in search activity worldwide — a story that spawned many publications. However, the statement went unnoticed in this release: "... the Russian search engine Yandex is growing most rapidly (94%) among the search engines are dozens of leaders." So why is Yandex developing so fast?
')
Google’s growth rate is 58%, Microsoft’s 41%, while others, including Baidu and Yahoo, show significantly lower growth rates (8% and 2%, respectively), and Yandex doubles year after year. Yandex continues to push its competitors to Yahoo, Microsoft, AOL and Ask even during these economically more difficult times.
An in-depth analysis of the figures shows that, according to comScore, Google’s share in the global volume of search queries increased the most - from 60.4% to 67.4%, while the share of Yandex increased by only 0.3% (the share of Microsoft and Ask did not change, while the share of Yahoo and Baidu has decreased).
Everything related to Yandex is a remarkable success story. Is this due to economic reasons? Or is it something related to search technology?
Yandex, which is often described as a copy of Google, was opened in 1997, shortly before Google. The first place in Russia, long preserved by Yandex, is explained by the fact that it was developed from the very beginning as a system that copes well with the difficulties of the Russian language - and he was the first to start doing this on the market.
The history of Yandex also has many parallels with the history of Google and Yahoo, with mathematicians Arkady Volozh and Arkady Borkovsky, who created the idea of ​​searching for the Russian language in their company Arkadia Inc. (later called CompTek). Initially they focused on searching for documents and intellectual property. After this, the parallels with Google become less pronounced, because they were not students at the time and did not start in the garage.
Yandex was a CompTek project, more like eBay. Volozh devoted himself entirely to this project, leaving his post as executive director of CompTek. Together with Arkady, a few more people left CompTek to create Yandex. Among them are Elena Kolmanovskaya and Mikhail Fadeev, also mathematicians. Borkovsky with time, in 2008, re-joined Yandex, after having worked for some time with Inktomi and Yahoo in the US, where he is now.
The emphasis on Russian is one of the key reasons why Yandex can be considered as an excellent example of how linguistics leads to success in multilingual search engine optimization. Western search engines and algorithms are essentially built from the building blocks of the English language. Of course, then they can be adapted, but the further the language is from English and the more complex the algorithm, the more difficult it is to implement such changes.
Russian is part of the Slavic language family, which includes a large number of languages ​​in Eastern Europe, from Polish and Czech to Serbian and Croatian. Most of these languages ​​do not use the Cyrillic alphabet, which is not, as it often seems, the reason for the complexity of indexing the Russian text by search engines - they are most likely due to the morphology or models and structure of the language.
In the present tense of the Russian language there is no verb “to be” (to be). The articles “the” and “a” are also missing. Instead, the endings of words are changed in Russian. Such languages ​​are called inflectional. Remember that “the” is considered by the search engines as a stop word.
In order to learn how to use a stop word in Russian, you need to know how much of the word disappears or changes. Unfortunately, the stemming algorithm, which is often used in English to improve search relevance, treats the plural and singular words as equivalents or shortens verbs to their basic parts, is much more difficult to use for the Russian language because of the huge number of possible variants.
For example, the Russian word “spew” (in English - “erupt”) has 166 different grammatical forms of the word. Perhaps now you are beginning to understand the difficulties that Yandex faced.
Meanwhile, the global economic crisis has also affected the Russian economy. During the first quarter of 2009, the state of its economy deteriorated by 9.8%, and only now the first signs of improvement have appeared. Therefore, this time the economic explanation does not fit.
The Russian statistical organization Liveinternet.ru provided data on how search engines work in Russia, the in-depth analysis of which is even more interesting. They show how stubborn the Google and Yandex struggle in Russia is. In the fall of 2007, Google moved to second place and then came closest to Yandex at the end of September 2008. But from this point on, Yandex easily overtakes it and increases the gap more and more.

Below are data on the percentage of the number of searches for September 2009, according to Liveinternet.ru. They show that Yandex is still the undisputed number one, and its share is increasing.

This is partly due to the fact that Yandex continues to firmly hold positions in the Russian, Ukrainian and Belarusian languages ​​thanks to its search technology. It is also worth noting that some of the key competitors of Yandex are completely entangled in their affairs.
Rambler, once confidently ranked second among Russian search engines, this year
announced a significant loss - and analysts explain this lack of concentration and the fact that Rambler allowed its competitors (Yandex and Google) to easily capture the market. Even Search.Mail.ru, which supports — guess who — Yandex, has now bypassed it.
Other aspects also give Yandex iron superiority in the Russian market. One of them is Yandex Money, which helps the development of the Russian e-commerce market, working as a bank for consumers' money and allowing them to buy on the Internet at a time when credit cards are not particularly popular and applicable, and the use of PayPal in Russia is a big inconvenience.
On top of all this, Yandex’s positions are strong in contextual advertising, it has its own PPC system, very similar to Google’s, and offers many of the most popular services that you can expect from a leader among search engines. In addition, Yandex recently launched an API that simplified access to its systems and also tools for researching keyword statistics.
Summing up, Yandex was the first search engine with a very deep connection with the Russian language and culture. He is in a good financial position and invests in his own future. Meanwhile, it is very difficult for competitors to maintain their positions. That is why Google has increased its share by 7 points worldwide, while Yandex scored its 0.3 points at home - in Russia and Eastern Europe.