What we should build Google
Today at the Google Developers Day event, I was able to talk with Vladimir Ofitserov , who flew to Moscow specifically for the event. Vladimir is a specialist in the Google Search Quality Department and I asked him a number of topical questions that were interesting to the users.
Let me remind you that since 1999, Vladimir Ofitserov worked at Inktomi , one of the first Internet search engines that became the basis of the Yahoo search engine. Vladimir participated in Yahoo and led projects that were aimed at improving the ranking algorithm, scanning and indexing infrastructure. In 2008, with a group of engineers from Yahoo, Vladimir moved to the Californian Yandex Labs, where he worked on projects aimed at improving search on the Russian Internet. Since 2010, he has been working in the Google Search Quality Department.
')
- Yahoo, Yandex, Google ... Vladimir, looking at such a sequence of places of your work, involuntarily begs the question: how is this possible? Is it really so easy to take and go from the first company with all its secrets (NDA) to the second, which is the main competitor of the first? Tell us about the reasons for the transition from one place to another.
- In 1999, one of my colleagues at Inktomi was Arkady Borkovsky , a friend of Arkady Volozh , with whom they organized Yandex in 1997. With Arkady I worked for a long time at Inktomi, later at Yahoo, and when Yandex decided to open an office, the choice of the head of this office naturally fell on Arkady Borkovsky. Since I worked with them for a long time, I just became one of the people who founded this office. And about the secrets - in California from the point of law, there are no problems with this.
- That is, in fact, poaching from one company, as such, was not?
- Exactly. Yahoo, as you know, stopped doing searches and completely switched to Bing. This trend was noticeable some time ago - the company did not invest in people, technology, or computers ... And, strictly speaking, the withdrawal from Yahoo just lasted long enough. Yandex is a very good company (in terms of management, decision-making, etc.), but when there is an opportunity to move from a company whose search is used by forty million people, to a company whose search is used by more than 500 million, the choice here remains obvious. Especially if there is a motivation to make the search really better.
- In Google, you have been working since January of this year. Compared to previous jobs, are there any fundamental differences? Is there no such thing that the new place lacks something that was at the previous places of work?
- The first thing you can see when you go to Google is that it is a global company. People are much easier to communicate among themselves, between offices, offices ... and communication is maintained very dynamic. Yandex, nevertheless, grew up in Russia and this is felt - it is more closed ... it has less experience and approaches to the organization of global business.
- Let's now talk a little about the technical side of the issue. Why did Google decide to switch to live search? And recently appeared site previews in search results.
- To begin with, about 30% of users do not see a live search at all - as a rule, those who type a query in the address bar of the browser. For the remaining average users, live search saves about two to three seconds. But we must not forget about people who immediately type the text with two fingers - for them “live search” saves 30 seconds and more. By pressing only a few letters, they are more likely to be able to select the request they need, which will no longer have to be printed. We decided that for many it can be a very big plus, and everyone else can always disable this feature.
- It seems to me that in most cases the user in general imagines what he is looking for - respectively, he simply does not need all the side effects. But is there, for example, such that in these “invisible results” money is spent by advertisers using Google AdWords?
- As for advertising, there is no problem with this - money starts to be spent only when a person has completed his request and there are mechanisms for determining this moment. On the contrary, in the case of live search advertising shows less.
- Does it happen that with such prediction of requests there will be an indirect manipulation of the person conducting the search? When he was just about to search for something, he was already obligingly offered what he “wants” to search for or something that someone has already paid for as “what to look for” (“everyone is looking for it!” Or “all consider it the answer! ”).
- The key factor in the fact that the search engine offers you to complete some kind of query is that we know that the result for it will be “good”. And if the user sees among the proposed options similar to the one he wants to find, then from the point of view of the search engine this is the ideal option to formulate a request, a goal. That is, if the user has difficulty with the formulation of the request, but in the list offered by the machine he sees something similar to what he wants to find, then he will find it, most likely, much faster.
- What can you say about social search? When does Google have a search creation plan that determines the relevance of issuance results based on the interaction and user assistance contrary to common search methods based on algorithms?
- I believe that such a search will appear in the very near future, next year or even this year. Moreover, it is already possible to specify your twitter-, facebook-account (and some other social networks) in Google-profile, after which Google will build a chain of your friends. And if one of them publishes some link in his blog or social network, and the information content of this link is similar to your search query, then you will get a kind of “friendly” result with the corresponding mark.
- One of the problems of the current search (any) is that it is looking not “deep down”, but “wide.” That is, for example, at the request of the “pen” in the results there will be a huge amount of a wide variety of pens - from writing to the door. But at the same time, “1,900,000 results were found at the request” - it's like asking someone, “What is a pen?” And getting an answer: “I know more than 1,900,000 answers to this question, you which one? That is, the entire instantaneous gain is lost against the background of information heterogeneity. With this, you can somehow fight?
- At the moment, the car really cannot predict at such a level what exactly you are looking for and mean, especially if it concerns such homonyms. However, for such situations, we try to display information about several values. Well, technology, as we see, does not stand still - I think nothing is impossible.
- What other problems exist in the modern search? What does not suit you personally at the moment and what would you like to fix as soon as possible?
- There are a lot of problems, but one of the most serious, I think, is search engine spam. At least for the Russian market this is one of the key factors and we will try to filter this information.
Another trend that has been visible for several years is the provision of information without having to press a large number of keys. For example, one of the steps to solving this problem was the recently introduced voice search.
- How do you see the search, let's say, in 5 years? After 10? Now they talk a lot about augmented reality ...?
" If I knew ... ", - said Vladimir and laughed. At this time, a hint from a colleague whispered from a colleague: “ I would have invested then! "
Perhaps the search will be even closer to the user. Personalization, localization, regionalization, socialization - I think all these aspects will significantly change the current search mechanism. For the reason that there is a lot of information in this data - no one will say more for you than your friends.
- Accordingly, the recent conflict with Facebook is actually much more significant than it might seem at first glance?
- We always take our competitors very seriously and, I think, do the same. But between search and social networks there are so many points of contact, thanks to which the work of these services can be done even better. The quality of our output could be much higher if we knew more about the likes that Facebook had placed throughout the internet.
- So now, in essence, search progress is hindering just this competition with Facebook?
- Exactly. Hopefully, sooner or later these barriers will become much lower.
- Vladimir, in more than 10 years of “search” work, you probably have a lot of interesting or unusual stories? Tell something from a particularly memorable, so to speak, couple of tales to laugh and a couple of rakes to cry.
- The year in 2002 was a funny case when I was working on a runtime system (a system that should respond to requests for half a second), I found a search query in its logs that was processed for more than a minute, I decided to look into. It turned out that the user entered the search string “i am alone in valentine's day” with a space after each letter (“I amalone I nvalent I ne 'sday”). As a result, the search engine was presented with almost an entire alphabet, where each letter contained hundreds and thousands of documents — the system struggled to combine them and produce the most appropriate result. [ note: I just tried to enter a similar query in Yandex - indeed, the batthurt, if not for a minute; google ok ]
I have here prepared a list of several amusing search questions ... sometimes it is fun to read.
- About titanium scrap in the train toilet? :)
- No, this is already a classic :) Here is something fresh. It is not clear what users are guided by when looking for this. Well, for example, “ what bangs fit me? ".
As for the rake ... it was also full of everything. But from the point of view of the programmer, the most terrible rake - to free the memory that you do not possess. It will not lead to anything good.
- Yeah, interesting. And what can you advise those who, for example, want to follow in your footsteps? Maybe you can advise some of the most interesting books ... that is, not just Google for Dummies, but something more serious, really worthwhile. Or is there nothing interesting in print and the most delicious things to look for in the vast expanses of the network?
- Walking in my footsteps is not at all necessary - everyone should choose their own path, really interesting. As far as information is concerned, I can advise two interesting books: “ Introduction to Information Retrieval ” [Christopher D. Manning, Prabhakar] is a more academic book written by a professor from Stanford. So to speak, for the basics. And “ Search Engines: Information Retrieval in Practice ” [Bruce Croft, Donald Metzler, and Trevor Stohman] is already a practical book written by Google engineers. It is perfect for those who, for example, want to make their own search engine - it tells about the work of many mechanisms, about writing effective code and many other useful things. And, of course, the Internet - if you wish, you can always find interesting, and most importantly, relevant information.
Thank! And finally, maybe share some secret? Anything out of the ordinary, exclusively for readers of our site?
- Well, they are secrets to secrets :) However, something periodically opens - for example, a recently opened programming language for the public, designed to handle VERY BIG volumes (more than an index!) Logs. It was written by one Russian (I’m not afraid to say this word, a scientist) and was our “know-how” for a long time, but about a week ago this information became public; If you wish, you can google. Well, or the same Closure and GWT for developing rich enough web applications in Java (which are then compiled into JavaScript and packaged so that they are optimal for compiling, loading, and execution time in browsers) - all this is made available to Google by developers.
We have Brad Fitzpatrick, the creator of LiveJournal - in fact, he is an “ordinary” programmer and, in his time, besides LJ, wrote several things for Google (mostly methods and classes) that are still relevant — now they literally work All and similar examples can still lead quite a lot. I can say simply - Google is the company that makes religion from engineering, creating projects "forever." Most often, these are much more complicated mechanisms (than those of competitors), all the subtleties of the work of which cannot be taken and opened.
At this point, our meeting came to an end, I took some photos for memory and went to the event itself - I was pleasantly surprised by a large number of people registered at Habré.
Successes!
Let me remind you that since 1999, Vladimir Ofitserov worked at Inktomi , one of the first Internet search engines that became the basis of the Yahoo search engine. Vladimir participated in Yahoo and led projects that were aimed at improving the ranking algorithm, scanning and indexing infrastructure. In 2008, with a group of engineers from Yahoo, Vladimir moved to the Californian Yandex Labs, where he worked on projects aimed at improving search on the Russian Internet. Since 2010, he has been working in the Google Search Quality Department.
')
- Yahoo, Yandex, Google ... Vladimir, looking at such a sequence of places of your work, involuntarily begs the question: how is this possible? Is it really so easy to take and go from the first company with all its secrets (NDA) to the second, which is the main competitor of the first? Tell us about the reasons for the transition from one place to another.
- In 1999, one of my colleagues at Inktomi was Arkady Borkovsky , a friend of Arkady Volozh , with whom they organized Yandex in 1997. With Arkady I worked for a long time at Inktomi, later at Yahoo, and when Yandex decided to open an office, the choice of the head of this office naturally fell on Arkady Borkovsky. Since I worked with them for a long time, I just became one of the people who founded this office. And about the secrets - in California from the point of law, there are no problems with this.
- That is, in fact, poaching from one company, as such, was not?
- Exactly. Yahoo, as you know, stopped doing searches and completely switched to Bing. This trend was noticeable some time ago - the company did not invest in people, technology, or computers ... And, strictly speaking, the withdrawal from Yahoo just lasted long enough. Yandex is a very good company (in terms of management, decision-making, etc.), but when there is an opportunity to move from a company whose search is used by forty million people, to a company whose search is used by more than 500 million, the choice here remains obvious. Especially if there is a motivation to make the search really better.
- In Google, you have been working since January of this year. Compared to previous jobs, are there any fundamental differences? Is there no such thing that the new place lacks something that was at the previous places of work?
- The first thing you can see when you go to Google is that it is a global company. People are much easier to communicate among themselves, between offices, offices ... and communication is maintained very dynamic. Yandex, nevertheless, grew up in Russia and this is felt - it is more closed ... it has less experience and approaches to the organization of global business.
- Let's now talk a little about the technical side of the issue. Why did Google decide to switch to live search? And recently appeared site previews in search results.
- To begin with, about 30% of users do not see a live search at all - as a rule, those who type a query in the address bar of the browser. For the remaining average users, live search saves about two to three seconds. But we must not forget about people who immediately type the text with two fingers - for them “live search” saves 30 seconds and more. By pressing only a few letters, they are more likely to be able to select the request they need, which will no longer have to be printed. We decided that for many it can be a very big plus, and everyone else can always disable this feature.
- It seems to me that in most cases the user in general imagines what he is looking for - respectively, he simply does not need all the side effects. But is there, for example, such that in these “invisible results” money is spent by advertisers using Google AdWords?
- As for advertising, there is no problem with this - money starts to be spent only when a person has completed his request and there are mechanisms for determining this moment. On the contrary, in the case of live search advertising shows less.
- Does it happen that with such prediction of requests there will be an indirect manipulation of the person conducting the search? When he was just about to search for something, he was already obligingly offered what he “wants” to search for or something that someone has already paid for as “what to look for” (“everyone is looking for it!” Or “all consider it the answer! ”).
- The key factor in the fact that the search engine offers you to complete some kind of query is that we know that the result for it will be “good”. And if the user sees among the proposed options similar to the one he wants to find, then from the point of view of the search engine this is the ideal option to formulate a request, a goal. That is, if the user has difficulty with the formulation of the request, but in the list offered by the machine he sees something similar to what he wants to find, then he will find it, most likely, much faster.
- What can you say about social search? When does Google have a search creation plan that determines the relevance of issuance results based on the interaction and user assistance contrary to common search methods based on algorithms?
- I believe that such a search will appear in the very near future, next year or even this year. Moreover, it is already possible to specify your twitter-, facebook-account (and some other social networks) in Google-profile, after which Google will build a chain of your friends. And if one of them publishes some link in his blog or social network, and the information content of this link is similar to your search query, then you will get a kind of “friendly” result with the corresponding mark.
- One of the problems of the current search (any) is that it is looking not “deep down”, but “wide.” That is, for example, at the request of the “pen” in the results there will be a huge amount of a wide variety of pens - from writing to the door. But at the same time, “1,900,000 results were found at the request” - it's like asking someone, “What is a pen?” And getting an answer: “I know more than 1,900,000 answers to this question, you which one? That is, the entire instantaneous gain is lost against the background of information heterogeneity. With this, you can somehow fight?
- At the moment, the car really cannot predict at such a level what exactly you are looking for and mean, especially if it concerns such homonyms. However, for such situations, we try to display information about several values. Well, technology, as we see, does not stand still - I think nothing is impossible.
- What other problems exist in the modern search? What does not suit you personally at the moment and what would you like to fix as soon as possible?
- There are a lot of problems, but one of the most serious, I think, is search engine spam. At least for the Russian market this is one of the key factors and we will try to filter this information.
Another trend that has been visible for several years is the provision of information without having to press a large number of keys. For example, one of the steps to solving this problem was the recently introduced voice search.
- How do you see the search, let's say, in 5 years? After 10? Now they talk a lot about augmented reality ...?
" If I knew ... ", - said Vladimir and laughed. At this time, a hint from a colleague whispered from a colleague: “ I would have invested then! "
Perhaps the search will be even closer to the user. Personalization, localization, regionalization, socialization - I think all these aspects will significantly change the current search mechanism. For the reason that there is a lot of information in this data - no one will say more for you than your friends.
- Accordingly, the recent conflict with Facebook is actually much more significant than it might seem at first glance?
- We always take our competitors very seriously and, I think, do the same. But between search and social networks there are so many points of contact, thanks to which the work of these services can be done even better. The quality of our output could be much higher if we knew more about the likes that Facebook had placed throughout the internet.
- So now, in essence, search progress is hindering just this competition with Facebook?
- Exactly. Hopefully, sooner or later these barriers will become much lower.
- Vladimir, in more than 10 years of “search” work, you probably have a lot of interesting or unusual stories? Tell something from a particularly memorable, so to speak, couple of tales to laugh and a couple of rakes to cry.
- The year in 2002 was a funny case when I was working on a runtime system (a system that should respond to requests for half a second), I found a search query in its logs that was processed for more than a minute, I decided to look into. It turned out that the user entered the search string “i am alone in valentine's day” with a space after each letter (“I amalone I nvalent I ne 'sday”). As a result, the search engine was presented with almost an entire alphabet, where each letter contained hundreds and thousands of documents — the system struggled to combine them and produce the most appropriate result. [ note: I just tried to enter a similar query in Yandex - indeed, the batthurt, if not for a minute; google ok ]
I have here prepared a list of several amusing search questions ... sometimes it is fun to read.
- About titanium scrap in the train toilet? :)
- No, this is already a classic :) Here is something fresh. It is not clear what users are guided by when looking for this. Well, for example, “ what bangs fit me? ".
As for the rake ... it was also full of everything. But from the point of view of the programmer, the most terrible rake - to free the memory that you do not possess. It will not lead to anything good.
- Yeah, interesting. And what can you advise those who, for example, want to follow in your footsteps? Maybe you can advise some of the most interesting books ... that is, not just Google for Dummies, but something more serious, really worthwhile. Or is there nothing interesting in print and the most delicious things to look for in the vast expanses of the network?
- Walking in my footsteps is not at all necessary - everyone should choose their own path, really interesting. As far as information is concerned, I can advise two interesting books: “ Introduction to Information Retrieval ” [Christopher D. Manning, Prabhakar] is a more academic book written by a professor from Stanford. So to speak, for the basics. And “ Search Engines: Information Retrieval in Practice ” [Bruce Croft, Donald Metzler, and Trevor Stohman] is already a practical book written by Google engineers. It is perfect for those who, for example, want to make their own search engine - it tells about the work of many mechanisms, about writing effective code and many other useful things. And, of course, the Internet - if you wish, you can always find interesting, and most importantly, relevant information.
Thank! And finally, maybe share some secret? Anything out of the ordinary, exclusively for readers of our site?
- Well, they are secrets to secrets :) However, something periodically opens - for example, a recently opened programming language for the public, designed to handle VERY BIG volumes (more than an index!) Logs. It was written by one Russian (I’m not afraid to say this word, a scientist) and was our “know-how” for a long time, but about a week ago this information became public; If you wish, you can google. Well, or the same Closure and GWT for developing rich enough web applications in Java (which are then compiled into JavaScript and packaged so that they are optimal for compiling, loading, and execution time in browsers) - all this is made available to Google by developers.
We have Brad Fitzpatrick, the creator of LiveJournal - in fact, he is an “ordinary” programmer and, in his time, besides LJ, wrote several things for Google (mostly methods and classes) that are still relevant — now they literally work All and similar examples can still lead quite a lot. I can say simply - Google is the company that makes religion from engineering, creating projects "forever." Most often, these are much more complicated mechanisms (than those of competitors), all the subtleties of the work of which cannot be taken and opened.
At this point, our meeting came to an end, I took some photos for memory and went to the event itself - I was pleasantly surprised by a large number of people registered at Habré.
Successes!
Source: https://habr.com/ru/post/108050/
All Articles