Russian Internet Week (RIW): Transcript of the round table “Recommender Systems - New Runet Trend: Problems and Benefits”
On the initiative of Imhonet, at the Russian Internet Week RIW (Crocus Expo, October 22-26), a round table discussion “Recommender services - a new trend of the Runet: problems and benefits” was held.
The trend of recommendation services spreads so fast that even in the questionnaires filled in when applying for work in Internet companies, the “possession of recommendation systems” column appeared. As often happens, popularity and demand lead to confusion. A typical situation is when a simple voting box is put on the site and the ratings are published, calling it the fashionable word “recommendation service”. Although it is only indirectly related to the case. Therefore, the following issues were discussed during the discussion:
- criteria to distinguish recommendation services in the narrow sense of the word from systems that give all sorts of hints and tips;
- various types of recommendation systems: for what purposes they are better suited;
- the value of rev.services for users and professional players of the Internet market;
- Ways of issuing recommendations: automatic generation of forecasts and fine, “manual” filters and adjustments, which make it possible to specify the recommendation in as much detail as possible;
- economic effects resulting from the installation of the recommendatory functionality on the site. In particular, improved navigation and, as a result, increased user loyalty, as well as additional opportunities for the monetization of resources. (The first experience of exporting the recommendation functional of Imhonet to third-party platforms shows that the merchantability of goods grows by 18 percent, the number of pages viewed is 24.) And so on.
The round table was attended by people standing in different professional positions ... Manufacturers of various advisory services: Imkhonet - service manager Alexander Dolgin, Afisha - project manager Elena Kuznetsova, working advisory service Guru.ru - general director Alexander Pyatigorsky; content access operators (AKADO - head of the service development department Ivan Volchenskov); experts (Alexander Sergeev - scientific editor of "Around the World").
As a result, it was possible to look at the problem in a rather voluminous manner - for details, see the transcript of the round table.
Leading:
Elena Lebedeva , Imkoneta PR Director
')
Speakers:
Alexander Dolgin, manager of Imkhonet, professor of the State University Higher School of Economics
Elena Kuznetsova , Afisha Project Manager
Alexander Pyatigorsky , general manager, gooroo.ru
Ivan Volchenskov , Head of AKADO Services Development Department
Alexander Sergeev , scientific editor of "Around the World"
Host : Recently, there has been a real fashion for recommendatory services. In connection with this, they just do not assign this big name. Even the simplest "voting", installed on the site and simply displaying the rating, is proudly called advisory service. Where does this hype come from? Recommended services improve navigation, which is extremely important in a situation of rapidly increasing volumes of content. Installing the recommender functionality leads to quite tangible economic effects: according to Imhonet’s experience, the number of page views increases by 24 percent, sales by 18. Plus, the recommender systems create prerequisites for a number of non-trivial monetization schemes (for example, highlight unrealized demand), etc. All this taken together and warms up the fashion on the recommendations.
Meanwhile, recommender systems are completely different. I give the floor to Alexander Dolgin, whom I ask to begin with a review of recommendatory services, their specifics, for which resource which advisory service is optimal.
Alexander Dolgin : I don’t think it makes sense to dwell in detail on why advisory services are generally needed. The level of noise in the hall, where we are now trying to establish a meaningful conversation, is an excellent illustration of my speech, because these interferences are nothing compared to the noisy information and product environment in which we all live.
As the presenter correctly noted, the terms “recommendatory service” and “recommendations” are now understood as completely different things. A recommendation in the most general sense is simply an indication that an object has some added value. This indication may be presented in different ways. For example, timeshare sellers who flourished 10 years ago, in a certain sense, made recommendations: they said that there is a good service that allows you to do this, this and that. Ratings, charts, billboards also perform an advisory function, informing that, on average, attracted a large number of people. Professional publications, publishing articles by critics and experts, reports about contests and festivals are also a kind of recommendation. Brand tips, advertisers and sellers are another form of recommendation. Even simple conversations in the circle of relatives and friends, exchanging their opinions, fall under the neck recommendation. And all these different types of recommendations today have a platform on the Internet. And each site, where one person to another tells something in streaming mode, claims the status of a recommendation.
I offer you a number of criteria with which you can structure all this recommendation space.
The first principle is the degree of automation in issuing recommendations. There are two poles here. On the one hand, the “manual method”: that is, the consumer reads a stream of texts and selects from there what he is interested in. The other pole is absolutely automated recommendations, when with one click you can get some kind of specialized information sample. In a sense, search engines can also be referred to as recommender systems that give answers to queries, setting priority in accordance with the reputation of the source of information. There is also a number of intermediate in terms of the degree of automation of technological solutions.
Imhonet's universal recommender system www.imhonet.ru , which I represent, deals with automated recommendations, that is, fast, highly accurate, not condemning the user to the “excavations” in the streaming tape. This does not mean that everything is limited to this level of recommendations - in addition to the automatically generated forecasts, there are many more entry points on the site, which make it possible to refine the recommendation. As a metaphor, I will give an image of a traveler who travels to another continent by plane, from the airport to the hotel - by taxi, and the last meters pass by foot. Because to immediately go to another continent on foot for a long time, it is also difficult to seek out the necessary information in the stream. It means that it is necessary to divide the search into several stages, using various methods (for example, by a landmark work or through profiles of critics with coinciding tastes) and seeking an increasingly targeted sample.
The second principle by which recommendation services can be classified is whose interests they represent. Here, too, two options are possible - systems that work for a manufacturer, agent, seller, promoter, and services that guard the interests of the consumer, helping him to make unprogrammable, unbound choices. In the first case, the system helps to sell. Historically, most of the recommendation services arose as a division of sellers, because it was the only opportunity for them to monetize their services and make money. The accuracy of the recommendations issued here is low, which, incidentally, corresponds to the objectives of the seller. For example, when you get this kind of information in the famous “Amazon” recommendation system: “This is the book that a person bought who bought the same book as you bought” - its goal is not to give you an accurate forecast, but to orient in some direction . The system does not know why the person bought this book. For yourself or someone else. The main thing is that he doesn’t know if he liked the purchase. It creates a rich field for omissions, beneficial for distribution.
Finally, the third principle of the classification of recommender systems is that based on which signals the prediction is made: direct or indirect. For example, a person says to his friend and like-minded person: “I watched this film and I liked it, go and see.” This is a direct signal from the consumer to the consumer. And it can be formulated differently: 10 thousand people watched a certain film - according to the statistics of visits it turns out to be “the best” - go see it too. In the second case, the consumer is oriented only on the actions of his predecessors, but not on the result that they actually received. This is the basis for the promotion of technology in the film industry, when it is important to ensure a high box office of the first weekend, because it automatically starts the so-called non-informative information cascade. And why do people go to the movies? Just because the skirmishers did it.
There are many indirect methods for calculating information about intentions. The problem is that they are all inaccurate. Today, the Internet is constantly collecting and analyzing information about what the user has paid attention to: what he flipped through, on what and how much he held his gaze, etc. Based on these statistics, conclusions are drawn about his true intentions. However, the gap between what the person intended to purchase, for which he eventually paid, and the result that he eventually received is very large. This creates a field for significant errors and inefficient choice.
The advisory service imhonet.ru , which I am developing, calculates the forecast solely on the basis of direct post-factum signals emanating from the consumers themselves, who exchange views on how much they really liked this or that work. In my opinion, this is the most relevant, reliable and useful information. Its quality is determined by how many people participate in the system - if there are few of them, then there is little field for sharing experiences. This opens the way for effective affiliate programs between different sites specializing in different areas, because it is rather difficult to set up such a system within one small site: there are not enough people and their ratings. Collectively make it easier and more efficient.
So, summing up, we can say that the quality of the recommendation service is determined, firstly, by the degree of automation of issuing recommendations, secondly, whose interests it serves (trade or consumer), and, thirdly, which signals it relies on (direct or indirect).
Without going deep into the difficult area of actually calculating recommendations (there are a number of methods), I’ll focus on two of them. The first is based on user clustering. For example, we can assume and statistically reveal that buyers of a particular toothbrush prefer such-and-such toothpaste. Or that people with average incomes give their children to certain kindergartens and go to work by bicycle (as it happens in Holland). Based on this, we are grouping people into clusters, and we advise those who have not yet bought a bike: buy it, because everyone like you have already done so.
The second method is called the collaborative, from the English. collaborative - “collaboration” (it is he who is at the heart of Imonet). In this case, people are chosen not on the basis of abstract assumptions about how it should be, but on the basis of clear facts that a person has consumed and how he has estimated a consumer product.
Question from the audience: The information you collect on your user profile is personal data, the protection of which is guaranteed by law. How open are these data on your website and what are the development prospects in this particular direction?
Alexander Dolgin: All rights to manage personal data are in the hands of the user. He can publish what he likes, what he likes, with whom he is friends, and can turn on the mode of anonymity, privacy.
Host: Another company participating in our round table has introduced a collaborative advisory service - this is the “Poster”. I give the floor to Elena Kuznetsova and ask her to elaborate on the specifics of their service.
Elena Kuznetsova: I would like to tell you how we apply various methods of recommendations in practice. “Poster” is a site about entertainment, one of its main goals is to help the user in choosing the way of spending his leisure time. And the recommendations here are very necessary. We offer the user recommendations of different levels. First, we provide complete information about what opportunities there are for entertainment in the city where he lives: the schedule of cinema, theaters, sporting events, a television program, and so on. Secondly, we publish online catalogs of films, TV shows, books, establishments that can be visited (restaurants, clubs, theaters, etc.). Since the amount of information is very large, we provide the user with quite good, in our opinion, navigation and search tools.
As for the recommendations directly, they are multi-layered.
First of all, our editorial board chooses the most important events for each topic every day - these are exclusively manual recommendations. The second level is the reviews of respected critics who have earned the trust of readers and users. Thirdly, these are reviews and ratings of the users themselves: every month we get about 300 thousand ratings and about 3 thousand reviews of objects (movies, restaurants, etc.). Then there are personalized selection tools - in particular, reviews and assessments of the user's friends. This is not just a review tape, but the opinions of only those people whom the user trusts and wants to see as their referees. And finally, the last - the collaborative method of recommendations "Kinobot". It is based on the Slope One algorithm and so far only works for movies. Briefly formulate its principle, the system recommends you to movies that you have not yet rated, but which users like, who liked the same movies as you.
In addition, “Kinorobot” takes into account the similarity of the genre of the recommended film with the genres of those films that you have already rated, and also takes into account the novelty of the object and the weighted average rating. That is, bad films, we do not recommend, in principle, even if somewhere, something coincided.
Host: A question for clarification: who decides whether a film is good or bad? Revision?
Elena Kuznetsova: No, this is determined by the weighted average rating. If it is below a certain value, then this film will not fall into the recommendations.
Alexander Dolgin: Do you get Fellini films good or bad?
Elena Kuznetsova: Good.
Host: They have a high rating?
Elena Kuznetsova : Enough to get into the recommendations.
Question from the audience: When issuing recommendations, who is given preference - the opinions of experts, friends, or the opinion of an unknown majority? Because it’s one thing when you are advised by a person who knows you personally and with whom your tastes roughly coincide, and the other is a generalized, weighted average recommender.
Elena Kuznetsova: Now on the site, automated recommendations are issued without taking into account the assessments of individual critics and individual assessments of friends. Although we have a small project where we generate recommendations for several people at the same time: for example, what might you and your girlfriend like where you could go together.
I would like to talk about combining various methods of issuing recommendations - automated and non-automated.
The table below shows which objects work best for which objects. According to our observations, collaborative recommendations work very well for objects that do not change over time — old films, books, CDs, etc. Reviews and assessments for them also work well, but since there are so many works, reading the reviews turns into a waste of time. The editorial choice in this case also does not work very well, because it is simply impossible to write reviews of all objects.
However, with regard to new, newly emerging works, we don’t really understand how here you can use collaborative recommendations, because there are either no estimates for a work yet or there are too few of them for the algorithm to be effective. In the case of new products, in our opinion, editorial choice and critics' reviews work much better. It is necessary to take into account that among our audience there is a large percentage of innovators, and for them it is important to receive recommendations specifically for new works.
The situation is even more complicated when it comes to events. We assume that there are some approaches and methods that could automate such recommendations. But it should be borne in mind that the event exists either in the future tense, when there are no assessments for it, or in the past, when it is already useless to recommend it. Therefore, it is not very clear how collaborative recommendations can work in this case.
Alexander Dolgin also mentioned places, places, restaurants, hotels. , : -, , . , , , . , , , , .
, : ? ? , : . , , .
: , , . ? , ? ?
:I'll start with the last question. There is an elegant solution that is used throughout the world. The recommendation service has a large rating base (for example, 100 million). Some part of this database, say, 20 million, is closed, and on the basis of the remaining data, the service tries to calculate these estimates unknown to it at the time of the experiment. And then the calculated estimates are compared with the real ones. In particular, the Netflix cinema advisory service announced a contest with a prize fund of $ 1 million in search of those who can improve their recommendation algorithm. Check there was conducted in this way. In principle, Elena mentioned this particular experience: everyone compares the predicted recommendations with their own opinion and decides whether the system guessed or not. But in isolated cases deviations are possible. And with a large statistics of errors does not happen.
, : ; , , ; , – .
: , , . . .
: , . , – , , , , . Which ones?
Ivan Volchenskov: Using the example of AKADO, I would like to talk about the need for recommender systems for access operators. The operator AKADO provides three basic access services: Internet, digital television and telephony.
All access operators like us seek to fill their services with additional content and services. As it seems to us, in the near future the struggle for the consumer will go not only at the level of attractive prices, but on the basis of quantity, quality and convenience of additional services in each of these three areas.
We began to develop additional services about 2.5 years ago. As a result, we offer a lot of services where our subscribers are faced with the problem of choice.
For example, a service for selling music: we now have 200,000 tracks, by the New Year this figure will reach 1 million, and our users will find it rather difficult to navigate in this information space. According to the existing statistics (for example, regarding such large operators as “America Online”), 90 percent of the tracks offered on the resource have never been downloaded. The same problem with games: there is a huge amount of them, they are expensive and the price of a mistake with the wrong choice is very high. A person wants to know in advance which game to buy. The same thing happens with software, books, etc. In the near future, we are launching the Video To Order service, where our subscribers will need to choose which movie to watch. We are trying to help our users and are going to solve this problem with the help of a recommendation system,which will facilitate the consumption of goods and services on our resources. In our opinion, this will lead to higher sales, as users will be satisfied with the purchase that is relevant to their tastes. Accordingly, customer loyalty will increase, which is another good result.
How are we going to implement this program? Initially, we thought about developing our own recommendation system, but we realized that it was a rather painstaking job. A year ago, we met with representatives of the Imhonet company, conducted our research and realized that the best solution for us is an affiliate program with Imhonet to implement recommendations for third-party resources. Our research has shown that this will be quite simple and will entail a lot of advantages. First of all - “hot start”. Here it has already been said that it is quite difficult to sell new products and goods for which there are no ready-made recommendations. For us Imkhonet is good because these recommendations already exist there. For example, a program guide.
We offer digital television with over 150 channels. But what exactly to watch? Users have a problem with choosing a program guide, it is followed by disappointment in digital television, and people return to Channel One, RTR, NTV and watch the advertised programs.
Therefore, we decided to develop the direction of "personal television", that is, to form a personal program guide for a particular subscriber, based on the channels available to us. Special software will offer the user, who settled in front of the TV, which program to watch now and which one to watch next. And we expect Imkhonet to help us with this thanks to his recommendation system, in which the television rating base has already been accumulated.
Leading:Here there are the most effective, live reference services in different niches - Afisha, Imkhonet, Guru - with their own specifics. But they are all manufacturers of recommendations. As you know, any specialist is like a flux, its completeness is incomplete. There is a person standing in a completely different position - Alexander Sergeev, a journalist, scientific editor of the magazine "Around the World", the author, in my opinion, one of the most fundamental and serious review articles on advisory services.
Alexander Sergeev: I will not speak as the creator of recommendatory services, but as a user, and will try to explain what is a recommendatory system for me and what I expect from such systems.
, , , . , , . . , - , – , .
The first is independence, that is, the absence of a conflict of interest between those who recommend and recommend. The second is collaborativeness, because a system built not on collaborative ideas, but, for example, on expert data, is a completely different thing. Expert advice has always existed; people have been using it for a long time - for example, legal advice. And finally, the third criterion is personalization, that is, the recommendation must be individually tailored to the specific consumer.
I will dwell on these points in more detail. Take advertising: this is also a kind of recommendation - to buy something, to go somewhere, etc. But this is not a recommendation service for one simple reason: the ultimate goal of advertising conflicts with the interests of the client. She wants to sell him something, and the recommendation service is always entirely on the consumer’s side. Therefore, for example, different systems that exist as an additional service to online bookstores, in my opinion of the user, are not advisory services. It is rather a certain type of contextual advertising and a system for the sale of related products ...
Do not confuse advisory service with advice. Consultations are good when they are given by specialists in a narrow field and, because of the highly qualified expert, are expensive. And the choice of a television program or a book is a small decision, for which the consulting approach is not suitable, because it is too expensive. If it is cheap, then, as a result, poor quality. That is why for me the advisory service is only that system, which is built on the analysis of the opinions of my fellow users, and a large number of them. From this point of view, such a wonderful system as “Yandex. Guru ”is not a recommendation: it is an expert system that evaluates and systematizes the opinions of several dozen experts, each of whom describes his or her area and advises what you should pay attention to.
And finally, the third aspect is personalization. It is obvious that any ratings and statistics are not recommendations, since they are the same for everyone. If you give a negative example, then I have a serious problem with the wonderful service “Yandex. News "and his ilk. What are they doing?Of the hundreds of news that appear daily, such systems distinguish 5 and say that they are the most important - for the whole country, for the whole world. But this is absurd - I proceed from my priorities and they can be completely different. I personally do not understand why there is still no advisory service on the news. Every day, as a journalist, I spend 2-3 hours on news, and as a private user, I spend half an hour. And no one helps me to optimize these half an hour, so that within my personal interests I read five news that is most important to me personally.
Now I would like to touch on the implementation of recommendation services. When the recommendation service is created as an independent body - like Imkhet, everything is clear. But when this functionality exists on the application site of some other direction, the question arises: why is it inserted there? Either it is used from tactical considerations, as a sales tool, or from strategic ones, in order to achieve customer loyalty.
, , , . , . , , – , . , . , : , , , - «», . , , - . . , . . , , . , - : « , , - ? , ». , , , , - . – .
The question is how to solve this problem correctly ?! How to make a deal with conscience when commercial and strategic interests overlap? In order to avoid flagrant distortions, the right people should make the decision - the company's management, who sees the whole situation as a whole. It may sometimes be worth a little sacrifice of strategic trust in order to stay afloat. But if this question is not brought up to the level of top management, then such decisions are made at the level of conflicts between departments, which should deal with purely technical solutions. And it all comes down to who won whom.
One of the best solutions to the problem, I consider the removal of the recommendatory division (as untargeted and not the main one for business) outside, turning it into an outsourcing division. Then all the negotiations with him will guide and decide whether the system provides impartial recommendations or use it as a specific mixing tool to achieve the desired results. But it will already be a clear business decision, and not a compromise between developers.
The distance serves as a guarantee of the correctness of the recommendation system. After all, when a company specializes in issuing recommendations, when it is its main activity, it is interested in the quality of the forecast, otherwise its business will fall apart.
I drew to you a certain ideal of a recommendation service and showed how it is possible to deviate from this ideal.
Host: Let me draw your attention to the pragmatic side of the issue. Having launched the Imkhonet affiliate program, we were convinced that even an honest advisory engine that does not convoys to the counter gives a visible economic effect. Introducing the recommendation functionality of Imkhonet on other sites, we retain the same equidistance from manufacturers and suppliers that Alexander Sergeev spoke about. At the same time, the first implementation experience shows that the listability of pages (the number of views) on third-party sites increases by 24 percent, and the merchantability of content increases by 18. Without any mixing and deceit. Just by improving navigation and optimizing choices.
Ivan Volchenskov:I have a comment. Alexander Sergeev noted that there is a discrepancy between the increase in sales and the increase in loyalty; one must choose either one or the other. I told in my report how we combine these two goals - increasing sales and loyalty. If you purchase a rather expensive product that you have been recommended and which turned out to be relevant to your expectations, will you not increase your loyalty to the service that supplied you with this recommendation? As such, there is no brutal conflict here. When it comes to products that need to be sold (roughly speaking, because there are large deductions to copyright holders and you need to recoup money), it is advisable to do it just with honest advertising: context, media, whatever. But this does not in any way cancel recommendations as a tool to increase sales and increase loyalty.Moreover, the recommendations are especially relevant in those products where taste matters play a decisive role and the opinion of the target audience is important.
Question from the audience: Today, the word “recommendations” sounded a lot, or rather, “rating”, assessments, something else. What is this, in your opinion? How many times does the user click the plus button? Or is it a rather difficult computed index made up of many components?
Alexander Dolgin: A recommendation is a forecast of how much a person will like a movie (a book, any other object) with which he is not yet familiar. The quality of this forecast is verified in practice. If the service predicts: “you will like it at 9,” and you looked, and it turned out to be a “two” - then a bad recommendation. A recommendation is some predictive statement about the future that nobody seems to know, but we have learned to count it.
Leading : : «» «». – . – , , , . , . , .
: , ? , , , ?
:Very good question. I asked him myself several times. I personally believe that in some areas expert judgment is more valuable than evaluation by a large community. For example, I believe that if a service is made on news, then it should be built solely on mass assessments. And if a sub-service on the news of science is made in it, then expert assessments must be mixed into it. This is necessary, of course. But first of all you need to understand that there is a recommendatory service in understanding our terminology and there are various other types of recommendations, for example, consulting. So, the inclusion of an element of consulting in a recommendation service, in my opinion, is possible. But this should be negotiated separately and separately. This is generally a topic for a special conversation - is it possible to make this relevant, and if so, how.
Question: I would like more details on the terms of the affiliate program.
Moderator: Quite briefly: Quite briefly: Imkhonet recommendation service has developed an export version of the recommendatory functionality, which provides other resources free of charge. At the same time Imkhonet solves two problems for partners at once: 1. provides the recommendation system itself (and also helps to establish, adapt, and maintain it later) 2. helps to overcome the “cold start” stage.
The trend of recommendation services spreads so fast that even in the questionnaires filled in when applying for work in Internet companies, the “possession of recommendation systems” column appeared. As often happens, popularity and demand lead to confusion. A typical situation is when a simple voting box is put on the site and the ratings are published, calling it the fashionable word “recommendation service”. Although it is only indirectly related to the case. Therefore, the following issues were discussed during the discussion:
- criteria to distinguish recommendation services in the narrow sense of the word from systems that give all sorts of hints and tips;
- various types of recommendation systems: for what purposes they are better suited;
- the value of rev.services for users and professional players of the Internet market;
- Ways of issuing recommendations: automatic generation of forecasts and fine, “manual” filters and adjustments, which make it possible to specify the recommendation in as much detail as possible;
- economic effects resulting from the installation of the recommendatory functionality on the site. In particular, improved navigation and, as a result, increased user loyalty, as well as additional opportunities for the monetization of resources. (The first experience of exporting the recommendation functional of Imhonet to third-party platforms shows that the merchantability of goods grows by 18 percent, the number of pages viewed is 24.) And so on.
The round table was attended by people standing in different professional positions ... Manufacturers of various advisory services: Imkhonet - service manager Alexander Dolgin, Afisha - project manager Elena Kuznetsova, working advisory service Guru.ru - general director Alexander Pyatigorsky; content access operators (AKADO - head of the service development department Ivan Volchenskov); experts (Alexander Sergeev - scientific editor of "Around the World").
As a result, it was possible to look at the problem in a rather voluminous manner - for details, see the transcript of the round table.
Leading:
Elena Lebedeva , Imkoneta PR Director
')
Speakers:
Alexander Dolgin, manager of Imkhonet, professor of the State University Higher School of Economics
Elena Kuznetsova , Afisha Project Manager
Alexander Pyatigorsky , general manager, gooroo.ru
Ivan Volchenskov , Head of AKADO Services Development Department
Alexander Sergeev , scientific editor of "Around the World"
Host : Recently, there has been a real fashion for recommendatory services. In connection with this, they just do not assign this big name. Even the simplest "voting", installed on the site and simply displaying the rating, is proudly called advisory service. Where does this hype come from? Recommended services improve navigation, which is extremely important in a situation of rapidly increasing volumes of content. Installing the recommender functionality leads to quite tangible economic effects: according to Imhonet’s experience, the number of page views increases by 24 percent, sales by 18. Plus, the recommender systems create prerequisites for a number of non-trivial monetization schemes (for example, highlight unrealized demand), etc. All this taken together and warms up the fashion on the recommendations.
Meanwhile, recommender systems are completely different. I give the floor to Alexander Dolgin, whom I ask to begin with a review of recommendatory services, their specifics, for which resource which advisory service is optimal.
Alexander Dolgin : I don’t think it makes sense to dwell in detail on why advisory services are generally needed. The level of noise in the hall, where we are now trying to establish a meaningful conversation, is an excellent illustration of my speech, because these interferences are nothing compared to the noisy information and product environment in which we all live.
As the presenter correctly noted, the terms “recommendatory service” and “recommendations” are now understood as completely different things. A recommendation in the most general sense is simply an indication that an object has some added value. This indication may be presented in different ways. For example, timeshare sellers who flourished 10 years ago, in a certain sense, made recommendations: they said that there is a good service that allows you to do this, this and that. Ratings, charts, billboards also perform an advisory function, informing that, on average, attracted a large number of people. Professional publications, publishing articles by critics and experts, reports about contests and festivals are also a kind of recommendation. Brand tips, advertisers and sellers are another form of recommendation. Even simple conversations in the circle of relatives and friends, exchanging their opinions, fall under the neck recommendation. And all these different types of recommendations today have a platform on the Internet. And each site, where one person to another tells something in streaming mode, claims the status of a recommendation.
I offer you a number of criteria with which you can structure all this recommendation space.
The first principle is the degree of automation in issuing recommendations. There are two poles here. On the one hand, the “manual method”: that is, the consumer reads a stream of texts and selects from there what he is interested in. The other pole is absolutely automated recommendations, when with one click you can get some kind of specialized information sample. In a sense, search engines can also be referred to as recommender systems that give answers to queries, setting priority in accordance with the reputation of the source of information. There is also a number of intermediate in terms of the degree of automation of technological solutions.
Imhonet's universal recommender system www.imhonet.ru , which I represent, deals with automated recommendations, that is, fast, highly accurate, not condemning the user to the “excavations” in the streaming tape. This does not mean that everything is limited to this level of recommendations - in addition to the automatically generated forecasts, there are many more entry points on the site, which make it possible to refine the recommendation. As a metaphor, I will give an image of a traveler who travels to another continent by plane, from the airport to the hotel - by taxi, and the last meters pass by foot. Because to immediately go to another continent on foot for a long time, it is also difficult to seek out the necessary information in the stream. It means that it is necessary to divide the search into several stages, using various methods (for example, by a landmark work or through profiles of critics with coinciding tastes) and seeking an increasingly targeted sample.
The second principle by which recommendation services can be classified is whose interests they represent. Here, too, two options are possible - systems that work for a manufacturer, agent, seller, promoter, and services that guard the interests of the consumer, helping him to make unprogrammable, unbound choices. In the first case, the system helps to sell. Historically, most of the recommendation services arose as a division of sellers, because it was the only opportunity for them to monetize their services and make money. The accuracy of the recommendations issued here is low, which, incidentally, corresponds to the objectives of the seller. For example, when you get this kind of information in the famous “Amazon” recommendation system: “This is the book that a person bought who bought the same book as you bought” - its goal is not to give you an accurate forecast, but to orient in some direction . The system does not know why the person bought this book. For yourself or someone else. The main thing is that he doesn’t know if he liked the purchase. It creates a rich field for omissions, beneficial for distribution.
Finally, the third principle of the classification of recommender systems is that based on which signals the prediction is made: direct or indirect. For example, a person says to his friend and like-minded person: “I watched this film and I liked it, go and see.” This is a direct signal from the consumer to the consumer. And it can be formulated differently: 10 thousand people watched a certain film - according to the statistics of visits it turns out to be “the best” - go see it too. In the second case, the consumer is oriented only on the actions of his predecessors, but not on the result that they actually received. This is the basis for the promotion of technology in the film industry, when it is important to ensure a high box office of the first weekend, because it automatically starts the so-called non-informative information cascade. And why do people go to the movies? Just because the skirmishers did it.
There are many indirect methods for calculating information about intentions. The problem is that they are all inaccurate. Today, the Internet is constantly collecting and analyzing information about what the user has paid attention to: what he flipped through, on what and how much he held his gaze, etc. Based on these statistics, conclusions are drawn about his true intentions. However, the gap between what the person intended to purchase, for which he eventually paid, and the result that he eventually received is very large. This creates a field for significant errors and inefficient choice.
The advisory service imhonet.ru , which I am developing, calculates the forecast solely on the basis of direct post-factum signals emanating from the consumers themselves, who exchange views on how much they really liked this or that work. In my opinion, this is the most relevant, reliable and useful information. Its quality is determined by how many people participate in the system - if there are few of them, then there is little field for sharing experiences. This opens the way for effective affiliate programs between different sites specializing in different areas, because it is rather difficult to set up such a system within one small site: there are not enough people and their ratings. Collectively make it easier and more efficient.
So, summing up, we can say that the quality of the recommendation service is determined, firstly, by the degree of automation of issuing recommendations, secondly, whose interests it serves (trade or consumer), and, thirdly, which signals it relies on (direct or indirect).
Without going deep into the difficult area of actually calculating recommendations (there are a number of methods), I’ll focus on two of them. The first is based on user clustering. For example, we can assume and statistically reveal that buyers of a particular toothbrush prefer such-and-such toothpaste. Or that people with average incomes give their children to certain kindergartens and go to work by bicycle (as it happens in Holland). Based on this, we are grouping people into clusters, and we advise those who have not yet bought a bike: buy it, because everyone like you have already done so.
The second method is called the collaborative, from the English. collaborative - “collaboration” (it is he who is at the heart of Imonet). In this case, people are chosen not on the basis of abstract assumptions about how it should be, but on the basis of clear facts that a person has consumed and how he has estimated a consumer product.
Question from the audience: The information you collect on your user profile is personal data, the protection of which is guaranteed by law. How open are these data on your website and what are the development prospects in this particular direction?
Alexander Dolgin: All rights to manage personal data are in the hands of the user. He can publish what he likes, what he likes, with whom he is friends, and can turn on the mode of anonymity, privacy.
Host: Another company participating in our round table has introduced a collaborative advisory service - this is the “Poster”. I give the floor to Elena Kuznetsova and ask her to elaborate on the specifics of their service.
Elena Kuznetsova: I would like to tell you how we apply various methods of recommendations in practice. “Poster” is a site about entertainment, one of its main goals is to help the user in choosing the way of spending his leisure time. And the recommendations here are very necessary. We offer the user recommendations of different levels. First, we provide complete information about what opportunities there are for entertainment in the city where he lives: the schedule of cinema, theaters, sporting events, a television program, and so on. Secondly, we publish online catalogs of films, TV shows, books, establishments that can be visited (restaurants, clubs, theaters, etc.). Since the amount of information is very large, we provide the user with quite good, in our opinion, navigation and search tools.
As for the recommendations directly, they are multi-layered.
First of all, our editorial board chooses the most important events for each topic every day - these are exclusively manual recommendations. The second level is the reviews of respected critics who have earned the trust of readers and users. Thirdly, these are reviews and ratings of the users themselves: every month we get about 300 thousand ratings and about 3 thousand reviews of objects (movies, restaurants, etc.). Then there are personalized selection tools - in particular, reviews and assessments of the user's friends. This is not just a review tape, but the opinions of only those people whom the user trusts and wants to see as their referees. And finally, the last - the collaborative method of recommendations "Kinobot". It is based on the Slope One algorithm and so far only works for movies. Briefly formulate its principle, the system recommends you to movies that you have not yet rated, but which users like, who liked the same movies as you.
In addition, “Kinorobot” takes into account the similarity of the genre of the recommended film with the genres of those films that you have already rated, and also takes into account the novelty of the object and the weighted average rating. That is, bad films, we do not recommend, in principle, even if somewhere, something coincided.
Host: A question for clarification: who decides whether a film is good or bad? Revision?
Elena Kuznetsova: No, this is determined by the weighted average rating. If it is below a certain value, then this film will not fall into the recommendations.
Alexander Dolgin: Do you get Fellini films good or bad?
Elena Kuznetsova: Good.
Host: They have a high rating?
Elena Kuznetsova : Enough to get into the recommendations.
Question from the audience: When issuing recommendations, who is given preference - the opinions of experts, friends, or the opinion of an unknown majority? Because it’s one thing when you are advised by a person who knows you personally and with whom your tastes roughly coincide, and the other is a generalized, weighted average recommender.
Elena Kuznetsova: Now on the site, automated recommendations are issued without taking into account the assessments of individual critics and individual assessments of friends. Although we have a small project where we generate recommendations for several people at the same time: for example, what might you and your girlfriend like where you could go together.
I would like to talk about combining various methods of issuing recommendations - automated and non-automated.
The table below shows which objects work best for which objects. According to our observations, collaborative recommendations work very well for objects that do not change over time — old films, books, CDs, etc. Reviews and assessments for them also work well, but since there are so many works, reading the reviews turns into a waste of time. The editorial choice in this case also does not work very well, because it is simply impossible to write reviews of all objects.
However, with regard to new, newly emerging works, we don’t really understand how here you can use collaborative recommendations, because there are either no estimates for a work yet or there are too few of them for the algorithm to be effective. In the case of new products, in our opinion, editorial choice and critics' reviews work much better. It is necessary to take into account that among our audience there is a large percentage of innovators, and for them it is important to receive recommendations specifically for new works.
The situation is even more complicated when it comes to events. We assume that there are some approaches and methods that could automate such recommendations. But it should be borne in mind that the event exists either in the future tense, when there are no assessments for it, or in the past, when it is already useless to recommend it. Therefore, it is not very clear how collaborative recommendations can work in this case.
Alexander Dolgin also mentioned places, places, restaurants, hotels. , : -, , . , , , . , , , , .
, : ? ? , : . , , .
: , , . ? , ? ?
:I'll start with the last question. There is an elegant solution that is used throughout the world. The recommendation service has a large rating base (for example, 100 million). Some part of this database, say, 20 million, is closed, and on the basis of the remaining data, the service tries to calculate these estimates unknown to it at the time of the experiment. And then the calculated estimates are compared with the real ones. In particular, the Netflix cinema advisory service announced a contest with a prize fund of $ 1 million in search of those who can improve their recommendation algorithm. Check there was conducted in this way. In principle, Elena mentioned this particular experience: everyone compares the predicted recommendations with their own opinion and decides whether the system guessed or not. But in isolated cases deviations are possible. And with a large statistics of errors does not happen.
, : ; , , ; , – .
: , , . . .
: , . , – , , , , . Which ones?
Ivan Volchenskov: Using the example of AKADO, I would like to talk about the need for recommender systems for access operators. The operator AKADO provides three basic access services: Internet, digital television and telephony.
All access operators like us seek to fill their services with additional content and services. As it seems to us, in the near future the struggle for the consumer will go not only at the level of attractive prices, but on the basis of quantity, quality and convenience of additional services in each of these three areas.
We began to develop additional services about 2.5 years ago. As a result, we offer a lot of services where our subscribers are faced with the problem of choice.
For example, a service for selling music: we now have 200,000 tracks, by the New Year this figure will reach 1 million, and our users will find it rather difficult to navigate in this information space. According to the existing statistics (for example, regarding such large operators as “America Online”), 90 percent of the tracks offered on the resource have never been downloaded. The same problem with games: there is a huge amount of them, they are expensive and the price of a mistake with the wrong choice is very high. A person wants to know in advance which game to buy. The same thing happens with software, books, etc. In the near future, we are launching the Video To Order service, where our subscribers will need to choose which movie to watch. We are trying to help our users and are going to solve this problem with the help of a recommendation system,which will facilitate the consumption of goods and services on our resources. In our opinion, this will lead to higher sales, as users will be satisfied with the purchase that is relevant to their tastes. Accordingly, customer loyalty will increase, which is another good result.
How are we going to implement this program? Initially, we thought about developing our own recommendation system, but we realized that it was a rather painstaking job. A year ago, we met with representatives of the Imhonet company, conducted our research and realized that the best solution for us is an affiliate program with Imhonet to implement recommendations for third-party resources. Our research has shown that this will be quite simple and will entail a lot of advantages. First of all - “hot start”. Here it has already been said that it is quite difficult to sell new products and goods for which there are no ready-made recommendations. For us Imkhonet is good because these recommendations already exist there. For example, a program guide.
We offer digital television with over 150 channels. But what exactly to watch? Users have a problem with choosing a program guide, it is followed by disappointment in digital television, and people return to Channel One, RTR, NTV and watch the advertised programs.
Therefore, we decided to develop the direction of "personal television", that is, to form a personal program guide for a particular subscriber, based on the channels available to us. Special software will offer the user, who settled in front of the TV, which program to watch now and which one to watch next. And we expect Imkhonet to help us with this thanks to his recommendation system, in which the television rating base has already been accumulated.
Leading:Here there are the most effective, live reference services in different niches - Afisha, Imkhonet, Guru - with their own specifics. But they are all manufacturers of recommendations. As you know, any specialist is like a flux, its completeness is incomplete. There is a person standing in a completely different position - Alexander Sergeev, a journalist, scientific editor of the magazine "Around the World", the author, in my opinion, one of the most fundamental and serious review articles on advisory services.
Alexander Sergeev: I will not speak as the creator of recommendatory services, but as a user, and will try to explain what is a recommendatory system for me and what I expect from such systems.
, , , . , , . . , - , – , .
The first is independence, that is, the absence of a conflict of interest between those who recommend and recommend. The second is collaborativeness, because a system built not on collaborative ideas, but, for example, on expert data, is a completely different thing. Expert advice has always existed; people have been using it for a long time - for example, legal advice. And finally, the third criterion is personalization, that is, the recommendation must be individually tailored to the specific consumer.
I will dwell on these points in more detail. Take advertising: this is also a kind of recommendation - to buy something, to go somewhere, etc. But this is not a recommendation service for one simple reason: the ultimate goal of advertising conflicts with the interests of the client. She wants to sell him something, and the recommendation service is always entirely on the consumer’s side. Therefore, for example, different systems that exist as an additional service to online bookstores, in my opinion of the user, are not advisory services. It is rather a certain type of contextual advertising and a system for the sale of related products ...
Do not confuse advisory service with advice. Consultations are good when they are given by specialists in a narrow field and, because of the highly qualified expert, are expensive. And the choice of a television program or a book is a small decision, for which the consulting approach is not suitable, because it is too expensive. If it is cheap, then, as a result, poor quality. That is why for me the advisory service is only that system, which is built on the analysis of the opinions of my fellow users, and a large number of them. From this point of view, such a wonderful system as “Yandex. Guru ”is not a recommendation: it is an expert system that evaluates and systematizes the opinions of several dozen experts, each of whom describes his or her area and advises what you should pay attention to.
And finally, the third aspect is personalization. It is obvious that any ratings and statistics are not recommendations, since they are the same for everyone. If you give a negative example, then I have a serious problem with the wonderful service “Yandex. News "and his ilk. What are they doing?Of the hundreds of news that appear daily, such systems distinguish 5 and say that they are the most important - for the whole country, for the whole world. But this is absurd - I proceed from my priorities and they can be completely different. I personally do not understand why there is still no advisory service on the news. Every day, as a journalist, I spend 2-3 hours on news, and as a private user, I spend half an hour. And no one helps me to optimize these half an hour, so that within my personal interests I read five news that is most important to me personally.
Now I would like to touch on the implementation of recommendation services. When the recommendation service is created as an independent body - like Imkhet, everything is clear. But when this functionality exists on the application site of some other direction, the question arises: why is it inserted there? Either it is used from tactical considerations, as a sales tool, or from strategic ones, in order to achieve customer loyalty.
, , , . , . , , – , . , . , : , , , - «», . , , - . . , . . , , . , - : « , , - ? , ». , , , , - . – .
The question is how to solve this problem correctly ?! How to make a deal with conscience when commercial and strategic interests overlap? In order to avoid flagrant distortions, the right people should make the decision - the company's management, who sees the whole situation as a whole. It may sometimes be worth a little sacrifice of strategic trust in order to stay afloat. But if this question is not brought up to the level of top management, then such decisions are made at the level of conflicts between departments, which should deal with purely technical solutions. And it all comes down to who won whom.
One of the best solutions to the problem, I consider the removal of the recommendatory division (as untargeted and not the main one for business) outside, turning it into an outsourcing division. Then all the negotiations with him will guide and decide whether the system provides impartial recommendations or use it as a specific mixing tool to achieve the desired results. But it will already be a clear business decision, and not a compromise between developers.
The distance serves as a guarantee of the correctness of the recommendation system. After all, when a company specializes in issuing recommendations, when it is its main activity, it is interested in the quality of the forecast, otherwise its business will fall apart.
I drew to you a certain ideal of a recommendation service and showed how it is possible to deviate from this ideal.
Host: Let me draw your attention to the pragmatic side of the issue. Having launched the Imkhonet affiliate program, we were convinced that even an honest advisory engine that does not convoys to the counter gives a visible economic effect. Introducing the recommendation functionality of Imkhonet on other sites, we retain the same equidistance from manufacturers and suppliers that Alexander Sergeev spoke about. At the same time, the first implementation experience shows that the listability of pages (the number of views) on third-party sites increases by 24 percent, and the merchantability of content increases by 18. Without any mixing and deceit. Just by improving navigation and optimizing choices.
Ivan Volchenskov:I have a comment. Alexander Sergeev noted that there is a discrepancy between the increase in sales and the increase in loyalty; one must choose either one or the other. I told in my report how we combine these two goals - increasing sales and loyalty. If you purchase a rather expensive product that you have been recommended and which turned out to be relevant to your expectations, will you not increase your loyalty to the service that supplied you with this recommendation? As such, there is no brutal conflict here. When it comes to products that need to be sold (roughly speaking, because there are large deductions to copyright holders and you need to recoup money), it is advisable to do it just with honest advertising: context, media, whatever. But this does not in any way cancel recommendations as a tool to increase sales and increase loyalty.Moreover, the recommendations are especially relevant in those products where taste matters play a decisive role and the opinion of the target audience is important.
Question from the audience: Today, the word “recommendations” sounded a lot, or rather, “rating”, assessments, something else. What is this, in your opinion? How many times does the user click the plus button? Or is it a rather difficult computed index made up of many components?
Alexander Dolgin: A recommendation is a forecast of how much a person will like a movie (a book, any other object) with which he is not yet familiar. The quality of this forecast is verified in practice. If the service predicts: “you will like it at 9,” and you looked, and it turned out to be a “two” - then a bad recommendation. A recommendation is some predictive statement about the future that nobody seems to know, but we have learned to count it.
Leading : : «» «». – . – , , , . , . , .
: , ? , , , ?
:Very good question. I asked him myself several times. I personally believe that in some areas expert judgment is more valuable than evaluation by a large community. For example, I believe that if a service is made on news, then it should be built solely on mass assessments. And if a sub-service on the news of science is made in it, then expert assessments must be mixed into it. This is necessary, of course. But first of all you need to understand that there is a recommendatory service in understanding our terminology and there are various other types of recommendations, for example, consulting. So, the inclusion of an element of consulting in a recommendation service, in my opinion, is possible. But this should be negotiated separately and separately. This is generally a topic for a special conversation - is it possible to make this relevant, and if so, how.
Question: I would like more details on the terms of the affiliate program.
Moderator: Quite briefly: Quite briefly: Imkhonet recommendation service has developed an export version of the recommendatory functionality, which provides other resources free of charge. At the same time Imkhonet solves two problems for partners at once: 1. provides the recommendation system itself (and also helps to establish, adapt, and maintain it later) 2. helps to overcome the “cold start” stage.
Source: https://habr.com/ru/post/43764/
All Articles