This article will discuss how to build personalized recommendations on Avito. Historically, the Avito business model has been designed so that the delivery of ads in the search takes place at the time of their placement. In this case, the user can buy additional services in order to raise his ad in the search in the event that over time the ad has fallen far in the search results and stopped typing views and contacts.

In the context of this business model, it is not obvious why we need personal recommendations. After all, they just violate the logic of sorting by time and those users who pay for raising an ad may be offended that we “raise” another person’s ad and show it to the user for free just because our recommendation model considered this ad more relevant to some user.

However, now the personal recommendations become the “must have” for classic (and not only) worldwide. We want to help the user in finding what he needs. Already, an increasingly significant share of the views of ads on Avito is made from the recommendations on the main page of the applications or the recommendations of similar ads on the product card. In this post I will tell you exactly what tasks our team at Avito solves.

Kinds of recommendations

First, consider what types of recommendations may be useful on Avito.

User-item recommendations

First of all, these are user-item recommendations, that is, recommendations of ads for the user. They can be of two types. The first is the goods or services that the user is currently looking for. The second type is complementary goods or services. For example, phone cases, if a person is looking for a phone. Or furniture transportation services, if a person buys or sells an apartment. Or klassery to store a collection of philatelist, if a person is looking for postage stamps.

User-item recommendations we deliver to users now in three ways:

• blocks with recommendations on the main page of mobile applications;
• banners with recommendations in search results on the desktop;
• email and push mailings with a selection of recommended ads.

User-category recommendations

It may also be necessary to recommend not specific ads, but product categories (user-category recommendations), by going to which the user himself specifies the search filters. User-category recommendations are also divided into two types: recommendations of categories of current interests of the user and cross-category recommendations. We now use this type of recommendation in the push lists and on the main application page.

Cross-category recommendations are especially important for Avito, since the majority of Runet users have used Avito at least once, but often “sit” in one category. Many people do not realize that in addition to personal items on Avito, you can effectively sell an apartment or a car. Cross-category recommendations help us expand the range of categories in which the user is a seller or buyer, and thus increase user involvement.

Item-item recommendations

Another promising area of ​​recommendation for Avito is item-recommendation, that is, product recommendations for other products. This type of recommendation is also divided into recommendations of similar products (analogues) and complementary goods or services. This direction is especially important because, unlike media portals (movies, music), the user usually comes to Avito for something specific, and it is difficult for us to predict the current user preferences in advance. But if the user is already looking at some product, then we can advise him alternatives or complementary products, and they are likely to be relevant to his current search. Recommendations of similar ads are shown on the announcement card, as well as used in email and push mailings.

User-item recommendations for classifications

Now we delve a little into the task user-item of recommendations, as the most interesting from a theoretical point of view. Input data are:

• history of user actions on the site: views, search queries, contacts, favorites;
• user profiles: data from linked social accounts networks location;
• all active ads on Avito: title, description, parameters, price.

At the same time, the amount of data is relatively large: 20 million active users, 35 million active ads.

Setting the task is as follows: for each active user, show top-N ads with the highest probability of contact request (call or send a message).

Despite the fact that the wording of the task sounds like a classic task of any recommendatory system, its construction for Avito has significant differences from the tasks of recommendations of the media content: films, music and other things. Firstly, liquid goods of private traders are quickly sold, without even having time to collect a good story on views and contact requests. Classical collaborative filtering algorithms are designed so that ads with a short history do not fall into the recommendations. Longer-lived ads are more often recommended, which tend to be of less interest to buyers.

Also, the user is usually interested in a typical product, for which there may be many active ads. For example, he needs to buy a specific iPhone model, and from whom it is no longer so important. Therefore, building recommendations is better not on ads, but on standard products. To do this, we build special clustering algorithms.

Another feature of Avito recommendations is that ads are created by ordinary users and contain errors, incomplete descriptions. This leads to the fact that we have to seriously work on text processing, extracting useful traits from ad descriptions.

Methods

Now a few words about what methods we use to build recommendations.

Offline models

Historically, we have used and continue to use models that process click stream users in “batch” mode. These algorithms allow you to respond to new actions performed by the user, with a delay of 1-2 hours. We call them offline models.

Offline-model recommendations are globally divided into collaborative and content. Obviously, each of these models has its pros and cons, and the best results show hybrid models that take into account both the history of user actions and the content of ads. We use the hybrid model as the main one for offline recommendations.

Online models

Offline-models are able to generate high-quality recommendations, but they can not quickly respond to changes in the interests of the user. This is their significant disadvantage. For example, if a user has started looking for some new product on Avito, then we want to begin recommending suitable products to him within the same session. To do this, we must take into account the interests of the user in real time. We call such models online-models.

Their feature is that they are more complex from an architectural point of view (the time from the moment the user performs the action until the recommendations are updated - no more than 1 second). The classic online model is based on the construction of an online profile of user interests, with the help of which the most recent and relevant ads are selected. Due to stringent performance requirements, online algorithms are usually simpler than offline.

Model quality assessment

After the new model is created, it needs to be somehow assessed. The target metric for the company is the increase in the number of transactions on Avito. All offline and online metrics must somehow correlate with it.

To evaluate offline-models, there are a number of excellent metrics, such as precision, recall , NDCG , R-score and others.

It is not always possible to select such offline-metrics that correlate well with the target metric of the company. This is where online metrics come to the rescue (CTR, conversion to contacts, growth in unique customers). On online split tests, we can compare recommendations from different models and different frontend interfaces. The method of multi-armed bandits is well suited to optimize the metaparameters of the models.

Results

The Avito recommendation team faces ambitious tasks that require in-depth and active research of recommendation methods that can withstand Avito's performance loads and show excellent results on target metrics.

In order to find optimal approaches, we read a lot of articles, we go and speak at conferences and hold contests. Not so long ago, our recommendation contest ended, and we do not plan to stop there. We call upon all interested to help us in this difficult work by participating in our contests. And we will try not to skimp on the prize :). We also occasionally open vacancies, which we are sure to announce in the slack channel ODS.

In addition, we ourselves participate in contests. BTW, in 2016 and 2017 we entered the top ten teams in the largest international competition in recommendation systems Recsys Challenge.

Thanks for attention!