Airbnb Search Technology

Three weeks ago, we wrote about how users and homeowners can more effectively use the search on our site. Today we want to talk about the algorithms on which our search engine is based.

The post was prepared based on the speech of Maxim Charkov:

')
Search is the power of matchings. Speaking essentially, here we are trying to connect the requests of our users with what is available on the market.

First I would like to say a few words about myself and my colleagues. I work in a search team. I started working in the company two years ago. Before that, I was an employee at Google, where I spent several years doing everything in my life, from search functions to web browsers. Of course, everything that I am going to present here would not have been possible without people from our team. Search Airbnb is a permanent team. Our engineers are working on the problems of search and streaming reservations, including infrastructure, user interface, etc. The scope of our activities also includes the development of equipment, design, user research, data processing and analysis.

First I want to present the problem of searching on Airbnb and how we help our guests find the best positions. Then I will talk about the issue of conversion booking. You will see that Airbnb does not always have enough positions to satisfy all user requests. And this is an interesting task. Also I will say a few words about the evaluation of modifications. When working on new search products, it is very important to establish evaluation tools and factors that would give confidence that every change made will have a positive effect for the user.

Airbnb is a global ad space. Today in our database there are more than 600 thousand positions, 34 000 cities in 192 countries of the world. On our site you can find housing for every taste, from ordinary apartments to tree houses, and even private islands.



Here you can see the density of the positions we offer in the San Francisco area. Each point is a corresponding sentence.

North America:



Europe is also a dense market.



The Asia-Pacific region is rapidly gaining momentum.



We need new solutions to work on a global scale, as we have all this huge variety of locations and types of offers.

Let's look at the search behavior of people who have successfully made a reservation on Airbnb. Travelers spend a lot of time searching. On average, each of them spends at least three days from the moment of the first search request to the completion of the reservation. During this time, they consider approximately 20 positions. The process of finding accommodation is not easy in itself for many reasons. Choosing a place is a serious decision. Often people want to consult with family or friends before making the final choice.

Unlike web search (I worked with it before), you almost never see the full picture of the search when the user simply enters a query, selects the first one in the list of results and makes a reservation. Even if the first result is objectively the best option for the user, people still want to consider a few others before making a choice.

Now let's see how the user interacts with search engines. They usually start from the page where they enter search parameters: place, date of travel and number of tourists. Next, they will see the most appropriate position.



This is what the user sees, but behind the scenes we have a complex ranking system that combines thousands and thousands of signals that make it clear that these proposed options are exactly what our guest wants.

Our task, as a search team, is to help the user achieve his goals. That is, we must make the process of finding a place to stay as easy as possible. Let's start with the Find section. We constantly measure how accurately the output results correspond to user requests. The most effective is to count the total number of users who have made a search query, and those who have completed a reservation on Airbnb. Increasing the last indicator is possible only by increasing the quality, relevance and personalization of search results for a particular user.

Therefore, we must first understand the concept of quality: this is, in fact, the attractiveness of a position in terms of a request. We have a model that calculates quality for each search result, and then we use this indicator when ranking companies. In a broad sense, the model looks like two groups of signals. One group is the attributes of positions: pictures, number of reviews, ratings, location attractiveness, estimated cost, etc. And the other is behavioral signals: user interaction and positions on our site.

Behavioral factors

They are unspoken user feedback on search results, and therefore very useful to us. Let's take a look at this sample search page.



We can assume that a typical user views the results from top to bottom, the black arrows indicate the order in which the user reviews the search results. If he believes that the position is worth his attention, then click on it for more detailed information, or immediately go to the next. Thus, we can already try to build a ranking factor by counting the number of clicks that each result received, but here it is worth making a small amendment. Let's look at the results, namely number 7.



Would it be true to say that the user saw the last position and found it most suitable? It is also possible that the guest has already found the best result among the 7 upstream. Or maybe he decided to completely abandon the request or change it. To prevent an unfair exclusion from the results of the issue, for each individual page we determine the position of the last click. In this case, this is the result number 7. Thus, when creating a ranking factor, the last considered result is taken into account, as well as all higher user-open links.

Of course, just clicking is not enough as such. We should not limit all behavioral signals when searching. For example, we look at the page and try to assess how much time it took the guest to take any action after briefly viewing the page with search results.
A behavioral approach to quality can be very effective. There are some proposals that received very low marks, but according to the behavioral signals listed above, their score was extremely high. First you can find some low-quality results, like this cave in Berlin.





This is probably not a real position. Sometimes the type of position may be unexpected: a typical Airbnb user did not expect to find a parking space. Here is another example: we have a garage.





And the car!



Now take a look at the other side. You can see what, as a rule, is the difference between the available positions: the attractiveness of the photo, competitive value and high rating. Of course, sometimes we fail to make a correct assessment, relying only on user behavior, so we combine behavioral signals with more obvious factors, such as feedback, to calculate the final quality assessment.

Oddly enough, we pay more attention to quality than to the result of the issue itself. For business travelers, the location of housing plays a crucial role. We always ask our users to fill out a small survey form upon completion of the use of the service. And in one of the points we ask the user to estimate the location (location). Thus, we can use these answers to see the algorithm, and then, if you like, assess the quality of the location of any position, view the estimates of nearby positions.





Here, on the left, we have a quality indicator for San Francisco, which is based on knn. And on the right, we have estimates of San Francisco positions obtained from another map.

Considering the location, it is important to consider not only the quality, it should also be relevant to the request. Here we see positions at the request of “Santa Cruz, CA”.



Obviously, all the housing offered is located in Santa Cruz, but what if the user wanted to stay in Santa Cruz County? The second position is from Airbnb, it has 500 views. But its location is Aptos, which is located nearby, but still this is not Santa Cruz, and few would have guessed to search for this particular place. Of course, we could say something like “If the query is“ Santa Cruz, ”then Aptos is also suitable.” But this problem remains unresolved only because we work around the world. In addition, we really do not want to make a choice for our users. On the contrary, we want to learn from their actions and reasoning about each request. Also requests can be much more complex than just the names of cities. The user can search for the neighborhood of the village, suburb, postal address, country or state. Sometimes - even a separate landmark.

Since we want to learn about relevance from our users, we need to understand what exactly people consider relevant, suitable for them. After we group the logs of user actions, another problem arises. Individual users can go to the site several times to solve various problems. For example, a guest visits the site and enters a Pacific request, then decides to make a reservation. The next day, he returns and is looking for accommodation in "Los Angeles", and booked in "Santa Monica". And we are trying to separate such actions.





Another important step is canonicalization. It is necessary to clear the data, especially to correct errors and differences in the spelling. For example, “SF” is San Francisco, but correctly in the request you need to enter “San Francisco, CA, USA”. We corrected the spelling error for Germany and added the Japanese spelling of the query “Buenos Aires” to the database.



After we have data on the session, the computer begins to generate signals. One of them is the conditional probability of an order in a given period of time, taking into account the specifics of a particular request. This effectively reflects how many people were searching for something already booked in a particular locality.



For example, if we analyze the reservation in the city of Aptos, it turns out that people found this position on the request of "Santa Cruz". This indicates a strong connection between these two places. As well as data signals, the number of positions in this location, and the distance between the place of reservation and the settlement in the request.



This is real data for Pacific, California. You can see some interesting features. For example, most people looking for accommodation in Pacific have booked in San Francisco. This may seem strange, but this is due to the fact that in Pacifica, Airbnb is not very popular. Upon request, only about 20 positions are displayed. But San Francisco will offer you a huge selection. Even if the user wants to stay in Pacifica, often in the end he will still book accommodation in San Francisco. We can finally debunk the doubts, taking into account the size of cities.

Look at the second table. A large number of users who decided to stop in El Granada, before that, wanted to choose Pacifica. On the last chart you can see the combined score. Not only can you find out that Pacific is relevant to the Pacific request, we also have a bunch of alternative places like El Granada, etc.



You can see them all on the map. People who were looking for accommodation here could also consider options in a number of other places. All cities located near Pacific are displayed, which correspond to the request.
In addition, we are trying to personalize all the results for a specific guest. For this we use social graphs. The fact that your friend reviewed the search results, and then chose a specific position, makes it more relevant to you. We also try to find the results most likely to meet the guest and host in real life. For example, you may have a mutual friend with the owner, or he went to the same university as you.

Reservation conversion

Our task is to maximally facilitate the search and booking of accommodation for our users. We are discussing how we can help them find the best place. But how can we be sure that they will actually complete the order? Airbnb is similar to other travel sites.



Here is an illustration of the flow of booking First you must find an attractive position. Further you contact the owner. If he agrees to accept you, then you book. The site supports hot keys for quick access to individual functions. For example, there is an instant booking option that the landlord can activate, but in most cases you will still need confirmation from the landlord. The owner may refuse to accept you or simply ignore you. This is really unpleasant when you are looking for a suitable position for three days, and then you are simply ignored.



We track the actions from the moment of establishing contact with the owner of the property until the confirmation of the guest’s candidacy. Essentially, "Contact to Accept" is the ratio between users, whose stay has been confirmed by the owners, and those who tried to contact the owners of the property.

We have come pretty far in recent years.
We did some research to determine why some guests were rejected. And again we have two sets of signals: one is “material and technical,” the second is “emotional.” The most common technical reason is accessibility. For example, the owner did not update the calendar time or simply did not mark the availability of housing on certain days on the calendar. Or he is not able to satisfy the wishes of the guest: the length of stay, the time required before or after the check-in. Some owners set a minimum stay, three nights, for example, or entry only on weekends. Of course, there is always an emotional component. Each owner has a preference according to which he decides whether to accept you or not.

We do a lot to improve performance. For example, we are trying to connect the host with a guest whom he would most likely agree to serve. A lot of effort is spent on optimization: the balance of attractiveness and bookability (bookable, armor) - how often a reservation is made, since a less attractive position can actually be better. And, of course, the user interface should have to interact with the site.

Host Compatibility

This is our attempt to substantially personalize the product for our owners. There are several important points for this. First: just because the owner accepts on average 50% of those who wish, does not mean that any request will be accepted or rejected with equal probability. In fact, the owner has his own preferences, the guest may not be suitable for personal reasons and is rejected without explanation. Different hosts have different preferences. One example is our homeowner from Miami. He prefers to receive guests for long periods of time and rejects all requests for accommodation for several days. He will not agree to accept a person for 2 nights if this prevents him from receiving other tourists for a longer period.

Another important factor for deciding is whether a specific guest request fits into the host calendar.




Suppose this is my calendar. I have a reservation from the 21st to the 24th. There is one more reservation. Between them I have three free days, from February 25 to February 27. Suppose I have a request for one-day stay on February 26th. If I accept it, it will mean that I will have gaps in the schedule and will have to look for two more tenants for two free days and nights. For this reason alone, it would be advisable to reject the request. Although the other owner might prefer to have free time between reservations and would accept this offer.
In Airbnb, we use information about the host’s past behavior, failures, and accepted requests to personalize the site’s response model to a particular action. We take into account the preferences of each owner and then apply this model when searching to calculate the probability of accepting a request for settlement, taking into account all the available information about the trip, guest and current status of the owner. Some of them do not have clear preferences.



And here is another example of how changing an AUI can have quite a big impact on performance. We used this function several years ago, which allowed us to search for certain attributes: distance, price, etc. The option was quite popular. In fact, about 10% of requests were made at the cost of rent. The problem was that the price was the only attribute. Strangely enough, the cheapest position could look very attractive for a guest, but he did not realize that, most likely, the owner would reject him. Thus, we abandoned this possibility and added a conversion on the control experiment.

Evaluation

At Airbnb, we test and evaluate every, even the slightest, change that we make in the search interface, booking stream or algorithms. It is important to have a set of tools that would allow us to quickly get an idea of ​​the work of experimental functions, of productivity. Therefore, we have created a number of tools, for example, for offline testing, which we use before introducing a new algorithm. We also have the power for online evaluation. Sometimes we do market-level experiments.

S x S

Side by side, one of the most important tools. It allows engineers to get quick answers to questions like: “What percentage of requests will this affect”, or “How a change in my ranking will change the price distribution on a given page”, or “Give me a few examples of requests that give different results in this experiment ranking. Engineer can also get answers to questions about traffic. For example, "How do my changes affect a top or less popular query?" At night, we start processing similar requests from developers, analyzing examples of real requests. The result is a list of queries with different probability distributions. After that, comparisons can be made, we simply indicate what we want to compare, ranking experiments, estimation options, sample queries, how many examples we need.

We compare different result sets using a special function that is based on the Kendal's tau rank correlation method. This is a very simple method by which the number of pairs of results that change positions is calculated. We made some changes to the algorithm. For example, they modified it to work with top queries, since the user sees only top positions on the site. We have already received some statistics. For example, changing rankings affects more than 30% of queries.

The engineer can also delve into the search and analysis of data. If you want to know why the results are ranked in a certain way in your control group, you can view the values ​​of all signals, all the estimates obtained as a result of ranking, and so on. Not all data can be displayed, as they may contain confidential user information, but this information is very important for making quick changes.
The main rule here is: always conduct experiments on real users. Thus, we can implement changes in the user interface or modify the ranking algorithm, and then compare the conversion in different groups of users.

I also want to talk about one problem that makes interaction on the site very difficult.

Take an example when we have two positions in the same city. Both options have approximately the same price and many reviews. The first position is higher, as it has a higher percentage of guest acceptance, 90%, while the second has only 50%. Let's assume that we are conducting a host selection experiment. Apparently, the second owner prefers couples. If it is not a couple, it simply rejects the candidacy. We are developing the following experiment: we say that the group will see the usual position rating, but if the number of guests in the request is two, then the second result will be in the top, as the guest confirmation rating will be 100%. The position will start receiving orders, the host calendar will start to fill up, the result will be less often displayed for a control group of people. Thus, fewer people from the control group will fall into the trap and will contact the owner of the property, which will allow the control group to work better for the benefit of the experiment.

There are completely opposite examples. The fact that this is an experiment can weaken your control over the situation. Certain aspects may be underestimated. The problem is that you can isolate the guests from each other, but they still interact in the market. And more importantly, they influence their development needs. For example, if a guest makes a reservation for a specific date, it means that another user will not be able to book it.

For us, this is a big problem. Of course, this is not a problem for a product like Google Search, because web results do not disappear when people click on them. However, everything is much more complicated when you are dealing with locations. One of the solutions is to separate from the market as much as possible. If you are conducting an experiment in Boston, for example, this should not affect the Chicago market. We can create control groups within these departments.Now this users are not at the level of the local market, but of entire geographic areas. And we can run an experiment for all users in a certain area, and then compare the results with the data for the control region.

The problem here is that it’s very difficult for Airbnb to find markets that we can compare with each other. Firstly, the tourism business has a pronounced seasonality. And in different markets, high seasons occur at different times of the year. Thus, the productivity and profitability of each region will vary depending on factors that are difficult to take everything into account. In addition, Airbnb continues to grow very quickly, and some markets are still not stabilized, and as they grow, they completely change the picture.

I talked about how we help our guests find attractive positions, how we display the availability of positions and evaluate the changes we have made. Finally, I’ll talk about some of the problems we have recently encountered. If you visit our site, you will see a clear enough message: “Give us the date and place of the trip, and you will receive accommodation.” By accessing Airbnb you need to clearly know where you want to go. But this should not be so. We want to learn how to produce suitable results for many queries. For example, “What if I don’t even know where I want to go, but I certainly know what I want to do. Can you find an amazing trip for me taking into account what you know about me? ”To help the user with such requests, we need to learn to“ think ”more widely and look“ deeper ”.



For example, on the left we have a very specific query. Location position - Venice, this is a view of the canal. Or “I live in San Francisco and I want to spend no more than two hours on the road. Find me the coolest place on the weekend so I can live in the house myself. ”

You will discover a lot of amazing travel opportunities, even if you don’t try to find them. Just open the app and see tips and personalized offers.

On Airbnb, each position is unique. Helping our guests with navigating through all the functions of the portal is one of their priorities. So we don’t want to overwhelm you with the thousands of offers available. Instead, we want to show only a few ideally suited for you. And, of course, we try to ensure ease of use of the service so that the booking does not require any effort.

Thanks for attention!