The reverse side of the ticket. How Tutu.ru helps to choose the best rate

In the spring of 2014, amendments to the Air Code of the Russian Federation were adopted, allowing airlines to enter into a contract for transportation without refund of fare in case of termination of the contract. In other words, the non-refundable tariffs appeared on the air transportation market. Before these changes, the airlines could only withhold a fine of not more than 25% of the ticket price, if the passenger handed over the ticket later than one day before departure. The new amendments allowed airlines to offer passengers cheaper, but non-refundable tickets.

At the same time appeared budget "unavailable tariffs." In fact, it is impossible to call them completely without a baggage: according to the law of the Russian Federation, a passenger has the right to bring up to 10 kg of personal belongings with him. And there is an interesting point: the law does not regulate how a passenger transports these 10 kg - in the cabin or in the luggage compartment. As you know, in the salon you can not take a lot of things: for example, liquid over 100 ml, nail scissors, a nail file and some gadgets. Even if the fare includes baggage allowance, each airline itself determines the maximum weight and size of baggage and carry-on baggage per passenger.

Passengers were confused in fares, there were many questions related to baggage transportation and surcharge overweight. In this article I want to tell you what and how we did to make it easier for Tutu.ru users to search for the most suitable air tickets among dozens of available options.

The ticket prices of many airlines did not look attractive against the background of the economic crisis, not at all cheapening fares for trains and the appearance of Victory, which really brought down prices in many directions. For example, before the low-cost airline appeared, I flew to Cheboksary and back for 8–10 thousand rubles, and now the flight will cost 3–6 thousand.

In the fall of 2015, with the aim of lowering prices, airlines began to actively introduce non-luggage and irrevocable tariffs. For us it was very important to provide the user with the opportunity to independently choose the options of interest to him. People were faced with the fact that the airport requires an extra charge for luggage, or that a ticket cannot be returned or exchanged. We wanted to give a person the opportunity to choose exactly what they need, making it as easy as possible and speeding up the search for tickets. On the other hand, the airlines themselves put forward to their agents, including Tutu.ru , the requirement to show information about the features of the tariff already at the search stage.

By the time we already showed customers whether it is possible to pass or exchange tickets. With luggage everything was more difficult.

The taste and color of GDS is not

Baggage information we receive from booking systems (GDS - Global Distribution System), which imposes some restrictions. First, we work with several GDSs, as well as with GDS aggregators, and this requires solution support for each of the systems.

Secondly, historically two types of airlines. Some limit the maximum luggage by weight, and others - by the number of seats. For example, "Aeroflot" gives a limit on the number of seats, and from the GDS we receive little informative data about baggage: "1 piece" without information about the maximum weight.

And finally, the most important thing. Information about baggage restrictions comes from the GDS, sometimes when the client has already selected a specific offer and proceeded to checkout. We tried to give the client an opportunity to compare offers at the stage of selection. Sending clarifying requests to the GDS, while the client chooses, it would be too expensive for both money and time: there may be dozens and hundreds of offers in one direction per day.

Of course, GDSs are not standing still, and more information comes to us in new API versions of the information we are interested in, and for some GDSs it is already available at the search stage. But to a greater extent this applies to large airlines that have switched to the system of branded fares (we will talk about it later). We use this information, but still this is not enough to make a truly convenient service. GDSs do not always ensure that airlines enter information about baggage, and especially do not bear responsibility, if the rules have changed, and the information has not yet been entered into the GDS, or if an error was made during the deposit. Therefore, for many small and regional airlines, this information is not in the GDS. Data on restrictions on carry-on baggage does not contribute to almost any airline.

There are some very complicated cases, for example, the Orenburg airline carries information in the GDS that baggage with a weight of no more than 10 kg is allowed. At the same time, flights are operated on An-2 airplanes, which do not have a luggage compartment, so it’s impossible to take things prohibited for carriage in the cabin. In this case, we show users that there is no baggage on the flight, but hand luggage weighing up to 10kg is allowed.

With the ability to exchange and return at that time everything was a little better, but only a little. Information about the possibility of returning and exchanging a ticket (as well as about baggage) was often available only on the selected offer, and not at the search stage. Moreover, the data from the GDS came to us not in the form of convenient for processing XML, but in the form of a “sheet” of text, in which the only formatting is a different number of spaces at the beginning of the line.

UNLESS OTHERWISE SPECIFIED CHANGES ANY TIME CHANGES PERMITTED. NOTE - CHARGE EUR 50.00 FOR REISSUE/REVALIDATION ---------- NO CHILD DISCOUNT APPLIES / INFANT WITHOUT A SEAT - NO CHARGE. ---------- CHANGES PERMITTED BEFORE ORIGINALY SCHEDULED FLIGHT ---------- THE TICKET MUST BE REVALIDATED OR REISSUED AT THE SAME TIME WHEN THE BOOKING IS CHANGED. ---------- THE CHANGE FEE APPLIES PER TRANSACTION. THE CHANGE FEE MUST BE COLLECTED AT THE SAME TIME WHEN THE BOOKING IS CHANGED ---------- WHEN COMBINING ON A HALF ROUNDTRIP BASIS THE PENALTY RULES FOR EACH FARE COMPONENT APPLY. CHARGE HIGHEST PENALTY FEE OF ALL CHANGED FARE COMPONENTS. ---------- IN CASE OF REISSUE-REROUTING/UPGRADE- WHEN THE FIRST FARE COMPONENT IS CHANGED CURRENT FARES VALID AT THE TIME OF REISSUE MUST BE USED. OTHERWISE HISTORICAL FARES VALID AT THE TIME OF ISSUANCE OF PREVIOUS TICKET MUST BE USED. ----------- IN CASE OF RE-ROUTING - NEW BASE FARE MUST BE EQUAL OR HIGHER THAN PREVIOUS FARE AND FARE DIFFERENCE HAS TO BE COLLECTED. WHEN NEW ITINERARY RESULTS IN LOWER FARE - UPGRADE TO A LEVEL WHICH IS AT LEAST EQUAL AS THE BASE FARE OF PREVIOUS TICKET. ---------- FARE DIFFERENCE AND A CHANGE FEE WILL BE COLLECTED. ---------- CHANGES PERMITTED ONLY WITHIN SAME OR HIGHER BRAND. ---------- ALL PROVISIONS OF THE NEW FARE MUST BE COMPLIED WITH. ---------- ONCE A FARE COMPONENT HAS BEEN COMPLETED FARE BREAK POINTS CAN NOT BE CHANGED. ---------- THE NON-REFUNDABLE AMOUNT -FARE/YQ/YR - OF THE ORIGINAL TICKET WILL BE NON-REFUNDABLE AND HAS TO BE INSERTED IN THE ENDORSEMENT BOX OF THE NEW TICKET. ---------- NO SHOW - NOT PERMITTED ---------- IF PASSENGER IS NO SHOW - FINNAIR HAS A RIGHT TO CANCEL ONWARD OR RETURN RESERVATION. ---------- NAME CHANGES PERMITTED. APPLIES ONLY TO ITINERARY WITH FLIGHTS MARKETED AND OPERATED BY AY. CANCELLATIONS BEFORE DEPARTURE CHARGE 50 PERCENT FOR CANCEL/REFUND. NOTE - NO CHILD DISCOUNT APPLIES / INFANT WITHOUT A SEAT - NO CHARGE. ---------- UNUSED YR/YQ FEES WILL BE REFUNDED. ---------- THE MOST RESTRICTIVE REFUND CONDITIONS APPLY WHEN COMBINING ON A HALF ROUNDTRIP BASIS. ---------- FULL REFUND PERMITTED IN CASE OF- - REJECTION OF VISA. EMBASSY STATEMENT REQUIRED. - DEATH OF PASSENGER OR FAMILY MEMBER. WAIVERS MUST BE EVIDENCED BY DEATH CERTIFICATE. ---------- NO SHOW - NOT PERMITTED. AFTER DEPARTURE TICKET IS NON-REFUNDABLE. NOTE - UNUSED YR/YQ FEES ARE REFUNDABLE. ---------- THE MOST RESTRICTIVE REFUND CONDITIONS APPLY WHEN COMBINING ON A HALF ROUNDTRIP BASIS. ---------- NO SHOW - NOT PERMITTED. 

We already knew how to determine the terms of exchange and return for such texts, and even before what time they are possible: for some companies before the end of registration, for others not later than 48 hours before departure, etc. To do this, we saved the texts of the tariff rules for all orders that made our users. Then, a simple but automatic system found the parts changing from the order to the order in the stored texts, such as the amount of the fine for the return, and cut them out. From the received text with gaps, some hash was calculated, and then the source, i.e. no gaps, the rules were grouped by this hash. Almost every order after payment was viewed by operators of our contact center. Operators read the specific text of the rules, defined the terms of exchange and return, and set a digestible description in accordance with the hash of this text. For example, “Return is possible no more than 24 hours before departure”. Subsequently, when another customer placed an order, and the hash was calculated for the text of the tariff rules, we could already show understandable terms of exchange and return. But at the search stage, this decision again did not fit: the information came already at the checkout stage.

New tariff system

We started developing a new solution with a simple system that can only show restrictions for S7 and Ural Airlines. They were the first in Russia to switch to a new tariff system. How the old system worked: for example, the airline opens a sale for a flight that will carry out an aircraft for 300 people. These 300 places are divided into 10-20 so-called booking classes (not to be confused with service classes: Economy, Business, First, etc.). Each class actually determines the services offered by the airline: baggage allowance, meals included, possibility to choose a seat, as well as exchange and return tickets, etc. Depending on the set of services varies and the cost of tariffs in this class. And on different flights within the same booking class, the range of services may differ.

The new system of tariff families (they are also called branded tariffs) is easier. Booking classes remain, but the fare code, namely its name, is responsible for the available options; the specific booking class and flight are no longer affected. For example, the S7 has 4 tariff families: Economy Basic - non-refundable and only with hand luggage, Economy Flexible - returnable and with luggage included 23 kg, Business Basic - non-returnable, without access to the business hall and with one piece of luggage up to 32 kg and Business Flexible - returnable, with access to the business hall and in two places 32 kg each. It is very simple to distinguish them from each other: in the code of tariffs of the Basic family there is a substring BS, and in the code of tariffs of the family Flexible it is a substring of FL. The booking class is no longer responsible for the available options. For example, if there are 9 places left on the booking class N, then you can buy tickets to these places with and without baggage.

Soon, several more major Russian airlines switched to tariff families. We set a goal to provide the user with a choice in most cases (even when you need to fly from Yakutsk to White Mountain ). In addition, with tariff families, everything is simple only as long as the client flies with his own flights of the airline that owns these fares. As soon as there is a transfer with the change of the airline, exceptions begin.

Develop your own solution

We began developing our own solution with a thorough analysis. We went to the operators of our contact center and for a long time asked out which airline was responsible for providing these or those options to the client. A little distracted, I note that the process of buying a ticket involves up to three airlines at once: the one that owns the plane (we call it operating), the one whose flight number is indicated on your ticket (marketing), and the one that sold you a ticket on its form and will distribute money between the rest (validating).

For example, you fly from Chelyabinsk to Rome with transfers in Moscow and Dusseldorf. On the Chelyabinsk - Moscow segment, you fly your own S7 - S7-8 flight. On the Moscow-Düsseldorf segment, the aircraft belongs to S7, but you have a flight with AirBerlin and the number AB-5911 on your ticket. This means that in this segment the operating carrier is S7, and the marketing carrier is AirBerlin and this combination is called codeshare agreement. Further from Düsseldorf to Rome you fly your own flight AB-8718 AirBerlin. At the same time, the entire flight from Chelyabinsk to Rome was issued with one electronic ticket, which was sold to you by AirBerlin airline despite the fact that on the first segment you are flying your own S7 flight. In this case, AirBerlin is a validating carrier, i.e. the airline, which, by mutual agreement with other airlines participating in the flight, issues a ticket and will carry out mutual settlements. Such an agreement is called interline. Most often, the validating company is also a marketing company in one of the segments, however, there are cases when all three airlines are different - operating, marketing and validating.

So, at the stage of collecting information, we found out that usually the options are set by the marketing airline - the one whose flight number is specified on your ticket and which owns the fare. However, it may turn out that the plane of the operating airline is not designed for marketing standards. This applies mainly to baggage. For example, in the plane may not be the luggage compartment, as, for example, in the An-2 . In addition, on flights with transfers on different segments there may be different baggage rules and it is doubtful that the passenger will be happy to get rid of the extra 2 kg of baggage somewhere in the intermediate point. On the other hand, Aeroflot most often guarantees its customers the baggage allowance, even if the operating airline has lower standards.

Even within the same airline, there are often exceptions in a particular direction, and even at a particular airport. For example, in Qatar Airways, the free baggage allowance in economy class for flights from Russia is 1 place weighing no more than 30 kg, and for flights from the USA 2 places no more than 23 kg each. Many companies change the baggage allowance on flights to the Middle East - to Turkey and Egypt. There are some very exceptional cases when, in any particular area, the right to set baggage allowances and charge a fee for their excess is bought by a company that is not related to the airline.

In parallel with the accumulation of knowledge about the problem, we began to collect the information that comes to us from the GDS, in the hope of getting patterns out of it. We needed to maintain our knowledge base, since the airline's tariff rules are constantly changing. The idea did not work: only for one Aeroflot with fairly standardized rules, we received a decision tree, which occupied A4 sheet in a very small font.

We introduce the FRule mechanism

As a result, we decided to use the FRule mechanism, which had been used in our team for a long time and was originally a flexible configuration mechanism. Later we successfully applied it to several applications.

FRule is a flexible configuration system that we developed for our services. Subsequently, the name FRule began to call the architectural approach to describe the system in which you need to flexibly set and select a particular case among many common cases.

He is like that. Suppose we have a function of N variables. First we ask

Then we fix some variable. $$$x_i = z$and ask

And, for example, for $$$x_i = w$ask

We fix further $$$x_j$and continue the process, essentially moving from the general to the particular. Then, the input of this function is fed any data. In the case of baggage allowance, these are flight segment data: from where, where, when, which airlines, fare code, etc. There is one nuance: variables in the function $$$F$may be unequal to each other. Consider the simplest case of a function of two variables. $$$F (x_1, x_2)$. Suppose we asked

For the function thus specified $$$F (0, 0)$undefined. To get rid of such uncertainties, we use a pre-set system of priorities: we say that the values ​​defined for a set in which only $$$x_1$, more priority than the values ​​defined for a set in which only $$$x_2$. In this case, the values ​​defined for the set in which are fixed and $$$x_1$and $$$x_2$, over the values ​​defined for sets in which only one of the variables is fixed. In this way, $$$F (0, 0) = 1$. These priorities usually stem from the subject area and business logic, for example, it is clear that specifying a city is more important than specifying a country.

Before developing a solution for production, we decided to check it “on the knee”, and for good reason. Physically, the FRule mechanism is a set of “parameter – value” correspondences for a function. $$$F$stored in mysql. Based on the priorities of the sets when accessing the FRule, some simple queries are executed until a result is found, which is then cached. For initial configuration tasks, this is more than enough. It turned out that the task of defining options needs to be refined. It turned out that it took 20 minutes to calculate the baggage of all offers on the route Moscow - St. Petersburg for one day. By reviewing the priority system and removing redundant and unused sets from there, we reduced it to 3 minutes. However, it was still a lot.

MySQL was a bottleneck, or rather, the quantity and overhead of requests that we sent to it if there is no result for a specific sentence in the cache. For each sentence, we began to look for coincidences: first on the most particular set, then on to more and more general sets. We give an example to present the scale of a disaster. Now we have 288 sets with different priority. Search proposals for one direction may be hundreds and thousands, and despite the fact that not all of them will be shown to the user, everything is required to be processed.

To solve this problem, we decided to transfer the calculations to the backend. The calculation of values ​​consists of two parts. The first is the construction of some special organized structure. The second is the calculation of values ​​based on this structure.

Consider what kind of structure we are building. Let's go back to our example function. $$$F$from two variables. For her, we defined the following priority sets:

Let me remind you that the topmost set is the most particular - if for all the fields it contains there is an explicitly specified function value, then it will be used. In essence, we will convert a flat table stored in a database into a set of trees for each set of variables. Consider the process of building a set of trees for a set $$$[x_1, x_2]$. First type request

 SELECT * FROM t WHERE x1 IS NOT NULL AND x2 IS NOT NULL 

We retrieve from the database all rows for which we have defined and $$$x_1$and $$$x_2$. Then group them by variable value. $$$x_1$. Then the rules in each group are grouped by variable. $$$x_2$, etc. So for the most private set $$$[x_1, x_2]$build a set of trees:

 [ [1] => [ [1] => 3, [2] => 4 ], [2] => [ [1] => 5 ] ] 

We repeat this procedure for each set of variables. For example, to dial $$$[x_1]$we will select the rules by request

 SELECT * FROM t WHERE x1 IS NOT NULL AND x2 IS NULL 

Unused variables in the set are marked with the value NOT_SPECIFIED. After the sets of trees for each set of parameters are obtained, we add them into one array with preserving the order of priorities of the sets.

As a result, for the function $$$F$we get this set of trees:

 [ [ [1] => [ [1] => 3, [2] => 4 ], [2] => [ [1] => 5 ] ], [ [0] => [ [NOT_SPECIFIED] => 1 ] ], [ [NOT_SPECIFIED] => [ [0] => 2 ] ], [ [NOT_SPECIFIED] => [ [NOT_SPECIFIED] => 6 ] ] ] 

The structure for verification is constructed, it remains only to pass through the input data of the function. We go through the trees from top to bottom and go around each of them, for each variable choosing the branch that corresponds to its value. If there is no such branch, then try to go along the NOT_SPECIFIED branch. If the value is found, then return it.

Data caching

In fact, it would be strange to build a set of trees every time you call FRule - it can be counted and remembered once. In addition, we have several servers on which this task can be performed, so it would be nice to also transfer a set of trees between them. We use memcache to cache and synchronize between servers of computed values. However, contact memcache every time to pick up a set from it, expensive - real sets take up several megabytes and there is no point in driving them over the network. Therefore, in addition to memcache, we also use APCu (and before switching to PHP7, xCache). If the server has built or received a set of trees from memcache, it will save it in APCu and take it from there next time.

There is also a problem with the synchronization of changes - the situation when the rules were changed and applied in one north, and everything else is considered to be still invalid due to old data in APCu in the other. To solve this problem, we store the current version number of the structure in memcache. It consists of a timestamp and a version of the code that built the structure. The same label is also stored in the APCu of each of the backends. Before taking trees from APCu, the backend checks whether the version number in its APCu matches the one stored in memcache. And if it does not match, it rebuilds the structure and installs a new number in memcache and its APCu. When editing rules from the admin, the memcache version number is simply reset: this way all backends will learn that the structure needs to be rebuilt.

What we got in the end

A user for whose issuance there is no data in the cache, the calculation of data on tariff options takes from fractions of a second to 15-20 seconds on very large outputs, where there are many flights and many transfers. Shortly after the backend part went into battle, our front-end developers made luggage filters, exchange and return options, so that customers can immediately cut off uninteresting offers. According to statistics, about 10% of visitors use new filters, and are interested in the parameters of baggage and the return of 15%. Now users do not have to search for the same flight for a long time, but with luggage or the possibility of return. We have learned how to display regional cases, for example, Polar Airlines airline allows for a total of 20 kg of baggage, of which no more than 5 are in hand luggage.

Here is an example of the results of our work. We help to buy tickets not only on traditional routes to St. Petersburg, Sochi or Simferopol, but regional directions are quite popular with us. Let's look at the flight of Alrosa from Krasnoyarsk to Mirny :

When searching, you can see all the details: the ability to exchange or return, and baggage allowance, hand baggage allowance, and even a special scheme for calculating them, when a passenger can take exactly 20 kg of baggage. Often, such information can be obtained either on the airline's website, or only from personal communication with a representative of the airline. It should be noted that it is not only our desire to show all the information honestly and clearly - all these parameters are very important for the buyers themselves when choosing, and without them we get feedback from users that this is not enough.

From a technical point of view, we have a knowledge base containing about 10,000 rules, which largely covers our range. Of course, there are still offers for which we have not yet established rules - especially for those areas and proposals for which we are rarely looked for and bought. In order for content managers to quickly find out about such offers, we have developed a special logging system: it is worth coming up with a value of one or another option that differs from the value in our knowledge base or which does not exist in it, within an hour the content managers talk about it will find out. To keep the database up to date, our content managers constantly interact with the airlines and check if there are any changes on the official websites, call up with their representatives.

The created mechanism is fairly easy to expand: we can start to show information about the amount of surcharge for luggage, the possibility of carrying sports equipment, the amount of bonus miles, etc.