Late update to the post: 2 months after the post was released, the formula for calculating the rating was amended, which is described in this post on the project blog. They bring the ratings of "Kinopoisk" to those that were before (although this is not a full rollback). The author of this post does not have an unequivocal opinion on whether everything is as it should now, only cautious optimism.

The events around the Kinopoisk reboot seem to be noisy, the users who have left are spending time on the previous site, the developers with parsers make insidious plans, the management has made personnel decisions of ambiguous focus, and everything else has returned to normal. But not all.
')
Rollback to the previous version of the site has not touched movie ratings. And although on the Top 250 we can see a familiar familiarity with many things with “Escape from the Shawshank” and “The Green Mile” as leaders, on the pages of the films themselves, in the Google issue and other places, the rating is displayed - the one that went to the basis of Topa-250 beta (I have it open intermittently, which was observed immediately after the restart).

And after a week nothing has changed, as if hinting to us that the new rating will remain for a long time, if not forever. I don’t exclude that the old top will be updated soon, maybe it keeps only because the rating calculation algorithm differs from the algorithm on the film pages (“in the top, special mechanisms are used that prevent unscrupulous users from cheating the rating”), and this algorithm corrects not a new one, but an old rating (mostly within a couple of hundredths).

General view

Well, let's look at the new rating . If you do not open it, you can use the version I saved . It is a week (I wanted to do this work a long time and started at the beginning of last week, but I managed to finish it only recently, then wrote another post) , but the ratings have changed only slightly (no more than a few thousandths, if we compare the detailed versions) stamp, with one important exception - see below).

What I have to say right away - in all the time, I have not seen a relatively new algorithm for comments by service representatives. According to the redesign, any meaningful comments slipped through, but the circumstances of entering the new rating calculation algorithm are shrouded in deaf fog. The only comment we have is criticism of the founder of Kinopoisk, Vitaly Tatsiya , who expressed concern that “they [the new Kinopoisk team will kill] the only decent rating of films in runet ... and whether all their users will be sent to imdb.

The fact that everyone noticed was a sharp drop by 35 positions of the two leaders mentioned above and the rise of Interstellar from 12th position to 2nd, as well as a scandalous 8.2 points instead of 4.8 in the trash film Green Elephant (and after the flash mob, users who saw this and decided to make it absurd were already 9.1). These changes may seem strange to someone, but they themselves are difficult to evaluate (with the exception, perhaps, of “Elephant”, which I, however, did not watch). Yes, the old rating on these points was closer to the IMDb rating (where Escape from the Shoushenk is also in the first place, and Interstellar on the 29th), but this is still not an argument.

A closer look reveals other strange and controversial things. To begin with - all the films with the number of votes less than 70 thousand were removed from the beta beta (the limit was established by practical consideration). This is not such a trifle: in connection with this decision, 21 films fly out of the top 100, 152nd place moves to the 100th, and 489th to the 224th (the full version of the new top without these exemptions, see this list on the old “Kinopoisk”, sorted by rating ). 70 thousand votes is more than a serious border; it has not yet been overcome, including the acclaimed "Leviathan" or launched in two weeks ago in the Russian box office and a month and a half ago in the world "Martian" . At the same time, in the old top the border was 500, and I did not hear any special complaints about it in this regard.


The arrows connect positions in the same top with a threshold of 70 thousand votes and without

Such a decision could be made from commercial considerations: less popular films are less likely to be available for online viewing at Yandex partners. And indeed: films with fewer than 70 thousand votes in the first hundred top beta versions are usually made far in the past century and are either available for free from Yandex partners or not available; There is no paid one. (To verify this, you can use the regular expression ^[^\t]+\t((\d\d?|100)\t|[^\t]+\t(\d\d?|100)\t).+$ on this table , which is described below under the spoiler. Movies outside the top are those with a dash in the second and third columns.)

Let us turn to the ratings themselves. Since the top beta version displays ratings up to a tenth instead of a thousandth, instead it’s better to work with the mentioned list on the old Kinopoisk .

Films from the 1st to the 42nd position cover the range from 9.465 to 9,000 points, moreover, in three films the rating is exactly 9,000, and the next one has a rating of 8,299 . Then follow immediately 55 films in the range from 8,271 to 8,200.



Wow a failure of 0.7 points! In turn, “nine exactly” - these are clearly traces of some kind of either external intervention, or incomprehensible rounding; the rest gives just some fancy ratio of factors in the formula, generating an uneven distribution.

And now pay attention that these are three films with 9,000: these are the very fallen “Shawshank Escape” and “The Green Mile”, and with them for the company and exchanging 5th place for the 38th “1 + 1” . Ahead of everything that will be said later, you can immediately draw attention to the fact that “Escape from the Shoushenk” and “1 + 1” are films available for free viewing at Yandex partners.

Further more. During the few days that I worked with the top, I fell to witness the miraculous metamorphosis with the rating of another film - "Chorus." He fell from 8.745 (the same “blind zone” for other films) to 8.260 in a couple of days. I even managed to capture this incredible “journey” on webcitation.org with the help of the Bing cache: here the movie has a rating of 8.745 , here it is 8.515 , and at the moment it has 8.260 . I feel like a paranormal hunter. No other movies have a rating even at close speed. This film is not available from partners of "Yandex" neither for free nor for a fee.


Another strange detail - the new ratings change in a strange relationship with the change of the old. Top for October 15 and yesterday , the films "Leon" and "Interstellar"

Preparation for calculations

I must say that at first I took the new rating as technical problems of the new Kinopoisk team, without seriously considering the version about its deliberate distortion. This is simply not the style of "Yandex" - a company that generally keeps its reputation; such manipulations for it must be the strictest taboos. But after the first alarm bells, suspicions began to creep in. And then I decided to sort things out.

And if we have two tops - new and old - and even both ratings for each film, as well as data on the availability of these films for paid / free viewing from Yandex partners, the solution suggests a simple one: to compare the positions of all films in one top and in another, and before the heap - and their ratings, calculate the arithmetic average of paid, free and all other films and compare. Of course, this method does not pretend to the accuracy of the findings, but it is capable of a general impression.

Simple arithmetic

Get down to business. In this place it is worthwhile to stipulate that what I have done can be repeated by everyone. Those interested can do it not for 100, but for all 250 films (if they can overcome the above difficulties). At the bottom of the post I gave regular expressions that allow you to select from the table above under the spoiler, in case you want to work with it.

Positions

1. First, let's see how the positions between the two tops have changed as is, i.e. without agreeing on the criteria for inclusion in the top (cutting out films from an old top with less than 70 thousand votes and subsequent position adjustments, see above under the spoiler).

All calculations in expanded form are shown here . (It would probably be more convenient to turn all this into a spreadsheet, but it was too lazy to remember them; regulars + multiple cursors + copy-to-calculator are also nothing.)

• The average position of all 102 films shifted to 16.12 up (due to the fact that low positions in the unadapted old top correspond to high in the new, but not vice versa).
• Each of the 29 pay- per-view movies shifted an average of 24.89 positions up.
• Each of the 21 films with free viewing shifted an average of 5.77 positions up.
• Each of the 52 inaccessible films has shifted on average by 15.43 positions up.

And the most important thing:

• The difference between the growth of paid films and the average height was 8.77 positions.
• The difference between the growth of free movies and the average height was −10.35 positions.

2. Now let's see what is the participation of the rating itself, and not the rating + criteria for inclusion in the top. Compare the position between the new top and the old one adapted for the harmonization of criteria .

Calculations - here .

• The average position of all 108 films shifted up by 0.96 .
• Each of the 30 pay- per-view movies shifted an average of 5.97 positions up.
• Each of the 24 movies with free viewing shifted an average of 5.91 positions down .
• Each of the 54 inaccessible films has shifted an average of 1.24 positions up.

Total:

• The difference between the growth of paid films and the average height was 5.01 positions.
• The difference between the growth of free movies and the average height was −6.87 positions.

Ratings

3. Since all films have ratings regardless of the number of votes, we can compare movie ratings in the aggregate sample of both top 100 directly. The number of films in the sample increases. Compare the new top with unadapted old .

Calculations .

• The average rating of all 120 films increased by 0.142 points.
• The rating of each of the 29 films with paid viewing has increased by an average of 0.232 points.
• The rating of each of the 28 films with free viewing has increased by an average of 0,023 points.
• The rating of each of the 63 inaccessible films has increased by an average of 0.154 points.

Total:

• The difference between the growth of paid films and the average height was 0.090 * points.
• The difference between the growth of free movies and the average height was −0.119 * points.

* - The dramatic change in the rating of “Choristov”, described above, will increase these values ​​by several thousandths.

Nine hundredth is not so little as it may seem. If the average rating on the old “Kinopoisk” is 7.2, and the maximum is 9.2, and if you consider all movies from 7 and above to have to watch, this is not less than 4% of the “watchable zone” range.

4. Finally, take a sample from the second comparison and apply it to the ratings. We compare the ratings between the new top and the old one adapted for the harmonization of criteria .

Calculations .

• The average rating of all 108 films increased by 0.142 points (as in the last paragraph).
• The rating of each of the 30 pay- per-view films has grown by an average of 0.223 points.
• The rating of each of the 24 films with free viewing has increased by an average of 0.054 points.
• The rating of each of the 54 inaccessible films has increased by an average of 0.136 points.

Total:

• The difference between the growth of paid films and the average height was 0.081 points.
• The difference between the growth of free movies and the average height was −0.088 points.

(There are no “choristers” in this sample.)

All calculations have been re-checked.

findings

Reasonable doubt

In principle, one paid film, which went 150 positions up, would give the result shown in the second comparison, and this could be considered an accident. But there are no such huge differences in positions between the two tops. The maximum difference in paid films is 63 positions, won back by “Mad Max: Dear Rage”, and the rise of Mad Max is compensated, for example, by the fall of “White Captivity” by 38 positions.

Also, probably, it would be possible to calculate how much such a deviation is likely if the calculation formula would give the films random positions within certain limits. Due to my lack of great knowledge in statistics, I leave it to those who have it. But I have big doubts that such a probability is somehow great. (Coarsely, one can be convinced of the uniformity of deviation by counting indicators separately for the top 50 and films from 51 to 100 positions according to the same principles; what I did is that the conclusions remain (growth 4.44 and 8.19 for paid ones in the second sample; the second value is greater, obviously, because the jumps on the top are stronger).)

In itself, raising the ratings of paid films and sagging free ones does not mean intent - there may be unobvious factors that by pure coincidence increase the value of some variables and lower the others. Relatively speaking, it is possible to imagine that for old Soviet films, which most often have the possibility of free viewing, there are some peculiarities of putting users in their ratings, which, from the point of view of the new algorithm, play down the rating of these films. On the other hand, the mere fact that these coincidences mysteriously play into the hands of the commercial interests of the company (and not, for example, against its interests) in any case cannot be ignored by the public.

What else? Since correlation does not mean a causal relationship, the inverse relationship hypothesis comes to mind.
Could it have been so that the new top was given to the partners of Yandex, and they drove their assortment under it?

Cynical version

Kinopoisk visitors, movie lovers, know what these guys are.

There is a more trivial explanation. It sounds like this:

The effective managers of Yandex are very effective and very strong in metrics, analysis and calculations.They were so tempted to optimize everything around that they not only nibbled the site’s functionality, having excluded everything that does not bring direct income , but even the opportunity to push the rating of paid films up to the tenth point did not miss. At the same time, it is reasonable to reason that if we correct the rating, it is unnoticeable, so as not to provoke mass discontent, be it clumsy. In fact - will anyone really sit with a calculator and calculate the difference in positions and rating for each film ?? OH SH ...

And so 9 positions in the top 100 or one-tenth rating points - it would seem nonsense - are converted into quite tangible thousands and millions of wood.

Total

From myself I’ll say that I was not convinced by the version for reasonable doubt, and I tend to the second, although I can only guess what logic the decision-makers had. However, in the light of the recent insider, I can almost imagine it.

Yes, and I can hardly imagine why, in principle , a rating “adjustment” may be necessary. I would like to ask: “What are you going to adjust? An objective opinion of tens and hundreds of thousands? " (And yet - and this already repeats the" big "story with the restarting of" Film Search "- " Why don't you need to give any comments on this? " But God bless him ...) What was not so with the old one?

Clearing up? But in the old top it has already been applied, and cope with the wrap can be a more direct way - removing fictitious ratings along with the users who put them. Yes, and I doubt that someone might need to wind the "Escape from the Shawshank."

Nevertheless, all this in any case will only suspicion. Therefore, it would be interesting to listen to the position of Yandex itself. I do not exclude that there is some more trivial explanation that escaped my attention.

Anyway, this whole undertaking with a new rating doesn’t cause anything but bewilderment, both for me and for everyone else whose opinion I had to face, whether on objective grounds or on the basis of purely subjective feelings. Therefore, as well as the results of the foregoingI urge Yandex to abandon the new algorithm in order not to undermine the credibility of the unique rating of films, TV series, etc. in Runet. - a worthy alternative to the IMDb rating, as well as to the Kinopoisk and Yandex.

In order not to end on a bad note, I will say that in the process of working with the top of the new site, I was able to evaluate, in my opinion, the function of short tags, which is far better than the notation , unjustly ridiculed in the memorable post andorro “What's wrong with the launch of“ Kinopoisk ” the genre makes it clear what this or that film is about, as well as the title pictures of films instead of advertising posters.

Appendage

Saved by me in the web archive:

• Classic Top 250
• Top-250 beta versions are not saved anywhere due to muddy redirects, and in any case there is uploading of 12 films. Once again the link to the saved me locally .
• Top 500, sorted by new rating: 1 , 2
• Top 500, sorted by classic rating, beyond 250 (for two films): 3 , 4

(I, in my opinion, took data from slightly earlier versions.)

Regulars for choosing from the table by the link above all films:

• first sample: ^((\d\d?|100)\t\d|[^\t]+\t[^\t]+\t(\d\d?|100)\t).+$
• second (it’s fourth) sample: ^[^\t]+\t((\d\d?|100)\t|[^\t]+\t(\d\d?|100)\t).+$
• third sample: ^((\d\d?|100)\t|[^\t]+\t[^\t]+\t(\d\d?|100)\t).+$