Hello, doctor, I have problems with karma

"90-60-90", or drive through the city past a traffic cop

I know a sufficient number of services, on which the concept of "karma" is present. And all of these services consider the rating according to some kind of vague algorithm. The introduction of even empirical non-linear dependencies of this indicator on all phases of the moon and the favor of other users leads to a noticeable number of problems. I would like to discuss today about them (and possible ways of solving them).




I will first outline all the problems that are visible to me, and then I will try to make wholesale conclusions.

Problem 1. Weighting factors

Karmic indicators can not simply add and subtract, as, for example, it is done on Habré. This is quite obvious: let's compare two users with the same karma (say, = 50) - Vasya and Petya. At the same time, Vasya got this fifty kopecks by subtracting one minus from fifty one plus points (I did not like the ratio ÷ = 50 ÷ 1). But Pettyr's aksakal has five hundred pluses and 450 minuses (that is, apologists and antagonists shared approximately equally). It seems to me to be very transparent that the karma of Vasya should have been greater than Petya, despite her youth and recent registration.
But if you just stupidly normalize karma to the total number of votes, you’ll get not exactly what we expect: Vasya will become 50 times better than Petit, for whom he has 20 times more people, and he managed to stay in the black. In addition, there may be angry stupid trolls in the amount of three hundred people, who are minus everyone at all, have Petit been in for a long time, and have not called Vasya yet?
')

Problem 2. Objective and subjective indicators

Yes, the indicators are purely objective (98% of those who saw the announcement went under the cut), and purely subjective (Vasya went to Petr in profile and spat in his karma), but the overwhelming majority still represent something in between. For example, the number of bookmarked is a statistically more or less objective indicator, and the number ± for an article is even on a large sample of those who voted closer to the subjective. In addition, when calculating the subjective parameters, it makes sense to take into account the propensities of the voter (if someone spends the entire released resource for minus, its disadvantages should be devalued, and the pros should be more expensive, and vice versa).

Problem 3. Obvious and non-obvious indicators

I consider an obvious indicator, for example, the number of people who added a note to their bookmarks. This is an important parameter, but it cannot be used directly: someone adds to bookmarks every second note, someone - one per year. Therefore, it makes sense to use the parameter, normalized by the tendency to add to the "laying". Something like the normalized reverse frequency of adding bookmarks. Then Vasya, who “lays down” for the first time on the service, and Peter, who is on the service a year and this is his 50th bookmark, will contribute equally to karma, and Kohl (a month on the service, 30 tabs, “Total Recall Syndrome” ) - 7 times smaller.
It makes sense to refer to the unobvious, say, the “atmosphere” in the comments (the total amount of pros / cons). For example, a note in the comments to which there is not a single minus (or their number is negligible) is probably more useful than holivarna. The total number of comments, while it seems to me, can not act as a parameter in any way.

Problem 4. Karma vs. Rating

There are two orthogonal indicators: karma and rating. Karma is a function of subjective non-obvious indicators; rating - on the contrary - a function of objective obvious. It seems to me obvious that the first one is suitable only for championships on measuring the length of the detoxal organ, whereas only objectively obvious indicators can be used to determine the user's “authority”.

Entities

I was able to come up with the following entities, which can be evaluated:

• a record
• comment
• action (± and similar)
• total activity

All subjective assessments should be normalized both to the user himself (the value of the estimate is inversely proportional to the balance of such assessments for a given user), and to their total number.
Now let's try to bring it all together.

Records and comments

Records and comments can be rated and or added to the bookmarks, in addition, they can be given a link both from the outside and from the inside of the habr. The weight of the comment (and associated ratings) is less than that of the record.

Actions and other activities

These two parameters can only be measured.

The summing up

Actually, I do not have a silver bullet. The coefficients need an empirical fit. But I suppose that the formulas for calculating the two parameters I mentioned should look something like this:

Karma = NORM _all (∑ NORM _this (± _{in karma} ))

Rating = NORM _all (∑ NORM _this (± _{for topics} )) + ⅓ × NORM _all (∑ NORM _this (± _{for comments} )) + F (number of external links) + ⅓ × F (number of internal links) + ∑ NORM _this ( in bookmarks)

The weights of each member of this formula must be customized manually.

Constructive as well as destructive criticism is very welcome. I am sure that I missed something important and I hope to supplement the method with your help. Unfortunately, I do not have the opportunity to go through all the records in Habré and calculate these numbers manually (and if you have direct access to the database, this is a couple of queries). If you could persuade the administration - no! Do not change the existing system! - but it would be cool to count these two values ​​for all users. Well, I think so.