Rating System Notes
Some considerations on the topic of rating systems that I systematized by reading posts on Habré on these topics (not all, of course; I don’t give any links, they are in my favorites), comments on these posts, as well as various relevant links, including These comments are contained. This systematization is not exhaustive and, moreover, not quite systematization, since I also took some of my thoughts from my head. However, some of them are obvious and I probably repeated someone.
1. It is difficult to make a good rating system, the initial version probably will not be very good anyway. The system will have to be finalized in the process, according to the results of user activity. However, it is often too bad to change too much - users will not understand. Therefore, 1) it is better to think over initially. Here the experience of various services to help us. It is necessary to accumulate materials on the topic of rating, make a list of useful comments / considerations both in posts and in comments. 2) To facilitate and accelerate debugging, you can try a stealth chip , as I suggested.
2. A rating system created to apply to a potentially large number of users may not work well on a small number. Because the statistics of estimates for each post will be too small and the influence of accidents is great. And for a small number of users, rating is generally not necessary, the moderator (s) can cope well with them. Therefore, firstly, at the initial stage, the role of “manual” control should be strong. Secondly, the principle of recruitment of initial users is needed accordingly. For example, the first users gathered among their trusted real and Internet friends, rummaged through specialized forums and other sites in the search for adequate and reputable people and invited them, lured by something. And then only register the rest by invites from these first users, and that they were responsible for the given ones with their rating. This initial group of adequate people will create the right atmosphere and tradition. Or another option - everyone can register, but those checked will immediately have a higher rating, features and voice weight. Probably this option is preferable.
')
3. It is necessary to exclude the factor of influence of ratings of those who have already voted on the opinion of those who have not yet voted. Correctly done on Habré, which initially hide the ratings of posts, but I would go further. Firstly, in general, I would never show the ratings of posts and comments to anyone except the author. A message would simply be issued: “Thank you, your vote has been counted.” Secondly, at first, the name of the author would be hidden (accordingly, the influence of his rating and karma on the voting result would be excluded). For example, on the main page, “Live broadcast” contains a tape of all posts, without ratings and authors, only with a rubric by topic. Users have the ability to customize (filter) the live broadcast in their profile to fit their thematic interests. And now the most-rated posts from live broadcast with the name of the author on the main tape.
4. It is necessary to exclude the influence of a stable negative attitude of one user to another. Suppose if someone constantly minus someone for any topics and comments, then immediately see if this coincides with the average rating (majority opinion). If it usually matches, it can mean that the materials are really bad. And if it does not coincide, then most likely the negative evaluations of this user in relation to another specific user should not be taken into account at all (or their weight should be greatly reduced). And also to lower the rating of the voter.
5. But the majority opinion is valuable only if there is no regular synchronized voting of certain groups of users. Those. if a group of users constantly votes almost equally on all topics or comments of a particular author (s), then all votes of this group should be rejected or reduced, as well as their rating should be lowered.
Although paragraphs 4 and 5 will not be so critical if you implement paragraph 3. This is just additional protection in case authorship of texts will be guessed according to a characteristic style (combined with a characteristic theme too). However, if clause 3 is not implemented, then clauses 4 and 5 become, on the contrary, critically important.
6. If there are too few votes (less than a certain threshold value), the vote is considered to have failed and the post hangs in some particular top where many users see it. Or for example in the top of the live broadcast, or even gets on the main (without authorship). There he will gain votes and remain on the main one, or will quickly disappear as a result of a more complete vote. When there are few users on the project, for example, at the beginning of its existence, all posts will practically go to the main one, along with the name. But if the topic has been hanging for a long time, but still they do not vote, it leaves the top without ratings. So such a special topic.
7. We stimulate activity - the rating grows, if there are more publications. And not only for the number of posts - everyone has different “specializations”, not everyone can write good topics. Some are limited to comments. The main thing that these comments were meaningful and interesting. Others generally limit only to the evaluation of topics and comments. It can be divided accordingly - for (positively rated) topics the rating grows the most, for comments less, for voting even less. The problem with the latter is how to evaluate their quality. Since this activity is not evaluated by other users, it is possible to apply a relatively objective criterion, as I suggested - if the evaluated materials really go to the top, then the rating increases, if not, then it falls. If people tend to constantly minus, then the rating drops, and the weight of his voice drops.
In general, I would bind the weight of the vote to the rating - slightly increased the weight of the user's voice with an increase in its rating.
8. The rating must take into account objective criteria. For example, the number of post views, the number of links to it (if not at all on the Internet, then at least within the resource).
9. Voting separately for (karma) of the author and for his topics / comments is a controversial decision. The author in the Internet space is represented mainly by its content, so the author as a person is difficult to evaluate separately from his (meaningful) content. And for non-content content, there are rules and moderators. Although the content, of course, can (and should) be evaluated independently of the authors.
10. The principle “more rating - more opportunities” is good, the only question is what exactly is this opportunity. I would not associate them with restrictions on writing posts / comments. Such restrictions should also be governed by rules and direct moderation. For example, punishment for flooding and oftop. Instead, I would give users with high rating moderator rights. The higher the rating, the more rights. Ratings are especially good in combination with “manual” control. The most high-rated participants could approach on the rights to developers of a resource and its hired managers. Not to mention my favorite trick - to share with the users the money earned. Different options are possible here, the link above contains an example of a link between a rating and money.
11. It is not bad to consider also the contextual dependence of the weight of votes on the competence of the authors and the voters.
12. To divide the vote for the topic on the "nomination" type of content, style, novelty, etc. hardly makes sense, because It is known from experience that in the overwhelming majority of cases people vote according to a double-bed system - plus / minus, like / dislike. Those. they minimize their efforts and do not want to strain themselves once again. Perhaps the exception is the case if you put under the post emoticons, corresponding to different estimates, and different nominations. Then, perhaps, poking at different emoticons is not too lazy - it's not very difficult, but fun. If such a thing stuck, then the overall rating system can be made more complex, multivariate, interesting.
1. It is difficult to make a good rating system, the initial version probably will not be very good anyway. The system will have to be finalized in the process, according to the results of user activity. However, it is often too bad to change too much - users will not understand. Therefore, 1) it is better to think over initially. Here the experience of various services to help us. It is necessary to accumulate materials on the topic of rating, make a list of useful comments / considerations both in posts and in comments. 2) To facilitate and accelerate debugging, you can try a stealth chip , as I suggested.
2. A rating system created to apply to a potentially large number of users may not work well on a small number. Because the statistics of estimates for each post will be too small and the influence of accidents is great. And for a small number of users, rating is generally not necessary, the moderator (s) can cope well with them. Therefore, firstly, at the initial stage, the role of “manual” control should be strong. Secondly, the principle of recruitment of initial users is needed accordingly. For example, the first users gathered among their trusted real and Internet friends, rummaged through specialized forums and other sites in the search for adequate and reputable people and invited them, lured by something. And then only register the rest by invites from these first users, and that they were responsible for the given ones with their rating. This initial group of adequate people will create the right atmosphere and tradition. Or another option - everyone can register, but those checked will immediately have a higher rating, features and voice weight. Probably this option is preferable.
')
3. It is necessary to exclude the factor of influence of ratings of those who have already voted on the opinion of those who have not yet voted. Correctly done on Habré, which initially hide the ratings of posts, but I would go further. Firstly, in general, I would never show the ratings of posts and comments to anyone except the author. A message would simply be issued: “Thank you, your vote has been counted.” Secondly, at first, the name of the author would be hidden (accordingly, the influence of his rating and karma on the voting result would be excluded). For example, on the main page, “Live broadcast” contains a tape of all posts, without ratings and authors, only with a rubric by topic. Users have the ability to customize (filter) the live broadcast in their profile to fit their thematic interests. And now the most-rated posts from live broadcast with the name of the author on the main tape.
4. It is necessary to exclude the influence of a stable negative attitude of one user to another. Suppose if someone constantly minus someone for any topics and comments, then immediately see if this coincides with the average rating (majority opinion). If it usually matches, it can mean that the materials are really bad. And if it does not coincide, then most likely the negative evaluations of this user in relation to another specific user should not be taken into account at all (or their weight should be greatly reduced). And also to lower the rating of the voter.
5. But the majority opinion is valuable only if there is no regular synchronized voting of certain groups of users. Those. if a group of users constantly votes almost equally on all topics or comments of a particular author (s), then all votes of this group should be rejected or reduced, as well as their rating should be lowered.
Although paragraphs 4 and 5 will not be so critical if you implement paragraph 3. This is just additional protection in case authorship of texts will be guessed according to a characteristic style (combined with a characteristic theme too). However, if clause 3 is not implemented, then clauses 4 and 5 become, on the contrary, critically important.
6. If there are too few votes (less than a certain threshold value), the vote is considered to have failed and the post hangs in some particular top where many users see it. Or for example in the top of the live broadcast, or even gets on the main (without authorship). There he will gain votes and remain on the main one, or will quickly disappear as a result of a more complete vote. When there are few users on the project, for example, at the beginning of its existence, all posts will practically go to the main one, along with the name. But if the topic has been hanging for a long time, but still they do not vote, it leaves the top without ratings. So such a special topic.
7. We stimulate activity - the rating grows, if there are more publications. And not only for the number of posts - everyone has different “specializations”, not everyone can write good topics. Some are limited to comments. The main thing that these comments were meaningful and interesting. Others generally limit only to the evaluation of topics and comments. It can be divided accordingly - for (positively rated) topics the rating grows the most, for comments less, for voting even less. The problem with the latter is how to evaluate their quality. Since this activity is not evaluated by other users, it is possible to apply a relatively objective criterion, as I suggested - if the evaluated materials really go to the top, then the rating increases, if not, then it falls. If people tend to constantly minus, then the rating drops, and the weight of his voice drops.
In general, I would bind the weight of the vote to the rating - slightly increased the weight of the user's voice with an increase in its rating.
8. The rating must take into account objective criteria. For example, the number of post views, the number of links to it (if not at all on the Internet, then at least within the resource).
9. Voting separately for (karma) of the author and for his topics / comments is a controversial decision. The author in the Internet space is represented mainly by its content, so the author as a person is difficult to evaluate separately from his (meaningful) content. And for non-content content, there are rules and moderators. Although the content, of course, can (and should) be evaluated independently of the authors.
10. The principle “more rating - more opportunities” is good, the only question is what exactly is this opportunity. I would not associate them with restrictions on writing posts / comments. Such restrictions should also be governed by rules and direct moderation. For example, punishment for flooding and oftop. Instead, I would give users with high rating moderator rights. The higher the rating, the more rights. Ratings are especially good in combination with “manual” control. The most high-rated participants could approach on the rights to developers of a resource and its hired managers. Not to mention my favorite trick - to share with the users the money earned. Different options are possible here, the link above contains an example of a link between a rating and money.
11. It is not bad to consider also the contextual dependence of the weight of votes on the competence of the authors and the voters.
12. To divide the vote for the topic on the "nomination" type of content, style, novelty, etc. hardly makes sense, because It is known from experience that in the overwhelming majority of cases people vote according to a double-bed system - plus / minus, like / dislike. Those. they minimize their efforts and do not want to strain themselves once again. Perhaps the exception is the case if you put under the post emoticons, corresponding to different estimates, and different nominations. Then, perhaps, poking at different emoticons is not too lazy - it's not very difficult, but fun. If such a thing stuck, then the overall rating system can be made more complex, multivariate, interesting.
Source: https://habr.com/ru/post/103065/
All Articles