Ranking criteria: From each according to ability, to each according to his work (№3)
The purely socialist slogan in the title of this note, the third in the series “How do we build a social web?”, May not be bad at all with regard to Web 2.0. Yes, and if you think well, it’s not so bad for real life. The only trouble is that no one ever intended to use it in this life.
In the previous post of this series (we will mark all notes of the series with their relative sequence number), we looked at the tools with which each registered user can express his attitude to various input messages from other users. In this we will focus on what for, how, when, and to what extent the system (resource, SRI) must use the assigned ratings when ranking information and users.
In the proposed approach to the selection of the main criterion for ranking participants, direct user ratings are either not taken into account at all, or come with less weight, indirect actions by readers, which objectively reflect their real interest in the posts of the participant being evaluated. After all, in fact it is difficult to assume that you will get acquainted with uninteresting and useless materials for you, vote on them and participate in their discussions. In other words, practice (your actions, not intentions and preferences) is the criterion of truth.
What are we talking about
')
In general, three types of objects are subject to ranking procedures in social resources:
1. Key messages (topics / topics)
2. Users
3. Sites (resources - sources of information)
In this case, the initial information for ranking can be:
1. Direct user ratings :
- of this object , for example, as we saw in the previous article in relation to messages
- another object , for example, a message when evaluating its author
2. Indirect actions of users who can say something about the interest to the ranked object
And now we will consider all this more specifically in relation to each of the three types of ranking objects. As always, we will first talk about existing resources, and then what is proposed for a typical intelligent social resource (SRI). At the same time I want to immediately warn you that, speaking about existing resources and in some cases without knowing the exact algorithms of their work, I will rely on my observations. The opacity of ranking algorithms is a matter for a separate discussion.
Ranking of main messages
The rank of a message in an existing resource is defined as:
digg is a certain function of the sum of all user “for ( digg it ) and“ against ”( Bury ) in relation to this message. Such an integral assessment here is called the popularity of the message.
reddit is a certain function of the sum of all user “for ( Up / like ) and“ against ”( Down / dislike ) in relation to this message. However, such a function is considered here only as one of the ranking options. Others are used almost equally, for example, on “controversy” ( Controversy ) and relevance ( Hot )
news2 is a function of the sum of all user “for ( Mark ) and“ against ”( problem? ) in relation to this message. Such an integral assessment here is called the quality of the message .
habrahabr is a certain function of the sum of all user pros and cons of this message. Such an integral estimate here is called Khabrasila
del.icio.us is a function of the sum of all user “for ( save it ) in relation to this message. Such an integral estimate is here called the publicity of the message .
bobrdobr.ru - message ranking is not used (or so it seemed to me?)
I suspect that the values of the pros and cons in some resources are also influenced by the ranks of the voting users. I also think that sometimes the value of such a weight factor is also influenced by the amount of comments to this message. Most likely, the latter assumption is true for reddit when ranking by "inconsistency" and relevance.
Offered for IMR:
1. Ranking by nominations - multicriterial ranking
2. Equality of all nominations - the user himself chooses which nomination suits him more and can even relate such a nomination to his home page
3. The relevance of the message is the most “objective” nomination for messages, although it is considered by the system only as the first among equals. It is by the criterion of relevance that the messages issued to the main page for silence are ranked. As we will see later, this same criterion is the main (if not the only) used in ranking users.
4. Equality of "evaluating" users in terms of the opportunity to express their assessment of any message (not his, of course). As for the weighting coefficients of such an assessment, the status of the voter in certain cases may affect such a weighting factor. But this is planned to devote a separate conversation.
The formulated approach, from my point of view, is a guarantee of a high level of ISR pluralism and prevents the possibility of any group of users ever imposing their “ideological” predilections on others.
First, a few words about the nominations of the ranking of basic messages, each of which is associated with its own criteria. Generally speaking, they can be arbitrarily many. I will give only a few (conditional names; as well as the nomenclature of nominations, this is a matter of the taste of the IRS developers):
- Relevance (popularity, demand) of messages - as we have already noted, this is the main criterion. Therefore, from my point of view, it is mandatory for any IRS. For details on this nomination, see below.
- Generally accepted messages - contain ideas (thoughts, suggestions, conclusions), with which the majority of those who acquainted them agree
- Unacceptability of messages - contain ideas (thoughts, suggestions, conclusions), with which the majority of those who acquainted them do not agree
- Contradictory messages - contain controversial ideas (thoughts, suggestions, conclusions).
Consider the ranking criteria for the relevance of information. The remaining criteria (nomination) ranking will be presented in the following note.
The rank of the message in the nomination relevance is determined as a function of the sum of all user:
1. views of this message. There is, of course, that a user first receives only a general list of messages.
2. direct replies - top level comments on this post
3. votes I agree - I do not agree , no matter how specifically each user voted
4. comments of lower levels associated with this message.
5. votes on these comments , again no matter what they are.
Each argument (sum) has its own weighting factor, the choice of which can play a decisive role in the correctness of the nomination. Without going deep into the subtleties of this question, one can, in general, outline the following priorities (in descending order of the weight coefficients): 2, 1, 3, 4, 5. At the same time, for arguments 4 and 5, associated with lower-level comments, the weighting factor should drop sharply for each next level.
Why do I still prefer comments in relation to the usual views (read) messages. The answer seems simple to me. We are all masters of making flashy annotations , as I, on the advice of a respected sheller, I made to this note. At the same time, it is hard to disagree with the statement of dik, that behind the “tempting preview”, it is often “to hide an open dummy” (see comments to the previous article in habrahabr). I hope this does not apply to my notes :). But the comment, each of us, as a rule, will write only if the material has touched him alive.
As we see, here, direct user assessments are either not considered at all (for example, “like - dislike”), or go with less weight (“I agree, I disagree”) than its indirect actions, which objectively reflect the user's real interest in this post. After all, in fact it is difficult to assume that you will get acquainted with uninteresting and useless materials for you, vote on them and participate in their discussions. In other words, practice (your actions, not intentions and preferences) is the criterion of truth .
Only one example to illustrate this. I have one loyal reader - my sworn friend. No, not the one you thought about - everything is within the limits of the permissible, although sometimes it pulls on Something is wrong . Who he is and where he comes from now has no meaning. The main thing that my friend does not miss almost any of my notes. At the same time he cleans me as soon as he can and puts the corresponding minuses. Almost rhetorical question, are my thoughts relevant for him or not? Or maybe this is just such a modern informational Robin Hood, who has devoted his whole life to protecting the orphaned and poor users of the social web?
It is also easy to note that in the definition of the relevance criterion I follow the well-known rule: “the new is a well-forgotten old”. After all, remember what postings we, first of all, read on forums built on traditional engines like IPB? That's right, those that have most viewed us and commented on others. True, in the forum engines I know in weekend listings, the ordering of messages is rarely made according to the relevance criterion, but the times are different now :). This is, by the way, once again to the hint sounded in my address about the use of the “turned forum”.
And now it will be easier for us to go to the very "sore question of social sphere." On the question of who, how and why we rank with you. Read about it in the extended version of the note , placed in iTech Bridge (Attention! Honestly I warn you, the note again turned out to be quite long) .
In the previous post of this series (we will mark all notes of the series with their relative sequence number), we looked at the tools with which each registered user can express his attitude to various input messages from other users. In this we will focus on what for, how, when, and to what extent the system (resource, SRI) must use the assigned ratings when ranking information and users.
In the proposed approach to the selection of the main criterion for ranking participants, direct user ratings are either not taken into account at all, or come with less weight, indirect actions by readers, which objectively reflect their real interest in the posts of the participant being evaluated. After all, in fact it is difficult to assume that you will get acquainted with uninteresting and useless materials for you, vote on them and participate in their discussions. In other words, practice (your actions, not intentions and preferences) is the criterion of truth.
What are we talking about
')
In general, three types of objects are subject to ranking procedures in social resources:
1. Key messages (topics / topics)
2. Users
3. Sites (resources - sources of information)
In this case, the initial information for ranking can be:
1. Direct user ratings :
- of this object , for example, as we saw in the previous article in relation to messages
- another object , for example, a message when evaluating its author
2. Indirect actions of users who can say something about the interest to the ranked object
And now we will consider all this more specifically in relation to each of the three types of ranking objects. As always, we will first talk about existing resources, and then what is proposed for a typical intelligent social resource (SRI). At the same time I want to immediately warn you that, speaking about existing resources and in some cases without knowing the exact algorithms of their work, I will rely on my observations. The opacity of ranking algorithms is a matter for a separate discussion.
Ranking of main messages
The rank of a message in an existing resource is defined as:
digg is a certain function of the sum of all user “for ( digg it ) and“ against ”( Bury ) in relation to this message. Such an integral assessment here is called the popularity of the message.
reddit is a certain function of the sum of all user “for ( Up / like ) and“ against ”( Down / dislike ) in relation to this message. However, such a function is considered here only as one of the ranking options. Others are used almost equally, for example, on “controversy” ( Controversy ) and relevance ( Hot )
news2 is a function of the sum of all user “for ( Mark ) and“ against ”( problem? ) in relation to this message. Such an integral assessment here is called the quality of the message .
habrahabr is a certain function of the sum of all user pros and cons of this message. Such an integral estimate here is called Khabrasila
del.icio.us is a function of the sum of all user “for ( save it ) in relation to this message. Such an integral estimate is here called the publicity of the message .
bobrdobr.ru - message ranking is not used (or so it seemed to me?)
I suspect that the values of the pros and cons in some resources are also influenced by the ranks of the voting users. I also think that sometimes the value of such a weight factor is also influenced by the amount of comments to this message. Most likely, the latter assumption is true for reddit when ranking by "inconsistency" and relevance.
Offered for IMR:
1. Ranking by nominations - multicriterial ranking
2. Equality of all nominations - the user himself chooses which nomination suits him more and can even relate such a nomination to his home page
3. The relevance of the message is the most “objective” nomination for messages, although it is considered by the system only as the first among equals. It is by the criterion of relevance that the messages issued to the main page for silence are ranked. As we will see later, this same criterion is the main (if not the only) used in ranking users.
4. Equality of "evaluating" users in terms of the opportunity to express their assessment of any message (not his, of course). As for the weighting coefficients of such an assessment, the status of the voter in certain cases may affect such a weighting factor. But this is planned to devote a separate conversation.
The formulated approach, from my point of view, is a guarantee of a high level of ISR pluralism and prevents the possibility of any group of users ever imposing their “ideological” predilections on others.
First, a few words about the nominations of the ranking of basic messages, each of which is associated with its own criteria. Generally speaking, they can be arbitrarily many. I will give only a few (conditional names; as well as the nomenclature of nominations, this is a matter of the taste of the IRS developers):
- Relevance (popularity, demand) of messages - as we have already noted, this is the main criterion. Therefore, from my point of view, it is mandatory for any IRS. For details on this nomination, see below.
- Generally accepted messages - contain ideas (thoughts, suggestions, conclusions), with which the majority of those who acquainted them agree
- Unacceptability of messages - contain ideas (thoughts, suggestions, conclusions), with which the majority of those who acquainted them do not agree
- Contradictory messages - contain controversial ideas (thoughts, suggestions, conclusions).
Consider the ranking criteria for the relevance of information. The remaining criteria (nomination) ranking will be presented in the following note.
The rank of the message in the nomination relevance is determined as a function of the sum of all user:
1. views of this message. There is, of course, that a user first receives only a general list of messages.
2. direct replies - top level comments on this post
3. votes I agree - I do not agree , no matter how specifically each user voted
4. comments of lower levels associated with this message.
5. votes on these comments , again no matter what they are.
Each argument (sum) has its own weighting factor, the choice of which can play a decisive role in the correctness of the nomination. Without going deep into the subtleties of this question, one can, in general, outline the following priorities (in descending order of the weight coefficients): 2, 1, 3, 4, 5. At the same time, for arguments 4 and 5, associated with lower-level comments, the weighting factor should drop sharply for each next level.
Why do I still prefer comments in relation to the usual views (read) messages. The answer seems simple to me. We are all masters of making flashy annotations , as I, on the advice of a respected sheller, I made to this note. At the same time, it is hard to disagree with the statement of dik, that behind the “tempting preview”, it is often “to hide an open dummy” (see comments to the previous article in habrahabr). I hope this does not apply to my notes :). But the comment, each of us, as a rule, will write only if the material has touched him alive.
As we see, here, direct user assessments are either not considered at all (for example, “like - dislike”), or go with less weight (“I agree, I disagree”) than its indirect actions, which objectively reflect the user's real interest in this post. After all, in fact it is difficult to assume that you will get acquainted with uninteresting and useless materials for you, vote on them and participate in their discussions. In other words, practice (your actions, not intentions and preferences) is the criterion of truth .
Only one example to illustrate this. I have one loyal reader - my sworn friend. No, not the one you thought about - everything is within the limits of the permissible, although sometimes it pulls on Something is wrong . Who he is and where he comes from now has no meaning. The main thing that my friend does not miss almost any of my notes. At the same time he cleans me as soon as he can and puts the corresponding minuses. Almost rhetorical question, are my thoughts relevant for him or not? Or maybe this is just such a modern informational Robin Hood, who has devoted his whole life to protecting the orphaned and poor users of the social web?
It is also easy to note that in the definition of the relevance criterion I follow the well-known rule: “the new is a well-forgotten old”. After all, remember what postings we, first of all, read on forums built on traditional engines like IPB? That's right, those that have most viewed us and commented on others. True, in the forum engines I know in weekend listings, the ordering of messages is rarely made according to the relevance criterion, but the times are different now :). This is, by the way, once again to the hint sounded in my address about the use of the “turned forum”.
And now it will be easier for us to go to the very "sore question of social sphere." On the question of who, how and why we rank with you. Read about it in the extended version of the note , placed in iTech Bridge (Attention! Honestly I warn you, the note again turned out to be quite long) .
Source: https://habr.com/ru/post/12531/
All Articles