Personalization for search services
This paper sets out the principles for the use of personalization technology, taking into account the psychological characteristics of users, when forming the issuance of information retrieval.
The purpose of this technology is to create for the user a personal comfort space on the Internet in general, and a positive experience of interaction with the search service in particular. As a result, search services receive tools to optimize the use of their resources.
Currently, for each user information request, the search engine finds thousands of resources. How to determine exactly what resources will interest the user to assign them a higher rank among all others? Solving the issue of the relevance of issuing an information request is one of the priority tasks for search services, and not because they want to save the user's time, but because of the resource-intensive process. Since existing systems now need to periodically scan and index all pages on the Internet, determine their popularity with reference to users' search requests, store information about all requests from all users, to sometimes refer to the query history, the issue of optimizing resource use becomes highly relevant.
One of the optimization options is a personalized search, designed to save the user from the need to spend time turning the pages of the issue, and the service - from "pulling up" all new and new pages.
Until recently, personal search used some data about the user (whether short-term or long-term search history, interests or anything else) to increase the relevance of the issue or did not use those.
This technology is universal for application in various spheres of the Internet activity, united by two concepts “person - digital information”. This could include recommendation systems, advertising, information retrieval, etc.
The described personalization technology is based on 2 basic principles. The first is that the needs of a person determine the vector of his activity (in other words, the behavior) that is conscious or unconscious. The exceptions are some mental disorders, but we will not take them into account. The second principle is the reactivity of the psyche, that is, when exposed to a stimulus, the psyche gives out some definite reaction to it.
In relation to human behavior on the Internet, these principles form a chain - “the search for relevant information - the consumption of information”. (It should be noted, the first - in this case, the word “actual” reflects the situational nature of the phenomenon. A person has a number of needs, updating one or several of which set the situational vector of behavior; second, even if a person goes on the Internet to communicate, this does not change essence - there is a satisfaction of a specific need, and the communication itself acts as a way to achieve a result). In other words, the user purposefully performs internet surfing (a sequence of transitions from page to page, and in fact - search), and the psyche for each unit of content gives a response - positive (suitable) or negative (not suitable). In the case of a positive reaction, surfing is interrupted and “content consumption” occurs.
Initially, the problem of personalized search lies at the junction of two areas of science - psychology and information technology. In this case, psychology must answer the questions - which elements of content and how should be taken into account when forming the issue for an information request, so that the issue could be called personalized, that is, satisfying the needs of this user personally. Information technologies, in turn, should provide the tools (algorithms) for isolating and interpreting these elements.
The proposed approach is to build a model of the user's psyche, which will make it possible, with a high degree of probability, to predict the user's reaction to the offered content unit. (In this case, “high probability” means a probability equal to or greater than 80%).
Building a model of the psyche of a particular user is carried out according to the following algorithm:
1) Registration of the user's reaction to the content unit. As a positive reaction, both the fact of clicking on the link (or indicating the resource in the address bar of the browser by the user), and the time spent on the page are taken into account.
2) Isolation in the link and its description (if there was a link following the link) or in the subject of the resource (if the browser’s address bar was used) semantic units (by “semantic units” here and below are certain ideas) that the user’s mind could react to . For this purpose, a semantic network was developed, which describes the connection between certain phenomena of the external world and their reflection in the internal (mental) plane of a person, until the identification of congenital or formed during the ontogeny of personal characteristics that determine precisely this variant of reflection.
3) Construction of a model that contains not only categories that are determined directly by the reaction to certain content elements, but also, mainly, the deep-seated personality characteristics associated with them (inborn, acquired during ontogenesis), but also [categories] related to actual cultural and informational space (accepted social norms, belonging to one or another social class, etc.).
')
However, as it turned out in practice, using the model of the psyche in this form for solving applied problems turned out to be difficult. Therefore, it was proposed to create a user profile that will contain a number of user characteristics, reflecting his reactions to a limited number of semantic units that are relevant to search behavior. The user profile is a simplified version of the psyche model that is easy to apply for solving applied problems.
After analyzing with the help of the expert method, requests for search activity of users, provided by Yandex to free access (due to the competition), it was possible to distinguish 9 main groups of queries, such as request for instructions, receiving on the Internet, selection, etc.
The analysis took into account the “idea” of the request, ignoring the form of expression of the given idea.
For each group, a glossary was compiled by an expert method. The words in the glossaries of different groups of queries did not intersect. Using this glossary, you can automatically define a group by the query keywords.
Also, for each group, a specific scale system was developed by an expert method. These scales reflect the invariance of the significant characteristics of the information that will be contained in the responses to queries related to this group. For example, the following scales were defined for the group “Request for Instruction”: level of intelligence, theory-practice, superficiality-depth, imagery, randomness-structuredness, and others. Different groups include different numbers of scales.
In view of the personality traits, each user has specific expectations from the response to the request, that is, each user “expects” to receive information with a certain set of characteristics. An example for a request for a new smartphone - if a young girl makes a similar request, then she most likely expects to see resources with a large number of photos in order to appreciate the design, color scheme, etc., since for her the smartphone is a trendy accessory. If this request makes a geek, then it will wait for technical information in order to evaluate the possibilities of the new product and compare it with its counterparts, if there are any.
The only exception is the situation of using a search engine as an analogue of a calculator, a slide rule or any other device, the information output in which is maximally unified in terms of interpretation and is usually done in the form of symbols (numbers and / or conventional symbols).
Based on the above, we want to emphasize 2 points: the first - the scales reflect the characteristics of the information; the second - the user's personal characteristics determine his expectations.
Consequently, personal search, in essence, comes down to defining the expectations of a particular user and offering him information with the necessary characteristics. This procedure does not affect the subject of the search.
The proposed technology of personalization on the one hand makes it possible to determine the expectations of a particular user. For this purpose, the history of user requests is analyzed and its profile is formed, where all the relevant scales for the given user will be displayed for each group of requests. On the other hand, the technology offers tools for characterizing information.
Both the formation of the initial user profile and its subsequent recalculations, and the characterization of information can be done in offline mode, which allows reducing operations in real time to calculating the current index of the [required] query group in the user profile and ranking the resources according to the maximum compliance with the calculated index.
To assess the possibility of using the proposed technology for the purposes of personalized search, an experiment was conducted. As a working hypothesis, the assumption was chosen that the use of this personalization technology in the search service increases the relevance of the issue on the information request, which means that the rank of the resource selected by the user will be less.
Sample Description
The experiment was conducted on a sample of data provided by Yandex in open access as part of an open competition for personalized network search (Personalized Wed Search Challenge). This “base” contained information about search activity ~ 41 thousand. users for a period of 60 days. The following data was suggested:
- User ID
- Request ID
- text of the user's search query
- unified search query text
- the list of resources on the first page of issue (or the first 10 resources) and the corresponding rank
- rank of the resource the user clicked on
Due to limited production capacities, the sample was reduced to a sample of up to 4 500 people randomly selected by the system.
As a result, the sampling parameters were as follows:
- number of users - 4 500 people.
- accounting period of activity - 60 days (from 19-09-13 to 17-11-13)
- total number of requests - 1 104 347
Experiment Description
The modified A / B test was chosen as the method for the experiment. The modification was that the division into experimental and control groups was carried out not among the subjects, but among the objects. In other words, the search activity of users, rather than the users themselves, was divided into groups. This was due to the fact that this approach implies the formation of a profile for each user with his personal characteristics.
At the 1st stage of the experiment, the system categorized requests, that is, assignment to one of the above 9 groups.
In the sample, selected groups blocked 48.7% (538,441) of all requests. The remaining 51.3% (565,906) included queries that the system could not identify due to grammatical errors, the use of transliteration, the use of words that were not included in any of the glossaries.
At the 2nd stage, the possibility of forming a personal profile for users of the sample was assessed. As for the formation of a functional profile within the framework of the proposed approach, there is a lower limit for user search activity (in our case, 40 queries), the system discarded users who do not meet this criterion and their queries. As a result, the sample decreased to 3,826 people. with a total number of requests - 523,007.
This stage is due solely to limited data. Search engines for this problem should not exist.
At the 3rd stage, there was a separation of requests for the experimental and control groups.
It was decided to divide into groups in the ratio of 80:20, that is, 80% of the activity of each user fall into the experimental group, 20% - in the control group.
As a result, in the experimental group it turned out 418 406 requests, and in the control group - 104 601.
At the 4th stage, requests from the experimental group (418,406 requests) were processed by the system according to the following scheme: definition of the request group -> assessment of the severity of significant scales for the resource selected by the user.
The definition of a query group was carried out by comparing query keywords with glossaries of query groups. Then the system determined significant scales for this group of requests and evaluated their severity. We cannot disclose the principles and mechanism for assessing expression, since this information is a trade secret, but we can say that they are accessible to machine learning.
According to the results of processing requests in the experimental group, user profiles were formed. In the profile for each of the groups of requests, the names of the scales relevant to the user and the estimated coefficients (the level of priority of the scale for the user) were indicated.
It should be noted that due to the limited design capacity, the system handled resources as a whole, rather than specific resource pages selected by users. For example, if the user selected slovo.ws/resh/007 , then the system analyzed slovo.ws . This fact, firstly, led to the fact that all content aggregators, such as social networks, Youtube, etc., fell out of processing due to the wide variety of content on them; secondly, this could not but affect the results of the experiment in the direction of their deterioration.
At the 5th stage, requests from the control group (104,601 queries) were processed by the system according to the following scheme: defining the query group -> determining the severity of significant scales -> calculating the index of compliance with the personal profile -> ranking resources in the search results by the correspondence index -> determining the rank of the resource selected by the user.
The definition of the query group was carried out, as in the previous step, by comparing the query keywords with glossaries of query groups. Then the system determined significant scales for this group of requests and evaluated their severity. After that, the compliance index was calculated for each resource from the search results (the first 10 resources). This index reflected how much the scales themselves and their expressiveness of the resource in the output correspond to the scales significant for the user for this group of requests in his personal profile. Based on the calculated compliance indexes, the system made a ranked list of resources in search results and determined the rank of the resource selected by the user.
At the final stage, there was an analysis of requests by the criterion of informativeness. After screening off non-informative queries, the average ranks of the selected resources were calculated and compared in the search service results and in the lists ranked by personal match indexes.
When analyzing requests for their informativeness, the following were eliminated:
a) queries when the user selected resources in the search results strictly in rank order, for example, (resource ranks) 1 * ; 12; 1, 2, 3; 1, 2, 3, 4; 1, 2, 3, 4, 5, etc., because, as we believe, this behavior does not reflect personal preferences, but rather is a consequence of stereotyping the perception of ranked information from the most significant (from the top) to the less significant (below). list). One could assume that in such a case the last resource is the most relevant for the user (after reading it, the user stopped / changed the search activity for the given request). In this case, it was necessary to analyze (compare) the following time parameters, - the time between the presentation of the search results and the first choice of the user, the time of interaction with the content, the time of the next request - and the parameters of the content uniqueness, - the presence of non-overlapping information, style of presentation, etc. We did not conduct it, as there were no time parameters in the database provided by the search service in open access, and the analysis of the uniqueness of the content required significant computational data. but it was useless without temporary data.
* - to determine that the user chose resource # 1 because of meeting his expectations (that is, the choice determined personal characteristics of the user), and not because this is a sample action, it is necessary to analyze the time between the presentation of the search results and click on the link. The time interval should be sufficient for comparing the parameters of resource No. 1 (wording of the link phrase, the contents of the snippet, address of the resource) with the corresponding parameters at least following in the list of resource No. 2. As mentioned above, there were no time parameters in the database provided by the search service.
The fact that search services have complete information on the query history can be used to determine the presence of patterned actions in the search behavior of the user and further optimize the use of personal search technology.
b) queries, if at least one of the resources in the corresponding search results had no compliance index. Despite the fact that the absence of an index could not increase the effectiveness of the proposed approach, as it placed the resources down the list, however, the elections, which were determined by the personal characteristics of users, and not the imperfections of the equipment, were the subject of analysis.
c) requests, where in the search results there were links to different pages of the same resource. Since the system analyzed not specific pages, but resources in general, it was not possible to separate different pages of the same resource when compiling a ranked list.
As a result, the calculation of average ranks was carried out on 74,279 requests (~ 71% of the total number of requests in the control group). The average rank of the selected resources in the issuance of a search service was 3.6. The value of the average rank in the lists, ranked by indices of personal matches, was 2.9. That is, the indicator improved by ~ 19.4%, which for the control group as a whole (including users for whom a profile was not formed) yielded a result of ~ 16.1%.
Interpretation of results
The results of the experiment showed that the use of personalization technology in the field of information retrieval increases the relevance of search results, which can be indirectly judged by the decrease in the rank of the resource selected by the user. In our case, the efficiency in the control group increased by ~ 16.1%.
We quite realistically look at the result. So, for example, having more data on user activity (specifically, time parameters) would not have to screen out all requests, where the user selected resources in the ranking sequence, and only a part from users with sample search behavior. This would undoubtedly reduce the final result. On the other hand, if we had more productive resources, we could analyze specific pages in search results, and not resources in general. This would slightly increase the efficiency, since it would allow to take into account a number of factors that, in our opinion, are important when choosing a source of information (unique information, style of presentation, etc.).
In our opinion, the proposed personalization technology for search services has several advantages designed to optimize the performance of search services. First, building a model of the user's psyche provides a high degree of predictability of this approach. Secondly, part of the operations can be displayed in offline mode, which allows optimal allocation of resources and reduce the load during online activity periods, for example, determining the severity of scales for resources and / or individual pages.
In the case of users with stereotypical perception of ranked information, the use of this approach will not affect the relevance of the issue directly (with or without personalized search, such users view resources according to the rank order). However, if we assume that such users stop / change search activity when a resource is found that meets their expectations, then using personal preferences will allow you to “raise” this resource higher in search results, thereby creating a positive experience of interaction with the search service.
The article is written in collaboration: Lepikhov Sergey, Golovan Alexander.
The purpose of this technology is to create for the user a personal comfort space on the Internet in general, and a positive experience of interaction with the search service in particular. As a result, search services receive tools to optimize the use of their resources.
Currently, for each user information request, the search engine finds thousands of resources. How to determine exactly what resources will interest the user to assign them a higher rank among all others? Solving the issue of the relevance of issuing an information request is one of the priority tasks for search services, and not because they want to save the user's time, but because of the resource-intensive process. Since existing systems now need to periodically scan and index all pages on the Internet, determine their popularity with reference to users' search requests, store information about all requests from all users, to sometimes refer to the query history, the issue of optimizing resource use becomes highly relevant.
One of the optimization options is a personalized search, designed to save the user from the need to spend time turning the pages of the issue, and the service - from "pulling up" all new and new pages.
Until recently, personal search used some data about the user (whether short-term or long-term search history, interests or anything else) to increase the relevance of the issue or did not use those.
This technology is universal for application in various spheres of the Internet activity, united by two concepts “person - digital information”. This could include recommendation systems, advertising, information retrieval, etc.
The described personalization technology is based on 2 basic principles. The first is that the needs of a person determine the vector of his activity (in other words, the behavior) that is conscious or unconscious. The exceptions are some mental disorders, but we will not take them into account. The second principle is the reactivity of the psyche, that is, when exposed to a stimulus, the psyche gives out some definite reaction to it.
In relation to human behavior on the Internet, these principles form a chain - “the search for relevant information - the consumption of information”. (It should be noted, the first - in this case, the word “actual” reflects the situational nature of the phenomenon. A person has a number of needs, updating one or several of which set the situational vector of behavior; second, even if a person goes on the Internet to communicate, this does not change essence - there is a satisfaction of a specific need, and the communication itself acts as a way to achieve a result). In other words, the user purposefully performs internet surfing (a sequence of transitions from page to page, and in fact - search), and the psyche for each unit of content gives a response - positive (suitable) or negative (not suitable). In the case of a positive reaction, surfing is interrupted and “content consumption” occurs.
Initially, the problem of personalized search lies at the junction of two areas of science - psychology and information technology. In this case, psychology must answer the questions - which elements of content and how should be taken into account when forming the issue for an information request, so that the issue could be called personalized, that is, satisfying the needs of this user personally. Information technologies, in turn, should provide the tools (algorithms) for isolating and interpreting these elements.
The proposed approach is to build a model of the user's psyche, which will make it possible, with a high degree of probability, to predict the user's reaction to the offered content unit. (In this case, “high probability” means a probability equal to or greater than 80%).
Building a model of the psyche of a particular user is carried out according to the following algorithm:
1) Registration of the user's reaction to the content unit. As a positive reaction, both the fact of clicking on the link (or indicating the resource in the address bar of the browser by the user), and the time spent on the page are taken into account.
2) Isolation in the link and its description (if there was a link following the link) or in the subject of the resource (if the browser’s address bar was used) semantic units (by “semantic units” here and below are certain ideas) that the user’s mind could react to . For this purpose, a semantic network was developed, which describes the connection between certain phenomena of the external world and their reflection in the internal (mental) plane of a person, until the identification of congenital or formed during the ontogeny of personal characteristics that determine precisely this variant of reflection.
3) Construction of a model that contains not only categories that are determined directly by the reaction to certain content elements, but also, mainly, the deep-seated personality characteristics associated with them (inborn, acquired during ontogenesis), but also [categories] related to actual cultural and informational space (accepted social norms, belonging to one or another social class, etc.).
')
However, as it turned out in practice, using the model of the psyche in this form for solving applied problems turned out to be difficult. Therefore, it was proposed to create a user profile that will contain a number of user characteristics, reflecting his reactions to a limited number of semantic units that are relevant to search behavior. The user profile is a simplified version of the psyche model that is easy to apply for solving applied problems.
After analyzing with the help of the expert method, requests for search activity of users, provided by Yandex to free access (due to the competition), it was possible to distinguish 9 main groups of queries, such as request for instructions, receiving on the Internet, selection, etc.
The analysis took into account the “idea” of the request, ignoring the form of expression of the given idea.
For each group, a glossary was compiled by an expert method. The words in the glossaries of different groups of queries did not intersect. Using this glossary, you can automatically define a group by the query keywords.
Also, for each group, a specific scale system was developed by an expert method. These scales reflect the invariance of the significant characteristics of the information that will be contained in the responses to queries related to this group. For example, the following scales were defined for the group “Request for Instruction”: level of intelligence, theory-practice, superficiality-depth, imagery, randomness-structuredness, and others. Different groups include different numbers of scales.
In view of the personality traits, each user has specific expectations from the response to the request, that is, each user “expects” to receive information with a certain set of characteristics. An example for a request for a new smartphone - if a young girl makes a similar request, then she most likely expects to see resources with a large number of photos in order to appreciate the design, color scheme, etc., since for her the smartphone is a trendy accessory. If this request makes a geek, then it will wait for technical information in order to evaluate the possibilities of the new product and compare it with its counterparts, if there are any.
The only exception is the situation of using a search engine as an analogue of a calculator, a slide rule or any other device, the information output in which is maximally unified in terms of interpretation and is usually done in the form of symbols (numbers and / or conventional symbols).
Based on the above, we want to emphasize 2 points: the first - the scales reflect the characteristics of the information; the second - the user's personal characteristics determine his expectations.
Consequently, personal search, in essence, comes down to defining the expectations of a particular user and offering him information with the necessary characteristics. This procedure does not affect the subject of the search.
The proposed technology of personalization on the one hand makes it possible to determine the expectations of a particular user. For this purpose, the history of user requests is analyzed and its profile is formed, where all the relevant scales for the given user will be displayed for each group of requests. On the other hand, the technology offers tools for characterizing information.
Both the formation of the initial user profile and its subsequent recalculations, and the characterization of information can be done in offline mode, which allows reducing operations in real time to calculating the current index of the [required] query group in the user profile and ranking the resources according to the maximum compliance with the calculated index.
To assess the possibility of using the proposed technology for the purposes of personalized search, an experiment was conducted. As a working hypothesis, the assumption was chosen that the use of this personalization technology in the search service increases the relevance of the issue on the information request, which means that the rank of the resource selected by the user will be less.
Sample Description
The experiment was conducted on a sample of data provided by Yandex in open access as part of an open competition for personalized network search (Personalized Wed Search Challenge). This “base” contained information about search activity ~ 41 thousand. users for a period of 60 days. The following data was suggested:
- User ID
- Request ID
- text of the user's search query
- unified search query text
- the list of resources on the first page of issue (or the first 10 resources) and the corresponding rank
- rank of the resource the user clicked on
Due to limited production capacities, the sample was reduced to a sample of up to 4 500 people randomly selected by the system.
As a result, the sampling parameters were as follows:
- number of users - 4 500 people.
- accounting period of activity - 60 days (from 19-09-13 to 17-11-13)
- total number of requests - 1 104 347
Experiment Description
The modified A / B test was chosen as the method for the experiment. The modification was that the division into experimental and control groups was carried out not among the subjects, but among the objects. In other words, the search activity of users, rather than the users themselves, was divided into groups. This was due to the fact that this approach implies the formation of a profile for each user with his personal characteristics.
At the 1st stage of the experiment, the system categorized requests, that is, assignment to one of the above 9 groups.
In the sample, selected groups blocked 48.7% (538,441) of all requests. The remaining 51.3% (565,906) included queries that the system could not identify due to grammatical errors, the use of transliteration, the use of words that were not included in any of the glossaries.
At the 2nd stage, the possibility of forming a personal profile for users of the sample was assessed. As for the formation of a functional profile within the framework of the proposed approach, there is a lower limit for user search activity (in our case, 40 queries), the system discarded users who do not meet this criterion and their queries. As a result, the sample decreased to 3,826 people. with a total number of requests - 523,007.
This stage is due solely to limited data. Search engines for this problem should not exist.
At the 3rd stage, there was a separation of requests for the experimental and control groups.
It was decided to divide into groups in the ratio of 80:20, that is, 80% of the activity of each user fall into the experimental group, 20% - in the control group.
As a result, in the experimental group it turned out 418 406 requests, and in the control group - 104 601.
At the 4th stage, requests from the experimental group (418,406 requests) were processed by the system according to the following scheme: definition of the request group -> assessment of the severity of significant scales for the resource selected by the user.
The definition of a query group was carried out by comparing query keywords with glossaries of query groups. Then the system determined significant scales for this group of requests and evaluated their severity. We cannot disclose the principles and mechanism for assessing expression, since this information is a trade secret, but we can say that they are accessible to machine learning.
According to the results of processing requests in the experimental group, user profiles were formed. In the profile for each of the groups of requests, the names of the scales relevant to the user and the estimated coefficients (the level of priority of the scale for the user) were indicated.
It should be noted that due to the limited design capacity, the system handled resources as a whole, rather than specific resource pages selected by users. For example, if the user selected slovo.ws/resh/007 , then the system analyzed slovo.ws . This fact, firstly, led to the fact that all content aggregators, such as social networks, Youtube, etc., fell out of processing due to the wide variety of content on them; secondly, this could not but affect the results of the experiment in the direction of their deterioration.
At the 5th stage, requests from the control group (104,601 queries) were processed by the system according to the following scheme: defining the query group -> determining the severity of significant scales -> calculating the index of compliance with the personal profile -> ranking resources in the search results by the correspondence index -> determining the rank of the resource selected by the user.
The definition of the query group was carried out, as in the previous step, by comparing the query keywords with glossaries of query groups. Then the system determined significant scales for this group of requests and evaluated their severity. After that, the compliance index was calculated for each resource from the search results (the first 10 resources). This index reflected how much the scales themselves and their expressiveness of the resource in the output correspond to the scales significant for the user for this group of requests in his personal profile. Based on the calculated compliance indexes, the system made a ranked list of resources in search results and determined the rank of the resource selected by the user.
At the final stage, there was an analysis of requests by the criterion of informativeness. After screening off non-informative queries, the average ranks of the selected resources were calculated and compared in the search service results and in the lists ranked by personal match indexes.
When analyzing requests for their informativeness, the following were eliminated:
a) queries when the user selected resources in the search results strictly in rank order, for example, (resource ranks) 1 * ; 12; 1, 2, 3; 1, 2, 3, 4; 1, 2, 3, 4, 5, etc., because, as we believe, this behavior does not reflect personal preferences, but rather is a consequence of stereotyping the perception of ranked information from the most significant (from the top) to the less significant (below). list). One could assume that in such a case the last resource is the most relevant for the user (after reading it, the user stopped / changed the search activity for the given request). In this case, it was necessary to analyze (compare) the following time parameters, - the time between the presentation of the search results and the first choice of the user, the time of interaction with the content, the time of the next request - and the parameters of the content uniqueness, - the presence of non-overlapping information, style of presentation, etc. We did not conduct it, as there were no time parameters in the database provided by the search service in open access, and the analysis of the uniqueness of the content required significant computational data. but it was useless without temporary data.
* - to determine that the user chose resource # 1 because of meeting his expectations (that is, the choice determined personal characteristics of the user), and not because this is a sample action, it is necessary to analyze the time between the presentation of the search results and click on the link. The time interval should be sufficient for comparing the parameters of resource No. 1 (wording of the link phrase, the contents of the snippet, address of the resource) with the corresponding parameters at least following in the list of resource No. 2. As mentioned above, there were no time parameters in the database provided by the search service.
The fact that search services have complete information on the query history can be used to determine the presence of patterned actions in the search behavior of the user and further optimize the use of personal search technology.
b) queries, if at least one of the resources in the corresponding search results had no compliance index. Despite the fact that the absence of an index could not increase the effectiveness of the proposed approach, as it placed the resources down the list, however, the elections, which were determined by the personal characteristics of users, and not the imperfections of the equipment, were the subject of analysis.
c) requests, where in the search results there were links to different pages of the same resource. Since the system analyzed not specific pages, but resources in general, it was not possible to separate different pages of the same resource when compiling a ranked list.
As a result, the calculation of average ranks was carried out on 74,279 requests (~ 71% of the total number of requests in the control group). The average rank of the selected resources in the issuance of a search service was 3.6. The value of the average rank in the lists, ranked by indices of personal matches, was 2.9. That is, the indicator improved by ~ 19.4%, which for the control group as a whole (including users for whom a profile was not formed) yielded a result of ~ 16.1%.
Interpretation of results
The results of the experiment showed that the use of personalization technology in the field of information retrieval increases the relevance of search results, which can be indirectly judged by the decrease in the rank of the resource selected by the user. In our case, the efficiency in the control group increased by ~ 16.1%.
We quite realistically look at the result. So, for example, having more data on user activity (specifically, time parameters) would not have to screen out all requests, where the user selected resources in the ranking sequence, and only a part from users with sample search behavior. This would undoubtedly reduce the final result. On the other hand, if we had more productive resources, we could analyze specific pages in search results, and not resources in general. This would slightly increase the efficiency, since it would allow to take into account a number of factors that, in our opinion, are important when choosing a source of information (unique information, style of presentation, etc.).
In our opinion, the proposed personalization technology for search services has several advantages designed to optimize the performance of search services. First, building a model of the user's psyche provides a high degree of predictability of this approach. Secondly, part of the operations can be displayed in offline mode, which allows optimal allocation of resources and reduce the load during online activity periods, for example, determining the severity of scales for resources and / or individual pages.
In the case of users with stereotypical perception of ranked information, the use of this approach will not affect the relevance of the issue directly (with or without personalized search, such users view resources according to the rank order). However, if we assume that such users stop / change search activity when a resource is found that meets their expectations, then using personal preferences will allow you to “raise” this resource higher in search results, thereby creating a positive experience of interaction with the search service.
The article is written in collaboration: Lepikhov Sergey, Golovan Alexander.
Source: https://habr.com/ru/post/243157/
All Articles