About analytics. How to calculate the integrity of IT companies
It is funny to consider any development index of a particular enterprise for internal use, publishing only average values for the industry, etc. There are a lot of positive emotions, and there is no need to quarrel with the respondents. It is more difficult to consider indicators publicly and impartially. Especially if the level of trust is assessed. Recently I found out about the petty scandal because of such personal indices. It was a little over a year ago.
Analytical company IDC publishes Russia IT Services Forecast and Vendor Shares , where it makes conclusions on the status of IT services in Russia, on the shares of different providers of these services and predicts how the market will change in the coming years. In 2007, IDC attracted LINEKS (the League of Independent IT Experts) to this study. This company introduced a “trust index” that shows how real the data on IT company revenues are in the IDC questionnaire. Those. in LINEKS they tried to measure whether the respondent strongly lied. Current index. And not only because it catches liars on the word, but because the respondent could not trite the question, make a mistake when filling out. The index rolls over - it means to call and clarify.
This measurement of honesty ended with a collective letter from the not quite recent IT companies Compulink, CROC, I-Teco, LANIT and R-Style to the president of IDC. In it, the signatories were outraged by the index and insisted that IDC prevent it from being published. As I understand it, there was no publication. There was a mention of the existence of the index at a press conference. In addition, the angry group now demanded to coordinate with the respondents the transfer of data from the questionnaires to a third party. If, however, the accuracy estimates of the data provided fall into the media, then all of the above will publicly break with the IDC.
')
How did it all end? IDC gently sent our IT companies, saying that they were satisfied with the subcontracting, and the rights to the research results were not transferred to anyone. LINEKS continues to consider the discrepancy between the data provided and the updated data. Converge more precisely than before, twice as accurate. Those who do not want to participate in the study - do not participate. Who got better? Rating - no, its representativeness suffered. Although my respect for IDC, for example, increased slightly, because at first glance, they connected analysts to the calculations not related to their company. Accuracy is likely to increase. To Gartner and IDC, I sometimes meet, to put it mildly, a complicated attitude among managers. Forrester is better. To managers of outraged companies, it is also not easier, I think, if they really get bonuses for getting into IDC rating closer to its top. Researchers have the whole demarche to be monopolistic, in principle, until a significant share of IT companies have joined the boycott and are buying their research.
For a couple of days in this story it is impossible to understand, I will try to keep it in sight. I’m not sure about the usefulness of ratings in general, but I’m sure about it for sure - in order to fight for the adequacy of one’s place on Olympus, one must constructively criticize researchers and indicate not errors in their calculations. And do not demand to remove specific analysts, leaving only "those who thought before and the way we liked." The lists of the best, in my opinion, should be a way of self-assessment and control for companies, and not a sales tool.
Analytical company IDC publishes Russia IT Services Forecast and Vendor Shares , where it makes conclusions on the status of IT services in Russia, on the shares of different providers of these services and predicts how the market will change in the coming years. In 2007, IDC attracted LINEKS (the League of Independent IT Experts) to this study. This company introduced a “trust index” that shows how real the data on IT company revenues are in the IDC questionnaire. Those. in LINEKS they tried to measure whether the respondent strongly lied. Current index. And not only because it catches liars on the word, but because the respondent could not trite the question, make a mistake when filling out. The index rolls over - it means to call and clarify.
This measurement of honesty ended with a collective letter from the not quite recent IT companies Compulink, CROC, I-Teco, LANIT and R-Style to the president of IDC. In it, the signatories were outraged by the index and insisted that IDC prevent it from being published. As I understand it, there was no publication. There was a mention of the existence of the index at a press conference. In addition, the angry group now demanded to coordinate with the respondents the transfer of data from the questionnaires to a third party. If, however, the accuracy estimates of the data provided fall into the media, then all of the above will publicly break with the IDC.
')
How did it all end? IDC gently sent our IT companies, saying that they were satisfied with the subcontracting, and the rights to the research results were not transferred to anyone. LINEKS continues to consider the discrepancy between the data provided and the updated data. Converge more precisely than before, twice as accurate. Those who do not want to participate in the study - do not participate. Who got better? Rating - no, its representativeness suffered. Although my respect for IDC, for example, increased slightly, because at first glance, they connected analysts to the calculations not related to their company. Accuracy is likely to increase. To Gartner and IDC, I sometimes meet, to put it mildly, a complicated attitude among managers. Forrester is better. To managers of outraged companies, it is also not easier, I think, if they really get bonuses for getting into IDC rating closer to its top. Researchers have the whole demarche to be monopolistic, in principle, until a significant share of IT companies have joined the boycott and are buying their research.
For a couple of days in this story it is impossible to understand, I will try to keep it in sight. I’m not sure about the usefulness of ratings in general, but I’m sure about it for sure - in order to fight for the adequacy of one’s place on Olympus, one must constructively criticize researchers and indicate not errors in their calculations. And do not demand to remove specific analysts, leaving only "those who thought before and the way we liked." The lists of the best, in my opinion, should be a way of self-assessment and control for companies, and not a sales tool.
Source: https://habr.com/ru/post/39692/
All Articles