Laughter and psevdoreyting sin

Yesterday I came across the news "country X takes% some place in the ranking of good countries." News as news, ratings of this type are made regularly and in the set. But the news indicated a list of the “nicest" countries and a source site. The data that is given there cause a healthy laugh, and the method of counting is perplexing. About ratings and data manipulation this post.

Top Rated

So which countries are compilers the best?

1. Ireland
2. Finland
3. Switzerland
4. Netherlands
5. New Zealand
6. Sweden
7. Great Britain
8. Norway
9. Denmark
10. Belgium

This is the top 10. This is already a rather strange list, countries are usually small, and nothing comes to mind about their “good” and “bad” affairs. The more interesting to go to their website and look at the full lists and methods of calculation.
')

Site

We look at their site . Hipster fashion design is already suspicious, okay, let's call it a nagging and will not be taken into account. See FAQ rating:

The World Country Index.

The Good Country Index attempts to measure the contribution to the planet and humanity of every country on Earth.

Try to think about how to count on a common good. So in this context “good” means the opposite of “selfish”, not the opposite of “bad”.

Try to think of “good” as how much a country invests in the common good. In this context, “good” is the antonym of “selfish”, not the antonym of “bad”

OK, this is served as a kind of countries altruism rating. Nothing is clear from the overall rating, let's see the rating by industry.

Ratings by industry

Numbers are only in numbers of a rating. The initial data is represented by some incomprehensible bars without indicating the value, magnitude, order, but in general at least something meaningful. What is sad and suspicious.

Science and Technology

Top 10: United Kingdom, Austria, Cyprus (!), Czech Republic, Israel, Switzerland, Finland, Sweden, Hungary, New Zealand.
Evaluation criteria: foreign students, export of journals, international publications in 2009, Nobel laureates, patents.

You can already laugh. Cyprus with third place gets high marks in the categories of "foreign students", "export of magazines", "Nobel laureates". Do you know at least one Nobel laureate from Cyprus? Google - only one . The United States is on the 26th place, behind the Nobel laureates of Bosnia and Herzegovina and Iceland.

Culture

Top 10: Belgium, the Netherlands, Malta (!), Austria, Germany, Estonia, Ireland, Czech Republic.
Criteria: export of “creative goods” and “creative services”, contribution to UNESCO, freedom of movement and freedom of the press. It is difficult to understand how the last two points relate to culture, but in any case, the third place in Malta is puzzling. Regardless of how culture is considered to be the artifacts of the film industry, indie game development, or the weight of books produced, Malta receives the highest score for what.

International peace and security

Top 10: Egypt (!), Jordan, Tanzania (!), Lesotho, Uruguay, Togo, Benin, Paraguay, Nigeria, Ecuador.
Criteria: peacekeepers, contribution to the UN peacekeeping budget, international conflicts, arms exports, Internet security.
To be honest, at this moment I had a feeling that the rating was made backwards, and the first place was the worst. But having flushed the list all the way down, I noticed that in the tail there were also no outstanding countries - Lithuania, Latvia, Hungary, Azerbaijan.
And now, attention, at the very bottom in small print the sources of data and the methodology for their processing are indicated. I hope everyone remembered that in advertisements and contracts in small print are usually written very unpleasant and unexpected things? Let's get a look:

All data is 2010 data unless otherwise indicated. This is a fractional rank (0 = top rank, 1 = lowest). This is a list of the best ranks for each indicator. Ranks.

All data refer to 2010, unless indicated otherwise. Countries receive points for each indicator as a fractional rank (0 = highest, 1 = lowest) relative to all other countries for which data are available. Simple ranks by category are based on fractional ranks by five indicators per category (up to two parameters may be missing). The final rank is based on the average of the rank categories .

It is a pity (although, I think, this was done intentionally) that there is no example of calculation. Because it is not clear how they consider this “fractional rank” - is it a fraction of the sum? Or something like distribution? In general, the lack of formulas directly hints at the authors' bad faith. What other problems can be identified here?

1. Values ​​are divided by GDP of countries. The need for this is not justified. It can be assumed that the authors wanted to calculate something like “how much each citizen gives to world prosperity from his salary,” but GDP is a very rough approximation to such a calculation. Further, why, for example, is the number of peacekeepers divided by GDP? In the military, absolute numbers are more important. Why do ten peacekeepers from a rich country turn out worse than one from a poor one (if the GDP of a poor country is eleven times less than rich, then so be it)?
2. The mechanism for converting a “fractional rank” to a “simple rank” is not shown.
3. Why, when calculating the “simple rank”, is the average of the ranks by categories taken? Why does a country that has made a big contribution in any area lose it because of a lag in other areas? Why not use a simple amount of points, if you have once "normalized" to GDP?

Important deception ratings

Ratings have one very deceptive property - without absolute data, they become much less informative. Consider an example, grotesque for clarity:
Rating of 5 countries in terms of availability of public services, the number of days without hot water. Countries A, B, C, D, E, F, G. Country C ranked third in the ranking. It seems to be a good position, is not it? Now let's look at the source data:

Agree, in this case, the third place in the ranking is a little worth.

Conclusion

If you see any rating, no matter which country occupies which place in it, deal with the data with a certain degree of criticality, try to think where this data came from, how it was processed, so as not to fall victim to dishonest manipulators.