We consider porn sites, we evaluate the effectiveness of Roskomnadzor

The whole story began with a rather heated discussion of the percentage of porn vs. all the rest. The discussion gradually turned into a tote. The one who calls the number closest to the “real” one is in queens. Bets are made, there are no more bets.

How and where?

It remained to find out the "real" indicator. The criteria and sample were determined in advance, within which the study was required. A sufficient criterion, in the opinion of the majority of participants in the special Olympiad, was a simple regularity on the word porn in the content + a few caustic words (we all know them). The sample took the top 1,000,000 . Despite the fact that the domains on the Internet ~ 400kk, Alex quite clearly distinguishes those who have at least some traffic.

Charged the crawler, rushed. I note that they did not try to google the participants of the tote, there is no reason not to trust anyone, the topic is delicate, and I really wonder how much? Or maybe it is, it is less and it's time to swing?
')

All shed sichnik

He with a rate of 2.9% was the closest to the values ​​obtained. We found 26044 pornik ( 2.6% ) of a million top-end domains (as of July 4, 2016 ). Frankly, the number of many surprised, the run-up in the predictions was not small, the top bar was given at the level of 35%. They used to google, found an article with a study from Forbes in 2011, with a figure of ~ 4%, which indirectly confirmed the adequacy of the results. But, really falls?

Dynamics

They came up responsibly, kept the regular season for almost 3 months, followed the dynamics. While they were watching, they burned the fall of a large grid (someone had hard-footed the hoster). You can notice a drop on the graph on August 4, ~ 2000 sites went down, but already 5 numbers of the guys recovered and continue to deliver content to this day, without a single break. Porn sites are consistently eating their share: in 3 months from 2.6% to 3.3% ( 33147 )

What is Roskomnadzor?

Just under the completion of our research, the HYIP has risen around adult content, the instigator of which was the beloved Roskomnadzor, which blocked particularly popular sites.

Joynim base of blocked sites (thanks to Roskomvoboda) on the list of pornography we received, we get 626 domains. This is 1.88% of all sites that we have identified as dirty, depraved.

I would like to understand what proportion of content the state blocks not by the number of domains, but by their importance. Suddenly, all these 626 domains aggregate 90% of traffic and users are forced to look for a replacement among lower-quality products. We don’t have traffic data, but we have Alexa rating, where:

1 . Google
...
...
999999 . Qoranona.com (I quote: "Sweet life to obey God," everything in Arabic, rescued Google translator).

A funny coincidence, the top 1, which is not related to religion in any way, helps to convey information to a religious site from the last place to the masses. If anything, I'm not trying to offend the feelings of believers, I am about technology with facts. Is this site extremist I have no opportunity to determine, so I do not advise anyone to go to it (just in case).

So, about the rating

In order to roughly estimate the share of traffic sites that Roskomnadzor has blotted, we will distribute to them the weights. For simplicity, the calculation will take away from the entire list from the rating value of 1,000,000 and take the module, ie:

Google , as the most visited site, gives us 999999 points on the drum, and Qoranona.com only 1 .

The total amount of weights for the list is considered as the sum of the natural series, we have it = 500000500000

The amount of site weights ( 33147 domains on the last day of the scan), which we defined as porn sites = 18096750838 ( 3.6% of the total list, with 3.3% of the number of domains occupied)

The sum of the weights is 626 domains that were blocked = 460251373, which is 2.54% of the total amount of weights of porn sites (with 1.88% by the number of domains). We think that this figure can be considered as an indicator of the effectiveness of the state's destruction of adult content at the moment.

UPD: interactive graph for the convenience of viewing posted here