Habramegairing: the best articles and Habra statistics for 12 years. Part 2/2

Hi, Habr.

In the first part , some patterns of development of such an interesting resource as habrahabr were considered. The material turned out to be long, so the continuation is here. In this part we will at the same time look at how to build such pictures, and finally, complete our statistics and rating.

')
Who are interested in the results, please under the cat.

Titles of articles (word cloud)

Before displaying the ranking of articles, it became interesting to see which keywords are the most popular in the headlines. It is quite obvious that the popularity of different technologies changes over time, I want to see it in a visual form.

This is easy to do with Python:

from wordcloud import WordCloud def split_words(s): try: words = re.split('[:?., "''()<>-\[\]|!]', s) return map(to_ascii, words) except: return [] def filter_words(s): s = s.decode('utf-8').encode("ascii", errors="ignore").decode() return len(s) > 2 def to_ascii(s): s = s.replace("'", '').replace("-", '').replace("|", '') return s.decode('utf-8').encode("ascii", errors="ignore").decode() titles = df['title'].str.lower() ts = titles.apply(lambda x: pd.value_counts(filter(filter_words, split_words(x)))).sum(axis = 0) ts = ts.sort_values(ascending=False) print(ts[:50]) print() s_all = "" for p in range(min(ts.shape[0], 200)): s_all += (ts.index[p] + ' ') * int(ts.values[p]) wc = WordCloud(width=1600, height=1200, background_color="white", relative_scaling=1.0, collocations=False, ).generate(s_all) plt.figure(figsize=(9,6)) plt.title("%d" % year) plt.xticks([]) plt.yticks([]) plt.tight_layout() file_name = 'habr-words-%d.png' % year plt.savefig(file_name)

However, it would be inconvenient to display the “sheet” of 12 pictures - we will do all this in the form of gif-animation. We take out the code in a separate function and run it cyclically for the desired range of years.

 import imageio images = [] for y in range(2006, 2019+1): file_name = make_words_cloud(df, y) images.append(imageio.imread(file_name)) imageio.mimsave('habr-words.gif', images, duration=2)

And the last point: to make words easier to compare, let's make sure that the same words always match the same colors.

 colors_dict = dict() def random_color_func_(word=None, font_size=None, position=None, orientation=None, font_path=None, random_state=None): if word in colors_dict: return colors_dict[word] else: c = random_color_func(word, font_size, position, orientation, font_path, random_state) colors_dict[word] = c return c wc = WordCloud(width=2600, height=2200, background_color="white", relative_scaling=1.0, collocations=False, color_func=random_color_func_).generate(text) ...

The final result in the form of a GIF:

And the size of the words (it is proportional to the number of entries) and their diversity speak for themselves. Some patterns are interesting - Google remained unshakable in the first place only, Flash, Opera and Yahoo are gone, 10 years ago nobody wrote about Amazon, and there were no names like Tesla, Kotlin or GDPR.

A similar attempt was made to create a distribution for Russian words, but it ended in complete failure - because of the declensions of the prefixes and endings in Russian, the result was more like a random number generator. An attempt to “normalize” all of this, singling out nouns, bringing them to the nominative case, etc., probably would not have demanded this article, but the length of the thesis. And it would be tempting to see how, for example, the word “Roskomnadzor” or “Duma” rose in the citation rating (but let's not talk about the sad).

At the same time, we will finally finish programming, and move on to the actual rating.

Rating

Once again, the rating is unofficial, and absolute certainty is not guaranteed. If for example, the server on some article returned timeout, then such an article will not be included in the list. Perhaps there are still some hidden indexes, about which I do not know. Manual check of 206 thousand articles from 2006 to 2019 is quite difficult. If one of the authors did not find himself in the rating, but I am sure that he should be there, write, add manually. Some articles from 10 years ago may have already become outdated, well, by the way, this is even more interesting - some forgotten moments can be remembered.

Let's go :) Well, and all the authors, congratulations in absentia on getting into the super-top. Although the names were not analyzed during the parsing and were not written into the rating, but I think those who created the articles will probably recognize themselves.

Edit : as some users noticed, a couple of articles are repeated twice. This is not a parsing error, these articles were really laid out again - the links are different, and the redirect leads to the same article.

Top 20 by the number of bookmarks

300 amazing free services 1482000 views, 9119 bookmarks
Memo to users of ssh 925000 views, 5822 bookmarks
27+ resources for online learning 939,000 views, 4851 bookmark
Networks for the smallest. Part zero. Planning 1388000 views, 4347 bookmarks
Stop twisting! 865,000 views, 4330 bookmarks
Sleep a little, but right? 464,000 views, 3,946 bookmarks
1000+ hours of Java video in Russian 1076,000 views, 3,616 bookmarks
Know the complexity of the algorithms 522,000 views, 3563 bookmarks
We make a private monitor from the old LCD monitor 486,000 views, 3539 bookmarks
The principle of cicada and why it is important for web designers 172,000 views, 3511 bookmarks
Acceleration of loading Windows for fun and profit 448000 views, 3497 bookmarks
Hack Wi-Fi for 10 hours 1416000 views, 3405 bookmarks
Guide to the design of HTML / CSS code from Google 266,000 views, 3349 bookmarks
Cheat sheet for design patterns 785,000 views, 3344 bookmarks
Several useful services 121,000 views, 3319 bookmarks
Top 5 most impressive books that every software developer should read 319,000 views, 3277 bookmarks
Many free books on programming 282,000 views, 3203 bookmarks
A selection of useful for fans of Twitter Bootstrap 248,000 views, 3079 bookmarks
Wi-Fi: unobvious nuances (for example, home network) 1186000 views, 3070 bookmarks
Want to be an iOS developer? Be it! 377,000 views, 2980 bookmarks

Top 20 in relation to the number of bookmarks to view

Cribs for those who make the first steps 1114 bookmarks, 15900 views
Linux hacking exercises 876 bookmarks, 13300 views
Anatomy of a font 1495 bookmarks, 22,700 views
38 articles on creating rounded corners on sites 677 bookmarks, 10,400 views
Where to nibble the granite of science 830 bookmarks, 13200 views
Advanced technologies, as a way to squeeze a maximum of 646 bookmarks from the server , 10,300 views
Lecture torrent Lectorium 947 bookmarks, 16200 views
Useful links for learning CSS animation 1280 bookmarks, 22300 views
New free online courses from Stanford 623 bookmarks, 11,300 views
UICloud: The largest database of user interfaces 1780 bookmarks, 32,400 views
A selection of html / javascript / css tools and libraries from SmashingMagazine 1000 bookmarks, 18,400 views
8 useful services for a web developer and designer 1658 bookmarks, 33,800 views
Lectorium recorded almost a thousand lectures in a year 2,516 bookmarks, 54,200 views
IPO for dummies. Part I: shares, majority shareholders, control over the company 564 bookmarks, 12,200 views
Cross-browser monochrome translucency of 499 bookmarks, 11000 views
Mediamagia: You come home, take the remote and choose to watch 518 bookmarks from the tracker , 11,700 views
Screencasts on how to cut and pull 448 bookmarks, 10,300 views
Noty - an unusually flexible jQuery plugin for displaying notifications 1099 bookmarks, 26200 views
Data Visualization 491 bookmark, 12200 views
9 articles on the topic of round buttons 396 bookmarks, 10,100 views

Top 20 most "controversial" articles

First post , 667 comments, rating + 596.0 / -445.0
Discrimination of VKontakte users , 319 comments, rating + 399.0 / -258.0
Why fell VKontakte , 380 comments, rating + 306.0 / -255.0
It's time to tie tabs in code , 217 comments, rating + 258.0 / -234.0
And the reader, and the dude igrets , 175 comments, rating + 337.0 / -233.0
Goodbye, Karma, or Who Needs an iPad? , 520 comments, rating + 661.0 / -223.0
Non-Apple products from Apple 504 comments rating + 397.0 / -218.0
Pointless 'Operating System' , 325 comments, rating + 394.0 / -215.0
Cho! Mail.ru What , 497 comments, rating + 316.0 / -205.0
Pepyaka , 255 comments, rating + 239.0 / -204.0
Chanterelle-Firefox Costume [photo] , 105 comments, rating + 285.0 / -204.0
Let's talk about Microsoft , 990 comments, rating + 261.0 / -201.0
OpenSource-CURATCH, or force the teacher to precipitate , 538 comments, rating + 276.0 / -200.0
Yandex browser , 825 comments, rating + 266.0 / -199.0
Merchant API , 136 comments, rating + 231.0 / -198.0
God is a stupid game designer , 531 comments, rating + 351.0 / -195.0
Why I refused Mozilla Firefox , 324 comments, rating + 225.0 / -193.0
Vogue magazine's 'secret' , 199 comments, rating + 225.0 / -189.0
Order of the White Knights of Habr , 553 comments, rating + 213.0 / -188.0
All PHP in two lines , 322 comments, rating + 240.0 / -187.0

Antitop-20 articles with the largest number of dislikes

Bonus

And a small bonus for those who read to here - we will post a mini-rating of articles written in English. This rating is essentially one year, since earlier from simply was not, but that is, that is. To get it, just add one line of code - select articles in the filter that have "/ en /" in the link field:

 df = df[df['link'].str.contains("/en/")]

The results are shown below. All categories will not bring, because English-language articles are still few, and much is repeated.

findings

Conclusions will not be. Thank you all for your attention, and have a nice reading :)

Source: https://habr.com/ru/post/442168/

All Articles

Habramegairing: the best articles and Habra statistics for 12 years. Part 2/2

Titles of articles (word cloud)

Rating

Top 20 articles by number of views

Top 20 articles by rating

Top 20 articles by relative rating

Top 20 by the number of bookmarks

Top 20 in relation to the number of bookmarks to view

Top 20 most "controversial" articles

Top 20 Most Commented Articles

Antitop-20 articles with the largest number of dislikes

Bonus

Top English-language articles by number of views

Top English articles on the number of bookmarks

findings

More articles: