Highload ++ 2014 through the eyes of a speaker who really loves Processing Large-Scale Graph Data in various bad ways
Hello!
You have probably heard about the conference of developers of high-loaded Highload ++ systems, or you yourself participated or spoke or even stopped driving it, cursing it on social networks.
I have a very reverent attitude to the Highload conference, because my first visit to it in 2012 stabilized my knowledge, which was chaotic at that time, into an orderly chain and structure and provided the most extensive overview of the current Highload technology market in Russia and abroad.
')
In 2013, I enjoyed watching the online broadcast in the evening from the Tamtek office, discussing with my colleagues the advantages of Rabbit MQ, duplicate attempts to cram everything in Neo4j, and also a report by our colleague Denis Nelyubin on comparing NoSQL database performance (Aerospike, Couchbase, Mongo, Cassandra ) on our test framework, in the development of which I myself participated a year earlier.
In 2014, March 14, on my birthday, I sent a request with a description of the report on Highload and waited. After six months, I received a confirmation and I set off, hoping not only to get acquainted with colleagues working in related fields, but also to raise my Big Data consciousness to a new level.
Even long before approaching the conference venue in the subway and at the stops, the admin-programmer public began to concentrate, which was very different from the average Muscovite for its slowness and relaxation.
The World Trade Center met me with the pathos of a hermes-like naked kid, which seemed to hint at the high cost of the conference and promised a wide scope and fun inside.
After the participant had defended in a perfectly shardirovannoi and balanced queue, he could climb into the main hall and join the waiting for the first big reports (keynote).
Those who came early had time to see a motivational video from the organizers.
The morning began with an introductory report in which two lecturers, drawing attention, tried to clarify the key terms often used in the industry with simple pictures and concepts.
On the sidelines, on the first and second day, it was really hot: for those who like to take a look at short skirts, there were their own stands, for those who like to talk with the developer from MongoDB, theirs.
In general, such a useful thing as techtalks is slowly penetrating into Russia: small discussion platforms outside the main lecture hall. And it's nice that at Highload these spaces were meaningfully filled with intelligent and sociable engineers from various companies working “at the edge.” Especially the guys standing opposite each other, preaching Mongo DB and PostgreSQL.
The presentation of each speaker was preceded by a cartoon drawn in real time. They all lie here . Here's an example of the process of drawing my face.
The reports went one by one: the famous companies where the speakers work, sounded more and more serious. However, the majority of serious companies were in no hurry to reveal the secrets of their main services, from which they have income, but rather shared the architectures of some secondary solutions. On the one hand, this is normal, but not listening to the Badoo architecture every year (it does not change so rapidly), on the other hand, it begins to seem that a “cool” company does not always solve the problem in an optimal way, but, on the contrary, it can even close its eyes for some shortcomings.
It is possible that because of the constant high loads at work, the speakers often could not find enough expressive words in Russian except for the first ones that came to mind. Someone succeeds in this, but the majority of young speakers, blindly replicating the manner of performance of the main stars, it turns out bad and looks out of place.
The main trends of this year were the complete dominance of PostgreSQL, the departure to the shadow of MySQL (only lazy did not scold it), the omnipresent Docker, and the uplifting dragon of the RTB world. However, all the reports that in one way or another were related to RTB-related topics (based on their descriptions) were more about languages, bases and algorithms of archiving, rather than revealing the tasty curtain of architecture and mutual dancing services.
It is worth noting an unusually successful idea with a separate training track, ideal for experienced developers, whose projects have slowly become “heavy” lately, but the recipes of “relief” on the Internet seem confusing and vague.
Also, a rather successful find was the arrangement of reports of similar subjects one after the other in the same room, this allowed for two or three reports to see the problem from different sides. The most successful and prominent in my opinion were the family of reports on PostgreSQL, cartography, work with indices and MongoDB.
In the cartographic block, the presentation of the cartographic service of our new national search engine Sputnik, based on data from the Open Street Maps project and the js-library of Ukrainian developer Vladimir Agafonkin Lefleat, was pretty much amused.
The speaker turned out to be a very serious person, well versed in how to render tiles and correctly “update the World once a month”.
In the evening of the second day, I gave a talk on “Thorny path to the Large-Scale Graph Processing”, where I talked for a long time and in detail about current problems and tasks on large graphs, as well as their processing tools and ways to solve some problems on certain types of graphs (road , web, social).
Two consecutive reports on graphs, graph DBMS, methods of their processing allowed me and the listeners to really dive deep into the subject area, so I think it is a good idea to build reports with a similar theme in one track.
If you are worried about the fate of large graphs, NoSQL databases, or you are just a person with a broad outlook and decent manners, then you can find me in VK, browse the blog or make friends in G + and twitter .
By the way, all presentations are already available here.
At the conference there were many reports devoted to the dry theory and complex mathematical models and algorithms, an insane number of diagrams and graphs that are difficult to fit tightly in one head. Those who came for the first time to such an event could get real pleasure from a high-quality educational track, which explained the basic terms, basics, popular cases of high-loaded systems, for example, it was possible to learn everything about the work of the indices of various databases.
It seemed to me that this year was too big a lurch towards reports for DevOps and database administrators, at the same time, the project’s evolution from the first storm loads to well-thought-out solutions was poorly presented, because most of the participants go here for these recipes.
Nevertheless, the Highload conference and all its participants breathed the spirit of the practice. He was saturated with all the talk, he was invisibly present during the handshakes and exchange of business cards. It was clear that many people will start new interesting projects after talking on the sidelines.
Conferences end and we always go home with a slightly changed consciousness to further configure our servers and write new code. Just the code will be slightly different, and the settings are a little better. And the response time is a little less ... For this, after all, all this is spinning.
You have probably heard about the conference of developers of high-loaded Highload ++ systems, or you yourself participated or spoke or even stopped driving it, cursing it on social networks.
I have a very reverent attitude to the Highload conference, because my first visit to it in 2012 stabilized my knowledge, which was chaotic at that time, into an orderly chain and structure and provided the most extensive overview of the current Highload technology market in Russia and abroad.
')
In 2013, I enjoyed watching the online broadcast in the evening from the Tamtek office, discussing with my colleagues the advantages of Rabbit MQ, duplicate attempts to cram everything in Neo4j, and also a report by our colleague Denis Nelyubin on comparing NoSQL database performance (Aerospike, Couchbase, Mongo, Cassandra ) on our test framework, in the development of which I myself participated a year earlier.
In 2014, March 14, on my birthday, I sent a request with a description of the report on Highload and waited. After six months, I received a confirmation and I set off, hoping not only to get acquainted with colleagues working in related fields, but also to raise my Big Data consciousness to a new level.
And the first day came ... and the gates opened ...
Even long before approaching the conference venue in the subway and at the stops, the admin-programmer public began to concentrate, which was very different from the average Muscovite for its slowness and relaxation.
The World Trade Center met me with the pathos of a hermes-like naked kid, which seemed to hint at the high cost of the conference and promised a wide scope and fun inside.
After the participant had defended in a perfectly shardirovannoi and balanced queue, he could climb into the main hall and join the waiting for the first big reports (keynote).
Those who came early had time to see a motivational video from the organizers.
The overall feeling of the content
The morning began with an introductory report in which two lecturers, drawing attention, tried to clarify the key terms often used in the industry with simple pictures and concepts.
On the sidelines, on the first and second day, it was really hot: for those who like to take a look at short skirts, there were their own stands, for those who like to talk with the developer from MongoDB, theirs.
In general, such a useful thing as techtalks is slowly penetrating into Russia: small discussion platforms outside the main lecture hall. And it's nice that at Highload these spaces were meaningfully filled with intelligent and sociable engineers from various companies working “at the edge.” Especially the guys standing opposite each other, preaching Mongo DB and PostgreSQL.
The presentation of each speaker was preceded by a cartoon drawn in real time. They all lie here . Here's an example of the process of drawing my face.
The reports went one by one: the famous companies where the speakers work, sounded more and more serious. However, the majority of serious companies were in no hurry to reveal the secrets of their main services, from which they have income, but rather shared the architectures of some secondary solutions. On the one hand, this is normal, but not listening to the Badoo architecture every year (it does not change so rapidly), on the other hand, it begins to seem that a “cool” company does not always solve the problem in an optimal way, but, on the contrary, it can even close its eyes for some shortcomings.
It is possible that because of the constant high loads at work, the speakers often could not find enough expressive words in Russian except for the first ones that came to mind. Someone succeeds in this, but the majority of young speakers, blindly replicating the manner of performance of the main stars, it turns out bad and looks out of place.
Trends this year
The main trends of this year were the complete dominance of PostgreSQL, the departure to the shadow of MySQL (only lazy did not scold it), the omnipresent Docker, and the uplifting dragon of the RTB world. However, all the reports that in one way or another were related to RTB-related topics (based on their descriptions) were more about languages, bases and algorithms of archiving, rather than revealing the tasty curtain of architecture and mutual dancing services.
It is worth noting an unusually successful idea with a separate training track, ideal for experienced developers, whose projects have slowly become “heavy” lately, but the recipes of “relief” on the Internet seem confusing and vague.
Also, a rather successful find was the arrangement of reports of similar subjects one after the other in the same room, this allowed for two or three reports to see the problem from different sides. The most successful and prominent in my opinion were the family of reports on PostgreSQL, cartography, work with indices and MongoDB.
In the cartographic block, the presentation of the cartographic service of our new national search engine Sputnik, based on data from the Open Street Maps project and the js-library of Ukrainian developer Vladimir Agafonkin Lefleat, was pretty much amused.
The speaker turned out to be a very serious person, well versed in how to render tiles and correctly “update the World once a month”.
Count, hefty ...
In the evening of the second day, I gave a talk on “Thorny path to the Large-Scale Graph Processing”, where I talked for a long time and in detail about current problems and tasks on large graphs, as well as their processing tools and ways to solve some problems on certain types of graphs (road , web, social).
Two consecutive reports on graphs, graph DBMS, methods of their processing allowed me and the listeners to really dive deep into the subject area, so I think it is a good idea to build reports with a similar theme in one track.
If you are worried about the fate of large graphs, NoSQL databases, or you are just a person with a broad outlook and decent manners, then you can find me in VK, browse the blog or make friends in G + and twitter .
By the way, all presentations are already available here.
At the conference there were many reports devoted to the dry theory and complex mathematical models and algorithms, an insane number of diagrams and graphs that are difficult to fit tightly in one head. Those who came for the first time to such an event could get real pleasure from a high-quality educational track, which explained the basic terms, basics, popular cases of high-loaded systems, for example, it was possible to learn everything about the work of the indices of various databases.
It seemed to me that this year was too big a lurch towards reports for DevOps and database administrators, at the same time, the project’s evolution from the first storm loads to well-thought-out solutions was poorly presented, because most of the participants go here for these recipes.
Nevertheless, the Highload conference and all its participants breathed the spirit of the practice. He was saturated with all the talk, he was invisibly present during the handshakes and exchange of business cards. It was clear that many people will start new interesting projects after talking on the sidelines.
Conferences end and we always go home with a slightly changed consciousness to further configure our servers and write new code. Just the code will be slightly different, and the settings are a little better. And the response time is a little less ... For this, after all, all this is spinning.
Source: https://habr.com/ru/post/245215/
All Articles