Big Data analytics - realities and prospects in Russia and the world

Today, only a person who has no external links with the outside world has heard about big data. On Habré, Big Data analytics and related topics are popular. But to non-experts who would like to devote themselves to the study of Big Data, it is not always clear what prospects this area has, where Big Data analytics can be applied and what a good analyst can expect. Let's try to figure it out.

The amount of information generated by a person increases every year. By 2020, the amount of stored data will increase to 40-44 zettabytes (1 ST, ~ 1 billion GB). By 2025, up to about 400 Zettabytes. Accordingly, the management of structured and unstructured data using modern technologies is an area that is becoming increasingly important. Both individual companies and entire countries are interested in big data.

By the way, it was in the process of discussing the information boom and methods of processing human-generated data that the term Big Data emerged. It is believed that Nature’s first editor, Clifford Lynch, proposed it in 2008.
')
Since then, the Big Data market has been increasing by several tens of percent every year. And this trend, according to experts, will continue. Thus, according to the company Frost & Sullivan, in 2021, the total volume of the global market for big data analytics will increase to $ 67.2 billion. Annual growth will be about 35.9%.

Why do you need big data analytics?

It allows you to identify highly valuable information from structured or unstructured data sets. Thanks to this, a business, for example, can determine trends, forecast production performance and optimize its own expenses. It is clear that for the sake of reducing costs, companies are ready to introduce the newest solutions.

Technologies and methods of analysis used to analyze Big Data:

• Data Mining;
  
• crowdsourcing;
  
• data mixing and integration;
  
• machine learning;
  
• artificial neural networks;
  
• pattern recognition;
  
• forecast analytics;
  
• simulation modeling;
  
• spatial analysis;
  
• statistical analysis;
  
• visualization of analytical data.
  

Big Data Analytics in the world

Now big data analytics is used in more than 50% of companies around the world. Despite the fact that in 2015 this figure was only 17%. Big Data is most actively used by companies that operate in the field of telecommunications and financial services. Then come the companies that specialize in health technology. Minimal use of Big Data analysts in educational companies: in most cases, representatives of this sphere declared their intention to use technologies in the near future.

In the US, Big Data analytics is used most actively: more than 55% of companies from various fields work with this technology. In Europe and Asia, the demand for big data analytics is not much lower - about 53%.

And what about Russia?

According to IDC analysts, Russia is the largest regional market for Big Data analytics solutions . The growth of the market for such solutions in Central and Eastern Europe is quite active, every year this figure increases by 11%. By 2022, it will reach in the quantitative terms $ 5.4 billion.

In many ways, this rapid development of the market is due to the growth of this sector in Russia. In 2018, revenue from the sale of relevant decisions in the Russian Federation accounted for 40% of the total investment in Big Data processing technologies of the entire region.

In the Russian Federation, companies from the banking and public sectors, the telecommunications industry and industry spend the most on processing Big Data.

What does Big Data Analyst do and how much does it get in Russia?

The Big Data Analyst is responsible for exploring vast amounts of information, both partially structured and unstructured. For banking organizations, these are transactions, for operators - calls and traffic, in retail - customer visits and purchases. As mentioned above, the analysis of Big Data reveals links between various factors of a “raw information history”, for example, a production process or a chemical reaction. Based on the analysis data, new approaches and solutions are being developed in various fields - from manufacturing to medicine.

Skills needed by Big Data analytics:

• The ability to quickly understand the features in the area for which the analysis is carried out, to immerse yourself in aspects of the desired field This may be retail, oil and gas, medicine, etc.
  
• Knowledge of statistical data analysis methods, construction of mathematical models (neural networks, Bayesian networks, clustering, regression, factor, dispersion and correlation analyzes, etc.).
  
• To be able to extract data from different sources, convert them for analysis, and load into an analytical database.
  
• SQL ownership
  
• Knowledge of English at a level sufficient for smooth reading of technical documentation.
  
• Knowledge of Python (at least the basics), Bash (it's very difficult to do without it in the process), plus it is desirable to know the basics of Java and Scala (needed for active use of Spark, one of the most popular frameworks for working with big data).
  
• Ability to work with Hadoop.
  

Well, how much does a Big Data analyst get?

Big Data specialists are now in short supply, demand exceeds supply. All because business comes to understanding: development requires new technologies, and technology development requires specialists.

So, Data Scientist and Data Analyst in the United States entered the top 3 best professions in 2017 according to the recruitment agency Glassdoor. The average salary of these specialists in America starts at $ 100,000 a year.

In Russia, machine learning specialists receive from 130 to 300 thousand rubles per month, big data analysts from 73 to 200 thousand rubles per month. It all depends on experience and qualifications. Of course, there are vacancies with lower wages, there are - with more. Maximum demand for big data analysts in Moscow and St. Petersburg. Not surprisingly, Moscow accounts for about 50% of active vacancies (according to hh.ru). Much less demand - in Minsk and Kiev. It is worth noting that some vacancies offer flexible hours and remote work. But in general, companies need professionals who work in the office.

Over time, we can expect an increase in demand for Big Data analysts and representatives of related specialties. As mentioned above, the hunger in technology has not been canceled. But, of course, in order to become a Big Data analyst you need to learn and work, improving both those skills mentioned above and additional ones. One of the opportunities to start the path of Big Data analytics is to enroll in a course from Geekbrains and test your abilities in the field of working with big data.