Graphic processors in solving modern IT problems

Graphics processors (GPUs) are a prime example of how technology designed for graphics processing has spread to an unrelated area of ​​high performance computing. Modern GPUs are the heart of many complex projects in the field of machine learning and data analysis. In our review article, we will describe how Selectel customers use equipment with a GPU, and think about the future of data science and computing devices, along with teachers from the Yandex Data Analysis School.

Graphics processors over the past ten years have changed a lot. In addition to the enormous performance gains, there was a separation of devices by type of use. So, video cards for home gaming systems and virtual reality installations stand out in a separate direction. Powerful, highly specialized devices are emerging: for server systems, one of the leading accelerators is the NVIDIA Tesla P100 , designed specifically for industrial use in data centers. In addition to the GPU, research is actively conducted in the field of creating a new type of processors that imitate the brain. An example is the Kirin 970 single-chip platform with its own neuromorphic processor for tasks related to neural networks and pattern recognition.

This situation makes you think about the following questions:
')

• Why has the field of data analysis and machine learning become so popular?
• How did graphics processors become dominant in the data hardware market?
• What research data analysis will be the most promising in the near future?

Let's try to deal with these issues in order, starting with the first simple video processors and ending with modern high-performance devices.

GPU Era

First, remember what a GPU is. The Graphics Processing Unit is a graphics processor widely used in desktop and server systems. A distinctive feature of this device is its focus on massively parallel computing. Unlike GPUs, the architecture of another CPU (Central Processor Unit) computing module is designed for serial data processing. If the number of cores in a conventional CPU is measured in dozens, then in the GPU they are counted in thousands, which imposes restrictions on the types of commands executed, but provides high computational performance in tasks involving parallelism.

The first steps

The development of video processors in the early stages was closely associated with the growing need for a separate computing device for processing two and three-dimensional graphics. Before the appearance of separate video controller circuits in the 70s, image output was carried out through the use of discrete logic, which affected the increased power consumption and large sizes of printed circuit boards. Specialized chips allowed to allocate the development of devices designed to work with graphics, in a separate direction.

The next revolutionary event was the emergence of a new class of more complex and multifunctional devices - video processors. In 1996, 3dfx Interactive released the Voodoo Graphics chipset, which quickly occupied 85% of the market for specialized video devices and became the leader in 3D graphics at the time. After a series of unsuccessful decisions of the company's management, among which was the purchase of a manufacturer of video cards STB, 3dfx gave way to NVIDIA and ATI (later AMD), and in 2002 declared bankruptcy.

General GPU Computing

In 2006, NVIDIA announced the launch of the GeForce 8 series product line, which marked the beginning of a new class of devices designed for general computing on graphics processors (GPGPU). During development, NVIDIA realized that a larger number of cores operating at a lower frequency are more efficient for parallel loads than a small number of more productive cores. Video processors of the new generation provided support for parallel computing, not only for processing video streams, but also for problems related to machine learning, linear algebra, statistics, and other scientific or commercial problems.

Recognized leader

Differences in the initial formulation of tasks for the CPU and GPU led to significant discrepancies in the architecture of devices - high frequency versus multi-core. For GPUs, this laid the computational potential, which is currently being fully realized. Video processors with an impressive amount of weaker computational cores do an excellent job with parallel computing. The central processor, historically designed to work with sequential tasks, remains the best in its field.

For example, let us compare the values ​​in the performance of the central and graphic processor on the performance of a common task in neural networks — multiplication of high order matrices. Choose the following devices for testing:

• CPU. Intel Xeon E5-2680 v4 - 28 threads with HyperThreading, 2.4 GHZ;
• GPU. NVIDIA GTX 1080 - 2560 CUDA Cores, 1607 Mhz, 8GB GDDR5X.

We use the example of calculating the multiplication of matrices on the CPU and GPU in Jupyter Notebook:



In the code above, we measure the time it took to calculate matrices of the same order on a central or graphics processor (“Runtime”). Data can be represented as a graph, on which the horizontal axis displays the order of the matrixes being multiplied, and the vertical axis - Runtime in seconds:



The line of the graph, highlighted in orange, shows the time it takes to create data in ordinary RAM, transfer it to the GPU memory, and subsequent calculations. The green line shows the time it takes to calculate the data that has already been generated in the video card's memory (without transferring it from RAM). Blue displays the counting time on the central processor. Matrices of the order of less than 1000 elements are multiplied by the GPU and the CPU in almost the same time. The difference in performance is well manifested with matrices larger than 2000 by 2000, when the computing time on the CPU jumps to 1 second, and the GPU remains close to zero.

More complex and practical tasks are more effectively solved on the device with graphic processors than without them. Since the problems that our clients solve on GPU equipment are very diverse, we decided to find out which most popular usage scenarios exist.

Who in Selectel lives well with the GPU?

The first option, which immediately comes to mind and turns out to be a correct guess, is mining, however, it is curious to note that some people use it as an auxiliary way to load equipment to "maximum". In case of renting a dedicated server with video cards, the time free from workloads is used to mine cryptocurrencies that do not require specialized installations (farms) to receive.

The tasks that have already become somewhat classical, associated with graphic processing and rendering, invariably find their place on the Selectel servers with graphic accelerators. The use of high-performance equipment for such tasks allows you to get a more efficient solution than the organization of dedicated workstations with video cards.

During the conversation with our customers, we also met with representatives of the Yandex Data Analysis School, which uses the power of Selectel to organize test training environments. We decided to learn more about what students and teachers do, what machine learning directions are popular now, and what future awaits the industry after young professionals join the ranks of employees of leading organizations or launch their start-ups.

Data science

Perhaps, among our readers there will not be those who would not have heard the phrase “neural networks” or “machine learning”. Rejecting marketing variations on the theme of these words, it turns out a dry residue in the form of an emerging and promising data science.

The modern approach to working with data includes several main areas:

• Big Data (Big Data). The main problem in this area is a huge amount of information that cannot be processed on a single server. From the point of view of infrastructure support, it is required to solve problems of creating cluster systems, scalability, fault tolerance, and distributed data storage;
• Resource-intensive tasks (machine learning, deep learning and others). In this case, the issue of using high-performance computing that requires a large amount of RAM and processor resources is raised. In such tasks, systems with graphic accelerators are actively used.

The boundary between the direction data is gradually erased: the main tools for working with big data (Hadoop, Spark) introduce support for GPU computing, and machine learning tasks cover new areas and require more data. Teachers and students of the School of Data Analysis will help us to understand more.



It is difficult to overestimate the importance of competent data manipulation and the appropriate implementation of advanced analytical tools. It is not even about big data, their “lakes” or “rivers”, but about intellectual interaction with information. What is happening now represents a unique situation: we can collect a wide variety of information and use advanced tools and services for in-depth analysis. The business introduces such technologies not only to obtain advanced analytics, but also to create a unique product in any industry. It is the last point that largely forms and stimulates the growth of the data analysis industry.

New direction

Everywhere we are surrounded by information: from the logs of Internet companies and banking operations to the readings in experiments at the Large Hadron Collider. The ability to work with this data can bring millions of profits and provide answers to fundamental questions about the structure of the Universe. Therefore, data analysis has become a separate area of ​​research among the business and scientific community.

The school of data analysis prepares the best specialized specialists and scientists who in the future will become the main source of scientific and industrial developments in this field. The development of the industry affects us as an infrastructure provider - more and more customers are requesting server configurations for data analysis tasks.

The specifics of the tasks facing our customers determine which equipment we must offer our customers and in what direction our product line should be developed. Together with Stanislav Fedotov and Oleg Ivchenko, we interviewed students and teachers of the School for Data Analysis and found out what technologies they use to solve practical problems.

Data Analysis Technologies

During training, students from the basics (basic higher mathematics, algorithms and programming) reach the most advanced areas of machine learning. We collected information on those that use servers with GPUs:

• Deep learning;
• Reinforcement training;
• Computer vision;
• Automatic text processing.

Students use specialized tools in their study assignments and research. Some libraries are designed to bring data to the desired form, others are designed to work with a specific type of information, such as text or images. Depth learning is one of the most difficult areas in data analysis that makes extensive use of neural networks. We decided to find out what kind of framework teachers and students use to work with neural networks.



The presented tools have different support from the creators, but nevertheless, continue to be actively used for training and working purposes. Many of them require productive equipment to handle tasks in adequate time.

Further development and projects

As with any science, the direction of data analysis will change. The experience that students receive today will undoubtedly become the basis for future developments. Therefore, it is worth noting separately the high practical orientation of the program - some students, during their studies or after, begin internships at Yandex and apply their knowledge already on real services and services (search, computer vision, speech recognition and others).

We talked about the future of data analysis with the teachers of the School of Data Analysis, who shared with us their vision of the development of data science.

According to Vlad Shakhuro , a teacher of the course “Image and Video Analysis”, the most interesting tasks in computer vision are to provide security in crowded places, control an unmanned vehicle and create an application using augmented reality. To solve these problems, it is necessary to be able to qualitatively analyze video data and develop, first and foremost, algorithms for detecting and tracking objects, recognizing a person by face, and three-dimensional reconstruction of the observed scene. Lecturer Viktor Lempitsky , who leads the course “Depth Education”, separately highlights auto-encoders, as well as generative and adversary networks.

One of the mentors of the School of Data Analysis shares his opinion on the distribution and the beginning of the mass use of machine learning:

“Machine learning from the lot of the few obsessed researchers is turning into another tool of an ordinary developer. Earlier (for example, in 2012), people wrote low-level code for learning convolutional networks on a pair of video cards. Now, anyone can in a matter of hours:

• download the weight of the already trained neural network (for example, in keras);
• use it to make a decision for your task ( fine-tuning, zero-shot learning );
• embed it in your website or mobile application (tensorflow / caffe 2).

Many large companies and startups have already won on such a strategy (for example, Prisma), but even more tasks are yet to be discovered and solved. And, perhaps, this whole story with machine / deep learning will someday become as commonplace as python or excel is now. ”

According to Oleg Ivchenko , administrator of the server infrastructure of the database, for standard deep learning tasks on standard data sets (for example, CIFAR, MNIST) the following resources are required:

• 6 CPU cores;
• 16 Gb of RAM;
• 1 GPU card with 6-8 Gb of video memory. This corresponds to such video cards as PNY NVIDIA GeForce GTX 1060 or MSI GeForce GTX 1070.

Accurately predict the technology of the future today, no one can, but when there is a certain motion vector, you can understand what should be studied now. And there are a lot of opportunities for this in the modern world.

Opportunities for beginners

The study of data analysis is limited to high requirements for students: extensive knowledge in the field of mathematics and algorithms, the ability to program. Truly serious machine learning tasks require the availability of specialized equipment. And for those who want to learn more about the theoretical component of data science, the School of Data Analysis, together with the Higher School of Economics , launched an online course on Introduction to Machine Learning .

Instead of conclusion

The growth of the graphics processor market is supported by the growing interest in the capabilities of such devices. GPU is used in home gaming systems, rendering and video processing tasks, as well as where general high-performance computing is required. The practical application of data mining tasks will penetrate deeper into our daily lives. And the implementation of such programs is most effectively carried out with the help of the GPU.

We thank our customers, as well as teachers and students of the School of Data Analysis for the joint preparation of the material, and invite our readers to get to know them better .

We are experienced and experienced in machine learning, data analysis and not only we offer to look at offers from Selectel for renting server equipment with graphic accelerators: from simple GTX 1080 to Tesla P100 and K80 for the most demanding tasks.