In the next few posts we will talk about the practical use of multi-core processors. After all, whatever is said about multi-core, in any case, the program must be “trained” in the effective use of several cores. And in this first post will be the announcement and the first "introductory" note.

Parallel programming technology

Immediately I must say that there are quite a lot of different parallel programming technologies. Moreover, these technologies differ not only and not so much in programming languages, as architectural approaches to the construction of parallel systems.

For example, some technologies assume the construction of parallel solutions based on several computers (both of the same type and of different types), while others assume that it will work on the same machine with several processor cores.
')
Systems based on several computers belong to the class of systems for distributed computing. Such solutions have been used for quite some time, they are well understood by industry professionals, there is quite a lot of literature on them. The most striking example of distributed computing technology is MPI (Message Passing Interface). MPI is the most common standard of data exchange interface in parallel programming, there are implementations of it for a large number of computer platforms. MPI provides the programmer with a single mechanism for the interaction of branches within a parallel application, regardless of the machine architecture (single-processor / multiprocessor with shared / separate memory), the relative position of branches (on a single processor or on different ones).

Since MPI is intended for systems with separate memory, it is not a good idea to use it for organizing a parallel process in a system with shared memory. This will be too redundant and difficult, which is why solutions like OpenMP began to develop. Although, of course, nothing prevents to make MPI-solutions for one machine.

But parallel programming systems for working on one machine began to develop relatively recently. No, of course, you should not think that these are fundamentally new ideas, but it is with the arrival (or rather, with the upcoming arrival) multi-core systems on the desktop, programmers should pay attention to technologies such as OpenMP , Intel Thread Building Blocks , Microsoft Parallel Extensions and a number of others.

It is very important that the parallel programming technology supports the ability to make the program parallel gradually. Yes, it is clear that an ideal parallel program should be immediately written parallel, or even better in some functional language, where the parallelization question is generally not worth it ... But programmers live and work in the real world, in which instead of the new-fashioned functional F # there is 10 MB of code in the best case is in C ++, or even C in general. And this code should be parallelized gradually. In this case, OpenMP technology (for example) would be a very good choice. It allows you to select the places most in need of parallelization in the application, first of all, to make them parallel. In practice, it looks like this. With the help of some kind of profiling tool, the programmer searches for “bottlenecks” in the program that last the longest. Why using a tool? Because the power of thought in a little-known project of 10 MB in size cannot find “bottlenecks”. Then the programmer makes these bottlenecks parallel with OpenMP. After that, you can search for the following bottlenecks, and so on, until you get the desired application performance. The process of developing a parallel version can be interrupted, release intermediate releases, return to it as needed. That is why, in particular, OpenMP technology has become quite popular.

What is OpenMP?

OpenMP (Open Multi-Processing) is a set of compiler directives, library procedures and environment variables that are designed for programming multi-threaded applications on shared memory multiprocessor systems (SMP systems).

The first OpenMP standard was developed in 1997 as an API focused on writing easily portable multi-threaded applications. At first it was based on the Fortran language, but later included the C and C ++ languages.

The OpenMP interface has become one of the most popular parallel programming technologies. OpenMP is successfully used both when programming supercomputer systems with a large number of processors, and in desktop user systems or, for example, in Xbox 360.

The OpenMP specification is developed by several large manufacturers of computing equipment and software, whose work is regulated by the non-profit organization OpenMP Architecture Review Board ( ARB ).

OpenMP uses the branch-merge parallel execution model. An OpenMP program begins as a single thread of execution called the initial thread. When a thread encounters a parallel construct, it creates a new group of threads, consisting of itself and a certain number of additional threads, and becomes the main one in the new group. All members of the new group (including the main one) execute the code inside the parallel structure. At the end of the parallel construction there is an implicit barrier. After a parallel construct, only the main thread continues to execute user code. Other parallel regions can be nested in a parallel region, in which each stream of the initial region becomes the main for its group of streams. Nested regions may in turn include regions of a deeper nesting level.

The number of threads in a group running in parallel can be controlled in several ways. One of them is the use of the environment variable OMP_NUM_THREADS. Another way is to call the omp_set_num_threads () procedure. Another way is to use the num_threads expression in conjunction with the parallel directive.

Announcement of upcoming parallel programming notes

With this post we begin a short cycle of publications devoted to acquaintance with the OpenMP technology and tools for developing parallel applications. In the following notes you will learn:
• what tools are needed to develop parallel programs;
• how to create a parallel program from scratch;
• how to add parallel execution to an existing program using OpenMP technology;
• what typical problems arise when developing OpenMP applications and how to diagnose them;
• optimization of parallel programs.

Wait for the next lesson release, and in the comments, please write down what parallel programming topics interest you. And then we will make further notes based on your wishes.

Additional materials

1. The Parallel Programming Community on the Intel Software Network.
2. Article " Brief about OpenMP technology ".