Parallel programming in the black box
About two months ago, the first experimental Russian-language online seminar from Intel " Intel Parallel Studio workflow " took place. At this seminar, an attempt was made to show that there are such magic tools that will help implement paralleling and optimizing C ++ code, even if you do not have a complete understanding of the principles of the program. I know that this approach is often criticized. And another reminder that the actions shown on the webinar are incorrect, encouraged me to this post.
Similar disagreements existed once between supporters and opponents of using debugging tools (“debuggers”). Opponents argued that debuggers are a consequence of the wrong methodology, and debugger users are simply illiterate. Their main argument was that the use of debugging tools encourages the writing of code by trial and error, without trying how to think out the algorithm and rewrite it, break it into small functions, make it more concise and simple - so that there is no room for errors. Since there are few opponents of debugging tools now, many may not even have heard of such disputes. Well, now we are seeing the beginning of a new round.
')
At present, two approaches to the problem of program parallelization have been formed and are being discussed. Proponents of the first want to get some means that will allow them with a minimal rework of the program to carry out its parallelization. These are practices that want to adapt their solutions to the capabilities of multi-core systems that appear on the tables of ordinary users. The latter like to talk about various parallelization theories, parallel programming languages ​​and the need to clearly understand how parallel code will work. Indeed, in their opinion, otherwise it is impossible to achieve its correctness and efficiency.
I agree with the second group, which calls for conscious action, whether debugging or experimenting with the use of OpenMP directives. Such an approach is provoked by the creation of less reliable programs, because a valid code is recognized not as a correct code, but as a code that works correctly on a number of tests. This is definitely bad. But with my soul, I fully support the first group.
I am a practitioner. And I think that a long time ago nobody knows how real programs work. And it makes no sense to say whether the program being debugged or parallelized is clear to the developer or not. It is always incomprehensible if we are talking about industrial software development. Opponents of the “black box theory” (the first group) are mainly representatives of the academic community, teachers, or people involved in solving narrow problems, where the extreme efficiency of parallelism is important. For example, numerical programming. Usually they deal with prototypes of programs that calculate something, which they are well versed in and know which parallelization algorithms should be applied in each specific case. Programs are small, they can be read or even rewritten if necessary - in another language. The mistake is that they are trying to scale their experience on large software projects.
Analogy with the second group (theorists, or opponents of the black box). A person with a flashlight and a magnifying glass, who is studying with interest a beautiful chest covered with painting and offers ways to make it even more beautiful and comfortable to use.
A real programmer, engaged in the development of a large project, developed over years or even decades, is in completely different conditions. Most likely, he came to the project when megabytes of code had already been written and he did not know the whole history of one or another architectural solution. Perhaps he does not even have anyone to ask. He had no time to read the source codes, and would never have that much time. He works in a particular area of ​​work and may not be aware of how many algorithms work, written long before he joined the development team, but which he has to use.
Analogy with the first group (practitioners, or black box supporters). Man with a flashlight and a magnifying glass in a huge maze. He can carefully study the frescoes on the walls, but the flashlight beam can not highlight the opposite walls and allows you to see the corridors of the maze only in fragments, without creating a holistic picture.
I do not consider the approach to parallel programming, as an experiment with a black box, a good methodology. But it is better than not doing anything or offering, to fully understand the structure of the existing software package and algorithms. This, unfortunately, just does not work. The complexity of many software solutions has grown beyond the consciousness of individuals and even the teams that work with them. Using specialized tools that allow you to somehow solve the problem of paralleling and debugging is the only available option. The experience of developing an extremely effective, but small parallel program is not applicable here.
The profiler will not offer a high-level way to change the algorithm. But it will show how you can at least partially speed up the program by parallelizing the most critical parts of the code. A parallel debugger or code analyzer will not prove the correctness of a parallel program. But they will allow the programmer to hedge up, who has forgotten to shine a flashlight in one of the directions, and thus has lost an important element from the look.
I support the method of using tools that allow and help parallelize not only well-understood code, but also black boxes, as practically the only way to work with large software systems.
Similar disagreements existed once between supporters and opponents of using debugging tools (“debuggers”). Opponents argued that debuggers are a consequence of the wrong methodology, and debugger users are simply illiterate. Their main argument was that the use of debugging tools encourages the writing of code by trial and error, without trying how to think out the algorithm and rewrite it, break it into small functions, make it more concise and simple - so that there is no room for errors. Since there are few opponents of debugging tools now, many may not even have heard of such disputes. Well, now we are seeing the beginning of a new round.
')
At present, two approaches to the problem of program parallelization have been formed and are being discussed. Proponents of the first want to get some means that will allow them with a minimal rework of the program to carry out its parallelization. These are practices that want to adapt their solutions to the capabilities of multi-core systems that appear on the tables of ordinary users. The latter like to talk about various parallelization theories, parallel programming languages ​​and the need to clearly understand how parallel code will work. Indeed, in their opinion, otherwise it is impossible to achieve its correctness and efficiency.
I agree with the second group, which calls for conscious action, whether debugging or experimenting with the use of OpenMP directives. Such an approach is provoked by the creation of less reliable programs, because a valid code is recognized not as a correct code, but as a code that works correctly on a number of tests. This is definitely bad. But with my soul, I fully support the first group.
I am a practitioner. And I think that a long time ago nobody knows how real programs work. And it makes no sense to say whether the program being debugged or parallelized is clear to the developer or not. It is always incomprehensible if we are talking about industrial software development. Opponents of the “black box theory” (the first group) are mainly representatives of the academic community, teachers, or people involved in solving narrow problems, where the extreme efficiency of parallelism is important. For example, numerical programming. Usually they deal with prototypes of programs that calculate something, which they are well versed in and know which parallelization algorithms should be applied in each specific case. Programs are small, they can be read or even rewritten if necessary - in another language. The mistake is that they are trying to scale their experience on large software projects.
Analogy with the second group (theorists, or opponents of the black box). A person with a flashlight and a magnifying glass, who is studying with interest a beautiful chest covered with painting and offers ways to make it even more beautiful and comfortable to use.
A real programmer, engaged in the development of a large project, developed over years or even decades, is in completely different conditions. Most likely, he came to the project when megabytes of code had already been written and he did not know the whole history of one or another architectural solution. Perhaps he does not even have anyone to ask. He had no time to read the source codes, and would never have that much time. He works in a particular area of ​​work and may not be aware of how many algorithms work, written long before he joined the development team, but which he has to use.
Analogy with the first group (practitioners, or black box supporters). Man with a flashlight and a magnifying glass in a huge maze. He can carefully study the frescoes on the walls, but the flashlight beam can not highlight the opposite walls and allows you to see the corridors of the maze only in fragments, without creating a holistic picture.
I do not consider the approach to parallel programming, as an experiment with a black box, a good methodology. But it is better than not doing anything or offering, to fully understand the structure of the existing software package and algorithms. This, unfortunately, just does not work. The complexity of many software solutions has grown beyond the consciousness of individuals and even the teams that work with them. Using specialized tools that allow you to somehow solve the problem of paralleling and debugging is the only available option. The experience of developing an extremely effective, but small parallel program is not applicable here.
The profiler will not offer a high-level way to change the algorithm. But it will show how you can at least partially speed up the program by parallelizing the most critical parts of the code. A parallel debugger or code analyzer will not prove the correctness of a parallel program. But they will allow the programmer to hedge up, who has forgotten to shine a flashlight in one of the directions, and thus has lost an important element from the look.
I support the method of using tools that allow and help parallelize not only well-understood code, but also black boxes, as practically the only way to work with large software systems.
Source: https://habr.com/ru/post/78350/
All Articles