Qt parallel compilation in Windows
Parallel compiling Qt projects under Windows using QtCreator is a mysterious and fastidous beast. In this small article I will tell you how to tame him after all. Parallel compilation can be done quite simply in theory, in practice, things are not quite smooth, which is supported by an infinite number of topics on forums where various solutions are offered. None of them, unfortunately, helped us.
In theory, for the MinGW case, it suffices to specify the –j parameter [number of compilation processes] for the make command, which is represented in mtw32-make in QtSDK MinGW. For example, the command mingw32-make.exe –j10 is obtained.
The recommended number of compilation processes is the number of processors + 1. If the parameter is not specified, it is considered that j = 1. If you specify too large a number, nothing terrible will happen either. Make will run exactly as much as a performance boost.
')
For convenience, you can use the environment variable% NUMBER_OF_PROCESSORS%, which indicates the number of processors in the system. It turns out a command of the form
However, not all so simple. Qmake creates three makefiles. A generic Makefile that, according to certain definees, chooses from the Makefile.Debug and Makefile.Release depending on the release build or debug. The fact is that the –j parameter is not inheritable . And when make is invoked in QtCreator, it is called for a Makefile. But to Makefile.Debug or Makefile.Release no longer gets.
In order to fix this, you will have to write a command explicitly . For example,
Not very convenient, but it greatly compensates for the increase in assembly speed. For example, rebuilding our project on core-i7-2630 without parallelization is done in 12 minutes, with the –j9 flag it takes less than two minutes to build. A performance gain of six times makes you wonder.
But even after specifying such build flags, we started working on two machines, but not on two. On the forums they write that the case may be in the Qt version, in the mingw curve, in the features of qmake, the OS version. However, the configurations of our machines are such that by the method of elimination it is possible to conclude that the versions of QtSDK, QtCreator, Windows are not related, or the system capacity to all this.
The solution is very simple, but at the same time not so obvious. You need to add the absolute path to mingw32-make to the PATH environment variable. Just in case, it is better to put a semicolon after the path. And after that, a parallel compilation miraculously begins to work, and programmers will not have to be distracted for 10 minutes each time some Q_OBJECT is added to the project and its reassembly.
All these compilation flags can be set in QtCreator on the Projects tab in the make parameters.
I hope the article will be useful and will solve the questions of many who have not had a parallel build in QtCreator before.
UPD (from comments) : As for Linux / MacOS users, they are much more fortunate - just the -jX switch is enough.
Also, if you use the MSVC compiler, then there should be no problems.
In the case of QtCreator, you need to add an entry.
in the .pro project file and setting the environment variable as I described.
In the case of Visual Studio, the properties of the VS project are: “Properties” => “Configuration Properties” => “C / C ++” => “Command Line”. And in the field “Additional options” add -MP [number of processes]. Thanks IGHOR
Also when using the MSVC compiler jom can help.
The constant part of the flags can be moved to a separate environment variable MAKEFLAGS. Make will take the set flags from there. For example, MAKEFLAGS = -j10. And in the make parameters you just have to remember to specify the file for the assembly. For example, -f Makefile.Debug. If you keep on assembling one version all the time, then you can put out all the flags of the pillars in MAKEFLAGS and forget about setting up future projects.
Thank you all for the additions.
In theory, for the MinGW case, it suffices to specify the –j parameter [number of compilation processes] for the make command, which is represented in mtw32-make in QtSDK MinGW. For example, the command mingw32-make.exe –j10 is obtained.
The recommended number of compilation processes is the number of processors + 1. If the parameter is not specified, it is considered that j = 1. If you specify too large a number, nothing terrible will happen either. Make will run exactly as much as a performance boost.
')
For convenience, you can use the environment variable% NUMBER_OF_PROCESSORS%, which indicates the number of processors in the system. It turns out a command of the form
mingw32-make.exe –j%NUMBER_OF_PROCESSORS%However, not all so simple. Qmake creates three makefiles. A generic Makefile that, according to certain definees, chooses from the Makefile.Debug and Makefile.Release depending on the release build or debug. The fact is that the –j parameter is not inheritable . And when make is invoked in QtCreator, it is called for a Makefile. But to Makefile.Debug or Makefile.Release no longer gets.
In order to fix this, you will have to write a command explicitly . For example,
mingw32-make.exe –j9 –f Makefile.Debug.Not very convenient, but it greatly compensates for the increase in assembly speed. For example, rebuilding our project on core-i7-2630 without parallelization is done in 12 minutes, with the –j9 flag it takes less than two minutes to build. A performance gain of six times makes you wonder.
But even after specifying such build flags, we started working on two machines, but not on two. On the forums they write that the case may be in the Qt version, in the mingw curve, in the features of qmake, the OS version. However, the configurations of our machines are such that by the method of elimination it is possible to conclude that the versions of QtSDK, QtCreator, Windows are not related, or the system capacity to all this.
The solution is very simple, but at the same time not so obvious. You need to add the absolute path to mingw32-make to the PATH environment variable. Just in case, it is better to put a semicolon after the path. And after that, a parallel compilation miraculously begins to work, and programmers will not have to be distracted for 10 minutes each time some Q_OBJECT is added to the project and its reassembly.
All these compilation flags can be set in QtCreator on the Projects tab in the make parameters.
I hope the article will be useful and will solve the questions of many who have not had a parallel build in QtCreator before.
UPD (from comments) : As for Linux / MacOS users, they are much more fortunate - just the -jX switch is enough.
Also, if you use the MSVC compiler, then there should be no problems.
In the case of QtCreator, you need to add an entry.
QMAKE_CXXFLAGS_RELEASE += -MP[ ]in the .pro project file and setting the environment variable as I described.
In the case of Visual Studio, the properties of the VS project are: “Properties” => “Configuration Properties” => “C / C ++” => “Command Line”. And in the field “Additional options” add -MP [number of processes]. Thanks IGHOR
Also when using the MSVC compiler jom can help.
The constant part of the flags can be moved to a separate environment variable MAKEFLAGS. Make will take the set flags from there. For example, MAKEFLAGS = -j10. And in the make parameters you just have to remember to specify the file for the assembly. For example, -f Makefile.Debug. If you keep on assembling one version all the time, then you can put out all the flags of the pillars in MAKEFLAGS and forget about setting up future projects.
Thank you all for the additions.
Source: https://habr.com/ru/post/137428/
All Articles