Accelerate Visual Studio Part I. Unity Builds

This is a translation of the Oliver Reeve article on one of the ways to speed up the compilation of a project. The author managed to speed up the compilation from 55 to 6 minutes. In my project, I received a performance boost of about 22% (about a minute). This is not as striking as the author’s achievements, but still, by multiplying this minute by the number of compilations per day, the number of developers, and the duration of project development, I received savings that exactly justified the costs of reading the article and setting up projects. The solution for Visual Studio and C ++ is described, but the idea also applies to other IDEs, compilers and programming languages ​​(not all). In the next article, I looked at a couple more ways to speed up the compilation of a solution.


Everyone who worked on a large C ++ (or C) project felt the full horror of a long compilation time. The first “big” C ++ project I was working on was about 10 minutes. It was much longer than what I worked with earlier. Later in my career, when I joined the game development industry, I felt the challenge of compiling a truly large project. Our game was going about an hour.

And here is the solution that we implemented: Unity Builds (hereinafter - UB ).
')
Some readers who are familiar with C and C ++ may decide that what I will describe next is some kind of "hack." I do not have a specific opinion on this. Of course, the idea has something “hack-like”, but it does solve the problem of lengthy compilation on most platforms and for most compilers.

Before I delve into the details and tell you how to configure UB , I would like to clarify that this mechanism is not intended to replace the usual way of building a release version of a solution. The main idea is to significantly reduce the build time for developers who at least 8 hours a day are busy modifying the code. Fixing bugs and adding functionality means constant recompilation. Each compilation is an expectation that everyone fills as best he can.


But still, every compilation is a loss of precious time, a drop in developer productivity. This is sad even when compiling small solutions, and even in large ones it becomes the number one problem. Nevertheless, let's not forget that normal release versions of projects should be assembled on separate computers and, of course, in the usual way. That is, there will be no changes in this procedure.

The basic principle of UB is quite simple. It consists in reducing:

• the number of openings for each individual source file.
• the number of files that open when compiled.

Each source file (.c, .cxx, .cc, .cpp, etc.) is compiled into an object file. When all source files are compiled into object files, the linker collects all object files and creates an executable program from them. For those of you who do not know: reading / writing to disk is the slowest operation that takes the compiler the most time (it's not so much reading and writing as in search (positioning) time, but this is not for us the essence is important). Thus, if we want to reduce the compilation time, it would be good to start with a decrease in the number of input / output operations.

I don't want to turn this article into a lesson on how C ++ programs are compiled, so I’ll just remind you of some details. We know that the preprocessor parses all files before compiling them (and for this, it opens and reads them). Later, the compiler again opens the already parsed files. Creating and writing each object file is again a waste of time. And finally, the linker, with its procedures for reading object and creating executable files. It all takes time to hell.

And here it is the way to speed up the whole thing essentially: we will not compile every single source code file in the project. Yes, we will not. Instead, we will create a master file in which we will include all other files in the project using the include directive. We will compile this master file.

Now we will try to demonstrate the idea in action in Visual Studio (however, the principle can be used in other IDEs and with other compilers). The first step is to create a separate configuration of the assembly, on which we will experiment without the risk of breaking the standard assembly procedure. Open Configuration Manager , click New Config and create a new configuration based on our configuration Debug (let's call it UnityDebug ). Next, we will add the master file to the project (let's call it UnityBuild.cpp) and include with the help of the include directive into it all the other files of the project source code.

If we try to compile the project at the moment, we will naturally receive a bunch of messages about duplication of various classes, functions and other things. To fix this, we exclude from the list of compiling files in the UnityDebug configuration all files except UnityBuild.cpp, and in other configurations - the UnityBuild.cpp file. (To exclude a file from the compilation list, open its properties, find the “Excluded From Build” item and set it to “Yes” for all the necessary configurations).

We have almost everything ready and we can try to compile the solution. Next, we have two options: either everything will compile or not. In the first case, I congratulate you - you can already estimate the performance gain. In the second case, have to think. We need to start thinking with the understanding that with our build scheme, the preprocessor collects all the source files into one large file. Thus, errors related to global and static variables and functions, whose names can be repeated in some places, can come out. In an amicable way, you have to have a few such things, right? Well, there will be another reason to reduce their number and influence on each other.

Now let's evaluate the performance gain. When UB was implemented in one game development company where I was working, the full compilation time decreased from 55 minutes to 6. In another case, the time changed from 10 minutes to just over two.

Do not forget that you still have the previous configurations of the project assembly and you can use them or UB to your liking at each assembly. In addition, you should not forget about the need to update the master file when adding new source files to the project. This, of course, somewhat complicates the process, but a few seconds to add one line to the master file are fully compensated by the significant acceleration of your work.

And finally, here is a link to the video , which shows in practice everything that was described in the article.