How to speed build 1000 VisualStudio C ++ projects
Good afternoon, dear habrayuzer.
I decided to write this topic, so as not to not copy my comment to this post. Here I will just describe our solution.
Before change:
Complete assembly (clean) took about 4.5 hours
The incremental assembly (continuous) took about 30 minutes.
After the change:
Complete assembly - about 40 minutes.
Incremental build - 2-3 minutes
Input data:
Iron (dedicated builder machine): Intel Xeon E5430@2.66Hz, 4 GB RAM
Development environment: Windows XP, 2003, 7, 2008. VisualStudio 6, 2005, 2008, 2012.
The build includes about 1000 C ++ projects: ~ 3MLN LOC
')
How it worked:
We had a file that contained a list of projects that should be compiled in order of priority.
The build script ran through this file and ran the compilation of each project in turn.
Accordingly, the build time grew from month to month linearly to the number of projects, and at some point, it ceased to suit us categorically.
What has been changed:
In sensation, every change accelerated the assembly by 2-3 times.
For us, the most difficult part was to create dependency files for each project. Here I will not describe how we did it, because Not sure if anyone is interested.
I hope that our experience may seem useful to someone.
I decided to write this topic, so as not to not copy my comment to this post. Here I will just describe our solution.
Before change:
Complete assembly (clean) took about 4.5 hours
The incremental assembly (continuous) took about 30 minutes.
After the change:
Complete assembly - about 40 minutes.
Incremental build - 2-3 minutes
Input data:
Iron (dedicated builder machine): Intel Xeon E5430@2.66Hz, 4 GB RAM
Development environment: Windows XP, 2003, 7, 2008. VisualStudio 6, 2005, 2008, 2012.
The build includes about 1000 C ++ projects: ~ 3MLN LOC
')
How it worked:
We had a file that contained a list of projects that should be compiled in order of priority.
The build script ran through this file and ran the compilation of each project in turn.
Accordingly, the build time grew from month to month linearly to the number of projects, and at some point, it ceased to suit us categorically.
What has been changed:
- Replacing drives for SSDs (before that there were quite good HDD drives from WD)
- Integration of IncrediBuild into the build script. Incredibuild we installed on 4 computers. (trivial change if you already have a build script)
- Compiling one huge solution with prescribed dependencies. Solution is generated automatically based on the project list and dependency file for each project.
- Another significant acceleration of the build process, although not related to the speed of compilation, was given to us by a switch to svn 7 (client only). Especially strongly it affects on. assembly when you need to make an update huge working copy. With svn 7, the time dropped from 10 minutes to 1 minute.
In sensation, every change accelerated the assembly by 2-3 times.
For us, the most difficult part was to create dependency files for each project. Here I will not describe how we did it, because Not sure if anyone is interested.
I hope that our experience may seem useful to someone.
Source: https://habr.com/ru/post/188622/
All Articles