Test C compilers for Windows

After numerous disputes on the topic “Which compiler generates code better,” an idea appeared to carry out the test itself. The main purpose of the test was to check the speed of the program with speed optimization. Test result: arithmetic average time of test function execution in milliseconds (1/1000 s). those. less is better.

Participated in testing:

• Intel C ++ Compiler Pro 11.1.054;
• GCC 4.5.0 (MinGW);
• MS C / C ++ Compiler 15.00.21022.08 (VS 2008);
• CodeGear C ++ Builder 11.0 (C ++ Builder 2007);
• Tiny C Compiler 0.9.25.

Iron for dough:

• Computer: CPU Intel E5200 (2-core) 2.5 GHz + 2 GB of RAM;
• Laptop: CPU AMD Athlon QL-62 (2-core) 2 GHz + 3 GB of RAM.

OS for test:

MS Windows XP SP3 Eng x32 on a laptop and on a computer (the same SP3 was installed from the same disk).
')
Compilation options:

1. Any optimization is disabled;
2. All possible optimization is included.

Restrictions on testing:

• The source code of the test program does not change depending on the compiler;
• The test function does not use the system functions, i.e. only computational operations, all functions associated with calling system functions are called before and after the time measurement;
• Parallel libraries of the OpenMP type are not used;
• Calculations are performed in only one thread;
• The computer is not loaded by any other programs anymore, only running Windows + Notepad + test program;
• For tests not used VCL, MFC, CLR, ATL;
• The program code was compiled as C code, not C ++;
• For Tiny C Compiler, only 1 compilation option was used, because it does not support code-level optimization. From the documentation: , , . , . , , . , .

Testing Method:

1. Memory allocation for buffers;
2. Getting UserTime of the current thread via GetThreadTimes;
3. Execution of the test function;
4. Getting UserTime of the current thread via GetThreadTimes;
5. Reception of a difference in time with an accuracy of milliseconds (1/1000 sec.);
6. Repeat the last 4 actions 10 times;
7. Calculation of the arithmetic mean time.

Algorithm of computing function:

1. Initialization of the key sequence for the RC4 encryption algorithm;
2. Initialization of the key sequence for the AES-128 encryption algorithm;
3. Filling the first test buffer with data obtained from the RC4 generator;
4. Calculate CRC32 for the first test buffer;
5. Encryption of the first test buffer by the AES-128 algorithm, in blocks of 128 bits, with the result placed in the second test buffer;
6. Filling the first test buffer with data obtained from the RC4 generator, i.e. initial data is completely erased;
7. Decryption of the second test buffer with placing the result in the first test buffer;
8. CRC32 count for decrypted first test buffer;
9. CRC comparison before encryption and after.

Test parameters:

• The number of data for encryption is 1600 kilobytes (102400 blocks);
• The number of test iterations for calculating the arithmetic average time is 10.

Test results:
Intel C ++ Compiler Pro 11.1.054

• Laptop without optimization: 6301 ms;
• Optimized laptop: 971 ms;
• Computer without optimization: 4541 ms;
• Computer optimized: 867 ms.

GCC 4.5.0 (MinGW)

• Laptop without optimization: 6568 ms;
• Optimized laptop: 1691 ms;
• Computer without optimization: 4979 ms;
• Computer optimized: 1521 ms.

MS C / C ++ Compiler 15.00.21022.08 (VS 2008)

• Laptop without optimization: 5149 ms;
• Optimized laptop: 1574 ms;
• Computer without optimization: 3740 ms;
• Computer with optimization: 1290 ms.

CodeGear C ++ Builder 11.0 (C ++ Builder 2007)

• Laptop without optimization: 4982 ms;
• Optimized laptop: 3854 ms;
• Computer without optimization: 4006 ms;
• Computer with optimization: 3185 ms.

Tiny C Compiler 0.9.25

• Laptop: 6275 ms;
• Computer: 4606 ms.

More clearly: Code execution timeline: Graph of the speed of code execution relative to the test leader (test leader - 100%)

Results:
According to the results of testing for speed optimization, compilers occupy the following places:

1. Intel C ++ Compiler Pro 11.1.054;
2. MS C / C ++ Compiler 15.00.21022.08 (VS 2008);
3. GCC 4.5.0 (MinGW);
4. CodeGear C ++ Builder 11.0 (C ++ Builder 2007);
5. Tiny C Compiler 0.9.25.

As you can see, the guys from Intel did a good job (32% of the code worked faster than the nearest opponent) and their code has excellent optimization, regardless of whether it runs on an Intel or AMD processor.

At the same time, C ++ Builder showed itself not with the best side (2 times behind), which indicates a slightly different specificity of its application.

Well, about Tiny C Compiler 0.9.25 and there can be no talk, because it doesn’t support conversion optimization at all, and it turns out that the speed of the program is on par with other compilers without optimization when compiling.

Of course, C ++ Builder turned out to be a bit old because I didn’t find a more recent version. Although it seems to me, there is little that has changed in this regard.

Conclusion
According to the results of testing, it is impossible to judge about the compiler, that it is good or bad, because everyone has his own application, and you can only talk about which one of them is suitable for creating software products related to computational operations.