Unit testing and continuous integration with Jenkins for C ++ projects

I think everyone knows what unit tests are, everyone knows, or at least heard what continuous integration is, and many people program in C ++. But I was faced with the fact that the Internet is not so much information about how to combine it all and make it work together. This article is an attempt to give newbies a step-by-step instruction that will allow you to take the first step in creating unit tests for C ++ projects and organize the run-through of unit tests using a CI server.
Update: Plan:

Write HelloWorld
Let's set up assembly of HelloWorld on Jenkins
Let's write a unit test for HelloWorld
Let's configure run of modular tests on Jenkins

Warning: a lot of letters and screenshots, half of which are redundant. Especially for those who are already in the subject.

Prerequisites

')
To achieve the task we need:

Access to version control system (subversion, mercurial or git). In this article, I use the subversion repository on my own server. You can use, for example, any free repository like GitHub or bitbucket.
An assembly server or a continuous integration server (Continuous integration server). This article uses Jenkins. But there are other good options. This server should be able to build C ++ projects. You also need the xUnit plugin.
Any development environment on your desktop. In this article I use Netbeans 7.3 C ++ edition.
C ++ compiler. I am using gcc.

Just a few words why Netbeans. This development environment is the only one that includes integration with version control systems and unit test support for C / C ++ at the IDE level. This does not mean that other IDEs cannot be used to create unit tests, this means that it may be slightly easier to start developing unit tests in this IDE than with other IDEs. That is, easier for beginners.

In order for me not to shower tomatoes on Windows and Visual Studio, I would like to make a reservation that Visual Studio in this sense, they say, is very good. But I do not use Windows.

Create a HelloWorld project

Optional. I recommend creating a new repository, if you have such an opportunity. The principle of "one project - one repository" is a very good principle, in my opinion.
Get a local copy of the repository you are using. For subversion it is checkout, for git and mercurial it is clone.
Create a folder for the project in the repository (this is if you do not have a separate repository). In my case, this is the 'unittests' folder in the svn repository.
Launch Netbeans:
Create a new project: menu File -> New Project.
Select "C / C ++ Application" as the project type
Specify the name of the project (I have 'helloworld'), and the project location (Project Location) in which it will be (in my screenshot this is the same folder that I received from the subversion server. In the 'Project Folder' field you will get the full path where the resulting project will be stored. The remaining fields can be left unchanged.
Click on the 'Finish' button.
After this, a project with a single source file will be created.
Please note that in the file list, the file 'main.cpp' is marked in green. This color marks new files that are missing in the repository.
Select in the main menu: Team -> Subversion -> Show Changes (you can right-click on this file and select Subversion -> Show Changes in the popup menu, but in the future I will write an option through the main menu). At the bottom of the screen there is a list of changed files (for now this is only main.cpp).
You can right-click on this file and see, for example, diff. Although this is not yet interesting, since the repository was still empty.
Let's change the changes. Select the root in the project tree and in the main menu: Team -> Subversion -> Commit. As we see here, Netbeans will create the 'helloworld' folder in the repository and the main.cpp file in it. And also all the project files. Here you can (need) add a comment to the commit.
Please note that after the commit in the list of files main.cpp became black. That is, there are no changes in this file.
It would be nice to try to build and run the resulting project. Press the Play button (or the main menu Run -> Run Project, or simply F6). If everything is normal, then after the assembly you will see a message like: “RUN FINISHED; exit value 0; real time: 0ms; user: 0ms; system: 0ms "
If this does not happen, then most likely there is some problem with the settings of the PATH environment variable or the lack of libraries. Look carefully at the messages you receive.
Add a message type to the main function:
```
cout << "Hello World!" << endl; 
```
And add a header file connection:
```
 #include <iostream> 
```
Run again, F6.
Note that the added lines are highlighted with a green highlight to the left of the line, and the modified file in the file list is now blue. In fact, modern IDEs are very good clients for version control systems. If you study the features that are available in the Team menu, then it is possible that for some of the version control systems you can refuse to use third-party version control clients.
Now diff files look more fun.
Let's make this change as well.

Build from the command line

In order to do the automatic assembly, we need some kind of assembly script for the assembly from the command line. A good option would be a 'makefile' (although I know people who prefer to build C ++ projects with 'ant'). If you know well how to create it, then do it and do not read this paragraph further. And if you are a beginner, you need to come up with something.

Creating a makefile is a separate large topic that does not fit into the scope of this article. Therefore, we will go the easiest way for us. Netbeans keeps its project, in fact, in the form of a makefile (but also with a bunch of additional files). We will use it as the makefile.

Generally speaking, the question of whether to store project files in the repository or not is a very controversial issue. I think not. But the build script must be in the repository. The Netbeans project in this case is such a compromise. Since, as I said, the Netbeans project files are, in fact, an assembly script. Not the best, but still an assembly script.

Open a terminal (cmd.exe or FAR for Windows users).
Go to the project folder ('/ home / risik / work / unittests / helloworld' for me).
Run the command 'make clean' from the command line, and then 'make'
If you had no problems building a project from Netbeans, then there should be no problems.
It is these files that we committed earlier together with the project files. But you, for some reason, did not do this, then this is the time to do it.
From Netbeans: right-click on a project, and further: Subversion -> Commit. However, it seems there is a slight error in Netbeans. Among other files in my project folder there is a file called '.dep.inc', which is needed for correct handling of dependencies. You can do without it - the assembly works, but it is better to add it now manually.
The second option is from the command line or what you usually use to work with your version control system.

I got the following list of files:
project root:

Makefile
.dep.inc

nbproject folder:

Makefile-Debug.mk
Makefile-Release.mk
Makefile-impl.mk
Makefile-variables.mk
Package-Debug.bash
Package-Release.bash
configurations.xml
project.xml

folder nbproject / private:

Makefile-variables.mk
configurations.xml
private.xml

Jenkins Initial Setup

Open Jenkins in your browser (in my screenshots it is located at 192.168.1.109:8080/jenkins).
Create a new task (Jenkins - New Job)
Specify a name for the task and select the task type “Build a free-style software project”.
Set the configuration, similar to that shown in the screenshots.
Let us consider in more detail the fields that should be modified.
- Job title. I just called it - helloworld-cpp
- In the 'Source Code Management' section, specify the type of version control system used. I have this 'Subversion'.
- Additional fields are different for different systems of version control, for SVN the most important of them is the URL of the repository.
- Attention: when you ask this URL, then most likely, your Jenkins will say something like "I can not connect to the repository." At the end of a long red trace of the log you can find the words: "(Maybe you need to enter credential?)" And a link to enter your login and password. Enter your login and password for the repository, and Jenkins will remember it. Another option is to specify a username and password directly in the URL. Or the third option is to place the authentication information right in the user's home folder under which Jenkins works.
- In the 'Build Triggers' section, I indicated the SCM build. That is, the assembly is scheduled. '* / 15 * * * *' means build every 15 minutes.
- Note: it is generally better to use push assembly by trigger from your version control system instead of SCM, but creating this configuration is beyond the scope of the article.
- In the 'Build' section, add a new build step of the 'Execute Shell' type. If your Jenkins works on Windows and you do not have cygwin installed on this server (and how do you live, if so?), Then it may be better to select the 'Execute Windows Batch Command' option.
Click the save button and let's try building a project.

I fell it. The problem is easily localized if you look at the 'Console Log':

 Started by user anonymous Building in workspace /var/lib/jenkins/jobs/helloworld-cpp/workspace Checking out a fresh workspace because there's no workspace at /var/lib/jenkins/jobs/helloworld-cpp/workspace Cleaning local Directory . Checking out https://sergeyborisov.com/svn/teach/kcup_unittests/helloworld at revision '2013-02-25T01:37:54.054 +0700' A main.cpp A nbproject A nbproject/Makefile-Release.mk A nbproject/Makefile-impl.mk A nbproject/Package-Release.bash A nbproject/project.xml A nbproject/Makefile-Debug.mk A nbproject/Makefile-variables.mk A nbproject/configurations.xml A nbproject/private A nbproject/private/Makefile-variables.mk A nbproject/private/configurations.xml A nbproject/private/private.xml A nbproject/Package-Debug.bash A Makefile U . At revision 132 [workspace] $ /bin/sh -xe /tmp/hudson8376440745271858508.sh + make all /tmp/hudson8376440745271858508.sh: 2: /tmp/hudson8376440745271858508.sh: make: not found Build step 'Execute shell' marked build as failure Finished: FAILURE

Well, everything is simple. I did not have the server command 'make'.
And after the second iteration, I learned that I was missing g ++. Well, it is also easy to eliminate.

Only the third time, everything turned out well.

 Started by user anonymous Building in workspace /var/lib/jenkins/jobs/helloworld-cpp/workspace Updating https://sergeyborisov.com/svn/teach/kcup_unittests/helloworld at revision '2013-02-25T01:42:03.200 +0700' At revision 132 no change for https://sergeyborisov.com/svn/teach/kcup_unittests/helloworld since the previous build [workspace] $ /bin/sh -xe /tmp/hudson8326466894395366933.sh + make all for CONF in Debug Release ; \ do \ "make" -f nbproject/Makefile-${CONF}.mk QMAKE= SUBPROJECTS= .build-conf; \ done make[1]: Entering directory `/var/lib/jenkins/jobs/helloworld-cpp/workspace' "make" -f nbproject/Makefile-Debug.mk dist/Debug/GNU-Linux-x86/helloworld make[2]: Entering directory `/var/lib/jenkins/jobs/helloworld-cpp/workspace' mkdir -p build/Debug/GNU-Linux-x86 rm -f build/Debug/GNU-Linux-x86/main.od g++ -c -g -MMD -MP -MF build/Debug/GNU-Linux-x86/main.od -o build/Debug/GNU-Linux-x86/main.o main.cpp mkdir -p dist/Debug/GNU-Linux-x86 g++ -o dist/Debug/GNU-Linux-x86/helloworld build/Debug/GNU-Linux-x86/main.o make[2]: Leaving directory `/var/lib/jenkins/jobs/helloworld-cpp/workspace' make[1]: Leaving directory `/var/lib/jenkins/jobs/helloworld-cpp/workspace' make[1]: Entering directory `/var/lib/jenkins/jobs/helloworld-cpp/workspace' "make" -f nbproject/Makefile-Release.mk dist/Release/GNU-Linux-x86/helloworld make[2]: Entering directory `/var/lib/jenkins/jobs/helloworld-cpp/workspace' mkdir -p build/Release/GNU-Linux-x86 rm -f build/Release/GNU-Linux-x86/main.od g++ -c -O2 -MMD -MP -MF build/Release/GNU-Linux-x86/main.od -o build/Release/GNU-Linux-x86/main.o main.cpp mkdir -p dist/Release/GNU-Linux-x86 g++ -o dist/Release/GNU-Linux-x86/helloworld build/Release/GNU-Linux-x86/main.o make[2]: Leaving directory `/var/lib/jenkins/jobs/helloworld-cpp/workspace' make[1]: Leaving directory `/var/lib/jenkins/jobs/helloworld-cpp/workspace' Finished: SUCCESS

Creating tests in Netbeans

For now, leave Jenkins alone and go back to Netbeans. To begin with, we modify our project a little. In order for it to have at least something that can be tested, let the welcome line form a separate class.

Right-click on 'Source Files' and select New -> C ++ Class from the context menu.
In the class creation dialog, specify the name of a new class, for example, Helloer (forgive me for this name, but I could not think of what to call it). In general, it would be right here to set the folder for the new file. Well, at least 'src'. But I was too lazy.
A list of files will appear in the file list — Helloer.cpp and Helloer.h. In Netbeans, they are created in one folder, and in other IDEs they can be created in two separate folders, for example, 'src' and 'include'.
As you can see, the constructor without parameters, copy constructor and virtual destructor are created automatically.
First, we commit to what was generated, and then add what we need.
That is, I added to the Helloer class a constructor with a string, as a parameter, which will indicate who exactly we will greet. And also added a method for receiving a greeting message.
Check out the changes. Please note that the project files have also been modified, they must also be committed.
Open Helloer.h, right-click inside the file and select Create Test -> Create CppUnit Test.
In the dialog that appears, select the items to be tested. I selected only the message method.
In the next dialog, enter the name of the test. By the way, if Netbeans did not find the required library (cppunit), he will tell you about it here. I have this package libcppunit-dev.
That's what I did after generating the tests.
The HelloWorldTests branch appeared in the project structure - this is the name I gave to the whole group of tests in the test configuration dialog. And in it, the test run module of this group (Runner), which I have called 'HelloWorldTestRunner.cpp' and so far the only test class is HelloerTest (.cpp + .h).
Let's try to run. Menu Run -> Test Project. uh ... I got a file — the unit tests didn't come together. Since there was no header file 'Helloer.h'. Well, here it is simple - you need to add the path to the file. To do this, call the context menu on the HelloWorldTests test group and select the Properties item. You will see the project folder properties dialog.
In this dialog, in the 'C ++ Compiler' section, find the 'Include Directories' item and click the '...' button on the right.
In the 'Include Directories' dialog that appears, click the 'Add' button
Finally, select the folder where your header file is located. Since this file is part of the project, the path to the file should be stored in a relative form, which is indicated in the right part of the dialog. Since I was too lazy to create even the 'src' folder when I created the Helloer class, I’ll have this folder as the project folder. That is, the folder '.'.
Close the properties dialog and try to run the unit test again.
This time he got ready, started, but fell, on Assertion.
However, I immediately commited, everything that was generated. And now I will correct the test:
And the changes are also commited. Please note that you need to commit not only source code, but also project files.
Open the file 'HelloWorldTestRunner.cpp' if you look closely, its structure is simple and clear:
- Prepare the test run.
- We chase them.
- We print the result.
Netbeans generated print results in the 'compiler compatible' format. This is a very convenient format for working with Netbeans, but an inconvenient format for Jenkins. Therefore, I will also add printing in XML format:
```
 ofstream xmlFileOut("cpptestresults.xml"); XmlOutputter xmlOut(&result, xmlFileOut); xmlOut.write(); 
```
Naturally you need to include the necessary header files:
```
 #include <cppunit/XmlOtputter.h> #include <ostream> 
```
Note: print in XML format, not instead of text, but with text.
The file name (“cpptestresults.xml” for me) and the path to it could have been chosen by another. But most importantly, what would you know what. We will need this information soon.

Customize the run of autotests in Jenkins

I opened the browser with Jenkins and I see that he has already done several assemblies. After all, I pointed out to him the SCM build every 15 minutes. And consequently, if at the next check, which is carried out every 15 minutes, as you might guess, there were new commits to the repository, then Jenkins collected the project. Well, since I wrote it all leisurely, he found reasons for a couple of runs :)
There is an important nuance here. If the system time of the server with your repository and the system time of the server with Jenkins diverge, then you can get very strange effects. For example, Jenkins will refuse to recognize that possible modifications to the source text in the future. Therefore, I highly recommend setting up time synchronization on all servers and best of all on a common ntp server. Yes, and your desktop would be good to synchronize in the same way.
Add “Post build action” to the job, for example:
We try to collect. and voila! Everything is collected and the test is passed! And you in the build have information about the passed tests:
And after collecting again in the job schedule will appear. Well, while I have it like this:
But in the 'Latest Tests Results' you can see more detailed information about all passed and failed tests (so far the only one):
Do not be embarrassed that you see (root) in the table of the general list of tests. Jenkins is designed for Java. And here should have been the names of the packages.
Let's do one more test. At the same time, we will immediately bear in mind a slight modification of the functionality: If there is nobody to greet (the 'who' line is not set), then there is no need to say any greetings.
That is, we must add the following lines to the header file:

and this method to the .cpp file:

 void HelloerTest::testMessageNobody() { Helloer helloer; string result = helloer.message(); if (true /*check result*/) { CPPUNIT_ASSERT(result == ""); } }

commit And now either we are waiting for the next launch of the assembly via SCM or we start the assembly manually.
Now we see that we have a fallen test!
True, the assembly is still marked as successful. We return to this issue later.
But the trend has become prettier:

Fix the resulting test fail:

 string Helloer::message() const { if (who.length() == 0) return ""; return (string)"Hello " + who; }

And again run the tests in the IDE. Now everything is OK. Let's finish it, collect Jenkins on the side and see that everything is fine there too (I managed to get through the assembly with the still broken dough).
A few words about the status of the assembly. Jenkins allows you to configure the formation of the status is quite flexible. This configuration can be seen in the setting of the xUnit step in the job - the 'Failed Tests' and 'Skipped Tests' methods. How it will be better specifically for you - I do not know. But I believe that every new test that has fallen should result in a breakdown of the assembly, that is, the assembly should be marked as red. But those tests that were broken earlier, should lead to a "yellow status." Experiment!

Well that's all! If you still got here through all the steps, and not just scrolled to the bottom of the page, then you have taken the first step in automating the run of the unit tests in your C ++ project. And the first step to TDD. Good luck!