About unit testing in C ++ and CxxTest

Most recently, during the development of a single project, I was faced with the task of finding a convenient means for writing unit tests in C ++. What results did my research lead to in more detail in this article.


Before embarking on specific tools for creating unit tests, let's define what unit (or module) testing is, its basic principles, and why, in general, we need a separate library for this business.

So, this principle consists in writing small tests for each non-trivial function or method of the source code of our program separately. What does this approach give us? First, after carrying out such testing, we can definitely say that individually all the parts are workable (not to be confused with the concept of “there are no errors”); secondly, we can conduct regression testing. And thirdly, the unit test is a good example of how to use the tested functions and classes, i.e. modular test - a means of indirect documentation.
')
Also, I would like to touch upon the basic, in my opinion, principles of unit testing. We list them:

• Tests should be small and fast.
• Tests should be automatic.
• Tests should not depend on each other and on their order.
• First, we write tests in the black box mode.
• And then, in the "white" mode.
• Found a mistake - a good reason to write a test that reproduces it.
• The test structure should follow the project hierarchy.
• For each of your class, get a separate test class.
• Separate tests from the main project.
• Have a unified system for naming tests (for example, for the class MyClass, the testing class will be named MyClassTest, and all of its methods will look like testWhat we Test ()).

As a result, we need a tool that automates various routine operations (such as, supporting main (), instantiating classes, displaying messages, etc.). Let's put forward several requirements:

• The tool should be easy to run, maintain and master.
• Must be powerful.
• Must be cross-platform.
• Well, if the tool sets some test build framework.

At the moment there are a huge number (as they say, over 9000) of libraries for writing unit tests under C ++. Here for everyone there is a means to their liking. Briefly list a few of them.

Boost :: Test
Boost is one of the most famous (and most large) libraries for C ++, and Boost :: Test is a testing framework included in it and built on macros.
My first acquaintance with Boost left not very good memories - too cumbersome and clumsy, in my opinion, the documentation, besides, this monster weighs quite a lot (yes, you do not need to attach the entire library to the project, you can use Boost :: Test, but download will have a whole).
I will give a small example of a test from the documentation:

Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }

1. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
3. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
4. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
5. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
6. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
8. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
9. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
10. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
11. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
13. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
14. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
15. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
16. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
17. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
18. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
20. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }
21. Copy Source | Copy HTML #include <boost/test/minimal.hpp> Int add( int i, int j ) { return i+j; } Int test_main( int , char *[ ] ) // note the name! { BOOST_CHECK( add( 2 , 2 ) == 4 ); // #1 continues on error BOOST_REQUIRE( add( 2 , 2 ) == 4 ); // #2 throws on error if ( add( 2 , 2 ) != 4 ) BOOST_ERROR( "Ouch..." ); // #3 continues on error if ( add( 2 , 2 ) != 4 ) BOOST_FAIL( "Ouch..." ); // #4 throws on error if ( add( 2 , 2 ) != 4 ) throw "Oops..." ; // #5 throws on error return add( 2 , 2 ) == 4 ? 0 : 1 ; // #6 returns error code }

QTestLib
Also no less well-known library, but according to the reviews it is complicated, and to pull all QT for the sake of tests is not justified.

UnitTest ++
Pretty not a bad and popular thing, but poor documentation.

And finally:
CxxTest
In my opinion one of the best xUnit-like libraries for writing unit tests.
Of the features:

• No need to embed anything in source code
• Does not require installation (you can easily embed in your project in the «test» branch)
• Does not pull any third-party libraries
• It has one of the best, in my opinion, manuals. Learning goes from simple to complex (with various examples and pictures;)).

Of the disadvantages:

• Perl or Python needed. Actually, on Linux, there are both defaults, for Windows Python, I use it as a good service tool.

Now let's see how to work with CxxTest.

1. Create * .h file
2. Create a class inherited from CxxTest :: TestSuite
3. We implement testMethod (void) methods (IMPORTANT: the first word must be test, otherwise the system will not recognize it as a test)
   

   Copy Source | Copy HTML

   1. class MyTest : public CxxTest :: TestSuite
   2. {
   3. public :
   4. void testMethod ( void )
   5. {
   6. TS_ASSERT ( 1 + 1 > 1 );
   7. TS_ASSERT_EQUALS ( 1 + 1 , 2 );
   8. }
   9. };

   
   
4. Run the preprocessor
   

   Copy Source | Copy HTML

   1. # perl cxxtestgen.pl --error-printer -o runner.cpp MyTest.h
   3. MyTest.h - the test we wrote
   4. runner.cpp - output file (the main () function will be created in it)
   5. cxxtestgen.pl or cxxtestgen.py - preprocessor

   
   
5. ... And compile
   

   Copy Source | Copy HTML

   1. # g ++ -o runner runner.cpp

If successful, we get an expression like:

Copy Source | Copy HTML

1. # ./runner
2. Running 1 test.OK!

Well, if something is not right in our source code, then CxxTest will politely indicate to us:

Copy Source | Copy HTML

1. # ./runner
2. Running 2 tests.
3. MyTest.h: 15: Expected (2 * 2 == 5), found (4! = 5)
4. Failed 1 of 2 tests
5. Success rate: 50%

Let's sum up.

CxxTest is quite a convenient framework for writing unit tests in C ++, besides it not only facilitates the work of the programmer, but also contributes to the correct writing of tests.
In addition to the listed features in CxxTest, you can easily make a simple interface for visualizing tests (in the form of a progress bar), and there is also support for Mock objects.

PS
Many thanks to Inversion for the invitation.