Custom jUnit processing of tests in TeamCity

TeamCity supports jUnit "on the fly" and there are no special problems with the execution of tests. But standard support does not cover all user cases. For example, one can never be sure in which order the tests will pass. In addition, there are other variations of the test architecture, which are simply impossible to do with default jUnit tools. For example, the definition in runtime, what tests you need to run, and what not. And with the output in the reports in TeamCity without ignored tests.

This is sometimes really confusing. I had a situation when tests for a web server were written and everything worked well. But somehow the web server itself fell (in the tests there was no logic to stop / start the web server), but some of the tests in the TeamCity reports were marked as successful (all ignored ones are marked in the list). Naturally the customer said "What the ...".
It seems to be nothing complicated - to use the TeamCity service messages. But in TeamCity there is an official bug in which the service messages in TestOutput are not read.

1. Displaying messages

For convenience, create a custom class for displaying messages. To transfer a command to TeamCity, you must send it to the standard out stream.

public class Log { public static void logTCTestSuiteStart( String message ) { System.out.println( "##teamcity[testSuiteStarted name='" + message + "']" ); } public static void logTCTestSuiteFinished( String message ) { System.out.println( "##teamcity[testSuiteFinished name='" + message + "']" ); } public static void logTCTestStart( String message ) { System.out.println( "##teamcity[testStarted name='" + message + "']" ); } public static void logTCTestFinished( String message ) { System.out.println( "##teamcity[testFinished name='" + message + "']" ); } public static void logTCTestFailed( String message, AssertionError e ) { System.out.println( "##teamcity[testFailed name='" + message + "' message='" + e.getMessage() + "']" ); } } 

Each test group should be placed in the TestSuite block using the ' testSuiteStarted ' and ' testSuiteFinished ' commands . Each test must begin with ' testStarted ' and end with ' testFinished '. A test will be considered successful if there is no ' testFailed ' command inside the test block.
More details about the supported service messages in TeamCity can be read here .
')

2. Simple test

Now we can write tests like this:

 public class BasicTests1 { BasicTester tester = new BasicTester( {constructor_args} ); @BeforeClass public static void setUp() { tester.initialize(); } @Test public void orderedTestRun() throws Exception { Log.logTCTestSuiteStart( "Basic tests" ); tester.testOne(); tester.testTwo(); tester.testThree(); Log.logTCTestSuiteFinished( "Basic tests" ); } @AfterClass public static void tearDown() { tester.dispose(); } } public class BasicTester { protected {field1}; protected {field2}; protected {field3}; protected BasicTester( {class_fields} ); protected void testOne() { Log.logTCTestStart( "testOne" ); try { //test logic } catch( AssertionError e ) { Log.logTCTestFailed( "testOne", e ); } Log.logTCTestFinished( "testOne" ); } protected void testTwo() { Log.logTCTestStart( "testTwo" ); try { //test logic } catch( AssertionError e ) { Log.logTCTestFailed( "testTwo", e ); } Log.logTCTestFinished( "testTwo" ); } protected void testThree() { Log.logTCTestStart( "testThree" ); try { //test logic } catch( AssertionError e ) { Log.logTCTestFailed( "testThree", e ); } Log.logTCTestFinished( "testThree" ); } } 

Since we report errors manually, it makes sense to intercept them. Otherwise, they will simply be ignored, and TeamCity, without waiting for the testFailed command, will consider the test successful.

3. Ant task

It's all standard. The only thing is that the ' junit ' task should contain ' showoutput = "yes" '. This will force TestsOuput to output to standard out.

4. TeamCity build settings

Since TeamCity does not read service commands from TestOutput junit, it is necessary to disable the recognition of junit tests altogether. To do this, open the build configuration, go to the 'Build parameters' tab and add the following parameters:

• system.teamcity.ant.junit-support.enabled = false
• system.teamcity.ant.testng-support.enabled = false

That's all. Now you can pervert over jUnit tests in TeamCity as you like.