Code Coverage analysis for iOS and OS X projects (xCode 4.4)

Foreword

This topic does not aim to talk about code coverage , and whether this tool is necessary or not. Also, the question of the expediency of tests in iOS projects will not be raised (assuming that they are nevertheless necessary to someone, which means there are).

Motivation

It is very convenient when the tools for profiling / analysis are built into the IDE. The history With code coverage in xCode is not completely cloudless: at the time of xCode 3.x and GCC everything was simple and the necessary links and compiler flags were Google. With the advent of xCode 4.1, everything became a bit more difficult due to the use of LLVM-GCC, I had to go for some tricks (up to build LLVM with my own hands). And in 4.3, the library libprofile_rt was moved to another directory, which also caused a lot of problems.

It was experimentally established that for xCode 4.4 code coverage takes a few minutes, and if it is cheap, why not use it? An alternative practical use case that has been tested in practice is testing the project code itself and searching for the dead code, followed by analysis and cleaning.
')

Setting up a project in xCode 4.4

Create a new project (iOS / OS X) with a tick Include Unit Tests . You can use my test project with ready-made unit tests.
Setting up a project involves two steps:
1. Open the target% project-name%, and set the flags in the Code generation section:
Generate Test Coverage Files = YES
Instrument Program Flow = YES



2. Only for iOS. To avoid the ticks described here , you need to add a file with the following contents to the * .c project:

#include <stdio.h> FILE* fopen$UNIX2003(const char* filename, const char* mode); size_t fwrite$UNIX2003(const void* ptr, size_t size, size_t nitems, FILE* stream); FILE* fopen$UNIX2003(const char* filename, const char* mode) { return fopen(filename, mode); } size_t fwrite$UNIX2003(const void* ptr, size_t size, size_t nitems, FILE* stream) { return fwrite(ptr, size, nitems, stream); } 

That's the whole project setup. Now, after running% projectname% Tests, you need to open the directory (for iOS) "/Users/%user%/Library/Developer/Xcode/DerivedData/%project-nameBLABLABLABLA%/Build/Intermediates/%project-name%.build/ Debug-iphonesimulator /% project-name% .build / Objects-normal / i386 " . In this directory, we are interested in the * .gcda and * .gcno files, which contain coverage data. Important : if you are going to test the coverage of the application code, and not the tests, you need to specify UIApplicationExitsOnSuspend = YES in * .plist, since the * .gcda files are created only after the program has been terminated.

For clarity, I cite the code of the class under test and several tests:

 #import "MyCalc.h" @implementation MyCalc - (CGFloat)performOperation:(MyMathOperation)operation withA:(CGFloat)a B:(CGFloat)b { CGFloat result = 0.f; switch (operation) { case MyMathOperationAdd: result = a + b; break; case MyMathOperationSubtract: result = a - b; break; case MyMathOperationDivide: result = a / b; break; case MyMathOperationMultiply: result = a * b; break; default: NSLog(@"Unsupported operation"); break; } return result; } - (CGFloat)negate:(CGFloat)number { //this method works incorrectly return number; } @end 

 - (void)testNegation { CGFloat input = 3; CGFloat expected = -3; CGFloat result = [self.calculator negate:input]; STAssertEquals(result, expected, @"Negation failed. Expected: %f, Actual: %f", expected, result); } - (void)testAddition { CGFloat a = 3; CGFloat b = 4; CGFloat expected = a + b; CGFloat result = [self.calculator performOperation:MyMathOperationAdd withA:a B:b]; STAssertEquals(result, expected, @"Addition failed. Expected: %f, Actual: %f", expected, result); } - (void)testMultiplication { CGFloat a = 14; CGFloat b = 3; CGFloat expected = a * b; CGFloat result = [self.calculator performOperation:MyMathOperationMultiply withA:a B:b]; STAssertEquals(result, expected, @"Addition failed. Expected: %f, Actual: %f", expected, result); } 

Results analysis

Consider several tools for presenting statistics in a human-friendly format.

gcov

gcov is a utility that generates coverage statistics based on * .gcda and * .gcno files. Until recently, it worked only with GCC, currently it works perfectly with LLVM. At the output we get a plain-text report.
For example, here is the result of running on the MyCalc.gcda files from the test project:



At the output we have statistics in percentage of coverage, as well as the file MyCalc.m.gcov:

  -: 0:Source:/Users/dlebedev/src/sandbox/Coverage/iOS/iOSCoverage/../../Common/MyCalc.m -: 0:Graph:MyCalc.gcno -: 0:Data:MyCalc.gcda -: 0:Runs:0 -: 0:Programs:0 -: 1:// -: 2:// MyCalc.m -: 3:// iOSCoverage -: 4:// -: 5:// Created by Denis Lebedev on 23.08.12. -: 6:// Copyright (c) 2012 Denis Lebedev. All rights reserved. -: 7:// -: 8: -: 9:#import "MyCalc.h" -: 10: -: 11:@implementation MyCalc -: 12: 2: 13:- (CGFloat)performOperation:(MyMathOperation)operation withA:(CGFloat)a B:(CGFloat)b { 2: 14: CGFloat result = 0.f; -: 15: 2: 16: switch (operation) { -: 17: case MyMathOperationAdd: 1: 18: result = a + b; 1: 19: break; -: 20: case MyMathOperationSubtract: #####: 21: result = a - b; -: 22: #####: 23: break; -: 24: case MyMathOperationDivide: #####: 25: result = a / b; -: 26: #####: 27: break; -: 28: case MyMathOperationMultiply: 1: 29: result = a * b; 1: 30: break; -: 31: default: #####: 32: NSLog(@"Unsupported operation"); #####: 33: break; -: 34: } 2: 35: return result; -: 36:} 1: 37:- (CGFloat)negate:(CGFloat)number { -: 38: //this method works incorrectly 1: 39: return number; -: 40:} -: 41: -: 42:@end 

#####: - The line failed.
n: - the line is executed n times.

More details can be read here .

CoverStory

CoverStory - GUI add-on over gcov, additionally allows you to generate html-reports using Apple Script.

lcov

lcov is another graphical front end for gcov. It is very convenient in the presence of a large number of files, as it groups html by directories, as well as in automating the process - the utility works from the terminal.

Install lcov:

 # sudo mkdir -p /usr/local/src; cd /usr/local/src # sudo wget http://downloads.sourceforge.net/ltp/lcov-1.6.tar.gz # sudo tar -xzvf lcov-1.6.tar.gz # cd lcov-1.6 

 # sudo vim /usr/local/src/lcov-1.6/bin/install.sh</code> 

On line 34 (install -D $ SOURCE $ TARGET), you must remove the -D flag.

 # sudo make install 

To get the report, execute the following commands in the folder with * .gcda-files:

 lcov -t 'Code coverage report' -o report.info -c -d . genhtml -o html-report report.info 

The result in the folder html-report:

Automation

As mentioned above, lcov is convenient in continuos integration. The demonstrated example will be purely academic and with annoying flaws (and it was not possible to apply the script to the project using CocoaPods).

Script code (it can also be found in the folder with the test iOS project):

 #!/bin/sh TARGET_NAME="iOSCoverage" TEST_TARGET_NAME="iOSCoverageTests" BUILD_CONFIG="Debug" SDK_VERSION="iphonesimulator5.1" rm -rf build rm -rf html-report echo Building and running tests xcodebuild -target $TEST_TARGET_NAME OBJECT_FILE_DIR_normal=/build/ TEST_AFTER_BUILD=YES -sdk iphonesimulator5.1 -configuration $BUILD_CONFIG -xcconfig settings.xcconfig echo Copying files mkdir build/gcda cp build/$TARGET_NAME.build/$BUILD_CONFIG-iphonesimulator/$TARGET_NAME.build/Objects-normal/i386/*.gcda build/gcda/ cp build/$TARGET_NAME.build/$BUILD_CONFIG-iphonesimulator/$TARGET_NAME.build/Objects-normal/i386/*.gcno build/gcda/ echo Generating report cd build/gcda lcov -t 'Code coverage report' -o report.info -c -d . cd .. cd .. genhtml -o html-report build/gcda/report.info 

Content settings.xcconfig (flags can be omitted from the project):

 GCC_INSTRUMENT_PROGRAM_FLOW_ARCS = YES GCC_GENERATE_TEST_COVERAGE_FILES = YES 

We put the script and the settings.xcconfig file next to the project (we first replace the target name variables and the SDK with the necessary ones), run ... and get an error. Since the original iPhone Simulator can not run tests from the command line. How to fix this annoying misunderstanding described here . After that, run the script again and get the html-report folder with statistics.

UPD: For tighter integration with Jenkins, you can use gcovr + Cobertura Plugin . Thank you moborb for the tip.