PHPUnit && ordered tests

All programmers are lazy. And everyone wants not to write additional code, but to use it already. Moreover, it is a good practice.

So I had a problem in which I wanted not to do copy-paste, but run a few tests for execution. But, each next test depended on the previous data, and so on, and so on ... As a result, I needed a strict sequence of tests and the ability to respond to dependencies. What is the solution, look under the cut ...

Precondition

There is a facade of the system that can perform any actions. There are dependent actions, and there are independent actions. Dependent actions require an independent action (and sometimes not one). Therefore, in the end, we should get a so-called ordered list of actions.
')
Since all the actions that are invoked, you need to test, then accordingly you need to get an ordered list of tests. PHPUnit was used as a test framework, as its conditions were quite enough for a certain type of action.

Condition

So, now I can see that they will want to throw tomatoes at me and say that there is dependency in PHPUnit. And indeed it is. BUT, as written by the author himself, dependencies do not allow you to specify a strict order of test execution.

PHPUnit supports the declaration of explicit dependencies between test methods. It is not necessary to determine the order.

In addition, as I found out later, to indicate the dependence of a single test method in a descendant class on a test method of the parent class, it is like immediately indicating that the test should be ignored.
For example:

Class ParentTestCase extends PHPUnit_Framework_TestCase { public function testOne() { self::assertTrue(true); } /** * @depends testOne */ public function testTwo() { self::assertTrue(true); } } 

 class ChildTestCase extends ParentTestCase { /** * @depends testTwo */ public function testThree() { self::assertTrue(true); } } 

I’ll point out that this is due to the use of reflection , because the methods from the descendant class are collected first, and then from the ancestor class.

In this regard, I had the idea to modify PHPUnit so that it supports the sequence of test execution and that the basis for this sequence lies in indicating dependencies on test execution.

Decision

First, let's determine where we are going to store the order of dependencies of the test. To do this, create a child TestCase and add control functions for this order.

 class MagicTestCase extends PHPUnit_Framework_TestCase { /** * @var array */ protected $order = array(); /** * Sets the orderSet of a TestCase. * * @param array $orderSet */ public function setOrderSet(array $orderSet) { $this->order = $orderSet; } /** * Get the orderSet of a TestCase. * * @return array $order */ public function getOrderSet() { return $this->order; } } 

After that, you must specify the initial dependence for each test. To do this, override the addTestMethod method for the PHPUnit_Framework_TestSuite class in the inherited class. Install the dependency:

 $test->setOrderSet(PHPUnit_Util_Test::getDependencies($class->getName(), $name)); 

Now we need to add the constructor of our TestSuite, so that it sorts all the tests in the order we set. In the same place for each test we define a recursive order.

  foreach($this->tests as $test) { $test->setOrderSet( array_unique($this->getRecursiveOrderSet($test, $test->getName())) ); } usort($this->tests, array("MagicUtilTest ", "compareTestOrder")); 

In addition, the addTestSuite function creates an instance of itself ( PHPUnit_Framework_TestSuite ), and it is necessary that it create a modified TestSuite , because in the first case, it does not use the modified TestSuite constructor . It all depends on one line, which we redefine inside the method:

 $this->addTest(new MagicTestSuite($testClass)); 

The resulting class is:

 class MagicTestSuite extends PHPUnit_Framework_TestSuite { /** * Constructs a new TestSuite: * * @param mixed $theClass * @param string $name * @throws InvalidArgumentException */ public function __construct($theClass = '', $name = '') { parent::__construct($theClass, $name); foreach($this->tests as $test) { $test->setOrderSet( array_unique($this->getRecursiveOrderSet($test, $test->getName())) ); } usort($this->tests, array("MagicUtilTest ", "compareTestOrder")); } /** * @param $object * @param $methodName * @return array */ protected function getRecursiveOrderSet($object, $methodName) { $orderSet = array(); foreach($this->tests as $test) { if ($test->getName() == $methodName && get_class($object) == get_class($test)) { $testOrderSet = $test->getOrderSet(); if (!empty($testOrderSet)) { foreach($testOrderSet as $orderMethodName) { if(!in_array($orderMethodName, $orderSet)) { $orderResult = $this->getRecursiveOrderSet($test, $orderMethodName); $orderSet = array_merge($orderSet, $orderResult); } } } $orderSet = array_merge($orderSet, $testOrderSet); } } return $orderSet; } /** * @param ReflectionClass $class * @param ReflectionMethod $method */ protected function addTestMethod(ReflectionClass $class, ReflectionMethod $method) { $name = $method->getName(); if ($this->isPublicTestMethod($method)) { $test = self::createTest($class, $name); if ($test instanceof PHPUnit_Framework_TestCase || $test instanceof PHPUnit_Framework_TestSuite_DataProvider) { $test->setDependencies( PHPUnit_Util_Test::getDependencies($class->getName(), $name) ); } $test->setOrderSet(PHPUnit_Util_Test::getDependencies($class->getName(), $name)); $this->addTest($test, PHPUnit_Util_Test::getGroups( $class->getName(), $name) ); } else if ($this->isTestMethod($method)) { $this->addTest( self::warning( sprintf( 'Test method "%s" is not public.', $name ) ) ); } /** * Adds the tests from the given class to the suite. * * @param mixed $testClass * @throws InvalidArgumentException */ public function addTestSuite($testClass) { if (is_string($testClass) && class_exists($testClass)) { $testClass = new ReflectionClass($testClass); } if (!is_object($testClass)) { throw PHPUnit_Util_InvalidArgumentHelper::factory( 1, 'class name or object' ); } if ($testClass instanceof PHPUnit_Framework_TestSuite) { $this->addTest($testClass); } else if ($testClass instanceof ReflectionClass) { $suiteMethod = FALSE; if (!$testClass->isAbstract()) { if ($testClass->hasMethod(PHPUnit_Runner_BaseTestRunner::SUITE_METHODNAME)) { $method = $testClass->getMethod( PHPUnit_Runner_BaseTestRunner::SUITE_METHODNAME ); if ($method->isStatic()) { $this->addTest( $method->invoke(NULL, $testClass->getName()) ); $suiteMethod = TRUE; } } } if (!$suiteMethod && !$testClass->isAbstract()) { $this->addTest(new MagicTestSuite($testClass)); } } else { throw new InvalidArgumentException; } } } 

Well, respectively, add a utility function that will compare the two tests and indicate in which direction to sort them.

 class MagicUtilTest { /** * @static * @param PHPUnit_Framework_TestCase $object1 * @param PHPUnit_Framework_TestCase $object2 * @return int */ public static function compareTestOrder(PHPUnit_Framework_TestCase $object1, PHPUnit_Framework_TestCase $object2) { if (in_array($object2->getName(), $object1->getOrderSet())) { return 1; } if (in_array($object1->getName(), $object2->getOrderSet())) { return -1; } return 0; } } 

Application

Create a test run file:

 require dirname(__FILE__) . DIRECTORY_SEPARATOR.' runSuite.php'; PHPUnit_Util_Filter::addFileToFilter(__FILE__, 'PHPUNIT'); require_once 'PHPUnit/TextUI/Command.php'; $tests= runSuite::suite(); PHPUnit_TextUI_TestRunner::run($tests); exit; 

Create a common class TestSuite, which will directly run the tests. We inherit it from our created:

 require_once 'MagicTestSuite.php'; class runSuite extends MagicTestSuite { public static function suite() { $suite = new self(); $suite->addTestSuite(“ChildTestCase”); } } 

Well, TestCases, which include tests and dependencies:

 require_once 'MagicTestCase.php'; class ParentTestCase extends MagicTestCase { public function testOne() { self::assertTrue(true); } /** * @depends testOne */ public function testTwo() { self::assertTrue(true); } } 

 require_once 'ParentTestCase.php'; class ChildTestCase extends ParentTestCase { /** * @depends testTwo */ public function testThree() { self::assertTrue(true); } } 

Restrictions

The current solution does not allow to break the closing dependencies. That is, if we have two tests that will depend on each other, then we get an infinite loop.

Of course, it was possible to build a dependency tree of the test ... but I honestly did not think up in which case (besides human error) you can use closure dependencies for tests.