Geekly Articles each Day
📜 ⬆️ ⬇️

Testing in Java. Testng


Surely everyone is familiar with such a concept as test-driven development (TDD) . Along with it, there is also such a thing as data-driven testing (DDT, no offense to Shevchuk) - a technique of writing tests in which data for tests are stored separately from the tests themselves. They can be stored in a database file, generated during the execution of the test. This is very convenient, since the same functionality is tested on different data sets, while adding, deleting or modifying this data is as simple as possible.

In a previous article, I looked at the possibilities of JUnit . There, Parameterized and Theories runners can serve as examples of this kind of approach, in both cases one test class can contain only one such parameterized test (in the case of Parameterized, several, but they all will use the same data).

In this article I will focus on the TestNG test framework. Many have already heard this name, and having gone to it, they hardly wish to return to JUnit (although this is only a guess).

Main features


So what is there? As in JUnit 4, tests are described using annotations, also tests written in JUnit 3 are supported. It is possible to use a doclet instead of annotations.
')
To begin, consider the test hierarchy. All tests belong to a sequence of tests (suite), include a number of classes, each of which may consist of several test methods. In this case, classes and test methods may belong to a particular group. It clearly looks like this:
+- suite/ +- test0/ | +- class0/ | | +- method0(integration group)/ | | +- method1(functional group)/ | | +- method2/ | +- class1 | +- method3(optional group)/ +- test1/ +- class3(optional group, integration group)/ +- method4/

Each member of this hierarchy may have before and after configurators. It all starts in this order:
 +- before suite/ +- before group/ +- before test/ +- before class/ +- before method/ +- test/ +- after method/ ... +- after class/ ... +- after test/ ... +- after group/ ... +- after suite/

Now more about the tests themselves. Consider an example. The utility for working with locales, can parse from a string, and also search for candidates ( en_US -> en_US, en, root ):
 public abstract class LocaleUtils { /** * Root locale fix for java 1.5 */ public static final Locale ROOT_LOCALE = new Locale(""); private static final String LOCALE_SEPARATOR = "_"; public static Locale parseLocale(final String value) { if (value != null) { final StringTokenizer tokens = new StringTokenizer(value, LOCALE_SEPARATOR); final String language = tokens.hasMoreTokens() ? tokens.nextToken() : ""; final String country = tokens.hasMoreTokens() ? tokens.nextToken() : ""; String variant = ""; String sep = ""; while (tokens.hasMoreTokens()) { variant += sep + tokens.nextToken(); sep = LOCALE_SEPARATOR; } return new Locale(language, country, variant); } return null; } public static List<Locale> getCandidateLocales(final Locale locale) { final List<Locale> locales = new ArrayList<Locale>(); if (locale != null) { final String language = locale.getLanguage(); final String country = locale.getCountry(); final String variant = locale.getVariant(); if (variant.length() > 0) { locales.add(locale); } if (country.length() > 0) { locales.add((locales.size() == 0) ? locale : new Locale(language, country)); } if (language.length() > 0) { locales.add((locales.size() == 0) ? locale : new Locale(language)); } } locales.add(ROOT_LOCALE); return locales; } }

Let's write a JUnit-style test for it (this example should not be considered as a guide to writing tests for TestNG):
 public class LocaleUtilsOldStyleTest extends Assert { private final Map<String, Locale> parseLocaleData = new HashMap<String, Locale>(); @BeforeClass private void setUp() { parseLocaleData.put(null, null); parseLocaleData.put("", LocaleUtils.ROOT_LOCALE); parseLocaleData.put("en", Locale.ENGLISH); parseLocaleData.put("en_US", Locale.US); parseLocaleData.put("en_GB", Locale.UK); parseLocaleData.put("ru", new Locale("ru")); parseLocaleData.put("ru_RU_xxx", new Locale("ru", "RU", "xxx")); } @AfterTest void tearDown() { parseLocaleData.clear(); } @Test public void testParseLocale() { for (Map.Entry<String, Locale> entry : parseLocaleData.entrySet()) { final Locale actual = LocaleUtils.parseLocale(entry.getKey()); final Locale expected = entry.getValue(); assertEquals(actual, expected); } } }

What is there?

All of these annotations have the following parameters:Apparently from an example the test practically differs in nothing from the same test on JUnit. If there is no difference, then why use TestNG?

Parameterized Tests


We write the same test in another way:
 public class LocaleUtilsTest extends Assert { @DataProvider public Object[][] parseLocaleData() { return new Object[][]{ {null, null}, {"", LocaleUtils.ROOT_LOCALE}, {"en", Locale.ENGLISH}, {"en_US", Locale.US}, {"en_GB", Locale.UK}, {"ru", new Locale("ru")}, {"ru_RU_some_variant", new Locale("ru", "RU", "some_variant")}, }; } @Test(dataProvider = "parseLocaleData") public void testParseLocale(String locale, Locale expected) { final Locale actual = LocaleUtils.parseLocale(locale); assertEquals(actual, expected); } }

Simpler? Of course, data is stored separately from the test itself. Conveniently? Of course, you can add tests by adding just a line to the parseLocaleData method.

So how does it work?
One more example, now we will spread the data and logic of the test into different classes:
 public class LocaleUtilsTestData { @DataProvider(name = "getCandidateLocalesData") public static Object[][] getCandidateLocalesData() { return new Object[][]{ {null, Arrays.asList(LocaleUtils.ROOT_LOCALE)}, {LocaleUtils.ROOT_LOCALE, Arrays.asList(LocaleUtils.ROOT_LOCALE)}, {Locale.ENGLISH, Arrays.asList(Locale.ENGLISH, LocaleUtils.ROOT_LOCALE)}, {Locale.US, Arrays.asList(Locale.US, Locale.ENGLISH, LocaleUtils.ROOT_LOCALE)}, {new Locale("en", "US", "xxx"), Arrays.asList( new Locale("en", "US", "xxx"), Locale.US, Locale.ENGLISH, LocaleUtils.ROOT_LOCALE) }, }; } } public class LocaleUtilsTest extends Assert { // other tests @Test(dataProvider = "getCandidateLocalesData", dataProviderClass = LocaleUtilsTestData.class) public void testGetCandidateLocales(Locale locale, List<Locale> expected) { final List<Locale> actual = LocaleUtils.getCandidateLocales(locale); assertEquals(actual, expected); } }

In this case, the dataProviderClass and dataProvider parameters are specified. The method that returns the test data must be static .

In addition to the above, there is another way to parameterize tests. The required method is annotated using @Parameters , where the names of all the necessary parameters are indicated. Some of the parameters can be annotated with the help of @Optional with the indication of the default value (if not specified, the default values ​​for the primitives will be used, or null for all other types). Parameter values ​​are stored in the TestNG configuration (which will be discussed later). Example:
 public class ParameterizedTest extends Assert { private DataSource dataSource; @Parameters({"driver", "url", "username", "password"}) @BeforeClass public void setUpDataSource(String driver, String url, @Optional("sa") String username, @Optional String password) { // create datasource dataSource = ... } @Test public void testOptionalData() throws SQLException { dataSource.getConnection(); // do some staff } }

In this case, the setUpDataSource method will accept database connection settings as parameters, with the username and password parameters optional, with the default values ​​specified. It is very convenient to use with data common to all tests (or almost all), for example, as in the example of setting up a database connection.

Well, in conclusion, I should say a few words about the factories that allow you to create tests dynamically. As well as the tests themselves, they can be parameterized using @DataProvider or @Parameters :
 public class FactoryTest { @DataProvider public Object[][] tablesData() { return new Object[][] { {"FIRST_TABLE"}, {"SECOND_TABLE"}, {"THIRD_TABLE"}, }; } @Factory(dataProvider = "tablesData") public Object[] createTest(String table) { return new Object[] { new GenericTableTest(table) }; } } public class GenericTableTest extends Assert { private final String table; public GenericTableTest(final String table) { this.table = table; } @Test public void testTable() { System.out.println(table); // do some testing staff here } }

Option with @Parameters :
 public class FactoryTest { @Parameters("table") @Factory public Object[] createParameterizedTest(@Optional("SOME_TABLE") String table) { return new Object[] { new GenericTableTest(table) }; } }

Multithreading


Need to check how an application behaves in a multithreaded environment? You can make tests run from multiple threads simultaneously:
 public class ConcurrencyTest extends Assert { private Map<String, String> data; @BeforeClass void setUp() throws Exception { data = new HashMap<String, String>(); } @AfterClass void tearDown() throws Exception { data = null; } @Test(threadPoolSize = 30, invocationCount = 100, invocationTimeOut = 10000) public void testMapOperations() throws Exception { data.put("1", "111"); data.put("2", "111"); data.put("3", "111"); data.put("4", "111"); data.put("5", "111"); data.put("6", "111"); data.put("7", "111"); for (Map.Entry<String, String> entry : data.entrySet()) { System.out.println(entry); } data.clear(); } @Test(singleThreaded = true, invocationCount = 100, invocationTimeOut = 10000) public void testMapOperationsSafe() throws Exception { data.put("1", "111"); data.put("2", "111"); data.put("3", "111"); data.put("4", "111"); data.put("5", "111"); data.put("6", "111"); data.put("7", "111"); for (Map.Entry<String, String> entry : data.entrySet()) { System.out.println(entry); } data.clear(); } }

The first test will occasionally fail with ConcurrentModificationException , since it will run from different threads, the second - not, since all the tests will be run sequentially from one thread.

You can also set the parallel parameter of the provider’s date to true, then the tests for each data set will be run in parallel, in a separate thread:
 public class ConcurrencyTest extends Assert { // some staff here @DataProvider(parallel = true) public Object[][] concurrencyData() { return new Object[][] { {"1", "2"}, {"3", "4"}, {"5", "6"}, {"7", "8"}, {"9", "10"}, {"11", "12"}, {"13", "14"}, {"15", "16"}, {"17", "18"}, {"19", "20"}, }; } @Test(dataProvider = "concurrencyData") public void testParallelData(String first, String second) { final Thread thread = Thread.currentThread(); System.out.printf("#%d %s: %s : %s", thread.getId(), thread.getName(), first, second); System.out.println(); } }

This test will display something like:
 #16 pool-1-thread-3: 5 : 6 #19 pool-1-thread-6: 11 : 12 #14 pool-1-thread-1: 1 : 2 #22 pool-1-thread-9: 17 : 18 #20 pool-1-thread-7: 13 : 14 #18 pool-1-thread-5: 9 : 10 #15 pool-1-thread-2: 3 : 4 #17 pool-1-thread-4: 7 : 8 #21 pool-1-thread-8: 15 : 16 #23 pool-1-thread-10: 19 : 20

Without this parameter there will be something like:
 #1 main: 1 : 2 #1 main: 3 : 4 #1 main: 5 : 6 #1 main: 7 : 8 #1 main: 9 : 10 #1 main: 11 : 12 #1 main: 13 : 14 #1 main: 15 : 16 #1 main: 17 : 18 #1 main: 19 : 20

Additional features


In addition to all of the above, there are other options, for example, to check for exceptions (it is very convenient to use for tests on incorrect data):
 public class ExceptionTest { @DataProvider public Object[][] wrongData() { return new Object[][] { {"Hello, World!!!"}, {"0x245"}, {"1798237199878129387197238"}, }; } @Test(dataProvider = "wrongData", expectedExceptions = NumberFormatException.class, expectedExceptionsMessageRegExp = "^For input string: \"(.*)\"$") public void testParse(String data) { Integer.parseInt(data); } }

One more example:
 public class PrioritiesTest extends Assert { private boolean firstTestExecuted; @BeforeClass public void setUp() throws Exception { firstTestExecuted = false; } @Test(priority = 0) public void first() { assertFalse(firstTestExecuted); firstTestExecuted = true; } @Test(priority = 1) public void second() { assertTrue(firstTestExecuted); } }

This example will be successful, since the method first will be executed first , then second . If you change the priority of first to 2, the test fails.

Similar behavior will be observed also if you specify dependencies for the test, for example, add to our test:
 public class PrioritiesTest extends Assert { // some staff @Test(dependsOnMethods = {"first"}) public void third() { assertTrue(firstTestExecuted); } // some staff }

This is usually convenient when one test depends on another, for example, Utility1 uses Utility0 , if Utility0 does not work correctly, then there is no point in testing Utility1 . On the other hand, dependencies are also conveniently used in @Before , @After methods, especially to associate the base test with the inheritance, and sometimes it is necessary to make sure that even if method A falls and method B depends on it, method B is still called. In this case, set the parameter alwaysRun to true.

Dependency injection


I want to please fans of the framework from the "corporation of good» Guice . TestNG has built-in support for the latter. It looks like this:
 public class GuiceModule extends AbstractModule { @Override protected void configure() { bind(String.class).annotatedWith(Names.named("guice-string-0")).toInstance("Hello, "); } @Named("guice-string-1") @Inject @Singleton @Provides public String provideGuiceString() { return "World!!!"; } } @Guice(modules = {GuiceModule.class}) public class GuiceTest extends Assert { @Inject @Named("guice-string-0") private String word0; @Inject @Named("guice-string-1") private String word1; @Test public void testService() { final String actual = word0 + word1; assertEquals(actual, "Hello, World!!!"); } }

All that is needed is to enchant the desired class using @Guice and specify all the necessary guice modules in the modules parameter. Later in the test class, you can already use dependency injection using @Inject .

I will also add that fans of other similar frameworks should not be upset, as they usually have their own support for TestNG, for example, Spring.

Expansion of functionality


Expansion of functionality can be implemented using the mechanism of listeners. The following listener types are supported:
One of the examples of interesting use of the mechanism of listeners can be read here .

Configuration


We now turn to the test configuration. The easiest way to run tests looks like this:
  final TestNG testNG = new TestNG(true); testNG.setTestClasses(new Class[] {SuperMegaTest.class}); testNG.setExcludedGroups("optional"); testNG.run();

But most often for start of tests XML or YAML a configuration is used. The XML configuration looks like this:
 <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"> <suite name="Test suite" parallel="classes" thread-count="10"> <test name="Default tests" verbose="1" annotations="JDK" thread-count="10" parallel="classes"> <parameter name="driver" value="com.mysql.jdbc.Driver"/> <parameter name="url" value="jdbc:mysql://localhost:3306/db"/> <groups> <run> <exclude name="integration"/> </run> </groups> <packages> <package name="com.example.*"/> </packages> </test> <test name="Integration tests" annotations="JDK"> <groups> <run> <include name="integration"/> </run> </groups> <packages> <package name="com.example.*"/> </packages> </test> </suite>

Similar YAML configuration:
 name: YAML Test suite parallel: classes threadCount: 10 tests: - name: Default tests parameters: driver: com.mysql.jdbc.Driver url: jdbc:mysql://localhost:3306/db excludedGroups: [ integration ] packages: - com.example.* - name: Integration tests parameters: driver: com.mysql.jdbc.Driver url: jdbc:mysql://localhost:3306/db includedGroups: [ integration ] packages: - com.example.*

Then to run the tests you will need to do the following:
  final TestNG testNG = new TestNG(true); //final Parser parser = new Parser("testing/testing-testng/src/test/resources/testng.xml"); final Parser parser = new Parser("testing/testing-testng/src/test/resources/testng.yaml"); final List<XmlSuite> suites = parser.parseToList(); testNG.setXmlSuites(suites); testNG.run();

Although, probably, the program launch of tests is unnecessary, since for this you can use the IDE features.

Let's go back to the configuration itself. At the very top level, the test sequence (suite) is configured. It can take the following parameters:
Also can be customized:
The suites, in turn, can include tests with practically the same settings as for the suites ( name, thread-count, parallel, time-out, junit, annotations , parameter, packages, method-selectors tags). The tests also have some peculiar settings, for example, the groups to be run:
  <test name="Default tests" verbose="1" annotations="JDK" thread-count="10" parallel="classes"> <!-- some staff here --> <groups> <run> <exclude name="integration"/> </run> </groups> <!-- some staff here --> </test>

In this example, the test will include only tests not related to the integration group.

Other tests may include test classes, which in turn may include / exclude test methods.
  <test name="Integration tests"> <groups> <run> <include name="integration"/> </run> </groups> <classes> <class name="com.example.PrioritiesTest"> <methods> <exclude name="third"/> </methods> </class> </classes> </test>


Conclusion


This is not all that can be said about this wonderful framework; it is very difficult to cover everything in one article. But I tried to reveal the main points of its use. Here, of course, it was possible to tell a little more about the mechanism of the listeners, about integration with other frameworks, about the configuration of tests, but then the article would turn into a book, and I don’t know everything. But I hope this article was useful to someone and what was said will encourage some readers to use TestNG, or at least on the DDT technique of writing tests.

Examples can be found here , various articles here .

Literature


Source: https://habr.com/ru/post/121234/

More articles:


All Articles