On Habré there is absolutely no information about TestContainers . At the time of this writing, search results have announcements of our same conferences , that's all. Meanwhile, in the project on GitHub , they already have more than 700 commits, 54 contributors and 5 years of history . It seems that all these five years the project has been carefully hidden by the special services and UFOs. It is time to get out of the shadows into the light.

Chukchi - a reader, not a writer. Therefore, instead of writing my own text, I asked permission to translate the relevant article from the RebelLabs blog .

So, here we will share a few words about the latest Java library for integration testing - TestContainers . In addition, there will be a little about why integration testing is so important for ZeroTurnaround and their requirements for integration tests. And of course, there will be a full-featured integration test sample for the Java agent. If someone has never seen the Java agent code, now is the time. Welcome under the cut!

Integration Testing at ZeroTurnaround

ZeroTurnaround's products integrate with much of the Java ecosystem. In particular, JRebel and XRebel are based on the technology of Java agents and integrate with Java applications, frameworks, application servers, and so on.

Using the Java agent, you can instrument the Java code to add the extra functionality you need. To test how an application behaves after applying a patch, you need to run it through a pre-configured Java agent. Once the application has started and started working, you can send an HTTP request to it to reproduce the desired behavior.

To run such tests on a large scale, you need to have an automated system that can start and stop the runtime environment, including the application server or any other external dependencies on which the application depends. It should be possible to run about the same things in the continuous integration environment and on the developer's computer.

The result is a bunch of tests, they do not work very quickly, and therefore we definitely want to run them in parallel. This automatically means that tests should be isolated so that there are no conflicts over resources. For example, if we run multiple instances of Tomcat on the same host, we would like to avoid port usage conflicts.

In such an integration test helps us a small beautiful library TestContainers . She didn’t just come up with the requirements voiced above - after its implementation we received an impressive performance increase.

Testcontainers

The official TestContainers documentation says the following:

“TestContainers is a Java library that supports JUnit tests and provides light, temporary instances of core databases, web browsers for Selenium, or anything else that can be run in a Docker container.”

TestContainers provides an API to automate environment configuration. It launches the necessary Docker-containers exactly for the duration of our tests and extinguishes them as soon as the tests are completed. Next we look at several demos based on official examples in their GitHub repository .

GenericContainer

When using TestContainers, the GenericContainer class is often used:

 public class RedisBackedCacheTest { @Rule public GenericContainer redis = new GenericContainer("redis:3.0.6") .withExposedPorts(6379); 

Its constructor takes as its parameter a string in which the Docker image is specified, which we will use in the future. During startup, TestContainers automatically loads the corresponding image (if it has not been downloaded before).

Important note: in the withExposedPorts(6379) method, 6379 is the port on which the container will hang. Next, we will be able to find the corresponding associated port by calling the getMappedPort(6379) method getMappedPort(6379) on the container instance. Combining this with getContainerIpAddress() , we can get the full URL of the service running in the container:

 String redisUrl = redis.getContainerIpAddres() + “:” + redis.getMappedPort(6379); 

You can see that the field from this example is marked with the @Rule annotation. The @Rule annotation from JUnit determines that we will receive a new GenericContainer instance in each test method of this class. If we want to reuse the container instance, for this there is the @ClassRule annotation.

Task containers

The GenericContainer are specialized containers. To test the level of access to data, out of the box are containerized MySQL, PostgreSQL and Oracle images.

 PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer("postgres:9.6.2") .withUsername(POSTGRES_USERNAME) .withPassword(POSTGRES_PASSWORD); 

With just this one line you can get a copy of the container, which will remain with us throughout the test. On the machine where the tests will run, you do not need to manually install the database. This gives an especially big gain if you want to test on several versions of the same database.

Own containers

Inheriting from GenericContainer , it is possible to make new container types. This is quite convenient if you want to encapsulate the corresponding services and logic. For example, you can use MockServer to lock the dependencies of a distributed system in which applications communicate with each other via HTTP:

 public class MockServerContainer extends BaseContainer<MockServerContainer> { MockServerClient client; public MockServerContainer() { super("jamesdbloom/mockserver:latest"); withCommand("/opt/mockserver/run_mockserver.sh -logLevel INFO -serverPort 80"); addExposedPorts(80); } @Override protected void containerIsStarted(InspectContainerResponse containerInfo) { client = new MockServerClient(getContainerIpAddress(), getMappedPort(80)); } } 

In this example, immediately after the container is initialized, the containerIsStarted(...) callback is used, which initializes the MockServerClient instance. Thus, we hid all the implementation details specific to the container inside our own container type. Thanks to this, we got a cleaner client code and a tidier API for testing.

Next we will see that manually defined containers help in structuring the environment for testing Java agents.

Testing the Java agent with TestContainers

To demonstrate the idea, let us use an example kindly provided by Sergey @bsideup Egorov , a co-driver of the TestContainers project.

Demo application

Let's start with a test application. We need a web application that responds to HTTP GET requests. Fat frameworks are not required - so why not take SparkJava ? To add fun, let's start code on Groovy right away! Here we will test this application:

 //app.groovy @Grab("com.sparkjava:spark-core:2.1") import static spark.Spark.* get("/hello/") { req, res -> "Hello!" } 

This is a simple Groovy script that uses Grape to load dependencies on SparkJava, and defines one HTTP endpoint that responds with a “Hello!” Message.

Java agent

The agent we are going to check patches the Jetty server and adds an additional header to it in the HTTP response.

 public class Agent { public static void premain(String args, Instrumentation instrumentation) { instrumentation.addTransformer( (loader, className, clazz, domain, buffer) -> { if ("spark/webserver/JettyHandler".equals(className)) { try { ClassPool cp = new ClassPool(); cp.appendClassPath(new LoaderClassPath(loader)); CtClass ct = cp.makeClass(new ByteArrayInputStream(buffer)); CtMethod ctMethod = ct.getDeclaredMethod("doHandle"); ctMethod.insertBefore("{ $4.setHeader(\"X-My-Super-Header\", \"42\"); }"); return ct.toBytecode(); } catch (Throwable e) { e.printStackTrace(); } } return buffer; }); } } 

In this example, Javassist is used to JettyHandler.doHandle method, to which an additional command is added that sets the X-My-Super-Header .

Of course, to become a Java agent, you need to properly assemble into a package and add the appropriate attributes to the MANIFEST.MF file. All this is done for us by the build script, so as not to clutter up the article, it is posted on GitHub, see the contents of the build.grade file.

Actually, the test!

The test will be quite simple: you need to make a request to our application and check the answer for the presence of a special header, which the Java agent, in theory, should add there. If the title is found and the value of the title matches the expected value - the test was successfully passed. Take a look at the code:

 @Test public void testIt() throws Exception { // Using Feign client to execute the request Response response = app.getClient().getHello(); assertThat(response.headers().get("X-My-Super-Header")) .isNotNull() .hasSize(1) .containsExactly("42"); } 

You can run it directly from the IDE, or from the command line, or even in a continuous integration environment. TestContainers help us launch an application so that the agent is in an isolated environment, in a Docker container.

To run the application, you need a Docker image with Groovy support. To make ourselves comfortable, we have a zeroturnaround / groovy Docker image, it lies on the Docker Hub. Here's how to use it, inheriting from the GenericContainer :

 public class GroovyTestApp<SELF extends GroovyTestApp<SELF>> extends GenericContainer<SELF> { public GroovyTestApp(String script) { super("zeroturnaround/groovy:2.4.5"); withClasspathResourceMapping("agent.jar", "/agent.jar", BindMode.READ_ONLY); withClasspathResourceMapping(script, "/app/app.groovy", BindMode.READ_ONLY); withEnv("JAVA_OPTS", "-javaagent:/agent.jar"); withCommand("/opt/groovy/bin/groovy /app/app.groovy"); withLogConsumer(new Slf4jLogConsumer(LoggerFactory.getLogger(script))); } public String getURL() { return "http://" + getContainerIpAddress() + ":" + getMappedPort(getExposedPorts().get(0)); } } 

See how the API provides us with methods for obtaining the IP address of the container, as well as the associated port (which is actually randomized). In a sense, the port will be different each time the test is run. Therefore, if you run all the tests at the same time, there will be no conflicts between the ports, and the tests will not sprinkle.

Now we have a special class GroovyTestApp for simple running scripts on Groovy, in our case - for testing a demo application:

 GroovyTestApp app = new GroovyTestApp(“app.groovy”) .withExposedPorts(4567); //the default port for SparkJava .setWaitStrategy(new HttpWaitStrategy().forPath("/hello/")); 

Run the tests, look at the exhaust:

 $ ./gradlew test 16:42:51.462 [I] d.DockerClientProviderStrategy - Accessing unix domain socket via TCP proxy (/var/run/docker.sock via localhost:50652) … … … 16:43:01.497 [I] app.groovy - STDERR: [Thread-1] INFO spark.webserver.SparkServer - == Spark has ignited ... 16:43:01.498 [I] app.groovy - STDERR: [Thread-1] INFO spark.webserver.SparkServer - >> Listening on 0.0.0.0:4567 16:43:01.511 [I] app.groovy - STDERR: [Thread-1] INFO org.eclipse.jetty.server.Server - jetty-9.0.2.v20130417 16:43:01.825 [I] app.groovy - STDERR: [Thread-1] INFO org.eclipse.jetty.server.ServerConnector - Started ServerConnector@72f63426{HTTP/1.1}{0.0.0.0:4567} 16:43:02.199 [I] ?.4.5] - Container zeroturnaround/groovy:2.4.5 started AgentTest > testIt STANDARD_OUT Got response: HTTP/1.1 200 OK content-length: 6 content-type: text/html; charset=UTF-8 server: Jetty(9.0.2.v20130417) x-my-super-header: 42 Hello! BUILD SUCCESSFUL Total time: 36.014 secs 

This test is not very fast. It takes some time to download Grapes - but only the very first time. However, this is a complete integration test that runs a Docker container, an application using the HTTP stack, and makes HTTP requests. In addition, the application runs in isolation, and it’s really easy to do. And all this thanks to TestContainers!

Conclusion

"Works on my computer" is a popular excuse, but it should no longer be an excuse at all. As containerization technology becomes available to more and more developers, it becomes possible to do more and more deterministic tests.

TestContainers reduce the amount of madness in Java application integration tests. This library is very easy to integrate into existing tests. You no longer need to manually manage external dependencies, and this is a huge victory, especially in a continuous integration environment.

If you like what you have just read, we highly recommend looking at the recording from the GeekOut Java conference, where Richard North, the original author of the project, provides background information on TestContainers , including development plans. Or at least look at the slides of this presentation .

A couple of words from the translator.

Firstly, if you find any inaccuracies, errors and typos - you need to go to the olegchir in person and describe everything as it is. I really read messages and fix bugs.

If you are interested in Java , new technologies and libraries, then you should visit our Java conferences. Nearest - JPoint and JBreak . By the way, ZeroTurnaround employees often speak at our conferences as speakers and work as members of the Program Committee.

If testing is more interesting for you , then we are holding a conference Heisenbug 2017 Moscow , which will be held in just a week and a half. The topic of testing using Docker there is somehow present in many reports .

Will you use TestContainers? Do you like the idea, have doubts? Write in the comments!