Testing in Openshift: Automated Testing

This is the final part of a series of three articles ( first article , second article ), which are devoted to automated software testing in Openshift Origin. This article will discuss aspects of testing in containers and features of building CI / CD with the participation of such products as:

1. Openshit Origin - as a system for deploying test environments.
2. Jenkins - as a tool for continuous integration.
3. Testlink - as a test management system.

4. Robot Framework - as a framework for writing tests.

   
   

   For better representativeness, I prepared an image of Vagrant, which contains a pre-configured environment of the above products (all the objects and mechanisms listed in this article can be easily inspected). To increase the degree of understanding of the material, I created two tasks: the task of assembly, the task of testing. Both tasks are divided into stages and described in detail.

   
   

Fast start:

1. Download Vagrant Image and Vagrantfile
2. vagrant box add --name viewshift viewshift-1.0.box && vagrant up

Build task:

The build task implies the build of the Docker image with the "curl" application, which will later be involved in the testing task.

Note: The supervisord is used as the root process (PID 1) in the container. supervisord and other similar tools are very useful in cases when you need to shut down the application completely or manage processes remotely.

Schematic diagram:

Stages :

1. We define the variables that will be involved in the task:
   
   PROJECT - the name of the project Openshift. For this project, ServiceAccount "jenkins" was created, which has administrator rights in the project. This ServiceAccount is used to access the project from Jenkins (this account is also used in the testing task).
   APP_NAME and APP_VERSION - conditional name and version of the application, which, however, appear in several places: the name and tag of the resulting Docker image, the name of the Build being launched, etc.

   
   

2. After the required variables have been defined (the granularity / distinctness of the tasks in the project has been considered), it is required to spread them across all YAML Openshift configurations and other Jenkins steps.

   
   

3. At this stage, a BuildConfig object is created, on the basis of which the Build object will be created and executed.

   
   

4. The build process starts on the basis of the created BuildConfig. If successful, the resulting image will be placed in the internal Docker register.

   
   

5. All created objects are deleted regardless of the success of the assembly.

   
   

Test task:

The test task is the process of testing the curl application, which interacts with the nginx service via the HTTP protocol. We want to make sure that the application works correctly and passes the specified tests.

Schematic diagram:

Stages :

1. Determine the parameters that will be involved in the task:
   
   PROJECT - the name of the project Openshift.
   TESTPLAN is the name of the test plan in Testlink. The task will fail if the specified test plan is not in Testlink.
   APP_NAME and APP_VERSION - conditional name and version of the application, which is the same way as in the assembly task.
   TEST_CMD is a variable that contains the name of the executable file that will be launched inside the container. Command line arguments are specified in the corresponding Jenkins step.
   TEST_TIMEOUT is a numeric expression that specifies the time to wait for the command to execute inside the container. After this time, the Jenkins task completes its execution with an error.

   
   

2. see build task.

   
   

3. At this stage, the Testlink configuration is specified, which indicates: with which server the connection will be established, which test plan will be used (from the test plan all tests assigned to this test plan are loaded for subsequent comparison), under which platform the tests were performed, etc. d. All this is required for the subsequent publication of passed tests back to Testlink and displaying the test report directly to Jenkins.

   
   

4. This step is intended to create a Service. Created services will indicate applications that will be launched later. These services check the availability of applications.

   
   

5. At this stage, a Pod is created for the "nginx" application.

   
   

6. At this stage, a Pod is created for the curl application. The image for this container is the image that is created during the build task process. Unlike "nginx", the data volume "share" has been added to this image, which allows the container to communicate with the file system of the working node.

   
   

7. After all the Pods are created, application availability check is required via the previously published services.

   
   

8. At this stage, the test command is launched on the Pod, followed by waiting for the completion of this command.

   
   

9. After passing all the tests, the test report is copied to the workspace of the task for subsequent import into Testlink.

   
   

10. At this stage, the strategy is indicated (there may be more than one) comparing the passed tests with what was obtained from the specified earlier test plan. In this case, there is a simple comparison of the names of test cases. After all operations, a test report is published in Testlink.

    
    

11. In addition to the Teslink report in Junit format, there is a test report in the Robot Framework format, which will set the status of the completed task based on the threshold values ​​of the passed tests.

    
    

12. At this stage, all Openshift objects created during the task are deleted.

    
    

Together:

Advantages and disadvantages of testing in containers:

Disadvantages:

1. Linux only. The so-called "light" virtualization is developing, and we should probably expect changes in the situation , but so far only Linux.
2. Single kernel version for all containers. Perhaps in paragraph 1
3. Only x86_64. No, of course, your image may be x86, but the kernel will be x86_64. For some, this can be an obstacle.
4. No nested SELinux ( nested CGroups exist ).
5. The lack of a full-fledged video device and access to the screen. Perhaps in paragraph 1

Pros:

1. The speed of work and flexibility of running environments, high density.
2. Unified delivery method, portability and repeatability of applications.

Conclusion:

Openshift Origin in conjunction with other tools allows you to achieve impressive flexibility and efficiency. A well-thought-out naming scheme for projects / objects allows you to avoid errors during mass launches of testing tasks.

Thanks:

I want to thank Google for making such a great platform.

I want to express my gratitude to the staff of the Red Hat company, who made the finished product from a wonderful platform.

Useful links:

1. Minikube is a quick way to get to know Kubernetes
2. Minishift - a quick way to get to know Openshift
3. Why is the embedded register using the IP address instead of the FQDN