Implementation of monitoring and integration testing information system using Scalatest. Part 1

In this article I want to share the experience of creating tests using the Scalatest framework for test automation. The article will consist of 2 parts. The first is a step-by-step instruction for creating and running a basic test, the second is a review of more complex cases and nuances of the test stack, information on creating test reports, and solving problems that arise.

There are many solutions to automate testing. Each of them has its own characteristics, advantages, disadvantages, differ in the threshold of entry, ease of use, efficiency, versatility, range of tasks for which is well suited. For the task of automating integration testing and monitoring of systems for one of the projects, the successful solution was the use of the Scala + ScalaTest + SBT bundle.

Task:
')
Automate integration testing;
To organize monitoring - automated testing of the performance / availability of the information system as a whole and its individual parts.
Provide quick localization of the problem component in case of system failure;
Provide a report on the test in an understandable form.

The general idea is as follows: Some action is performed that initiates the start of processing in the information system (sending a message to the service, queue manager, performing a certain sequence of actions on the web interface). The system processes the message, driving it through its components, subsystems. We check that the subsystem has performed its task on certain grounds: the presence of a record in the log, the presence of a new record in the database, the correct answers to requests, the receipt of response messages from the system, the display on the site. Each such test turns into a test case step, which can either succeed or fail.

After passing the test, a report is generated, to which some files are attached (System responses, screenshots, query results), from which you can determine whether everything is good in the system and where the problem is. Examples of the report are presented below:









Brief description of the technologies used:

Scala is a programming language designed to be concise and type safe for simple and fast creation of component software, combining the functionality of functional and object-oriented programming. Works on top of jvm.
Sbt - (scala build tool) - an automated build system for projects written in Scala and Java.
Scalatest is an application testing framework. On the main page, we read “simply productively” and “we’re able to increase both the code and communications”.
Selenium WebDriver is a tool for testing web applications, a set of tools for controlling the browser, and emulating user actions.

This stack pleased us with its versatility (development and launch on win / linux / macos are not particularly different), the wealth of possibilities (if you can’t realize something with scala, you can use slightly modified java code and java libraries), efficiency, support from the IDE (IntelliJ IDEA, Eclipse), good documentation. Particularly pleased that this “Just works” (if you omit problems with the encoding in Windows), that is, the number of problems solved by a tambourine is not so great, and to start working with the stack it is not necessary to have extensive experience in programming and tuning information systems. Writing on Scala is quite nice, especially if you use a good IDE.

Oracle JDK must be installed on the machine, preferably version 8 (1.8), on previous versions some components may not start, Idea will not work on openJDK. www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html ,

For the development of the test will use IntelliJ IDEA Community Edition from jetbrains.

To do this, put the idea, run, set the plugin scala.







Restart the application.



Create a new project, Scala -> SBT.



Enter the name, directory, select the path to the JDK, put the checkbox “Use auto-import” so that the changes in the build.sbt file are immediately updated.



After that, after a while, the project structure will be created. The first time it may take a few minutes, because Idea has to create, index a project, get the right version of components from the repository. The system can throw out several Warning messages, ignore them.



The file “build.sbt”, which was created in the project directory, contains information about the project, as well as a list of components used.
To connect the framework “scalatest”, add the corresponding line to “build.sbt”:

libraryDependencies += "org.scalatest" % "scalatest_2.11" % "3.0.0-M7" 

This is the latest version at the time of this writing.
(SBT uses the MAVEN repository, so you can find the library on the site, for example, mvnrepository.com/artifact/org.scalatest/scalatest_2.11/3.0.0-M7 ).

(In SBT, it is possible to load the latest version automatically by specifying “latest.integration” in the field with the version. But, this can be a problem - changing the behavior when updating. It is better to fix the version).

Please note that the lines of this file are separated by empty lines (So it is, although without it will work without separating the lines).
After adding, the idea will select the appropriate library from the repository and load it. This operation may also take considerable time. Sometimes there are some problems with the update, you can close the project and reopen it.



The indicator at the bottom of the form indicates the project is ready. Let's create a simple test, see how skaltest works. Test classes are created in the / src / test / scala directory from the context menu.



Let's call it "DummyTest". Class text:

 /** * Created by user on 26.08.2015. */ import org.scalatest.{Matchers, FreeSpec} class DummyTest extends FreeSpec with Matchers{ "    " in { val num = 2+3 num should be (5) } "    " in { val num = 2+3 num should be (4) } } 

We describe the test. From the styles presented here, I like FreeSpec. We will use it (we add "extends FreeSpec" after the class name). "With Matchers" is needed to use comparisons, for example, x should be (y) checks that the value of x matches the value of y. A more detailed comparison of the values here and here .

The test contains 2 steps. The first is obviously correct, the second must fall. The step has the name “Two plus three equals five”, which is displayed in the reports. This is a string that is enclosed in quotes, it is allowed to use Cyrillic. The word in separates the name of the step and the body, enclosed in brackets "{", "}". The test is considered passed if all the steps in the body were successful. The test is considered to be inundated, if something went wrong and the test fell on the step.

“Val num = 2 + 3” - we declare a variable and assign a value to it.

In general, an ad in scala looks like this:

val or val VariableName [: DataType] = Initial Value

val is a value that does not change (looks like a constant), var is a variable — changes during the execution of the code. Using val is preferred. "Num should be (5)" - compares the values, checks that the value of the variable "num" is equal to five.

If an error is displayed in the import line, then probably the necessary libraries have not yet loaded, and the idea cannot import the necessary components.

Run the script in Idea. We call the context menu inside the class - the “Run” item appears in it (you can not run any class, the “extends FreeSpec” option allows us to use trait FreeSpec). After this, the class will be added to the list in the upper right part of the screen, where it can be launched using the button, or by using the Shift + f10 key combination



If we work in windows - an error will appear - a problem with the encoding. 1251 - default Windows encoding does not work with Cyrillic. Therefore, we change to UTF-8 in the lower right part of the window. In the dialog box, click "Convert". In linux / mac, there should be no problems, utf-8 is the default encoding for new files in Idea.



Run the test again, see the result at the bottom of the form.



We see some kind of tree with checks on the left, a description of the error in the middle and the result on the right. Expected second step dropped.

Running a test from the IDE allows you to quickly debug tests in a convenient form. But when the test runs on a continuous integration server, it will run from the console. Let's see how to do it in your car.

To begin with, we put SBT, version not lower than 0.13.8 . It is not recommended to install sbt from the linux repository, because it is either absent there or the version may be outdated.

After installing sbt, check the version by executing the command "sbt --version" .

Since the “MaxPermSize” parameters are specified in the sbt startup parameters, and it is not needed in the eighth java, a corresponding warning will appear. We can ignore it or change the SBT settings in the configuration files. Version “0.13.8” is fine for us. Go to the SBT project directory. In it, we execute the command for running tests - “sbt test”. The test will pass, the last step will be green, zaflenny - red.



For the standard windows console, there is again an encoding problem. To display beautiful Russian text, you need to change the page encoding and fonts. In the properties of the command line window set "Lucida Console".



In the window we write the command "chcp 1251". After that, re-run the "sbt test" again - everything should be displayed beautifully.



Also, the correct display of Russian characters can be achieved by changing the SBT configuration files. Basic dough preparation is ready.

Consider an example to test the performance of the site. The test will check that the page is returned and the contents of the “title” tag are correct.
Check in 2 ways - Get-request and selenium script. To implement the test using the web-driver, you need “Firefox” (in principle, it is possible to use other browsers, including “HtmlUnit”, which does not need a graphical interface, but there are usually fewer problems with firewall, and the delivery of many Linux distributions it is present by default).


To execute the Selenium script, add the line to “build.sbt”

 libraryDependencies += "org.seleniumhq.selenium" % "selenium-java" % "2.46.0" 

The idea will begin to pull components from the repository for the work of Selenium.

It is important to note that after upgrading Firefox, seleniumDriver, which comes in the “selenium-java” library, may not work. For example, "2.46.0" does not work with the 42 version of Firefox. You need to update the version to the current one, or specify the version “latest.integration” in the field so that the version is checked and, if necessary, updated.

  libraryDependencies += "org.seleniumhq.selenium" % "selenium-java" % "latest.integration" 

Create a class, call it, for example, “GetTest”. We import the necessary libraries and, at the beginning of the class, define the “get” method to get the text of the page and create an instance of “FirefoxDriver” (implicit value c driver for using Selenium DSL, which is in ScalaTest).

 import org.openqa.selenium.WebDriver import org.openqa.selenium.firefox.FirefoxDriver import org.scalatest.selenium.WebBrowser import org.scalatest.{Matchers, FreeSpec} import scala.io.Source class GetTest extends FreeSpec with Matchers with WebBrowser{ val pageURL = "http://scalatest.org/about" def get(url: String) = Source.fromURL(url, "UTF-8").mkString implicit val webDriver: WebDriver = new FirefoxDriver() "Get   %s   ".format(pageURL) in { get(pageURL) should include("<title>ScalaTest</title>") } "  " + pageURL+ "   " in { go to pageURL pageTitle should be ("ScalaTest") quit() } } 

The get method gets the page code from the web server and converts it to a string, which then checks for the presence of the substring "ScalaTest".

The second test opens a page in firelight and compares its title with the template. Since the step name is a string, you can add variables to it by gluing the string together, or using the format command. Right click in the class - run. The result is observed in the lower part of the form.



So you can quickly create a test that checks that the page loads and has the correct title. Such a test can already be used to monitor the site, launching it using a continuous integration server on a schedule, sending notifications to interested parties in the event of a fall.

Examples of actions that can be performed in tests:

• GET / POST request. Check the contents of the response;
• Database request. Check the contents of the response;
• Generate messages and send them to the queue;
• Receive messages from the queue. Check the contents of the response;
• Checking the web interface (Correctness of work, display of values ​​on the form, getting screenshots);
• Run command line scripts (for example, monitoring remaining memory, hard disk space).

To cover the system with tests, it is necessary to wrap the check of each component (and in some cases, check each step of the message processing by the system) in a separate step or in a separate test class, in order to accurately localize the problem, to build informative reports.

Issues to be addressed in the following article:

• Report generation;
• Connection of the configuration file;
• Parallel test run;
• Requests to the database;
• Work with the queue manager;
• More complex scenarios using Selenium WebDriver;
• Execution of system commands;
• Work with time (pause, wait, timeout);
• Some more features of the stack.

The second part of the article came out:
Implementation of monitoring and integration testing information system using Scalatest. Part 2

I hope this article will be useful to those who choose a testing or monitoring tool.

Useful links:
ScalaTest official page
ScalaTest Documentation
Useful books:
Scala Cookbook
Testing in scala