About ScalaCheck

Part 1. Introduction .

ScalaCheck is a combinatorial library that greatly facilitates the writing of unit tests on Scala. It uses a property-based testing approach, first implemented in Haskell's QuickCheck library. There are many implementations of QuickCheck: as for Java , C , as well as other languages ​​and platforms. Using this approach can significantly reduce the time to develop tests.

This series of articles is in many ways similar to my previous Parboiled, so the structure of the story will be similar. I will tell you what all this is for, then we will learn to look at the world through the prism of properties and generators, and then we move on to more complex things. Interested? I ask under the cat.

Cycle structure

• Introduction
• Generators
• Properties
• Minimization and state testing
• Integration and Settings

Introduction

Unit testing is one of the most important approaches in software development. Even if your program passes tests at the level of the type system when compiling, this does not mean that there are no logical errors in it. This means that no matter how powerful your programming language may be, testing a code is indispensable. However, the cost of testing is very high: in addition to the spent man-hours, it is required to spend non-human efforts on the routine writing of unit tests. Because of this, many customers save on testing, which many programmers use with great joy: it is boring to write unit tests (but you need to!).

Agree to cover a huge number of cases and write syntactic constructions of the same type day after day is a dubious pleasure. Unfortunately, even type-safe code in functional languages ​​needs to be tested. And although the use of the functional paradigm itself makes modular testing much easier - the absence of side effects allows you to consider fewer marginal cases and completely forget about state testing, this does not significantly reduce the amount of testing code. In addition, although manual testing will give you some confidence in your code, it cannot guarantee that, over time, there will be no particular case that you have missed.

The ScalaCheck approach allows moving to the next level of abstraction and avoiding a significant part of this routine, at the same time increasing the readability of the testing code and reducing its volume (which will positively affect its maintainability). Coverage of your code with tests will also increase significantly. And to achieve this, you only need to get acquainted with several new concepts.

Property-based testing

What are properties?

First of all, let's understand what is a property. If in a nutshell, a property is a certain logical statement relating the input values ​​of the function under test and the results obtained on them. In this case, the very word “property” here should be understood not in the everyday programmer sense (data belonging to some object), but in the mathematical sense - as some law or rule, valid entirely for a certain set of objects. Recall, for example, the properties of associativity or distributivity in the algebra of real numbers.

When testing properties, we do not care about the implementation details of the function being tested,
and only its input and output specifications. Let us now take, for example, the following simple property, written in the Leibniz language:

∀x ∈ ℝ: x ≠ 0 ⟹ x² > 0 

If you do not understand Leibniz, do not despair: then everything will be on Scala.

ScalaCheck allows you to write it in almost the original mathematical form:

 forall { x: Double => (x != 0) ==> x * x > 0 } 

In Russian, it reads something like this: “For any real number x not equal to zero, x² is always greater than zero.”

The author is well aware that the type Double is not a bit, but for
to simplify the presentation of the absolute correctness of the wording will have to sacrifice.

As you can see, the property is a higher level of abstraction than traditional assertion tests in JUnit. However, in the end, it all comes down to them: based on the abstract description of the properties, ScalaCheck generates very specific tests of individual values ​​that are comparable in quality to those that would be written by hand.

Properties are not theories.

In JUnit4, an interesting mechanism appeared under the general name theories . (Oh, they would call them hypotheses ...) Theories work in a very similar way to ScalaCheck, they just don't know how to generate random input data and perform minimization ( shrinking ).

Alas, I did not think of a better translation for the completely understandable term “shrinking”
in Russian, than "minimization". It sounds not entirely correct, but it is better,
than my other attempts.

So what is the theory in the JUnit view? First of all, this is a special type of unit test. The theory checks whether a certain condition is true for each element of a given set of test data (they are called data points ). This allows you to program the validation logic once, and then quickly run it on various data sets.

In order for your method to become a theory, it needs to be annotated accordingly: add @Theory . Input data is annotated as @DataPoint . This is enough for the runner to run the test several times: once for each data point. Here is a small example, blatantly borrowed from the JUnit documentation:

 @RunWith(Theories.class) public class UserTest { //    — . @DataPoint public static String GOOD_USERNAME = "optimus"; //    — ,     . @DataPoint public static String USERNAME_WITH_SLASH = "optimus/prime"; @Theory public void filenameIncludesUsername(String username) { assumeThat(username, not(containsString("/"))); assertThat(new User(username).configFileName(), containsString(username)); } } 

This mechanism is similar to the one that uses ScalaCheck. There are only two small differences:

• ScalaCheck generates test data itself;
• if a test falls on some data set, ScalaCheck tries to finish it with a counter-example easier, pick up some simpler particular case to facilitate further debugging.

It is these differences that allow ScalaCheck to defeat the theories in terms of both code size and number of tests. But if you are still interested in learning more about theories, you can
read about them here .

Pros and cons of property-based approach

Contrary to possible suspicions of the reader, ScalaCheck was created not to completely force out ScalaTest or the notorious JUnit, but to bring additional benefits to the process of unit testing, such as:

• Conciseness: less code - higher coverage compared to the standard (assertion-based) approach
• High-level: we focus on input data in general, and not on special cases.
• Minimization: when something is broken, we will be helped to find exactly where (or we will help ourselves).
• The presence of entities that are easier to test as a whole, rather than testing them component-wise.

However, property-oriented testing is not a silver bullet.
There are disadvantages to this approach:

• The time of testing the braked functions makes you howl at the moon with anguish. (Although it may be a plus. My practice has some cases where the introduction of ScalaCheck made all project participants think about optimization.)
• False sense of security. With ScalaCheck it is easy to believe that we covered everything with tests with all we could, but often this is not the case.
• Insensitivity to boundary conditions.
• Uniform random distribution of test cases. In general, this is good, but not always convenient.

When to use?

ScalaCheck can be used just like any other testing framework. You will have to think a little differently, but, ultimately, ScalaCheck will allow you to write almost any test - though not always in the most efficient, convenient and readable way. However, there are areas where ScalaCheck is really good:

• code that is extremely sensitive to input data;
• finite state machines or any state dependent systems;
• parsers (this is what I personally use ScalaCheck for);
• various data converters :
  
  • validators;
  • classifiers;
  • aggregators;
  • sorters, etc.
• Spark RDD, as well as mapera and reducers for Hadoop.

ScalaCheck

Library features

ScalaCheck is:

• compact library (less than twenty files with code);
• no additional dependencies;
• support of tests with the internal state (stateful testing);
• avoiding the use of java.util.Random as a pseudo-random number generator (moreover, ScalaCheck is careful to ensure that random test sets are not repeated using a return search ).
• support scala-js and Dotty.

Inside java.util.Random linear congruent method is used.
generation of pseudo-random sequences (hereinafter - LCG). More you can
read the official documentation . LCG does not provide enough
quality of generating pseudo-random numbers, and are used in most
libraries solely due to simplicity and high performance.
ScalaCheck uses its own generator, which behaves much better in
serious statistical applications. More information about the generator, you can find out
here

Preparatory work

Once we have decided on whether we need property-oriented testing and ScalaCheck in particular, let's proceed to the preparatory work. Add the following dependency to your project (I expect that you, dear reader, have already switched to Scala 2.12):

 <!--  sbt  gradle,  ,   Maven. --> <dependency> <groupId>org.scalacheck</groupId> <artifactId>scalacheck_2.12</artifactId> <version>1.13.4</version> </dependency> 

It is also assumed that you are using the latest version. There is a problem when using outdated versions of the library in conjunction with Scala 2.12. Be careful.

As mentioned earlier, ScalaCheck is built on two main concepts: properties and generators. Properties are well covered in a variety of blogs, including Russian. Therefore, I will try to pay more attention to the generator. In this part, we will briefly review both the properties and the generators.

Little about properties

The property is the minimum testable module. Represented by an instance of the org.scalacheck.Prop class. The simplest example is:

 import org.scalacheck.Prop val propStringLengthAfterConcat = Prop forAll { s: String => val len = s.length (s + s).length == len + len } //      ,    //  val propDivByZero = Prop.throws(classOf[ArithmeticException] {1/0}) //    ,       //     ,    //  IndexOutOfBoundsException val propListIndexOutOfBounds = Prop forAll { xs: List[Int] => Prop.throws(classOf[IndexOutOfBoundsException]) { xs(xs.length + 1) } 

Little about generators

In practice, generators have to write no less than properties. In order to take advantage of them, you should import org.scalacheck.Gen .

 import org.scalacheck.Gen // ,       //  . val binaryDigit = Gen.choose(0, 1) val octDigit = Gen.choose(0, 7) // ,       //   . val vowel = Gen.oneOf('a', 'e', 'i', 'o', 'u') 

ScalaCheck also has a set of ready-made generators for standard types:

 // ,    . val alphaLower = Gen.alphaLowerChar // ,    (  // ,      ,   //      ). val identifier = Gen.identifier // ,      Long. val natural = Gen.posNum[Long] 

You can also combine existing generators by applying map and for comprehension to them:

 val personGen = for { charValue <- Gen.oneOf("Jason", "Oliver", "Jessica", "Olivia") ageValue <- Gen.posNum[Int] } yield Person (name = nameValue, age = ageValue) 

We will take a closer look at the properties and generators in the following articles. The next article in the series will be devoted to generators. Thank you for reading, stay in touch.