The history of testing the project "K": Kotlin & Spek

Hi, Habr!

In this article we will talk about automated testing on one of the many QIWI projects, which received the code name "K".



When we organized the testing of this project, we decided to choose a practical and HYIP Kotlin , as well as Spek , saying “You call them tests, we call them specifications” (You call them tests, we call them specifications). Perhaps this approach will suit you if you run into similar tasks.
')
Why Kotlin, and not something else? Kotlin was chosen by development, to experiment, since this particular product was not critical, and it was possible to practice it live without fearing that there would be problems.

The official documentation tells us that “Spek is written in Kotlin, and the specifications you write will be written in Kotlin” - this very clearly answers the question: “Why is it necessary?”.

So…

What is it and why is it needed?

The project provides its partner with software, which is an Android application. The lion's share of tests falls on the back-end, so we will focus on testing the REST API.

For a bundle that allows you to write tests and get results, everything is clear: you need a programming language, a test framework, an HTTP client and reports. And what about the entry point to our test universe?

Requirements, they are specifications, the project developers decided to write in the form of tests. The result was an interesting picture - BDD. Thus, Kotlin, Spek and khttp appeared in the arena.
The attentive reader will ask - OK, but where are the testers?

Testers

Having finished two birds with one stone, the development gave the product tester both the requirements and the autotests. Since then, the tester extends the test coverage, according to the requirements, and also supports and creates, together with the developers, new tests.

“This can not last forever and should not end tragically for the testing process!” - when colleagues had such an idea, the service team of the Testing Department entered the game. The service department was faced with the task: to study Kotlin in a short time in order to immediately take over the support of the tests if necessary.

Getting started

In service, the service department has IntelliJ IDEA , and since Kotlin works on top of the JVM and is developed by JetBrains , it wasn’t necessary to install something extra for writing code.

The process of learning the language for obvious reasons, skip.

The first thing to start with was to clone the repository:
git clone https://gerrit.project.com/k/autotests

Then the project was opened and the gradle settings were imported :



For complete satisfaction and comfort (* In fact, it is necessary), the Spek plugin was installed:



He ensured the launch of tests in the development environment:



The first stage was completed, and it was time to start writing the tests themselves.

Tests

Brave guys from the service department do not belong to a particular product. These are the employees who are in a hurry to assist in setting up automation on the project, including all the steps of the process, and then handing over the tests and testing to product testers.

And since the interaction of the internal teams of the testing department is organized in a similar way, then at the entrance the service department “asks” at least the requirements for the feature.

It may seem like a deadlock in the case of “K”. But it was not there:

• Required read access to the repository where the source of the project is stored;
• Cloned the repository;
• Began to dive into the functionality of the product through reading the source code written in Java.

What did you read?

Development "K" asked to write tests for the feature, which allowed you to add, update and delete products for sale. The implementation consisted of two parts: “web” and “mobile”.

In the case of web:

• To add products using a POST request, the body of which contains JSON with data.
• To update or edit products use a PUT request, the body of which contains JSON with modified data.
• To delete products use a DELETE request, the body of which is empty.

In the case of mobile:

To add, update and delete products use a POST request, the body of which contains JSON with data for the specified operations.

Those. JSON has three nodes:

• "Added": list of added goods,
• "Removed": list of products to be removed,
• Updated: list of updated products.

What did you write?

A test class containing test-specifications was already created and contained test methods (* a little out of Spek), so all that was needed was to expand it.

For web

Test for successful addition of goods:

• Add product
• Check that the item is added.
• Delete created product (postcondition)

Code:

 on("get changed since when goods added") { val goodsUpdateName: String = RandomStringUtils.randomAlphabetic(8) val date = Date.from(Instant.now()).time - 1 val goodsAdded = post(baseUrl + "goods/${user.storeId}", authHeader, json = dataToMap(goods.copy(name = goodsUpdateName))) it("should return the status code OK") { goodsAdded.statusCode.should.be.equal(OK) } val goodId = goodsAdded.jsonObject?.optLong("id") val goodsUpdates = get(baseUrl + "updates/goods/since/" + date, authHeader.plus("AppUID" to user.AppUID)) it("should return the status code OK") { goodsUpdates.statusCode.should.be.equal(OK) } val goodsInsert = goodsUpdates.jsonObject.getJSONArray("updated").toList() .map { it as JSONObject } .find { it.optLong("goodId") == goodId } it("should contain goods insert") { goodsInsert.should.be.not.`null` goodsInsert?.optString("name").should.be.equal(goodsUpdateName) } delete(baseUrl + "goods/${user.storeId}/$goodId", authHeader) } 

Test for successful removal of goods:

• Add a product (precondition)
• Remove the goods
• Check that the product is deleted.

Code:

 on("get changed since when goods deleted") { val goodsUpdateName: String = RandomStringUtils.randomAlphabetic(8) val date = Date.from(Instant.now()).time - 1 val goodsAdded = post(baseUrl + "goods/${user.storeId}", authHeader, json = dataToMap(goods.copy(name = goodsUpdateName))) it("should return the status code OK") { goodsAdded.statusCode.should.be.equal(OK) } val goodId = goodsAdded.jsonObject?.optLong("id") val responseDelete = delete(baseUrl + "goods/${user.storeId}/$goodId", authHeader) it("should return the status code NO_CONTENT") { responseDelete.statusCode.should.be.equal(NO_CONTENT) } val goodsUpdates = get(baseUrl + "updates/goods/since/" + date, authHeader.plus("AppUID" to user.AppUID)) it("should contain goods deletes") { goodsUpdates.statusCode.should.be.equal(OK) goodsUpdates.jsonObject.getJSONArray("removed").toList() .map { it as Int } .find { it == goodId.toInt() } .should.be.not.`null` } } 

Negative test for unauthorized user request execution

• Add product
• Check the status of the response
• A request to add a product is sent without an authorization header. The answer comes with a status of 401 Unauthorized.

Code:

  on("get changed since when goods added without authorization") { val response = post(baseUrl + "goods/${user.storeId}", json = dataToMap(goods)) it("should contain an Unauthorized response status and an empty body") { response.statusCode.should.be.equal(UNAUTHORIZED) response.text.should.be.equal("") } } 

For mobile

Auxiliary functions were written to get the nodes from the response body and form the request body.

Code:

 package com.qiwi.k.tests import com.fasterxml.jackson.databind.ObjectMapper import khttp.responses.Response import org.json.JSONObject val mapper = ObjectMapper() fun arrayAdded(n: Int): Array<GoodsUpdate> { return Array(n) { i -> GoodsUpdate() } } fun getGoodsIds(list: List<GoodsUpdate>): List<Long> { return Array(list.size) { i -> list[i].goodId }.toList() } fun getResult(response: Response): List<GoodsUpdate> { return mapper.readValue( response.jsonObject.getJSONArray("result").toString(), Array<GoodsUpdate>::class.java ).toList() } fun getCountryIdFromTheResult(response: Response): List<Int> { val listGoods = mapper.readValue( response.jsonObject.getJSONArray("result").toString(), Array<GoodsUpdate>::class.java ).toList() return Array(listGoods.size) { i -> listGoods[i].countryId }.toList() } fun getBody(added: Array<GoodsUpdate> = emptyArray(), removed: List<Long> = emptyList(), updated: List<GoodsUpdate> = emptyList()): JSONObject { return JSONObject( mapOf( "added" to added, "removed" to removed, "updated" to updated ) ) } 

Test for the successful addition of goods

• Add product
• Check that the item is added.
• Delete the item (postcondition)

Code:

 on("adding goods") { val respAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count))) val resultOfAdding = getResult(respAdd) it("should return the status code OK") { respAdd.statusCode.should.be.equal(OK) } it("should be equal to the size of the variable count") { resultOfAdding.should.be.size.equal(count) } post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(removed = getGoodsIds(resultOfAdding))) } 

Test for successful product update

• Add a product (precondition)
• We update the product
• Check that the added product is updated.
• Delete the item (postcondition)

Code:

 on("updating goods") { val respAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count))) val resultOfAdding = getResult(respAdd) it("should return the status code respAdd OK") { respAdd.statusCode.should.be.equal(OK) } it("should be equal to the size of the variable count (resultOfAdding)") { resultOfAdding.should.be.size.equal(count) } val respUpdate = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(updated = resultOfAdding.map { it.copy(countryId = REGION_77) }) ) it("should return the status code respUpdate OK") { respUpdate.statusCode.should.be.equal(OK) } it("should be equal to the size of the variable count (respUpdate)") { getResult(respUpdate).should.be.size.equal(count) } it("should be all elements are 77") { getCountryIdFromTheResult(respUpdate).should.be.all.elements(REGION_77) } post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(removed = getGoodsIds(resultOfAdding))) } 

Test for successful removal of goods:

• Add a product (precondition)
• Remove the goods
• Check that the added item has been deleted.

Code:

 on("deleting goods") { val respAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count))) val resultOfAdding = getResult(respAdd) it("should return the status code respAdd OK") { respAdd.statusCode.should.be.equal(OK) } it("should be equal to the size of the variable count") { resultOfAdding.should.be.size.equal(count) } val respRemoved = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(removed = getGoodsIds(resultOfAdding)) ) it("should return the status code respRemoved OK") { respRemoved.statusCode.should.be.equal(OK) } it("should be empty") { getResult(respRemoved).should.be.empty } } 

After writing the tests, it was necessary to go through the review code.

Review

More than a dozen commits, a lot of correspondence with dev, visiting forums, chatting with Google. And that's what the result.

Code:

 package com.qiwi.k.tests.catalog import … class GoodsUpdatesControllerSpec : WebSpek({ given("GoodsUpdatesController") { val OK = HttpResponseStatus.OK.code() val NO_CONTENT = HttpResponseStatus.NO_CONTENT.code() val UNAUTHORIZED = HttpResponseStatus.UNAUTHORIZED.code() val REGION_77 = 77 val auth = login(user) val accessToken = auth.tokenHead + auth.tokenTail val authHeader = mapOf("Authorization" to "Bearer $accessToken") val baseUrl = "http://test.qiwi.com/catalog/" val count = 2 val authHeaderWithAppUID = mapOf("Authorization" to "Bearer $accessToken", "AppUID" to user.AppUID) val urlGoodsUpdate = "http://test.qiwi.com/catalog/updates/goods/" on("get changes since") { val goodsName: String = goodsForUpdate.name + Random().nextInt(1000) val date = Date.from(Instant.now()).time - 1 put(baseUrl + "goods/${user.storeId}", authHeader, json = dataToMap(goodsForUpdate.copy(name = goodsName))) val goodsUpdates = get(baseUrl + "updates/goods/since/" + date, authHeader.plus("AppUID" to user.AppUID)) it("should contain goods updates") { val updates = goodsUpdates.jsonObject.getJSONArray("updated").toList() .map { it as JSONObject } .find { it.optLong("goodId") == goodsForUpdate.id } updates.should.be.not.`null` updates?.optString("name").should.be.equal(goodsName) } } on("get changed since when goods added") { val goodsUpdateName: String = RandomStringUtils.randomAlphabetic(8) val date = Date.from(Instant.now()).time - 1 val goodsAdded = post(baseUrl + "goods/${user.storeId}", authHeader, json = dataToMap(goods.copy(name = goodsUpdateName))) it("should return the status code OK") { goodsAdded.statusCode.should.be.equal(OK) } val goodId = goodsAdded.jsonObject?.optLong("id") val goodsUpdates = get(baseUrl + "updates/goods/since/" + date, authHeader.plus("AppUID" to user.AppUID)) it("should return the status code OK") { goodsUpdates.statusCode.should.be.equal(OK) } val goodsInsert = goodsUpdates.jsonObject.getJSONArray("updated").toList() .map { it as JSONObject } .find { it.optLong("goodId") == goodId } it("should contain goods insert") { goodsInsert.should.be.not.`null` goodsInsert?.optString("name").should.be.equal(goodsUpdateName) } delete(baseUrl + "goods/${user.storeId}/$goodId", authHeader) } on("get changed since when goods deleted") { val goodsUpdateName: String = RandomStringUtils.randomAlphabetic(8) val date = Date.from(Instant.now()).time - 1 val goodsAdded = post(baseUrl + "goods/${user.storeId}", authHeader, json = dataToMap(goods.copy(name = goodsUpdateName))) it("should return the status code OK") { goodsAdded.statusCode.should.be.equal(OK) } val goodId = goodsAdded.jsonObject?.optLong("id") val responseDelete = delete(baseUrl + "goods/${user.storeId}/$goodId", authHeader) it("should return the status code NO_CONTENT") { responseDelete.statusCode.should.be.equal(NO_CONTENT) } val goodsUpdates = get(baseUrl + "updates/goods/since/" + date, authHeader.plus("AppUID" to user.AppUID)) it("should contain goods deletes") { goodsUpdates.statusCode.should.be.equal(OK) goodsUpdates.jsonObject.getJSONArray("removed").toList() .map { it as Int } .find { it == goodId.toInt() } .should.be.not.`null` } } on("get changed since when goods added without authorization") { val response = post(baseUrl + "goods/${user.storeId}", json = dataToMap(goods)) it("should contain an Unauthorized response status and an empty body") { response.statusCode.should.be.equal(UNAUTHORIZED) response.text.should.be.equal("") } } on("adding goods") { val respAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count))) val resultOfAdding = getResult(respAdd) it("should return the status code OK") { respAdd.statusCode.should.be.equal(OK) } it("should be equal to the size of the variable count") { resultOfAdding.should.be.size.equal(count) } post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(removed = getGoodsIds(resultOfAdding))) } on("updating goods") { val respAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count))) val resultOfAdding = getResult(respAdd) it("should return the status code respAdd OK") { respAdd.statusCode.should.be.equal(OK) } it("should be equal to the size of the variable count (resultOfAdding)") { resultOfAdding.should.be.size.equal(count) } val respUpdate = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(updated = resultOfAdding.map { it.copy(countryId = REGION_77) }) ) it("should return the status code respUpdate OK") { respUpdate.statusCode.should.be.equal(OK) } it("should be equal to the size of the variable count (respUpdate)") { getResult(respUpdate).should.be.size.equal(count) } it("should be all elements are 77") { getCountryIdFromTheResult(respUpdate).should.be.all.elements(REGION_77) } post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(removed = getGoodsIds(resultOfAdding))) } on("deleting goods") { val respAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count))) val resultOfAdding = getResult(respAdd) it("should return the status code respAdd OK") { respAdd.statusCode.should.be.equal(OK) } it("should be equal to the size of the variable count") { resultOfAdding.should.be.size.equal(count) } val respRemoved = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(removed = getGoodsIds(resultOfAdding)) ) it("should return the status code respRemoved OK") { respRemoved.statusCode.should.be.equal(OK) } it("should be empty") { getResult(respRemoved).should.be.empty } } } }) 

Total

The code itself, language proficiency and knowledge of the framework are far from perfect, but the beginning is generally good.

When meeting Kotlin, there was a feeling that he was syntactic sugar in Java. And while writing the code with all the fibers of the soul, I managed to feel the words: “fully compatible with Java”.

Spek, which uses simple language constructs to describe specifications, provides a complete pool of test methods. Those. gives what they want from it as from a test framework.

Total - all tests in master. Everything turned out, and the service department now knows for sure that it will be able to support colleagues from “K” in difficult times.