Moki and explicit contracts

Probably everyone who started writing unit and integration tests, faced with the problem of mock abuse, which leads to fragile tests. The latter, in turn, create for the programmer the wrong belief that tests only interfere with the work.

Below is a free translation of an article in which José Valim - the creator of the Elixir language - expressed his opinion on the problem of using mocks, with which I fully agree.

A few days ago I shared my thoughts about mocks on Twitter:

Mock is a useful tool for testing, but existing test libraries and frameworks often lead to abuse of this tool. Below we look at the best way to use mocks.

What is mock?

We use the definition from the English-language Wikipedia : Mock - customizable object that simulates the behavior of a real object. I will focus on this later, but for me, mock is always a noun, and not a verb [ for clarity, the verb mock will be translated everywhere as “lock” - approx. trans. ].

On the example of the external API

Let's look at a standard real-life example: an external API.

Imagine that you want to use the Twitter API in a web application on a Phoenix or Rails framework. The application receives a request, which is redirected to the controller, which, in turn, makes a request to the external API. The external API is called directly in the controller:

defmodule MyApp.MyController do def show(conn, %{"username" => username}) do # ... MyApp.TwitterClient.get_username(username) # ... end end 

The standard approach when testing such a code is to lock (it is dangerous! To lock in this case is a verb!) HTTPClient , which MyApp.TwitterClien uses:

 mock(HTTPClient, :get, to_return: %{..., "username" => "josevalim", ...}) 

Next, you use the same approach in other parts of the application and you are trying to pass unit and integration tests. Time to move on?

Not so fast. The main problem with HTTPClient is to create a strong external dependency . coupling everywhere translated as "dependency" - approx. trans. ] to a specific HTTPClient . For example, if you decide to use a new, faster HTTP client without changing the behavior of the application , most of your integration tests will fall, because they all depend on a particular HTTP HTTPClient . In other words, changing the implementation without changing the behavior of the system still leads to a drop in the tests. This is a bad sign.

In addition, since the mock shown above modifies the module globally, you can no longer run these tests in parallel with Elixir.

Decision

Instead of using HTTPClient , we can replace MyApp.TwitterClient with something else during the tests. Let's see how the solution might look like on Elixir.

In Elixir, all applications have configuration files and a mechanism for reading them. We use this mechanism to configure the Twitter client for various environments. The controller code will now look like this:

 defmodule MyApp.MyController do @twitter_api Application.get_env(:my_app, :twitter_api) def show(conn, %{"username" => username}) do # ... @twitter_api.get_username(username) # ... end end 

Appropriate settings for different environments:

 # config/dev.exs config :my_app, :twitter_api, MyApp.Twitter.Sandbox # config/test.exs config :my_app, :twitter_api, MyApp.Twitter.InMemory # config/prod.exs config :my_app, :twitter_api, MyApp.Twitter.HTTPClient 

Now we can choose the best strategy for getting data from Twitter for each of the environments. Sandbox can be useful if Twitter provides some kind of sandbox for development. Our locked-in version of HTTPClient allowed HTTPClient to avoid real HTTP requests. Implementing the same functionality in this case:

 defmodule MyApp.Twitter.InMemory do def get_username("josevalim") do %MyApp.Twitter.User{ username: "josevalim" } end end 

The code is simple and clean, and there is no longer a strong external dependency on HTTPClient . MyApp.Twitter.InMemory is mock , that is, a noun , and you do not need any libraries to create it!

Need for explicit contracts

A mock is intended to replace a real object, which means it will be effective only when the behavior of a real object is clearly defined. Otherwise, you may find yourself in a situation where the moke starts to get harder, increasing the dependency between the tested components. Without an explicit contract, it will be difficult to notice.

We already have three implementations of the Twitter API and it’s better to make their contracts explicit. In Elixir, you can describe an explicit contract with the help of behavior :

 defmodule MyApp.Twitter do @doc "..." @callback get_username(username :: String.t) :: %MyApp.Twitter.User{} @doc "..." @callback followers_for(username :: String.t) :: [%MyApp.Twitter.User{}] end 

Now add @behaviour MyApp.Twitter to each module that implements this contract, and Elixir will help you create the expected API.

At Elixir, we rely on such behaviors all the time: when we use Plug , when we work with a database in Ecto , when we test Phoenix channels and so on.

Boundary testing

At first, when there were no explicit contracts, the boundaries of the application looked like this:

[MyApp] -> [HTTPClient] -> [Twitter API]

Therefore, a change in HTTPClient could lead to a drop in integration tests. Now the application depends on the contract and only one implementation of this contract works with HTTP:

[MyApp] -> [MyApp.Twitter (contract)]

[MyApp.Twitter.HTTP (contract impl)] -> [HTTPClient] -> [Twitter API]

Tests of such an application are isolated from the HTTPClient and from the Twitter API. But how do we test MyApp.Twitter.HTTP ?

The difficulty of testing large systems lies in defining clear boundaries between components. Too high level of isolation in the absence of integration tests will make tests fragile, and most of the problems will be detected only in production. On the other hand, a low isolation level will increase the test passing time and make tests difficult to maintain. There is no one right decision, and the level of isolation will vary depending on the team's confidence and other factors.

Personally, I would test MyApp.Twitter.HTTP on the real Twitter API, running these tests as needed during development and every time a project is built. The tagging system in ExUnit — the library for testing in Elixir — implements this behavior:

 defmodule MyApp.Twitter.HTTPTest do use ExUnit.Case, async: true #      Twitter API @moduletag :twitter_api # ... end 

Exclude tests with Twitter API:

 ExUnit.configure exclude: [:twitter_api] 

If necessary, include them in the general test run:

mix test --include twitter_api

You can also run them separately:

mix test --only twitter_api

Although I prefer this approach, external restrictions, such as the maximum number of requests to the API, can make it useless. In such a case, you may actually need to use the HTTPClient if its use does not violate the previously defined rules:

1. The change in HTTPClient only leads to a drop in the tests on MyApp.Twitter.HTTP
2. You are not mocking (caution! In this case, the mock is a verb!) HTTPClient . Instead, you pass it as a dependency through a configuration file, just as we did for the Twitter API
3. You still need a way to test your client's work before rolling out to production.

Instead of creating a mock HTTPClient you can raise a dummy server that will emulate the Twitter API. bypass is one of the projects that can help with this. All possible options you should discuss with your team.

Notes

I would like to finish this article by analyzing a few common problems that surface in almost every discussion of mocks.

Creation of "tested" code

Quote from elixir-talk mailing list :

It turns out that the proposed solution makes the production code more "testable", but creates the need to go into the configuration of the application for each function call? Having an unnecessary overhead to make something “testable” doesn't seem like a good solution.

I would say that this is not about creating a "testable" code, but about improving the design [ from English. design of your code - approx. trans. ].

A test is a user of your API, like any other code you write. One of the ideas of TDD is that tests are code and are no different from code. If you say: “I don’t want to make my code testable,” it means “I don’t want to reduce the dependency between components” or “I don’t want to think about the contract (interface) of these components.”

There is nothing wrong with not wanting to reduce the dependency between components. For example, if we are talking about the module of work with URI [ meaning the URI module for Elixir - approx. trans. ]. But if we are talking about something as complicated as the external API, defining an explicit contract and having the ability to replace the implementation of this contract will make your code comfortable and easy to maintain.

In addition, the overhead is minimal, since the configuration of the Elixir application is stored in the ETS , which means it is read directly from the memory.

Local moki

Although we used application configuration to solve an external API problem, it is sometimes easier to pass the dependency as an argument. For example, some function performs many calculations that you want to isolate in tests:

 defmodule MyModule do def my_function do # ... SomeDependency.heavy_work(arg1, arg2) # ... end end 

You can get rid of the dependency by passing it as an argument. In this case, passing an anonymous function will suffice:

 defmodule MyModule do def my_function(heavy_work \\ &SomeDependency.heavy_work/2) do # ... heavy_work.(arg1, arg2) # ... end end 

The test will look like this:

 test "my function performs heavy work" do #         heavy_work = fn(_, _) -> send(self(), :heavy_work) end MyModule.my_function(heavy_work) assert_received :heavy_work end 

Or, as described earlier, you can define a contract and transfer the entire module:

 defmodule MyModule do def my_function(dependency \\ SomeDependency) # ... dependency.heavy_work(arg1, arg2) # ... end end 

Change the test:

 test "my function performs heavy work" do #         defmodule TestDependency do def heavy_work(_arg1, _arg2) do send self(), :heavy_work end end MyModule.my_function(TestDependency) assert_received :heavy_work end 

You can also represent a dependency in the form of a data structure and define a contract using protocol .

Passing the dependency as an argument is much simpler, so if possible, this method should be preferable to using the configuration file and Application.get_env/3 .

Mok is a noun

It is better to think of mocks as nouns. Instead of mopping the API (moka is a verb), you need to create mok (mok is a noun) that implements the necessary API.

Most of the problems of using mocks arise when they are used as verbs. If you wash something, you change already existing objects, and often these changes are global. For example, when we mock the module SomeDependency , it will change globally:

 mock(SomeDependency, :heavy_work, to_return: true) 

When using mock as a noun, you need to create something new, and, naturally, this cannot be an already existing module SomeDependency . The rule "mok is a noun, not a verb" helps to find "bad" mocks. But your experience may be different from mine.

Libraries to create mocks

After reading you may have the question: "Do I need to abandon libraries to create mocks?"

It all depends on the situation. If the library pushes you to substitute global objects (or use mocks as verbs), change static methods in object-oriented, or replace modules in functional programming, that is, violate the rules described above, you should refuse it.

However, there are libraries for creating mocks that do not encourage you to use the anti-patterns described above. Such libraries provide "mock objects" or "mock modules", which are passed to the system under test as an argument and collect information about the number of mock calls and about which arguments it was called with.

Conclusion

One of the tasks of testing the system is finding the right contracts and boundaries between components. Using mocks only if you have an explicit contract will allow you to:

1. To protect against the dominance of mocks, since contracts will be created only for the necessary parts of the system. As mentioned above, you are unlikely to want to hide the interaction with standard URI and Enum modules behind the contract.
2. Simplify component support. When adding new functionality to a dependency, you need to renew the contract (add a new @callback to Elixir). The endless growth of @callback will indicate a dependency that takes on too much responsibility, and you can deal with the problem earlier.
3. Make your system suitable for testing, because the interaction between complex components will be isolated.

Explicit contracts let you see the complexity of dependencies in your application. Difficulty is present in every application, so always try to make it as obvious as possible.