Settings

This article is a translation of the Hypothesis -Settings page taken from the official manual.

Hypothesis tries to use acceptable values ​​in defaults for its behavior, but sometimes this is not enough, and you need to configure it.

The mechanism for this is the object of hypothesis.settings . You can change the basic test settings based on @given using the settings decorator:

The @given call looks like this:

 from hypothesis import given, settings @given(integers()) @settings(max_examples=500) def test_this_thoroughly(x): pass 

In this case, the hypothesis.settings object is used, with the result that the test receives a much larger set of examples than the usual one.

It can be applied either before or after given . This will not affect the results. Therefore, the following example is exactly equivalent to the previous one:

 from hypothesis import given, settings @settings(max_examples=500) @given(integers()) def test_this_thoroughly(x): pass 

Controlling What Runs

Hypothesis divides the tests into four logically different phases:

1. Running explicit examples provided with the @example decorator .
2. Re-run a sample of previously unsuccessful examples to reproduce a previously noticed error.
3. Creating new examples.
4. Attempt to compress the example found in steps 2 or 3 to a more manageable one (explicit examples cannot be compressed).

Phase tuning provides precise control over which of them is performed, with each phase corresponding to the value in the hypothesis._settings.Phase listing:

1. Phase.explicit manages the execution of explicit examples.
2. Phase.reuse manages the reuse of previous examples.
3. Phase.generate determines whether new examples will be created.
4. Phase.shrink manages the reduction (compression) of examples.

The phases argument takes a collection with any subset of them. for example, settings(phases=[Phase.generate, Phase.shrink]) will generate new examples and compress them, but will not run explicit examples or reuse previous failures, while settings(phases=[Phase.explicit]) only explicit examples will be executed.

View intermediate result

To see what happens while Hypothesis performs your tests, you can enable Verbosity in the configuration. It works with both ~hypothesis.core.find and @given .

 >>> from hypothesis import find, settings, Verbosity >>> from hypothesis.strategies import lists, integers >>> find(lists(integers()), any, settings=settings(verbosity=Verbosity.verbose)) Tried non-satisfying example [] Found satisfying example [-1198601713, -67, 116, -29578] Shrunk example to [-67, 116, -29578] Shrunk example to [116, -29578] Shrunk example to [-29578] Shrunk example to [-115] Shrunk example to [115] Shrunk example to [-57] Shrunk example to [29] Shrunk example to [-14] Shrunk example to [-7] Shrunk example to [4] Shrunk example to [2] Shrunk example to [1] [1] 

Four levels: quiet (quiet), normal (normal), verbose (detailed) and debug (debug). normal is the default value, while in quiet mode hypothesis will not print anything, even the final example of tampering. debug is essentially the same verbose, but in a bit more detail. You probably do not want this.

When using pytest, you may also need to disable output capturing for passing tests (writing output for passing tests).

Build settings objects

Settings can be created by calling hypothesis.settings with any of the available settings. Any missing will be set by default:

 >>> from hypothesis import settings >>> settings().max_examples 100 >>> settings(max_examples=10).max_examples 10 

As the first argument, you can also pass an object - the "parent" settings, and any parameters not specified as named arguments will be copied from the parent parameters:

 >>> parent = settings(max_examples=10) >>> child = settings(parent, deadline=200) >>> parent.max_examples == child.max_examples == 10 True >>> parent.deadline not_set >>> child.deadline 200 

Default settings

At any time in your program there are current default settings available as settings.default . In addition to the settings object itself, all newly created settings objects that are not explicitly based on other settings are based on default values, so they will inherit any values ​​that are not explicitly set.

Default values ​​can be changed using profiles (See the next section), but they can also be overridden locally using the parameters object as the context manager

 >>> with settings(max_examples=150): ... print(settings.default.max_examples) ... print(settings().max_examples) 150 150 >>> settings().max_examples 100 

Note that after exiting the block, the default value returns to normal.

This can be used by putting test definitions in context:

 from hypothesis import given, settings with settings(max_examples=500): @given(integers()) def test_this_thoroughly(x): pass 

All created settings objects or tests defined inside the block inherit default values ​​from the settings object used as context. Of course, you can still redefine them with custom settings.

Caution !: If you use the definition of test functions that do not use @given inside the context block, they will not use nested parameters. This is because the context manager only affects the definition, not the execution of the function.

settings profiles

Depending on the environment, various default parameters may be required. For example: during development, you can reduce the number of examples to speed up the tests. However, in the CI environment, more examples may be required in order to more likely find errors.

Hypothesis allows you to define various profile settings. These profiles can be downloaded at any time.

Loading a profile changes the default settings, but does not change the behavior of tests that explicitly change the parameters.

 >>> from hypothesis import settings >>> settings.register_profile("ci", max_examples=1000) >>> settings().max_examples 100 >>> settings.load_profile("ci") >>> settings().max_examples 1000 

Instead of loading a profile and overriding default values, you can get profiles for specific tests.

 >>> with settings.get_profile("ci"): ... print(settings().max_examples) ... 1000 

If necessary, you can define an environment variable for loading the profile. This is the recommended template for running tests in CI. The code below should run in conftest.py or any setup / initialization section of the test suite. If this variable is not defined, the default values ​​determined by Hypothesis will be loaded.

 >>> import os >>> from hypothesis import settings, Verbosity >>> settings.register_profile("ci", max_examples=1000) >>> settings.register_profile("dev", max_examples=10) >>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose) >>> settings.load_profile(os.getenv(u'HYPOTHESIS_PROFILE', 'default')) 

If you are using the hypothesis pytest plugin and your profiles are registered with your conftest, you can load one with the command line --hypothesis-profile .

 $ pytest tests --hypothesis-profile 

Timeouts

The timeout hypothesis functionality is obsolete and will be removed. At this time, the timeout parameters can still be assigned, and the old default timeout remains in one minute.

If you want to use your code in the future, you can evaluate its behavior in the future by setting the timeout to hypothesis.unlimited .

 from hypothesis import given, settings, unlimited from hypothesis import strategies as st @settings(timeout=unlimited) @given(st.integers()) def test_something_slow(i): ... 

This will cause your code to be executed until it selects the usual example within the Hypothesis framework, no matter how long it takes. timeout=unlimited will remain a valid parameter after dropping the timeout function (but then will have its own aging cycle).

However, there is now a health check associated with deadlines and health check, which is designed to catch tests that are ready to work for centuries. If you really want your test to run forever, the following code will allow this:

 from hypothesis import given, settings, unlimited, HealthCheck from hypothesis import strategies as st @settings(timeout=unlimited, suppress_health_check=[ HealthCheck.hung_test ]) @given(st.integers()) def test_something_slow(i): ... 

Back