Consumer Driven Contracts or Gitlab CI through the eyes of QA test automation

The objectives of this publication :

• Brief Introduction to Consumer Driven Contracts (CDC)
• Configure CI pipeline based on CDC

Consumer Driven Contracts

In this part we will go through the main points of the CDC. This article is not exhaustive on the subject of contract testing. There is a sufficient amount of materials on this topic on the same Habré .

To continue, we need to get acquainted with the basic provisions of the CDC:

• Contact testing is at the level of Service / Integration Tests above Unit Tests according to the autotest pyramid (Mike Cohn)
• Contract testing can be used when there are 2 (or more) services that interact with each other.
• The consumer-driven approach means that the first step in implementation is to write a test on the consumer side. The result of the test is a pact (contract) in json format that describes the interaction between a consumer (for example, a web interface / mobile interface: a service that wants to get some data) and a provider (for example, a server API: a service that provides data)
• The next step is to check the contract with the provider. This is fully implemented by the Pact framework.

Let's start with the test on the consumer side . I used Pactman . Here is the test:

import pytest from pactman import Like from model.client import Client @pytest.fixture() def consumer(pact): return Client(pact.uri) def test_app(pact, consumer): expected = '123456789' (pact .given('provider in some state') .upon_receiving("request to get user's phone number") .with_request( method='GET', path=f'/phone/john', ) .will_respond_with(200, body=Like(expected)) .given('provider in some state') .upon_receiving("request to get non-existent user's phone number") .with_request( method='GET', path=f'/phone/micky' ) .will_respond_with(404) ) with pact: consumer.get_users_phone(user='john', host=pact.uri) consumer.get_users_phone(user='micky', host=pact.uri) 

Using Pact DSL, we describe request / response interactions. After running the test, we get a new file ({consumer} - {provider} -pact.json):

 { "consumer": { "name": 'basic_client' }, "provider": { "name": 'basic_flask_app' }, "interactions": [ { "providerStates": [ { "name": "provider in some state", "params": {} } ], "description": "request to get user's phone number", "request": { "method": "GET", "path": "/phone/john" }, "response": { "status": 200, "body": "123456789", "matchingRules": { "body": { "$": { "matchers": [ { "match": "type" } ] } } } } }, { "providerStates": [ { "name": "provider in some state", "params": {} } ], "description": "request to get non-existent user's phone number", "request": { "method": "GET", "path": "/phone/micky" }, "response": { "status": 404 } } ], "metadata": { "pactSpecification": { "version": "3.0.0" } } } 

Next, we need to transfer the pact to the provider for verification. This is done using Pact Broker.

Pact Broker is a repository of contracts with some additional functions that allow us to track the compatibility of service versions, as well as generate network diagrams (service interaction).

Pact broker

Pact

Version Matrix

Provider check

This part of the test is fully implemented by the framework. After verification, the results are sent back to Pact Broker.

 provider-verifier_1 | Verifying a pact between basic_client and basic_flask_app provider-verifier_1 | Given provider in some state provider-verifier_1 | request to get user's phone number provider-verifier_1 | with GET /phone/john provider-verifier_1 | returns a response which provider-verifier_1 | WARN: Skipping set up for provider state 'provider in some state' for consumer 'basic_client' as there is no --provider-states-setup-url specified. provider-verifier_1 | has status code 200 provider-verifier_1 | has a matching body provider-verifier_1 | Given provider in some state provider-verifier_1 | request to get non-existent user's phone number provider-verifier_1 | with GET /phone/micky provider-verifier_1 | returns a response which provider-verifier_1 | WARN: Skipping set up for provider state 'provider in some state' for consumer 'basic_client' as there is no --provider-states-setup-url specified. provider-verifier_1 | has status code 404 provider-verifier_1 | provider-verifier_1 | 2 interactions, 0 failures 

Run both parts of the test in the pipeline

Now that both parts of contract testing have been disassembled, it would be nice to run them at every commit. This is where Gitlab CI comes to the rescue. Pipeline jobs are described in .gitlab-ci.yml . Before we get to the pipeline, we need to say a few words about GitLab Runner, which is an open-source project, and is used to start jobs and send the results back to GitLab. Jobs can be executed locally or using Docker containers. In our project we use Docker. The test infrastructure is implemented in containers and described in docker-compose.yml , located in the root of the project.

 version: '2' services: basic-flask-app: image: registry.gitlab.com/tknino69/basic_flask_app:latest ports: - 5005:5005 postgres: image: postgres ports: - 5432:5432 env_file: - test-setup.env volumes: - db-data:/var/lib/postgresql/data/pgdata pactbroker: image: dius/pact-broker links: - postgres ports: - 80:80 env_file: - test-setup.env provider-states: image: registry.gitlab.com/tknino69/cdc/provider-states:latest build: provider-states ports: - 5000:5000 consumer-test: image: registry.gitlab.com/tknino69/cdc/consumer-test:latest command: ["sh", "-c", "find -name '*.pyc' -delete && pytest $${TEST}"] links: - pactbroker environment: - CONSUMER_VERSION=$CI_COMMIT_SHA provider-verifier: image: registry.gitlab.com/tknino69/cdc/provider-verifier:latest build: provider-verifier ports: - 5001:5000 links: - pactbroker depends_on: - consumer-test - provider-states command: ['sh', '-c', 'find -name "*.pyc" -delete && CONSUMER_VERSION=`curl --header "PRIVATE-TOKEN:$${API_TOKEN}" https://gitlab.com/api/v4/projects/$${BASIC_CLIENT}/repository/commits | jq ".[0] .id" | sed -e "s/\x22//g"` && echo $${CONSUMER_VERSION} && pact-provider-verifier $${PACT_BROKER}/pacts/provider/$${PROVIDER}/consumer/$${CONSUMER}/version/$${CONSUMER_VERSION} --provider-base-url=$${BASE_URL} --pact-broker-base-url=$${PACT_BROKER} --provider=$${PROVIDER} --consumer-version-tag=$${CONSUMER_VERSION} --provider-app-version=$${PROVIDER_VERSION} -v --publish-verification-results=PUBLISH_VERIFICATION_RESULTS'] environment: - PROVIDER_VERSION=$CI_COMMIT_SHA - API_TOKEN=$API_TOKEN env_file: - test-setup.env volumes: db-data: 

So, we have services that run in containers as needed.

Service Provider:

 basic-flask-app: image: registry.gitlab.com/tknino69/basic_flask_app:latest ports: - 5005:5005 

Pact Broker and its DB. Volumes allow us to have a permanent repository for provider pacts and verification results:

 postgres: image: postgres ports: - 5432:5432 env_file: - test-setup.env volumes: - db-data:/var/lib/postgresql/data/pgdata pactbroker: image: dius/pact-broker links: - postgres ports: - 80:80 env_file: - test-setup.env 

Service Provider States. In practice, he must bring the provider into a certain state (for example, log the user in the database). However, in our example, it simply performs a dummy function.

 provider-states: image: registry.gitlab.com/tknino69/cdc/provider-states:latest build: provider-states ports: - 5000:5000 

The service that runs the Consumer Test. Note the command that runs in the container find -name '* .pyc' -delete && pytest $$ {TEST}

 consumer-test: image: registry.gitlab.com/tknino69/cdc/consumer-test:latest command: ["sh", "-c", "find -name '*.pyc' -delete && pytest $${TEST}"] links: - pactbroker environment: - CONSUMER_VERSION=$CI_COMMIT_SHA 

Provider Verifier Service:

 provider-verifier: image: registry.gitlab.com/tknino69/cdc/provider-verifier:latest build: provider-verifier ports: - 5001:5000 links: - pactbroker depends_on: - consumer-test - provider-states command: ['sh', '-c', 'find -name "*.pyc" -delete && CONSUMER_VERSION=`curl --header "PRIVATE-TOKEN:$${API_TOKEN}" https://gitlab.com/api/v4/projects/$${BASIC_CLIENT}/repository/commits | jq ".[0] .id" | sed -e "s/\x22//g"` && echo $${CONSUMER_VERSION} && pact-provider-verifier $${PACT_BROKER}/pacts/provider/$${PROVIDER}/consumer/$${CONSUMER}/version/$${CONSUMER_VERSION} --provider-base-url=$${BASE_URL} --pact-broker-base-url=$${PACT_BROKER} --provider=$${PROVIDER} --consumer-version-tag=$${CONSUMER_VERSION} --provider-app-version=$${PROVIDER_VERSION} -v --publish-verification-results=PUBLISH_VERIFICATION_RESULTS'] environment: - PROVIDER_VERSION=$CI_COMMIT_SHA - API_TOKEN=$API_TOKEN env_file: - test-setup.env 

Consumer Pipeline
.gitlab-ci.yml in the root of the consumer project describes the processes that are performed on the consumer side:

 image: gitlab/dind:latest variables: TEST: 'tests/docker-compose.app.yml' CONSUMER_VERSION: $CI_COMMIT_SHA BASIC_APP: '11993024' services: - gitlab/gitlab-runner:latest before_script: - docker login -u $GIT_USER -p $GIT_PASS registry.gitlab.com stages: - clone_test - get_broker_up - test - verify_provider - clean_up clone test: tags: - cdc stage: clone_test script: - git clone https://$GIT_USER:$GIT_PASS@gitlab.com/tknino69/cdc.git && ls -ali artifacts: paths: - cdc/ broker: tags: - cdc stage: get_broker_up script: - cd cdc && docker-compose -f docker-compose.yml up -d pactbroker dependencies: - clone test test: tags: - cdc stage: test script: - cd cdc && CONSUMER_VERSION=$CONSUMER_VERSION docker-compose -f docker-compose.yml -f $TEST up consumer-test dependencies: - clone test provider verification: tags: - cdc stage: verify_provider script: - curl -X POST -F token=$CI_JOB_TOKEN -F ref=master https://gitlab.com/api/v4/projects/$BASIC_APP/trigger/pipeline when: on_success clean up: tags: - cdc stage: clean_up script: - cd cdc && docker-compose stop consumer-test dependencies: - clone test 

Here the following happens:

In before_script we log in to our gitlab registry using the $ GIT_USER and $ GIT_PASS variables, which we set in the “Settings”> “CI / CD” section

• Next, we clone the test project.
• In the next step, we raise Pact Broker
• Then run the Consumer Test
• After that, use the Gitlab API to launch provider verification
• And finally, we clean up

Provider Pipeline
The configuration of the pipeline provider is stored in .gitlab-ci.yml in the root of the project provider.

 image: gitlab/dind:latest variables: TEST: 'tests/docker-compose.app.yml' PROVIDER_VERSION: $CI_COMMIT_SHA services: - gitlab/gitlab-runner:latest stages: - clone_test - provider_verification - clean_up clone test: tags: - cdc stage: clone_test script: - git clone https://$GIT_USER:$GIT_PASS@gitlab.com/tknino69/cdc.git artifacts: paths: - cdc/ verify provider: tags: - cdc stage: provider_verification before_script: - cd cdc - docker login -u $GIT_USER -p $GIT_PASS registry.gitlab.com && docker-compose -f docker-compose.yml up -d basic-flask-app script: - PROVIDER_VERSION=$PROVIDER_VERSION docker-compose -f docker-compose.yml -f $TEST up provider-verifier dependencies: - clone test .clean up: tags: - cdc stage: clean_up script: - cd cdc && docker-compose down --rmi local 

Just like in Consumer Pipeline, we have several jobs:

• Clone a test project
• We verify provider
• We clean up after ourselves

Summarize :

• Written a contract test in Python
• Set up a test environment in Docker containers
• Configured CI based on contract tests, i.e. commit to the consumer project will launch the CI pipeline ( on the consumer side : cloning the test environment -> launching Pact Broker -> testing the consumer -> launching provider verification -> clean up; on the provider side : cloning the test environment -> provider verification -> clean up ).
  Commit to the provider’s project initiates provider verification to ensure the provider’s compliance with the pact.

Thanks for attention.