Consumer-Driven Contracts as a way to develop a service

- The secret of supplier success is to provide consumers with a quality product ... oh, that is, service. Well, it is also important not to indulge in all serious ones with backward compatibility.
Walter White

From translator

What is it

This is a translation of an article describing Consumer-Driven Contracts (CDC).
The original is published on the Martin Fowler website by Jan Robinson .

Why is it

In the microservice architecture, dependencies between services are a source of problems. The CDC template helps to solve these problems in a way that suits both the developers of the service and its customers. Fowler refers to Consumer-Driven Contracts in his key article on microservice architecture: Microservices .

For whom is it

The article will be especially useful for teams developing services for several consumers within the same organization, and teams-consumers of such services.

About terms

• I did not translate Consumer-Driven Contracts into Russian - we do not translate in conversation Test-Driven Development. If necessary, you can use the option Consumer-oriented .
• Provider Contract and Consumer Contract are translated as supplier contract and consumer contract - they have sustainable use in Russian.
• Development or evolution of service (service evolution) - refinement, expansion of the functionality of the service.
• The service community is the supplier plus all the consumers of this service.

Preamble

This article discusses the problems that arise between developers and consumers of services, for example, when suppliers change part of their contract, in particular, the document schema. Two strategies for dealing with them are described:

1. Adding extension points.
2. "Sufficient" validation of received messages.

Both strategies help protect consumers from changes in the contract, but do not give the service provider an understanding of:

• how best to use these strategies;
• what needs to be done as the service develops.

The article describes the template Consumer-Driven Contracts , which allows suppliers to better understand their customers and focuses the development of the service on what consumers need.

Example of service evolution

To illustrate some of the problems we face when developing services, consider a simple ProductSearch that allows you to search our product catalog. The search result has the following structure:

Figure 1: Search result layout

An exemplary search results document is as follows:

<?xml version="1.0" encoding="utf-8"?> <Products xmlns="urn:example.com:productsearch:products"> <Product> <CatalogueID>101</CatalogueID> <Name>Widget</Name> <Price>10.99</Price> <Manufacturer>Company A</Manufacturer> <InStock>Yes</InStock> </Product> <Product> <CatalogueID>300</CatalogueID> <Name>Fooble</Name> <Price>2.00</Price> <Manufacturer>Company B</Manufacturer> <InStock>No</InStock> </Product> </Products> 

The ProductSearch service is currently used by two applications: an internal marketing application and an external reseller web application. Both clients use XSD to verify received documents before processing them. The internal application uses the fields CatalogueID, Name, Price and Manufacturer; external application - the fields CatalogueID, Name and Price. None of them use the InStock field: although it was considered for a marketing application, it was dropped early in the development.

One of the most common ways to develop a service is to add a field to a document for one or several consumers. Depending on how the client and server parts were implemented, even simple changes like this can have costly consequences for the business and its partners.

In our example, after the ProductSearch service has been in operation for some time, the second reseller wants to use it, but asks to add a Description field to each product. Because of the way customers are structured, this change has significant and costly implications for both the supplier and existing customers. The cost of each of them depends on how we make the change. There are at least two ways in which we can distribute the cost of change between members of the “service community”. First, we could change our original schema and require each consumer to update their copy of the schema in order to properly validate the search results. The cost of changing the system here is shared between the supplier, who, faced with such a request for change, will still make some changes; and consumers who are not interested in the updated functionality. On the other hand, we could add a second operation and scheme for a new consumer and keep the original operation and scheme for existing customers. All improvements are now on the side of the supplier, but the complexity of the service and the cost of its support are increasing.

Interlude: Tired SOA

The main advantages of using services are:

1. Increased organizational flexibility.
2. Reducing the total cost of implementing changes.

SOA increases flexibility by deploying business functions in dedicated, reusable services. These services are then used and orchestrated to perform core business processes. This reduces the cost of change by reducing dependencies between services, allowing them to quickly re-configure and tune in response to changes or unplanned events.

However, a business can fully realize these benefits only if the implementation of SOA allows services to evolve independently of each other. To increase independence, we create service contracts. Despite this, we are forced to refine consumers as often as the supplier, mainly because consumers are dependent on the specific version of the supplier’s contract. In the end, suppliers begin to be very careful about changing any element of the contract; in particular, because they have no idea how consumers implement this contract. In the worst case, consumers are tied to the supplier, describing the entire document scheme as part of their internal logic.

Contracts provide service independence; paradoxically, they can also bind suppliers and consumers in undesirable ways. Without thinking about the function and role of contracts that we implement in our SOA, we subject our services to “hidden” communication. The absence of any information about how service consumers implement the contract in code, as well as the absence of restrictions on implementation (for both the supplier and the consumer), collectively undermine the perceived benefits of SOA. In short, the company begins to tire of services.

Schema versioning

We can start researching contract and dependency issues that interfere with our ProductSearch service with versioning the schema. The WC3 Technical Architecture team (TAG) described a number of version control strategies that can help us develop schemas for our services in ways that reduce dependency problems. These strategies range from overly liberal none , which requires services not to distinguish between different versions of the scheme and accept all changes, to an extremely conservative big bang , which requires the service to issue an error if it receives an unexpected version of the message.

Both extremes create problems that do not provide business value and increase the total cost of ownership of the system. Explicit and implicit "no version" strategies lead to systems that are unpredictable in their interactions, poorly developed, and have a high cost of refinement. On the other hand, the big bang strategies create tightly coupled service landscapes in which changes to the scheme affect all suppliers and consumers, worsening accessibility, slowing development and reducing opportunities for profit.

The “service community” of our example effectively implements the big bang strategy. Given the cost to increase the value of the system for business, it is clear that suppliers and consumers will benefit from a more flexible version management strategy - what TAG calls a compatibility strategy. It provides forward and backward compatibility schemes. With the development of services, backward-compatible schemes allow consumers to accept instances of the old scheme: the provider created to handle backward-compatible new versions can, however, accept a request in the format of the old scheme. On the other hand, direct compatibility allows consumers of old schemes to process an instance of a new scheme. This is a key point for existing ProductSearch users: if the search results scheme was originally made with direct compatibility in mind, consumers could handle the responses of the new version without needing to be improved.

Expansion points

Backward and forward compatibility mapping is a well-understood design task, best expressed by the Must Ignore extensibility pattern (see David Orchard and Deira Obasanjo’s articles ). The Must Ignore pattern recommends that schemas include extension points that allow you to add elements to types and additional attributes for each element. The template also recommends that XML describe a processing model that defines what consumers do with extensions. The simplest model requires consumers to ignore elements that they do not recognize - hence the name of the template. The model may also require consumers to process elements with the “Must Understanding” flag, or give an error if they cannot understand them.

Here is our original outline of the search results document:

 <?xml version="1.0" encoding="utf-8"?> <xs:schema xmlns="urn:example.com:productsearch:products" xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" targetNamespace="urn:example.com:productsearch:products" id="Products"> <xs:element name="Products" type="Products" /> <xs:complexType name="Products"> <xs:sequence> <xs:element minOccurs="0" maxOccurs="unbounded" name="Product" type="Product" /> </xs:sequence> </xs:complexType> <xs:complexType name="Product"> <xs:sequence> <xs:element name="CatalogueID" type="xs:int" /> <xs:element name="Name" type="xs:string" /> <xs:element name="Price" type="xs:double" /> <xs:element name="Manufacturer" type="xs:string" /> <xs:element name="InStock" type="xs:string" /> </xs:sequence> </xs:complexType> </xs:schema> 

Let's now roll back in time, and from the very beginning let us specify a compatible, expandable scheme for our service:

 <?xml version="1.0" encoding="utf-8"?> <xs:schema xmlns="urn:example.com:productsearch:products" xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" targetNamespace="urn:example.com:productsearch:products" id="Products"> <xs:element name="Products" type="Products" /> <xs:complexType name="Products"> <xs:sequence> <xs:element minOccurs="0" maxOccurs="unbounded" name="Product" type="Product" /> </xs:sequence> </xs:complexType> <xs:complexType name="Product"> <xs:sequence> <xs:element name="CatalogueID" type="xs:int" /> <xs:element name="Name" type="xs:string" /> <xs:element name="Price" type="xs:double" /> <xs:element name="Manufacturer" type="xs:string" /> <xs:element name="InStock" type="xs:string" /> <xs:element minOccurs="0" maxOccurs="1" name="Extension" type="Extension" /> </xs:sequence> </xs:complexType> <xs:complexType name="Extension"> <xs:sequence> <xs:any minOccurs="1" maxOccurs="unbounded" namespace="##targetNamespace" processContents="lax" /> </xs:sequence> </xs:complexType> </xs:schema> 

This scheme includes an additional expansion element for each product. The extension element itself can contain one or more elements from the target namespace:

Figure 2: Extensible search result schema

Now, when we receive a request to add a product description, we can publish a new schema with an additional Description element, which the supplier inserts into the extension point. This allows the ProductSearch service to return results containing product descriptions, and consumers use the new scheme to validate the entire document. Consumers using the old scheme will not break, although they will not process the Description field. The new document with the search result looks like this:

 <?xml version="1.0" encoding="utf-8"?> <Products xmlns="urn:example.com:productsearch:products"> <Product> <CatalogueID>101</CatalogueID> <Name>Widget</Name> <Price>10.99</Price> <Manufacturer>Company A</Manufacturer> <InStock>Yes</InStock> <Extension> <Description>Our top of the range widget</Description> </Extension> </Product> <Product> <CatalogueID>300</CatalogueID> <Name>Fooble</Name> <Price>2.00</Price> <Manufacturer>Company B</Manufacturer> <InStock>No</InStock> <Extension> <Description>Our bargain fooble</Description> </Extension> </Product> </Products> 

The revised scheme looks like this:

 <?xml version="1.0" encoding="utf-8"?> <xs:schema xmlns="urn:example.com:productsearch:products" xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" targetNamespace="urn:example.com:productsearch:products" id="Products"> <xs:element name="Products" type="Products" /> <xs:complexType name="Products"> <xs:sequence> <xs:element minOccurs="0" maxOccurs="unbounded" name="Product" type="Product" /> </xs:sequence> </xs:complexType> <xs:complexType name="Product"> <xs:sequence> <xs:element name="CatalogueID" type="xs:int" /> <xs:element name="Name" type="xs:string" /> <xs:element name="Price" type="xs:double" /> <xs:element name="Manufacturer" type="xs:string" /> <xs:element name="InStock" type="xs:string" /> <xs:element minOccurs="0" maxOccurs="1" name="Extension" type="Extension" /> </xs:sequence> </xs:complexType> <xs:complexType name="Extension"> <xs:sequence> <xs:any minOccurs="1" maxOccurs="unbounded" namespace="##targetNamespace" processContents="lax" /> </xs:sequence> </xs:complexType> <xs:element name="Description" type="xs:string" /> </xs:schema> 

Please note that the first version of the extensible scheme is compatible with the second, and the second is backward compatible with the first. However, this flexibility comes at the cost of increased complexity. Expandable schemas allow us to make unforeseen changes, but they provide opportunities that may never be needed; while we lose:

• expressiveness coming from simple design
• clear presentation of data by entering the meta-information elements of the container.

Further we will not discuss expandable schemas. Suffice it to say that expansion points allow us to make directly and backward compatible changes in charts and documents without affecting service providers and consumers. However, the expansion of the scheme does not help us manage the evolution of the system when we need to make a change that violates the contract.

Breaking compatibility

- Yes, our team violated backward compatibility in the latest release! Just could not resist a small optimization of the protocol ... Well, do not be offended, baby!
Carla Borin

Our ProductSearch service includes a field in the search results indicating the availability of this product. The service fills this field using an expensive call to an ancient inventory system - a dependency that is expensive to maintain. The vendor would like to remove this dependency, clean the design and improve the overall performance of the system. And preferably, without affecting consumers. By communicating with consumers, the supplier team finds out that none of the consumer applications actually do anything with this field; that is, being expensive, it is redundant.

Unfortunately, in the current situation, if we remove the InStock field from our extensible scheme, we break existing customers. To fix the supplier, we need to fix the entire system: when we remove the functionality from the supplier and publish a new contract, each consumer application will need to be reinstalled with a new scheme, and the interaction between services will be thoroughly tested. The ProductSearch service in this regard cannot develop independently of consumers: the supplier and consumers must "jump at the same time."

Our service community is disappointed in its evolution, because every consumer has a “hidden” dependency, which reflects the entire supplier contract in the internal logic of the consumer. Consumers, using XSD validation and, to a lesser extent, static binding to the document scheme in the code, implicitly accept the entire supplier contract, regardless of their intention to use only a part.

David Orchard gives some clues as to how we could avoid this problem, referring to the principle of reliability : "The implementation must be conservative in its behavior and liberal in what it takes from others." We can reinforce this principle in the context of service development, stating that message recipients must perform “sufficient” verification: that is, they must process only the data they use, and must perform explicitly limited or targeted verification of the data they receive - as opposed to implicitly unlimited validation “All or nothing” inherent in XSD processing.

Schematron

One of the ways we can customize consumer-side validation is to use masks or regular expressions for paths to document elements, possibly using the tree structure validation language, such as Schematron . Using Schematron, each user of the ProductSearch service can specify what he expects to find in the search results:

 <?xml version="1.0" encoding="utf-8" ?> <schema xmlns="http://www.ascc.net/xml/schematron"> <title>ProductSearch</title> <ns uri="urn:example.com:productsearch:products" prefix="p"/> <pattern name="Validate search results"> <rule context="*//p:Product"> <assert test="p:CatalogueID">Must contain CatalogueID node</assert> <assert test="p:Name">Must contain Name node</assert> <assert test="p:Price">Must contain Price node</assert> </rule> </pattern> 

Schematron implementations typically transform a Schematron schema, such as this, into an XSLT transform that the message recipient can apply to the document for validation.

Please note that this scheme does not contain assumptions about elements in the original document that are not needed by the consumer application. Thus, validation of only the required elements is explicitly described. Changes to the schema of the original document will not be rejected during validation if they do not violate the explicit expectations described in the Schematron schema, even if it is the removal of previously required elements.

Here is a relatively easy solution to our problems with contract and dependency, and this does not require us to add implicit meta-information elements to the document. So let's roll back again in time and restore the simple scheme described at the beginning of the article. But this time we will also insist that consumers are free in their behavior and validate and process only the information that they need (using Schematron schemes, and not XSD to check received messages). , , , . , , InStock , — , .

ProductSearch :

 <?xml version="1.0" encoding="utf-8"?> <xs:schema xmlns="urn:example.com:productsearch:products" xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" targetNamespace="urn:example.com:productsearch:products" id="Products"> <xs:element name="Products" type="Products" /> <xs:complexType name="Products"> <xs:sequence> <xs:element minOccurs="0" maxOccurs="unbounded" name="Product" type="Product" /> </xs:sequence> </xs:complexType> <xs:complexType name="Product"> <xs:sequence> <xs:element name="CatalogueID" type="xs:int" /> <xs:element name="Name" type="xs:string" /> <xs:element name="Price" type="xs:double" /> <xs:element name="Manufacturer" type="xs:string" /> <xs:element name="Description" type="xs:string" /> </xs:sequence> </xs:complexType> </xs:schema> 

Consumer-Driven Contracts

Schematron , . , .

, — , , . .

, . , , -. :

• . . , , ; , , - , , , .
• Interfaces , . - , . , . RPC- , , - . , , , , .
• . , , "-" "fire-and-forget". , . , , . , «» , , , "" . , " ", , . , , .
• . , , , , . , . Web- WS-Policy , WS-SecurityPolicy, " ", .
• . , , , , . .

, , , , . , : , . , - , , , , . , , WS-Basic, .

, . , , , .

, ? , , (, ) , . , , , , - . .

:

• . , , , .
• . .
• . / (. "Validity of Data in Bounded Space and Time" Data on the Outside vs. Data on the Inside ). , .

, — , — . Schematron . , , . , , , , "" , . , , , - , .

, /, , ( ). , , , .

, , , , , . , , .

3:

:

• . . .
• . , . - , - . , , . , . , ; .
• . , / .

Consumer-Driven Contracts

. , , , . , , . , . , Consumer-Driven Contracts .

---, . , , . ; , , .

Consumer-Driven Contracts :

• . , . , / , / .
• . , , .
• . . , Consumer-Driven Contract , . , , .

, :

Implementation

Consumer-Driven Contracts , Consumer-Driven Contracts. , , , / .

. , -. unit-, , , . Schematron WS-Policy, .

, , . Consumer-Driven Contracts , , , , . / , . , , - .

Benefits

Consumer-Driven Contracts , . -, . , . Consumer-driven contracts , , — , , . "" , -, . — — , .

, , . Consumer-Driven Contracts . Consumer-driven contracts , , . , , , , . , , .

Consumer-Driven Contracts , , Consumer-Driven Contracts , . , , CDC.

Consumer-Driven Contracts , , : , , , . , , , . .

, , Consumer-Driven Contracts, , . , — : , . , , , . , , , , . , — , deprecated , .

Consumer-Driven Contracts . , , . , , «» , .

, Consumer-Driven Contracts . , - — - — , , , WS-Agreement WSLA, SLA. , ; . — — , " " .

, : , . , , , , .

Links

1. Ian Robinson. Consumer-Driven Contracts ( )
2. Martin Fowler. Microservices , Decentralized Governance
3. . , " "
   
4. .