Hello! My name is Dmitry Pavlov, at Align Technology, my colleagues and I are developing a Web API for interfacing internal systems and integrating our company with third-party vendors. I would like to tell you about the ideas of creating an API for the web, or rather the RESTful API, in this article.

In recent years, the topic of Web API has become very popular, many companies are engaged in the creation of similar interfaces, both open and for internal use. In the description of the Web API, one can almost always come across the acronym REST, but what does this term mean and is it used correctly?

REST or not REST?

Most developers, especially in Russia, understand REST as a program interface that works using the HTTP [S] protocol, subject to the following properties:

• The server does not store the state of the client: no sessions, all that is required to fulfill the request, the client sends with the request itself.

  
  

• Human readable URLs in which resources are identified separately. No more /index.php?productId=1 , instead use /products/1

  
  

• More extensive use of HTTP methods: not limited to GET and POST, add PUT and DELETE. In some APIs, PATCH can also be found.

  
  
• JSON is used as the data transfer format.

The algorithm by which this API is used is usually standard. First you need to go to the site with the documentation, find a page with a list of URL-templates for access to resources. Usually it looks like this:

    API " " --- /recipes/cookies -   , GET   URL     [ { "name" : "   ", "rating" : 5, "shortDescription" : "...." } ] POST   URL     .      json   { "name" : " ", "shortDescription" : "...." ...... } --- /recipes/cookies/:name -     ${name} { "name" : "   ", "rating" : 5, "shortDescription" : "....", "description" : "...." "ingredients" : [ { "name" : "", ..... }, { "name" : "", ..... }, { "name" : "", ..... } ], "cookingSteps" : [ .... ] } //    API   HTTP   URL  

and having studied it to fulfill requests to resources (which is usually expressed in writing a client, which, according to specified URL formats, substitutes the parameters and processes the answers).

There are plenty of examples of such APIs on the network, until recently, Yandex has many APIs ( one , two ) declared as REST worked according to this scheme.

If we turn to primary sources, i.e. to Roy Fielding’s thesis (which is often referred to, but read less often), we’ll see that APIs created in this way cannot be called REST, because they violate some of the principles described in the thesis, the most important of which is the use of hypermedia as state control tools (Hypermedia As The Engine Of Application State, HATEOAS), indirectly affecting the issues of self-descriptive messages.

Hypermedia in messages

The essence of HATEOAS is the approach to describing the resources of our API. Instead of simply listing a set of resources, with a list of all possible operations that a client can invoke, guided by some internal logic, we perform an inversion of control - now the server is responsible for the state of the resource and it dictates to the client what operations can be performed on the resource at the moment. This information must be present in the actual representation of the resource that the client receives. Thus, the representation of a resource describes itself sufficiently so that the client understands what can be done with it.

Applying this approach usually means that the client knows a certain final set of "entry points" (you can think of them as analogous to the start pages on websites), from which he begins his interaction with the API using the information provided in the resource view to navigate to other resources and perform actions .

Hyperlinks are used to achieve this task:

• All resources are addressable by reference, with links to other resources present within the messages themselves for their communication with each other. The client, instead of targeting the URI format, is guided by identifiers by which it selects links that are located directly in the resource view. If earlier we indicated in the documentation that we need to take some ID and, on its basis, build a special URL, thereby making URL sets a part of our API, then now the details of URL formation are just the implementation features of the server and the client do not care. After all, it is important for the client to access the resource, and not to generate URLs from templates from the documentation.
• Available operations on a resource are also representable as links.

The absence of a link to both related resources and available actions means that this operation is not available in the current state of the resource.

An example of the processing of the Cookie Recipes API in Hypermedia View

Returning to the example of our API about the catalog of recipes for cookies, we transform it into a Hypermedia-view.

As you remember, we had a list of recipes and a resource detailing a specific recipe with a list of ingredients and cooking steps. Here is what they will look like using the hypermedia approach:

 //  { "links": { "self" "/recipes/cookies" } "items": [ { "name": "   ", "rating": 5, "shortDescription": "...." "links": { "self": "/recipes/cookies/   " } } ] } //  { "links": { "self": "/recipes/cookies/   " } "name": "   ", "rating": 5, "shortDescription": "....", "description": "...." "ingredients": [ { "name": "", ..... }, { "name": "", ..... }, { "name": "", ..... } ], "cookingSteps": [ .... ] } 

A significant difference from the original version is the appearance of the links object inside each resource. The keys of this object are relations (they are identifiers), and the values ​​are references. As a result, our resources do not require additional information (outside the resource itself) about how to go from the recipe catalog to a detailed description, the link is embedded in the resource representation.

This approach allows you to easily extend the functionality of our API. Suppose that for each recipe we want to provide the client with a set of recommendations that can be represented as a list of recipes. This is very easy to do, just add a new key to our links object:

 "links" : { "self" : "/recipes/cookies/   ", "http://acme.com/recipes/rels/you-can-also-like" : "/recipes/cookies?related_to=+++" } 

Likewise, it’s not at all difficult to add the identification of ingredients as separate resources if the need arises.

The content of the URI does not matter, because now the element of the API is a relation, and we can change the link to /recipes/related-to/ or /recipes/234892skfj45sdlkfjdsa12 without any changes on the client

Hypermedia in the service of change

Hypermedia is used not only to navigate, but also to perform actions, it is enough only to determine that some relation are responsible for performing certain operations on resources, and also to identify the semantics and details of these operations.

For clarity, consider an example with our API, adding hypermedia controls to create a new recipe.

 //  { "links" : { "self" : "/recipes/cookies", "http://acme.com/recipes/rels/add-recipe" : "/recipes/cookies" } "items" : [ ..... ] } 

We just added a link with a special relation. The basic rule is that the client ignores unknown relationships: the "old" customers who do not know how to add a new recipe will work as before, and for those who support the creation, this will be a signal that it is possible to add a new recipe by sending a request on the URI that is specified in http://acme.com/recipes/rels/add-recipe .

This approach allows us not to describe a static set of operations and conditions for their execution in the documentation, but directly to the server to control which operations a client can perform on a resource at a given time, and which cannot. Adding new actions is also not difficult: we simply declare a new relation, and begin to include it in the resource representation that the server forms.

Of course, providing links does not remove responsibility from the server for correctly handling HTTP methods and observing their semantics :).

What about relation?

At this moment, you probably have a question: what is the point of plotting all this, if the client still needs to understand the meaning of relationals for effective work? For him, the documentation should be available.

In fact, to work effectively, the client really needs to understand what each relationship means. The main idea behind the replacement of the interpretation of URIs for working with relationes is the greater durability of the latter. The URI is an implementation detail and can change over time or from server to server. Relation is a semantic description of the connection and is not tied to the details of storage.

Suppose I want to make a compatible API for storing recipes, but because of the storage characteristics I want to identify each recipe by its UUID, and not by its name. In the case of the original API, this is not possible, and for the hypermedia API it is completely transparent to the client.

As a result, it becomes possible to create more universal clients that are less susceptible to changes in the event of modifications on the server.

Hypermedia types or why application / json does not suit us

Deciding to take advantage of the Hypermedia approach, we modified our API in the manner indicated above, and now we have resources linked to each other by reference. At first glance, it may seem that our API is fine, but before declaring that we have the Hypermedia API, we will look at the Content-Type header that we return in the responses. If there is application/json or even text/plain , then we still have to work hard.

Resources through the eyes of the machine

Looking at the resources we got, a person immediately allocates links, which creates the impression of the correct format of our message. We conclude this by analyzing the content of the message, then the standard prescribes to look at the Content-Type response header.

Consider the following server response:

 200 OK Content-Type: text/plain <?xml version="1.0"?> <hello>world</hello> 

It is obvious to us that the answer contains an xml-document, but the Content-Type prescribes to perceive the content as plain text, so that it looks like an xml-document can be just a coincidence or a special case. That is why the true Content-Type is so important.

Let's figure out what the application/json n't right for our task? The fact is that the standard describing this type does not provide any place or mechanism for defining the links in it. And even if the message generated by us contains links, the machine cannot distinguish them from the line containing the text in the format resembling a link. We need to unambiguously determine where the link is in the message, and where not, so we need a different type.

Vendor specific types

One way to solve the Content-Type correctness problem is to use your own. In the documentation we will clearly indicate where we have links in the message. If the client received a response from the server with our personal Content-Type, he will not need to dynamically guess what the link is and what not, unless of course he understands our Content-Type. It is worth noting that often the documentation describing the type contains not only the details of the format itself (that is, where the links are located and where the properties are located), but also other information:

• semantic description of properties, i.e. what they mean in terms of business logic;
• details of the client's interaction with the server, such as the HTTP methods required to send the request.

These types are called vendor specific, as they are often created for a specific task and specific organization. They do not need to be registered with IANA. It is recommended to give them the name of the type application/vnd.${vendor}+${base_format} , where ${vendor} is the company's inverted domain, ${base_format} is the type we took as the basis. If the company has an acme.com domain and we use json to represent our resources, then for our recipe API the type name will look like application/vnd.com.acme.recipes+json .

Hypermedia general purpose types

At first glance, vendor specific types solve the problem with links, but they have their own problems:

• types are not compatible with each other, so customers have to support many different implementations, if they interact with more than one API, i.e. support of a separate library is required to parse the format of each type, select properties, references and other things in it;
• Creating a vendor specific type for each task leads to a very noticeable increase in their total number.

Alternatively, I did not keep myself waiting for the new approach, which brought the types of general purpose. If you think about it, all we need from the message format is a specification that answers the questions:

• how to find the properties of our resources,
• how to find hypermedia controls inside the resource.

It is this problem that is being solved: the general-purpose type does not try to adapt to a specific domain domain, they can describe most of the resources we are dealing with.

An important feature of all types of general purpose is that they do not set the task of a semantic description of a document, i.e. they do not say what kind of resource this is - a description of the recipe or a blog comment is not their task. They are responsible more for the details of the format, leaving the semantic specification beyond the scope. It is assumed that the semantics will be contained in the so-called profile - a separate document describing the semantics of properties and relations (relations).

At the moment there are already quite a large number of such formats, so we list only some of them:

• application/hal+json - one of the first to emerge and the most popular format in our days;
• application/vnd.siren+json ;
• application/mason+json .

In the description of all such formats you will find how and where to place the properties of the resource, in what form to make links to other resources.

They differ in format and capabilities that are contained in the type itself.

The difference in formats or principled creators

Most general purpose types are distinguished by minor details, such as how to format links. So, in the HAL, the links look like this:

 "_links" : { "self" : .... "relToResource": ..... } 

Whereas Siren presents them like this:

 "links" : [ {"rel" : ["self"], "href" : "...."}, {"rel" : ["relToResource"], "href" : "...."} ] 

The main difference here is in the representation of relation values. The creator of HAL wanted to make the format more concise, while the creator of Siren did it more completely: the relation at the link can be really complicated (therefore, in Siren it is an array of values), but this is not always used (therefore, in HAL it is a scalar and also the key in the object).

Such different views led to the creation of different formats, but could not agree on one format.

Difference in possibilities

We will not list here all the differences in the formats, we denote only the main ones, by the example of the types already mentioned:

• In HAL, there is no concept of actions as a separate entity, only the links and the method that should be used to send a request is also not included in the message. In Siren and Mason, there are tools for describing forms: the message contains the parameters that are required for input by the client, the method to which all this needs to be sent and the Content-Type.
• HAL has a separate concept of a search form - a safe, idempotent action for obtaining information from the server. Siren and Mason, as mentioned above, has the ability to describe any action.
• HAL and Siren do not contain details on the error description in the specification, it remains at the discretion of the user of the format ( application/vnd.error+json can be used), whereas in Mason this aspect is included in the format.

Generic vs vendor specific

What option is preferable: a specially created type or one of the existing options? As is usually the case with such questions, there is no definite answer to it, it all depends on the circumstances of use, so we will try to highlight the advantages and disadvantages of each of them.

One of the main advantages of a general purpose hypermedia-type is the time savings for you and your API customers. That is why it is achieved:

• The message format is already invented for you. You can focus on solving your problem by creating an API in the already established restrictions of the chosen format.
• To create and parse messages, libraries have already been written for different programming languages. Agree, it is convenient to connect a ready-made library and work with high-level concepts like link, action, object properties, and not with details of the paths in JSON.
• For many generic types, there are so-called browsers. These are simple clients who understand elements of a certain type and dynamically form a web interface, allowing you to immediately demonstrate the created API without spending a minute to create a client.
• Your clients will only need to study the documentation for your API, i.e. they can only focus on the business sense, without wasting time reading the details of the format.

At the same time, some of the advantages of this approach may look like disadvantages for someone. Due to the fact that the type is not tied up under any domain area and task, the representation of resources is more “bloated” compared to the special type that we could create.

As a result, for most tasks, we can recommend to use one of the existing general-purpose Hypermedia-formats and choose vendor-formats in complex or specific cases (unless of course you aim at vendor lock-in).

How much do I need all this?

The described approach is not another silver bullet, designed to solve all the problems in the development of the API.

It can be noted that the concept of entry points can lead to an increase in the number of requests to “get” to the desired resource and that, including links, we make the message more voluminous compared to bare data.

These shortcomings can be objected that these problems are solved by a well-thought-out resource structure (who prevents to perform resource search operations at the entry point for fast navigation?), Caching, which Fielding also noted as an important component of this architectural approach, and the banal compression on web servers.

The main advantage of the REST approach (here I mean the full REST) ​​is in the flexibility and extensibility that it provides, allowing us to add new features or simply change the organization of resources on our server without disrupting existing clients.

Even if you decide not to use hypermedia in your API, now you know that without it, REST is not REST, but just a Web API. This does not make the API bad or good, I just state a fact. , API API, , :).

• API — hypermedia API, HAL.
• Github — ( JSON), hypermedia.
• Paypal .
• Foxycart — API — HAL Siren.
  ( , opensource)