RAML routing in the Play Framework

The Play framework is a very flexible tool, but there is little information on how to change the format of the route file on the Internet. I will talk about how you can replace the standard route description language based on the route file with a description in RAML format. And for this we will have to create our SBT-plugin .

In a nutshell about RAML (RESTful API Modeling Language). As quite rightly stated on the main page of the project, this language greatly simplifies working with the application API throughout its life cycle. It is concise, easily reused, and most valuablely, it is equally easy to read by machine and man. That is, it is possible to embody the documentation as a code approach, when one artifact (RAML script) becomes an entry point for all participants in the development process - analysts, testers and programmers.

Formulation of the problem

Our team is working on a large and complex online banking system . Special attention is paid to interface documentation and tests. Once I wondered if it was possible to combine tests, documentation and code generation. It turned out that to a certain extent it is possible. But first, a few words about why we needed it.
')
In a large project, documentation of interfaces is of particular importance. Analysts, developers of services and client applications should have a description of the API of the system on which they are working. Analysts have important documentation in an easy-to-read form, programmers ultimately describe the interface in code, with different commands writing code in different languages. Information is repeatedly duplicated, which makes it difficult to synchronize. When there are many teams, it is almost impossible to manually ensure that the various API formats match each other. Therefore, we have approached the solution of this problem thoroughly.

To begin, we chose a single API description format for all commands. They became RAML. Next, we would need to ensure (as far as possible) that our services match the description, and the client applications work with the described services. To do this, we use testing tools, which I will discuss in another article. And the final step was the implementation of code generation, which creates code for us based on data from RAML. Today we will talk about the code generation tools used in the server project.

The documentation usually contains information about the REST endpoints, a description of the parameters and the request body, HTTP codes, and a description of the responses. Moreover, for all the above elements, examples are often indicated. This information is quite enough to test the endpoint's performance - you just need to take an example request for it and send it to the server. If the end point has parameters, then its values ​​should also be taken from the examples. Compare the response with an example response or validate its JSON scheme based on the documentation. For the sample answers to match the server's responses, the server must work with the correct data in the database. Thus, if there is a database with test data and the API documentation of the service with a description of the answers and examples of requests, we can provide simple testing of the performance of our service. About our documentation, testing system and database, I have now mentioned to complete the picture, and we will definitely talk about them another time. Here I will talk about how, on the basis of such documentation, to generate as much useful server code as possible.

Our server is written in Play 2.5 and provides the REST API to its clients. The data exchange format is JSON. The standard description of the API in the Play framework is in the conf / route file. The syntax of this description is simple and is limited to the description of endpoint names and their parameters, as well as the binding of endpoints to the controller methods in the routes file. Our goal will be to replace the standard syntax with a description in RAML format. For this we need:

1. Understand how routing is arranged in Play and how route files are processed.
2. Replace the standard routing mechanism with our mechanism using RAML.
3. Look at the result and draw conclusions :)

So let's go in order.

Routing in the Play framework

Play framework is designed for use with two languages ​​- Scala and Java. Therefore, the authors of the framework did not use DSL on the basis of a particular language to describe the routes, but wrote their own language and compiler for it. Next, I will talk about Scala, but everything said is true for Java. The play application is built using SBT . During the project build, route files are compiled into files on Scala or Java, and then the result of the compilation is used during the build. The SBT plugin com.typesafe.play.sbt-plugin is responsible for processing the route file. Let's see how it works. But first, a few words about SBT.

The basic concept of SBT is key. Keys are of two types: TaskKey and SettingsKey. The first type is used to store functions. Each call to this key will call this function. The second type of key stores a constant and is calculated once. Compile is TaskKey, in the process of execution it calls another TaskKey, sourceGenerators, for code generation and creating source files. The SBT-plugin itself adds the function of processing the route file to sourceGenerators.

Usually, two main artifacts are created based on route — the file target / scala-2.11 / routes / main / router / Routes.scala and target / scala-2.11 / routes / main / controllers / ReverseRoutes.scala. The Routes class is used to route incoming requests. ReverseRoutes is used to call the endpoints from the controller code and view by the endpoint name. Let's illustrate the above with an example.

conf / routes

GET /test/:strParam   @controllers.HomeController.index(strParam) 

Here we declare the parameterized endpoint and map it to the HomeController.index method. Compiling this file results in the following Scala code:

target / scala-2.11 / routes / main / router / Routes.scala

 class Routes(   override val errorHandler: play.api.http.HttpErrorHandler,   HomeController_0: javax.inject.Provider[controllers.HomeController],   val prefix: String ) extends GeneratedRouter {   ...   private[this] lazy val controllers_HomeController_index0_route = Route("GET",       PathPattern(List(           StaticPart(this.prefix),           StaticPart(this.defaultPrefix),           StaticPart("test/"),           DynamicPart("strParam", """[^/]+""",true)       ))       )   private[this] lazy val controllers_HomeController_index0_invoker = createInvoker(       HomeController_0.get.index(fakeValue[String]),HandlerDef(           this.getClass.getClassLoader,           "router","controllers.HomeController","index",           Seq(classOf[String]),"GET","""""",this.prefix + """test/""" + "$" + """strParam<[^/]+>""")       )   def routes: PartialFunction[RequestHeader, Handler] = {       case controllers_HomeController_index0_route(params) =>           call(params.fromPath[String]("strParam", None)) { (strParam) =>               controllers_HomeController_index0_invoker.call(HomeController_0.get.index(strParam))           }       } } 

This class deals with the routing of incoming requests. As arguments, it is passed the link to the controller (more precisely, the injector, but this is not significant) and the prefix URL path, which is configured in the configuration file. Further in the class, the “mask” of routing controllers_HomeController_index0_route is declared. The mask consists of an HTTP verb and a route pattern. The latter consists of parts, each corresponding to the URL element of the path. StaticPart defines a mask for the unchanged part of the path, DynamicPart sets a template for the URL parameter. Each incoming request falls into the routes function, where it is associated with the available masks (in our case it is one). If no match is found, the client will receive a 404 error, otherwise the corresponding handler will be called. In our example, one handler is controllers_HomeController_index0_invoker . It is the responsibility of the handler to call the controller method with the necessary set of parameters and transform the results of this call.

target / scala-2.11 / routes / main / controllers / ReverseRoutes.scala

 package controllers {   class ReverseHomeController(_prefix: => String) {       ...        def index(strParam:String): Call = {           import ReverseRouteContext.empty           Call("GET", _prefix + { _defaultPrefix } +               "test/" +               implicitly[PathBindable[String]].unbind("strParam", dynamicString(strParam)))       }   } } 

This code allows us to access the endpoint through an appropriate function, for example, in view.

So, to change the format of the route description we just need to write our own Routes file generator. We do not need ReverseRoutes, since our service gives JSON and it has no view. In order for our generator to work, you need to turn it on. You can copy the source of the generator in each project where it is needed, and then connect it to build.sbt. But it would be more correct to arrange the generator in the form of a plug-in to SBT.

SBT plugin

SBT plug-ins are fully documented. Here I will mention the main, in my opinion, points. A plugin is a set of additional functionality. Usually plugins add new keys to the project and extend existing ones. For example, we will need to expand the sourceGenerators key. Some plugins may depend on others, for example, we could use the com.typesafe.play.sbt-plugin plugin as the basis and change only what we need. In other words, our plugin depends on com.typesafe.play.sbt-plugin. In order for SBT to automatically connect all dependencies for our plugin, it must be AutoPlugin . And finally: due to compatibility issues, plugins are written on Scala 2.10.

So, we need to generate Routes.scala based on the RAML file. Let this file be called conf / api.raml. In order for documentation in RAML format to be used for routing, it is necessary in some way to specify in it for each end point a controller method that must be called when a request is received. RAML 0.8, which we will use, does not have the means to indicate such information, so you have to do a dirty hack (RAML 1.0 solves this problem with annotations, but at the time of this writing, this version of the standard is still cheese). Add information about the controller method being called to the first description line for each endpoint. Our example in RAML format from the previous chapter will look like this:

 /test/{strParam}:   uriParameters:       strParam:       description: simple parameter       type: string       required: true       example: "some value"   get:       description: |           @controllers.HomeController.index(strParam)       responses:           200:               body:                   application/json:                       schema: !include ./schemas/statements/operations.json                       example: !include ./examples/statements/operations.json 

I will not dwell on the details of RAML parsing, I will just say that you can use the parser from raml.org. As a result of the parsing, we get a list of rules - one for each endpoint. The rule is defined by the following class:

 case class Rule(verb: HttpVerb, path: PathPattern, call: HandlerCall, comments: List[Comment] = List()) 

The names and types of fields speak for themselves. Now for each rule we can create our own mask, handler and case element in the route function in the Routes.scala file. To solve this problem, you can manually generate a line with the code Routes.scala based on the list of rules, or use macros. But it is better to choose an intermediate version, which was preferred by the developers of Play - to use the template engine. PB Play uses the twirl templating engine , and we also use it. Here is a template from our plugin that generates the route function:

 def routes: PartialFunction[RequestHeader, Handler] = @ob   @if(rules.isEmpty) {   Map.empty   } else {@for((dep, index) <- rules.zipWithIndex){@dep.rule match {   case route @ Rule(_, _, _, _) => {       case @(routeIdentifier(route, index))(params) =>           call@(routeBinding(route)) @ob @tupleNames(route)               @paramsChecker(route) @(invokerIdentifier(route, index))                   .call(@injectedControllerMethodCall(route, dep.ident, x => safeKeyword(x.name)))@cb       }   }}}@cb 

It looks somewhat confusing, but if you look closely, everything becomes clear. Expressions starting with @ are directives and variables of the template engine. The @ob and @cb variables will be expanded to {and}, respectively. And, for example, @ (routeIdentifier (route, index)) will unfold according to the following rule:

 def routeIdentifier(route: Rule, index: Int): String = route.call.packageName.replace(".", "_") +   "_" + route.call.controller.replace(".", "_") +   "_" + route.call.method + index + "_route" 

Now it’s clear how to write code that creates Routes.scala based on RAML, and understand how to connect it to the assembly. Sources of the finished plug-in are on Github .

Future plans

The plugin allowed us to use the documentation as the source code for the server. But code generation does not use all the available information from the RAML file. Namely, we do not use information about the type of request and response. In Play, request parsing and response generation takes place in controller methods, but we want to generate this code automatically. In addition, we have plans to use RAML version 1.0.

That's all for today, thank you for your attention!