Kotlin DSL: Theory and Practice

Sql, RegExp, Gradle - what unites them? These are all examples of using problem-oriented languages ​​or domain-specific language. Each such language solves its own narrowly focused task, for example, requesting data from a database, searching for matches in the text, or a description of the application building process. The Kotlin language provides a large number of opportunities to create your own problem-oriented language. In the course of the article, we will figure out what tools are in the programmer’s arsenal, and implement DSL for the proposed subject area.

I will explain the entire syntax presented in the article as simply as possible, however, the material is intended for practicing engineers who view Kotlin as a language for building problem-oriented languages. At the end of the article will be given the shortcomings that need to be prepared. The code used in the article is relevant for Kotlin version 1.1.4-3 and is available on GitHub.

What is DSL?

Programming languages ​​can be divided into 2 types: universal languages ​​(general-purpose programming language) and domain-specific language. Popular DSL examples are SQL, regular expressions, build.gradle. A language reduces the amount of functionality provided, but at the same time it is able to effectively solve a specific problem. This is a way to describe the program not in an imperative style (how to get the result), but in a declarative or close to declarative (describe the current problem), in which case the solution to the problem will be obtained on the basis of the specified information.

Suppose you have a standard execution process, which can sometimes change, be refined, but in general you want to use it with different data and format of the result. Creating a DSL, you make a flexible tool for solving various problems from one subject area, and the end user of your DSL does not think about how the solution will be obtained. This is some API, masterfully using which you can greatly simplify your life and long-term system support.

In the article I reviewed the construction of the "internal" DSL in the Kotlin language. This kind of problem-oriented languages ​​is implemented on the basis of the universal language syntax. You can read more about this here .

Application area

One of the best ways to apply and demonstrate Kotlin DSL, in my opinion, is tests.

Suppose you come from the world of Java. How often did you have to describe standard instances of entities for a rather large data model again and again? Is it possible that for this you used some builders or, even worse, special utility classes that filled the default values ​​under the hood? How many overloaded methods do you have? How often do you need “quite a bit” to deviate from the default values ​​and how much work does it have to do now? If nothing but a negative, you do not cause these questions, then you are reading the correct article.

For a long time on our project devoted to the educational sphere, we, in the same way, with the help of builders and utility classes, covered with tests one of the most important modules of the system - a module for building a training schedule. This approach was replaced by the Kotlin and DSL language for the formation of various options for the use of a planning and verification system. Below you will see examples of how we took advantage of the possibilities of the language and turned the development of tests on the planning subsystem from torture to pleasure.

In the course of this article, we will look into the DSL design for testing a small demonstration system of class planning between a student and a teacher.

Main features

Let's list the main advantages of Kotlin, which allow you to write clean enough in this language and are available for building your own DSL. Below is a table with the main language syntax enhancements you should use. Review this list carefully. If most of the designs are not familiar to you, then it is advisable to read sequentially. However, if you are not familiar with one or two points, you can go directly to them. If everything is familiar to you here, then you can go on to review the shortcomings of using DSL at the end of the article. If you want to add to this list, then please write your options in the comments.

Found something new for yourself? Then we continue.

In the table, delegated properties are intentionally omitted, since, in my opinion, they are useless for building DSL in the form that we will consider. Thanks to these features, you can write code cleaner, get rid of a lot of “noisy” syntax and at the same time make development even more enjoyable (“where is more pleasant?”, You ask). I liked the comparison from the book Kotlin in Action, in natural languages, for example, in English, sentences are built from words and grammatical rules govern how to combine words with each other. Similarly in DSL, a single operation can be made up of several method calls, and type checking will ensure that the design makes sense. Naturally, the order of calls may not always be obvious, but this remains on the conscience of the designer DSL.

It is important to understand that in this article we will consider the “internal DSL”, i.e. Problem-oriented language is based on the universal language - Kotlin.

Example of the final result

Before we begin to build our problem-oriented language, I want to demonstrate the result of what you can build after reading the article. You can find all the code on the GitHub repository by reference . Below is a DSL for testing teacher search for students in their subjects of interest. In this example, there is a fixed time grid, and we check that the classes are placed in the teacher’s and student’s plans at the same time.

 schedule { data { startFrom("08:00") subjects("Russian", "Literature", "Algebra", "Geometry") student { name = "Ivanov" subjectIndexes(0, 2) } student { name = "Petrov" subjectIndexes(1, 3) } teacher { subjectIndexes(0, 1) availability { monday("08:00") wednesday("09:00", "16:00") } } teacher { subjectIndexes(2, 3) availability { thursday("08:00") + sameDay("11:00") + sameDay("14:00") } } // data { } doesn't be compiled here because there is scope control with // @DataContextMarker } assertions { for ((day, lesson, student, teacher) in scheduledEvents) { val teacherSchedule: Schedule = teacher.schedule teacherSchedule[day, lesson] shouldNotEqual null teacherSchedule[day, lesson]!!.student shouldEqual student val studentSchedule = student.schedule studentSchedule[day, lesson] shouldNotEqual null studentSchedule[day, lesson]!!.teacher shouldEqual teacher } } } 

Instruments

A complete list of tools for building DSL, was given above. Each of them was used in the example and, by examining the code by reference , you can study the construction of such constructions. We will often return to this example to demonstrate various tools. It is important to note that the decisions to build DSL are demonstrative, although you can repeat what you see in your own project, this does not mean that the presented option is the only correct one. Below we take a detailed look at each tool.

Some features of the language are especially good in combination with others and the first tool in this list is lambda outside the brackets.

Lambda outside brackets

Documentation

Lambda expressions or lambdas are blocks of code that can be passed to functions, saved, or called. In the Kotlin language, the lambda type is denoted as follows ( ) -> . Following this rule, the most primitive type of lambda is () -> Unit , where Unit is an analogue of Void with one exception. At the end of the lambda or function, we do not
must write the "return ..." construct. Due to this, we always have a return type, just in Kotlin this happens implicitly.

Below is the simplest example of how you can save lambda to a variable:

 val helloPrint: (String) -> Unit = { println(it) } 

For lambda without parameters, the compiler is able to independently derive the type from the already known ones. However, in our case there is one parameter. Calling such a lambda looks like this:

 helloPrint("Hello") 

In the example above, lambda takes one parameter. Inside the lambda, this parameter defaults to the name "it", but if there are several parameters, you must explicitly list their names, or use the underscore "_" to ignore it. The example below demonstrates this behavior.

 val helloPrint: (String, Int) -> Unit = { _, _ -> println("Do nothing") } helloPrint("Does not matter", 42) //output: Do nothing 

The basic tool that you might have already encountered, for example, in Groovy, is lambda outside the brackets. Pay attention to the example at the very beginning of the article; almost every use of curly brackets, with the exception of standard constructions, is the use of lambda. There are at least two ways to make a construction of the form x { … } :

• the object x and its unary operator invoke (we will discuss this method later);
• x function to which lambda is passed.

Regardless of the option, we use lambda. Suppose there is a function x() . In the Kotlin language, the following rule applies: if lambda is the last argument of a function, then it can be taken out of the brackets, if the lambda is a single parameter, then you can leave out the brackets. As a result, the x({…}) construct can be converted to x() {} , and then, removing the brackets, we get x {} . The declaration of such a function is as follows:

 fun x( lambda: () -> Unit ) { lambda() } 

or in abbreviated form for single-line functions, you can write this:

 fun x( lambda: () -> Unit ) = lambda() 

But what if x is an instance of a class, an object, not a function? There is another interesting solution, which is based on one of the fundamental concepts used in the construction of problem-oriented languages, redefinition of operators. Let's look at this tool.

Operator Overriding

Documentation

Kotlin provides a wide but limited range of operators. The operator modifier allows you to define functions by convention that will be called under certain conditions. An obvious example is the plus function, which will be executed when using the "+" operator between two objects. A complete list of operators can be found at the link above in the documentation.

Consider a slightly less trivial operator "invoke". The main example of this article begins with the schedule {} construct. The purpose of the construction is to isolate the block of code that is responsible for planning testing. To construct such a construction, a method slightly different from that discussed above is used: the invoke + operator “lambda outside parentheses”. After the definition of the invoke operator, the schedule (...) construction becomes available to us, while schedule is an object. In fact, the call to schedule (...) is interpreted by the compiler as schedule.invoke (...). Let's look at the schedule declaration.

 object schedule { operator fun invoke(init: SchedulingContext.() -> Unit) { SchedulingContext().init() } } 

It is necessary to understand that the schedule identifier refers us to a single instance of the schedule class (singleton), which is marked with the special keyword object (for more details on such objects, please read here ). Thus, we call the invoke method on the schedule instance and, with the only parameter of the method, we determine the lambda, which we put out of the brackets. As a result, the schedule {...} construction is equivalent to the following:

 schedule.invoke( {    } ) 

However, if you look closely at the invoke method, you will see not the usual lambda, but the lambda with a handler or the lambda with the context, the type of which is written as follows: SchedulingContext.() -> Unit
It's time to figure out what it is.

Lambda with handler

Documentation

Kotlin gives us the opportunity to set the context for lambda expressions. Context is a regular object. The context type is defined along with the lambda expression type. Such a lambda acquires the properties of a non-static method in a context class, but with access only to the public API of this class.
While the type of a conventional lambda is defined as: () -> Unit , the type of lambda with a context of type X is defined as: X.()-> Unit and, if the first type of lambda can be run in the usual way:

 val x : () -> Unit = {} x() 

then for lambda with context context is needed:

 class MyContext val x : MyContext.() -> Unit = {} //x() // , ..    val c = MyContext() //  cx() //  x(c) //   

Let me remind you that in the schedule object we have an invoke operator defined (see the previous paragraph), which allows us to use the construction:

 schedule { } 

The lambda we use has a context like SchedulingContext. In this class, the data method is defined. As a result, we get the following construction:

 schedule { data { //... } } 

As you probably guessed, the data method takes the lambda with context, however, the context is different. Thus, we get nested structures within which several contexts are available simultaneously.

To understand in detail how this example works, let's remove all syntactic sugar:

 schedule.invoke({ this.data({ }) }) 

As you can see, everything is extremely simple.
Let's take a look at the implementation of the invoke operator.

 operator fun invoke(init: SchedulingContext.() -> Unit) { SchedulingContext().init() } 

We call the constructor for the context: SchedulingContext() , and then on the created object (context) we call the lambda with the identifier init, which we passed as a parameter. This is very similar to calling a regular function. As a result, in one line SchedulingContext().init() we create a context and call the lambda passed to the operator. If you are interested in other examples, then pay attention to the apply and with methods from the standard Kotlin library.

In the last examples, we looked at the invoke operator and its interaction with other tools. Next, we focus on another tool that is formally an operator and makes our code cleaner, namely the convention for get / set methods.

Convention for get / set methods

Documentation

When developing DSL, we can implement the syntax of access to an associative array using one or more keys. Take a look at the example below:

 availabilityTable[DayOfWeek.MONDAY, 0] = true println(availabilityTable[DayOfWeek.MONDAY, 0]) //output: true 

To use square brackets, you must implement the get or set methods depending on what you need (read or write) with the operator modifier. An example of the implementation of this tool can be found in the Matrix class on GitHub by reference . This is the simplest implementation of a wrapper for working with matrices. Below is the part of the code that interests us.

 class Matrix(...) { private val content: List<MutableList<T>> operator fun get(i: Int, j: Int) = content[i][j] operator fun set(i: Int, j: Int, value: T) { content[i][j] = value } } 

The types of parameters for the get and set functions are limited only by your imagination. You can use one or several parameters for get / set functions and provide a comfortable syntax for accessing data. Operators in Kotlin bring many interesting features that you can find in the documentation .

Surprisingly, the Kotlin standard library has a Pair class, but why? Most of the community believes that the Pair class is bad, the meaning of the connection of two objects disappears with it and it becomes not obvious why they are paired. The following two tools demonstrate how to preserve the meaningfulness of a couple, and not to create extra classes.

Type aliases

Documentation

Imagine that we need a wrapper class for a point on a plane with integer coordinates. In principle, the Pair<Int, Int> class is suitable for us, but in a variable of this type at one moment we may lose understanding of why we connect values ​​in a pair. Obvious fixes are either writing your own class or something worse. In Kotlin, the developer’s arsenal is replenished with type aliases, which are written as follows:

 typealias Point = Pair<Int,Int> 

In fact, this is the usual renaming of the structure. Thanks to this approach, we do not need to create a class Point, which in this case would simply duplicate the pair. Now, we can create points like this:

 val point = Point(0, 0) 

However, the Pair class has two properties, first and second, and how can we rename these properties to erase any differences between the desired class Point and Pair? The properties themselves cannot be renamed, but there is a great opportunity in our toolkit, which craftsmen have identified as multi-declarations.

Multi-declaration (destructuring declaration)

Documentation

For ease of understanding the example, consider the situation: we have an object of type Point, as we know from the example above, this is just the renamed type Pair<Int, Int> . As can be seen from the implementation of the Pair class of the standard library, it is marked with the data modifier, which means that, among other things, in this class we get the generated componentN methods. Let's talk about them.

For any class, we can define the componentN operator, which will provide access to one of the object properties. This means that calling the point.component1 method is equivalent to calling point.first. Now let's see why this duplication is needed.

What is a multi-declaration? This is a way to "decompose" an object into variables. Thanks to this functionality, we can write the following construction:

 val (x, y) = Point(0, 0) 

We have the opportunity to declare several variables at once, but what will be as values? For this we need the generated methods componentN , in accordance with the sequence number, instead of N, starting from 1, we can decompose an object into a set of its properties. So, for example, a record above is equivalent to the following:

 val pair = Point(0, 0) val x = pair.component1() val y = pair.component2() 

which in turn is equivalent to:

 val pair = Point(0, 0) val x = pair.first val y = pair.second 

where first and second are properties of the Point object.

The for construction in Kotlin has the following form, where x sequentially takes the values ​​1, 2 and 3:

 for(x in listOf(1, 2, 3)) { … } 

Note the assertions block in DSL from the main example. For convenience, I will provide part of it below:

 for ((day, lesson, student, teacher) in scheduledEvents) { … } 

Now everything should be obvious. We iterate through the scheduledEvents collection, each element of which is decomposed into 4 properties describing the current object.

Extension function

Documentation

Adding custom methods to objects from third-party libraries or adding methods to the Java Collection Framework is a long-time dream of many developers. And now we all have that opportunity. The declaration of expanding functions is as follows:

 fun AvailabilityTable.monday(from: String, to: String? = null) 

Unlike the usual method, we add the class name before the method name to indicate which class we are expanding. In the example, AvailabilityTable is an alias for the Matrix type and, since the aliases in Kotlin are just renaming, as a result such a declaration is equivalent to the one shown in the example below, which is not always convenient:

 fun Matrix<Boolean>.monday(from: String, to: String? = null) 

But, unfortunately, nothing can be done about it, except for not using the tool or adding methods only to a specific context class. Then magic appears only where it is needed. Moreover, you can extend even these interfaces with these functions. A good example would be the first method, which extends any Iterable object as follows:

 fun <T> Iterable<T>.first(): T 

As a result, any collection based on the Iterable interface, regardless of the type of the element, receives the first method. The interesting thing is that we can put the extension method in the context class and thus have access to the extension method only in a specific context (see the lambda with the context above). Moreover, we can also create extension functions for Nullable types (the explanation of Nullable types is beyond the scope of the article, but you can read it here if you like). For example, the isNullOrEmpty function from the Kotlin standard library, which extends the CharSequence? Type, can be used as follows:

 val s: String? = null s.isNullOrEmpty() //true 

The signature of this function is shown below:

 fun CharSequence?.isNullOrEmpty(): Boolean 

When working from Java with such Kotlin functions, extension functions are available as static.

Infix functions

Documentation

Another way to add a syntax is to use infix functions. Simply put, thanks to this tool, we were able to get rid of unnecessary code noise in simple situations.
The assertions block from the main sample article demonstrates the use of this tool:

 teacherSchedule[day, lesson] shouldNotEqual null 

Such a construction is equivalent to the following:

 teacherSchedule[day, lesson].shouldNotEqual(null) 

There are situations when brackets and points are unnecessary. In this case, we need the infix modifier for functions.
In the code above, the teacherSchedule[day, lesson] construct returns a schedule element, and the function shouldNotEqual checks that the element is not null.

To declare such a function is necessary:

• specify the infix modifier;
• define exactly one parameter.

You can combine the last two tools, as in the code below:

 infix fun <T : Any?> T.shouldNotEqual(expected: T) 

Please note that the generic type is the default Any inheritor (non-Nullable type hierarchy), however, in such cases, we cannot use null, so you need to explicitly specify the Any? Type.

Context control

Documentation

When we use a lot of nested contexts, an explosive mixture is produced at the lowest level, so, for example, without any control, the following construction can be obtained which does not make sense:

 schedule { // SchedulingContext data { // DataContext +   SchedulingContext data { } // -    } } 

Prior to Kotlin 1.1, there was already a way to avoid this. Create your own data method in the nested DataContext context, and then mark it with the Deprecated annotation with the ERROR level.

 class DataContext { @Deprecated(level = DeprecationLevel.ERROR, message = "Incorrect context") fun data(init: DataContext.() -> Unit) {} } 

Thanks to this approach, we could exclude the possibility of inadmissible building DSL. However, with a large number of methods in the SchedulingContext, we received a certain amount of routine work, discouraging all the desire to control the context.

In Kotlin 1.1 a new tool for control appeared - the @DslMarker annotation. It is applied to your own annotations, which, in turn, are needed to label your contexts. Let's create our own annotation, which we will mark with the help of a new tool in our arsenal:

 @DslMarker annotation class MyCustomDslMarker 

Then you need to mark contexts. In our main example, these are SchedulingContext and DataContext. Due to the fact that we mark each of the classes with a single DSL marker, the following happens:

 @MyCustomDslMarker class SchedulingContext { ... } @MyCustomDslMarker class DataContext { ... } fun demo() { schedule { // SchedulingContext data { // DataContext +     SchedulingContext // data { } // , ..    DSL  } } } 

Despite all the delight of this approach, which reduces a lot of time and effort, one problem remains. If you pay attention to our main example, you will see the following code:

 schedule { data { student { name = "Petrov" } ... } } 

Student, , , , , @MyCustomDslMarker , , , .

Student data {} , .. DataContext , :

 schedule { data { student { student { } } } } 

, , . :

1. , , StudentContext. @DslMarker.
2. , , IStudent ( ), -, , , .
   

    @MyCustomDslMarker class StudentContext(val owner: Student = Student()): IStudent by owner 

3. @Deprecated, . , , , .
   deprecated extension Identifiable .

 @Deprecated("Incorrect context", level = DeprecationLevel.ERROR) fun Identifiable.student(init: () -> Unit) {} 

, , DSL .

DSL

DSL Kotlin , DSL .

DSL

, DSL, . DSL extension , .

, , : " callback'", DSL, . , , . , DSL , .

This, it!?

this it DSL. - it, , , , . , .

, . " " DSL. , , , val mainContext = this

Nesting

. . , , , " ", . , DSL, , , DSL , - . DSL (, ), , .. .

, ?

- DSL, : " ?". . DSL, , . , . , .. - : " , ?" , , .

Conclusion

, - . , .

, - , , . , DSL . , , .

" ", , DSL , , , , .
- !