Macros and quasiquotes in Scala 2.11.0
Not long ago, the release of Scala 2.11.0 . One of the remarkable innovations of this version are quasiquotes - a convenient mechanism for describing Scala syntax trees using strings compiled during compilation; Obviously, this mechanism is primarily intended for use with macros.
Surprisingly, in Habré, while the topic of macros in Scala is not considered very actively; last post
with serious consideration of macros was already a whole year ago.
This post will discuss in detail the writing of a simple macro, designed to generate JSON deserialization code into the class hierarchy.
There is a wonderful JSON library for working with Scala - spray.json .
')
Usually, in order to deserialize some JSON object using this library, a couple of imports are enough:
Simple enough, isn't it? And if we want to deserialize the entire class hierarchy? I will give an example of the hierarchy, which we will consider further:
As you can see, several deserializable classes with different numbers of arguments of different types are inherited from the abstract parent. A completely natural desire when deserializing such entities is to add a
The spray.json library provides the ability to determine the conversion of JSON to any types by user-defined rules through the formatter extension
This code looks a bit ... patterned. This is a great task for the macro! The rest of the article is devoted to the development of a macro that can generate such code, having only the
The first obstacle that a programmer encounters when developing macros is that SBT does not want to compile both the macro and the code using it at the same time. This problem is discussed in the SBT documentation and I recommend the solution described below.
It is necessary to divide the macro code and the main application code into two projects, which should be referred to in the main
Another subtlety is that if you want the macro to work with the class hierarchy, this hierarchy should be known when the macro is expanded. This causes some problems, because the sequence of file processing by the compiler is not always obvious. The solution to this issue is either to have the classes with which the macro should work in the same project with the macro (the macro should still be in another project), or simply place the necessary classes in the same file in which the macro is expanded.
When debugging macros, the
Scala's macros work almost like reflection. Please note that the Scala reflection API is significantly different from Java reflection, since not all Scala concepts are known to the standard Java library.
The macro mechanism in Scala provides the ability to create portions of code at compile time. This is done using a strongly typed API that generates syntax trees corresponding to the code you want to create. Scala macros are significantly different from all the usual C macros, so they should not be confused.
At the heart of Scala macros is the
In essence, a macro is a function that accepts and returns syntax trees. Let's write our macro template:
The
The argument of type
One of the drawbacks of macros is the non-obvious description of syntax trees. For example, the code snippet
The aforementioned lack of macros from version Scala 2.11.0 can be easily resolved with quasi-quotation. For example, the above construction, describing the expression
As well as for other string interpolators of the Scala language, from quasiquote you can refer to the variables of their surrounding scope. For example, to generate the expression
For variables of different types, interpolation can occur in different ways. For example, variables of string type in generated trees become string constants . In order to refer to a variable by name, you need to create a
As you can see from the example of the generated code given at the beginning of the article, we need to be able to generate the following elements:
First of all, we consider the generation of a common tree of the whole
In this section of the code, a special kind of interpolation is used. The
In the form of quasi such a branch can be written as follows:
In this code fragment, several quasiquotes are used: the
Finally, the resulting syntax tree, consisting of a
This tree will be built in by the compiler at the place of application of the macro.
Throughout the development of technology, metaprogramming has become an increasingly important element of programming languages, and it is increasingly being used in everyday code to implement various concepts. Scala macros are an actual tool that can save us from various routine work, which in the JVM world was previously decided to be implemented through reflection or code generation.
Certainly, macros are a powerful tool that should be used with caution: if used improperly, it is enough to simply shoot off your leg and fall into the abyss of unsupported code. However, you should always try to automate routine activities, and if macros can help us in this task, they will be used and will benefit the community.
Surprisingly, in Habré, while the topic of macros in Scala is not considered very actively; last post
with serious consideration of macros was already a whole year ago.
This post will discuss in detail the writing of a simple macro, designed to generate JSON deserialization code into the class hierarchy.
Formulation of the problem
There is a wonderful JSON library for working with Scala - spray.json .
')
Usually, in order to deserialize some JSON object using this library, a couple of imports are enough:
// , : case class MyClass(field: String) // spray.json: import spray.json._ import DefaultJsonProtocol._ implicit val myClassFormat = jsonFormat1(MyClass) val json = """{ "field\": "value" }""" val obj = json.parseJson.convertTo[MyClass] // ok Simple enough, isn't it? And if we want to deserialize the entire class hierarchy? I will give an example of the hierarchy, which we will consider further:
abstract sealed class Message() case class SimpleMessage() extends Message case class FieldMessage(field: String) extends Message case class NestedMessage(nested: Message) extends Message case class MultiMessage(field: Int, nested: Message) extends Message As you can see, several deserializable classes with different numbers of arguments of different types are inherited from the abstract parent. A completely natural desire when deserializing such entities is to add a
type field to a JSON object, and when deserializing, dispatch over this field. The idea can be expressed by the following pseudocode: json.type match { case "SimpleMessage" => SimpleMessage() case "FieldMessage" => FieldMessage(json.field) // ... } The spray.json library provides the ability to determine the conversion of JSON to any types by user-defined rules through the formatter extension
RootJsonFormat . It sounds quite like what we need. The core of our formatter should look like this: val typeName = ... typeName match { case "FieldMessage" => map.getFields("field") match { case Seq(field) => new FieldMessage(field.convertTo[String]) } case "NestedMessage" => map.getFields("nested") match { case Seq(nested) => new NestedMessage(nested.convertTo[Message]) } case "MultiMessage" => map.getFields("field", "nested") match { case Seq(field, nested) => new MultiMessage(field.convertTo[Int], nested.convertTo[Message]) } case "SimpleMessage" => map.getFields() match { case Seq() => new SimpleMessage() } } This code looks a bit ... patterned. This is a great task for the macro! The rest of the article is devoted to the development of a macro that can generate such code, having only the
Message type as a starting point.Project organization
The first obstacle that a programmer encounters when developing macros is that SBT does not want to compile both the macro and the code using it at the same time. This problem is discussed in the SBT documentation and I recommend the solution described below.
It is necessary to divide the macro code and the main application code into two projects, which should be referred to in the main
project/Build.sbt . In the code accompanying the article, these preparations have already been made, here are links to the resulting files:- main
project/Build.sbtfileproject/Build.sbt; - macro project:
macro/build.sbt; - main project:
main/build.sbt.
Another subtlety is that if you want the macro to work with the class hierarchy, this hierarchy should be known when the macro is expanded. This causes some problems, because the sequence of file processing by the compiler is not always obvious. The solution to this issue is either to have the classes with which the macro should work in the same project with the macro (the macro should still be in another project), or simply place the necessary classes in the same file in which the macro is expanded.
When debugging macros, the
-Ymacro-debug-lite compiler option helps you to display the results of deploying all macros in a project to the console (these results are very similar to Scala code, and can often be compiled manually without any changes when passed to the compiler, which can help debugging non-trivial cases).Macros
Scala's macros work almost like reflection. Please note that the Scala reflection API is significantly different from Java reflection, since not all Scala concepts are known to the standard Java library.
The macro mechanism in Scala provides the ability to create portions of code at compile time. This is done using a strongly typed API that generates syntax trees corresponding to the code you want to create. Scala macros are significantly different from all the usual C macros, so they should not be confused.
At the heart of Scala macros is the
Context class. An instance of this class is always passed to the macro when expanded. Then you can import the internals of the Universe object from it and use them in the same way as in runtime reflection — request from there descriptors of types, methods, properties, etc. The same context allows you to create syntax trees using classes like Literal , Constant , List , etc.In essence, a macro is a function that accepts and returns syntax trees. Let's write our macro template:
import scala.language.experimental.macros import scala.reflect.macros.blackbox.Context import spray.json._ object Parsers { def impl[T: c.WeakTypeTag](c: Context)(typeName: c.Expr[String], map: c.Expr[JsObject]): c.Expr[T] = { import c.universe._ val cls = weakTypeOf[T].typeSymbol.asClass val tree = ??? // c.Expr[T](tree) } def parseMessage[T](typeName: String, map: JsObject): T = macro Parsers.impl[T] } The
parseMessage[T] macro takes type T , which is the base for the hierarchy of classes being deserialized, and a syntax tree to get the type of the map deserialized, and returns a syntax tree to get the deserialized object converted to the base type TThe argument of type
T described in a special way: it is indicated that the compiler must attach an implicitly generated object of type c.WeakTypeTag . Generally speaking, the implicit TypeTag argument TypeTag used in Scala to work with generic argument types that are usually not available at runtime due to type erasure . For macro arguments, the compiler requires using not just TypeTag , but WeakTypeTag , which, as far as I understand, is related to the features of the compiler's work (it does not have a “full-fledged” TypeTag for a type that may not yet be fully generated during macro expansion). The type associated with a TypeTag can be obtained using the typeOf[T] method of the Universe object; accordingly, for WeakTypeTag there is a weakTypeOf[T] method.One of the drawbacks of macros is the non-obvious description of syntax trees. For example, the code snippet
2 + 2 should look like Apply(Select(Literal(Constant(2)), TermName("$plus")), List(Literal(Constant(2)))) when generated; even more serious cases begin when we need to present larger pieces of code with pattern substitution. Naturally, we do not like this complexity and we will overcome it.Quasiquotes
The aforementioned lack of macros from version Scala 2.11.0 can be easily resolved with quasi-quotation. For example, the above construction, describing the expression
2 + 2 , in the form of quasiquotes will look just like q"2 + 2" , which is very convenient. In general, quasiquotes in Scala are a set of string interpolators that are located in the Universe object. After importing these interpolators in the current scope, it is possible to use a series of characters before the string constant, which determine its processing by the compiler. In particular, the interpolators pq for patterns, cq for branches of the match expression, and q for complete expressions of the language will be useful to us when implementing the problem in question.As well as for other string interpolators of the Scala language, from quasiquote you can refer to the variables of their surrounding scope. For example, to generate the expression
2 + 2 you can use the following code: val a = 2 q"$a + $a" For variables of different types, interpolation can occur in different ways. For example, variables of string type in generated trees become string constants . In order to refer to a variable by name, you need to create a
TermName object.As you can see from the example of the generated code given at the beginning of the article, we need to be able to generate the following elements:
matchtypeNamevariable withcasebranches corresponding to each type of hierarchy;- in each branch - the transfer of the list of the names of the arguments of the constructor of the corresponding class to the
map.getFieldsmethod; - ibid - deconstruction of the obtained sequence (using the same
matchexpression) on variables and passing these variables to the type constructor.
First of all, we consider the generation of a common tree of the whole
match expression. To do this, you have to use interpolation of variables in the context of quasiquotes: val clauses: Set[Tree] = ??? // . val tree = q"$typeName match { case ..$clauses }" In this section of the code, a special kind of interpolation is used. The
case ..$clauses expression case ..$clauses inside the match block will be expanded as a list of case branches. As we remember, each branch should look like this: case "FieldMessage" => map.getFields("field") match { case Seq(field) => new FieldMessage(field.convertTo[String]) } In the form of quasi such a branch can be written as follows:
val tpe: Type // val constructorParameters: List[Symbol] // val parameterNames = constructorParameters.map(_.name) val parameterNameStrings = parameterNames.map(_.toString) // pq: val parameterBindings = parameterNames.map(name => pq"$name") // , : val args = constructorParameters.map { param => val parameterName = TermName(param.name.toString) val parameterType = param.typeSignature q"$parameterName.convertTo[$parameterType]" } // case: val typeName = tpe.typeSymbol val typeNameString = typeName.name.toString cq"""$typeNameString => $map.getFields(..$parameterNameStrings) match { case Seq(..$parameterBindings) => new $typeName(..$args) }""" In this code fragment, several quasiquotes are used: the
pq"$name" expression creates a set of patterns, which are subsequently substituted into the Seq(...) expression. Each of these expressions is of type JsValue , which must be converted to the appropriate type before passing to the constructor; for this, a quasiquote is used that generates a call to the convertTo method. Note that this method can recursively call our formatter if necessary (that is, you can nest Message objects into each other.Finally, the resulting syntax tree, consisting of a
match expression with the case branches generated by us, can also be constructed using interpolation: val tree = q"$typeName match { case ..$clauses }" This tree will be built in by the compiler at the place of application of the macro.
findings
Throughout the development of technology, metaprogramming has become an increasingly important element of programming languages, and it is increasingly being used in everyday code to implement various concepts. Scala macros are an actual tool that can save us from various routine work, which in the JVM world was previously decided to be implemented through reflection or code generation.
Certainly, macros are a powerful tool that should be used with caution: if used improperly, it is enough to simply shoot off your leg and fall into the abyss of unsupported code. However, you should always try to automate routine activities, and if macros can help us in this task, they will be used and will benefit the community.
Used materials
Source: https://habr.com/ru/post/224229/
All Articles