Utilize Groovy DSL features for configuring Java applications

Prehistory

Hello! I would like to tell the story of the terrible configs and how they managed to comb and make sane. I am working on a rather large and relatively old project that is constantly finishing and growing. The configuration is set using mapping xml-files to java-bins. Not the best solution, but it has its advantages - for example, when creating a service, you can transfer a bin with the configuration responsible for its section to it. However, there are downsides. The most significant of them is that there is no normal inheritance of configuration profiles. At some point, I realized that in order to change one setting, I had to edit about 30 xml files, one for each of the profiles. This could not continue any longer, and a willful decision was made to rewrite everything.

Requirements

• Inheritance and redefinition (or fallback). It should be possible to set a certain basic profile, inherit from it the children and redefine or add to them the places that are necessary
• Mapping in java-bins. I didn’t want to rewrite the use of the configuration from bins to the mongodb.directory.host type for the whole project, use maps from map too.
• Ability to write comments in the config. Not critical, but comfortable and pleasant.

I would like the config to look like this:

 name = "MyTest" description = "Apache Tomcat" http { port = 80 secure = false } https { port = 443 secure = true } mappings = [ { url = "/" active = true }, { url = "/login" active = false } ] 

As I achieved it - under a cat.

Maybe for this there is already a library?

Probably yes. However, from those that I found and looked, nothing came up to me. Most of them are designed for reading configs, merging them into one large one and then working with the received config via separate properties. Almost no one knows how to mapping bins, and it is too long to write dozens of converters for converters. The most promising seemed to be the lightbend config , with its nice HOCON format and inheritance / redefinition out of the box. And she was almost able to fill in a java-bin, but as it turned out, she doesn’t know how to map and expands very poorly. While I was experimenting with it, a colleague looked at the resulting configs and said: “This is somewhat similar to Groovy DSL”. So it was decided to use it.

What it is?

DSL (domain-specific language, domain-specific language) is a language "sharpened" for a specific scope, in our case - for the configuration of our particular application. An example can be seen in the spoiler before the cut.

Running groovy scripts from a java application is easy. You just need to add groovy depending, for example, Gradle

 compile 'org.codehaus.groovy:groovy-all:2.3.11' 

and use GroovyShell

  GroovyShell shell = new GroovyShell(); Object value = shell.evaluate(pathToScript); 

How it works?

All magic is based on two things.

Delegation

To begin with, the groovy script is compiled into bytecode, a class is created for it, and when the script is run, the run () method of this class is called, containing all the script code. If the script returns some value, then we can get it as the result of evaluate() . In principle, it would be possible in the script to create our beans with the configuration and return them, but in this case we will not get a nice syntax.

Instead, we can create a script of a special type - DelegatingScript . Its peculiarity is that it can pass a delegate object to it, and all calls to methods and work with fields will be delegated to it. The documentation for the link is an example of use.
Let's create a class that will contain our config

 @Data public class ServerConfig extends GroovyObjectSupport { private String name; private String description; } 

@Data - annotation from the lombok library: adds getters and setters to fields and implements toString, equals and hashCode. Thanks to her, POJO turns into bin.

GroovyObjectSupport is the base class for "java-objects that want to appear as groovy-objects" (as written in the documentation). Later I will show what it is for. At this stage, you can do without it, but let it be right away.

Now create a script that will fill in its fields.

 name = "MyTestServer" description = "Apache Tomcat" 

It's all obvious. So far, as you see, we do not use any features of DSL, I will tell about them later.

And finally run it from java

 CompilerConfiguration cc = new CompilerConfiguration(); cc.setScriptBaseClass(DelegatingScript.class.getName()); //      groovy     DelegatingScript GroovyShell sh = new GroovyShell(Main.class.getClassLoader(), new Binding(), cc); DelegatingScript script = (DelegatingScript)sh.parse(new File("config.groovy")); ServerConfig config = new ServerConfig(); //     script.setDelegate(config); //    run()  " "  config     name  description script.run(); System.out.println(config.toString()); 

ServerConfig(name=MyTestServer, description=Apache Tomcat) is the result of the lombok implementation of toString ().

As you can see, everything is quite simple. The config is a real executable groovy-code, you can use all features of the language in it, for example, substitutions

 def postfix = "server" name = "MyTest ${postfix}" description = "Apache Tomcat ${postfix}" 

will return us ServerConfig(name=MyTest server, description=Apache Tomcat server)

And in this script you can even set breakpoints and debug!

Method call

We now turn to the actual DSL. Suppose we want to add connector settings to our config. They look like this:

 @Data public class Connector extends GroovyObjectSupport { private int port; private boolean secure; } 

Add the fields for the two connectors, http and https, to our server config:

 @Data public class ServerConfig extends GroovyObjectSupport { private String name; private String description; private Connector http; private Connector https; } 

We can set the connectors from the script using the groovy code

 import org.example.Connector //... http = new Connector(); http.port = 80 http.secure = false 

As you can see, it worked, but, of course, this syntax is completely unsuitable for configuration. Let's rewrite the config as we would like it to look:

 name = "MyTest" description = "Apache Tomcat" http { port = 80 secure = false } https { port = 443 secure = true } 

It looks like we are trying to call the http(Closure) method, and groovy cannot find it from either the delegate object or the script. We could, of course, declare it in the ServersConfig class:

  public void http(Closure closure) { http = new Connector(); closure.setDelegate(http); closure.setResolveStrategy(Closure.DELEGATE_FIRST); closure.call(); } 

And similar - for https. This time everything is fine:

Here we need to clarify what we have done, because this is the first step to DSL. We declared a method that accepts the groovy.lang.Closure parameter, creates a new object for our groovy.lang.Closure field, delegates it to the received closure, and executes the closure code. Line

 closure.setResolveStrategy(Closure.DELEGATE_FIRST); 

means that when accessing fields or methods, groovy will first look at the delegate, and only then, if it does not find anything suitable, on the closure. For the script, this strategy is used by default, for closing it must be set manually.

In principle, we already have some DSL, but it is far from ideal. Firstly, I would like to avoid manually writing the code to install each field, and secondly, I would like to avoid duplicating the code for all classes of bins that are used in our config. And here the second component of the groovy DSL magic comes to our rescue ...

methodMissing ()

Each time groovy encounters a method call that is not present in the object, it tries to call methodMissing (). As parameters, the name of the method that they tried to call and the list of its arguments are passed there. Remove the http and https methods from the ServerConfig class and declare the following instead:

 public void methodMissing(String name, Object args) { System.out.println(name + " was called with " + args.toString()); } 

args is actually of type Object[] , but groovy is looking for a method with exactly that signature. Check:

 http was called with [Ljava.lang.Object;@16aa0a0a https was called with [Ljava.lang.Object;@691a7f8f ServerConfig(name=MyTest, description=Apache Tomcat, http=null, https=null) 

Exactly what is needed! It remains only to expand the arguments and, depending on the type of the parameter, set the values ​​of the fields. In our case, an array of one element of the Closure class is passed there. For example, let's do this:

  public void methodMissing(String name, Object args) { MetaProperty metaProperty = getMetaClass().getMetaProperty(name); if (metaProperty != null) { Closure closure = (Closure) ((Object[]) args)[0]; Object value = getProperty(name) == null ? metaProperty.getType().getConstructor().newInstance() : getProperty(name); closure.setDelegate(value); closure.setResolveStrategy(Closure.DELEGATE_FIRST); closure.call(); setProperty(name, value); } else { throw new IllegalArgumentException("No such field: " + name); } } 

Here we see several calls specific for groovy objects.

• we see that the method called by name matches one of the fields by referring to the metaclass. The metaclass is present in each groovy object and works approximately as reflection, but more convenient. Metaclass, in particular, allows you to receive information about fields and access to them through accessors, even if the fields themselves are private. This will come in handy later.
• we get the type of the field through the same metaclass to create a new instance of it. Here we are counting on the fact that all classes that we are going to use in configs have a default constructor, but in principle no one bothers to make the logic as complex as you need.
• we get the value of the field through getProperty () and set the new value through setProperty (). These are methods from GroovyObjectSupport and they access the field via accessors if they find them, or directly. This saves us from having to change the field through reflection or some other not very convenient way, especially if this field is somewhere in the heir class.

So far we have added methodMissing and all dsl-buns for only one class, ServerConfig. We could implement the same method for Connection, but why duplicate the code? Let's create some basic class for all our config-bins, say, GroovyConfigurable, transfer methodMissing to it, and inherit ServerConfig and Connector.

 public class GroovyConfigurable extends GroovyObjectSupport { @SneakyThrows public void methodMissing(String name, Object args) { MetaProperty metaProperty = getMetaClass().getMetaProperty(name); if (metaProperty != null) { Closure closure = (Closure) ((Object[]) args)[0]; Object value = getProperty(name) == null ? metaProperty.getType().getConstructor().newInstance() : getProperty(name); closure.setDelegate(value); closure.setResolveStrategy(Closure.DELEGATE_FIRST); closure.call(); setProperty(name, value); } else { throw new IllegalArgumentException("No such field: " + name); } } } @Data public class ServerConfig extends GroovyConfigurable { private String name; private String description; private Connector http; private Connector https; } @Data public class Connector extends GroovyConfigurable { private int port; private boolean secure; } 

It all works, even though GroovyConfigurable knows nothing about the fields of its heirs!

Inheritance

The next step is to make it possible to include in the config some parent config and redefine some separate fields. It should look something like this.

 include 'parent.groovy' name = "prod" https { port = 8080 } 

Groovy allows you to import classes, but not scripts. The easiest way is to implement the include method in our GroovyConfigurable class. Add the path to the script itself and a couple of methods:

  private URI scriptPath; @SneakyThrows public void include(String path) { //        URI uri = Paths.get(scriptPath).getParent().resolve(path).toUri(); runFrom(uri); } @SneakyThrows public void runFrom(URI uri) { this.scriptPath = uri; //  ,     main- CompilerConfiguration cc = new CompilerConfiguration(); cc.setScriptBaseClass(DelegatingScript.class.getName()); GroovyShell sh = new GroovyShell(Main.class.getClassLoader(), new Binding(), cc); DelegatingScript script = (DelegatingScript)sh.parse(uri); script.setDelegate(this); script.run(); } 

Let's make the parent.groovy config, in which we will describe a certain basic config:

 name = "PARENT NAME" description = "PARENT DESCRIPTION" http { port = 80 secure = false } https { port = 443 secure = true } 

In config.groovy, we leave only what we want to override:

 include "parent.groovy" name = "MyTest" https { port = 8080 } 

As you can see, the name has been redefined, just like the port field in https. The secure field in it remains from the parent config.

You can go even further and make it possible to include not the whole config, but its individual parts! To do this, in methodMissing, you need to add a check that the field to be set is also GroovyConfigurable and set the path to the parent script.

  public void methodMissing(String name, Object args) { MetaProperty metaProperty = getMetaClass().getMetaProperty(name); if (metaProperty != null) { Closure closure = (Closure) ((Object[]) args)[0]; Object value = getProperty(name) == null ? metaProperty.getType().getConstructor().newInstance() : getProperty(name); if (value instanceof GroovyConfigurable) { ((GroovyConfigurable) value).scriptPath = scriptPath; } closure.setDelegate(value); closure.setResolveStrategy(Closure.DELEGATE_FIRST); closure.call(); setProperty(name, value); } else { throw new IllegalArgumentException("No such field: " + name); } } 

This will allow us to include not only the whole script, but also its parts! For example, so

 http { include "http.groovy" } 

where http.groovy is

 port = 90 secure = true 

This is an excellent result, but there is a small problem.

Generics

Let's say we want to add mappings and their status to our server config.

 name = "MyTest" description = "Apache Tomcat" http { port = 80 secure = false } https { port = 443 secure = true } mappings = [ { url = "/" active = true }, { url = "/login" active = false } ] 

 @Data public class Mapping extends GroovyConfigurable { private String url; private boolean active; } 

 @Data public class ServerConfig extends GroovyConfigurable { private String name; private String description; private Connector http; private Connector https; private List<Mapping> mappings; } 

Oops. Type erasure in all its glory. Unfortunately, magic ends here and we have to correct what we read with our hands. For example, using a separate method GroovyConfigurable#postProcess()

 public void postProcess() { for (MetaProperty metaProperty : getMetaClass().getProperties()) { Object value = getProperty(metaProperty.getName()); if (Collection.class.isAssignableFrom(metaProperty.getType()) && value instanceof Collection) { //      ParameterizedType collectionType = (ParameterizedType) getClass().getDeclaredField(metaProperty.getName()).getGenericType(); //       ,  ,    ,    //  ,      Class itemClass = (Class)collectionType.getActualTypeArguments()[0]; //      ,       GroovyConfigurable //   , ,    if (GroovyConfigurable.class.isAssignableFrom(itemClass)) { Collection collection = (Collection) value; //      ,    ,      Collection newValue = collection.getClass().newInstance(); for (Object o : collection) { if (o instanceof Closure) { //      Object item = itemClass.getConstructor().newInstance(); ((GroovyConfigurable) item).setProperty("scriptPath", scriptPath); ((Closure) o).setDelegate(item); ((Closure) o).setResolveStrategy(Closure.DELEGATE_FIRST); ((Closure) o).call(); ((GroovyConfigurable) item).postProcess(); //     ? newValue.add(item); } else { newValue.add(o); } } setProperty(metaProperty.getName(), newValue); } } } } 

It turned out, of course, ugly, but it does its work. In addition, we wrote it for only one base class, and do not need to repeat for the heirs. After calling config.postProcess(); we will get usable bins.

Conclusion

Of course, the code given here is just a small (simplest) part of what is needed in a real library for configuration, and the more complicated your use case is, the more manual processing and checks must be added. For example, support for maps, enums, nested generics, etc. The list is endless, but for my needs I had enough of what I quoted in the article. I hope this helps you too and your configs will become more beautiful and comfortable!