Handling annotations during compilation

Metaprogramming is a type of programming associated with the creation of programs that generate other programs as a result of their work (in particular, at the stage of compiling their source code), or programs that change themselves during execution.

Annotations as a metaprogramming tool appeared along with the release of Java 5 back in 2004. Together with them appeared the Annotation Processing Tool , which was replaced by the JSR 269 specification or the Pluggable Annotation Processing API . What is interesting, this specification is almost 10 years old, but it began to acquire its popularity in Android development just now.

We will talk about the possibilities that this specification opens a bit later (there will be a lot of code), but first, would you like to talk about compiling Java code?

Couple of words about javac

The whole compilation process is controlled by the tools from the com.sun.tools.javac package and, according to the OpenJDK specifications, in general, looks like this:

Parse - the compiler parses the input stream into a sequence of tokens and forms an abstract syntax tree (AST) using the tools from the com.sun.tools.javac.parser. * Package
Enter - at this stage, a pass through the syntactic tree is performed and a symbol table is created. It is worth noting that this process consists of two phases: the first is the passage through the AST from the Parse phase, the second pass through all dependencies (interfaces, supertypes, parameters).
Annotation processing - this phase, in fact, will be discussed later.
Attribute - most context-sensitive operations are performed during this phase: name resolution, type checking, constant computation.
Flow - at this stage there is a check of the data flow, the reachability of all parts of the code, that all intercepted exceptions are processed, that final variables are set once, and so on.
Desugar - removing syntactic sugar, replacing inner classes, expanding foreach cycles.
Generate - generation of .class files

Metaprogramming in Android

Where does metaprogramming help us in Android development? Many of you already know the answer to this question - it is a considerable number of libraries that somehow offer solutions for injecting components, installing listeners and much more.

public class MainActivity extends AppCompatActivity { @Bind(R.id.fab) private FloatingActionButton mFab; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.ac_main); mFab.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { } }); ButterKnife.bind(this); } }

What problems does this code have? First, he will not meet! The compilation process will be interrupted by an error.

 Error:(15, 34) error: @Bind fields must not be private or static. (moscow.droidcon2015.activity.MainActivity.mFab)

Great, remove private and everything is going to code. But, by doing so, we are violating one of the fundamental principles of OOP - encapsulation . Secondly, at startup, the application will crash with NPE , because the mFab field is initialized at the time of the call to ButterKnife.bind (this) . Third, Proguard can cut classes generated by the ButterKnife library. You can say that these are all far-fetched problems and they are all solved within five minutes. Of course, this is true, but it would be great to save yourself from having to think about these possible problems.
')

Forward! To heavy substances!

So let's start reinventing the wheel! The first thing we need, oddly enough, is the annotation itself, which we will subsequently process:

 @Retention(RetentionPolicy.SOURCE) @Target({ElementType.FIELD}) public @interface BindView { int value(); }

RetentionPolicy.SOURCE means that this annotation is available only in source codes (which completely suits us) and it will not be possible to reach it through reflection. ElementType.FIELD says that the annotation applies only to class fields.

Next, we need to create the processor itself and register it in a special file:

 src/main/resources/META-INF.services/javax.annotation.processing.Processor

The content of this file is a single line containing the full class name of the connected processor:

 moscow.droidcon2015.processor.DroidConProcessor

DroidConProcessor.java

 @SupportedAnnotationTypes({"moscow.droidcon2015.processor.BindView"}) public class DroidConProcessor extends AbstractProcessor { private final Map<TypeElement, BindViewVisitor> mVisitors = new HashMap<>(); @Override public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) { if (annotations.isEmpty()) { return false; } final Set<? extends Element> elements = roundEnv.getElementsAnnotatedWith(BindView.class); for (final Element element : elements) { // element == MainActivity.mFab final TypeElement object = (TypeElement) element.getEnclosingElement(); // object == MainActivity BindViewVisitor visitor = mVisitors.get(object); if (visitor == null) { visitor = new BindViewVisitor(processingEnv, object); mVisitors.put(object, visitor); } element.accept(visitor, null); } for (final BindViewVisitor visitor : mVisitors.values()) { visitor.brewJava(); } return true; } @Override public SourceVersion getSupportedSourceVersion() { return SourceVersion.latestSupported(); } }

No matter how paradoxical it may be, we mark the processor itself with an annotation, which indicates what kind of annotations the processor can handle. It is not difficult to guess that the main method is the process method. The first parameter is the set of annotations from the list supported by our processor, which were discovered during the first two phases of the compilation. The second parameter is the compiler environment. In an amicable way, in the implementation of the method we have to go through the entire set of annotations found and process them all, but in this case, the processor supports only one single annotation, so we will process it head-on. Consider the process method in steps:

We check that at least the summary of the supported
We get from the environment the set of all elements marked with the annotation @BindView
We pass on the given set. As we remember, annotation can only be applied to a class field; accordingly, the element.getEnclosingElement () method returns an object of the class in which the field is contained.
Create a class visitor for each object containing marked fields.
We apply our visitor to each field
After all visitors have worked, we generate the final classes of source code

BindViewVisitor.java

 public class BindViewVisitor extends ElementScanner7<Void, Void> { private final CodeBlock.Builder mFindViewById = CodeBlock.builder(); private final Trees mTrees; private final Messager mLogger; private final Filer mFiler; private final TypeElement mOriginElement; private final TreeMaker mTreeMaker; private final Names mNames; public BindViewVisitor(ProcessingEnvironment env, TypeElement element) { super(); mTrees = Trees.instance(env); mLogger = env.getMessager(); mFiler = env.getFiler(); mOriginElement = element; final JavacProcessingEnvironment javacEnv = (JavacProcessingEnvironment) env; mTreeMaker = TreeMaker.instance(javacEnv.getContext()); mNames = Names.instance(javacEnv.getContext()); } }

Now let's look at the class in which all the main work is done. The first thing that the eye clings to is ElementScanner7 . This is an implementation of the ElementVisitor interface, and 7 is the minimum version of the JDK we want to use. Let's walk through the fields (more precisely, according to their types):

CodeBlock.Builder is part of the javapoet library from the guys from Square, which is designed to simplify code generation.
Trees is a class from the com.sun.source.util package that allows access to the AST.
Messager - logger. You can display messages during the compilation process or interrupt the process if you send a message with an ERROR priority.
Filer is a class that allows you to create source code files in the current compiler sandbox. Knows exactly where to place the file in the file system. For example, for gradle this is build / intermediates / classes.
TreeMaker is a class from the com.sun.tools.javac.tree package, which is absolutely responsible for all the magic that will happen next! It is used in the first phase of compilation to build AST.
Names is a class from the com.sun.tools.javac.util package that converts the element names into AST constructs.

As you remember, we applied the ElementVisitor to the class field, so the method that interests us is

visitVariable

  @Override public Void visitVariable(VariableElement field, Void aVoid) { ((JCTree) mTrees.getTree(field)).accept(new TreeTranslator() { @Override public void visitVarDef(JCTree.JCVariableDecl jcVariableDecl) { super.visitVarDef(jcVariableDecl); jcVariableDecl.mods.flags &= ~Flags.PRIVATE; } }); final BindView bindView = field.getAnnotation(BindView.class); mFindViewById.addStatement("(($T) this).$L = ($T) findViewById($L)", ClassName.get(mOriginElement), field.getSimpleName(), ClassName.get(field.asType()), bindView.value()); return super.visitVariable(field, aVoid); }

A small digression to understand what will happen next: the classes from javax.lang.model.element (VariableElement, TypeElement, etc.) are, let's say, a high-level abstraction over AST. With the help of the Trees class, we get a low-level abstraction, set the TreeVisitor 's implementation on it and get into the visitVarDef method in whose parameters the AST representation of our field is located ( JCTree.JCVariableDecl ). Next dirty hack - remove the private flag from the field. Yes, yes, we violate the principle of encapsulation, but we do it at the level of the compiler (where we are, in principle, because what is happening). At the source code level, the encapsulation is preserved: IDE will not allow access to the field from the outside, and the static analyzer will quietly report that there are no problems with this field. Add an expression to CodeBlock.Builder to initialize the field and that's it.

Generating the source file

After we have visited all the fields of our class, it is necessary to generate the source code file.

brewJava

  public void brewJava() { final TypeSpec typeSpec = TypeSpec.classBuilder(mOriginElement.getSimpleName() + "$$Proxy") // MainActivity$$Proxy .addModifiers(Modifier.ABSTRACT) .superclass(ClassName.get(mOriginElement.getSuperclass())) // extends AppCompatActivity .addOriginatingElement(mOriginElement) .addMethod(MethodSpec.methodBuilder("setContentView") .addAnnotation(Override.class) .addModifiers(Modifier.PUBLIC) .addParameter(TypeName.INT, "layoutResId") .addStatement("super.setContentView(layoutResId)") .addCode(mFindViewById.build()) // findViewById... .build()) .build(); final JavaFile javaFile = JavaFile.builder(mOriginElement.getEnclosingElement().toString(), typeSpec) .addFileComment("Generated by DroidCon processor, do not modify") .build(); try { final JavaFileObject sourceFile = mFiler.createSourceFile( javaFile.packageName + "." + typeSpec.name, mOriginElement); try (final Writer writer = new BufferedWriter(sourceFile.openWriter())) { javaFile.writeTo(writer); } // TODO: MAGIC } catch (IOException e) { mLogger.printMessage(Diagnostic.Kind.ERROR, e.getMessage(), mOriginElement); } }

All work on generating the source code was taken over by the javapoet library. Of course, it would have been possible to do without it, but then the whole source would have to be generated manually using string concatenation, which, you see, is not very convenient. At this stage, all the creators of libraries like ButterKnife finish . We received a class file, which we then find with the help of reflection and, with its help, we pull the corresponding method, which performs useful work. But I promised that we will get rid of this need!

We need to go deeper!

TODO: MAGIC

 JCTree.JCExpression selector = mTreeMaker.Ident(mNames.fromString(javaFile.packageName)); selector = mTreeMaker.Select(selector, mNames.fromString(typeSpec.name)); ((JCTree.JCClassDecl) mTrees.getTree(mOriginElement)).extending = selector;

Yes! Three lines. What happens in them:

Choose one of the AST nodes. In our case, the package in which the generated class lies.
Go deep into the tree and select the next node - the generated class itself.
At the source element (MainActivity) we change the extending property, which, in fact, means from which this class is inherited.

In an even simpler language, we embed the generated class in the inheritance hierarchy:

 MainActivity extends MainActivity$$Proxy extends AppCompatActivity

MainActivity $$ Proxy.java

 // Generated by DroidCon processor, do not modify package moscow.droidcon2015.activity; import android.support.design.widget.FloatingActionButton; import android.support.v7.app.AppCompatActivity; import java.lang.Override; abstract class MainActivity$$Proxy extends AppCompatActivity { @Override public void setContentView(int layoutResId) { super.setContentView(layoutResId); ((MainActivity) this).mFab = (FloatingActionButton) findViewById(2131492965); } }

MainActivity.class (decompiled)

 // // Source code recreated from a .class file by IntelliJ IDEA // (powered by Fernflower decompiler) // package moscow.droidcon2015.activity; import android.os.Bundle; import android.support.design.widget.FloatingActionButton; import android.view.View; import android.view.View.OnClickListener; import moscow.droidcon2015.activity.MainActivity$$Proxy; public class MainActivity extends MainActivity$$Proxy { FloatingActionButton mFab; public MainActivity() { } protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); this.setContentView(2130968600); this.mFab.setOnClickListener(new OnClickListener() { public void onClick(View v) { } }); } }

Conclusion

Unfortunately, within the framework of a single article it is impossible to tell about all the intricacies of Annotation processing and the treasures that lie inside com.sun.tools.javac. * . What is even more distressing is the complete lack of any documentation on these treasures and the lack of compatibility between releases. Then scary words will sound: to provide support for the java7 and java8 compilers, you will need to use reflection in the compilation process! From this turn! True? But I repeat once again - this only applies to com.sun.tools.javac .

Based on DroidCon

It is more convenient to read the article while flipping through the presentation .
Project repository here .

Answers on questions:

This is not a research task. All this is already actively working in a number of projects.
The advantage of this approach over the modification of bytecode by libraries like ASM is that annotation processing is performed at the time of compilation and not after, and the ability to grab a compilation error and not runtime, IMHO, is much better.
You can see it in the DroidKit library. andkulikov , the documentation will undoubtedly appear. When? When is done. =)

More hardcore!

visitMethodDef

 @Override public void visitMethodDef(JCTree.JCMethodDecl methodDecl) { super.visitMethodDef(methodDecl); methodDecl.body.stats = com.sun.tools.javac.util.List.<JCTree.JCStatement>of( mTreeMaker.Try( mTreeMaker.Block(0, methodDecl.body.stats), com.sun.tools.javac.util.List.<JCTree.JCCatch>nil(), mTreeMaker.Block(0, com.sun.tools.javac.util.List.<JCTree.JCStatement>of( mTreeMaker.Exec(mTreeMaker.Apply( com.sun.tools.javac.util.List.<JCTree.JCExpression>nil(), ident(mPackageName, mHelperClassName, "update"), com.sun.tools.javac.util.List.of( mTreeMaker.Literal(TypeTag.CLASS, mColumnName), mTreeMaker.Select(mTreeMaker.Ident(mNames._this), mNames.fromString(mFieldName)), mTreeMaker.Select(mTreeMaker.Ident(mNames._this), mNames.fromString(mPrimaryKey.call())) ) )) )) ) ); }

Here is such a scary at first glance code that only modifies the code of the setter method so that the changes are written directly to the database.

pants are turning smoothly ...

 //  public void setText(String text) { mText = text; } //  public void setText(String text) { try { this.mText = text; } finally { Foo$$SQLiteHelper.update("text", this.mText, this.mId); } }

Sources

Source: https://habr.com/ru/post/206208/

All Articles