Create a plugin for Netbeans using the example of the Vala language
Vala is a programming language that was created as a replacement for the C language when developing applications for Gnome. In this case, the language in my opinion was a success, in its syntax took a lot of C # and Java. You can read more about it on Habré or the Gnome website . Wanting to try this project in the case faced with the fact that at the moment there is no high-quality IDE or plug-in for the IDE that would allow it to develop comfortably.
As a result, a plugin for NetBeans IDE was created, which in my opinion is one of the best open source IDE (along with Eclipse). Nevertheless, information on how to develop modules and plugins for NetBeans Plaform is rather fragmented, and there is almost nothing in Russian.
')
The first thing that needs to be done to support a new language in NetBeans is to write the lexical and parser for this language. You can write them yourself, or use some tools to help you do this. Tools can be for example:
To implement the plugin, I chose ANTLR, as one of the most popular, namely ANTLRv3 . He can generate Java code, which makes it relatively easy to integrate it into a plugin for the NetBeans Platform.
For ANTLR, you must write a language description using LL grammar. This language is close to the RBNF and is largely intuitive.
If simplified, the description of the grammar of ANTLR consists of two types of rules - lexer and grammar. The lexer rules describe tokens that are analyzed at the lexical parsing stage. The names of the lexer rules are written in upper case, and the grammar rules in lower case.
Example of a lexer rule:
This rule identifies identifiers in the source code, namely, words that begin with a Latin letter or underscore and do not contain digits, Latin characters or underscores, respectively. All lexer rules matching the rules will be allocated to tokens.
After parsing the tokens, a syntax analysis is performed that generates an abstract syntax tree or AST. To generate an AST, grammar rules are used.
Example grammar rule:
This rule describes a try ... catch block. A rule consists of a sequence of tokens and / or other grammar rule names. The symbol '*' in this case means that the block in catch_clause brackets should be repeated several times or absent. The symbol '?' - the finally_clause block may be present only once or it may not be present.
Perhaps this will end with the language of ANTLR, the site has documentation and many examples. There you can also find ANTLRWorks , which will help you debug the rules. It displays a disassembled tree in a graphical and fairly visual form.
You can see the description of the syntax of the Vala language that was written for the plugin on github.
First you need to create a project for a NetBeans module, through the menu New Project | NetBeans Modules | Module. Check the box "Standalone Module". You must specify the correct name of the project, the name of the module and check the box “Generate XML Layer”. The last action will create a layer.xml file that will contain the names of the classes that implement the behavior of the plugin and other parameters of your module.
NetBeans works with project files through the Datasystems API and Filesystems API . You can add support for new extensions by selecting the root element of the project, then the menu New | Other | Module Development | File Type.
These actions will lead to the creation of a simple class for working with a DataObject, in our case this is a ValaDataObject , and this class will be registered in layer.xml as an .vala file handler:
Now we have everything to add support for syntax highlighting of the new language to the module.
We now have a parser written in ANTLR, and we can connect it to the module for Netbeans. To do this, you need to generate lexer and parser files in ANTLRWorks and add them to the project. You can specify a package for the generated files:
First you need to implement the abstract class org.netbeans.spi.lexer.LanguageHierarchy. In fact, this class defines a language in the form of tokens, tokens are divided into categories. Categories are needed just to implement syntax highlighting.
I missed the implementation of ValaTokenId here, it is quite simple
It remains to create a class
The main purpose of the
Another point here is that you need to adapt the stream of characters that the NetBeans Platform gives us to the stream that the ANTLR classes will perceive, namely, the
We create the file FontAndColors.xml, in which we describe the correspondence of the categories of tokens (as mentioned above), to the categories of elements highlighted in different colors. In particular, our file looks like this:
In the Bundle.properties file, we describe the human names for the categories:
Next, add a sample file with the source code ValaExample.vala to the project. It will appear in the NetBeans Token Color Options dialog. And the last step is to add lines to the layer.xml file for highlighting:
If you did everything right, and I did not miss anything - you can compile and run the project. When opening files with the extension .vala, we see that the keywords are highlighted in the desired color. In the settings dialog, the Vala language should appear, which you can select and view the highlighted contents of the ValaExample.vala file.
After all done, we just got syntax highlighting. But the analyzer, which was written using ANTLR, can be used for other tasks, namely:
The current day in the plugin for Vala implemented the first two points. In the future I will try to describe how the project can add support for indents and highlighting of erroneous constructions.
There was also an idea to try to parse the Vala code not using ANTLR, but with the parser, which itself is written in the Vala language. If specifically, then with its implementation, which is used in the Vala compiler - libvala. If anyone is interested in connecting to the finalization of the plugin, write!
As a result, a plugin for NetBeans IDE was created, which in my opinion is one of the best open source IDE (along with Eclipse). Nevertheless, information on how to develop modules and plugins for NetBeans Plaform is rather fragmented, and there is almost nothing in Russian.
')
Parser for language
The first thing that needs to be done to support a new language in NetBeans is to write the lexical and parser for this language. You can write them yourself, or use some tools to help you do this. Tools can be for example:
- JavaCC is a parser generator for the Java language;
- ANTLR - parser generator for many languages;
- Bison - parser generator;
- Yacc - parser generator;
- many other parsers
To implement the plugin, I chose ANTLR, as one of the most popular, namely ANTLRv3 . He can generate Java code, which makes it relatively easy to integrate it into a plugin for the NetBeans Platform.
For ANTLR, you must write a language description using LL grammar. This language is close to the RBNF and is largely intuitive.
If simplified, the description of the grammar of ANTLR consists of two types of rules - lexer and grammar. The lexer rules describe tokens that are analyzed at the lexical parsing stage. The names of the lexer rules are written in upper case, and the grammar rules in lower case.
Example of a lexer rule:
ID : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_')* ;This rule identifies identifiers in the source code, namely, words that begin with a Latin letter or underscore and do not contain digits, Latin characters or underscores, respectively. All lexer rules matching the rules will be allocated to tokens.
After parsing the tokens, a syntax analysis is performed that generates an abstract syntax tree or AST. To generate an AST, grammar rules are used.
Example grammar rule:
try_statement : TRY block (catch_clause)* (finally_clause)?;This rule describes a try ... catch block. A rule consists of a sequence of tokens and / or other grammar rule names. The symbol '*' in this case means that the block in catch_clause brackets should be repeated several times or absent. The symbol '?' - the finally_clause block may be present only once or it may not be present.
Perhaps this will end with the language of ANTLR, the site has documentation and many examples. There you can also find ANTLRWorks , which will help you debug the rules. It displays a disassembled tree in a graphical and fairly visual form.
You can see the description of the syntax of the Vala language that was written for the plugin on github.
NetBeans Module
1) Create a project
First you need to create a project for a NetBeans module, through the menu New Project | NetBeans Modules | Module. Check the box "Standalone Module". You must specify the correct name of the project, the name of the module and check the box “Generate XML Layer”. The last action will create a layer.xml file that will contain the names of the classes that implement the behavior of the plugin and other parameters of your module.
2) Add support for .vala and .vapi files to the module
NetBeans works with project files through the Datasystems API and Filesystems API . You can add support for new extensions by selecting the root element of the project, then the menu New | Other | Module Development | File Type.
These actions will lead to the creation of a simple class for working with a DataObject, in our case this is a ValaDataObject , and this class will be registered in layer.xml as an .vala file handler:
< filesystem >
< folder name ="Loaders" >
< folder name ="text" >
< folder name ="x-vala" >
< folder name ="Actions" >
...
</ folder >
< folder name ="Factories" >
< file name ="ValaDataLoader.instance" >
< attr name ="SystemFileSystem.icon" urlvalue ="nbresloc:/org/carbonfx/valaproject/vala.png" />
< attr name ="dataObjectClass" stringvalue ="org.carbonfx.valaproject.ValaDataObject" />
< attr name ="instanceCreate" methodvalue ="org.openide.loaders.DataLoaderPool.factory" />
< attr name ="mimeType" stringvalue ="text/x-vala" />
</ file >
</ folder >
</ folder >
...
* This source code was highlighted with Source Code Highlighter .Syntax highlighting
Now we have everything to add support for syntax highlighting of the new language to the module.
1) Connect parser files generated by ANTLR to the module
We now have a parser written in ANTLR, and we can connect it to the module for Netbeans. To do this, you need to generate lexer and parser files in ANTLRWorks and add them to the project. You can specify a package for the generated files:
@header { package org.carbonfx.valaproject.antlr; }
@lexer::header { package org.carbonfx.valaproject.antlr; }
* This source code was highlighted with Source Code Highlighter .2) Adapt the ANTLR parser to the NetBeans API
First you need to implement the abstract class org.netbeans.spi.lexer.LanguageHierarchy. In fact, this class defines a language in the form of tokens, tokens are divided into categories. Categories are needed just to implement syntax highlighting.
public class ValaLanguageHierarchy extends LanguageHierarchy<ValaTokenId> {
@Override
protected Collection<ValaTokenId> createTokenIds() {
return Arrays.asList(tokens);
}
@Override
protected Lexer<ValaTokenId> createLexer(LexerRestartInfo<ValaTokenId> lri) {
return new org.carbonfx.valaproject.lexer.ValaLexer(lri);
}
@Override
protected String mimeType() {
return "text/x-vala" ;
}
static synchronized ValaTokenId getToken( int id) {
return idToToken. get (id);
}
private static final Map<Integer, ValaTokenId> idToToken = new HashMap<Integer, ValaTokenId>();
private static final ValaTokenId[] tokens = new ValaTokenId[] {
token(ValaLexer.AND, "operator" ),
token(ValaLexer.AND_ASSIGN, "operator" ),
token(ValaLexer.BACKSLASH, "operator" ),
// ...
token(ValaLexer.UNICODE_CHAR, "error" ),
token(ValaLexer.OTHER_CHAR, "error" ),
token(ValaLexer.UNKNOWN_CHAIN, "error" ),
};
static {
for (ValaTokenId token : tokens) {
idToToken.put(token.ordinal(), token);
}
}
private static ValaTokenId token( int antlrToken, String category) {
return new ValaTokenId (ValaParser.tokenNames[antlrToken], category, antlrToken);
}
}
* This source code was highlighted with Source Code Highlighter .I missed the implementation of ValaTokenId here, it is quite simple
It remains to create a class
ValaLexer ValaLexer , which implements the Lexer<ValaTokenId> interface.The main purpose of the
ValaLexer class is to receive a sequence of characters and return a sequence of tokens. In the simplest implementation, it is enough to save the sequence of characters in the constructor and in the nextToken() method to implement the return of tokens in turn.Another point here is that you need to adapt the stream of characters that the NetBeans Platform gives us to the stream that the ANTLR classes will perceive, namely, the
CharStream interface. Fortunately, there is already a ready implementation of this class in a similar project.3) Backlight configuration
We create the file FontAndColors.xml, in which we describe the correspondence of the categories of tokens (as mentioned above), to the categories of elements highlighted in different colors. In particular, our file looks like this:
<? xml version ="1.0" encoding ="UTF-8" ? >
<! DOCTYPE fontscolors PUBLIC "-//NetBeans//DTD Editor Fonts and Colors settings 1.1//EN" "http://www.netbeans.org/dtds/EditorFontsColors-1_1.dtd" >
< fontscolors >
< fontcolor name ="character" default ="character" />
< fontcolor name ="error" default ="error" />
< fontcolor name ="identifier" default ="identifier" />
< fontcolor name ="keyword" default ="keyword" />
< fontcolor name ="comment" default ="comment" />
< fontcolor name ="number" default ="number" />
< fontcolor name ="operator" default ="operator" />
< fontcolor name ="string" default ="string" />
< fontcolor name ="separator" default ="separator" />
< fontcolor name ="whitespace" default ="whitespace" />
< fontcolor name ="method" default ="method" />
< fontcolor name ="javadoc" default ="javadoc" foreColor ="ff006666" />
< fontcolor name ="regex" default ="regex" foreColor ="ff660066" />
< fontcolor name ="command" default ="command" foreColor ="ff006633" />
</ fontscolors >
* This source code was highlighted with Source Code Highlighter .In the Bundle.properties file, we describe the human names for the categories:
operator=Operator
string=String
separator=Separator
whitespace=Whitespace
method=Method
regex=Regular Expression
command=Command
...
* This source code was highlighted with Source Code Highlighter .Next, add a sample file with the source code ValaExample.vala to the project. It will appear in the NetBeans Token Color Options dialog. And the last step is to add lines to the layer.xml file for highlighting:
< filesystem >
< folder name ="Editors" >
< folder name ="text" >
< folder name ="x-vala" >
< file name ="language.instance" >
< attr name ="instanceCreate" methodvalue ="org.carbonfx.valaproject.lexer.ValaTokenId.getLanguage" />
< attr name ="instanceOf" stringvalue ="org.netbeans.api.lexer.Language" />
</ file >
< attr name ="SystemFileSystem.localizingBundle" stringvalue ="org.carbonfx.valaproject.Bundle" />
< folder name ="FontsColors" >
< folder name ="NetBeans" >
< folder name ="Defaults" >
< file name ="FontAndColors.xml" url ="FontAndColors.xml" >
< attr name ="SystemFileSystem.localizingBundle" stringvalue ="org.carbonfx.valaproject.Bundle" />
</ file >
</ folder >
</ folder >
</ folder >
</ folder >
</ folder >
</ folder >
< folder name ="OptionsDialog" >
< folder name ="PreviewExamples" >
< folder name ="text" >
< file name ="x-vala" url ="ValaExample.vala" />
</ folder >
</ folder >
</ folder >
</ filesystem >
* This source code was highlighted with Source Code Highlighter .If you did everything right, and I did not miss anything - you can compile and run the project. When opening files with the extension .vala, we see that the keywords are highlighted in the desired color. In the settings dialog, the Vala language should appear, which you can select and view the highlighted contents of the ValaExample.vala file.
Conclusion
After all done, we just got syntax highlighting. But the analyzer, which was written using ANTLR, can be used for other tasks, namely:
- verification of the code validity and highlighting of erroneous constructions;
- automatic indentation;
- hint method names, function arguments.
The current day in the plugin for Vala implemented the first two points. In the future I will try to describe how the project can add support for indents and highlighting of erroneous constructions.
There was also an idea to try to parse the Vala code not using ANTLR, but with the parser, which itself is written in the Vala language. If specifically, then with its implementation, which is used in the Vala compiler - libvala. If anyone is interested in connecting to the finalization of the plugin, write!
Links
- The source code of the project on github , you can also download the compiled module
- ANTLR and ANTLRWorks website
- NetBeans + ANTLR Plugin Tutorial
- NetBeans + JavaCC Plugin Tutorial
Source: https://habr.com/ru/post/124659/
All Articles