Battle "Listener vs Visitor" at the stadium antlr4

Attend or listen? A matter of taste - no more. Or not?
Prehistory

Having examined the source code, a tree was formed at the output:


')
In itself, the tree does not have any meaning, it is “Wooden”, the result of the analysis (bypass) of this tree has meaning and value. For those who are not ready to strain and write samopisnye sleigh on descent from the tree (for example, me) in antlr4 added the ability to get the analyzer almost for free.

1. Visitor

Classic is a behavioral design pattern. When traversing nodes, the method that processes the current node type is determined, after which the method is called and this is where the development begins, namely the analysis of the incoming subtree.

2. Listener

Innovation, which appeared in the fourth version. The behavior of this class is already far from classical (Observer or Publish / Subscribe). In the classic version there is a manager who notifies subscribers about the occurrence of events. The behavior of the listener in question is more like an inspector’s job. The inspector makes a note “I check the X node” before checking the node, then follows the descendants of the node, after the traversal, which can be done “Conclusion on the results of the X traversal”.

Practice

For a better understanding of what is happening, let's turn to the author of the recognizer pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference . In the book “The Definitive ANTLR 4 Reference” In section 8.2 Translating JSON to XML, the author translates using the Listener.

The examples in the book are based on JAVA, I am not familiar with JAVA, but they are translated to C # without pain (that's what best practice means cloning).

To prepare the listener, we need VS, C # proj and JASON.g4 with something like this

grammar JASON; json: object | array ; object : '{' pair ( ',' pair )* '}' # AnObject | '{' '}' # EmptyObject ; pair: STRING':'value; array : '['value(','value)*']' # ArrayOfValues | '['']' # EmptyArray ; value : STRING # String | NUMBER # Atom | object # ObjectValue | array # ArrayValue | 'true' # Atom | 'false' # Atom | 'null' # Atom ; STRING : '"' ( ESC | ~["\\] )* '"'; fragment ESC: '\\'(["\\/bfnrt]|UNICODE); fragment UNICODE:'u'HEX HEX HEX HEX; fragment HEX:[0-9a-fA-F]; NUMBER : '-'? INT'.'INT EXP? //1.35,1.35E-9,0.3,-4.5 | '-'? INT EXP //1e10-3e4 | '-'? INT //-3,45 ; fragment INT: '0'|[1-9][0-9]*;//noleadingzeros fragment EXP: [Ee][+\-]? INT;//\-since-means"range"inside[...] WS : [\t\n\r]+->skip; 

This is a grammar allowing to recognize JASON. In the properties of the file, you must set Generate Listener and Generate Visitor (this is still useful). The result of the Listener’s work in the original example from the book is the xml text, I’m not satisfied with it, I’ll get the XElement (anyway, the xml text will need to be translated into something, although the plus text is that it doesn’t pinch into the framework of using specific classes).

The algorithm is simple: if antlr4 uses descending parsing (in our case from the root to the nodes), then xml will be formed in the same way, creating an ancestor element to which descendants will be added.

Example Listener:

  class XmlListener : JASONBaseListener { #region fields ParseTreeProperty<XElement> xml = new ParseTreeProperty<XElement>(); #endregion #region result public XElement Root { get; private set; } #endregion #region xml api XElement GetXml( IParseTree ctx ) { return xml.Get( ctx ); } /// <summary> ///   xml  /// </summary> XElement GetParentXml( IParseTree ctx ) { var parent = ctx.Parent; XElement result = GetXml( parent ); if ( result == null ) result = GetParentXml( parent ); return result; } void SetXml( IParseTree ctx, XElement e ) { xml.Put( ctx, e ); } #endregion #region listener public override void ExitString( JASONParser.StringContext context ) { var value = GetStringValue( context.STRING() ); AddValue( context, value ); } public override void ExitAtom( JASONParser.AtomContext context ) { var value = context.GetText(); AddValue( context, value ); } public override void EnterPair( JASONParser.PairContext context ) { var name = GetStringValue( context.STRING() ); XElement element = new XElement( name ); XElement ParentElement = GetParentXml( context ); ParentElement.Add( element ); SetXml( context, element ); } public override void EnterJson( JASONParser.JsonContext context ) { Root = new XElement( "JSON" ); SetXml( context, Root); } #endregion #region private private string GetStringValue( ITerminalNode terminal ) { return terminal.GetText().Trim( '"' ); } private void AddValue( ANTLR_CSV.JASONParser.ValueContext context, string value ) { var parent = GetParentXml( context ); if ( context.Parent.RuleIndex == JASONParser.RULE_array ) { XElement element = new XElement( "elemnt" ); element.Value = value; parent.Add( element ); SetXml( context, element ); } else parent.Value = value; } #endregion } 

EnterJson corresponds to the entry to the node described in the grammar as follows:

 json: object | array ; 

ExitString corresponds to the output from the node described in the grammar as follows:

 STRING # String 

In contrast to the original example, I do not use all the charms of Enter and Exit. For that is, ParseTreeProperty is recognized to store pairs [subtree, value], it is probably better to replace it with a regular dictionary (it definitely won't be worse).

Visitor Sample:

 class XmlVisitor : JASONBaseVisitor<XElement> { #region fields private XElement _result; ParseTreeProperty<XElement> xml = new ParseTreeProperty<XElement>(); #endregion #region xml api XElement GetXml( IParseTree ctx ) { return xml.Get( ctx ); } XElement GetParentXml( IParseTree ctx ) { var parent = ctx.Parent; XElement result = GetXml( parent ); if ( result == null ) result = GetParentXml( parent ); return result; } void SetXml( IParseTree ctx, XElement e ) { xml.Put( ctx, e ); } #endregion #region visitor /// <summary> ///    -   xml /// </summary> protected override XElement DefaultResult { get { return _result; } } public override XElement VisitJson( JASONParser.JsonContext context ) { _result = new XElement( "JSON" ); SetXml( context, _result ); return VisitChildren( context ); } public override XElement VisitString( JASONParser.StringContext context ) { var value = GetStringValue( context.STRING() ); AddValue( context, value ); return DefaultResult; } public override XElement VisitAtom( JASONParser.AtomContext context ) { var value = context.GetText(); AddValue( context, value ); return DefaultResult; } public override XElement VisitPair( JASONParser.PairContext context ) { var name = GetStringValue( context.STRING() ); XElement element = new XElement( name ); XElement ParentElement = GetParentXml( context ); ParentElement.Add( element ); SetXml( context, element ); return VisitChildren( context ); } #endregion #region private private string GetStringValue( ITerminalNode terminal ) { return terminal.GetText().Trim( '"' ); } private void AddValue( ANTLR_CSV.JASONParser.ValueContext context, string value ) { var parent = GetParentXml( context ); if ( context.Parent.RuleIndex == JASONParser.RULE_array ) { XElement element = new XElement( "elemnt" ); element.Value = value; parent.Add( element ); SetXml( context, element ); } else parent.Value = value; } #endregion } 

As the saying goes, “Find the 10 differences”, the first difference of VisitJson, managing the visit without calling VisitChildren (context), the descendants' visit stops, and hence the detour. Each of the visiting methods should return a value, that is, there is always a result of the visit, and this is convenient:

 var result = visitor.Visit( tree ); 

When working with a listener:

 walker.Walk( listener, tree ); var result = listener.Root; 

In the original example, without the Listener, it would be rather tight, for this solution there is not much difference, but I vote for the decision on the Visitor.

Well, so that you can try it yourself:

  private static IParseTree CreateTree() { StringBuilder sb = new StringBuilder(); sb.AppendLine( "{" ); sb.AppendLine( "\"description\":\"Animaginary server config file\"," ); sb.AppendLine( "\"count\":500," ); sb.AppendLine( "\"logs\":{\"level\":\"verbose\",\"dir\":\"/var/log\"}," ); sb.AppendLine( "\"host\":\"antlr.org\"," ); sb.AppendLine( "\"admin\":[\"parrt\",\"tombu\"]," ); sb.AppendLine( "\"aliases\":[]" ); sb.AppendLine( "}" ); AntlrInputStream input = new AntlrInputStream( sb.ToString() ); JASONLexer lexer = new JASONLexer( input ); CommonTokenStream tokens = new CommonTokenStream( lexer ); JASONParser parser = new JASONParser( tokens ); IParseTree tree = parser.json(); return tree; } 

The sample JSON text is taken almost unchanged from the original example:

  private static void ListenerXml() { IParseTree tree = CreateTree(); ParseTreeWalker walker = new ParseTreeWalker(); XmlListener listener = new XmlListener(); walker.Walk( listener, tree ); var result = listener.Root; } private static void VisitorXml() { IParseTree tree = CreateTree(); XmlVisitor visitor = new XmlVisitor(); var result = visitor.Visit( tree ); } 

Well, the result of the implementation:

 <JSON> <description>Animaginary server config file</description> <count>500</count> <logs> <level>verbose</level> <dir>/var/log</dir> </logs> <host>antlr.org</host> <admin> <elemnt>parrt</elemnt> <elemnt>tombu</elemnt> </admin> <aliases /> </JSON> 

Oddly enough, but both methods gave the same thing.

Further

PS Listener vs Visitor - 0: 1