Using Roslyn to edit game content

Chatter is worth nothing. Show me the code.
- Linus Torvalds

Hello! I work as a programmer in a small (but proud) gamedev-office. In the past few years, the company has been producing casual games for mobile phones in the match3 genre. We write on C # (which is good news) and do not use Unity (well, almost). It so happened that the main area of ​​my responsibility is gameplay and UI. And I'm also crazy about C # 's and its ecosystems. Today I want to tell you how I managed to use the Roslyn code analysis and modification tool to edit game content. Who cares - I ask under the cat.

Note. Sections that understand the implementation method and provide code examples that are marked with [technical section] in the headers. If you do not want to dive into the details at this level, just skip them. This will not affect the general understanding of the idea.

Prehistory One match3 will not be full

In many games of the match3 genre, there is a separate entity that represents the game world. We call it a card. The card’s functionality may be different: from very simple (an indicator of progress in the game) to a full-fledged action scene (with its own internal story, plot, characters, etc.).

In one of the projects I'm working on, special attention is paid to the plot and map. Enough requirements: a large amount of content, variety, excitement, well-directed cut-scenes, etc. It should also be easy to edit and expand. Until now, there was nothing like this in the game projects of our studio. Both the plot system and the tools for it had to be developed almost from scratch.
')

Plot architecture [technical section]

We divide the whole story into separate gaps - quests. A quest is a set of stages (states), each of which has its own logic. Let's look at the Stub-quest code:

enum StubQuestState { Initial, } class Stub : SectorComponent<StubQuestState> {} class StubQuest : SectorBehaviour<Stub, StubQuestState> { [State(StubQuestState.Initial)] private IEnumerator<object> InitialState() { yield break; } } 

So we see:

• enum StubQuestState - quest status;
• class Stub - quest component, stores state and optional data;
• class StubQuest - logic of the quest. Each state of the quest must correspond to a method that returns an iterator (the method is linked through the State attribute and determines the actions that are performed directly in this state).

Also, each quest has an activation condition - when it should be launched (in the code it is represented as Func and is called ReachedCondition).


Quests are divided by locations, in our terminology by sectors (static classes with names like SectorN). Each sector has the Initialize method, initializing the map (here I mean graphics, animations directly), tutorials (some of them are not implemented through quests and are separate entities) and the quests themselves. Here is an example of the initialization code:

 var fixSubmarineQuest = new FixSubmarineQuest { ReachedCondition = () => plot.Levels.Completed > 3, }; var removeSeaweedQuest = new RemoveSeaweedQuest { ReachedCondition = () => fixSubmarineQuest.Component.IsFinished, }; var fixGateStatuesQuest = new FixGateStatuesQuest { ReachedCondition = () => removeSeaweedQuest.Component.IsFinished && fixSubmarineQuest.Component.IsFinished, }; 

Visualize the link quests:



As we can see, quests form a disconnected acyclic directed graph.
Let us examine the logic of the state of the quest. Each state is in its essence a sequence of actions that must be performed in this state. Actions can be completely different: activate animation, move the camera around the scene, launch the game level, launch and control the cut-scene, etc. Example:

 yield return WaitForMapScreenOnTop(); MapScreen.HideUi(); yield return MapScreen.ScrollToMapNodeTask( MapScreen.Map._Sector02._RadioTower.It, zooming: MapZooming.Minimal, screenPivot: new Vector2(0.5f, 0.4f) ); yield return MapScreen.Map._Sector02._RadioTower.RunAnimationFixTower(); yield return MapScreen.Map._Sector02._RadioTower.RunAnimationCommunicate(); var dialog = new CutScene(CharacterEmotion.Playfulness); dialog.AddBubble("Bla-bla-bla"); yield return Task.WaitWhile(() => !dialog.IsNearlyHidden); MapScreen.ShowUiWithDelay(); SetState(FixCommunicationCrystalQuestState.Final); 

In this example, we:

1. We are waiting until the top dialog is a map.
2. Hide the UI and move the camera.
3. Successively launch plot animations.
4. Show a cut-scene with a happy character saying “bla-bla-bla”.
5. Show UI and go to the next state.

Roslyn and content

As you know, gamedev is a rather extreme software development area. Not only is the game a “hodgepodge” of systems and components, so also the relationships between them change with lightning speed (and sometimes even according to the principle of a pendulum: remove, return, remove half ...). At the stage of active development, the plot is constantly undergoing changes. This affects all levels of the described architecture: the relationship between quests (their order), and the division of quests into sectors, and directly the logic of the quest (state and action). I honestly got tired of writing everything by hand (or doing Ctrl + C, Ctrl + V, and then renaming, etc.), so first I wrote a small WinForms application to generate StubQuest with the specified name and states. And then I remembered Roslyn and decided to create a plot editing tool.

As I already described above, the whole plot is set by code. Without programming skills you will not edit. To provide the ability to change all the above components (quests, order, logic, actions) to other people (producers, game designers), and even to work many times faster, an editor was created, an important part of which is Roslyn. The principle of its work:

1. Using Roslyn (in particular, Microsoft.CodeAnalysis.CSharp), the game code is analyzed and a logical model is built.
2. The logical model is displayed in the diagram.
3. Chart changes are displayed in the model, and then (again through Roslyn, but using the API for generating and modifying code) back to the code base.

Schematically, this can be expressed as:



The logical representation is quite simple in itself - a set of classes / structures representing game entities, various data with which to describe / supplement them, and the connections between these entities. Thanks to the logical model, it is easy to check the correctness of behavior (for example, the presence of cycles in the graph or the forgetfulness of the developers to show the UI after the cut scene). Let's talk about code analysis and building a logical structure in more detail.

Code analysis and model building [technical section].

What is a code? For starters, it's just text. Then, at the parsing stage, AST ( Abstract Syntax Tree ) is obtained from it. After the semantic analysis, a symbol table appears. Using Roslyn, getting the required data structures is easy (an example is taken from here ):

 var tree = CSharpSyntaxTree.ParseText(@" public class MyClass { int MyMethod() { return 0; } }" ); var Mscorlib = MetadataReference.CreateFromFile(typeof(object).Assembly.Location); var compilation = CSharpCompilation.Create( "MyCompilation", syntaxTrees: new[] { tree }, references: new[] { Mscorlib } ); //Note that we must specify the tree for which we want the model. //Each tree has its own semantic model var model = compilation.GetSemanticModel(tree); 

So, we have the code in a form suitable for analysis. It remains to link this with the logical structure of the plot. The project’s file structure and the architecture described above help us in this:

 public void LoadSectorCode(string dirPath) { var sectorFile = Directory.EnumerateFiles(dirPath) .FirstOrDefault(p => new FileInfo(p).Name.StartsWith("Sector")); if (sectorFile == null) { throw new SectorFileNotFoundException(); } code.ReadFromFile(Path.Combine(dirPath, sectorFile), CodeBulkType.Sector); var questsDir = Path.Combine(dirPath, "Quests"); if (Directory.Exists(questsDir)) { var files = Directory.EnumerateFiles(questsDir); foreach (var filePath in files) { code.ReadFromFile(filePath, CodeBulkType.Quest); } } } 

Each file in the codebase is associated with a type (enum CodeBulkType): quest, sector, configuration file, etc. Knowing the type, you can perform a higher level analysis. For example, read quest data:

 class QuestReader : CodeReader { public override CodeBulkType[] AcceptedTypes => new[] { CodeBulkType.Quest, }; public override void Read(CodeBulk codeBulk, Code code, ref Flow flow) { var quest = FromCodeTransformer.ReadQuest(codeBulk.Tree); //... sector.Quests.Add(quest); //... } } 

For the analysis and editing of the syntax tree, the Visitor pattern, or LINQ , is actively used. The latter is rather inconvenient, suitable for very simple analysis. In the project code, I actively used the SyntaxVisitor , SyntaxWalker and SyntaxRewriter classes. Use-case of the first two - analysis of the syntax tree. For example, a quest can be added to the plot, or just lie quietly, do not touch anyone. To determine this characteristic you need to look, the quest constructor is called somewhere. With Roslyn, this is quite simple: inherit from SyntaxWalker and “visit” all ObjectCreationExpression:

 // SyntaxWalker -  CSharpSyntaxWalker'a  .  private class QuestInitializationFinder : SyntaxWalker { //... public override void VisitObjectCreationExpression(ObjectCreationExpressionSyntax node) { base.VisitObjectCreationExpression(node); var model = Code.Compilation.GetSemanticModel(node.SyntaxTree); if (model.GetTypeInfo(node).Type == questTypeToFind) { //…. } } //... } 

Display of the constructed model.

So, we built the model. It remains to display it, and then learn to edit.

Model mapping was implemented using the open NShape library. This project turned out to be the easiest to learn and has rich functionality (although sometimes you want more opportunities for expansion ...). The Sugiyama algorithm from the Microsoft Automatic Graph Layout library was used to locate the vertices of the quest graph , and I also had to write a few heuristics.

I will give an example of the quests of the first sector:



It looks much more user-friendly than the initialization code, right? (This is a very simple example, where there are few quests and they are initialized in a linear manner.) But the display of the logic of the quest (the mode “for programmer” is to show the code; in the mode “for all others” the verbal description of the action is shown):

Editing model [technical section].

The NShape library is intended for creating and editing diagrams. Displaying the logical structure in the form of a diagram, we have the possibility of its (structure) visual editing. The main thing is to correctly interpret the changes and to prohibit invalid.

The translation of actions on a diagram into logical ones takes place in special wrappers over the NShape library tools (writing this code was still a pleasure). Only actions that carry a certain meaning in the context of a logical model are transformed (for example, the rotation of a shape denoting a quest does not imply any structural change in our display method). All actions are encapsulated in objects of type Command (did you recognize the pattern ?) And are reversible.

 delegate void DoneHandler(bool firstTime); delegate void UndoneHandler(); interface ICommand { DoneHandler Done { get; set; } UndoneHandler Undone { get; set; } void Do(); void Undo(); } 

This interface (and the abstract class implementing it) is responsible for changing the logical model and code. Updating of visual representation occurs through a subscription to Done / Undone delegates.


For editing syntax trees, the extensions of the CSharpSyntaxRewriter class are used, as described above (the technique is well described here ). An example of deactivating a quest (removing a constructor call):

 class DeactivateQuestCommand : Command { //… public override void Do() { var classDecls = codeBulk.Tree.GetRoot().DescendantNodes().OfType<ClassDeclarationSyntax>(); foreach (var classDecl in classDecls) { var typeRemover = new ClassConstructorCallRemover( Context.CodeEditor.GetSymbolFor(classDecl, codeBulk), Context.Code ); Context.CodeEditor.ApplySyntaxRewriter(typeRemover); } //... } //... } class ClassConstructorCallRemover : SyntaxRewriter { //… public override SyntaxNode VisitExpressionStatement(ExpressionStatementSyntax node) { var model = Compilation.GetSemanticModel(node.SyntaxTree); var expr = node.Expression; if (expr is ObjectCreationExpressionSyntax oces) { if (ReferenceEquals(model.GetTypeInfo(expr).Type, typeToRemove)) { return null; } } return base.VisitExpressionStatement(node); } public override SyntaxNode VisitLocalDeclarationStatement(LocalDeclarationStatementSyntax node) { if (node.Declaration.Variables.Count == 1) { var model = Compilation.GetSemanticModel(node.SyntaxTree); var type = model.GetTypeInfo(node.Declaration.Variables.First().Initializer.Value).Type; if (ReferenceEquals(type, typeToRemove)) { return null; } } return base.VisitLocalDeclarationStatement(node); } //... } 

I want to note that some of the editing features Roslyn provides out of the box. A vivid example is renaming ( Renamer ) and formatting spaces ( Formatter ). And it greatly simplifies life.
Up to now, we have dealt only with deleting / changing the already existing code. And what about the generation of a new one? This is done simply - via SyntaxFactory .Parse * and node modification functions:

 var linkExprText = $"ReachedCondition = {newLinkText}"; var newInitializer = node.Initializer.AddExpressions(SyntaxFactory.ParseExpression(linkExprText)); 

As you can see (if you carefully looked at the second line), the AddExpressions function returns us a new node. In general, the whole Roslyn follows a functional approach in architecture: if you want to change something, create a new one with the required parameters. At first, it is inconvenient, and then you begin to love it.


To write such code, you need a good knowledge of grammar (what syntactic units you need to check / edit / create) and Roslyn API. But after you get everything you need, programming with Roslyn turns into one pleasure (especially in architectural terms - I'm a big fan of beautiful code and functionalism).

Editor features and tradeoffs. A practical example.

Up to this point little has been said about the editor and its functionality. Let's fix the situation: let's discuss, in the end, what exactly can be edited, what compromises had to be made to achieve a balance between the wealth of features and complexity, and then consider an example from real practice (albeit somewhat simplified).

So, decided to edit the code. We admit to ourselves that the wording is rather scary. Here people do not always write as they should, but we need to write a program that will analyze and change other code. And not necessarily written by us. No restrictions can not do.

The first thing I want to describe is the behavior of the editor in case of detection of code that he doesn’t know how to interpret: some internal affairs of programmers, hacks, new features, about which it is impossible to say for sure whether they will enter the project or not, etc. . This situation must be “resolved” very carefully. Otherwise, the tool will do more harm than good.

How do we solve this problem? Do not play with what you do not understand . The term is referring to the knowledge of the program about the logical value of a piece of code. Too abstract? Here are a couple of examples.

The first example. The quest must be activated not only after another, but also after three levels passed. Or five. And even after the user was given a daily reward. In general, there is some additional condition. From the editor, to set such a condition in the general case is impossible - the intervention of the programmer is required. Okay, the programmer finally got free and added a couple of lines to the code. What's next? First, the editor displays this in a diagram:



The user immediately sees that everything is not so simple with this quest. Secondly, it is forbidden to edit the unknown (not parsed by the editor) conditions. The maximum that we can do is to read its source code and the comment left to it (a useful function - allows the programmer to explain the meaning of the code to the uninitiated). Thirdly, it is only allowed to add new conditions (and, of course, to edit and delete them). Those. the condition may become stricter , and as much as the editor allows.

An example of the second. Unassembled code in the logic (actions) of the quest. Probably the most common situation. The philosophy here is the same: do not give to edit and unobtrusively show a sign that says WARNING (in general, there is a tick to hide / show the unparsed code).

The second thing to discuss is editing the flow of execution. For example, in one of the quests there is a part of logic, which in itself is optional and is launched only according to a definite and undetectable condition. It would be foolish to ban its editing altogether. Therefore, it was decided to leave this possibility to the user, but with some reservations: no editing of the condition of occurrence is allowed (conditions in if, while), as well as moving the whole block in the general case (yes, the bitter truth - moving can break the compilation or what is worse, it is not obvious to change the logic). Need to move the logic? Two options: either turn off the block and in the right place create a new one, or give the task to the programmer.

Now consider a simple example for illustration. The game designer gets the task: to insert the TalkToLeeroy quest between FindKey and OpenTheDoor quests. In the quest, we show the cut-scene, wait for the direct passage of the quest, then show the second cut-scene and complete the quest. Algorithm action game designer (from his face):

1. We find quests, between which it is necessary to insert a new one:
   
2. Add a new quest with the desired name, throws connections:
   
3. Open the quest editing window, add the required states:
   
4. Then we fill each state with the necessary action. For example, the Initial status task is to show the first cut-scene:
   
5. By analogy, the remaining states are filled.
6. If errors are suddenly detected (for example, we have hidden the interface and forgot to show it), we are warned about this visually (with special marks on the diagram). We correct errors.
7. We start the game, make sure that everything works as it was intended.

So, the task is completed, the joyful designer calmly sends it for testing and leaves to drink tea.

The purpose of the creation of the editor was rapid prototyping and content filling. In the end, only the programmer, the little king and god of the virtual world, has absolute power over the logic of the game. It is cheaper (and, as a result, more expedient) to give him the task of polishing / refining the game functionality, rather than complicating the tool a lot. And then so before such a situation is not far:

Conclusion

I described not all. I had to omit some points (analysis and visualization of errors, operations with sectors, etc.). If there is interest - write in the comments, perhaps I will write another article. The purpose of this was to show that even such tools, which, it would seem, are poorly related to gamedev, can be successfully applied in practice.
What was finally done:

1. Work speeds up at times
2. Improved structure and architecture of the game
3. The author learned Roslyn, deepened knowledge of C #, improved design skills

Would I recommend Roslyn to learn and use? Of course. Be patient and go.