Why visual programming and D3NE may be useful to you

Visual programming allows you to describe processes in graphical form, in contrast to the textual representation, where you need to make additional efforts to think in the way that a computer should do. It sounds promising, but let's try to figure out the essence and find out why you should try it.

By itself, programming involves not only the process of writing code, but often it takes up most of the time during development. Just imagine how much effort you have to spend on keeping in mind the many rules and specifications for a particular programming language, instead of focusing on the problem being solved. Especially annoying is the variety of syntax in languages: a semicolon is needed somewhere, curly brackets are not needed somewhere, no expression can do without brackets somewhere at all. What to say about holivarah, reminiscent of religious disputes.

A shot from the series "Silicon Valley"

It is often advised to start developing software with a graphic description of the future system, its components and the connections between them, in order to determine the most advantageous system structure at the early stages and minimize possible problems in the future. A graphical representation is easier to understand than a text version, but it may have its limitations, besides, it will still have to be translated into compiler-friendly code. Of course, this does not apply to small applications (as they may seem at first), you can immediately start writing code, but the problem still remains - you need to think within a certain programming language. Especially when you do it in a long imperative style that has bored you.

Programmers by their very nature should be lazy in order to find profitable ways to solve problems and not waste energy on a routine, especially looking at the trends of increasing software complexity. This is what stimulates the birth of paradigms, programming languages ​​and absolutely new, seemingly little-known tools of visual programming.

Why and where to use visual programming

One of the first tools, more famous and friendly for the ordinary citizen, can be considered Scratch. It is intended solely for educational purposes, as it represents the same blocks of code, only wrapped in multi-colored puzzles. There is no practical use if you already know how to write code.

A similar tool from Google called Blocky

There is another kind of visual programming, more useful in my opinion, this is Data-flow programming. It is not as flexible as the previous one and serves as some kind of add-in for programming processes of a specific subject. Its essence consists in the manipulation of data transmitted between blocks (nodes).

Sample Node Editor from Blender

The advantage of this approach is as follows: you do not need to think about what is happening with the data in the nodes, but only to know what they are doing, get input and give it to the output. It remains only to choose the necessary nodes and connect them with each other by lines, along which one can understand without any difficulty what is connected with what.

The node editor in Blender is exactly the case when programming data streams is used for rendering control, creating shaders and textures. This approach in 3D modeling and rendering is quite popular, since no one wants to write code, and it is necessary to create instructions for nonlinear data management.

Also known to everyone in the community of 3D modelers is the Substance Designer, which allows you to create 3D materials according to the principle described above.

Editor to create material in Substance Designer

Although in official sources there is no mention of programming, it uses a real data-flow. Probably, this is such a marketing move - not to talk about what might scare away.

So, you understand why visual programming is used in 3D modeling, but what about those areas of activity where it’s impossible to make a sufficiently functional product without the ability to write code? It would seem that if you know how to write code, then you just need to develop this skill, but no - it was in geymdev that tools for programming without coding were among the first to be used. On the one hand, it was beneficial for popularizing gamedev among those who are deterred by the code with only one type, on the other hand, progress does not stand still and even programming gurus are tired put spaces instead of tabs reading thousands of lines of code and its support, and they decided to try new techniques for describing logic and processes.

Many people know UE4 with its Blueprint. This is not just a data-flow, but something more, because it allows you to generate instructions that will be executed not in one pass, but throughout the entire life cycle.

Example of displaying a string by event

For example, it has nodes for sending events. The nodes connected to them are executed only at the moment when the event is triggered. It turns out very conveniently, it is not necessary to operate with some objects and methods, but simply to make a connection from the output to the input.

There are many more separate tools for visual programming (more often they are positioned as a graphic programming environment), but they are not so attractive, have a narrow focus, or even more similar to designers than to programming tools.

General aspects of the development of node editors

Suppose once you saw how make magic programmed using some kind of editor with blocks and lines. To your surprise, this turned out to be exactly what will allow you to embody your idea (for example, some advanced dialog generator that can’t be represented as a regular graph).

The main thing that will depend on the development process - the platform and technology. Recently, a lot of things can be done right in the browser, even if sacrificing performance, which in most cases will not affect the quality of the application (otherwise asm.js / wasm to help). One of the features of such applications is that they can be embedded in other native applications. The choice of tools and programming languages ​​should be in the interests of the developer to make the development process more comfortable and efficient and reduce the likelihood shoot yourself in the foot appearance of problems.

Having decided on the platform and language, it would be good to use ready-made solutions (in our case, this is a node editor), but even here everything is not smooth.
There are several options:

1. you are using a ready-made solution, which will soon prove to be not so suitable and convenient, in addition to this, to you laziness unable to tinker with it
2. write your own implementation, at the same time examining how it all works from the inside and where to fasten the wheels .
3. use a ready-made solution that is quite flexible and allows you to customize it for yourself without any problems, without sacrificing profitable features for your product.

The problem of the latter is that this is a rather rare phenomenon, arising after a lot of effort. Actually, the essence of this article is to stimulate the achievement of such a result on the example of the library presented below.

What is D3NE

D3 Node editor (D3NE) is a JS library for creating node editors for solving visual programming problems. The library completely undertakes work on display and processing of nodes. Uses D3.js and Angular Light

A tutorial is available on the GitHub Wiki , as well as a detailed description of all the components and terminology used in the library (English wikis and grammatical errors are possible, so I will be happy to help with their correction)

It all started with a single project (as it lies in the draft), in which the idea appeared to apply something similar to UE4 Blueprint, only with the correction that you just need a data-flow. Among the ready-made open source solutions on JS were not suitable: the criteria were the appearance and the necessary features (for example, different types of sockets to limit the connection of incompatible outputs and inputs). If the first one could somehow be corrected from the outside, the addition of the second one could cause problems, especially if the structure and source code leave much to be desired.

And so it began writing code and shaping it as a separate library. Yes, yes, it is the writing of the code, without planning and design, although it was not known in advance what it would ultimately be like (because of which I had to correct my mistakes more than once).

Fortunately, a few months after the start of development, it was possible to reach a fully functional version with the necessary components to provide flexibility in creating editors. But this does not mean that this is the final. Rather, the benchmark, after which you need to strive to improve what is already there, and introduce the necessary capabilities.

How to apply in your project

First you need to find out if you need it. Of course, trying something new for yourself will never hurt (if it doesn’t hurt your health :)), but it’s worth thinking about when solving real problems.

It is better to see once than ... in general, see examples:

• Data-flow programming - an example of the addition of numbers
• Events and conditional data management - something like this is done in UE4
• Customization - the ability to change not only styles, but also the structure of nodes and the way of connections
• Allmatter is a project based on this library.

UE4 style customization example

Consider the basic components that are needed to create a minimally usable editor.

Sockets

These are connectors that can be inputs or outputs and are necessary to ensure that they are properly connected to each other. By assigning this socket to the inputs and outputs, you can be sure that the user will not be able to transfer the wrong data there (or rather, he only assigns connections). It is also possible to combine sockets for connection between the inputs and outputs of different types.

Knots

Nodes are usually rectangles (but you can change this within your imagination and CSS and HTML capabilities). They can have entrances, exits and controls. The inputs and outputs represent the sockets and are located on the left and right side of the node, respectively (again, you can arrange them as you like, using templates ).

Controls

Controls are needed to add arbitrary elements to nodes, and from them you can assign data to the node at the time of working in the editor (not during data processing). For example, using a simple input field, you can put a value in a node, which can then be used during processing.

Components

Components are needed in order for the editor to know which nodes can be added in it and how to process them, namely, they are necessary to support import / export. The so-called builders and workers are used, the usual functions with the corresponding key, which are set when the component is created (and must be in the same place).

The builder is responsible for creating an instance of the node - in the function you create an instance, add the necessary inputs, outputs and controls to it. This is created by software, and not through configs (usually found such implementations) in order to allow reuse, for example, of controls, and even the inheritance of nodes.

A worker as parameters gives you inputs, outputs, and an object with node data (not an instance that is created in the builder, but something that was obtained from it through toJSON ). The input array contains data that came to the input from another node and an array with output data into which you must put the data, depending on what the node should do)

Editor

One of the main objects that manages all of the above for the purpose of displaying and interacting with the user. The identifier, components, context menu are transferred to it.

Engine

Serves to process data in nodes. The peculiarity is that it needs only the data exported from the editor, thus it is possible to perform processing of the nodes without an editor at all (for example, there is its implementation in C ++). It is this component of the library that is responsible for traversing the nodes, determining which one to start processing, calls your workers, passing them the input data received from the previous nodes. It can work with asynchronous functions, which can not but rejoice against the background of support from async / await browsers. Moreover, due to asynchronous processing it can execute several nodes in parallel.

Conclusion

As you can see, visual programming, although it is intended to replace the writing of code, is rarely used at the moment. Probably because many are skeptical about the idea of ​​programming with a mouse. This is partly true, since most of the tasks in software development will be more familiar to solve in any programming language than using an insufficiently proven tool.

The essence of progress is to find more profitable ways to solve any problems. So in the case of visual programming, it is possible to describe processes in an easy-to-understand presentation and a sufficient level of abstraction. But at the same time, you cannot get rid of the source code, as it is the basis for all this.

To make sure of this and check, as they say, all the features described above, you can build your own visual programming editor based on D3NE.