The use of procedural generators in creating content for real-time 3D applications: Part 1. Oil Rush

Oil Rush game

This article, which consists of two parts, will focus on using the capabilities of procedural generators to create content for the computer game Oil Rush and the Valley benchmark (to be released in February), developed on our own Unigine engine . All of the following is solely the opinion of the authors, based on personal experience, and does not pretend to absolute truth and completeness of coverage.

In addition to the theoretical part, describing the process of creating content and the problems that have arisen in the series of articles, we present some of the source materials we developed (procedural textures). All these materials are available for download. They are given not only for familiarization - they can also be freely used (with or without changes) in their projects, but only not sold in a pure form and / or as part of any libraries.
')

The authors of the article decided that the files provided for downloading should not be subjected to any special preparation, processing or optimization - in order not to embellish anything. All this is taken from real combat production conditions with its tight time constraints and constant alterations to the requirements of the customer (whether it be art director, producer or publisher). We sincerely believe that those who find these files useful for their project will be able to easily figure them out on their own and redo them at their discretion.

The problems of traditional methods of creating textures

Photoshop layers hell

In most cases, when creating textures, artists use two methods or a combination of them. The first method is based on the use of real photographic images with subsequent retouching and minor adjustments. The second is to create a texture completely from scratch using a graphics tablet. These methods, as well as their combination, have one major drawback - the inability to quickly iterate the texture options. In both cases, such textures are deprived of even the minimum parameterization of the image and, as a result, the possibility of quick change by editing these parameters. As a result, even minimal changes in the diffuse texture entail the need to make appropriate changes in all related textures, and for modern graphic content this is at least a normal and specular map. The maximum that can be done quickly is to change the brightness, contrast and color of certain areas.

In addition, the creation of textures with these methods comes down to routine work: a lot of brush strokes, endless fingering of photo textures in search of suitable ones, retouching to correct unwanted elements, making the borders of images seamless, etc. However, all this routine work can be avoided by competently applying, where possible, procedural texture generation.

Procedural texture generation is a method of creating a texture, in which the color of each pixel of an image is formed algorithmically. This makes it possible to use various mathematical functions, such as Perlin noise or Voronoi mosaic (Fortune's algorithm).

Benefits of procedural textures

An example of a filter tree in Filter Forge

Compared to traditional methods, procedural texture generation has the following significant advantages:

Ability to make changes to any stage of image formation (non-destructive editing). Almost all applications related to procedural generation are built on a block architecture (node-based), which allows you to make logical chains of any complexity by adding elementary blocks and creating connections between them. This architecture retains all the links between resources and the sequence of texture formation, allows the use of the same resource (source texture or output from any block) as input for parallel use by many subsequent blocks, allows you to switch to different color spaces anywhere ( for example, disassemble / assemble a picture on RGB or HSL components). As a result, even quite complex graphs are easily readable, and thanks to this, an artist, even a non-author of such a graph, can easily figure out the process of forming the final picture and modify it to fit his needs as soon as possible.
The ability to increase or decrease the size of the texture without loss of detail. Whatever resolution we choose in the future, the texture clarity will not suffer from this (if we increase the size, the texture will not be blurred by any methods of resampling images), since procedural generation is based on mathematical functions, and not on raster data. For example, if we have a procedural texture, which is based on the function of fractal noise, then when changing the size of such a texture, only the number of solutions of the equations of fractal noise will change to determine the color of each pixel. In this regard, you can easily choose the size of the texture under the current scale of the 3D model for which it is created.
High speed of creating new content based on existing developments by modifying and combining them.
Fast creation of similar textures. After the procedural generator has been implemented, which forms the necessary texture, in order to obtain a similar one, it is only necessary to change the input value (seed) of the random number generator. Thanks to this, for example, by creating one texture generator for one bronze statue, it will be possible to create unique textures for an arbitrarily large number of such statues.
The ability to quickly view several texture variants generated by the algorithm by changing the input value (seed) of the random number generator and select the most suitable one.
Quick change of the final image by changing the parameters of the blocks at any stage of work.
The ability to use procedural generators to create a basic texture with its further rasterization and processing already in raster image editors.
Free seamless. As a rule, procedural texture generators have automatic tiling algorithms (seamless tiling), which allow to remove seams along the edges of an image by installing just one tick in the parameters.
The ability to form images in the HDR (High Dynamic Range) range with further adjustable conversion to LDR (Low Dynamic Range). In the case of procedural textures, we are not limited to the pixel depth, and calculations are performed in floating point numbers.
Convenient processing of multiple images. In case you need to process a group of pictures using some identical algorithm, you are not limited to the filters and actions available in the application. You can implement the required algorithm and apply it to all source images.
The small size of the stored data, which does not depend on the resolution of the image. Only the description of the algorithm, occupying several kilobytes, and not rasterized data of 4 bytes per pixel in each layer is saved to the file.
Saving the memory occupied by the application, since the same texture can be used as input in several handlers at once, and not copy it every time.

In addition to the above, I would also like to note that there are tasks for which the creation of textures in linear image editors like Photoshop or GIMP becomes extremely unacceptable or even impossible. One of these tasks is the creation of landscape textures (terrain). The key feature of landscape textures is their huge size - for example, 8192 * 8192 pixels (the Unigine engine allows you to use very high resolution textures to create landscapes in real time, up to 65536 * 65536 pixels). Drawing all the necessary textures of the landscape manually using a graphics tablet will take weeks, and making at least some minimal changes will take days. All this is completely unreasonable, if your project has at least some time limitations, not to mention the elementary inconvenience of working with textures of this size.

Without the use of procedural tools, the process of forming all the necessary textures of the earth's surface is as follows:

In a specialized sculpting application, a high-poly landscape of the required size is created.
All the necessary textures are baked: height maps (height), normals (normal), ambient occlusion, various masks (for example, a water surface mask).
All generated textures are imported into one document (Photoshop / GIMP) by layers.
For baked textures, masks are formed for various types of surfaces - mountains, stones, grass. At this stage, we are faced with the disadvantages of linear image editors:

The absence of the concept of reference (reference) to the image and, accordingly, the inability to use such links as instances (instances) in new layers. The introduction of Smart Objects in Photoshop has slightly improved the situation, but still it looks more like a stub solution than a universal solution (for example, there is still no normal way to use Smart Object as a mask on any layer). Therefore, it is necessary to repeatedly copy the same rasterized images several times, to memorize all the connections and dependencies between them and, in the event of a change in the original image, to completely produce the whole process again.
Very often there is a need to use a separate set of layers not only for the color, but also for the formation of the alpha channel of the image. It lacks the ability to use the alpha channel, thus formed as a mask for other layers. It is necessary to use solutions-stubs of the type of “nested in each other layer with masks”, which are not applicable everywhere.

Working with such a bulky file, even on powerful computers, causes problems with the lack of memory, constantly used hard disk, very slow work due to the huge amount of computation. This forces you to close other used programs, reduce the number of cached Undo steps in applications, split the created texture into several files according to types of surfaces (water, sand, grass), work with each of them separately, rasterize and only after that finally assemble in one file . In such conditions, work becomes uncomfortable, which can not but affect the quality of the result.

Photoshop, being the de facto standard of a texture application, unfortunately, remains a tool only for photographers and retouchers. Until now, there have been no significant improvements aimed at introducing non-destructive editing and parameterization of images, which greatly complicates the work not only for content creators for 3D graphics, but also for interface designers, 2D artists, and web designers.

Since its inception, GIMP (even though it is forgivable as an absolutely free application) has been trying to copy the functionality of Photoshop, while implementing it on an even more inconvenient interface instead of trying to go its own way and implement a more viable non-destructive approach to image editing.

In our opinion, the main problems of raster image editors at the moment are:

The inability to reuse existing images in several places (the reference / instance mechanics are not implemented).
The inability to create and separate editing layers for masks.

Procedurally generated content in the game Oil Rush

We encountered the problem of procedural content generation, in particular, when working on the Oil Rush marine strategy, which takes place in a flooded post-apocliptic world. In the process of creating content for the levels of the game Oil Rush, we were faced with the task of creating a realistic natural environment in 6 different settings: the Arctic Ocean, fishing villages in the tropics, settlements in canyons, an industrial zone, destroyed cities and reefs, which differ in relief, climate and type buildings. In order to make the content look natural and diverse, when creating it, we used the capabilities of procedural generators.

Before solving any problem of procedural content creation, you should think about the physical processes of the object or object being modeled, simplify it as much as possible and select only the key parameters that determine its visual component.

At the pre-production stage of the game, we tried almost all the procedural generators that existed at that time, and among them was selected the most acceptable option - the Filter Forge program.

Filter Forge was used in the development of the game Oil Rush as a generator of procedural textures to create an environment and some special effects. Below is a list of content in the creation of which the features of this program were used.

Environment

Waves on the water
Icebergs
Canyons
Rocks
Ice floes
Ocean floor
Oil film
Elements of the ruins
Masks distribution of algae, ice and debris
Vegetation on the rocks

Effects

Foaming water (for example, from getting into her subject)
Loops on the water from units moving along it
Bursts of light (eg when welding)
Sprawling oil stains

Waves on the water

Water in the game Oil Rush

Creating waves on the water by any linear means is a rather difficult task. When creating repetitive textures of large surfaces, the seamlessness of such textures plays a huge role: the less noticeable the repetitive character of the resulting image is, the better. It is quite problematic and inefficient to generate textures of the normals and heights of the water surface in programs like CrazyBump or nDo2 from her photographs due to the reflective and refractive properties of water, and to sculpt realistic seamless waveforms in ZBrush, Mudbox or 3D-Coat applications for subsequent baking of the resulting models in the normal map will take an unreasonable amount of time. Taking into account all these factors, it was decided to use procedural means.

Waves on the water are formed under the influence of wind on its surface. Since water is a more inert substance in comparison with air, the formation of waves occurs only in the case of a constant wind effect on some of its surface area. Thus, for our case, the key parameters are the direction, area of impact and intensity of the wind, which together determine the direction, height and width of each generated wave. It is also worth considering that the water surface is subject to vibrations and the imposition of several waves on each other.

With great simplifications, models of the phenomenon were made two types of waves - for strong and for weak wind. Each of them required two variations of normal textures with different main directions of wave movement.

The water shader in the Unigine engine implements wave simulation by superimposing two normal maps with animated texture transformation parameters (the shift of texture coordinates changes in time according to its law separately for each texture) - this is how a unique pattern of the water surface is formed at almost any time.

Normal maps used in the water material to simulate high winds in the game Oil Rush

Download filter (ocean_waves.ffxml)

Creation of icebergs, canyons, rocks and ice floes

Icebergs in the game Oil Rush

In this case, the traditional linear means of content production did not suit us for two reasons:

The task was set in a short time to realize a sufficiently large number of different variants of environmental objects.
It was necessary at the early stages to see the forms of objects, select only those that we are completely satisfied with, and throw out unnecessary ones without any hesitation.

In the area of procedural generators for creating 3D objects, the situation is not the best. It is quite possible that it will change if the scripting language (for example, Python) is enabled in the voxel editor 3D-Coat API, with the help of which it will be possible to create algorithms for generating three-dimensional objects. And at the moment, we are interested in looking at the Houdini programs from Side Effects Software and Voxelogic Acropora (unfortunately, judging by the lack of news updates on the site, this program no longer develops). But for our project, it was decided to use the capabilities of the Filter Forge two-dimensional procedural generator using a small trick: a filter was implemented that forms an iceberg elevation map, and then a high polygon mesh was displaced using the resulting elevation map in a 3D modeling program.

The height map for icebergs (above) and the mesh extruded through it (below)

Download filter (icebergs_forms.ffxml)

From all this diversity, the most suitable objects were selected and their retopology was made.

Iceberg rotopology

The analysis of photographs of real-life objects (references) made it clear that the surface texture for our objects is very different in nature in the top and side views. Therefore, after dividing objects into side and top elements and creating scans for them, two more filters were created, generating height maps separately for the top and separately for the side surfaces of the objects.

Unfortunately, Filter Forge has no means of visualizing the results of applying the height map to the displacement mesh, so each time you have to exit the filter creation mode, render the resulting texture to a file and update it in a third-party 3D application.

Formed height maps of the side and top elements were used for:
1. Extrusion (displacement) of a flat high-polygon mesh, from which, after optimizing its geometry, the necessary ambient occlusion, normals and cavity textures were baked.

The elevation map for forming the side surface of the iceberg (above) and the mesh extruded over it, ready for baking textures (below). It is easy to see that it would take a lot of time to sculpture one version of such a surface.

Download filter (iceberg_side.ffxml)

The elevation map for forming the upper surface of the iceberg (above) and the mesh extruded through it, ready for baking textures (below)

Download filter (iceberg_top.ffxml)

2. Giving extra relief to low-poly meshes also through extrusion (displacement).

After that, the joints of the side and top elements were sewn together.

Squeezing the side surfaces of icebergs with a height map (top), mesh after optimization (in the middle) and gluing vertices to the upper part (bottom)

Diffuse textures for objects were created in Photoshop from the resulting height, ambient occlusion, cavity textures, as this program still has more convenient tools for compositing multiple images than Filter Forge — you don't need to go out of the filter every time to render and saving pictures, it is possible to memorize a sequence of routine, frequently performed actions (for example, to save a file in the required format) in order to re-execute them by pressing a hot key. Therefore, the final color texture for viewing the result in the engine here is much easier and faster to form.

In a similar way, canyons, rocks and ice floes were created.

Canyons in the game Oil Rush

Map of the heights of the side surfaces of the canyons (top) and the mesh extruded through it (bottom)

Download filter (canyon_side.ffxml)
Download filter (canyon_top.ffxml)

Rocks in the game Oil Rush

Map of the heights of the rocky surface (above) and the mesh extruded over it (below)

Download filter (rock.ffxml)

This elevation map gave relief to several elements of the rocks at once, which later grouped together to form reefs.

The process of forming rocks and giving them the necessary relief

Ice Rush in the game Oil Rush

The height map of the floes (above), the mesh extruded through it (in the middle) and the retopology of the extruded mesh (below)

Download filter (ice_forms.ffxml)

Ice Surface Normal Map

Download filter (ice_top.ffxml)

Ocean floor

Filter Forge is well suited for generating small landscape objects with low relief and a mild erosion effect (for everything else, it is better to use specialized landscape generators). In the case of Oil Rush, this program is well suited for creating ocean bottom textures.