Nvidia's neural network turns simple sketches into beautiful landscapes

Smoker's waterfall and a healthy person's waterfall

We all know how to draw an owl. You must first draw an oval, then another circle, well, and then - it turns out a gorgeous owl. Of course, this is a joke, and very old, but Nvidia engineers tried to make the fantasy a reality.

The new development , called GauGAN, creates chic landscapes from very simple sketches (really simple ones - circles, lines and everything). Of course, at the heart of this development are modern technologies - namely, generative competitive neural networks.

GauGAN allows you to create colorful virtual worlds - not only for fun, but also for work. So, architects, specialists in landscape design, game developers - they can all learn something useful. Artificial intelligence immediately "understands" what a person wants and complements the original idea with a huge amount of detail.
')
“Brainstorming in terms of design development is much easier with the help of GauGAN, because a smart brush can complement the initial sketch by adding quality images,” said one of the developers of GauGAN.

Users of this tool can change the original idea, modify the landscape or another image, add sky, sand, sea, etc. Everything your heart desires, and the addition takes place in just a couple of seconds.

The neural network was trained using a base of millions of images. Thanks to this, the system can understand what a person wants and how to achieve the desired. And the neural network does not forget about the smallest details. So, if you sketch a pond and some trees near it, after the landscape is revived, all nearby objects will be reflected in the pond's water mirror.

The system can indicate what should be the visible surface - it can be covered with grass, snow, water or sand. All this can be transformed in a second, so that the snow will become sand and instead of a snow-covered wasteland, the artist will receive a desert landscape.

“It's like a coloring book, which says, where to place the tree, where is the sun, and where is the sky. Then, after the initial task, the neural network revives the picture, adds the necessary details and textures, draws reflections. All this is based on real images, ”says one developer.


Despite the fact that the system lacks an “understanding” of the real world, the system creates impressive landscapes. This is because here two neural networks are used, the generator and the discriminator. The generator creates an image and shows it to the discriminator. The one, based on millions of previously seen images, selects the most realistic options.

That is why the generator "knows" where the reflections should be. It is worth noting that the tool is very flexible and equipped with a large number of settings. So, you can use it to paint pictures, adjusting to the style of a particular artist, or just messing with the quick addition of sunrise or sunset.

The developers claim that the system does not just take images from somewhere, adds them together and gets the result. No, all the resulting "pictures" are generated. That is, the neural network "creates" as a true artist (or even better).

So far, the program is not in the public domain, but soon it can be tested in the work. This can be done at the technology exhibition GPU Technology Conference 2019, which is right now in California. The lucky ones who could visit the exhibition, can already test GauGAN.

Neural networks have long been taught to take part in the creative process. For example, last year, some of them could create 3D models . In addition, the developers from DeepMind have trained the neural network to restore three-dimensional spaces and objects from drawings, photographs, sketches. In order to recreate a simple figure, a neural network lacks one image; to create more complex objects, five images are required for “dragging”.

As for GauGAN, this tool will clearly find a worthy commercial application - many businesses and science have a need for such services.