The neural network turns scribbles into artistic masterpieces

The Neural Doodle program, made on the basis of a convolutional neural network, is a doodle.py script that generates images, taking three to four pictures as input parameters. This includes a simple sketch at the entrance (what the authors call “scrawl”) and a sample of the style with its sketch. For example, in the case of the example above, a sample of the style is such a Renoir painting .

The neural network extracts the characteristic style features - and transfers them to the sketch.

Here is another example.
')


Sample style from Claude Monet.



To run the program, you need Python 3.4+, the installed numpy and scipy , and also python3-dev . The script requires a previously trained neural network (VGG19, 80 MB). For instructions on setting up a local environment, see here .

Rendering to GPU requires a good Nvidia card with support for CUDA technology and 2-4 GB of memory (for large images - 8-12 GB). Rendering can also be run on the CPU, in which case you need approximately the same amount of RAM.

The design of the neural network is described in the scientific work of the author " Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks ", which he prepared for the conference nucl.ai Conference 2016.

The neural network uses an image synthesis algorithm, which was proposed by researchers Chuan Li and Michael Wand in the research paper " Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis ".