Video: neural network "repaints" the movie under Van Gogh (Munch, Picasso, Kandinsky, rock art)
Google has already demonstrated how the neural network creates pictures in the style of Van Gogh and Picasso, but this method is not suitable for video: the result of a frame-by-frame change of the film will be difficult to glue. German scientists coped with this problem - their neural network recognizes objects in the frame, remembers them and uses the same style for drawing when they appear again.
Processing frames from the movie "Ice Age" in the style of "Starry Night" by Van Gogh
Neural networks are taught using a large number of images. Each such network consists of a series of layers - from two to 30 - with a different level of abstraction. A photo or drawing for the end result passes through all these layers. In July 2015, Google demonstrated how the neural network creates works of art from random noise.
')
In the same year, an experiment was conducted at the University of Tübingen, Germany, which demonstrated the ability of the neural network to forge the artistic style of great masters - Van Gogh, Picasso, Munch and others. As the source, use any images and photos. The results are in the image below. The main achievement of the experiment, the authors considered the ability to independently manage the content and style in the neural network.
The research of scientists from the University of Tübingen is referred to by Alexey Dosovitsky, Manuel Ruder and Thomas Broks from the University of Freiburg, Germany. Scientists have improved the capabilities of the neural network in order to change not a single image, but a video sequence. In the case of a frame-by-frame change, the objects in each frame can be processed differently, and the result will be difficult to combine. Scientists have coped with this problem by teaching the neural network to recognize objects, ensure that their display does not change too much in the video, and use the same style each time they appear.
Processing frames from the movie "Ice Age" in the style of "Starry Night" by Van Gogh
Neural networks are taught using a large number of images. Each such network consists of a series of layers - from two to 30 - with a different level of abstraction. A photo or drawing for the end result passes through all these layers. In July 2015, Google demonstrated how the neural network creates works of art from random noise.
')
In the same year, an experiment was conducted at the University of Tübingen, Germany, which demonstrated the ability of the neural network to forge the artistic style of great masters - Van Gogh, Picasso, Munch and others. As the source, use any images and photos. The results are in the image below. The main achievement of the experiment, the authors considered the ability to independently manage the content and style in the neural network.
The research of scientists from the University of Tübingen is referred to by Alexey Dosovitsky, Manuel Ruder and Thomas Broks from the University of Freiburg, Germany. Scientists have improved the capabilities of the neural network in order to change not a single image, but a video sequence. In the case of a frame-by-frame change, the objects in each frame can be processed differently, and the result will be difficult to combine. Scientists have coped with this problem by teaching the neural network to recognize objects, ensure that their display does not change too much in the video, and use the same style each time they appear.
Source: https://habr.com/ru/post/394195/
All Articles