We draw the rest of the owl on the basis of neural networks.
The idea to draw an owl from circles is implemented using neural networks.
Based on machine learning and a neural network, developer Christopher Hesse created an online service that can "finish" sketches of full color photographs.
The website http://affinelayer.com/pixsrv/index.html Christopher Hesse presents a neural network capable of drawing cats.
')
The user is invited to draw a sketch of a cat in the left window, click on “process” and see how the neural network completes the illustration. The page shows such a beautiful example:
Apparently, the more lines used in the outline and the more the outline resembles a cat from a training set, the better the neural network will create a full-fledged drawing based on it.
Here is what the author himself writes:
Recently, I made a Tensorflow port pix2pix on Isola and others, discussed in the article Image-to-Image Translation in Tensorflow . I took a few previously created models and made an interactive web page for testing. Chrome browser is recommended for viewing.
The pix2pix model works by training on pairs of images, such as sketching the facade of a building and a full image of building facades, and then trying to generate the corresponding output image for any input transmitted image. This idea originates from the article pix2pix , which is recommended for reading.
Facades
The model was trained on sampling sketches of facades of buildings for full-fledged images of facades. It is unlikely that this model will work on sketches with a large empty area, but if you draw a sufficient number of windows in the outline, the model often gives good results. In the sketch picture, the elements of the facade are drawn with colored rectangles to indicate not only the borders, but also the entire elements.
I did not have the names of the various parts of the facades of the buildings, so I just roughly outlined them.
edges2cats
The training took place on about 2000 photos of cats and sketches with automatically generated borders for these photos. The model creates color images of cats from sketches, but some are confronted with nightmarish results. One of those I saw here .
Some pictures look especially creepy, I think, because cats were painted wrong, especially because of the eyes. The procedure for autogeneration of boundaries was not very high quality, and in many cases did not detect the eyes of a cat, which degrades the quality of the prepared image database for training the model.
edges2shoes
The training was conducted on the basis of approximately 50 thousand shoe images collected from Zappos , as well as with automatically generated sketches of borders for these images. If you’re really good at drawing the boundaries of your shoes, you can try creating some new designs. Keep in mind, the model was trained on real objects, so if you can draw a better 3D sketch, the result will look better.
edges2handbags
By analogy with the previous ones, the training was conducted on the basis of approximately 137 thousand photographs of bags collected from Amazon, with automatically generated sketches of boundaries for these photographs. If you draw shoes here instead of a handbag, you will get a very strange texture of shoes.
Implementation
Training and export of models was performed using pix2pix.py script from pix2pix-tensorflow . An interactive demo is made based on JavaScript using the Canvas API, which interacts with the server, which sends images to Tensorflow. The server can run Tensorflow itself or forward requests to Google Cloud ML with Tensorflow.
Trained models are available in the Datasets section on GitHub. Also should be available models coming with the original implementation of pix2pix. Models can be exported from trained examples using the pix2pix.py script and links to exported models are in the README file on the GitHub server.
Borders for cat photos were obtained using the Holistically-Nested Edge Detection algorithm and this functionality was added to the process.py script and the corresponding dependencies were added to the Docker image .
Based on machine learning and a neural network, developer Christopher Hesse created an online service that can "finish" sketches of full color photographs.
The website http://affinelayer.com/pixsrv/index.html Christopher Hesse presents a neural network capable of drawing cats.
')
The user is invited to draw a sketch of a cat in the left window, click on “process” and see how the neural network completes the illustration. The page shows such a beautiful example:
Apparently, the more lines used in the outline and the more the outline resembles a cat from a training set, the better the neural network will create a full-fledged drawing based on it.
Here is what the author himself writes:
Recently, I made a Tensorflow port pix2pix on Isola and others, discussed in the article Image-to-Image Translation in Tensorflow . I took a few previously created models and made an interactive web page for testing. Chrome browser is recommended for viewing.
The pix2pix model works by training on pairs of images, such as sketching the facade of a building and a full image of building facades, and then trying to generate the corresponding output image for any input transmitted image. This idea originates from the article pix2pix , which is recommended for reading.
Facades
The model was trained on sampling sketches of facades of buildings for full-fledged images of facades. It is unlikely that this model will work on sketches with a large empty area, but if you draw a sufficient number of windows in the outline, the model often gives good results. In the sketch picture, the elements of the facade are drawn with colored rectangles to indicate not only the borders, but also the entire elements.
I did not have the names of the various parts of the facades of the buildings, so I just roughly outlined them.
edges2cats
The training took place on about 2000 photos of cats and sketches with automatically generated borders for these photos. The model creates color images of cats from sketches, but some are confronted with nightmarish results. One of those I saw here .
Some pictures look especially creepy, I think, because cats were painted wrong, especially because of the eyes. The procedure for autogeneration of boundaries was not very high quality, and in many cases did not detect the eyes of a cat, which degrades the quality of the prepared image database for training the model.
edges2shoes
The training was conducted on the basis of approximately 50 thousand shoe images collected from Zappos , as well as with automatically generated sketches of borders for these images. If you’re really good at drawing the boundaries of your shoes, you can try creating some new designs. Keep in mind, the model was trained on real objects, so if you can draw a better 3D sketch, the result will look better.
edges2handbags
By analogy with the previous ones, the training was conducted on the basis of approximately 137 thousand photographs of bags collected from Amazon, with automatically generated sketches of boundaries for these photographs. If you draw shoes here instead of a handbag, you will get a very strange texture of shoes.
Implementation
Training and export of models was performed using pix2pix.py script from pix2pix-tensorflow . An interactive demo is made based on JavaScript using the Canvas API, which interacts with the server, which sends images to Tensorflow. The server can run Tensorflow itself or forward requests to Google Cloud ML with Tensorflow.
Trained models are available in the Datasets section on GitHub. Also should be available models coming with the original implementation of pix2pix. Models can be exported from trained examples using the pix2pix.py script and links to exported models are in the README file on the GitHub server.
Borders for cat photos were obtained using the Holistically-Nested Edge Detection algorithm and this functionality was added to the process.py script and the corresponding dependencies were added to the Docker image .
Source: https://habr.com/ru/post/401835/
All Articles