Reconstruction of a moving face 3D model
An interesting presentation was prepared for the European Conference on Computer Vision (ECCV-2014) by researchers from the University of Washington. This technology is a time-lapse reconstruction of a 3D model on video .
Using a video from YouTube, the program automatically builds high-resolution 3D models for each frame.
')
This is a very impressive result, given the complexity of the task, because the mimicry of a human face is very complex. For recognition of emotions, it is important to see the exact position of the eyes, the bend of the eyebrows, wrinkles. The slightest error in such a 3D model is strongly striking.
The vast majority of other programs for 3D tracking and face digitization use the method of smooth modifications (blend shapes) when the shape of the object changes, “flowing” from one state to another. The method of smooth deformations lack just the loss of small details that are so important for the perception of the face. The authors of the new algorithm abandoned this approach.
On the other hand, although in the frame-by-frame reconstruction each model exists “separately” from each other, but when playing at 30/60 frames per second, the result should be more realistic than in the case with a smooth modification.
And further. Unlike other similar technologies, there is no need for human participation in the test video. Instead, a large archive of his photographs is used in different lighting conditions and postures, using the optical flow method (3D optical flow). The author of the scientific work says that in our time there is a large archive of photos for each person.
The program has successfully proved itself in different light conditions and on different facial expressions.
The work is presented at the European Conference on Computer Vision, which was held September 6-12 in Zurich.
Research Article
Accompanying material
The technology can be used in cinema, 3D printing, computer games and virtual reality, perhaps to restore the exact appearance and facial expression of a deceased person from his photographs and video recordings.
Well, the most direct application of technology seems to be the creation of more realistic 3D movies and animation of facial expressions of characters in computer games.
Using a video from YouTube, the program automatically builds high-resolution 3D models for each frame.
')
This is a very impressive result, given the complexity of the task, because the mimicry of a human face is very complex. For recognition of emotions, it is important to see the exact position of the eyes, the bend of the eyebrows, wrinkles. The slightest error in such a 3D model is strongly striking.
The vast majority of other programs for 3D tracking and face digitization use the method of smooth modifications (blend shapes) when the shape of the object changes, “flowing” from one state to another. The method of smooth deformations lack just the loss of small details that are so important for the perception of the face. The authors of the new algorithm abandoned this approach.
On the other hand, although in the frame-by-frame reconstruction each model exists “separately” from each other, but when playing at 30/60 frames per second, the result should be more realistic than in the case with a smooth modification.
And further. Unlike other similar technologies, there is no need for human participation in the test video. Instead, a large archive of his photographs is used in different lighting conditions and postures, using the optical flow method (3D optical flow). The author of the scientific work says that in our time there is a large archive of photos for each person.
The program has successfully proved itself in different light conditions and on different facial expressions.
The work is presented at the European Conference on Computer Vision, which was held September 6-12 in Zurich.
Research Article
Accompanying material
The technology can be used in cinema, 3D printing, computer games and virtual reality, perhaps to restore the exact appearance and facial expression of a deceased person from his photographs and video recordings.
Well, the most direct application of technology seems to be the creation of more realistic 3D movies and animation of facial expressions of characters in computer games.
Source: https://habr.com/ru/post/237827/
All Articles