Digital actors

In 2001, a full-length animated film Final Fantasy: Spirits Within (Final Fantasy: Spirits) was released. From a huge budget of $ 145 million, about a third was spent on purchasing servers, workstations, and software. And the filmmakers paid particular attention to the realistic modeling of live characters. At that time, this was a real breakthrough in the field of creating electronic actors. Of course, they could not be confused with living people, the difference was already small. And many of the flesh and blood actors had bad feelings that Tom Hanks sounded shortly after the premiere of the film: “This is very disturbing to me, very much. This threat is approaching and we, the actors, must, by rallying in front of her face, undertake something. ” Since then, 13 years have passed, technology has gone far ahead. How did this affect the electronic actors, was it really worth worrying about Forrest Gamp?

Sinister valley

Three years later, ironically, Tom Hanks himself became the “source” for the character in the full-length animated film Polar Express (Polar Express):
')


Some kind of creepy feeling from the faces of all the characters, right? This is the so-called “ominous valley” , a psychological effect in which the audience is evoked by a feeling of discomfort (anxiety, fear, discomfort) and the contemplation of characters that look almost like a living person. This unpleasant underlying feeling arises in the case when a character loses all the features of caricature and simplicity, which clearly tell our consciousness that this is a fake, imitation. And it became, oddly enough, one of the main problems on the way to creating perfect digital actors.

In the mid-2000s, it seemed that we could never achieve such realistic artificial people to finally get rid of the effect of an ominous valley. However, in 2008, the wonderful film The Curious Case of Benjamin Button ( The Curious Case of Benjamin Button) is released. Through the efforts of animators, Brad Pitt throughout the film received the appearance of both an old man and a teenager, while remaining recognizable by the old Brad.



And now, six years later, in most scenes, it seems that this is just a very well-made actor. But this is not the case:



An example of this film showed that in episodes it is already possible to get rid of the effect of an ominous valley, since cost and laboriousness do not yet allow making a full-fledged counterpart. For example, in some scenes of the film Iron Man 3 (Iron Man 3) a digital copy of the actor Guy Pearce was involved, because he managed to grow a beard for filming in another film.

Someone even very high-quality modeling of monkeys in Dawn of the planet of apes (Planet of the Apes: Revolution) is not only surprising, but also some inexplicable wariness. Although this can be explained by the unusual humanity of monkey faces.



The disadvantage of the “sinister valley” effect for the film industry is that instead of surprise and joy, the viewer may experience negative emotions. And this is a direct path to the commercial failure of the film, and for what was then spent time, effort and a lot of money when creating special effects? And in modern blockbusters this is the main item of expenditure of all kinds of resources, not only financial ones.

How many resources are required today for realistic human modeling? Of course, this very much depends on a number of conditions, but on average in a hospital each frame requires about 30 hours of operation of a large number of very powerful servers. And in five minutes of a film of 7200 frames with a frequency of 24 frames / sec. At one time, the real breakthrough was the use of motion capture technology, without which it is impossible to achieve complete realism of the movements of 3D models.

Digital Ira

A year and a half ago, Nvidia published a demo of face rendering technology Digital Ira.



Here is a very realistic picture of an early half-grown man of about thirty, talking a little in the nose. If you look from afar or glimpse, then from a living person can not be distinguished. One can even be confused if one does not see the eyes, despite the fact that serious computational powers are used for realistic modeling of their movements.

Surprisingly, but still the eyes of digital actors can not "give in" to animators, they cannot be made completely indistinguishable. Here you will inevitably think about the meaning of the phrase “Eyes are the mirror of the soul” ... Each feature of this digital model requires serious calculations: folds and skin irregularities, blood vessels, movements of mimic muscles. A man can express pleasure, bewilderment, irritation, surprise, boredom, pleasure and anxiety. And with almost the same speed as a living person. This is a very important factor, as our brain instantly notes a too low or high rate of change in facial expressions, with the result that we have a sensation of falsity.

How this technology works: first, the head of a man-model is scanned, the movements of facial muscles, jaw and eyes are digitized. In addition to the mechanics, the reflection of light sources, the shadow map with a particular facial expression, during a conversation is calculated.

So far, two versions of the Digital Ira were demonstrated: from Nvidia in March 2013 at a conference dedicated to GPU technologies, and from Activision in July 2013 at the Siggraph. Nvidia ran it on one of its video cards, while it performed 4.9 trillion operations per second. The Activision version worked on the PlayStation 4 with a graphics processor with a performance of 1.84 teraflops.

Movie

As you can see, we are already close to the main goal that the special effects industry has set itself, the creation of interactive photo-realistic digital actors. And here a number of questions and doubts arise.
Imagine that developers finally managed to create digital people who are indistinguishable from real ones (and this will happen sooner or later). This will completely reverse our understanding of acting skills and the general role of actors. Games will turn into such interactive films. Believing TV news in general will be meaningless, given the possibility of faking any video. Although now they can not be believed, to be honest.
Despite the technical ability to create an almost completely realistic digital actor, the cost and labor intensity is such that it is cheaper to hire a highly paid Hollywood star. The main reason is the need to use very large computing power, which is constantly getting cheaper. And when the cost of creating the perfect, perfect “digital camera” becomes comparable or lower than hiring a good actor, then there will be stormy changes.

Artificial actors will not need to be divided into drama, comedy, etc. They will be able to play any role, any character. Almost all films with digital actors use three-dimensional models of living people, as in the same “The Mysterious History of Benjamin Button”. However, the main goal of special effects specialists is to create arbitrary characters that are indistinguishable from people. Arbitrary age, appearance, gender, height, fullness, etc. Although digital copies will also be in demand: imagine how many great, great actors could be perpetuated in the ever-young digital cameras, using them for many decades. While we all age, and the same actors with age can no longer fulfill many roles.

Games

It is quite possible that the flowering of “film games” with a large number of subject forks is waiting for us, in which the story will evolve according to one of several scenarios, depending on the choice made by the game player. This principle is already used in games, one of the examples is two games from the Walking Dead series.





Here the player is shown a certain story, which he looks like a spectator. But from time to time there are situations requiring the player to be interactive and choose how to act on the character. Depending on this, the plot may develop in a different direction, but within certain boundaries.

This approach can be developed further by creating a small world with a large number of subjects. Players will be able to choose characters whose storyline they want to play, with the ability to switch to another character if their story seems more interesting.

Logical step

Today in movies and games, characters are usually created with the help of motion capture technology. True, the cinema games are still far off in terms of confidence. On the other hand, this is a matter of budget and computing power: in the cinema, the viewer is shown a ready-made rendered picture, and in games all work falls on the much-suffering computers and user consoles.



But still it is impossible to call an actor a digital copy of a living person. After all, the actor himself forms his image, behavior, emotions, gestures. And after we are able to reproduce a person's appearance with an acceptable cost, the next step will be the creation of an “acting” technology for digital characters. So that they do not repeat the algorithm of actions laid by the operator, they should not “lose” the previously scanned mimic schemes. In fact, a digital actor must be to some extent an artificial intellect, who will choose himself how to play this or that role or scene, how to express emotions. So that he acts according to the same principle as you and I, when we choose how to conduct ourselves in this or that situation, how to express our feelings and thoughts.

Of course, dreaming is not bad. The creation of such a technology will require the solution of the most difficult tasks that today stand in the way of developing artificial intelligence. And even more, because this intellect must be capable of not only “speaking”, but also “expressing itself” non-verbally.

The problem is complicated by the fact that we still need to understand ourselves how the acting talent works, what psychological (or physiological?) Features make some people actors? We also do not know the mechanism by which the manifestation of some kind of emotion can affect our facial expressions, posture, gestures. Scientists are still arguing that the primary from the point of view of deep psycho-physiological mechanisms - whether we feel happy because we smiled, or we smile, because we have felt happiness.

So it will take more than a decade, while we can talk about the possibility of creating full-fledged digital actors. Until then, we will be “content” with digital twins of varying degrees of credibility. By the way, there is an opinion that now we need to begin to record in detail the appearance, facial expressions and behavior of various politicians, scientists, influential persons - all those whose digital copies could be used in the future to create films and games. For the sake of historical authenticity, so to speak. In fact, it is no different from the constant photography and video, using these materials for a variety of tasks. Although there is such an opinion that digital actors are evil, and the real art of acting is available only to living people. After all, how can artificial intelligence compare with the talents of the brilliant actors of the past and present?