What's wrong with Mass Effect: Andromeda animations?

Much attention was attracted by the problems with animation in the reviews of the new big project Bioware - Mass Effect: Andromeda . Therefore, we decided that it would be interesting to collect experienced animators and discuss the difficulties encountered when working on animations in such projects. Many thanks to Daniel Floyd for moderating this roundtable. So let's start by introducing the participants.

Dan Floyd: I think it’s worth starting with me, because I’m mainly here as a moderator and will be asking questions to smart people. My name is Dan Floyd, I am one of the leading YouTube channel Extra Credits . On this channel I do a show about game animations called Extra Frames . In addition, I am an animator with experience in the film and gaming industry, but I do not reach the level of my interlocutors at all.

Simon Unger: Hi, my name is Simon Unger. I’m an animator at Phoenix Labs and am working on a new Dauntless game. I used to be a lead animator at Electronic Arts, Square Enix and Robotoki. I studied in such educational institutions as Gnomon, iAnimate and Vancouver Film School.
')

Gwen Frey: Hi, my name is Gwen Frey, I have been working in the gaming industry for ten years. My last work is the creation of a small indie studio in Boston and the development of The Flame in The Flood . In the project, I was the only animator and effects artist. Before that, I worked on technical animation in Bioshock Infinite and DLC, and even earlier on MMO, for example, on Marvel Heroes Online . I participate in public speaking about game development, lead podcast ( The Dialog Box ) and YouTube channel ( GwenFreyTheTA ).

Tim Borelli (Tim Borelli): Hi! I am Tim Borelli. I have been working with animation since 1998, and I have more failed projects than successful ones. I worked at Volition (from Freespace 2 before starting to develop Saints Row 3 ), at 5TH Cell (where I first learned to be a good leader), and now I’m director of animation and video effects at First Strike Games .



Dan: So let's start with general questions: what problems are inherent in your work? What are the limitations most often encountered by an animator working on video games?

Simon: There are many minor technical difficulties that you have to cope with day after day. Often they are associated with a specific project, product or engine. I think that throughout my entire career there was always the problem of achieving a high “believability” of the character’s actions. We have to deal with a lot of variables and external incoming data (usually the player likes to turn the joystick in different directions and tear off the video sequences), so the interesting task is to process all this in such a way that the sensations from the game process do not suffer. Sometimes the smallest roughness interfere with the game.

Tim : Problem number one is that you should always remember - the animation in games does not end after the animation process is complete. Most often at this moment only half of the way is covered. After exporting animations, many different systems will interact with them, affect their reproduction, appearance and spirit. The time of mixing transitions from one animation to another, inverse kinematics systems that change the position of the legs and displacement relative to the ground, additive systems that add poses to animations or groups of animations, change their appearance, procedural systems that must be perfectly tuned - all this affects the work system. If any of these components fail, the animation will look completely wrong.

Dan: We all saw videos from the early release of Mass Effect Andromeda , and noticed a rather large number of animation errors in them, especially in navigation systems and conversations. Before turning to the particulars, can someone explain how the talk systems work? How does the animation of the Mass Effect or Witcher dialog system differ from the animated scenario inserts in games like Uncharted or Kingdom Hearts ?

Tim: I worked with systems of a much smaller scale, and even they were a nightmare for people working on them. Instead of creating special animations for each phrase of the dialogue and choosing a suitable voice actor, you need to prepare phonemes (“A”, “E”, “O”, “Mmm”, etc.), emotions (anger, sadness, joy etc.) for the upper and lower parts of the face, animation of body language (happiness, irritation, sadness, delight, etc.) with different levels of intensity (very angry, a little angry, boiling with anger, etc.) . I will confine myself to this in order to simplify the description of the process, but usually a very small team of animators is working on all of this, and in my experience only one person does.

On top of this data are procedural systems, both in-game and in the editor. These systems control the turns of the eyes and heads in the direction of the goal (when the head and eyes are watching something interesting), sometimes auto-generation of lip-synchronization in the dialogues is used, and much more, but this can be stopped.

Depending on the size of the project and the team, this data is used in different ways:

1. Unite the team involved in setting up all the conversations in the game. Take for example the phrase: “And then he stole a pig! HA-HA-HA! ”, Which will be the ending of the joke. The animation team needs to find the right body language animations and facial expressions. Lip sync should already be generated by a tool that reads a phrase and creates an animation from it.
2. Procedurally connected by the system that reads the "tags" of data. These “tags” are assigned by the animator or designer for two aspects: for body language and facial emotions that define their essence (for example, “anger”) and for the voice acting phrase, so that the voice actor can play it. Then the system takes a random version of “anger” and plays it at the right moment. Lip sync is already generated by a tool that reads a phrase and creates an animation from it.
3. Finally, the data can be used as a basis, which animators can customize and enhance in a 3D editor (for example, in Maya). They choose the movements to be used, connect them together, and then manually improve them.

In this process, the character can be assigned different points of interest. They should correspond to the current conversation, but sometimes the flags do not change the meaning, and important NPC suddenly pass next to the character, and his eyes start to rotate madly.

All such decisions have their price, and I think we'll talk about it later. But given the complexity of even the simplest versions I have described, it is very easy to understand how frightening the task of creating a convincing system of conversations is.

Let's compare with the games of the Uncharted series - they create their own animation for such scenes, which is being worked on by a huge team of animators, video artists, video camera crews, etc. They are created with voice actors, re-shot, and are always prepared taking into account the emotional mood of the game.

Dan: In such large projects, one inevitably faces the realities of the production plan. Time and money are limited and the team needs to plan to spend it as optimally as possible. Therefore, without a lot of predictions can not do. How is the selection made for animation teams? What is included in the graphics process of the AAA-class video game animations?

Gwen: [laughs] Planning the production process in big studios is a rather complicated process. After the preproduction stage, the producer asks the animation team to prepare a bunch of predictions. "How many hours of work will animators need to animate the movement of a two-legged NPC?" How many hours does an animator need for a new two-handed weapon for the player’s hands and for all NPCs? ”And so on. Animators answer these questions on the basis of their own assessments, and then the production team starts work. The production team collects data for each aspect, and then provides the director with data based on these estimates: “We can either add one new weapon to the game, or add three new NPCs”, and stuff like that.

From this point on you can no longer manage the schedule. In a large studio, there is too much different work, and many tasks depend on when the previous task performed by another team is completed. When everyone is a specialist who performs a very narrow task, then many interdisciplinary dependencies arise. Therefore, the producers tell the employees what tasks are most important and how much time they have to complete. They earn their living by this, and you must obey them (because obedience to them is what you earn by bread).

Obviously, for the good of the game, priorities change in the production process. Large chunks of player and scene abilities are thrown away, dialogs change at the last moment, and so on. Sometimes important dates change completely, which greatly influences the tasks of each. This is an “exciting” adventure.

Simon: Yes, Gwen explained everything correctly. If the team conducted the preproduction stage well enough (that is, studied all the unknown, created prototypes, tested the performance of the basic gameplay cycles, etc.), then the estimates will be close to the realities of production. The animation team may have problems when it has to evaluate an ill-defined design or function. It helps experience, but in games it is often necessary to do something that no one has done before. It is very difficult to estimate in hours (or even days) how long it will take to implement. It is necessary to arrange the functions and tasks in order of priority, try to relate them to the schedules of other departments and start with the most important. Anything that does not fit within the time frame, or cut, or is then implemented in the update or DLC.



Dan: I want to touch on another topic: the animation in the game (good or bad) that the player sees is the result of the work of not only the animation team. I think that from your own experience you can all answer this differently, but in general, game animation depends on coordinating many disciplines: from programming to design. This is really a team work. How do other disciplines within your team affect your work?

Gwen: I thought about the fact that people blame animators for bugs, although in fact they probably have nothing to reproach. In an AAA project of this scale, most of the small inserts are not even seen by animators. A system for designers is created, and all hope is placed in them in the correct writing of scripts. For example, they create a system of "moods" so that the character can be happy, sad or angry. Then the designers are assigned to create a mood for the NPC in the scene. Designers are given a bunch of background NPCs with basic behavior to place these characters in logical places. The animator mainly deals with scenes with characters (scenes that are fully animated manually) or works with animation programmers to ensure the “tightness” of gameplay systems. For example, you need to make the character lean forward when running uphill, so the animator is working on mixing various motion animations. All the bugs seen in the videos are like technical errors or implementation errors. I'm not saying that the animators have nothing to blame: everyone is responsible for ensuring the AAA-quality of the game and everyone must beat all the bells when something goes wrong. However, I look at all these errors as a technical animator and I think that they look like problems with inverse kinematics and script errors.

Tim: I’ll add to what Gwen said: the aspect of creating and customizing a character always comes into play. Character creation and animation teams should always take into account the differences in facial features and proportions that are available to players when setting up a character. Any such settings can lead to imperfect animation of faces. Just google “Ugliest character in Saints Row 2 ” to see examples. Often in big games the same customization system is used to fill the NPC world, and although we strive to control the quality of the characters created by this system, sometimes something is missed or generated randomly.

Dan: I don’t think that any of us can understand exactly what’s wrong with the Mass Effect Andromeda animation (without knowing BioWare workflow and the complexities of the system). But when you watch these videos, what reasons can you find these oddities? What exactly could go wrong in the system of conversations and lead to such animation errors?

Simon: I think that before making assumptions about the causes of such problems with animation, it is important to explain to people what numbers are behind the games of this level. I do not know the exact estimates for ME: A , but we know that in Mass Effect 3 there were more than 40 thousand lines of dialogs, and in Dragon Age - about 60 thousand. If we take as an average 50 thousand and estimate that each line lasts about three seconds, then as a result we get about 41.5 hours of dialogues. That would be enough for 21 films. Teams of 70 or more animators work on most animation films. Making a movie takes two years or more. Five to ten animators worked on Mass Effect , and they had to create twenty times more content over the same period of time. In addition, it is necessary to take into account the localization of the game in at least 4-5 additional languages.

It is clear that it is impossible to break such a volume of content into key phases of animation with an acceptable level of quality. Therefore, in order to cope with this scale, studios seek to transfer as much work as possible to procedural systems. I know that they used to use a turnkey solution called FaceFX. It analyzes audio tracks and creates animations based on waveforms, projections, etc. At a basic level, the result may look like the movements of robots, and I suspect that this is what we see in the commercials. Of course, you can work with sound and procedural tools to enhance animations. But looking at the thousands of minutes of dialogues that need to be improved, the definition of “permissible” is shifting in the direction close to the capabilities of the team and the approaching deadlines. If the team and the project were mine, I would collect metrics about which scenes are watched most often on the basis of game tests, and polish them in the first place, and leave the rest as is.

Gwen: I agree with Simon, many errors look like an unsuccessful use of FaceFX. I suspect that a lot was realized even by an animator. Often the intern or junior is involved in this work, simply copying the written script to FaceFX. The system automatically generates facial animation based on the letters / sounds of the entered text. In general, it looks bad, and you need to pronounce the words phonetically, and not just to drive in their text. But it takes a lot of time and resources, which are often lacking.

Have you seen a crab gait bugs when a character is running fast? They have standard causes. If the collision with the surface is not configured correctly, the character will fall down a bit to the floor. However, the inverse kinematic in-game system does not expect the character to partially pass through the ground, so she raises her legs back to the surface level. This leads to a strange rotation of the knees. Another variant of the reason for such a problem: the system of inverse kinematics detects a nonexistent rise, or there is some kind of mathematical error when lowering the pelvis to the ground. For example, if a character quickly runs up the stairs, the system detects that he is on a climb, and lowers his pelvis to the ground to compensate. If the system breaks down and it is not reported that the lift has already ended, the pelvis will always be lower, which will lead to a very funny “crab” movement.

A convulsive eye movement can also have many possible causes. ME definitely has a dynamic eye tracking system, used when NPCs look at interesting objects moving alongside them. It is possible that in some inserts this system is not turned off, and procedural tracking of eye movements is struggling with the planned eye animation. Or maybe the object is in an intermediate state between being in the field of visibility (when it is tracked by the eyes) and outside this area (when it is not tracked). Or maybe the NPC is trying to track two objects at the same time, and the system constantly receives targeting information from two different targets?

Tim: Probably, we should not forget that everyone involved seeks to invest in the game maximum, but the bigger the game, the greater the likelihood of misunderstanding. Animator A has optimal plans for using conversation animations created according to a design document received from Screenwriter A, but Screenwriter B in another city (or simply in another part of the building) may need one of the animations in a task for which it was not intended. Of course, he can create a request to create a specific animation, but animator A may not see it because of the approaching deadline and sorting requests by priority.

It would be wrong to blame one of the two employees in this situation. You can criticize their work, and it is quite acceptable - as creators, we often welcome criticism, because this is a way to improve ourselves and the product that we offer players. But in such a big task, as, for example, creating a system of conversations, it is important to know that such work requires a lot of effort and time. And if we do not meet the expectations of the players, then do not blame yourself for this. This is quite normal, we are at the very beginning of training, and the results will improve over time.

And we need this time to come.

Dan: I want to end up honoring the Mass Effect Andromeda development team. What matters now is how the game will be accepted by the players. Many people have invested in its creation, and this deserves respect. Congratulations on reaching the final line!