Flash animation in Unity3D do it yourself. Part One, Lyrical

In this series of articles, I will discuss how and why we decided to create our own solution for importing flash-animations into Unity, about optimizations and the internal kitchen of the resulting plug-in. Also in the program: a story about the internals of the swf format, the features of the Unity-editor extension, and in general about animations in general. I ask under the cat!

Introduction

At the beginning of any project, there are flour choices of technology for many of its parts. One of these parts is the animation system. There are several variables on which this choice depends. First, what your animators are used to (can) work with or what tools of these animators are easier for you to find. It is clear that it does not make sense to choose an extremely specific product, because if current employees leave, for whatever reasons (sudden bus?), It will be extremely difficult to replace them, part of the project will just stop for an indefinite period, which, in the case of a commercial product can be very expensive. The second variable is more technical: integration into your engine. Are there third-party solutions and the quality of these decisions, is there the strength and the ability to create one's own, as well as the performance and convenience of all of the above. Well, the third variable is the capabilities of this tool, because if you need blood from the nose, you need inverse kinematics , then it will be completely strange to choose non-skeletal animation.

Types of 2d animations

Consider the most popular options for the implementation of animation in games. I will talk only about 2d animations, 3d animations - this is a topic for a completely different conversation, approaches and tools. You can safely skip a section if you are savvy in these matters (there are no revelations and no disruptions of cover), or if you went to see the features of flash-animations only, they are lower.

Frame-by-frame animations

Our mini-review opens with the simplest and most ancient type of animation: frame-by-frame animation. Each frame of such an animation is represented by a separate picture, with a quick change of which, there is an illusion of movement.

Pros:

Elementary implementation, usually there in any engine. Any complexity and style of animations, even if the movie is frame-by-frame, fold and paste (in theory, of course). Any tools: almost anything can give you a sequence of frames, which you can, if necessary, glue into one texture atlas.

Minuses:

The main disadvantage is, of course, the size of this good in memory. Naturally, there are all sorts of tricks such as cutting frames into blocks and then reusing repeated blocks, background loading frames from disk and freeing up already shown ones, and others. But these tricks have their drawbacks. Cutting into blocks is only suitable for pixel art , background loading creates additional load on the disk, takes up a lot of space on this disk, the size of your game increases accordingly, as well as the amount of data you need to download to start or update it.

Total:

Only suitable for small animation frames and in the number of frames, ideally for pixel art and retro-style games a la NES .

Video animations

As an apotheosis of the madness of frame-by-frame animations, video animations are performing. And yes, it is used and even sometimes to the point. We implemented this for games in the " Hidden Object " genre for large and complex animations, as well as for cut scenes with a touch of realism. Using codecs like theora and vp8, you can implement quite tolerable video animations (and even with an alpha channel ) with your own hands.

The advantages here, of course, are that you can show anything in such animations that your heart desires, from staged scenes with real actors to rendered 3d battles. The minuses are the wild load on the CPU when decoding this and, of course, the picture quality. It is possible to find a compromise between quality and performance in very limited and specific cases, as in the above-mentioned Hidden Object games, for example.

In general, it will suit not everyone and not always, or rather almost never, if we do not consider rendered cut scenes - where such "animations" are and a place. A good implementation of such animations cannot boast simplicity either, there are a lot of nuances and tricks, both in terms of the quality of the output image and the received loads on the central processor. Due to the specificity, there are few ready implementations, they also do not differ in quality, or they cost big money (hello, Bink_Video ), so many people write their own.

Skeletal animations

So we got to the popular and modern types of 2d animations. Skeletal animations more and more captures the hearts and minds of developers in the 2d realm, becoming the standard of animation for games. It's funny that they came so widely in 2d relatively recently, unlike 3d, where they appeared in the time of the King of Peas, when fathers wrote Half-Life . The essence of skeletal animations, oddly enough, in the skeleton, which creates an animator of bones connected with each other by a tree structure. Fragments in the form of separate pictures are attached to the bones. In motion, the whole structure is driven by displacements and rotations of the bones relative to each other. Offsets and turns of bones in the skeleton follow the general animation timeline.

The advantages are obvious: we do not need to store each frame of the animation as separate pictures, only fragments (pens and legs of the character, for example), which we will move with the bones of the skeleton. There are excellent implementations of this type of animation with runtimes for the necessary platforms and engines, with a reasonable price of use or completely free: Spine , Spriter , Anima2D , DragonBones and others. Thanks to the skeleton and its features, you can achieve tremendous smoothness of animations by interpolating the positions of its bones, as well as mix transitions from animation to animation and even mix two different animations: shooting on the run, shooting sneaking and so on. I will not list all the possibilities of skeletal animation, there are really a lot of them and they are very cool. I will give a better reference where they are clearly shown with pictures and descriptions on the Spine site .

Looks like a silver bullet, but no. Without the cons was not done. At once I will make a reservation that for many projects and animations - these disadvantages may not exist. If skeletal animations are suitable for you - this is great, use them, they are beautiful and modern.

Let's go back to the minuses. An additional stage in creating animation is rig (or rigging, like many terms in our industry, Russified as they could). That same creation of a skeleton and snapping bones to fragments. If the object has one animation, and not several, like, for example, the character of the game in the genre " Platformer ", then the stage is completely superfluous. Not all animations are easy to animate with bones. There are many animations where the bones interfere and without them more conveniently, and also faster, both in terms of creation and in terms of game performance. A separate item I make experts on the market. Their number with the growing popularity of skeletal animation is growing, but still not the easiest thing to find them in the team. The scatter of means and tools also affects, someone is used and uses one thing, someone else needs to be retrained on the go, which creates additional difficulties in finding and training employees.

Timeline Animation

I brought this type of animations to the very end, not because it is better than all the others, but because these are the very flash-animations about the insides of which will be discussed in the technical part of the article. Flash is not the only representative of this type of animation, but certainly their leader, so everything will be described relative to it. Timeline animations consist of fragments that are animated in separate layers on the timeline. Almost the same as we saw in skeletal animation, but without the skeleton. By keyframes on the timeline, we can move, rotate, replace, draw new fragments into these frames and, of course, interpolate translations, scaling, and rotations between them. That is, we move not the bones with attached fragments, but the fragments themselves. Of course, we lose many of the possibilities that skeletal animations give us, but not everyone needs them. Instead, we acquire others, such as:

• the ability to draw and insert new fragments in the middle of the animation;
• no rig, no bones, which at times only interfere;
• Adobe Flash (now Adobe Animate ) is the oldest and proven tool for animations over the years, a very large number of animators, one way or another, own it;
• the ability to use old animations when translating their flash-projects to Unity, including animations performed in vector graphics , and not only in bitmap .

Flash is dead? As a player in browsers - unconditionally, as a tool for animations - it is alive and well, there are no alternatives yet and it is not foreseen. Minuses? Of course. The main disadvantage is that this is not skeletal animation, huh. We lose the ability to mix animations with each other, we do not have inverse kinematics and similar features, inherent only in the skeleton. But we have timelines nested in each other, as well as raster and vector masks! All animators love masks! Here it is worth making a reservation that the Spine also recently had the possibility of clipping , but so far only geometric and strongly limited compared to flash masks.

Totals by types of animations

Naturally, I listed not all types of animations, only the main ones. For example, the so-called procedural animations were left behind, when objects or their fragments are driven only by code, but it’s not possible to oppose them to others, and it’s not very desirable to inflate the article, it’s better to look at something more specialized in animations to complete the picture.

To summarize Silver bullets, as always and almost everything, no. You need to choose a project, tasks and people. In the end, no one limits anyone to using one type of animation. Everywhere there are projects where all types of animations are used at the same time and everyone is happy. Cut scenes - video, characters - skeleton, scenery - timeline, flying emitters of particle systems - procedural, the particles themselves in their systems - time-lapse. We take the best from each type.

Flour choice and the decision to write his

Well, for some objective reasons, we decided that we need flash-animations for our project. There was a question about their integration into the engine. Having tried several plug-in options, we settled on one that suited us with its capabilities, was alive and supported, and also had a completely immodest price for an license, but you can be patient for a good product. I deliberately will not give any names, so as not to make anyone any advertising or anti-advertising.

With this plugin we lived for almost a year of development, during which the details were found out, which we could not use further. Namely - the quality of its integration into Unity. It all poured into frankly serious bugs, which are not to say that they were repaired easily, quickly and without a fight on the side of the developers, and in terrible performance on target devices, and we still have this level of iPad 2 . This happened because of the illiterate integration into a specific engine, ignorance of its specifics and pitfalls, according to the possibilities we were more than satisfied, but the source code, even in a personal order, was refused to open, and it was decided to write our own solution.

I had some quite successful experience in writing extensions for Unity and had little experience converting flash animations into my own formats. The latter was long enough, but something was remembered. In the same place, it was decided to write it all as a home project, so as not to depend on the project or the company where I work, and it’s no secret that I wanted to have such a product. So, armed with a 250-page specification of the swf-format , I went into battle.

Export options

First of all, you should discuss the options for exporting animations from a flash editor. There are several of them, naturally with their own advantages and disadvantages. Let's go through the main ones.

.xfl format

In the flash animation editor, you can save the animation source to an uncompressed .xfl format instead of the default .fla format, which is offered by default. The format is fairly simple, but undocumented. It consists of several subdirectories and a bunch of .xml-files, where all the states of the clips stored inside are described. The description format is also simple and clear, here is an example:

static_clip.xml

<DOMSymbolItem name="static_clip" itemID="5c719f28-00000051" lastModified="1550950184"> <timeline> <DOMTimeline name="static_clip"> <layers> <DOMLayer name="Layer_1" color="#00FFFF" current="true" isSelected="true"> <frames> <DOMFrame index="0" keyMode="9728"> <elements> <DOMBitmapInstance selected="true" libraryItemName="bitmap.png"/> </elements> </DOMFrame> </frames> </DOMLayer> </layers> </DOMTimeline> </timeline> </DOMSymbolItem> 

Here we have a static clip static_clip with one layer Layer_1 and one frame in which there is a bitmap image with the name bitmap.png .

movie_clip.xml

 <DOMSymbolItem name="movie_clip" itemID="5c719f30-00000053" lastModified="1550950713"> <timeline> <DOMTimeline name="movie_clip"> <layers> <DOMLayer name="Layer_1" color="#00FFFF" current="true" isSelected="true"> <frames> <DOMFrame index="0" duration="4" tweenType="motion" motionTweenSnap="true" keyMode="22017"> <elements> <DOMSymbolInstance libraryItemName="static_clip"> <matrix> <Matrix tx="-50" ty="-50"/> </matrix> <transformationPoint> <Point x="28.5" y="27.5"/> </transformationPoint> </DOMSymbolInstance> </elements> </DOMFrame> <DOMFrame index="4" tweenType="motion" motionTweenSnap="true" keyMode="22017"> <elements> <DOMSymbolInstance libraryItemName="static_clip" centerPoint3DX="128.5" centerPoint3DY="127.5"> <matrix> <Matrix tx="100" ty="100"/> </matrix> <transformationPoint> <Point x="28.5" y="27.5"/> </transformationPoint> </DOMSymbolInstance> </elements> </DOMFrame> </frames> </DOMLayer> </layers> </DOMTimeline> </timeline> </DOMSymbolItem> 

This describes the movie_clip animation clip, which contains two keyframes with indices 0 and 4, respectively. In the frames is our static clip static_clip coordinates (-50;-50) and (100;100) . There is a motion tween between frames (procedural animation, in our case it is only a transfer, without scaling and rotations), respectively, the position of the static clip, between key frames, can be obtained by linear interpolation of coordinates from these frames.

Naturally, everything will be in real animation, ahem ... somewhat more complicated and more voluminous, but this can all be understood without documentation. It would seem that here it is happiness. And I know several projects and companies that quite successfully went this way, but with some limitations and difficulties. These difficulties are the spoon of tar in the barrel .xfl, namely:

1. Tweens (tweens, Russified as they could) are very different, both simple: with linear interpolation, and more complex: with user functions and graphs of this interpolation, morphing vector graphics on tweens, with their own rules, only for macromedia and adobe;
2. Vector graphics, like all animation, except for raster graphics, is described in text form, an example . Accordingly, it is necessary to write your rasterizer, which, in view of the complexities and peculiarities of the vector in flash, is not possible in practice;
3. The rules for playing animations, including nested ones, need to be invented from the very beginning, there is no documentation for this and when which frame should be shown, and what situations to twin and leave without interpolation, we have to find out with long experiments, trying to cover all possible cases in test runs, which is a very non-trivial exercise for a general, rather than a private, solution.

These are not all the difficulties of this approach, but they are enough to understand that for a general solution it is only suitable with strong reservations. Long ago, I went down this path for one of the projects in which I participated. There were flash cut scenes, with classic tweens (without user interpolation rules and special functions of this interpolation), only raster graphics and no other advanced features provided by the flash editor. The private solution was written fairly quickly and efficiently, but we had to severely limit the animators so that they did not use anything that was somewhat complicated. The approach has the right to life and there are several libraries, of varying degrees of neglect, based on it, but, as I said, with a lot of reservations and restrictions.

.jsfl scripts

Another, almost working, option to get information about our animations. jsfl-scripts allow you to extend the flash-editor, interact with its environment, change the animation and, of course, get all the necessary information about the timelines, layers, clips and frames in them. And it is also used by animators to automate various actions, but this is a story from a nearby book. In general, the approach is endowed with all the drawbacks of the previous approach, so I will not dwell on it, I can only say that with it you can unload all of our animation in the form of, for example, frame-by-frame animation, but this is not the way of a real Jedi (but, naturally, takes place in some projects). We will return to these scripts in the approach that I chose: to rasterize vector graphics and optimize the paged animation.

AIR application

But this approach is completely working and can be used as it should. I chose not him, but for reference, and give this option. The essence of the approach is that we create an AIR application on the flash itself or, for aesthetes, on Haxe , for example, which will get all the information from the animation already compiled into the swf format. We play frame by frame animation within our application, get all the information about frames and save it in the format we need for our runtime. All the problems outlined above are solved here:

• no need to rasterize vector graphics, flash-runtime will do it for us, all that remains is to get this information and save the raster result of vector fragments;
• no need to invent playing rules and deal with user interpolation of twins, the flash player inside the AIR application knows exactly how it is done and does almost everything for us;

As a bonus, we get the opportunity to use personnel scripts in animations (all kinds of play() , stop() and other gotoAndPlay() , some animators love them very much). By minus, we get the inability to export the looped ones in the flash animation itself, but it doesn’t matter, since you can loop them in your runtime, and ask the animators to prepare everything for this.

Here it is worth making a reservation that I am sure that everything will be somewhat more complicated than it sounds in my description, since I personally did not go there, so I cannot tell the details. Let those who have passed this way share their experience, be happy to read about their joys and adversities!

Own flash player

This is probably the most honest, obvious and straightforward option of all, but also the most difficult. In the end, this is what a real flash-player does, which stood with most of us in our favorite browser as an addition to it. Popular representatives of this genre: gameswf and, grown from the previous, scaleform . Both are now dead. Interestingly, they were used mainly for implementing GUI in various projects, including AAA . But we are interested in the internal structure, and not in the specific applications of the dead libraries.

Any self-respecting flash player has at least the following things on board:

• parser swf-format;
• rasterizer vector graphics;
• implementation of a virtual machine to run ActionScript code;
• implementation of the standard library for custom code;

Each of these points screams that if it is possible to implement, you will have to change more than one command for this task and exchange several years of development to the minimum version. Even the implementation of a small part, sufficient to play something, will take a very long time, and the additional complexity, in the form of so many undocumented parts of the entire pipeline, will double this already huge period.

Yes, the network is full of attempts to implement each of these parts, of varying degrees of neglect and misinterpretation. But even taking these developments as a basis is not possible, only bugs will have to be caught from there for years, and unwritten unit tests should be written before realizing that it will not be possible to fully implement everything. Those who tried to supplement gameswf with the necessary capabilities understand what I'm talking about. An honest flash player should also rasterize vector graphics not in advance, but in runtime, which will affect such a performance hit that not every project can survive. Taking this opportunity, I say hello to the survivors who used the scaleform on mobile devices. In general, the darkest and hopeless way, not our choice.

We combine approaches

That finally came to my option. Having studied the possible ways, I had one interesting combination. Relatively simple to implement, manageable to one person in size and at the same time functional enough to cover the needs of most 2d timeline animations.

First, we will use the animation compiled into the swf, so as not to invent various playback options and almost never guess how the flash player works from the inside, since we can’t cover all possible cases, but we still try to write a common solution. Also, the flash editor gets rid of all the twins that were used in the animation, slyly presenting, in the compiled swf, the bare positions of the fragments.

Secondly, we prohibit the use of scripts in animations. Yes, this is a very tough and sad requirement, but I do not have a team of programmers to implement an honest virtual machine and a standard library that I would have to write almost blindly. , , AIR-, - , . , . frame labels , , .

-, , , flash. (!) , jsfl-. , , , , : / . / (, HD SD ).

. , , (!), , . , ! flash- ZeptoLab Playrix .