Unity3D + Kinect, or animation on the knee
Introduction
Hello, dear reader. Before I begin, I would like to talk a little bit about the purpose and format of my article. Usually, with such headings, articles are of a reviewing nature, they describe the technical aspects of the implementation of a decision or provide instructions for use, etc. The reader makes his own conclusions. Before writing this article, I conducted a little research work on the subject of analogues. And I found several articles both at the very hub, and a lot of video materials on YouTube. Therefore, I decided not to write another technology review, but to share my impressions, talk about the rake we stepped on, and bring in a bit of “kitchen” analytics.
The purpose of this article is to help you decide whether or not to use this method. First of all, it is designed for technical specialists, although it is not replete with technical details. But it will also be useful for everyone else - art specialists, project managers and just everyone who is somehow connected with the development of games. Perhaps a link to this article will help to beat the budget with the authorities and speed up the development / increase the quality of your future game. So let's get started.
Prerequisites
Animation is often a “problem place” for many 3D projects. Of course, everything depends on the specifics of the project, but often this specificity is determined precisely by the complexity and high cost of producing high-quality animation. Models, textures, sketches, effects and even the interface to find / order is easier than a package of specific animations for the character of your game. Sometimes game design is limited by the framework that the animation sets. This is most clearly seen in indie projects and projects of small teams. Alas, sometimes we see problems with animation and large developers. For example, a new game comes out, in which everything seems to be good: the gameplay is up to par, and the models are beautiful, and the style is sustained, and even the effects seem to be nothing ... But the game looks dull, because the characters run like planted or all abilities in the game have three animations. Repeatedly, I witnessed the death of interesting projects that were never born, because the developers could not find the resources specifically for the animation. As a result, among developers there is a perception that high-quality and diverse animation is expensive in terms of resources. And for the money, and for the time. Especially if we want realistic animation of humanoid characters. We faced the problem of animation in one of our projects and want to share the experience of its solution, which may be useful to you, dear readers. So, we had two problems.
First, the conflict of desires and possibilities. I wanted a lot, high quality and diverse. The budget and resources were limited, and time was running out.
Secondly, experimental gameplay. During the development process, some game concepts changed, and so did the visualization. I wanted to be able to quickly respond to possible changes or adding features. After a bit of research into options and opportunities, we thought about using Motion Capture technology, as it met all our needs. Quality, variety, the ability to change animations on the fly, etc. But the price of technology was too high. On average, renting a Motion Capture studio in Moscow costs 800-1000 USD. per day, cleaning animation after recording 1-2 cu second. We began to look for alternatives ... and found - using Kinect + iPi Motion Capture Software.
')
Impressions and solutions to problems
I will not describe the process of recording animation and go into technical details, as it is described in detail on the website of software developers (iPi Motion Capture Software) and in this article . Impressions of using this technology are more than positive. With minimal resources we got a quality result. The whole process — from recording movements to importing results into Unity3D — takes an average of 1-3 hours, depending on the length of the recording (most of the time is an automatic process of recognition and post-processing). At the output, you get a ready-made fbx file, which you simply import into Unity3D, and you can get quite an acceptable result without even doing cleaning. And the participation of a professional animator is not required. No need for special room and clothing. The toolkit provides the ability to automatically “clean” the animation, and manual refinement in the process. The main advantage is that you can export your own skeleton and link it with the skeleton from the record. This allows “on the fly” to see the future result and adjust it as necessary, and of course, export the final result tailored to your needs.
Naturally, in the process of work, we stuffed cones and made conclusions:
1. Knowledge - power - do not rush to record and export animation. Carefully read the examples and manuals, watch the video on YouTube. A complete understanding of the process will save you a lot of time and nerves.
2. From simple to complex - do not rush to immediately record long and complex animations. Start simple. In this way, you will gradually become familiar with technology and gain experience using it.
3. Calibration - when calibrating 2x Kinects, we did not immediately figure out how to properly adjust the cameras, and for a couple of days we got bad results. It is very important that after automatic calibration by a special recording of the camera in the software editor as closely as possible coincide with the spatial location that you used when recording the calibration video. This can be achieved by moving the cameras in the editor after calibration. Usually it is necessary to do several iterations until the match is accurate.
4. Prepare a place - although a special place is not required for recording, but it is still worth preparing in advance. Note that you need 4x4 meters of free space. Also it is necessary to mark the position for the legs on the floor, for example with the help of electrical tape. This will help to maintain the correct position between movement recordings. Put a large mirror out of sight of the cameras so that you can see yourself with it: this is how you can control the accuracy of movements.
5. Training and doubles - in advance rehearse and memorize those movements that you plan to record and write several duplicates of each movement in order to choose the best one afterwards.
6. Post-processing - the software provides quite good post-processing tools. Even if the recognition of any bones were lost, you can move, adjust them directly in the editor. There are also tools for smoothing fine jitter, etc. After several entries, we came to the conclusion that it is more convenient to do the revision in the editor, since after exporting it is not very convenient to set up animations in another editor. Although this item is rather subjective.
Results
Using Kinect to record animations in practice shows itself as a fast, easy-to-use and inexpensive way to get high-quality animations for your games and more. The price of the issue on average is 12–40 thousand rubles, depending on the chosen license for ipiSoft and the number of Kinects, which is not a big financial barrier not only for studios, but also for indie developers. Wherein
Considering how much time and resources will be saved when working with outsourcers or full-time animators, this approach turns out to be very profitable. In addition, the technology can be tested absolutely free. The 30-day trial extends to all the necessary software, and getting 1–2 Kinect is not a problem at all. Of course, Kinect is not a panacea for all problems. It is impossible, of course, to record animations.
non-humanoid characters. And complex animation with sharp movements cannot be recorded. But technologies are developing, for example, the same Kinect - Demonstrating the capabilities of Kinect 2.0 - with this recognition, the quality of the animations will be simply superb. The benefits of using Kinect for MotionCapture are obvious. Just gotta try. I am pleased to answer any questions related to the topic of the article in the comments.
Links
Since laziness is the engine of progress, I provide a list of useful links to whom laziness to google yourself:
1. ipiSoft Motion Capture - actually the site of the software, where there is everything you need. Lessons, examples, software distributions, etc.
2. Exporting MoCap data to Unity3d is a useful article describing the process of calibration, export of animation and post-processing.
3. Selection of videos on YouTube - many examples of how, what, where and why.
4. Demonstration of Kinect 2.0 features - demonstration of the second version of the controller. Prospects are very interesting.
Hello, dear reader. Before I begin, I would like to talk a little bit about the purpose and format of my article. Usually, with such headings, articles are of a reviewing nature, they describe the technical aspects of the implementation of a decision or provide instructions for use, etc. The reader makes his own conclusions. Before writing this article, I conducted a little research work on the subject of analogues. And I found several articles both at the very hub, and a lot of video materials on YouTube. Therefore, I decided not to write another technology review, but to share my impressions, talk about the rake we stepped on, and bring in a bit of “kitchen” analytics.
The purpose of this article is to help you decide whether or not to use this method. First of all, it is designed for technical specialists, although it is not replete with technical details. But it will also be useful for everyone else - art specialists, project managers and just everyone who is somehow connected with the development of games. Perhaps a link to this article will help to beat the budget with the authorities and speed up the development / increase the quality of your future game. So let's get started.
Prerequisites
Animation is often a “problem place” for many 3D projects. Of course, everything depends on the specifics of the project, but often this specificity is determined precisely by the complexity and high cost of producing high-quality animation. Models, textures, sketches, effects and even the interface to find / order is easier than a package of specific animations for the character of your game. Sometimes game design is limited by the framework that the animation sets. This is most clearly seen in indie projects and projects of small teams. Alas, sometimes we see problems with animation and large developers. For example, a new game comes out, in which everything seems to be good: the gameplay is up to par, and the models are beautiful, and the style is sustained, and even the effects seem to be nothing ... But the game looks dull, because the characters run like planted or all abilities in the game have three animations. Repeatedly, I witnessed the death of interesting projects that were never born, because the developers could not find the resources specifically for the animation. As a result, among developers there is a perception that high-quality and diverse animation is expensive in terms of resources. And for the money, and for the time. Especially if we want realistic animation of humanoid characters. We faced the problem of animation in one of our projects and want to share the experience of its solution, which may be useful to you, dear readers. So, we had two problems.
First, the conflict of desires and possibilities. I wanted a lot, high quality and diverse. The budget and resources were limited, and time was running out.
Secondly, experimental gameplay. During the development process, some game concepts changed, and so did the visualization. I wanted to be able to quickly respond to possible changes or adding features. After a bit of research into options and opportunities, we thought about using Motion Capture technology, as it met all our needs. Quality, variety, the ability to change animations on the fly, etc. But the price of technology was too high. On average, renting a Motion Capture studio in Moscow costs 800-1000 USD. per day, cleaning animation after recording 1-2 cu second. We began to look for alternatives ... and found - using Kinect + iPi Motion Capture Software.
')
Impressions and solutions to problems
I will not describe the process of recording animation and go into technical details, as it is described in detail on the website of software developers (iPi Motion Capture Software) and in this article . Impressions of using this technology are more than positive. With minimal resources we got a quality result. The whole process — from recording movements to importing results into Unity3D — takes an average of 1-3 hours, depending on the length of the recording (most of the time is an automatic process of recognition and post-processing). At the output, you get a ready-made fbx file, which you simply import into Unity3D, and you can get quite an acceptable result without even doing cleaning. And the participation of a professional animator is not required. No need for special room and clothing. The toolkit provides the ability to automatically “clean” the animation, and manual refinement in the process. The main advantage is that you can export your own skeleton and link it with the skeleton from the record. This allows “on the fly” to see the future result and adjust it as necessary, and of course, export the final result tailored to your needs.
Naturally, in the process of work, we stuffed cones and made conclusions:
1. Knowledge - power - do not rush to record and export animation. Carefully read the examples and manuals, watch the video on YouTube. A complete understanding of the process will save you a lot of time and nerves.
2. From simple to complex - do not rush to immediately record long and complex animations. Start simple. In this way, you will gradually become familiar with technology and gain experience using it.
3. Calibration - when calibrating 2x Kinects, we did not immediately figure out how to properly adjust the cameras, and for a couple of days we got bad results. It is very important that after automatic calibration by a special recording of the camera in the software editor as closely as possible coincide with the spatial location that you used when recording the calibration video. This can be achieved by moving the cameras in the editor after calibration. Usually it is necessary to do several iterations until the match is accurate.
4. Prepare a place - although a special place is not required for recording, but it is still worth preparing in advance. Note that you need 4x4 meters of free space. Also it is necessary to mark the position for the legs on the floor, for example with the help of electrical tape. This will help to maintain the correct position between movement recordings. Put a large mirror out of sight of the cameras so that you can see yourself with it: this is how you can control the accuracy of movements.
5. Training and doubles - in advance rehearse and memorize those movements that you plan to record and write several duplicates of each movement in order to choose the best one afterwards.
6. Post-processing - the software provides quite good post-processing tools. Even if the recognition of any bones were lost, you can move, adjust them directly in the editor. There are also tools for smoothing fine jitter, etc. After several entries, we came to the conclusion that it is more convenient to do the revision in the editor, since after exporting it is not very convenient to set up animations in another editor. Although this item is rather subjective.
Results
Using Kinect to record animations in practice shows itself as a fast, easy-to-use and inexpensive way to get high-quality animations for your games and more. The price of the issue on average is 12–40 thousand rubles, depending on the chosen license for ipiSoft and the number of Kinects, which is not a big financial barrier not only for studios, but also for indie developers. Wherein
Considering how much time and resources will be saved when working with outsourcers or full-time animators, this approach turns out to be very profitable. In addition, the technology can be tested absolutely free. The 30-day trial extends to all the necessary software, and getting 1–2 Kinect is not a problem at all. Of course, Kinect is not a panacea for all problems. It is impossible, of course, to record animations.
non-humanoid characters. And complex animation with sharp movements cannot be recorded. But technologies are developing, for example, the same Kinect - Demonstrating the capabilities of Kinect 2.0 - with this recognition, the quality of the animations will be simply superb. The benefits of using Kinect for MotionCapture are obvious. Just gotta try. I am pleased to answer any questions related to the topic of the article in the comments.
Links
Since laziness is the engine of progress, I provide a list of useful links to whom laziness to google yourself:
1. ipiSoft Motion Capture - actually the site of the software, where there is everything you need. Lessons, examples, software distributions, etc.
2. Exporting MoCap data to Unity3d is a useful article describing the process of calibration, export of animation and post-processing.
3. Selection of videos on YouTube - many examples of how, what, where and why.
4. Demonstration of Kinect 2.0 features - demonstration of the second version of the controller. Prospects are very interesting.
Source: https://habr.com/ru/post/192820/
All Articles