Kwiki - control your hands!

Good day, the community

This publication will interest those who are not indifferent to innovations in user interfaces, as well as those who are passionate about Augmented Reality .

If we look closer at the work of the human brain, try to recreate the tasks that it performs, in particular, those that we take for granted, we can see how enormous the amount of calculations performed by the brain is. For example, motion detection and tracking, or the difference of more than one object, their movement relative to each other, their speed and the distance between them.
')
I note immediately that Kwiki The Bird and the development of such games is not the main focus of our company . The goal of the game was to check the capabilities of various algorithms for determining / tracking movements of a computer webcam.

What we did:

You can play the game here: http://kwikithebird.com

Marketing part:

For the promotion of the game, we chose a platform for distributing free indie games - like kongregate . The best in our opinion were: http://www.kongregate.com , http://www.gamemontage.com . The number of views / games on the day of publication amounted to 500 people, without additional methods of promotion.

Game concept:

The technology of motion recognition, the algorithms of which exist now ( below the link ), is quite cheese. Inaccuracy of tracking. At first we thought about hiding such inaccuracies by the original idea of ​​the game - a drunken bird, which is difficult to manage.

Our task was not to keep the user in the game for a long time. Since this is not possible for such games, since controlling the camera (in most cases embedded in the top panels of laptops, iMacs and just standing on the monitor), your hands get tired very quickly. Namely, this is your controller, which is constantly in the air. Therefore, I wanted to see the viral distribution of the game among the audience.

Actually it happened, according to statistics (each user who played the game, brought his friend, as can be seen from the photos taken during the game).

Therefore, I want to call this item more or less successful. Still, games with a new control interface, namely a web camera, are viral and can spread themselves.

Next, watching the popular video about the bird Kiwi, which can not fly and tends to the sky - Kiwi (YouTube Most Adorable video of 2006). We decided to make a kind of sequel, using the idea of ​​a bird kiwi, which was supposed to hide some flaws in technology.

Technical part:

Designer work - 20 hours



1. Creating an image of the main character - Kviki.
2. Design 20 levels of the game.
3. Design of trophies (3 for each subgroup of levels - Earth, Above ground at night, Sky, Space). Trophies do not carry any special value, but increase the user's interest, which, in general, stimulates to continue the game and fly higher.
4. “Bad objects”, their own for each level, for the capture of which points are taken away from the user.
5. "Deadly" enemies, a collision which means the end of the game.

The work of a flash programmer (game logic) - 40 hours

With logic there was a small amount of rework. At the beginning there was one screen and the objects just fell from top to bottom. We decided to add the feeling of flying, every time Kwiki eats a good object, he moves n pixels up to his dream, and, accordingly, when he eats a bad object, he goes down.

Launching the first test of the game among our colleagues / friends, surprisingly, we saw that many manage to control the bird in spite of the technology flaws and different light conditions (dark / bright room). And therefore, it was decided to add “deadly objects” in the senior groups of levels:

This is a Thundercloud, a plane and a UFO, of course not, but impressive enough on the last subgroup of levels - Cosmos.

Moreover, some testers even managed to read, cheat, control the bird only at the top of the screen and collect all the bad / good objects, gaining the maximum points.

Programming work (facebook) - 20 hours

Because We already had experience in creating Facebook applications, with this game we decided to try to enter the market (although this is loud, given the goals we set for ourselves) on Facebook.

The main aspects were added:

• facebook connect for stand alone applications (games from the site),
• the function of automatic publication of the results in the news feed,
• The ability to view game activity directly from the stand alone website and the I Like button, which affects the output of search results on Facebook, is the most.

Work with gesture recognition algorithms - 80 hours

Here, perhaps, the most interesting thing I want to share with you. What algorithms we used, what changed, and most importantly, why:

Inspired by the work of Grant Skiner and of course Mario Klingermann aka Quasimondo . The latter, by the way, has a fairly large number of examples ported from C, openCV, to flash, to recognize faces and tracking, as well as what is called NFT (Natural Feature Tracking) and tracking of color objects.

The last algorithm in the list, but not the last one for us, but on the contrary the first one I would like to point out is the Justin Wind motion recognition algorithm.

Because Our bird should fly, we had to not only determine the movement, but also do its tracking.

Actually the similarity of our algorithms was in the division of the screen into a grid, we average the colors within each cell / segment at regular intervals. The difference in brightness within each segment (from one calculation to the next) will determine whether or not movement occurred in this area of ​​the screen.

After analyzing all the works, this approach turned out to be the most accurate for us. Although, in reality, it takes a lot of action to get Pixel () , to set Pixel () , and also to convert these results into color, saturation, brightness of objects, each of which is heavy enough for the brain of our computer - the processor. A good computer, of course, copes with this easily, for example, iMac 3.06GHz copes with it with a bang.

We summarize how it works in Flash:

• After the camera object is created, it is passed to the MotionTracker class, which extends the Point object. The MotionTracker class contains two BitmapData objects, one representing the current data from the webcam, and the other used to store the previous frame.
• When updating the method is called, a new frame is drawn on top of the previous one using the filter difference.
• The results are obtained by ColorMatrixFilter , increasing the contrast of the image and dropping dark areas (with less movement) further to the background.
• The blur filter is then applied by us in order to further reduce the noise level and form the “Part of the Movement” from areas where more movement occurs.
• Trashhold is called as a result of BitmapData , in fact, as a result of which the grid next to black pixels should be ignored, and “motion detected” is assigned to all bright pixels.

The treshhold level can be adjusted in the game settings, with a simple slider that moves from a relatively bright room to a dark one.

The parameters that make up are brightness, blur (blur), contrast and spacing of the segments of that grid itself.

Future plans

1. Starting VKontakte after creating the second beta version and eliminating all the bugs.
2. Add game mode training or how to.
3. Automatic detection of the active camera.
4. Adding more design and sound effects.
5. Changing the animation action Kviki.
6. Posting photos of the user on the walls of friends on facebook, increasing the sharing parameter.
7. Change background music :).

Thank you for attention.

Lastly, a fan-like video of Kwiki playing: