Making your Gmail Motion

A few days ago, I don’t remember the exact date, Google introduced its new email tool, Gmail Motion . About it even wrote on Habré. Having decided to take advantage of this beautiful feature, I, to my great regret, did not find an opportunity to enable it in my Gmail account. Probably, I did not join the group of beta testers. This, of course, is frustrating. But not us! Below I will tell you how I made my Gmail Motion for a bottle of one hour. Immediately video with the result, under the cut - technical details.

Assembly Instructions

1. Take Microsoft Kinect.

We put on a flat surface, turn on the outlet and USB-port.
')

2. Take some kind of gesture recognition library for Kinect.

I used FAAST as a good working and able to send different messages to other applications, which gives us no need to program anything. (But in general there are already dozens of such libraries, and I don’t presume to say which one is the best). How to install FAAST (and the necessary libraries for it), detailed here .

3. Find a list of hotkeys for Gmail.

Here it is . We enable hotkeys in the Gmail settings .

4. Making a config file for FAAST.

I limited myself to the “Open Letter”, “Reply”, “Reply to all” and “Send” commands shown in the demo. In addition, I had to implement the “Open Inbox” command, because I do not understand how without this command you can fully demonstrate the work of the others without the need for a keyboard or mouse. The gesture was not shown in the video for this command, but the full specification of the standard includes it - this is the body inclination to the left. In addition, I replaced the "Answer All" command with the swing in the video back with both hands on the back swing with my left hand. There are two reasons for this:

• I believe that Google experts have not worked enough on the ergonomics of the standard. “Reply to all” is a quite frequent task and there is nothing to hang up with two hands on it, if it can be done with one left hand (in both senses of this phrase). And this is not to mention the infringement of the rights of one-armed.
• In the standard version, the FAAST library does not allow catching a complex action with several hands. You can, of course, take another library or add an intermediate gesture interpretation layer for FAAST, but I wanted to get at least something working as soon as possible.

Config file:

# -
lean_left 20 key_press u

#
right_arm_out 7 key_press o

#
right_arm_up 8 key_press r

#
left_arm_up 7 key_press a

#
right_foot_up 9 key_press tab
right_foot_up 10 key_press enter

5. Run FAAST, open the config, run the emulator.

Voila! Everything is working. Gmail is open on the left of the video, the skeleton recognized by FAAST is in the upper right, the picture below is from the camera, and even lower is the keystroke that is transmitted to the browser window. You can optionally add support and other actions.

findings

Using Kinect, it’s possible to implement many (if not all) Google’s suggested Gmail user interface responses. Of course, if you take the matter seriously, you will still have to get into programming, solve the problem of false positives, perhaps change the gesture recognition library and add an understanding of complex actions in several parts of the body simultaneously. But now I am sure it is possible. Of course, you can not detract from the achievements of Google - because their functionality works on ordinary cameras, and not on Kinect. And the experiment was conducted solely for non- scientific purposes.

PS Despite the fact that Gmail Motion is Google's wild banter, everything described in this article really exists and works as described . The movie is also real.