Kinect taught to recognize and translate sign language

Despite the fact that the latest gaming gadget from Microsoft, Xbox One, was perceived somewhat skeptical (for several reasons, some of which have already been eliminated by the company), the same Kinect remains one of the most interesting technological advances of recent years. And not in the sense of adaptation to the game, but in the sense of the possibility of working with a person as if “directly”. The other day, Kinect “taught” to recognize sign language and translate it into phrases that we all understand on the screen.

The new development is a joint project of Microsoft Research Asia and the Institute of Computer Technology of the Chinese Academy of Sciences, Beijing. Of course, the software plays a big role in the project, Kinect itself, despite all its might, is just a piece of hardware without software. So, Kinect in this case recognizes the visual information provided by the person, and the corresponding software perceives this information, analyzes it and translates it into a language that we all understand. There are two components in the project - actually, “translation”, and work in two directions, when the system translates into sign language and plain text printed by a person with normal hearing and language skills.
')
In the latter case, a 3D avatar is used, which communicates in sign language. By the way, all this is not part of the game world of the Xbox, it is a serious scientific study (with the corresponding achievements), the results of which were published .

Most likely, this development will find its place in the system of communication technologies from Microsoft. As mentioned above, not only for the game.



Via microsoft