SignAloud gloves: translation of gestures into words
Surely each of us would be glad to know as many foreign languages as possible. In addition to the teaching literature, online courses and lessons, there are many technologies, devices and applications that contribute to this in every possible way. Most of the developments are aimed at studying oral speech and grammar, while millions of people live in the world using sign language for everyday communication. And now it is much more difficult to master such a language than a foreign one.
However, this imbalance usually precedes progress. Such a multifaceted niche could not remain free for a long time. And it was decided to take it by two students at Washington University - Thomas Pryor and Navid Azodi. They came up with a way to make sign language more accessible to others. Young inventors have developed a system that provides the ability to translate "gestural" speech into ordinary oral.
')
The guys have created advanced SignAloud technology gloves, consisting of various sensors that monitor the slightest movements of fingers, wrists and hands, have a Bluetooth wireless interface for transferring the collected data to a PC and special software. Gloves are able to catch the slightest movements of the user's hands, read signs of sign language in movements and, after deciphering the received data, convert them into oral speech using a special synthesizer.
Of course, the device can be effectively used and when there is no possibility to hear the received translation - sign language signs are converted into written text. By Bluetooth gloves send information about gestures to the central computer, where using statistical data processing it is decided which word was “spoken” in the gesture. Data analysis is based on the processing of sequential statistical regression.
The IT community has embraced the creators of SignAloud as true technological geniuses. The gloves were invented as part of the Lemelson-MIT Student Prize competition (2016) and of course brought victory to their authors. As a result, the guys received a grant of $ 10,000 for further continuation of the improvement and production of the product.
Navid Azodi emphasizes that there are currently about 70 million deaf-and-dumb people who use sign language daily in the world. But this is a large group (by the way, it exceeds the population of France), which is scattered among people without such functional limitations. And the question of their constant communication with the others is more than relevant, which, in turn, places additional demands on the device. Gloves should be more comfortable, compact and adapted to different operating conditions.
About the introduction of the device in everyday life, Thomas Pryor notes the following:
Of course, there are other sign language devices on the market, but really practical products are very rare. SignAloud’s success lies in the easy and immediate translation of sign language into spoken language. The gloves themselves are not bulky, the technical device is also not large. True, the appearance of SignAloud may seem somewhat awkward (unless you are not in a superhero costume), but the developers have already begun to work on a more casual and pleasant design.
Sign language translation is an impressive accomplishment in itself. But the creators of SignAloud do not stop there. They believe that this technology will be used for successful development in a wide range of other tasks. Whether it will be a medical field or a virtual reality is not known. But Pryor and Azodi hope to decisively change the world for the better.
The video shows the work of the demo version of gloves performed by their creators.
At the moment, the development exists only in the form of a prototype and when the gloves go on sale it remains a mystery. In addition, the translation is carried out only in English.
This is what Navid Azodi says about this:
Friends invented their gloves on the campus where they currently live. Young talents work in the community for CoMotion development students.
However, this imbalance usually precedes progress. Such a multifaceted niche could not remain free for a long time. And it was decided to take it by two students at Washington University - Thomas Pryor and Navid Azodi. They came up with a way to make sign language more accessible to others. Young inventors have developed a system that provides the ability to translate "gestural" speech into ordinary oral.
')
The guys have created advanced SignAloud technology gloves, consisting of various sensors that monitor the slightest movements of fingers, wrists and hands, have a Bluetooth wireless interface for transferring the collected data to a PC and special software. Gloves are able to catch the slightest movements of the user's hands, read signs of sign language in movements and, after deciphering the received data, convert them into oral speech using a special synthesizer.
Of course, the device can be effectively used and when there is no possibility to hear the received translation - sign language signs are converted into written text. By Bluetooth gloves send information about gestures to the central computer, where using statistical data processing it is decided which word was “spoken” in the gesture. Data analysis is based on the processing of sequential statistical regression.
The IT community has embraced the creators of SignAloud as true technological geniuses. The gloves were invented as part of the Lemelson-MIT Student Prize competition (2016) and of course brought victory to their authors. As a result, the guys received a grant of $ 10,000 for further continuation of the improvement and production of the product.
Navid Azodi emphasizes that there are currently about 70 million deaf-and-dumb people who use sign language daily in the world. But this is a large group (by the way, it exceeds the population of France), which is scattered among people without such functional limitations. And the question of their constant communication with the others is more than relevant, which, in turn, places additional demands on the device. Gloves should be more comfortable, compact and adapted to different operating conditions.
About the introduction of the device in everyday life, Thomas Pryor notes the following:
Our gloves are lightweight, compact and completely ergonomic, to be used as a daily accessory, just like using a hearing aid or contact lenses.
Of course, there are other sign language devices on the market, but really practical products are very rare. SignAloud’s success lies in the easy and immediate translation of sign language into spoken language. The gloves themselves are not bulky, the technical device is also not large. True, the appearance of SignAloud may seem somewhat awkward (unless you are not in a superhero costume), but the developers have already begun to work on a more casual and pleasant design.
Sign language translation is an impressive accomplishment in itself. But the creators of SignAloud do not stop there. They believe that this technology will be used for successful development in a wide range of other tasks. Whether it will be a medical field or a virtual reality is not known. But Pryor and Azodi hope to decisively change the world for the better.
The video shows the work of the demo version of gloves performed by their creators.
At the moment, the development exists only in the form of a prototype and when the gloves go on sale it remains a mystery. In addition, the translation is carried out only in English.
This is what Navid Azodi says about this:
Keep in mind, we have not fixed all the languages. This is still far away. English is more than just a set of words or specific phrases. And we understand that perfectly. The language is filled with various complex grammatical structures. What do you want to achieve in the final result? Allow SignAloud gloves to recognize and process most sign language.
Friends invented their gloves on the campus where they currently live. Young talents work in the community for CoMotion development students.
Source: https://habr.com/ru/post/369355/
All Articles