Crowdfunding vOICe sound vision technology - for the blind and not only
Ears can be loved, they can hang noodles. And with ears you can ... see.
Many years ago, the Dutchman Peter Meyer, an employee of the Philips research and development department, began work on the Seeing With Sound project, designed to replace vision for blind people. The idea was very unusual and at first glance quite controversial. The human brain is quite plastic. And if you turn the image from the camera into a sound sequence, it is possible that after training the user's brain can learn to perceive the screeching whistling trill not as a sound, but as a monochrome picture and to distinguish between dark and light areas of this “audio image”. At first glance, the idea is not obvious. But its authors managed to find like-minded people and risked their own time and money spent on equipment. Thus was born the “sound” vision system vOICe - (the abbreviation of the three capital letters sounds like “Oh I See”).
')
According to the experimenters, after long and regular training, blind people who used vOICe could move around in the premises without running into objects, and even read large text, such as signs on signs.
Such a project could not fail to attract the attention of the transhumanist community of the RTD even in the 00s, but only by the mid-10s the progress of iron and Internet resources allowed the development of vOICe to give a new impetus and organize a campaign on Boomstarter , which is successfully completed by more than 100% today . The basic project provides for the creation of 5 pilot copies of the system for blind users plus their training, but if someone wants to help another person with disabilities to become a little Batman, and at the same time get software and educational materials for experiments and a souvenir, then it can be done do until 20:34 MSK on August 4th. You can read more about the project on the campaign page, and now a few words about the history of its development from Yana Artisheva aka telegamochka :
Five years ago we decided to create a similar system. I was interested in this topic, because I was directly related to the creation of a wearable computer (NK). In addition, at that time head-mounted displays were inaccessible, expensive and cumbersome, and I was interested in the development of a sound interface to the NK. So when my friends from the RTD told me about the vOICe project, I joined the development with great interest.
We have created a vOICe-system based on the ASUS netbook and a sports “hirat” with a webcam embedded into it. The system turned out to be cumbersome, not the most convenient, with a small (less than an hour) operation time - but quite suitable for experiments.
Unfortunately, we did not find support in the blind forums. There were not even people who would be interested in trying out the system. However, a netbook with a standard OS, webbing straps and a wire from a webcam mounted on the head was of little use so that a blind person could use it without outside help. The prototype is a prototype - and no more than that ...
What is much less surprising - did not interest our project and state institutions.
And we began to independently master the system. But seeing it is much more difficult. First of all, the motivation is much less strong. Yes, we made the system. Yes, it is interesting to learn how to use it. Interesting - but no more. Secondly, in a sighted person, of all the ways of perception, sight is in the first place, the rest of the senses, including hearing, are usually in the background and the sighted person’s brain is usually less sophisticated in the processing of sound information compared to the perception of visual images. To catch the fake singer, to distinguish the notes of the played chord - yes, we are capable of that. But in order to see with the help of hearing, the sighted one needs to develop and develop the auditory info channel. While a person who has not used his eyesight for many years, hearing is an order of magnitude better suited to “fine work” with sound information. It is enough to remember about blind people who have managed to develop their ability to echolocate, like dolphins and bats.
And nevertheless, I managed to achieve some success in mastering vOICe. I did not learn to walk around the room blindfolded, but after several weeks of training I was able to first determine which figure was drawn on the voiced picture - a rectangle, an oval or a segment. Then I learned to “hear” several mutually intersecting figures, sketch their mutual position and determine the brightness of each object. Not 100%, but at 60-70 I managed to quite accurately perceive the picture. I tried to play in the audio version of the good old "Wolfenstein" - but, alas, absolutely unsuccessful.
Having existed just over a year, our project gradually faded away.
As you can see, the era of Raspberry Pi, Epson Moverio, Google Glass and crowdfunding * starters gives such projects a much better chance of success than 5 or 10 years ago. It so happened that it was possible to publish this project specifically on GT and in general in the habrasemeet only today, at the finish line, but this is only the first pilot experiment. In this connection, I propose to think: what other initiatives for helping people with disabilities can be realized precisely thanks to today's technologies, and what other technologies of a universal nature (platforms like the ones mentioned above) are still not enough for us to significantly increase in 5-10 years the quantity and quality of such projects?
Many years ago, the Dutchman Peter Meyer, an employee of the Philips research and development department, began work on the Seeing With Sound project, designed to replace vision for blind people. The idea was very unusual and at first glance quite controversial. The human brain is quite plastic. And if you turn the image from the camera into a sound sequence, it is possible that after training the user's brain can learn to perceive the screeching whistling trill not as a sound, but as a monochrome picture and to distinguish between dark and light areas of this “audio image”. At first glance, the idea is not obvious. But its authors managed to find like-minded people and risked their own time and money spent on equipment. Thus was born the “sound” vision system vOICe - (the abbreviation of the three capital letters sounds like “Oh I See”).
')
According to the experimenters, after long and regular training, blind people who used vOICe could move around in the premises without running into objects, and even read large text, such as signs on signs.
Such a project could not fail to attract the attention of the transhumanist community of the RTD even in the 00s, but only by the mid-10s the progress of iron and Internet resources allowed the development of vOICe to give a new impetus and organize a campaign on Boomstarter , which is successfully completed by more than 100% today . The basic project provides for the creation of 5 pilot copies of the system for blind users plus their training, but if someone wants to help another person with disabilities to become a little Batman, and at the same time get software and educational materials for experiments and a souvenir, then it can be done do until 20:34 MSK on August 4th. You can read more about the project on the campaign page, and now a few words about the history of its development from Yana Artisheva aka telegamochka :
Five years ago we decided to create a similar system. I was interested in this topic, because I was directly related to the creation of a wearable computer (NK). In addition, at that time head-mounted displays were inaccessible, expensive and cumbersome, and I was interested in the development of a sound interface to the NK. So when my friends from the RTD told me about the vOICe project, I joined the development with great interest.
We have created a vOICe-system based on the ASUS netbook and a sports “hirat” with a webcam embedded into it. The system turned out to be cumbersome, not the most convenient, with a small (less than an hour) operation time - but quite suitable for experiments.
Unfortunately, we did not find support in the blind forums. There were not even people who would be interested in trying out the system. However, a netbook with a standard OS, webbing straps and a wire from a webcam mounted on the head was of little use so that a blind person could use it without outside help. The prototype is a prototype - and no more than that ...
What is much less surprising - did not interest our project and state institutions.
And we began to independently master the system. But seeing it is much more difficult. First of all, the motivation is much less strong. Yes, we made the system. Yes, it is interesting to learn how to use it. Interesting - but no more. Secondly, in a sighted person, of all the ways of perception, sight is in the first place, the rest of the senses, including hearing, are usually in the background and the sighted person’s brain is usually less sophisticated in the processing of sound information compared to the perception of visual images. To catch the fake singer, to distinguish the notes of the played chord - yes, we are capable of that. But in order to see with the help of hearing, the sighted one needs to develop and develop the auditory info channel. While a person who has not used his eyesight for many years, hearing is an order of magnitude better suited to “fine work” with sound information. It is enough to remember about blind people who have managed to develop their ability to echolocate, like dolphins and bats.
And nevertheless, I managed to achieve some success in mastering vOICe. I did not learn to walk around the room blindfolded, but after several weeks of training I was able to first determine which figure was drawn on the voiced picture - a rectangle, an oval or a segment. Then I learned to “hear” several mutually intersecting figures, sketch their mutual position and determine the brightness of each object. Not 100%, but at 60-70 I managed to quite accurately perceive the picture. I tried to play in the audio version of the good old "Wolfenstein" - but, alas, absolutely unsuccessful.
Having existed just over a year, our project gradually faded away.
As you can see, the era of Raspberry Pi, Epson Moverio, Google Glass and crowdfunding * starters gives such projects a much better chance of success than 5 or 10 years ago. It so happened that it was possible to publish this project specifically on GT and in general in the habrasemeet only today, at the finish line, but this is only the first pilot experiment. In this connection, I propose to think: what other initiatives for helping people with disabilities can be realized precisely thanks to today's technologies, and what other technologies of a universal nature (platforms like the ones mentioned above) are still not enough for us to significantly increase in 5-10 years the quantity and quality of such projects?
Source: https://habr.com/ru/post/366011/
All Articles