A ray of light in a dark kingdom: how technology returns lost sight
The main value of a post-industrial society is technology. Ever updated and improved, running up to progress and cultivated on the basis of tough competition, they change everything.
Life, work, entertainment and improvement of the world are no longer possible without new technologies. People think that with their help they can do what they want. And they do.
')
For a modern person, technology is an integral part of life - its working tool, means of communication, movement, education, and comfort. They are the “crutches” of our mind and physically limited body. But there are people for whom technology is even more important and serves as crutches not only in a figurative sense.
One of the global goals of progress is to remove all existing restrictions. Features of health or illness should not become a hindrance for a person, affect the possibilities of his self-realization and quality of life.
Following this rule, many IT companies that are at the forefront of scientific and technological progress create technologies and solutions that allow people with disabilities to overcome their limitations. We will talk about this interesting facet of progress today.
One of the most difficult and at the same time promising areas in the field of innovation is the development of rehabilitation tools for people who have visual impairment or do not see at all. The originality and adaptability of projects in this area are amazing.
What is a prosthesis - everyone knows. People with disabilities have been using artificial body parts for thousands of years. Modern prostheses have bionic compatibility, and sometimes functionality that tends to perform the work of a lost organ.
Today, almost the same success was achieved by scientists who create visual prostheses. And we are not talking about cosmetic replacement of the eye with a glass one, but about implants that allow completely blind people to see the world around again.
In 2012, the bionic eye of Argus II was created. The implantable part is an electronic retinal implant, and the external system consists of a camera mounted in glasses in combination with a small processor. Such a prosthesis actually gives vision to people living in total darkness. With some reservations, of course. The patient’s blindness must be caused by certain diseases, and the artificial vision of the prosthesis will be very limited.
Antonina Zakharchenko - the first woman in Russia who installed Argus II. In December 2018, she visited the “Hope for Technology” forum dedicated to the rehabilitation industry and support for people with disabilities. Antonina arrived with her husband, but she moved independently, and when she was talking, she turned her head as if she could see her interlocutors perfectly.
For many years she lived in absolute darkness, not seeing a ray of light. But in 2017 she had an operation to introduce a retinal prosthesis. Then Antonina was installed 60 photosensitive electrodes, tuned cameras and an external device processor. And as soon as the woman put on her glasses, she received her sight.
Of course, it is impossible to call it a full-fledged vision: Antonina sees light and dark silhouettes, obstacles, and outlines of objects. But the main thing she sees light. After years of darkness. “The grandchildren no longer want to play hide and seek with me,” Antonina complains jokingly.
Anyone can see the world as Antonina Zakharchenko sees it right now. Laboratory "Sensor-Tech", actively promoting the implantation of the prosthesis, has developed a VR-simulator, showing the surrounding picture of the world as it is shown by Argus II. The application is laid out in open access, it can be downloaded for free at one of the links: App Store , Google Play .
In addition to bionic vision, the VR-simulator of the Laboratory "Sensor-Tech" shows the majority of known diseases and visual impairments. The development is designed to help ophthalmologists and other specialists understand patients, and ordinary people - their visually impaired friends and relatives.
The number of people who use bionic vision today is over a hundred worldwide. Of these, only two are citizens of Russia. The installation of a visual prosthesis is a very complicated, high-tech business and as a result, very expensive. Both Russian patients were fully paid for the operation by a well-known in Russia charitable foundation associated with big business.
But over time, progress will inevitably make solutions for retinal bionic implantation easier, and technologies more accessible and effective. Probably, in two or three decades, each person will be able to use artificial vision, there will be no blind people on Earth and blindness will disappear into history.
But so far, neither technology nor the economy is able to provide all people who have lost their sight with bionic implants. And progress seeks detours. There are a lot of them and each deserves a separate attention.
One of the most interesting is touch substitution technology. This path is based on the mechanism of neuroplasticity of the human brain. In fact, a person is invited to perceive the surrounding reality with “ears”.
However, natural sounds are not enough to “see” the world. Therefore, the Russian project vOICe vision has developed the concept of sound vision.
Project engineers created a device that captures the surrounding space with the help of a camera and converts the image into sounds using a certain technology. The sound is perceived as an analogue of a black and white image. And the bone conduction headphones that the device is equipped with do not block the ear canal and do not interfere with the perception of the environment.
As conceived by the developers, vOICe vision replaces one way of perception with another, and without limitation for the second. A person hears and “sees” at the same time, using one perception organ for this purpose. Hearing.
To transform a picture into a signal, sounds of various durations and heights are used. A brighter part of the image is accompanied by a strong signal, and from the side from which the object is located. If the object in the field of view is moving, then the signal is moving. Two separate objects are decoded by two signals, three by three, and so on. Increasing the tonality of "neighboring" sounds means an obstacle, while the higher the tone - the higher the obstacle in front of a person.
Due to the peculiarities of neuroplasticity, the brain over time is rebuilt under the conditions offered by the vOICe vision technology. The areas of the cortex, which are usually responsible for hearing, learn to convert sound signals into visual images. As a result, after a few weeks of training, a completely blind person begins to “see” at the level of the visually impaired, and even to distinguish between images, and eventually, household items, signs with large print and the depth of space.
With proper diligence and constant use of the device, the person will again see subjectively. That is, his brain will literally begin to turn the audio signals into the corresponding image, embossed and volumetric, similar to what our eyes transmit to us.
And the man will begin to see clearly, remaining blind.
Learn more about vOICe vision or contact the developers on the official project website .
The idea of ​​“looking at the world with ears” is also popular abroad, albeit in a somewhat simplified format. Imagine a technology that gives a “voice” to every object or object surrounding a person - a desk, a door, a road sign, a wall, and a parked car.
A device that looks like vOICe vision glasses also scans the space, but instead of “black and white” sounds, gives the user the exact name of the objects and the distance to them. This is possible thanks to artificial intelligence technologies that quickly and accurately process the image from the camera.
It was on artificial intelligence that the developers of the Department of Neurobiology from the Institute of Translational Research im. Tianqiao and Chrissie Chen (China) created CARA or Cognitive Augmented Reality Assistant [intelligent assistant in augmented reality].
The system combines technologies of augmented reality and digital vision. The program uses so-called spatial sounds that make objects sound differently, depending on their location in the room. For example, if an object is located to the left of the user, then his voice will sound as if it comes from the left, and the closer the object is, the higher its “voice” will sound.
To avoid potential cacophony when all objects speak at the same time, scientists programmed CARA into several different modes. In the first mode, surrounding objects speak only if the user is looking directly at them. As the device owner turns his head, the next object in sight calls itself. Thus, a blind person can “look around” to explore the surrounding space.
In the second mode, all objects in turn call themselves from left to right, regardless of the user's view. In the third mode - focal - the user selects a specific object and “communicates” only with it, using it as a guide when navigating.
Dubai version of "smart glasses" for the blind Amal 1 has a slightly different functionality. Unlike the Chinese device, Amal 1 reads newspapers and books printed in standard font, and also understands the image on a computer monitor. Such a device actually allows a blind person to work with text in print or electronic form.
In addition, the developer has equipped his product with a GPS-system that helps the blind to determine their location and build routes. For emergency situations, Amal 1 has the SOS function, which, when activated, instantly transmits a message to relatives or special services, attaching a map and photos to it from the user's location.
By the way, something similar is done in Russia. Laboratory "Sensor-Tech" has created an intelligent assistant for the blind, which combines the basic functions of foreign analogues: it scans the space around, guesses objects, and also recognizes signs and road signs. In the domestic market, it is better known as the “Smart Assistant“ Robin ” .
“Robin” is similar to a portable video camera and, in addition to what its foreign analogs do, can recognize faces. And the signal about the surrounding space is transmitted not only by sound, but also by vibration, like a sonar or rangefinder, reporting the distance to obstacles or objects.
Another interesting solution proposed by the Israeli company ICI Vision. Although the device is powerless to help totally blind people, it can make life easier for the visually impaired.
Israel’s Orama glasses combine artificial intelligence, light tracking software and computer vision technology. The device includes a 3D camera and a miniature laser scanner that projects a three-dimensional image onto healthy parts of the retina, allowing a person to see even those areas of the visual field that have disappeared from his illness.
The procedure for mapping healthy and damaged cells of the retina allows you to customize the glasses to the individual characteristics of the user. This is the uniqueness of the Israeli development.
It is important to note here that with certain types of blindness, a part of the visual cells may remain, and a person may not see the space around, but see black and white spots. It is these people who will again make ICI Vision points visually sighted.
Now Orama are intended for indoor use. Glasses allow the carrier to read books, watch TV and navigate through space. Thanks to this, today the device can help those who have difficulties in the workplace.
Over the past year, VR technologies have become more affordable. There are about 340 million blind people all over the world, and IT giants like Microsoft cannot ignore this. Several technological solutions of this corporation are fully focused on those who have lost their vision.
So, last year, Microsoft engineers presented a virtual reality system, adapted for totally blind people. Interaction with game objects occurs in it with the help of a virtual white cane and a joystick synchronized with it, equipped with a vibration motor and sensors that track its position relative to the virtual reality helmet.
Such a reed allows you to feel the fact of collision with objects in the VR environment, as well as to distinguish objects and surfaces of different types through vibration and sound.
During the virtual simulation, the user walks around the game room and moves around with a short cane-joystick, as well as real life. But in virtual reality, the cane is much longer. When it collides with an obstacle, the brake stops its movement, creating the feeling of a real collision.
At the same time, the motor on the cane vibrates, and the speakers in the helmet make a sound corresponding to the situation in the virtual world. For different surfaces, the system simulates a different tactile response.
The latest Microsoft developments are designed to include visually impaired people in an already existing virtual reality. A month ago, corporation engineers created a set of software functions for VR helmets that simplify the perception of the virtual world.
This set includes 14 services that adapt graphics for people with visual impairments . It includes increasing the contrast, brightness, increasing the image, highlighting the contours of objects, or repainting them in different colors, as well as sound annotation of objects and actions.
Nine functions are implemented as a simple plug-in, available with any applications for HTC Vive helmets. The remaining five need to be embedded in the application using the package for the popular Unity game engine.
Microsoft developers hope that this functionality will help people close to the loss of vision, to get full pleasure from immersion in virtual reality.
In the modern world, technology is no longer just a working tool. Now it is also a way for people to explore the surrounding reality. In any case, for those who are deprived of one of the five ways of perceiving the world, bestowed by nature.
Science has not yet learned to completely replace the sense organs on the same qualitative level. But she is relentlessly moving towards this. Every year assistive technologies offer more and more alternatives and ways to rehabilitate, significantly improving the quality of life of those whose health is limited due to illness.
Sooner or later there will come a time when technological solutions for people with disabilities will exceed the capabilities of an ordinary person. This hypothetical moment of the future will change everything. What today is referred to as “assistive technology” and is being developed for a relatively small category of people, suddenly everyone will need it. After all, everyone wants to have the best.
When this happens, the person in his classic image will not. This biological type will be replaced by cyborg, with more perfect organs of perception and physical, and possibly intellectual abilities. They will get a new world, the foundation of which is now laid by the rehabilitation industry.
And in this world there will be no restrictions caused by disability.
Life, work, entertainment and improvement of the world are no longer possible without new technologies. People think that with their help they can do what they want. And they do.
')
For a modern person, technology is an integral part of life - its working tool, means of communication, movement, education, and comfort. They are the “crutches” of our mind and physically limited body. But there are people for whom technology is even more important and serves as crutches not only in a figurative sense.
One of the global goals of progress is to remove all existing restrictions. Features of health or illness should not become a hindrance for a person, affect the possibilities of his self-realization and quality of life.
Following this rule, many IT companies that are at the forefront of scientific and technological progress create technologies and solutions that allow people with disabilities to overcome their limitations. We will talk about this interesting facet of progress today.
One of the most difficult and at the same time promising areas in the field of innovation is the development of rehabilitation tools for people who have visual impairment or do not see at all. The originality and adaptability of projects in this area are amazing.
Artificial vision
What is a prosthesis - everyone knows. People with disabilities have been using artificial body parts for thousands of years. Modern prostheses have bionic compatibility, and sometimes functionality that tends to perform the work of a lost organ.
Today, almost the same success was achieved by scientists who create visual prostheses. And we are not talking about cosmetic replacement of the eye with a glass one, but about implants that allow completely blind people to see the world around again.
In 2012, the bionic eye of Argus II was created. The implantable part is an electronic retinal implant, and the external system consists of a camera mounted in glasses in combination with a small processor. Such a prosthesis actually gives vision to people living in total darkness. With some reservations, of course. The patient’s blindness must be caused by certain diseases, and the artificial vision of the prosthesis will be very limited.
Antonina Zakharchenko - the first woman in Russia who installed Argus II. In December 2018, she visited the “Hope for Technology” forum dedicated to the rehabilitation industry and support for people with disabilities. Antonina arrived with her husband, but she moved independently, and when she was talking, she turned her head as if she could see her interlocutors perfectly.
For many years she lived in absolute darkness, not seeing a ray of light. But in 2017 she had an operation to introduce a retinal prosthesis. Then Antonina was installed 60 photosensitive electrodes, tuned cameras and an external device processor. And as soon as the woman put on her glasses, she received her sight.
Of course, it is impossible to call it a full-fledged vision: Antonina sees light and dark silhouettes, obstacles, and outlines of objects. But the main thing she sees light. After years of darkness. “The grandchildren no longer want to play hide and seek with me,” Antonina complains jokingly.
Anyone can see the world as Antonina Zakharchenko sees it right now. Laboratory "Sensor-Tech", actively promoting the implantation of the prosthesis, has developed a VR-simulator, showing the surrounding picture of the world as it is shown by Argus II. The application is laid out in open access, it can be downloaded for free at one of the links: App Store , Google Play .
In addition to bionic vision, the VR-simulator of the Laboratory "Sensor-Tech" shows the majority of known diseases and visual impairments. The development is designed to help ophthalmologists and other specialists understand patients, and ordinary people - their visually impaired friends and relatives.
The number of people who use bionic vision today is over a hundred worldwide. Of these, only two are citizens of Russia. The installation of a visual prosthesis is a very complicated, high-tech business and as a result, very expensive. Both Russian patients were fully paid for the operation by a well-known in Russia charitable foundation associated with big business.
But over time, progress will inevitably make solutions for retinal bionic implantation easier, and technologies more accessible and effective. Probably, in two or three decades, each person will be able to use artificial vision, there will be no blind people on Earth and blindness will disappear into history.
You hear? No, I see that
But so far, neither technology nor the economy is able to provide all people who have lost their sight with bionic implants. And progress seeks detours. There are a lot of them and each deserves a separate attention.
One of the most interesting is touch substitution technology. This path is based on the mechanism of neuroplasticity of the human brain. In fact, a person is invited to perceive the surrounding reality with “ears”.
However, natural sounds are not enough to “see” the world. Therefore, the Russian project vOICe vision has developed the concept of sound vision.
Project engineers created a device that captures the surrounding space with the help of a camera and converts the image into sounds using a certain technology. The sound is perceived as an analogue of a black and white image. And the bone conduction headphones that the device is equipped with do not block the ear canal and do not interfere with the perception of the environment.
As conceived by the developers, vOICe vision replaces one way of perception with another, and without limitation for the second. A person hears and “sees” at the same time, using one perception organ for this purpose. Hearing.
To transform a picture into a signal, sounds of various durations and heights are used. A brighter part of the image is accompanied by a strong signal, and from the side from which the object is located. If the object in the field of view is moving, then the signal is moving. Two separate objects are decoded by two signals, three by three, and so on. Increasing the tonality of "neighboring" sounds means an obstacle, while the higher the tone - the higher the obstacle in front of a person.
Due to the peculiarities of neuroplasticity, the brain over time is rebuilt under the conditions offered by the vOICe vision technology. The areas of the cortex, which are usually responsible for hearing, learn to convert sound signals into visual images. As a result, after a few weeks of training, a completely blind person begins to “see” at the level of the visually impaired, and even to distinguish between images, and eventually, household items, signs with large print and the depth of space.
With proper diligence and constant use of the device, the person will again see subjectively. That is, his brain will literally begin to turn the audio signals into the corresponding image, embossed and volumetric, similar to what our eyes transmit to us.
And the man will begin to see clearly, remaining blind.
Learn more about vOICe vision or contact the developers on the official project website .
Artificial Intelligence sees for you
The idea of ​​“looking at the world with ears” is also popular abroad, albeit in a somewhat simplified format. Imagine a technology that gives a “voice” to every object or object surrounding a person - a desk, a door, a road sign, a wall, and a parked car.
A device that looks like vOICe vision glasses also scans the space, but instead of “black and white” sounds, gives the user the exact name of the objects and the distance to them. This is possible thanks to artificial intelligence technologies that quickly and accurately process the image from the camera.
It was on artificial intelligence that the developers of the Department of Neurobiology from the Institute of Translational Research im. Tianqiao and Chrissie Chen (China) created CARA or Cognitive Augmented Reality Assistant [intelligent assistant in augmented reality].
The system combines technologies of augmented reality and digital vision. The program uses so-called spatial sounds that make objects sound differently, depending on their location in the room. For example, if an object is located to the left of the user, then his voice will sound as if it comes from the left, and the closer the object is, the higher its “voice” will sound.
To avoid potential cacophony when all objects speak at the same time, scientists programmed CARA into several different modes. In the first mode, surrounding objects speak only if the user is looking directly at them. As the device owner turns his head, the next object in sight calls itself. Thus, a blind person can “look around” to explore the surrounding space.
In the second mode, all objects in turn call themselves from left to right, regardless of the user's view. In the third mode - focal - the user selects a specific object and “communicates” only with it, using it as a guide when navigating.
Dubai version of "smart glasses" for the blind Amal 1 has a slightly different functionality. Unlike the Chinese device, Amal 1 reads newspapers and books printed in standard font, and also understands the image on a computer monitor. Such a device actually allows a blind person to work with text in print or electronic form.
In addition, the developer has equipped his product with a GPS-system that helps the blind to determine their location and build routes. For emergency situations, Amal 1 has the SOS function, which, when activated, instantly transmits a message to relatives or special services, attaching a map and photos to it from the user's location.
By the way, something similar is done in Russia. Laboratory "Sensor-Tech" has created an intelligent assistant for the blind, which combines the basic functions of foreign analogues: it scans the space around, guesses objects, and also recognizes signs and road signs. In the domestic market, it is better known as the “Smart Assistant“ Robin ” .
“Robin” is similar to a portable video camera and, in addition to what its foreign analogs do, can recognize faces. And the signal about the surrounding space is transmitted not only by sound, but also by vibration, like a sonar or rangefinder, reporting the distance to obstacles or objects.
Another interesting solution proposed by the Israeli company ICI Vision. Although the device is powerless to help totally blind people, it can make life easier for the visually impaired.
Israel’s Orama glasses combine artificial intelligence, light tracking software and computer vision technology. The device includes a 3D camera and a miniature laser scanner that projects a three-dimensional image onto healthy parts of the retina, allowing a person to see even those areas of the visual field that have disappeared from his illness.
The procedure for mapping healthy and damaged cells of the retina allows you to customize the glasses to the individual characteristics of the user. This is the uniqueness of the Israeli development.
It is important to note here that with certain types of blindness, a part of the visual cells may remain, and a person may not see the space around, but see black and white spots. It is these people who will again make ICI Vision points visually sighted.
Now Orama are intended for indoor use. Glasses allow the carrier to read books, watch TV and navigate through space. Thanks to this, today the device can help those who have difficulties in the workplace.
Unlimited Virtual Reality
Over the past year, VR technologies have become more affordable. There are about 340 million blind people all over the world, and IT giants like Microsoft cannot ignore this. Several technological solutions of this corporation are fully focused on those who have lost their vision.
So, last year, Microsoft engineers presented a virtual reality system, adapted for totally blind people. Interaction with game objects occurs in it with the help of a virtual white cane and a joystick synchronized with it, equipped with a vibration motor and sensors that track its position relative to the virtual reality helmet.
Such a reed allows you to feel the fact of collision with objects in the VR environment, as well as to distinguish objects and surfaces of different types through vibration and sound.
During the virtual simulation, the user walks around the game room and moves around with a short cane-joystick, as well as real life. But in virtual reality, the cane is much longer. When it collides with an obstacle, the brake stops its movement, creating the feeling of a real collision.
At the same time, the motor on the cane vibrates, and the speakers in the helmet make a sound corresponding to the situation in the virtual world. For different surfaces, the system simulates a different tactile response.
The latest Microsoft developments are designed to include visually impaired people in an already existing virtual reality. A month ago, corporation engineers created a set of software functions for VR helmets that simplify the perception of the virtual world.
This set includes 14 services that adapt graphics for people with visual impairments . It includes increasing the contrast, brightness, increasing the image, highlighting the contours of objects, or repainting them in different colors, as well as sound annotation of objects and actions.
Nine functions are implemented as a simple plug-in, available with any applications for HTC Vive helmets. The remaining five need to be embedded in the application using the package for the popular Unity game engine.
Microsoft developers hope that this functionality will help people close to the loss of vision, to get full pleasure from immersion in virtual reality.
Biological singularity
In the modern world, technology is no longer just a working tool. Now it is also a way for people to explore the surrounding reality. In any case, for those who are deprived of one of the five ways of perceiving the world, bestowed by nature.
Science has not yet learned to completely replace the sense organs on the same qualitative level. But she is relentlessly moving towards this. Every year assistive technologies offer more and more alternatives and ways to rehabilitate, significantly improving the quality of life of those whose health is limited due to illness.
Sooner or later there will come a time when technological solutions for people with disabilities will exceed the capabilities of an ordinary person. This hypothetical moment of the future will change everything. What today is referred to as “assistive technology” and is being developed for a relatively small category of people, suddenly everyone will need it. After all, everyone wants to have the best.
When this happens, the person in his classic image will not. This biological type will be replaced by cyborg, with more perfect organs of perception and physical, and possibly intellectual abilities. They will get a new world, the foundation of which is now laid by the rehabilitation industry.
And in this world there will be no restrictions caused by disability.
Source: https://habr.com/ru/post/452028/
All Articles