Created a bionic hand with a neural network that instantly recognizes and grabs items

Past generation bionic prostheses are usually controlled with the help of myoelectric signals, which arise as a result of muscular contractions of the human hand. To manage such a prosthesis is not easy: it requires a certain concentration, and the effectiveness of successful actions leaves much to be desired. Not easy from the first time to do what you want. In terms of the accuracy of such actions, such prostheses are far from the “intuitive” actions of a living real hand.

In recent years, researchers have focused mainly on the accuracy of recognition of myoelectric signals, and the accuracy of recognition of movements of individual fingers has reached 90%. But due to a number of technical reasons, the massive use of such "smart" prostheses is very limited. New development of engineers from the University of Newcastle (UK) offers a fundamentally different approach. A hand equipped with a video camera recognizes an object in front of it - and determines for itself how to grab it most effectively. It acts automatically and almost instantly, without additional efforts on the part of the person. In fact, the bionic arm has its own vision.

Previously, scientists experimented with stereo cameras and various object recognition algorithms. At the same time, new models of manipulators for robots were created - there, computer vision technologies are very similar to bionic prostheses for humans. It was in the field of robotics that the most promising technologies of machine vision and in-depth training were tested.
')
Engineers from the University of Newcastle used these developments of their predecessors and aimed the machine vision system to recognize the type of capture for objects of different types, rather than on the basis of a specific measurement of its dimensions. That is, the objects after learning of the neural network are classified precisely by the type of capture, and not by the type or category of the object. The authors believe that due to such a fundamentally new approach, they managed to significantly improve the speed of the system, since it ignores unnecessary details.


For comparison: classification by object category (above) or one of the four types of capture (below)

For training the system was used convolutional neural network. It turned out that its architecture is perfect for this type of task, namely for bionic prosthetic hands. For example, other methods of computer vision experienced problems when faced with objects that do not fall into any of the known categories. But the identification of unknown objects is one of the most important qualities of a bionic prosthesis with machine vision. Therefore, the convolutional neural network is ideal for this task.

The training system took place on the Amsterdam Image Bank , where there are a large number of home objects.

The architecture of a two-layer convolutional neural network for feature extraction and classification is shown in the illustration below.



In tests on real patients with prostheses, the system was tested on 8 known and 16 unknown objects in a random position. The results for the two volunteers are shown in the graphs on the left and right. Taking into account the allowable errors, the accuracy of recognition and capture of objects was 88% and 87% for the first and second volunteers, respectively.



Most importantly, such a bionic prosthesis works almost in real time: the choice of the type of capture is carried out in milliseconds, as opposed to 0.75-24 seconds for bionic hands, where machine vision performs the classification of objects. Even the best bionic prostheses of this type CyberHand and SmartHand cope with recognition in 4 and 1 second, respectively. They run powerful computers and show capture accuracy of 93% and 94%, respectively. Although there is a little higher accuracy, but the capture in real time or with a pause of one second is a big difference, so the achievement of British biomedical engineers should not be underestimated. This is the first bionic hand, which is able to grab objects "intuitively", as if not thinking. A person only gives a signal with a small movement of the muscle that the object needs to be captured - and a smart hand with a neural network quickly does the rest on its own.


The scientific article was published on May 3, 2017 in the Journal of Neural Engineering (doi: 10.1088 / 1741-2552 / aa6802).