Neural Interfaces Today

Since the invention of the mouse, half a century has passed, and this is still one of the main ways of human-computer interaction. I went to the conference at the Institute of Cognitive Neurosciences of HSE to learn about the latest achievements in the field of BCI, which is beyond the horizon, which is why it is so interesting.

The conference report I reworked into an article for related narration. Some moments I simplify and omit, and some I add from my observations and reports from other events. After reading it, I hope you will have a general understanding of the approaches to BCI and the current state in this area. For original interpretations, it is better to refer to original articles, fortunately, almost everything is publicly available.

Story

The history of BCI began in 1973 with the publication of Toward direct brain-computer communication [1], where Jacques Vidal outlined ideas in communication between man and machine and described the laboratory for analyzing the EEG signal for such purposes. A decade later, Wolpaw focused the use of BCI on helping paralyzed people and described the concept of BCI [2]:

The main BCI implementations made it possible for people to enter text with isolation syndrome . These were difficult to use systems, because the user had to go through a long training session [3], in contrast, there appeared “spellers” based on the recognition of the P300 - a component that occurs at the moment when a person makes a choice, which reduced the requirements for the user [4] .

In the 90s, the topic is becoming more and more known, especially with the advent of machine learning techniques [5]. With the increase in BCI reliability, people are interested in expanding applications to new areas.

Thorsten Zander proposed the following BCI classification [6]:

• Active BCI - the user initiates the command unconditionally
• Reactive BCI - the user initiates a command in response to the impact of the system
• Passive BCI - the user does not give a command, but the system reads and analyzes its state

Separately, it is worthwhile to consider the issue of brain stimulation, this topic, although not directly related to BCI, is a related technology that expands the possibilities of BCI to control.

BCI can also be classified by the method of receiving the signal:

• Invasive (implanted electrodes, ECoG and others)
• Non-invasive ( EEG , NIRS and others)

EEG is the most common way to get a signal, so, unless otherwise indicated, I mean it by default.

Active BCI

Basketparadigm

This is a symbol of the possibility of control by activating imaginary movements. The fact is that the motor cortex is compactly located in the center of the head, therefore, imaginary movements of different parts of the body are well classified and used to construct the BCI. The user, to work with such BCI, must be mentally imagined as he makes movements with different parts of the body.

To facilitate experiments, scientists develop their own frameworks, for example, BCILAB . With his help, an experiment was conducted to demonstrate to the skeptic the possibility of controlling with imaginary movements. The result was 80% - so-so result in conditions where a person has familiar alternatives, but deserves a high mark, especially for an unprepared respondent [7].
')
The same approach was used to control the horizon for the flight simulator. The results are ambiguous, for 3 respondents it was possible to achieve a result of 94%, for another 4 64% and less than 60% for three more. The success lies in the fact that the first trinity controlled the aircraft in the same way as the steering wheel. The rest of the pilots did not focus enough on the internal state and made muscular movements, which made a negative contribution to the management.

Rehabilitation systems

BCI, which recognize motor commands, are well studied and are already being used to rehabilitate patients after a stroke: to restore the broken bonds needed to control paralyzed limbs. Pavel Bobrov demonstrated the results of clinical trials of a rehabilitation complex to restore the motor functions of the hands, which proved the effectiveness of use. Moreover, there is a significant difference for patients who began rehabilitation after a month and 6 months after a stroke, the earlier rehabilitation begins, the better the effect. [eleven]

The head of g.tec Gunter Edlinger spoke about the work of special gyms for rehabilitation, an interesting point is that electrical stimulation of the extremities was added to the rehabilitation process, and if the electro-mechanical device was used above, then there is current stimulation, which reduces the cost of the complex.

If we add elements of the game and competition to the process, the involvement will be higher, which means the patient will go through rehabilitation better. At the Center for Bioelectric Interfaces, the HSE, under the leadership of Alexey Osadchiy, developed prototypes to improve the rehabilitation process. The video demonstrates a prototype system for two people, where they control a vessel, performing imaginary motor commands, trying to incline the vessel in their direction:

Single player game:

Or, for example, the handwriting recognition algorithm for muscle activity using a compact array of electrodes allows you to reconstruct what is written: [12]


The pinnacle in their work is the work on BCI in the ExoAtlet project, which allows people with disabilities to move independently or use it for rehabilitation.

Invasive BCI is a more complex subject, and now experiments are being conducted on animals or on people who have electrodes installed for medical reasons. A series of studies was highlighted, which showed that it is possible to determine not only single components (meaning all the same fictional movements), but also to divide movement, attention, and the direction of gaze between them. A recording of a similar report from the conference in Samara is available.

Reactive BCI

The classic example of reactive BCI is the “speller” on the P300 effect, this is the “wave” that appears in response to the choice of the shown stimulus, and in the “speller”, the stimulus is a certain way coded characters of the alphabet or command. The user must mentally interact with the stimuli that the system displays — count the number of flashes of the selected symbol.

Not to mention the project Neurochat , which allows people with disabilities to communicate:

Passive BCI

The basic idea of ​​passive BCI is an assessment of a person’s condition, for example, an assessment of cognitive load (workload), it can be applied in training systems, a study was conducted to solve this problem.

The qualifier was trained on the following tasks:

• For high loads, the respondent subtracted 1-2 digits from 3-4 figures, excluding simple options with dozens.
• For a light load, they asked the respondent to focus on a pleasant memory.

The accuracy of the algorithm was 70%. The classifier was tested on other tasks (multiplication, playing the scramble), and obtained similar accuracy, thereby confirming the fact that it is possible to make the classifier independent of the person and the tasks. [13]
This idea can be used to control the surgeon during surgery [14]. The task of determining the load on the surgeon during the execution of various on the complexity of the manipulations on the simulator was solved. The system has learned to determine how the surgeon performs the operation with high accuracy.

Another option is to measure the degree of relaxation. On the basis of the state of the visitor of the interactive installation in the " Museum of Silence " a vivid picture was created that reflected its internal state. [15]

Passive BCIs can also be used for control tasks; a rather original approach is to give a person not direct control over the cursor, but only the right to judge whether the cursor is moving along the correct path to the goal. The experiment was conducted on small matrices of 4x4 and 6x6 pixels. At first, the system was trained on an arbitrary point motion, and the person’s task was to determine whether the point was moving in the right direction, then they were tested in live mode and the result was found to be close to the optimal path. [16] You can see a demonstration .

Midas touch problem and E-BCI interfaces

Controlling the cursor with the help of the gaze direction is a simple task, which is solved with the help of eye-tracker'ing (also known as video eye). But in these interfaces there are problems, for example, involuntary eye movements and the problem of choice, by the way, it is very symbolic that it is called the problem of the touch of Midas, the Phrygian king, any touch of which turned an object into gold. The applications of passive BCI allow you to solve these problems.

The approach where active BCI was used to make choices when controlling with the eye-tracker, has been known for a long time, but does not differ in speed. The study, where respondents evaluated different methods of selection on the NASA TLX scale, showed that the BCI variant is not faster in time than the long fixation option for selecting an object, but the BCI causes a lower degree of frustration [10].

Further work of the team Torsten Zander showed that it is possible to distinguish conscious fixation on the object from the unconscious with an accuracy of 90% [17]. For the experiment, the paradigm "Oddball" was used - the respondent looked through a series of figures containing a figure that he wants to choose in combination with distracting figures.

Sergey Shishkin spoke about the improvement of the above approach [8]. A significant plus of their solutions is a reduction in the speed of selection up to 300ms-500ms, which requires very fast classification, for this they used EEGNet [9].

Attention mechanisms are a separate topic that can expand the scope of BCI and create systems for the rehabilitation of patients with ADHD, Mehdi Ordikhani talks about the basic idea in his Tedtalk

Stimulation

The question of the ethics of conducting experiments is very acute for neuroscience, and animals take the brunt of research for research beyond the limits. What if we want to affect a particular area in the depths of the brain? Now it is possible only with the help of implanted electrodes. But, for example, in nature there are creatures that are sensitive to a magnetic field, the Galit Pelled team from the University of Michigan have isolated this gene in fish, introduced it to rats, and learned to control their behavior by the action of a magnetic field [18]. Thus, it is possible to have a targeted effect on the necessary areas, for example, to stop epileptic seizures.

And a whole group of studies of invasive interfaces from Mikhail Lebedev on rhesus monkeys: a brain-computer-brain interface was built, which allowed us to receive tactile feedback by controlling virtual limbs. You can take a closer look at the excerpt from the lecture " The interface between the brain and the computer ."

Kingdom deep learning

Besides the fact that the “deep learning” algorithms allow us to finish the already high accuracy of “machine learning”, it can be noted that people are working on the “inverse problem”. Based on fast EEG and MEG data, you can try to restore the actual activation of neurons in the brain, which is now shown, for example, by the fMRI method, but with a very low temporal resolution. One can only rejoice at optimism and believe in the speedy success of this work.

Another BCI problem based on EEG or MEG is that the activity results in different areas of the brain for the same components differ among users, we have to teach the neural network for each user and the task, which complicates the work with the system and makes it more expensive. Nevertheless, changes with the “transfer of training” are possible here, when the neural network uses data from different users / in different tasks and is trained online, as a result, the calibration step can be skipped. [nineteen]

Hardware

Finally, we got to the glands!

Here it is important to say about 2 points, on the one hand, the equipment for BCI is rather cumbersome, the person in it attracts attention, in one of the speeches miniature electrodes were demonstrated, such that the person does not stand out in them. [20]

Unfortunately, there is no possibility to insert a larger photo, but you can view a photo via Google .

Despite all the miniature, to install these electrodes is not convenient, it is necessary to glue each individual electrode. For acceleration use various devices:

• EEG caps, which are marked with holes for the electrodes
• Hoops and helmets of various designs, where basically the position of the electrodes is fixed, only OpenBCI is allocated Ultracortex, in which the electrodes can be rearranged depending on the tasks.

A relatively fresh idea is the arrays from CeeGrid electrodes for attachment in the ear, which are both invisible and easy to install, but a significant disadvantage is the limited application, although there are works that show that it is realistic to use this option for BCI ERP [21] .

And the second problem is the need for a conductive gel for a high-quality signal, it is shown here that the differences are permissible, and the use of dry electrodes is justified [22], but it all depends on the amount of hair. Over this issue, for example, recently Florida Research Instruments began to sell an elongated dry electrode (in the picture below it is on the left), which differs from the original version with a larger roundness of pins and, as you understand, causes less negative sensations among users. Even more advanced options are when the pins themselves on the electrodes are supplied with shock absorption, thanks to the material or with the help of springs (in the picture below they are in the center and on the right).

Conclusion

The spread of BCI to the masses will not be quick and easy, very limited possibilities are now open for understanding brain states, but progress in this area cannot be ignored. The main thing is that there is a correct tendency to reduce the cost of devices / provision of devices by subscription and the emergence of projects that are aimed at enthusiasts.

Personally, I am very pleased that among the promoted Emotive, MUSE, OpenBCI Russian projects are beginning to appear. At the recent Neuroforum, which was held in St. Petersburg, were demonstrated:

• Neurointerface Optometrograph
• Neurolaboratory "Young neurophysiologist-engineer"
• Project FreeEEG32
• Rehabilitation complex iBrain based on EEG SmartBCI

The expansion of available devices makes the interface area attractive for study and experimentation. The threshold of entry is low, you can always find an adequate task, and you can improve algorithms all the way, acquiring new knowledge and skills. What I wish for you.

This is how I saw the BCI area, let's see what will be interesting next year.

UPD corrected description of EBCI interfaces, by error they were assigned to active BCI, which is not true