Neuro-headset for every day - how it is done, why it is needed and what it will turn us into

Neural networks, blockchains, AI, flying cars, self-tying shoelaces - all this is, of course, cool, but the thought of the perfect neural interface makes me feel the greatest thrill. Reflections about the work of the brain and the device of consciousness usually give rise to the most severe dead ends, religious disputes and breeds trillions of conjectures. Therefore, I decided to look for a person who knows about working with the brain and the neural interface without a hearsay.

My interlocutor is Alexander Smirnov ( Bioalex ), the founder of CleverPoint , a Minsk startup.
')
In his youth, he studied at the doctor, received a diploma surgeon. After university, went to the department of biological chemistry and defended his thesis. At that time, Alexander began his business - the production of dental implants - and has been engaged in it for 20 years.

From school, his hobby was programming. The friends with whom he went to the computer club remained for the rest of his life - now they are all serious engineers. Together with them, for fun, Alexander decided to make a wearable neural interface, not lagging behind professional medical equipment. The joke has gone far, and now it is an ambitious technology startup.

Before talking, I tried to understand what exactly their product is used for. The goals turned out to be suspiciously many. What are the main ones - the mental control of devices? Monitoring the state of the brain? Research, data collection?

As a result, our conversation, like a joke with a startup, went too far. We climbed not only into the brain, but also into conflicting ethical problems, and into the future, where humanity seems to be waiting for a couple of difficult trials.



Alexander Smirnov

- I saw that you were recently in Skolkovo. How did everything go?

- In general, good. We presented the project, we were told what opportunities there are for startups in terms of grants, funding, a platform for communication in Skolkovo. I really liked the meeting, it was positive. We decided that we will apply, we will try to become a resident. Our project is quite extensive - we are making a technological platform, and the neural headset itself, and algorithms. We are currently speculating with which fragment we will go there.

- Does the idea resonate?

- Most people do not understand what we do. Although neural interfaces are already ten years old at the hearing, they still remain a new sphere. I have to explain, tell. The first reaction is surprise, then some skepticism, of course. But they react, in general, positively.

- Is it true that investors look first at the team?

- A team in a startup is the most important. We are fine with her. There are engineers, biophysicists, and radio electronics specialists, and physiologists with clinicians. As long as we don’t have the finished hardware, the doctors simply advise us, pass on the experience to our techies, so that they understand what to look for, what useful signals should be extracted from the total noise that our brain generates. When we make an alpha version of the product, then more serious and large-scale research will begin.

There are five engineers in the team, they are experts in their fields. Engaged in digital processing of radio signals, structural and functional components, programming logic integrated circuits, working in MatLab, write in C. There are specialists in machine learning, industrial design. The team is serious enough for their field.

- How did you manage to assemble such a team in Minsk?

- These are connections that have remained since school days. I was fond of programming then. And since then we have been friends for many years - already families. They tell me how they make their projects in the field of radio electronics, I tell them about medicine. Once I had an idea, and they caught fire.

In the beginning, everything seemed like a joke - well, we will try and try - but everything has gone very far. We were deep in the subject, filled with cones and developed a knowledge base.

- And how far did you go?

- Developed a prototype, picked up the iron, bought it, collected a test mock, tested its performance. We estimated the received signal, how much noise there is, how much useful signal. Tested known electroencephalographic paradigms in humans. It is known that under certain tasks the brain reacts in a certain way - we have confirmed all these concepts, that is, the iron of our device works.

In addition, we are developing the headset itself, relatively speaking, a helmet that fits over the head. Domestic headsets on the market, in my opinion, do not satisfy many requirements - ranging from convenience to technical specific requirements. The design of our neuro headset went through the first iteration, and now we are preparing to print a prototype.



And the third area that we are now actively engaged in is the testing of algorithms, the removal of artifacts that arise in the electroencephalogram. Audio will be built into the headset, so we are testing how the simultaneous transmission of brain impulses and audio signal goes, how much it interferes with each other, and in general, what is the real bandwidth of our wireless channel.

- Audio?

“We can encode brain activity, logically tie it to sound — clicks or some kind of signals — and translate them into headphones so that a person can hear what exactly is happening in his brain. For example, when a person is in a state of stress or mental overload, you can use a beep to inform him about it. The frequency and intensity of the signals may correlate with the degree of stress. Then the person makes an effort, consciously calms down - and the sounds disappear.

Another example is that while driving, the driver begins to fall asleep. Neuro headset warns you should stop and rest.

With the help of audio headphones, you can also give neurofeedback - this is brain training based on the feedback principle. If you send a sound or music of a certain rhythm and emotional coloring to the audio channel, the brain reacts to the audio signal and changes its rhythms. By repeating such training sessions, you can regularly learn how to manage your brain rhythms without audio support. For example, quickly moving from a state of relaxation to a state of concentration and vice versa.

- And what kind of iron do you use?

- A mobile headset is, in fact, a wearable mobile encephalograph. In classical encephalography, the signal is recorded, then it is cleared of artifacts and noise offline. The session is processed by various algorithms, and useful information is extracted from the signals.

Unlike classical encephalographs, our task is to process brain signals in real time. When this is done online, the requirements for the device itself grow. Therefore, it will stand high-precision analog-to-digital converter, which will remove the signal from the electrodes and convert them into numbers.

The device will have a programmable logic integrated circuit (FPGA). On the FPGA, we can paralyze the processing of multiple data streams, we can write preliminary noise cleaning algorithms, filtering, or even decision logic. Our device in two different basic versions will have 6 and 12 electrodes. Theoretically, we can scale and increase the number of channels to 16 or 32. But this is already beyond the scope of the everyday neuroheadership.



Along the way, we are developing our own board. Initially, it will be quite cumbersome - a large starter kit with a variety of different connection options and support for several electrode mounting schemes.

- So now this is a headset layout printed on a 3D printer, and wires go from it to the board?

- Yes, for the time being this is not a very wearable device - it will fit into the breast pocket and be connected with wires to the headset. Then we will release a more miniature version, we will build the board into the helmet and again we will test everything. How many ultimately pass the iterations, I do not know yet.

Usually wearable neuroheaders are either a cap or a plastic frame with arcs that supply electrodes to certain positions at the head. People have different head sizes, and the position of the electrodes on the head should be clearly exposed. If the neuroheadset is the same size for both children and adults, for women and men, then the electrodes are located on different heads in absolutely different ways. Fine-tune the work of the algorithms will not work - the algorithms expect the electrodes to stand correctly. To date, no manufacturer of non-medical neuroheternologies has solved this problem.

We are just trying to put the electrodes in any point of the head, make a system of various fasteners. At the same time, we constantly remind ourselves that this is not a professional device, but a domestic one, it should be convenient.



As a result, we will be able to decode the signal directly to the FPGA and immediately issue the user with data about his brain activity in the form of sound signals to the headphones - bypassing any other devices that go beyond the headset.

Most of the existing devices for online processing of electroencephalograms transmit a signal to either a computer or a smartphone, and there they are already deciphering it. With us, all the basic things will be implemented in a wearable device, and in a desktop or mobile application - only more complex logic. For example, interaction with a specialist in neurology.

- Will there be a server between the device and the application?

- No, the signal will be processed directly by the processor of the phone or any workstation. In the future, we plan to create a cloud service, it already outlines the first features. But they have not yet decided how and in what volume we will store the user's bio data, and how he will be able to interact with the specialist, conduct online treatment sessions.

- And how far did you go in software development?

- We have written drivers for the main software products for electroencephalography and concentrated on the removal algorithms for various artifacts that pollute the signal. For example, when a person blinks, the impulses that are transmitted to the oculomotor muscles are much stronger in amplitude than the brain waves, but are in the same range of useful frequencies. It is necessary that the signal becomes clear, and at the same time we have not lost some of the useful information on the filters. In classical multichannel electroencephalography, this problem is solved. And for a household neuro headset with real-time processing and a small number of electrodes, there are no good solutions yet.

What problems does neurointefrays have?

• Signal quality. The human head is absolutely not adapted to remove electrical signals from it. The skin has high resistance, and there is also hair - through them it is necessary to somehow fix the electrode on the skin.
  
  Most manufacturers offer wet electrodes. They use a special conductive gel, which ensures a tight contact of the electrode and the scalp, bypassing even the hairy part. Some make the electrodes active, they incorporate additional microelectronics.
  
  We are trying to develop a headset on dry polymer electrodes. They will be made in the form of combs or flat buttons, which we will also test on different heads. We will create a focus group and see how comfortable the headset is for practical use.
• Installation time When a medical device is worn on the head for an experiment, the installation takes 10 to 45 minutes to complete. Each electrode must be fixed, inject gel, check resistance at each, then tie all the wires together and connect to the measuring device.
  
  For the consumer, this is absolutely unacceptable. Ideally, a person should put a neural headset on her head, and she should start working right away. Maybe with minimal tightening and calibration to get a high-quality signal. In fact, no one has yet solved this problem. Until we decide, it makes no sense to move on.
• The lack of reliable algorithms to determine emotions, to determine the performance of the brain and other electroencephalographic paradigms.
• Unpreparedness of ordinary users. At present, a person without special training in principle cannot work with neural interfaces. Technology requires high motivation. Now people with disabilities are very highly motivated. They are ready to learn, to undergo endless calibrations of instruments. For the average user, this all seems strange and scary - he quickly loses interest.
  
  It is necessary to carry out a gigantic job in order to turn a scientific gadget into a device, without which it is impossible to manage in ordinary life, like, for example, a smartphone.

- Doesn't it seem dishonest to you that developers are highly motivated by people with disabilities and work everything out on them, so that they can use them on others?

- For people with disabilities, the neuro headset is sometimes the only window into the world. Therefore, they are ready to participate in experiments, sometimes very complex. Sometimes even in those where no one else agrees. They are ready to risk their health, to make invasive interfaces. These people are moving science forward.

To date, many technologies have frozen in anticipation of the brain interface. This is a new channel of interaction with the outside world and a new tool for man’s knowledge of himself. If a more or less cheap device appears that allows you to reliably record and interpret brain rhythms, then an instantaneous start and a serious breakthrough will happen.

- Can your device develop science?

- It will definitely help the popularization of science. The price of wearable medical devices is measured in thousands of dollars, on average, a good device costs 600-800 dollars for one information channel, that is, more than 7 thousand for 12 channels. If we make the device cheap, convenient and at the same time sufficiently accurate, we will contribute to the development of science.

- It seems that you are not inventing a neural interface, but are trying to make existing medical samples wearable.

- Partly it is. Our device does not have such expensive components as in professional medical systems. We have a simpler analog front-end, although it has been verified by scientific publications that it is comparable with medical ones in terms of the quality of the received signal.

- Do you feel yourself as a commercial project or as a scientific one?

- While we are just wondering. We have a feeling that we are doing research and development work with great commercial prospects. There must be a happy medium. We are all humans. In the team, everyone is over forty, we are not young guys who are willing to work for the idea. We all understand that we need to feed families, live. Therefore, initially hoped that the device should be commercially applicable.

We are aware of the complexity of the problem. Seven-eight years ago, neural interfaces started abruptly, they were predicted with great prospects. But the imperfection of products does not allow to fully apply them. And we still do not know in which area exactly our device will take its place.

- Does it cost you dearly?

- We spent about 18 thousand dollars only on iron and all sorts of experimental work. I do not count the time spent on development, on programming, on searching for scientific information, on a trip. That is - already not cheap.

- And you do all this on your own?

- Yes, while we do everything on their own.



- Recently, a lot of news about the use of machine learning algorithms in medical devices. Are you looking this way?

- We have to solve the problem of classification of the selected features using machine learning. We use different methods that have already been tested in the field of electroencephalography - discriminant analysis, the method of supporting vectors, the method of principal components, regression analysis.

As for neural networks, they are used but so far the reliability is not higher compared to other methods of machine learning.

- But there are prospects.

- The prospect is huge, but they still have a long experimental path to go. We are also working in this direction.

Around of the advanced and fashionable technologies about which all speak, the heap of myths soars. It seems that they can more than they actually are.

There is a good joke. “If everyone goes jumping off a cliff, will you go too?” Yes - meets the neural intelligence, built on machine learning. " It is impossible to predict when the algorithm is wrong. I think this is one of the main problems for which neural networks are not insured. They need in-depth testing, serious scientific study. Be careful with this.

- You probably have no mythical sensations and illusions from the technology with which you work?

- When in 2012 I saw the neuro-device for the first time, there was a feeling of witchcraft. The technology seemed magical. But the more I studied this question, the faster the sobering came, then disappointment, rethinking, and, finally, interest re-emerged.

We are far from being the first startup in this area; there are many laboratories that are likely to be many steps ahead of us. But we are interested, we want to try too.

“We didn’t think that when we also study our brain well, the mythical idea of ​​it will also be dispelled - it will just be an easily explained mechanism.

Well ... I don't think so.Even with the simplest neural networks, no one can predict which result will produce an algorithm after learning. This is always experimental work. But the simplest neural networks can be drawn on one piece of paper, the whole principle of their operation - and the result is unpredictable.

And imagine when you have the same thing in your head, only scaled a trillion times. I do not think it will be soon.

- And tell me more about the work of the brain?

- It's complicated. I will briefly discuss the features of the brain that generally make electroencephalography possible.

Functioning the brain generates various electrical potentials. Some of these potentials, mainly from the neurons of the cerebral cortex, can be registered with the help of electrodes from the surface of the scalp - the skin and the hairy part of the skull.

Synchronized activity between neuron ensembles creates electrical voltage microscopic oscillations - brain rhythms, which are visible on the electroencephalogram. Depending on the frequency range, rhythms are conventionally divided into intervals and denoted by letters of the Greek alphabet - alpha, beta, gamma, theta, delta.

By changing brain rhythms in different areas, we can judge what tasks the brain is currently solving. For example, rhythms change during the transition from wakefulness to falling asleep, with mental tension and relaxation, with the testing of various emotions, with solving mental problems of any type, as well as in response to sensory (auditory, visual and other) stimulation.

For example, in response to photostimulation with a frequency of 5–30 Hz, a rhythm with an identical frequency appears in the occipital lobes of a person. This is due to the fact that in the back of the head is the cortical end of the vision analyzer. Normally, he has a pronounced alpha rhythm. If a person closes his eyes, the amplitude of the waves increases. When the eyes open, the amplitude will fall again.

Another example of the desynchronization of rhythms is associated with the intention of a person to make a movement. That is, we imagine that we move with our right foot, and we have a response, which can be fixed on the electrodes, and then learn to identify, training various classifiers.

We fix which pattern of changes in brain rhythms corresponds to the mental notion of raising the right leg, and which - of the left leg. So we receive the elementary qualifier and we can program interaction with household external devices. The man thought about raising his right leg, and his wheelchair went forward.

Or an example from another area. When a person experiences some kind of emotion, he begins to get out of sync rhythms between the hemispheres. Several algorithms are now built to define human emotions — we can interpret from four to six kinds of emotions.

Based on what paradigm we want to digitize and which program logic to build, we need to select a different number and location of the electrodes. But it all depends on how motivated the person is to train the algorithms, to increase their accuracy and reliability of the prediction.



- At first, I couldn’t understand for a long time what exactly you want to use your device for. And it seemed to me that your emphasis is simply on collecting data from the brain.

- Today, while we are testing the initial device, we are accumulating a knowledge base, and yes - first of all we collect data and conduct standard experiments. But we are already preparing new algorithms that will be scientifically tested and published.

“Just collecting data now is a sensitive question.”

From the point of view of personal data, yes, of course. But such work must be carried out, because the currently existing databases of brain patterns are heterogeneous and incomplete. To date, there is no definitive concept of the norm. And of course we are interested in collecting and analyzing such data in order to improve our algorithms. Naturally, all with the consent of users.

I do not like it when my data is collected. Therefore, each user of our software product will have the opportunity to refuse to use his personal bio data for scientific and statistical purposes.

“A man walks with a headset on his brain, she constantly analyzes him — when he listens to music, runs, watches TV, does something else. Too many scenarios open up to influence a person.

- Of course, neurotechnology, as a potential tool of influence, carries a certain danger. It is necessary to think carefully about what security guarantees we must provide to a person in that his data are not used to the detriment or for the purpose of manipulation.

We are considering various options for group interaction on our neurohearnes. For example, a group emotiogram - this term has not been specifically mentioned in the Russian-speaking segment of the Internet. Its essence is that we can record the emotional response of a large group of people to an event, for example, to a movie in a movie theater. And a person who has not watched the movie will be able to see where the most terrible moment or the most fun. He will understand how the room reacted.

Or, for example, an emotional reaction to the speech of a politician. Here, of course, the user must be given clear guarantees that his data will not be hacked and not used against him. We are considering various options for protecting information from hacking and spoofing using hardware encryption and blockchain. But so far, this issue has not been worked out deeply.

- Suppose everything that you are talking about goes according to the ideal scenario. Our life will be easier, everything will be safe and good. Have you wondered what the constant improvement of the world will lead to?

- I, of course, thought about it. I must admit that you cannot escape from neural interfaces. Someday this technology will be implemented anyway. The channel that we are now using for communication is bytes per minute. This is very little against the background of the channels that operate computer networks and the Internet - tens of gigabytes per second. I give you an interview, and the information comes with very low bandwidth.

We need some additional channel of interaction with the brain, faster and more effective. Speaking specifically about implementation, you will most likely have to get into the anatomy and physiology of a human as a biological species, change it, genetically modify it, add something else directly to the surface of the brain, put some devices that amplify electromagnetic signals - and here I'm leaving in the realm of fantasy. But the fact that this, unfortunately, is inevitable is my firm confidence.

The same dental implants that I do - it's the same biomechanical device, in fact, we make a cyborg from a person.

A completely new world is coming, and it is not yet clear on what principles it will work. We are spurred by scientific and technological progress, we have become its hostages at this stage, and we cannot turn off - this is inevitable. In 30 years the world will not be the same as it is now, and we are putting our hand to it.

It sounds scary and unusual, but this is the trend. We can make this transition to a “new person” less painful and avoid social upheavals.

- I heard that if you manage to connect the electrode to the section that is responsible for getting pleasure, and give the person a button - he will simply press it and will not want anything more from life.

- In humans, such experiments were not conducted. They were carried out on rats. And the rats killed themselves. They died of exhaustion because they needed nothing more. They constantly stimulated the so-called pleasure center - one might say, experienced a chronic mental orgasm.

- If a person is given greater access to the brain, the same thing will not happen?

- Humanity is always on the brink of disaster. I think we will come up with some kind of defense mechanism in order to preserve our Self and our humanity.

A person already becomes an extra link in the society that we have created. The world is ruled by ideas that force us to do something and develop. Ideas spin a new round of scientific and technological progress, and we just serve it. More and more, he begins to work under laws that we do not understand. A person needs to be modified in some way so that he can understand what is happening, so that he can quickly receive and transmit information, so that he does not become a weak link in this system.