In the psychophysiology laboratory of Moscow State University: EEG as a tool for reverse engineering of the brain and brain-computer interface
One of the most accurate and high-speed encephalographs in the world is in the research complex of the psychophysiology of the faculty of psychology of Moscow State University. Up to 258 channels in the passive mode can be used simultaneously for scientific research, allowing simultaneous recording and analysis of the electroencephalogram (EEG) in real time.
(weconnect Katya to the EEG matrix of 32 channels with active electrodes)
How “pakmeny eat a non-existent square”, how much of the brain generates illusions, what color the numbers are, how psychophysiologists process multidimensional signals and whether this can lead to reverse engineering of the brain.
')
Under the cat, a superficial description of the device, a little about signal processing and about the studies that are conducted by young scientists, psycho-neurophysiologists, and what challenges are there for programmers who want to study the brain and / or work in a project to study the brain.
Sit comfortably in a chair, sit back and take a deep breath
The EEG-helmet with the ActiChamp system from Brainproducts is equipped with active biosensors (up to 128 channels), which catch the smallest changes in electrical signals from the human brain with extremely high sampling rates - up to 100 KHz!
Active, as opposed to passive, electrodes are equipped with microchips embedded in them - preamplifiers, which digitize the analog signal and transmit it with the least distortion.
Electrodes are installed over all areas of the brain that are important for analysis and synchronously transmit a signal to blocks of amplifiers, either using a loop or via wi-fi — 32 channels per block with a 1000 Hz bandwidth and a sampling frequency of up to 5000 Hz as opposed to 500 Hz of some medical EEG, not to mention neurogneatures).
Wi-fi module
After the signals arrive at the amplifier units (noise level ≤ 2 μVpp), their path does not end. Then they are transmitted via a dual fiber optic cable to the USB adapter, which also receives external signals (triggers) for synchronization with external devices, such as EyeTracker (about it next time). The USB adapter serves as a PC connector.
The main task of all mediating elements in minimizing distortion and delays in a signal whose power is measured in dozens of microvolts
Additional blocks for registration of a so-called. peripheral indicators that reflect vegetative processes, such as: heart rate, vascular tone, respiration, temperature, movement, muscle activity, etc., up to a total of 128 channels in addition.
Sensors
Skin resistance, thermal sensor, accelerometer, optical heart rate monitor
That's who they make measurements on when there are no students.
The software for processing and analyzing EEG has broad functionality with the ability to integrate MRI data to localize electrical signals, taking into account the individual constitution of a particular person’s brain. The system itself has a module that is called a “low resolution tomography” (LORETA) (* this is not a technical implementation - magnetic or positron emission, but the principle of visualization and brain mapping).
One of the main problems for accuracy is artifacts. This term refers to the distortion of a signal by external or internal factors, whether it is eye movement or aiming at 100 Hz from fluorescent lamps.
Despite the seeming perfection of the EEG recording technology, there are still flaws. Perhaps the main one is the need to use an electrically conductive paste or gel to reduce the resistance between the scalp and the electrodes. The technology as a whole is also not self-sufficient, and the recent trend converges to a combination of different methods. Already used electrodes that work in the magnetic field of the scanner (about 10 Tesla).
The dotted line is an external stimulus. Phase EEG-signal (left), attempts to localize the source in three-dimensional space (middle) and in the projection on the scalp (right)
Fragment of the recorded EEG
Here such krivulkami our brain sees the world. But seriously, it is the MSS (event-related potentials) that reflect the net (specially recorded and filtered) response of groups of neurons to a particular stimulus. Here they are all in a crowd for all leads. The potential and so far (and perhaps in general) unsolvable task is the opposite: to restore, what did the person see / hear / imagine / touch at this moment?
Below - PSS, but not by all, but by the way they were registered in different leads.
The map of the electrode localization of activity, per second - from image to image
3d view
Fourier transform of a signal registered under different electrodes.
Continuous EEG wavelet transform. (The first rule of psychophysiologists is not to tell anyone about the wavelet. A joke.) We need the wavelet in order to look at the frequency component of the EEG (different rhythms consist of different frequencies and reflect different states of a person) and look at it all in a time perspective.
Vyacheslav Lebedev , employee and postgraduate student, Department of Psychophysiology, Moscow State University Mv Lomonosov, co-founder of the project NeuroFuture , psychophysiologist:
“The essence of my scientific work is the study of the role of attention in the processes of cross-modal integration.
Decoding: Information coming from the external environment, as a rule, consists of signals of different modalities (auditory, visual, etc.). Processing this multisensory information requires synchronous joint work of the brain structures involved in processing signals of a certain modality. Modern studies at the level of individual neurons show the existence of neurons in various regions and levels of the brain, whose function is the processing of multisensory information. But the question remains open of the systemic unification of these multilevel structures in the course of processing the flow of multisensory information, as well as on the role of voluntary (initiated by the person himself) and involuntary (caused by external events) attention in this process. EEG is one of the most informative methods for studying this process over time. ”
(Find the dog)
Ilya Zakharov , graduate student, Department of Psychophysiology, Moscow State University. Mv Lomonosov, co-founder of the project NeuroFuture , psychophysiologist:
“Psychophysiology tries to connect the ideal (psychology and“ soul ”, if it exists) and material (that which can be investigated by natural scientific methods). It turns out with varying success, but it's interesting
There are such things - illusions. What happens in the brain when we see them? Is there a universal pattern of brain activity, the same for all people, characteristic only for those moments when we see visual illusion? And what if we look at a loved one and we suddenly have this pattern?
By recording various patterns on the EEG, one can objectively register a person's perception of a new type of illusion for him (for example, a new version of the illusion of a subjective contour). The method of event-related potentials is used for this: extracting from the common EEG signal the component that appears when the illusion is presented.
Black graph - just a stimulus, red - an illusion, dotted - the moment of presentation of the stimulus
On the right, the brain’s response to the presentation of a classic type of illusion compared to a stimulus, when the brain does not need to complete construction
On the left - the answer to the presentation of a new type, again, compared with when it is not necessary to finish building
We see that in the same place (component N170) the amplitude of the signal in response to the illusion is greater. This answer is an objective measure of our perception of illusion. If we see both straight and wavy illusory contours at the same psychophysiological level, it is likely that in both cases we use the same mechanisms of image formation. ”
Kostya Slavnov , programmer
- What is the internal motive of the maths encouragement to go to work on a neurosines project?
- Interest. I was always interested to know how the human brain works. By that time, I already knew quite a lot about the possibilities of the data analysis method applied to the brain. All of them are based on processing multi-dimensional signals. And I had just such a topic of course work.
As always, accident played a big part. I read an article about Ilya and decided to write to him about myself: that I am a programmer in mathematics, this topic is interesting. And the guys just lacked a man with my skills. And I quickly joined the work.
Maria Stepanenko , a student at the Department of Psychophysiology, Moscow State University. Mv Lomonosov, psychophysiologist, researcher synesthesia
“Synesthesia is, roughly speaking, the connection of various modalities. It manifests itself in the form of experiencing sensations in one modality while presenting stimuli in another.
Can you quickly find the “colored” triangle?
In general, synesthesia is an incredibly interesting, but also very little-studied phenomenon. There are an incredible variety of different kinds of it: some synesthetes "see" the colors of certain letters / days of the week / months, some of them "feel" smells when saying certain words, someone "feel" the voice / smell texture, "feel" taste names, "hears" the movement and so on. Moreover, most synesthetes show a connection not of two, but of multiple modalities, and, thus, various types of synesthesia manifest.
There are many hypotheses about how synesthetic sensations arise. In my research, the question is asked at what level this process takes place: is it only perceptual sensations that are processed at lower levels, or is it a high-level, cognitive process.
The emphasis is on audiovisual synesthesia, namely, on the connection of tones and timbres with different colors. The subjects are both synesthetes and people who do not possess synesthetic sensations. In the course of the experiment (namely, at the time of presentation of sound stimuli), an EEG recording is made, and subsequently - data analysis by comparing the MSS registered at synesthetes and nesynesthets.
Speaking of the subjects: if you are interested in this topic, or suddenly you have familiar synesthetes, or, suddenly, you yourself discovered that you have an appropriate type of audiovisual synesthesia for this study - feel free to contact me, I will be glad to see you among your subjects! ”
Neuroscience TED Talk
The synesthesia test “color of a digit” (on the left is how an ordinary person sees a picture; on the right is a vision of synesthe: a triangle of twos immediately stands out with a different color, as a result of which a synesthete can easily and quickly indicate all the two in the picture)
Andrei Uchaev , a student at the Department of Psychophysiology, Moscow State University. Mv Lomonosov, psychophysiologist:
“My position on obtaining experimental data was well reflected in the words of the hero of one game:“ ... we just throw science at the wall and see what will stick. "This suggests that in the research I try to get maximum data that can be squeezed from the electrophysiological signal.
Now I’m rather fixated on the following: on working out various methods of signal processing, regardless of the task. Help programmers here is huge. There are various problems: it happens that not all processing methods are presented in one program, or there is no access to all the functionality - you have to dodge, and this is not always possible.
And there is another important problem - saving time. Quite often it is necessary to process the data of ten people using 10-15 different transformations. There would not prevent writing scripts that will automate the task. I pressed the button and went home - and in the morning everything was processed. ”
"Neuroscience needs programmers." So say, and Russian colleagues, and foreign neuroscientists even turn to hackers. Because very soon,God forbid, we may need not only tools to ensure the privacy of messages from technical interception, but also thoughts .
As Azimov wrote that “there are two riddles left for humanity - the depths of space and the secret of the brain”, I am glad that Habr very much welcomes the space program, I hope that the same will happen with the study of the brain.
(we
How “pakmeny eat a non-existent square”, how much of the brain generates illusions, what color the numbers are, how psychophysiologists process multidimensional signals and whether this can lead to reverse engineering of the brain.
')
Under the cat, a superficial description of the device, a little about signal processing and about the studies that are conducted by young scientists, psycho-neurophysiologists, and what challenges are there for programmers who want to study the brain and / or work in a project to study the brain.
Sit comfortably in a chair, sit back and take a deep breath
The EEG-helmet with the ActiChamp system from Brainproducts is equipped with active biosensors (up to 128 channels), which catch the smallest changes in electrical signals from the human brain with extremely high sampling rates - up to 100 KHz!
256 channels
256 channels
256 channels
Active, as opposed to passive, electrodes are equipped with microchips embedded in them - preamplifiers, which digitize the analog signal and transmit it with the least distortion.
Electrodes are installed over all areas of the brain that are important for analysis and synchronously transmit a signal to blocks of amplifiers, either using a loop or via wi-fi — 32 channels per block with a 1000 Hz bandwidth and a sampling frequency of up to 5000 Hz as opposed to 500 Hz of some medical EEG, not to mention neurogneatures).
Wi-fi module
After the signals arrive at the amplifier units (noise level ≤ 2 μVpp), their path does not end. Then they are transmitted via a dual fiber optic cable to the USB adapter, which also receives external signals (triggers) for synchronization with external devices, such as EyeTracker (about it next time). The USB adapter serves as a PC connector.
The main task of all mediating elements in minimizing distortion and delays in a signal whose power is measured in dozens of microvolts
Additional blocks for registration of a so-called. peripheral indicators that reflect vegetative processes, such as: heart rate, vascular tone, respiration, temperature, movement, muscle activity, etc., up to a total of 128 channels in addition.
Sensors
Skin resistance, thermal sensor, accelerometer, optical heart rate monitor
That's who they make measurements on when there are no students.
Soft and signal processing
The software for processing and analyzing EEG has broad functionality with the ability to integrate MRI data to localize electrical signals, taking into account the individual constitution of a particular person’s brain. The system itself has a module that is called a “low resolution tomography” (LORETA) (* this is not a technical implementation - magnetic or positron emission, but the principle of visualization and brain mapping).
One of the main problems for accuracy is artifacts. This term refers to the distortion of a signal by external or internal factors, whether it is eye movement or aiming at 100 Hz from fluorescent lamps.
Despite the seeming perfection of the EEG recording technology, there are still flaws. Perhaps the main one is the need to use an electrically conductive paste or gel to reduce the resistance between the scalp and the electrodes. The technology as a whole is also not self-sufficient, and the recent trend converges to a combination of different methods. Already used electrodes that work in the magnetic field of the scanner (about 10 Tesla).
The dotted line is an external stimulus. Phase EEG-signal (left), attempts to localize the source in three-dimensional space (middle) and in the projection on the scalp (right)
Fragment of the recorded EEG
Here such krivulkami our brain sees the world. But seriously, it is the MSS (event-related potentials) that reflect the net (specially recorded and filtered) response of groups of neurons to a particular stimulus. Here they are all in a crowd for all leads. The potential and so far (and perhaps in general) unsolvable task is the opposite: to restore, what did the person see / hear / imagine / touch at this moment?
Below - PSS, but not by all, but by the way they were registered in different leads.
The map of the electrode localization of activity, per second - from image to image
3d view
Fourier transform of a signal registered under different electrodes.
Continuous EEG wavelet transform. (The first rule of psychophysiologists is not to tell anyone about the wavelet. A joke.) We need the wavelet in order to look at the frequency component of the EEG (different rhythms consist of different frequencies and reflect different states of a person) and look at it all in a time perspective.
People
Attention
Vyacheslav Lebedev , employee and postgraduate student, Department of Psychophysiology, Moscow State University Mv Lomonosov, co-founder of the project NeuroFuture , psychophysiologist:
“The essence of my scientific work is the study of the role of attention in the processes of cross-modal integration.
Decoding: Information coming from the external environment, as a rule, consists of signals of different modalities (auditory, visual, etc.). Processing this multisensory information requires synchronous joint work of the brain structures involved in processing signals of a certain modality. Modern studies at the level of individual neurons show the existence of neurons in various regions and levels of the brain, whose function is the processing of multisensory information. But the question remains open of the systemic unification of these multilevel structures in the course of processing the flow of multisensory information, as well as on the role of voluntary (initiated by the person himself) and involuntary (caused by external events) attention in this process. EEG is one of the most informative methods for studying this process over time. ”
(Find the dog)
Illusions
Ilya Zakharov , graduate student, Department of Psychophysiology, Moscow State University. Mv Lomonosov, co-founder of the project NeuroFuture , psychophysiologist:
“Psychophysiology tries to connect the ideal (psychology and“ soul ”, if it exists) and material (that which can be investigated by natural scientific methods). It turns out with varying success, but it's interesting
There are such things - illusions. What happens in the brain when we see them? Is there a universal pattern of brain activity, the same for all people, characteristic only for those moments when we see visual illusion? And what if we look at a loved one and we suddenly have this pattern?
By recording various patterns on the EEG, one can objectively register a person's perception of a new type of illusion for him (for example, a new version of the illusion of a subjective contour). The method of event-related potentials is used for this: extracting from the common EEG signal the component that appears when the illusion is presented.
Black graph - just a stimulus, red - an illusion, dotted - the moment of presentation of the stimulus
On the right, the brain’s response to the presentation of a classic type of illusion compared to a stimulus, when the brain does not need to complete construction
On the left - the answer to the presentation of a new type, again, compared with when it is not necessary to finish building
We see that in the same place (component N170) the amplitude of the signal in response to the illusion is greater. This answer is an objective measure of our perception of illusion. If we see both straight and wavy illusory contours at the same psychophysiological level, it is likely that in both cases we use the same mechanisms of image formation. ”
Programming and processing multidimensional signals
Kostya Slavnov , programmer
- What is the internal motive of the maths encouragement to go to work on a neurosines project?
- Interest. I was always interested to know how the human brain works. By that time, I already knew quite a lot about the possibilities of the data analysis method applied to the brain. All of them are based on processing multi-dimensional signals. And I had just such a topic of course work.
As always, accident played a big part. I read an article about Ilya and decided to write to him about myself: that I am a programmer in mathematics, this topic is interesting. And the guys just lacked a man with my skills. And I quickly joined the work.
Synesthesia
Maria Stepanenko , a student at the Department of Psychophysiology, Moscow State University. Mv Lomonosov, psychophysiologist, researcher synesthesia
“Synesthesia is, roughly speaking, the connection of various modalities. It manifests itself in the form of experiencing sensations in one modality while presenting stimuli in another.
Can you quickly find the “colored” triangle?
In general, synesthesia is an incredibly interesting, but also very little-studied phenomenon. There are an incredible variety of different kinds of it: some synesthetes "see" the colors of certain letters / days of the week / months, some of them "feel" smells when saying certain words, someone "feel" the voice / smell texture, "feel" taste names, "hears" the movement and so on. Moreover, most synesthetes show a connection not of two, but of multiple modalities, and, thus, various types of synesthesia manifest.
There are many hypotheses about how synesthetic sensations arise. In my research, the question is asked at what level this process takes place: is it only perceptual sensations that are processed at lower levels, or is it a high-level, cognitive process.
The emphasis is on audiovisual synesthesia, namely, on the connection of tones and timbres with different colors. The subjects are both synesthetes and people who do not possess synesthetic sensations. In the course of the experiment (namely, at the time of presentation of sound stimuli), an EEG recording is made, and subsequently - data analysis by comparing the MSS registered at synesthetes and nesynesthets.
Speaking of the subjects: if you are interested in this topic, or suddenly you have familiar synesthetes, or, suddenly, you yourself discovered that you have an appropriate type of audiovisual synesthesia for this study - feel free to contact me, I will be glad to see you among your subjects! ”
Neuroscience TED Talk
The synesthesia test “color of a digit” (on the left is how an ordinary person sees a picture; on the right is a vision of synesthe: a triangle of twos immediately stands out with a different color, as a result of which a synesthete can easily and quickly indicate all the two in the picture)
Data processing
Andrei Uchaev , a student at the Department of Psychophysiology, Moscow State University. Mv Lomonosov, psychophysiologist:
“My position on obtaining experimental data was well reflected in the words of the hero of one game:“ ... we just throw science at the wall and see what will stick. "This suggests that in the research I try to get maximum data that can be squeezed from the electrophysiological signal.
Now I’m rather fixated on the following: on working out various methods of signal processing, regardless of the task. Help programmers here is huge. There are various problems: it happens that not all processing methods are presented in one program, or there is no access to all the functionality - you have to dodge, and this is not always possible.
And there is another important problem - saving time. Quite often it is necessary to process the data of ten people using 10-15 different transformations. There would not prevent writing scripts that will automate the task. I pressed the button and went home - and in the morning everything was processed. ”
Conclusion
"Neuroscience needs programmers." So say, and Russian colleagues, and foreign neuroscientists even turn to hackers. Because very soon,
As Azimov wrote that “there are two riddles left for humanity - the depths of space and the secret of the brain”, I am glad that Habr very much welcomes the space program, I hope that the same will happen with the study of the brain.
Source: https://habr.com/ru/post/235399/
All Articles