The use of MEMS gyroscopes and accelerometers to track human body movements

Tracking human body movements is a task that has been solved with varying success for more than one thousand years. Once I read a story about an ancient Greek orator Demosthenes, who had a bad habit of raising his shoulder to the ear if he was nervous. To get rid of this, during daily training, he hung his sword over his shoulder, which he very unpleasantly stabbed if his shoulder was raised. As a result, the speaker became so famous that there is even an article about him in Wikipedia.

Many have seen another good example in Chinese martial arts films. For example, Kung Fu Panda strongly elbowed while performing techniques. In order to track this moment, he put burdocks in the armpits zone. Burdock falls - the student gets a scolding from the master. Surely some of us parents promised to screw the stick to the back, if we slouched. True, these threats were never fulfilled and therefore did not act very convincingly.

Very often, motion tracking is required during sports training. For example, you can find patent US3820783 , which describes a training device that simultaneously guides an athlete and prevents him from moving incorrectly.

Installation diagram of patent US 3820783
')

Motion registration using video

In the mid-70s, systems appeared that processed motion video recording made from several points. As a result, a mathematical model of a movement appeared. If you watch a video on how the Lord of the Rings movie was filmed, you can see interesting episodes of filming Gollum's movements. A man in a special suit actually moved, and then with the help of a clever mathematical apparatus and software he turned out to be a pretty bald character.

Video fixation of movements has obvious advantages, but since I want to describe an alternative solution in this article, I will dare to criticize a bit and present the shortcomings:

• video shooting must be carried out from several angles;
• often placed markers on the body;
• need expensive hardware (video cameras);
• we need good computational power and the corresponding software product to convert the video into a model of human movement;
• a person cannot move freely over long distances, otherwise he will inevitably go beyond the limits of the video shooting zone;
• in addition to the measurement object, a team of specialists is needed, that is, an ordinary user will hardly be able to record body movement during the morning run.

Movement measurement is a very useful thing, not only in sports. It is also used in industrial design - in the development of automobiles, conveyors, sewing machines and much more. Such systems already exist, for example, Siemens - Jack ( Human Simulation and Ergonomics ). How to find out whether it will be convenient for the driver to press the air conditioner on button in the designed vehicle? You can, of course, make a car, put a person in and check it out. But it is much easier to put a virtual person in a virtual car. A virtual car is already there, since all modern drawing systems provide for the development of 3D models. It remains only to tie the movements of the human model to the movements of his real prototype. This can be done with the help of all the same motion video recordings or with the help of the method described below.

Smart clothes

In this article I want to tell you about how it would be great if you could measure movement without restricting yourself to the scope of the set. For example, if the measurement functions were embedded in clothing. You walk, run, jump, and the clothes record everything and then reproduce your movements on the smartphone screen, give recommendations and tell you how to run and not hurt your knees, sit and not slouch, how to pedal on a bicycle without injuries.

It turns out that science and technology already provide such opportunities. Of course, we are not talking about smart everyday clothes, but there are already special suits made up of wearable sensors that record body movements at a very good level. Such costumes are made by XSENS . They cost dearly, but as dozens of computer units appear in each family, chips become cheaper and more and more intelligent portable systems become. We are moving by leaps and bounds to a brighter future. Without going deep into the technical details, I will try to tell you how the movement is recorded, I will describe the work and the principle of operation of the main components of the movement measurement system based on electronic-mechanical sensors.

MEMS

As microelectronics develops, various miniature sensors appear. A separate group of such sensors is called MEMS - micro electromechanical sensors. Acceleration sensors - accelerometers and angular velocity sensors - gyroscopes are used to measure movements. Accelerometer is a miniature sensitive element that changes its properties under the action of acceleration. This may be a piezoelectric sensor or a variable capacitance element - a capacitor with a movable plate. The piezoelectric sensor generates a small voltage on its electrodes, which can be measured and recalculated into acceleration. The situation is similar with the capacity of a variable capacitor.

MEMS gyroscope most often uses in the design the action of the Coriolis force, which rejects the vibrating plate, the magnitude of the deviation is recorded and converted into angular velocity.

As we know from the course of physics and mathematics, any vector can be decomposed into the vector components. For example, acceleration and speed are decomposed into mutually perpendicular components: X, Y, Z. The sensitive elements of MEMS measure acceleration and speed separately along each of these vectors.

It is important to note that now there are microcircuits, which contain several MEMS sensors at once.

For example, the MEMS accelerometer chip measures the acceleration simultaneously in three axes x, y, z. The same applies to gyro chips, which can measure the angular velocity at once along all three axes. There are even chips that simultaneously measure both acceleration and speed. Such sensors are called six-coordinate.

MEMS - controller

MEMS sensors, as a rule, are equipped with a built-in controller that calculates acceleration or angular velocity, provides digital filtering and configuration of the chip.

The data inside the controller is stored in special memory cells called registers. They are presented in integer format with a sign. The unit of measurement is usually g [acceleration due to gravity - 9.8 m / s2] for accelerometers and rad / s [radian per second] for gyroscopes. A description of the data format, addresses of registers, units of measurement, ranges of measurement and other parameters are always given in the documentation for the corresponding chip.

The controller also provides the connection of the MEMS sensor with the outside world through one of the common interfaces. This is usually SPI or I2C. SPI is an interface with two data lines and one clocking line. I2C is an interface with one data line and one clocking line. In principle, we only need to know that transferring data from the MEMS sensor is easy and pleasant, for this there are standardized common interfaces and ready-made libraries.

A computer, tablet or smartphone does not have SPI or I2C interfaces available to the user, therefore, in order to connect the sensor to them, some other matching device is also needed. This may be, for example, a microcontroller connected to a Bluetooth radio transmitter. The buffer microcontroller, as a rule, has responsibilities for preliminary data processing in order to reduce the load on the communication channel.

Generally speaking, the choice of communication channel is a separate big task. Of course, this channel should preferably be wireless, but which wireless technology to choose? 2.4GHz communication standards, such as Bluetooth or WiFi, are good because they are supported by most user devices. But on the other hand, they limit the communication range due to the short wavelength. Of course, there are Bluetooth radio modules with a stated range of about a kilometer, but let's not flatter ourselves, because nobody canceled the laws of physics, and such a range can be obtained only under conditions of direct visibility and sufficient height of sensors above the ground. It is important to determine the model of measurement and data processing. It’s one thing when all the calculations are done on a smartphone that is in a person’s pocket, and it’s quite another when the computer is at a distance of tens of meters on the table at the trainer / operator. This is more a question of marketing and the choice of the target audience of the complex motion measurement. I will only note that in any case the task of transmitting data to the destination can be solved and there are specialized hardware and software solutions for this.

Human body model

Let the data nevertheless have reached the destination and their processing begins. To carry out calculations and visualize movements, we simply need a mathematical model of the human body. Such a model should certainly take into account the different lengths of arms, legs, waist, chest grip, that is, various anthropometric features of people. Perhaps such a model should also take into account the internal structure of the body. The more complex the model, the harder, more expensive and longer its creation. I personally believe that the model should contain only those elements that you can wear a sensitive element. That is, if we are talking about a hand, then it is advisable to make its model from the following parts:

• shoulder;
• forearm;
• brush;
• fingers;

Building a model with all the bones, muscles and tendons is impractical. A cone can act as the simplest model of one part of the body. This is a simple geometric figure that can be easily implemented in any graphical environment and does not require a lot of resources, which is especially important for mobile platforms. Actually, the volumetric shape of the cone is used to visualize the model, and the vector, which coincides with the longitudinal axis of symmetry, is used for various calculations. Different lengths of the lower and upper circumference of the cone easily simulate differences in diameters, for example, the hips above and around the knee.


Cone, as an element of the human body model


A complete model of the human body composed of cones

Agree that the model looks quite recognizable. This model was built in the Microsoft XNA environment; Primitives3D libraries found on the Internet were used to draw a cone. For the placement of model elements, the mathematical apparatus of matrix computing of the XNA environment is used.

A little bit about calculations

The space in which the model is located is called the world space. To move model elements in global space, it is necessary to compose displacement matrices in order to rotate the rotation matrices necessary. Generally speaking, these are the same matrices, only for different purposes they use different cells.


Matrix Structure in XNA Environment

The elements highlighted in red are responsible for the rotation, the elements highlighted in blue for the movement, and the black elements are needed to comply with the 4x4 dimension. To create a displacement matrix, use the method:

Matrix.CreateTranslation(vector3) 

Which as a parameter takes the radius-vector of the desired position of the point. To accommodate all the elements of the body, you need to create a displacement matrix for each of them, let's call such a matrix fBaseWorldi.

Everything related to the movement of 3D models is well known to computer game developers and other professionals working in the field of computer 3D graphics.

We move on to the most interesting, namely, how to connect the measurements of angular velocity and acceleration with the position of the model on the screen. The position of the body in space can be specified using Euler angles, displacement and rotation matrices, or using quaternions. Many copies are broken in disputes about which of the ways to choose. I use the quaternion position representation. From one method to another you can easily navigate using well-known mathematical transformations.

A quaternion is a set of four numbers defining an axis in space, around which the body and the angle of rotation should be rotated. Quaternion is written in the form:

q = [W, X, Y, Z],
where W is the cosine of the half angle; X, Y, Z - coordinates of the axis of rotation.

Quaternion is preferable to rotation matrices, since the matrix contains 16 numbers, and the quaternion is only 4, which obviously saves data transmission time and does not load the communication channel so much.

An attentive reader can rightly argue that quaternion only allows to describe the rotational motion, but what about the translational? The fact is that the proposed method is intended to register movements without being tied to the surrounding terrain. And all human movements can be built only by rotations.

Indeed, our body is practically not subject to pure (without rotation) stretching and compression. For example, to linearly move the brush forward or backward, up or down, you will have to perform rotational movements of the forearm at the elbow or shoulder at the shoulder.

It remains the case for small, convert the acceleration and angular velocity into quaternions. The mathematical apparatus that produces such transformations constitutes the holy of holy firms producing systems for measuring movements. This device is generally known, on the Internet you can even find the source codes ( Open source IMU and AHRS algorithms ), but as usual all the difficulties lie in the details. Therefore, do not be surprised if motion measurements contain position errors. This is due to errors in the readings of sensors that are integrated and significantly affect the result. Also, errors in the location of the sensors on the human body contribute to their share of inaccuracies. A good solution is provided by the manufacturer of the MEMS-chips Invensense, they produce microchips into which they can load the computation library supplied by them. In earlier versions, the library was a code written in C, which was run on a third-party microcontroller. Now the library is an array of hexadecimal numbers that need to be loaded into the MEMS chip after power is applied to it. A similar solution is provided by other firms, for example Microchip .

After we calculated the position of a part of the body and expressed it in the form of a quaternion, it is necessary to draw up a general model from the testimony relating to individual parts of the body. This is where the XNA environment and Primitives3D code come in handy. The Primitives3D library used to move or rotate uses the data in a matrix representation. Therefore, to convert the quaternions received from the sensors, you must use the built-in XNA function:

 Matrix.CreateFromQuaternion(qi) 

where qi is the quaternion from the sensor.

Then it is necessary to transfer the rotated part of the body from the origin to the corresponding point where it should be located, the forearm, for example, is “attached” to the elbow. Although you have already seen in the picture a completely “assembled” body, so that it is always in the correct position after the arrival of each new quaternion, the body position must be re-calculated. This is due to the fact that the quaternion calculation library produces a quaternion, which relates the body position at zero time with the current moment. In order to carry out the transfer of a body part, it is only necessary to multiply two matrices:

 Matrix.CreateFromQuaternion(qi)* fBaseWorldi 

The fBaseWorldi matrix is ​​constantly being adjusted, because if the shoulder is moved, then the elbow will move accordingly. Therefore, in the program after the arrival of a quaternion for any part of the body, it is necessary to calculate the fBaseWorldi matrices for all other related parts of the body. <



The video recorded movement of a person, obtained using seven sensors, three sensors on each arm and one on the body in the lumbar region.

Now that we have obtained a record of body movements, it is possible on the basis of these data to make calculations of various interesting quantities. For example, to help Demosthenes and attach a sensor to his shoulder, which will monitor the correct position and issue any signal when deviating from this position, just as if you run the application on your smartphone, and place several sensors on the child’s back, this will help him to control your posture. And how many more different useful applications for MEMS sensors.

Conclusion

In this article I tried to review the main points related to the use of MEMS sensors as applied to the measurement of human body movements. Of course, many of the issues mentioned here require a more detailed explanation, in addition, I did not specifically mention some of the nuances in order not to overload the article with details.