Gesture Recognition with APDS-9960

Reading the comments to my previous article about the APDS-9960, where it was about color recognition and the level of illumination, two things became obvious to me: 1) the topic of gesture recognition is interesting and 2) this topic is not disclosed.

Indeed, if we took up the description of the APDS-9960, without a description of the gestures, the description looks somewhat incomplete. Therefore, I found free time to explore this topic too.

In this article, I bring to your attention a review of the possibilities for gesture recognition provided by the APDS-9960 sensor.

The article will discuss the mechanism for configuring the sensor, data collection, processing and presentation. You can see for yourself how easy it is to work with gestures with the help of the APDS-9960.
')
Like last time, the article will be accompanied by a code, everything that happens will be described in detail. The full version of the code is available at the end of the article.

Immediately a small remark: the built-in automatic mechanism for determining gestures in the APDS-9960 is not provided; that is, there is such a thing that is right here, I read it, it means that the register, and already there the gesture is processed - there is no such thing in the APDS-9960; This means that you will have to write your own gesture interpretation algorithm, which we will do later on.

In general, it is both good and not so. Not very much - because it can complicate the study of this sensor for a beginner, but it is good, because, together with the data on the approximation, it is possible, with a little extravagance, to make up your own gestures of various different kinds and types.

But, since this article only has an overview function, we will limit ourselves to only basic UP-DOWN-LEFT-RIGHT gestures.

Well, let's get started.

Theory

Beg a little materiel.

To obtain the necessary information on the movement and direction of movement, the APDS-9960 uses an IR LED and four photodiodes, which, as illustrated in the figure below, record signals in the near-IR range (NIR).



The IR LED (LED) has the function of illumination, and the photodiodes (UDLR) register the light reflected from the “obstacle”.

Photodiodes are located on the sensor in such a way that, depending on the direction of movement of the “obstacle”, the corresponding photodiode will receive most of the reflected IR signal at the input and a smaller part at the output. At the same time, the documentation on the APDS-9960 unambiguously tells us that it is possible to interpret the direction of motion by measuring and comparing the amplitude and phase difference of signals from UDLR photodiodes.

Practice

To work with the APDS-9960, as in the previous time , we will use STM32VLDISCOVERY. The connection also has not changed.

APDS-9960 Setup

We make the initial adjustment of the sensor.

So here it is:


What is going on here? Let's figure it out.

 i2c1_write(APDS9960_CONTROL, DEFAULT_PGAIN); 

PGAIN (Proximity Gain Control) is the parameter that controls the gain of the sensitivity of the approximation. Give it a value of 2, which is four times the gain.

 i2c1_write(APDS9960_GPENTH, DEFAULT_GPENTH); i2c1_write(APDS9960_GEXTH, DEFAULT_GEXTH); 

GPENTH (Gesture Proximity Enter Threshold Register) - this parameter sets the proximity threshold value to determine the start of gesture recognition.

GEXTH (Gesture Exit Threshold Register), respectively, sets the threshold for determining the end of gesture recognition.

 i2c1_write(APDS9960_GCONF2, DEFAULT_GGAIN); 

In the GCONF2 register (Gesture configuration two), we explicitly set only the GGAIN parameter (Gesture Gain Control) to the gain value four times.

 i2c1_write(APDS9960_GPULSE, DEFAULT_PULSE_LENGTH); i2c1_write(APDS9960_PPULSE, DEFAULT_PULSE_LENGTH); 

Backlight. By default, the value for the current source of the IR LED of the backlight is set to 0, which corresponds to a current of 100 mA, which is fine with us — we will not change it.

The IR illumination in the APDS-9960 is a sequence of pulses and is characterized by the corresponding register parameters for gestures GPULSE (Gesture pulse count and length): GPLEN (Gesture Pulse Length) and GPULSE (Number of Gesture Pulses), as well as the PPULSE approximation (Proximity Pulse Count Register ): PPLEN (Proximity Pulse Length) and PPULSE (Proximity Pulse Count) that specify the number of pulses and the period of each individual pulse.

We define that GPLEN and PPLEN take a value of 2 equal to 16 μs, and a GPULSE and PPULSE value of 9, which corresponds to 10 pulses.

As you can see, the setup turned out to be slightly more difficult than the similar one for color recognition and lighting from the previous review of the APDS-9960.

Reading data

Now let's move to the main cycle of the program, in which we start to register and interpret data from photodiodes every now and then, and also learn how to find differences between one gesture and another.

First of all, let's start the APDS-9960 with the functions of working with gestures and approximation.

 GesturesSet(GESTURES_START); 

And immediately begin to track the parameter GVALID. GVALID (Gesture FIFO Data) is a parameter of the GSTATUS register (Gesture Status Register), which, being in a state other than zero, tells us that the sensor has usable gesture data.

Documentation teaches us that gesture information is in a buffer, in the area of ​​RAM, which in general has a size of 32 x 4 bytes.

In practice, the actual size of this buffer can be found by reading the value of the register GFLVL (Gesture FIFO level), i.e. according to my purely empirical experimental observations, it turns out GFLVL * 4. Something like this:



Well, as the name of the buffer, the data in it are arranged in the order First In - First Out. That is, roughly speaking, the “earlier” the signal came from each of the photodiodes, the “higher” in GFLVL it is located.

Data from photodiodes (UDLR) can be read from the corresponding Gesture FIFO Register:

- GFIFO_U (Gesture FIFO Data, UP)
- GFIFO_D (Gesture FIFO Data, DOWN)
- GFIFO_L (Gesture FIFO Data, LEFT)
- GFIFO_R (Gesture FIFO Data, RIGHT)

After each reading of the values ​​from these registers, GFLVL is decremented; thus, for good, it is necessary to read the entire buffer completely until GFLVL reaches zero.

To define gestures, we need only the first four bytes of this buffer, not more. Therefore, we read only them.

 GestureUp = i2c1_read(APDS9960_GFIFO_U); GestureDown = i2c1_read(APDS9960_GFIFO_D); GestureLeft = i2c1_read(APDS9960_GFIFO_L); GestureRight = i2c1_read(APDS9960_GFIFO_R); 

Gesture recognition

To interpret what kind of gesture happened, let's make some simple calculations:

 GestUpDown = GestureUp-GestureDown; GestLeftRight = GestureLeft-GestureRight; 

To determine which of the gestures at the moment happened to us, it is not the GestUpDown and GestLeftRight values ​​that are important to us, but only a sign, so to speak, of a real number.

That is, in other words, taking as input the negative and positive values ​​of the GestUpDown and GestLeftRight variables determine which gesture is performed.

The truth table for the variables GestUpDown and GestLeftRight is shown in the figure below.



Now let's reset GFLVL:

 GesturesSet(GESTURES_STOP); 

... and return to the beginning of the main program cycle.

And now the whole code:


I want to note that the gesture mechanism of the APDS-9960 works very well. The recognition is stable, the filters built into the APDS-9960 UV and IR work well.

I hope this material will be useful to someone. Thanks for attention.