See the world through the eyes of animals: new horizons of eytreking

When we talk about technologies for recognizing emotions, we mean by default a person, that is, the entire array of data - verbal and non-verbal - that is generated and used by them for a comprehensive assessment of his condition. However, the further, the more attention is paid to animals: in recent years, studies of the “emotional” world of horses or dogs, for example, have been regularly conducted. In our article, we will discuss one of the reversals of this topic: the way in which the tracking technology is used to study the characteristics of some of the fauna living side by side with us. So how to look around with the eyes of a dog, a peacock or even a mouse?

Dogs

Probably, every dog ​​lover wants to know how human friends perceive us and their relatives. Therefore, this direction of registration of animal eye movements leads in the diversity of experimental non-invasive techniques.

For dogs, as for other social animals, the look is an important social signal. The predominant function of a dog’s gaze in the eyes is a signal of dominance or ritualized aggression (Schenkel, 1967). However, such oculomotor behavior demonstrated by the dog towards the owners is interpreted as the initialization of communication (Kis, Hernádi, Miklósi, Kanizsár, & Topál, 2017). Also, analysis of the oculomotor activity of dogs is a promising direction for the study of not only communication, but also face recognition and emotional facial expressions of people.

In the course of research using the methods of eytreking on dogs, it was shown that the latter are able not only to recognize people by their faces, but also to distinguish individual human emotions: joy, sadness, anger. When viewing images of people's faces, dogs prefer the eye area, regardless of the facial expression of the stimulus (Kis et al., 2017).
Tornqvist et al. (Törnqvist et al., 2015) conducted a curious comparative study of image viewing by dogs and humans. Photographs depicting social interaction or avoiding social interaction between two people or two dogs were chosen as stimulus material. So, both people and dogs demonstrated a longer time viewing objects in images with social interaction. Note that people more time looked at the pictures with the social interaction of dogs than people, while the dogs - on the contrary.
')
As for the study of human eye movements, there are two main types of experimental setups for dogs: stationary and wearable Aytrekeri.

Stationary installations are a monitor-mounted tracker or projector on which stimuli are presented. In such installations, a dog or pre-trained to stay for a long time (for example, Tornqvist et al. (2015) used this version of the experiment), and it is also used in Clever Dog Lab - the course of experiments using video-celography techniques in this laboratory can read here ), or the owner fixes it in the right position, as was done in the study of Kis et al. (Kis et al., 2017). With regard to the preliminary training of dogs in the experimental setup, there is an opinion that training in itself creates an additional cognitive task for the dog, which distorts the distribution of visual attention (Kis et al., 2017). In addition, it is logical to assume that retention in the installation by the owner can also introduce some kind of distortion in the results obtained, and they are fundamentally important when studying the psycho-physiological state of animals.

Therefore, the most promising method for studying the eye movements of dogs in more natural conditions are eytreker points systems. So, at the University of Lincoln (UK), Williams and colleagues (Williams, Mills, & Guo, 2011) developed on the basis of the eytreker VisionTrak (60 Hz) - tracker glasses for dogs, working with an accuracy of 2.25-2.71 °. The whole structure is fixed on the muzzle: between the ears is a scene camera, in front of the left eye - a dichroic mirror , above the left eye - a camera for recording eye movements and infrared illumination. During calibration, a calibration structure is attached to the muzzle center (in the nose area), which consists of two crossed axes, the ends of which correspond to the frame boundaries of the scene camera. At the end of each calibration axis, there is an area where something tasty is fixed during the calibration — the dog’s task is to look at the desired piece of food, with successful calibration it receives it as a reward. Thus, this tracker does not require special training of the dog's behavior during the experiment, but involves only a brief training for calibration.

Another project to create a tracker points for dogs - DogCam (Rossi, Parada, & Allen, 2010).
A lot of amazing facts about the study of the behavior of dogs can be found on the page CleverDogLab .

Birds

Birds have extremely sharp eyesight, and among them there are no species that would lack vision as such, therefore studying their visual perception is of particular interest. The peculiarity of focusing on the object in birds is that they have practically no vestibulo - ocular reflex , that is, the head can move faster than the eyes. For example, in pigeons, eye movements cause only 10–20% of gaze movement (Yorzinski, Patricelli, Platt, & Land, 2015).

Today, there are eye tracking systems for recording eye movements of only large birds. Such an eytreker consists of a helmet on which a camera for recording eye movements (30 Hz) with infrared illumination and a scene camera is mounted, and a backpack with a battery and a transmitter. Similar trackers produce only monocular registration (the second eye of the bird is closed), since the design with two cameras for recording eyes would be too heavy.

For example, during the use of the described system in the study of the visual attention of female peacocks, significant results were obtained: when females see a peacock male from close range, their visual attention is directed to the lower half of the tail, and when the male peacock dismisses the tail, being far from the female - her attention is focused on the upper half of the tail (Yorzinski, Patricelli, Babcock, Pearson, & Platt, 2013).

At present, there is no adequate equipment to study the eye movements of birds of a small size (such as starlings), so the problems of visual attention of small birds are reduced to tracking head movements. Thus, the Butler and Fernandez-Juricic study (Butler & Fernandez-Juricic, 2018) showed that starlings (ordinary or European starling) use foveal vision when viewing a predator (the authors suggest that at this time the bird receives detailed information that can be required to successfully avoid the enemy).

Mice

H. Payne and JL Raymond (Payne & Raymond, 2017) developed a method for registering eye movements on a freely moving mouse with a very high resolution of 0.1 °. To do this, they applied the method of magnetic light tracking. Mice are implanted with a cylindrical magnet (0.75 x 1 mm in size, weighing 6.8 mg) in a biocompatible polymer shell on the temporal side of the conjunctiva, the incision in which is healed with a tissue adhesive. The magnet is perpendicular to the axis of the horizontal saccades . A small angular magnetic sensor (4.8 x 5.8 mm in size, weighing 76 mg) is attached above the implanted magnet on the skull. The authors also conducted pilot experiments with smaller disk implantable magnets (1.5 x 0.5 mm), but they gave a too weak signal.

Calibration of this system is a synchronous recording of a magnetic tracker signal and video using the dual-angle video-oculography method on a mouse whose head is fixed. The essence of the dual-angle video-oculografy lies in the video recording of eye movements by two fixed cameras with infrared illumination at an angle of 40 ° from each other and at an equal distance from the center of the pupil (5 cm). The position of the eye is calculated for each frame by comparing the images from both cameras, the position of the eye is tied to the signal from the magnetic tracker. Calibration is carried out only once and in the future the magnetic tracker can be used on a freely moving mouse in complete darkness. The original article is available online; you can read it here .

Primates

In our article, of course, you can not ignore the closest relatives of a person from the animal kingdom - primates. A great deal of successful primitive research on primates has been done than on other animals, since monkeys are a classic model object for studying the neurophysiology of the visual system. The most common method for registering eye movements in primates is using electromagnetic raytracking (we used this method of recording eye movements on a person in our previous article ), however, in the case of monkeys, the movable element is attached not to the lens, but implanted. Often, this method is used in combination with invasive neurophysiological techniques that allow detailed analysis of the features of the brain and visual attention.
About the study of the movements of the eyes of monkeys in Russia can be read here .

Literature

Material author:
Maria Konstantinova, Researcher at Neurodata Lab , biologist, physiologist, specialist in visual sensory system, oculography and ocular motorics.