The Kinect game controller is the first device on the market for direct three-dimensional measurement.
Launched on the market in November 2010, the Kinect game controller is the first low-cost home appliance for direct, remote three-dimensional measurement of objects and scenes. I sincerely thank Microsoft for this step.
The first device of this type was any type of radar and sonar, ranging from air defense system devices and ending with a parking assistant on a vehicle. If the speed of propagation of a probe pulse in a medium is known, then multiplying the speed by half the time between the moments of radiation and receiving a pulse will give the distance from the emitter to the object that reflected the signal. By orienting the emitter-receiver system in different ways in space, it is possible to measure a set of distances to reflecting objects in a given direction.
Production of compact solid-state lasers made it possible to create measurement systems that used light as a probe pulse. A mechanical method of orienting the light beam to such systems to perform consecutive measurements in the spatial sector (or scan it, respectively, the measurement device became known as laser scanners ).
Any scanner produces a set of three-dimensional coordinates of the points of reflection of the laser beam relative to the point of radiation. Representation in such a local coordinate system is quite sufficient for measurements of local dimensions up to the building ( stationary scanner ). If you provide such a scanner with a sufficiently accurate GPS receiver and a system for measuring its angular orientation in three directions, then you can install it on an aircraft as an aerial survey scanner or on a vehicle as a mobile scanner .
The working wavelength of scanners is near infrared light with a wavelength of about 1 micron for aircraft scanners (less scattering in atmospheric vapors) and greater than 1 micron for stationary scanners (safer for vision).
The result of laser scanners is a point cloud .
The whole history of laser scanning happened literally "just now", right before my eyes, and fits into 20 years.
There are many advantages:
- Direct measurement method of three-dimensional coordinates, as opposed to stereo.
- Ability to measure through optically transparent covers. In particular, airborne laser scanning makes it possible to obtain relief coordinates under the cover of vegetation.
- High performance, which is especially important for mobile systems.
But there are drawbacks, there are three main ones:
- The duration of the process of mechanical scanning of a designated spatial sector (full space for a stationary scanner, a strip or a circle for a mobile). In addition, additional scanning systems may distort the beam path, which requires separate processing.
- Lack of brightness information. Modern scanners allow you to measure the intensity of the reflected light, which is only to some extent able to replace the brightness image, but does not completely solve the problem of interpreting the reflection points.
- The combination of a laser scanner with parallel digital photography allows you to get the brightness characteristic of the scene only indirectly by projecting the photos on the relevant parts of the point cloud. Especially laborious is the process for
- Large size and power consumption, although recently manufacturers are working on this issue . This winter, I saw an electric unmanned helicopter with a very compact scanner on board, which we surveyed a part of the forest in Lapland.
')
An absolutely obvious approach to remote three-dimensional measurement is the use of an optical focusing system and a digital matrix. If the parameters of the optical system are known, then the task of restoring the spatial position of any beam arriving at the matrix is trivial. The matter remains in order to create a flash of light on the body of the measuring apparatus and somehow measure the delay in the arrival of light reflected from the object on each pixel of the digital matrix.
In 2006, at one of the industry conferences, there was a discussion of the newly emerged star of Google Maps, coupled with the concept of "neogeography". Virtually every speech was accompanied by the mention of the need to move to a truly three-dimensional textured data. Listening to the speakers, I clearly felt that in five to seven years, portable household 3D cameras would enter the market. But scanner solutions for this role are poorly suited.
The idea of a new solution occurred to many, while the investment plan was proposed according to the same scheme: “Now you will give us a million (or ten, or a hundred), and we will give you a machine in ten years (or five, or in a year)” . We did not invest.
In the spring of 2009, the company acquired an Israeli firm 3DV Systems. The Israelis had previously launched their rangefinder ZCam into the market and set their sights on the game controller market. Apparently, Microsoft has evaluated the technology as promising and decided to use it for implementation in the new controller for the Xbox 360 game console, calling this development “Project Natal”
The design of the camera allows to refer it to the so-called "time-of-flight", which, in essence, means a parallel implementation of the above principles.
The main components of the camera are the emitter, the optical system, a special digital matrix and an electronic measuring and computing system.
It should be noted that the characteristic dimensions of the measurement area (several meters) and the required accuracy (several centimeters) impose very stringent requirements on the time accuracy of operation of all measuring devices.
For example, to detect an object at a distance of several meters from the camera, the characteristic reflection delay time is only 10 ns (at a speed of light c = 3 10 8 m / s). Accordingly, it can be calculated that the stated accuracy of determining the distance requires determining the moment of arrival of the response with an accuracy of two orders of magnitude more. The characteristic operating frequencies of the electronic circuit are in the gigahertz region.
Lens and flash appearance
The emitter creates a very short unidirectional light flash (equivalent in duration to a distance of several meters, which determines the maximum range of the camera, for example, 7.5 meters with a flash duration t 0 = 50 ns). The working wavelength is most likely 1.5 microns. The light reflected from the scene comes back and is focused by the optical system so that it falls on the digital matrix.
This matrix has a special type and consists of photodiodes. When light hits the photodiode, it begins to produce a current that is transmitted through the key to the storage elements (capacitors). In the simplest case, two capacitors can be used.
Initially, the first capacitor S 1 is charged, 50 ns after the start of the light flash, the key switches to the charging of the second capacitor S 2 . After another 50 ns, the key goes to the neutral position.
After a short interval, the whole process is repeated again. Since the flash frequency is large enough and the whole scene can be considered stationary, the charges of capacitors S 1 and S 2 accumulate from flash to flash, thereby increasing the signal / noise level and increasing the reliability of the calculation. Several thousand cycles are used for accumulation.
As a result, the ratio between the charges S 1 and S 2 allows to determine the position of the front of the reflected pulse relative to the flash. You can finally calculate the distance D along the beam that came to this pixel using the formula
D = c t 0 S 2/2 ( S 1 + S 2 )
Timing Charts (from wikipedia)
According to the stated specifications, the Kinetic controller maintains a working range of distances from 1 to 6 meters and has a field of view of approximately 50 angular degrees in both directions. The resolution of the ranging matrix is small and is most likely 320 x 240 pixels.
Simultaneously with the collection of long-range information, the usual digital matrix performs video shooting of the scene.
Clouds of dots as a result of Kinect
They are parasitic illumination, which can exceed the brightness of the flash and multiple reflections of light, which gives fake points of reflections. In principle, it is possible to deal with this by introducing a particular modulation of the signal to highlight it from the background.
Among them, I would call compact three-dimensional home scanners and use in robotics and transport (I recall a three-dimensional scanner installed on an IV skateboard in Stevenson's “Avalanche”)
It amazes me that the level of development of electronics now allows us to directly measure the time required for the fastest process in the world to spread over several meters. I hope that a series of similar devices will not be limited to Kinect alone and the proposed technology will take its rightful place.
Recent history
The first device of this type was any type of radar and sonar, ranging from air defense system devices and ending with a parking assistant on a vehicle. If the speed of propagation of a probe pulse in a medium is known, then multiplying the speed by half the time between the moments of radiation and receiving a pulse will give the distance from the emitter to the object that reflected the signal. By orienting the emitter-receiver system in different ways in space, it is possible to measure a set of distances to reflecting objects in a given direction.
Production of compact solid-state lasers made it possible to create measurement systems that used light as a probe pulse. A mechanical method of orienting the light beam to such systems to perform consecutive measurements in the spatial sector (or scan it, respectively, the measurement device became known as laser scanners ).
Any scanner produces a set of three-dimensional coordinates of the points of reflection of the laser beam relative to the point of radiation. Representation in such a local coordinate system is quite sufficient for measurements of local dimensions up to the building ( stationary scanner ). If you provide such a scanner with a sufficiently accurate GPS receiver and a system for measuring its angular orientation in three directions, then you can install it on an aircraft as an aerial survey scanner or on a vehicle as a mobile scanner .
The working wavelength of scanners is near infrared light with a wavelength of about 1 micron for aircraft scanners (less scattering in atmospheric vapors) and greater than 1 micron for stationary scanners (safer for vision).
The result of laser scanners is a point cloud .
The whole history of laser scanning happened literally "just now", right before my eyes, and fits into 20 years.
Advantages and disadvantages of laser scanning
There are many advantages:
- Direct measurement method of three-dimensional coordinates, as opposed to stereo.
- Ability to measure through optically transparent covers. In particular, airborne laser scanning makes it possible to obtain relief coordinates under the cover of vegetation.
- High performance, which is especially important for mobile systems.
But there are drawbacks, there are three main ones:
- The duration of the process of mechanical scanning of a designated spatial sector (full space for a stationary scanner, a strip or a circle for a mobile). In addition, additional scanning systems may distort the beam path, which requires separate processing.
- Lack of brightness information. Modern scanners allow you to measure the intensity of the reflected light, which is only to some extent able to replace the brightness image, but does not completely solve the problem of interpreting the reflection points.
- The combination of a laser scanner with parallel digital photography allows you to get the brightness characteristic of the scene only indirectly by projecting the photos on the relevant parts of the point cloud. Especially laborious is the process for
- Large size and power consumption, although recently manufacturers are working on this issue . This winter, I saw an electric unmanned helicopter with a very compact scanner on board, which we surveyed a part of the forest in Lapland.
')
The obvious solution
An absolutely obvious approach to remote three-dimensional measurement is the use of an optical focusing system and a digital matrix. If the parameters of the optical system are known, then the task of restoring the spatial position of any beam arriving at the matrix is trivial. The matter remains in order to create a flash of light on the body of the measuring apparatus and somehow measure the delay in the arrival of light reflected from the object on each pixel of the digital matrix.
In 2006, at one of the industry conferences, there was a discussion of the newly emerged star of Google Maps, coupled with the concept of "neogeography". Virtually every speech was accompanied by the mention of the need to move to a truly three-dimensional textured data. Listening to the speakers, I clearly felt that in five to seven years, portable household 3D cameras would enter the market. But scanner solutions for this role are poorly suited.
The idea of a new solution occurred to many, while the investment plan was proposed according to the same scheme: “Now you will give us a million (or ten, or a hundred), and we will give you a machine in ten years (or five, or in a year)” . We did not invest.
And Microsoft has invested
In the spring of 2009, the company acquired an Israeli firm 3DV Systems. The Israelis had previously launched their rangefinder ZCam into the market and set their sights on the game controller market. Apparently, Microsoft has evaluated the technology as promising and decided to use it for implementation in the new controller for the Xbox 360 game console, calling this development “Project Natal”
ZCam ranging camera device (based on materials from the Internet)
The design of the camera allows to refer it to the so-called "time-of-flight", which, in essence, means a parallel implementation of the above principles.
The main components of the camera are the emitter, the optical system, a special digital matrix and an electronic measuring and computing system.
It should be noted that the characteristic dimensions of the measurement area (several meters) and the required accuracy (several centimeters) impose very stringent requirements on the time accuracy of operation of all measuring devices.
For example, to detect an object at a distance of several meters from the camera, the characteristic reflection delay time is only 10 ns (at a speed of light c = 3 10 8 m / s). Accordingly, it can be calculated that the stated accuracy of determining the distance requires determining the moment of arrival of the response with an accuracy of two orders of magnitude more. The characteristic operating frequencies of the electronic circuit are in the gigahertz region.
Lens and flash appearance
General principle of the camera
The emitter creates a very short unidirectional light flash (equivalent in duration to a distance of several meters, which determines the maximum range of the camera, for example, 7.5 meters with a flash duration t 0 = 50 ns). The working wavelength is most likely 1.5 microns. The light reflected from the scene comes back and is focused by the optical system so that it falls on the digital matrix.
This matrix has a special type and consists of photodiodes. When light hits the photodiode, it begins to produce a current that is transmitted through the key to the storage elements (capacitors). In the simplest case, two capacitors can be used.
Initially, the first capacitor S 1 is charged, 50 ns after the start of the light flash, the key switches to the charging of the second capacitor S 2 . After another 50 ns, the key goes to the neutral position.
After a short interval, the whole process is repeated again. Since the flash frequency is large enough and the whole scene can be considered stationary, the charges of capacitors S 1 and S 2 accumulate from flash to flash, thereby increasing the signal / noise level and increasing the reliability of the calculation. Several thousand cycles are used for accumulation.
As a result, the ratio between the charges S 1 and S 2 allows to determine the position of the front of the reflected pulse relative to the flash. You can finally calculate the distance D along the beam that came to this pixel using the formula
D = c t 0 S 2/2 ( S 1 + S 2 )
Timing Charts (from wikipedia)
According to the stated specifications, the Kinetic controller maintains a working range of distances from 1 to 6 meters and has a field of view of approximately 50 angular degrees in both directions. The resolution of the ranging matrix is small and is most likely 320 x 240 pixels.
Simultaneously with the collection of long-range information, the usual digital matrix performs video shooting of the scene.
Clouds of dots as a result of Kinect
Disadvantages of the scheme
They are parasitic illumination, which can exceed the brightness of the flash and multiple reflections of light, which gives fake points of reflections. In principle, it is possible to deal with this by introducing a particular modulation of the signal to highlight it from the background.
Perspectives
Among them, I would call compact three-dimensional home scanners and use in robotics and transport (I recall a three-dimensional scanner installed on an IV skateboard in Stevenson's “Avalanche”)
It amazes me that the level of development of electronics now allows us to directly measure the time required for the fastest process in the world to spread over several meters. I hope that a series of similar devices will not be limited to Kinect alone and the proposed technology will take its rightful place.
Source: https://habr.com/ru/post/109337/
All Articles