Kinect for Windows SDK. Part 1. Sensor

• [Kinect for Windows SDK. Part 1. Sensor]
• Kinect for Windows SDK. Part 2. Data Flows
• Kinect for Windows SDK. Part 3. Functionalities
• Playing cubes with Kinect
• Program, aport!

The three parts of the review article can be considered introductory to the other two: “We play cubes with Kinect” and “Program, anport!”. Although chronologically they go last. Moreover, while they were writing, Microsoft managed to release a new version of the SDK - 1.5. What a blow from the corporation!

As you should know, Kinect is a contactless controller, i.e. let's say in games, instead of violently knocking on the keyboard or tormenting the gamepad, you swing your arms and legs and occasionally voice you give some commands. It's time to hide the expensive Chinese vases X century away!

Starting with Kinect is best to get acquainted with the sensor itself. This is a small black box with three cameras, a little more than 20 cm long and about 5 cm high (not counting the legs). One nuance - in Kinect two cameras. And this image (taken from MSDN ) is a confirmation.


')

• IR Emitter - Infrared Emitter. Yes, this is not a camera. Its purpose is to emit infrared rays, which, being reflected from objects, fall back into the sensor, where they are received:
• IR Depth Sensor - Infrared receiver. Collects the reflected rays, converting them into the distance from the sensor to the object (s). And in this way, we can say that the distance matrix is ​​built - a whole frame. The maximum resolution is 640x480 (30 fps). If we talk about translation, it seems to me that the “IR receiver” reflects the essence better than the literal “IR depth sensor”.
• Color Sensor - Color camera. Capture video with a maximum resolution of 1280x960 (12fps). Camera viewing angle: 43 ° vertically and 57 ° horizontally (the rangefinder viewing angle is identical). You can select the image format: RGB or YUV .
• Microphone Array - a set of microphones. 4 built-in microphones allow you to determine the location of the sound source and the direction of the sound waves. The built-in audio signal handler includes echo cancellation and noise reduction.
• Tilt Motor - Tilt Correction. Allows you to programmatically adjust the tilt of the sensor in the range of ± 27 ° vertically.

The sensor is connected to the computer via USB, although, strictly speaking, a USB device is not, because has a slightly different firmware than a standard USB device. Find any comments on this issue did not work, but we can assume that this was done solely in order to improve performance. You can connect up to four sensors simultaneously.

Another important aspect of the hardware is the range of “working” distances, i.e. at what distance the sensor is able to see objects (rangefinder data). And again, the picture helps to demonstrate the capabilities of Kinect.



Default Range mode is available for both Kinect for Windows and Kinect for XBox, but Near Range is only available for the Kinect for Windows sensor. A simple example: for all objects in a frame that are within Normal Values, the sensor will return the measured distance, but for objects outside this range, the sensor will return either a zero or a constant defined for each range (depending on your settings). One of the significant differences of the SDK 1.5 from 1.0 is that now in the Near Range mode you can get information about a sitting person, this is the so-called tracking of a sitting figure (seated skeletal tracking) . Kinect tracks 10 points: head, neck and hands. But more on that later.

I would like to draw attention to the differences between the two Kinect (XBox and Windows). The first was developed and tested with one device - XBox. The second, on the contrary, should work with a wide range of devices (computers), so its cable is shorter, and a special device is designed to prevent Kinect from fighting with the rest of the USB peripherals. The license agreement allows you to use Kinect for Windows for commercial purposes, but Kinect for XBox does not.

We now turn to the installation of the SDK. In version 1.5, Microsoft split the SDK into two packages. You can download them from the official page :

• Kinect for Windows SDK . Includes APIs and sensor drivers.
• Kinect for Windows Developer Toolkit . Includes Kinect Studio (a useful debugging tool), Face Tracking SDK, code samples, and documentation.

We will not stop at installing the SDK, there is nothing complicated or interesting in this. But what is really interesting is that what a developer can get from Kinect? Take a look at the following features:

• Skeletal Tracking
• Face Tracking
• Speech Recognition (speech recognition)

An overview of these functions can be found in the following parts of the article.

Useful links:

• Kinect for Windows SDK
• Developer's Guide . You can find examples in C # (managed code) and in C / C ++ (unmanaged code)
• Kinect for Windows Quickstart Series . A series of video tutorials about the features of Kinect (very useful for exploring the device).
• Near Mode: What it is (and isn't) . The article about near mode, although written in the run-up to SDK 1.0, is useful in terms of exposing some myths.
• Kinect for Windows SDK forums . Here you can seek advice from experts.