Switching from Hand Mode to Intel RealSense SDK R4 (v6.0) to Cursor Mode in Intel RealSense SDK 2016 R1

After the appearance of the Intel RealSense SR300 camera and the Intel RealSense SDK 2016 R1 package, a new gesture-based interaction mode appeared - pointer mode ( Cursor ), available only when using the SR300 camera. This tutorial describes the code changes needed to enable this new functionality.

Prior to the release of the Intel RealSense SDK 2016 R1, applications in which you need to control the movement of the pointer and detect click actions used Hand mode, and gesture recognition was used to determine clicks. This Hand Mode functionality has now become a new Cursor mode. Applications that use the previous functionality will be able to use higher accuracy and more advanced pointer control in Cursor mode after modifying the code.

Please note that the Cursor mode is available only on devices with the Intel RealSense SR300 Camera. Intel RealSense application developers using SR300 cameras need to upgrade their operating system to Windows * 10 and use the Intel RealSense SDK version 2016 R1.

Most likely, you already have an application written for the F200 camera using Intel RealSense SDK R4 (v6.0). How to move on and use the new Cursor mode?
')
The process pipeline must be initialized in the same way as in the previous version of the Intel RealSense SDK. You need to create an instance of Sense Manager and make sure that there are no errors in the process.

PXCSenseManager *pSenseMgr = new PXCSenseManager::CreateInstance(); if( !pSenseMgr ) { < continue on to creating the modes > } 

Previously, in the Hand mode on the F200 cameras, to get something, even remotely resembling a pointer, it was necessary to use the Hand module and track hands in various configurations. The code might look something like this (note that the following code is for reference, it will not be compiled directly as shown here):

 PXCHandModule *pHandModule; PXCHandData *pHandData; int confidence; . . . <    > . . . pxcStatus status; if( !pSenseMgr ) { status = pSenseMgr->EnableHand() if(status == pxcStatus::PXC_STATUS_NO_ERROR) { // Get an instance of PXCHandModule handModule = pSenseMgr->QueryHand(); // Get an instance of PXCHandConfiguration PXCHandConfiguration handConfig = handModule handConfig->EnableGesture("cursor_click"); handConfig->ApplyChanges(); . . . <  > . . . } } 

In Intel RealSense SDK, starting from version 2016 R1, a new Cursor mode is implemented, the actions with the pointer are separated from the Hand mode. This means that you need to revise the old code that requested the Hand mode in the Sense Manager. The new code will look like this:

 PXCHandCursorModule *pCursorModule; PXCCursorData::BodySideType bodySide; //  ,   Confidence   . . . <    > . . . pxcStatus status; if( !pSenseMgr ) { // Enable handcursor tracking status = pSenseMgr::EnableHandCursor(); if(status == pxcStatus.PXC_STATUS_NO_ERROR) { // Get an instance of PXCCursorModule pCursorModule = pSenseMgr->QueryHandCursor(); // Get an instance of the cursor configuration PXCCursorConfiguration *pCursorConfig = CursorModule::CreateActiveConfiguration(); // Make configuration changes and apply them pCursorConfig.EnableEngagement(true); pCursorConfig.EnableAllGestures(); pCursorConfig.ApplyChanges(); . . . <  > . . . } } 

For examples of basic computational cycles for synchronous and asynchronous functions, see the Intel RealSense SDK 2016 R1 documentation in Implementing the basic computational cycle “Cursor Module [SR300]” section .

The asynchronous (recommended) approach will look like this:

 class MyHandler: public PXCSenseManager::Handler { public: virtual pxcStatus PXCAPI OnModuleProcessedFrame(pxcUID mid, PXCBase *module, PXCCapture::Sample *sample) { // check if the callback is from the hand cursor tracking module if (mid==PXCHandCursorModule::CUID) { PXCHandCursorModule *cursorModule=module->QueryInstance<PXCHandCursorModule>(); PXCCursorData *cursorData = cursorModule->CreateOutput(); // process cursor tracking data } // return NO_ERROR to continue, or any error to abort return PXC_STATUS_NO_ERROR; } }; . . . < SenseManager> . . . // Initialize and stream data MyHandler handler; // Instantiate the handler object // Register the handler object pSenseMgr->Init(&handler); // Initiate SenseManager's processing loop in blocking mode // (function exits only when processing ends) pSenseMgr ->StreamFrames(true); // Release SenseManager resources pSenseMgr ->Release() 

So, in the Intel RealSense SDK 2016 R1, the implementation and access to the hand pointer have changed, but all changes are homogeneous, which simplifies code modification. The above code example demonstrates this simplicity: it is shown that the structure of the program during initialization, tuning and frame-by-frame execution can remain unchanged, while the program will use the advanced features of the Cursor mode.

Recall that the new Cursor mode is available only for systems with the SR300 camera (the camera can be built-in or connected as a separate peripheral device) and with the RealSense SDK 2016 R1 version. The ability of the code to detect the camera model and maintain the F200 and SR300 cameras simultaneously will be described in other tutorials.