Have you thought of creating a simple color image streaming application using the Intel RealSense camera and the Intel RealSense SDK , or are you just going to use the color image stream in one of the applications? You need a clear application, simple in tracking his work, acting directly, without a huge amount of additional code, distracting from what you want to learn? In this case, you are lucky, because I tried to achieve exactly this result here: I tried to create a simple but effective sample application and a document describing the use of the camera and the Intel RealSense SDK.
This example was written using Intel RealSense SDK R4 and Visual Studio * in C #, and was tested in the R5 release. It requires an Intel RealSense F200 camera.

Project structure

In this simple application, I tried to separate the functionality of the Intel RealSense SDK from the code of the Windows UI level GUI to make it easier for developers to focus on the functionality of the SDK related to image streaming. To do this, I created a shell class (RSStreaming) in C #, enclosing several Intel RealSense SDK classes.

The Windows Form application contains only a few buttons and a PictureBox control that displays a stream of color images.

Please note that I am not trying to create the perfect application. I added certain means of exception handling, but this was limited. Everything else is your concern if you want to apply the correct programming principles and get the application stability and convenience for users.
')
The structure of the project is based on the use of events for data transmission, so that you can do without close links. An auxiliary event class RSNewImageArg was created, inheriting from EventArg. It is used to publish the current frame from the camera to the client application.

Beginning of work

To work, you need an Intel RealSense F200 camera. Intel RealSense SDK version R4 or later is also required. In addition, the appropriate Depth Camera Manager (DCM) package must be installed on the computer. The SDK and F200 DCM can be downloaded here .

Requirements

equipment requirements

• 4th generation Intel® Core ™ processor (based on Haswell microarchitecture).
• 8 GB of free hard disk space.
• Intel RealSense F200 Camera (you need to connect it to a USB 3 port).

Software Requirements

• 64-bit version of Microsoft Windows 8.1 or Windows 10 operating system.
• Microsoft Visual Studio * 2010–2015 with the latest service pack.
• Microsoft .NET * 4.0 Framework for C # development.
• Unity * 5.x or later for developing games on the Unity engine.

Project Components

RSNewImageArg.CS

RSNewImageArg is derived from the C # EventArgs class. As you can see, this is a small wrapper class to which one private data item has been added. This private bitmap _bitMap data element contains the current bitmap extracted from the camera stream.
This class is used as an event argument when the RSStreaming class sends an event back to the Form class, indicating that the new bitmap image is ready for display.

RSStreaming.CS

RSStreaming is a wrapper class (a kind of “engine”) with data flow of a color image from an Intel RealSense camera. I wrote this class for the following purposes.

• Clearly and clearly identify as many features of the Intel RealSense SDK as possible and separate them from the client application as far as possible.
• Try to provide comments in the code to help the reader understand exactly what the code does.

Below are described all the functions included in the class RSSpeechEngine .

public event EventHandler <RSNewImageArg> OnStreamingImage

The OnStreamingImage event is used to switch a message back to the client application, informing it that the new color bitmap image is ready for display. The client creates event processing to handle the RSNewImageArg object.

public bool Initialized

The get method property is used as a flag, indicating that the RSStreaming class is initialized.

public bool IsStreaming

The get method property is used as a flag, indicating that the RSStreaming class is currently streaming color image data.

public void StartStreaming ()

Checks if the class is initialized. If not, calls InitCamera to make sure that the class is up and running. This function then calls the _senseManager.StreamFrames (...) function.
If you had time to get acquainted with a sufficient amount of materials on developing applications for Intel RealSense, then you probably noticed that the data from the camera is often received in a while loop. For example, something like this.

while (!Stop) { /* Wait until a frame is ready: Synchronized or Asynchronous */ if (sm.AcquireFrame(Synced).IsError()) break; /* Display images */ PXCMCapture.Sample sample = sm.QuerySample(); /* Render streams */ EventHandler<RenderFrameEventArgs> render = RenderFrame; PXCMImage image = null; if (MainPanel != PXCMCapture.StreamType.STREAM_TYPE_ANY && render != null) { image = sample[MainPanel]; render(this, new RenderFrameEventArgs(0, image)); } if (PIPPanel != PXCMCapture.StreamType.STREAM_TYPE_ANY && render != null) render(this, new RenderFrameEventArgs(1, sample[PIPPanel])); /* Optional: Set Mirror State */ mirror = Mirror ? PXCMCapture.Device.MirrorMode.MIRROR_MODE_HORIZONTAL : PXCMCapture.Device.MirrorMode.MIRROR_MODE_DISABLED; if (mirror != sm.captureManager.device.QueryMirrorMode()) sm.captureManager.device.SetMirrorMode(mirror); sm.ReleaseFrame(); /* Optional: Show performance tick */ if (image!=null) timer.Tick(PXCMImage.PixelFormatToString(image.info.format)+" "+image.info.width+"x"+image.info.height); } 

Yes, this is a solid piece of code. This code probably does much more than my sample application, but I just want to say that my while loop is not used in this way. My application uses the StreamFrames function (...) . This function handles a while loop inside itself, and for each frame it includes an event that RSStreamingRGB subscribes to . It works like this.

1. We start the stream PXCMSenseManager.StreamFrames (...) .
2. We catch event event handler.
3. At the end of the streaming call PXCMSenseManager.Close () .

I like this approach because I don’t want to manually fiddle with the while loop, knowing when and how to stop the loop. Better to let the SDK take care of it instead of me. You will see how this technique works when I talk about the InitCamera () function, so I will not talk about it here. Just make sure you understand how to stream data and allow the SDK to manage the loop over the raw data coming from the camera.

After calling StreamFrames, I set the boolean flag _isStreaming to true to notify the class and client application of the start of streaming.

public void StopStreaming ()

Stopping streaming is the opposite of StartStreaming. This function instructs the SDK to stop the transfer of data from the camera and calls Dispose () to destroy the data objects.

private void InitCamera ()

InitCamera () creates an instance of PXCMSenseManager and includes the stream type we need. As you can see, I specify a color image stream with a resolution of 320 x 240 at 30 frames per second.
If you remember, I talked about the possibility of using the event from the PXCMSenseManger to inform the class about the availability of a new frame of color image data available. To do this, use the event class PXCMSenseMananger.Handler . The process is quite simple: create an instance of the Handler class, assign it to an event handler via onNewSample , then initialize the PXCMSenseManager object ; _senseMananger with a handler class.
Then we set the _initialized flag to true. As mentioned above, using this flag, we inform either this class or the client application that the RSStreaming class is initialized.

private pxcmStatus OnNewSample ()

This is the event handler for the PXCMSenseMananger.Handler () object. If you remember, in the InitCamera () function, I set an object event handler for this function.
The event handler must match the specified function signature. The function must return the value of pxcmStatus and take two parameters.

• Mid. Thread id If multiple streams are requested via the EnableVideoStream s function, then it can be PXCMCapture.CUID + 0 , PXCMCapture.CUID + 1 , etc.
• Sample. Available image.

You need to convert the PXCMCapture.Sample object to a usable bitmap image that can be displayed by the client application.

First, I check that the sample.color object is not empty and that the internal bitmap image of the _colorImageData class is also not empty. You need to make sure that _colorImageData does not contain any data, and if it does, then you need to free them.

Then you need to use the sample.colors object to fill in _colorImagedata . This is, in essence, a metadata object for the PXCMCapture.Sample color image object . After that, you can proceed to creating a bitmap image by specifying its size.
After receiving the bitmap and making sure that it is not empty, I enable the OnStreamingImage event, indicating the source of the event and the new RSNewImageArg object.

Finally, we MUST release the current frame from the PXCMSenseMananger object, and, since it is required by the function signature, return pxcmStatus . At this point, you could add code to handle exceptions, but I chose not to do this for the sake of simplicity. If I wrote such code, I could return another state of pxcmStatus , but I just return a success state.

private void Dispose ()

Dispose () performs a cleanup. I check that the dispatcher is not empty and that it was initialized. If both conditions are met, then I call his cleaning method. I check that the RSStreaming bitmap is not empty, and I clear it. Then I set the value to null everywhere.

MainForm.CS

The main form is a graphical user interface that displays a stream of color images. Here you can control the RSStreaming object. It has two global variables: an RSStreamingRGB instance and a bitmap image. The bitmap will contain the current image from the current frame, which is sent by the RSStreamingRGB class.

public MainForm ()

Form Designer. It creates a new RSSTreamingRGB object and provides an event handler for the OnStreamingImage event.

private void btnStream_Click ()

Event handler when you click Start Streaming. Commands the _rsStreaming object to start streaming by calling the StartStreaming () function of this object.

private void btnStopStream_Click ()

Event handler when you click Stop Streaming. Tells the _rsStreaming object to stop streaming by calling the StopStreaming () function of this object.

private void UpdateColorImageBox (object source, RSNewImageArg e)

UpdateColorImageBox - the event handler for the _rsStream.OnStreamingImage event. Checks that the newImage argument is not empty and in this case assigns _currentBitMap to the new bitmap image, using newImage as the source of the bitmap image.

If you do not create a new bitmap, then the _currentBitMap of the form will point to the original bitmap created by the SDK. This can cause problems when calling the RSStreaming.Dispose method. The client program has a picture frame, and in this frame is an image that is taken from the SDK. If, with the active form and frame of the picture, I try to call RSStreaming.Dispose , which releases the SDK resources, then I get a crash, because the original image for the frame of the picture is deleted.

After assigning a new image to _currentBitMap, I call pictureBox.Invalidate () and thus trigger the Paint event of the picture frame.

private void pictureBox_Paint (object sender, PaintEventArgs e)

This is the Paint event handler for the picture frame; it is switched by calling the pictureBox.Invalidate () . He gives the command to redraw the picture frame using the current source image.
First, we check that _currentBitMap is not empty, and then set the most recent bitmap for it, which is saved in _currentBitMap .

private void btnExit_Click ()

Everything is elementary here. Just call Close () . There is no need to process cleanup, since all this happens in the MainForm_FormClosing method.

private void bMainForm_FormClosing ()

This form event closes an event handler. When you call the Close () method in any given function, the FormClosing event is raised . In order not to duplicate the code, I posted here the entire cleaning code. We check that _rsStream is not empty and that the stream is being transmitted. If these conditions are met, we call _rsStream.StopStreaming () . There is no need to call the cleanup method for _rsStreaming , since everything you need is already done in StopStreaming .

Conclusion

I hope that with the help of this article and the sample code, you better understand how to use the Intel RealSense SDK to create simple color image transfer applications. I tried to demonstrate that this can be done in a simple and understandable application: it implements all the elements necessary to successfully create your own color image transfer application.