In this section i will give a tutorial for installing Intel RealSense SDK and OpenCV in Microsoft Visual Studio 2015.
With the Intel® RealSense™ SDK, you have access to robust, natural human – computer interaction (HCI) algorithms such as face tracking, finger tracking, gesture recognition, speech recognition and synthesis, fully textured 3D scanning and enhanced depth augmented reality.With the SDK you can create Windows* desktop applications that offer innovative user experiences.
Previous, i had posted tutorial about installing opencv in visual studio, that this tutorial abput installing intel realsense sdk might more simple but first there are several hardware requirement which you should have. Intel has been releasing hardwares there are Intel RealSense F200, R200 and SR300.
At least you have one of those hardware to use Intel RealSense SDK. Second, you should download and install the latest Intel RealSense SDK (latest SDK is 2016 R2 or version 10.0.26.0396) you can download at here.
The next hardware you should have is your PC support usb 3 because all intel real sense camera have 3 camera and high resolution RGB that required a usb 3 for highspeed data transfer.
And then we’re going to software configuration, Make sure you are already installed Microsoft Visual Studio 2015 within C++ language. If you’re already read tutorial installing opencv in visual studio you can continue to this tutorial.
- Create new project with empty project visual c++
- Create cpp files with named ‘main’
- Go to Project properties
- Go to C/C++ -> Additional Include Directories, put these entry.
- Go to Linker -> General -> Additional Library Directories, put these entry .
- Go to Linker -> Input ->Additional Dependencies, put these entry.
- Chose the x64 platform at Project Properties.
- Go to next section of this tutorial at below.
In this tutorial, i join with opencv sdk to exploit opencv image viewer because using windows image panel for c++ and its stuff is not simple.
Import all dependencies header
Put this converted from realsense image format to opencv image format function
cv::Mat PXCImage2CVMat(PXCImage *pxcImage, PXCImage::PixelFormat format)
pxcImage->AcquireAccess(PXCImage::ACCESS_READ, format, &data);
int width = pxcImage->QueryInfo().width;
int height = pxcImage->QueryInfo().height;
format = pxcImage->QueryInfo().format;
int type = 0;
if (format == PXCImage::PIXEL_FORMAT_Y8)
type = CV_8UC1;
else if (format == PXCImage::PIXEL_FORMAT_RGB24)
type = CV_8UC3;
else if (format == PXCImage::PIXEL_FORMAT_DEPTH_F32)
type = CV_32FC1;
cv::Mat ocvImage = cv::Mat(cv::Size(width, height), type, data.planes);
and then put the main function
int main(int argc, char* argv)
PXCSession *session = PXCSession::CreateInstance();
PXCSenseManager *sm = PXCSenseManager::CreateInstance();
cv::Size frameSize = cv::Size(640, 480);
float frameRate = 60;
cv::Mat frameColor = cv::Mat::zeros(frameSize, CV_8UC3);
cv::Mat frameDepth = cv::Mat::zeros(frameSize, CV_8UC1);
sm->EnableStream(PXCCapture::STREAM_TYPE_COLOR, frameSize.width, frameSize.height, frameRate);
sm->EnableStream(PXCCapture::STREAM_TYPE_DEPTH, frameSize.width, frameSize.height, frameRate);
if (sm->AcquireFrame(true) < PXC_STATUS_NO_ERROR) break;
sample = sm->QuerySample();
frameColor = PXCImage2CVMat(sample->color, PXCImage::PIXEL_FORMAT_RGB24);
PXCImage2CVMat(sample->depth, PXCImage::PIXEL_FORMAT_DEPTH_F32).convertTo(frameDepth, CV_8UC1);
int key = cv::waitKey(1);
if (key == 27)
The RealSense SDK have several modules, I/O modules and algorithm modules, a PXCSession. Other SDK’s might create a single or multiple SDK session. each session maintains its own context for I/O and algorithm models and is not aware of other sessions. This PXCSession should use before using other classes or algorithms in RealSense SDK.
And then for PXCSenseManager is a main interface to call algorithm modules, camera input device and of course algorithm modules executed within camera input that serves to the algorithm. The algorithms that included in SDK are HandTracking and FaceTracking.
the project files can be download in here.