Ask Your Question

openni2_tracker and rgb data

asked 2015-03-04 10:51:51 -0500

Chaos gravatar image

updated 2015-03-05 05:13:53 -0500

Hi all! I'm using the openni2_tracker with an ASUS Xtion Pro Live for controlling a mobile robot through a gesture recognition node I made. I'm using ROS Indigo on Ubuntu 14.04. I also need the RGB image provided by the Xtion. I tried

roslaunch openni2_launch openni.launch

while running the openni2_tracker, but as also stated here it does not work.

Is there a way to get the RGB image while running the openni2_tracker node? I'm interested ONLY in the RGB raw image.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2015-05-05 09:18:06 -0500

Yianni gravatar image

updated 2015-05-06 09:38:45 -0500

Using openni::Videostream. Find below some example relevant source code (header and footer to see where it goes in the main of tracker.cpp). I am also copying the complete file but it is a mess and read the bold text as you need to change two variables.

Relevant snippet

// Get Tracker Parameters
if(!pnh.getParam("tf_prefix", tf_prefix)){
    ROS_ERROR("tf_prefix not found on Param Server! Check your launch file!");
    return -1;
if(!pnh.getParam("relative_frame", relative_frame)){
    ROS_ERROR("relative_frame not found on Param Server! Check your launch file!");
    return -1;

// Initial OpenNI
if( openni::OpenNI::initialize() != openni::STATUS_OK )
    ROS_ERROR("openni initial error: \n");
    //cerr << "OpenNI Initial Error: " << OpenNI::getExtendedError() << endl;
    return -1;

// Open Device
openni::Device  devDevice;
if( openni::ANY_DEVICE ) != openni::STATUS_OK )
    ROS_ERROR("Can't Open Device: \n");
    return -1;
ROS_INFO("device opened\n");

// NITE Stuff
nite::UserTracker userTracker;
nite::Status niteRc;
niteRc = userTracker.create(&devDevice);
if (niteRc != nite::STATUS_OK){
    printf("Couldn't create user tracker\n");
    return 3;

// Create color stream
openni::VideoStream vsColorStream;

if( vsColorStream.create( devDevice, openni::SENSOR_COLOR ) == openni::STATUS_OK )
    // set video mode
    openni::VideoMode mMode;
    //mMode.setResolution( 640, 480 );
    mMode.setResolution( 640, 480 );
    mMode.setFps( 30 );
    mMode.setPixelFormat( openni::PIXEL_FORMAT_RGB888 );

    if( vsColorStream.setVideoMode( mMode) != openni::STATUS_OK )
        ROS_INFO("Can't apply videomode\n");
        //cout << "Can't apply VideoMode: " << OpenNI::getExtendedError() << endl;

    // image registration
    if( devDevice.isImageRegistrationModeSupported( openni::IMAGE_REGISTRATION_DEPTH_TO_COLOR ) )
        devDevice.setImageRegistrationMode( openni::IMAGE_REGISTRATION_DEPTH_TO_COLOR );
    ROS_ERROR("Can't create color stream on device: ");// << OpenNI::getExtendedError() << endl;
    //cerr <<  "Can't create color stream on device: " << OpenNI::getExtendedError() << endl;
    return -1;

// Loop
printf("\nStart moving around to get detected...\n(PSI pose may be required for skeleton calibration, depending on the configuration)\n");

nite::UserTrackerFrameRef userTrackerFrame;

// create OpenCV Window
cv::namedWindow( "User Image",  CV_WINDOW_AUTOSIZE );

// start

// while (!wasKeyboardHit()){
ros::Rate rate(100.0);
while (nh.ok()){
    niteRc = userTracker.readFrame(&userTrackerFrame);
    if (niteRc != nite::STATUS_OK){
        printf("Get next frame failed\n");

    // get depth frame
    openni::VideoFrameRef depthFrame;
    depthFrame = userTrackerFrame.getDepthFrame();

    // get color frame
    openni::VideoFrameRef vfColorFrame;
    cv::Mat mImageBGR;
    if( vsColorStream.readFrame( &vfColorFrame ) == openni::STATUS_OK )
        // convert data to OpenCV format
        const cv::Mat mImageRGB( vfColorFrame.getHeight(), vfColorFrame.getWidth(), CV_8UC3, const_cast<void*>( vfColorFrame.getData() ) );
        // convert form RGB to BGR
        cv::cvtColor( mImageRGB, mImageBGR, CV_RGB2BGR );

    const nite::Array<nite::UserData>& users = userTrackerFrame.getUsers();
    for (int i = 0; i < users.getSize(); ++i) {

Complete file and fairly messy but to compare with yours:

You need to change in main the following two variables as I am saving the images and the skeleton: std::string path("/home/yiannis/datasets/recorded/scrap/"); std::string filename_skeleton("/home/yiannis/datasets/recorded/scrap/skeleton.csv");

Full file

* Software License Agreement (BSD License)
* Copyright (c) 2013, Johns Hopkins University
* All rights reserved.
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* copyright notice ...
edit flag offensive delete link more


Hi Yianni! Unfortunately it gives me the error:

[ERROR] [1430909562.362226973]: Can't Open Device:
Chaos gravatar image Chaos  ( 2015-05-06 05:54:23 -0500 )edit

Hi @Chaos Please use ATtags otherwise it was fluke I saw this comment. Does the "original/unmodified" openni2_tracker run fine? roslaunch openni2_tracker tracker.launch Bear in mind that the above is a snippet rather than the complete file. If you need the complete file I can post it.

Yianni gravatar image Yianni  ( 2015-05-06 07:15:34 -0500 )edit

@Yianni The original node works fine. Actually, starting from your suggestion, I'm working on it. I managed to make the node run by adding:


Unfortunately, it's still not working. Anyway, I'm still working on it

Chaos gravatar image Chaos  ( 2015-05-06 07:39:21 -0500 )edit

@Chaos Although it is not ideal to put not cleaned up code, I updated my answer with my complete file that runs fine, for me at least :). Feel free to use it. Make sure you read the bold text as you need to change a couple of variables.

Yianni gravatar image Yianni  ( 2015-05-06 09:41:28 -0500 )edit

@Yianni Thank you very much! Thanks to your code I managed the openni2_tracker node to publish also the captured video stream! Your help was extremely useful!

Chaos gravatar image Chaos  ( 2015-05-07 04:16:19 -0500 )edit

@Chaos Glad it worked :). Out of curiosity do you save as raw format the images or in some other format such as jpg/png?

Yianni gravatar image Yianni  ( 2015-05-08 10:07:59 -0500 )edit

@Yianni Actually I'm not saving the images. I'm just streaming the video over ros. Now I'm trying to publish also the point cloud.

Chaos gravatar image Chaos  ( 2015-05-08 10:43:05 -0500 )edit

@Chaos This might be of general interest. Do you have it anywhere in a public repository?

Yianni gravatar image Yianni  ( 2015-05-18 03:45:30 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools



Asked: 2015-03-04 10:51:51 -0500

Seen: 1,617 times

Last updated: May 06 '15