Hi everyone,

I am currently working on a robotic project involving shape recognition using Kinect. I need to retrieve the x y z camera coordinates of an object (i.e. a cube). For now, I am working on easy task such as detecting a square using static image.

Kinect have to camera: rgb camera and a depth camera.

My idea is to do image processing (on RGB camera) to detect corners of the object in order to get x y screen coordinates. Using these x y screen coordinates, I save the value of the pixel located at x y screen coordinates of the depth camera. Using some geometric transformation, I can calculate x y z coordinates in camera coordinates system.

First question: Is that the right way to get that ?

Before working with the Image stream in "real-time image processing", I would like to work on 2 static images. Saving the RGB image onto jpeg is okay since openni programmed saving option by right clicking. My issue is how do I retrieve depth data ? I was thinking to do the same with the disparity image but I think that the value of the pixel does not contain the depth information but only a value used for visualization purpose. (what I want is value that represents distance in mm or cm or m).

So I thought of directly look into the uint8[] data array contained into the sensor_image/Image ROS message.

Second question: Is it possible to extract specific part of ROS message and save it ?

PS: I am using Linux Ubuntu 12.04 / ROS Hydro / Kinect first generation / openni ROS package / openCV / C++.

Thanks in advance !