Frame of reference of depth sensors
Hello everyone,
I am working on a project to do the 3D reconstruction of the environment using ASUS Xtion pro and using openni2launch, openni2camera and rgbd_launch packages to get the data from the device drivers.
In rgbdlaunch, there is a file **kinectframes.launch** in which four frames - camerargbframe, cameradepthframe, camerargbopticalframe and cameradepthopticalframe, with different orientations are defined. And there is a frame camera_link in which all of these frames are transformed. You can see the static transform between these frames here.
My question is that when the point cloud is created by the device, which frame among the above mentioned frames is considered as reference? Also when the image (rgb and/or depth) is acquired by the device, which frame is the reference frame?
The orientation of the baselink or cameralink is different from camerargbopticalframe or cameradepthopticalframe as mentioned in REP-103. Hence my confusion.
Any guidance is appreciated. Thanks in advance !!
Asked by KamalDSOberoi on 2016-07-04 05:55:23 UTC
Answers
The reference frame should be listed in the frame_id field in the header of each message.
Asked by ahendrix on 2016-07-04 22:16:00 UTC
Comments
Hi. Thanks for your answer. But if we talk in more general sense than ROS, do you know if there is any predefined convention by the manufacturer (Microsoft or Asus) that a particular reference frame will be used to display the point cloud?
Asked by KamalDSOberoi on 2016-07-05 02:42:21 UTC
Most depth sensors capture depth as the distance along the ray from the pixel on the image plane, through the center of the image lens. This is a fairly standard pinhole camera model, but reporting depth instead of light intensity or color.
Asked by ahendrix on 2016-07-05 03:06:41 UTC
Some sensors (like the kinect and Asus) appear to have low distortion, or they correct for lens distortion in hardware, but otherwise they don't usually transform the data into a different reference frame.
Asked by ahendrix on 2016-07-05 03:08:07 UTC
Image coordinates correspond to the *_optical_frame, and I believe they are used for the captured rgb and depth images. Sensor-relative coordinates correspond to the camera_depth_frame and are used for the computed point cloud.
Asked by ahendrix on 2016-07-05 03:10:30 UTC
Note that the sensor sends a depth image over USB, and the ROS driver publishes that depth image in addition to converting it into a 3D point cloud and publishing that point cloud, too.
Asked by ahendrix on 2016-07-05 03:12:51 UTC
IMO there should be a reference frame in which the frames for depth and rgb cameras are fixed (static transform) and the point cloud is displayed. What do you think?
Asked by KamalDSOberoi on 2016-07-05 04:24:35 UTC
Comments