How to use depth_image_proc/register to convert a depth image from a tf to another?

asked 2014-07-30 11:11:53 -0500

Sergio MP gravatar image

updated 2014-08-01 05:37:51 -0500

Hello everybody! I'm working on ROS Hydro on Ubuntu 12.04. I have two simulated cameras: one depth camera and one RGB camera. They have a 4 cm horizontal offset between them, as specified on the urdf. I've successfully transformed the depth image into a point cloud using the depth_image_proc/point_cloud_xyzrgb nodelet as follows:

<!- Load processing nodelets for the depth camera -->
<launch>
     <arg name="cloud_input_ns" value="/some_namespace/cameras/" />
     <arg name="upm_ns"         value="/upm_perception"/>
     <arg name="cam_info_subfolder" value="$(arg upm_ns)/cameras"/> 

     <!-- Nodelet manager for this pipeline -->
    <node pkg="nodelet" type="nodelet" args="manager" name="depth_transforms_manager" output="screen"/>

     <!-- Convert to point cloud -->
    <node pkg="nodelet" type="nodelet" name="cloudify"
    args="load depth_image_proc/point_cloud_xyzrgb depth_transforms_manager --no-bond">

        <!-- Input: Rectified depth image, registered to the RGB camera. -->
        <remap from="depth_registered/image_rect" to="$(arg cam_info_subfolder)/scene_depth_cam_wf"/> 
        <!-- Input: Rectified color image. -->
        <remap from="rgb/image_rect_color" to="$(arg cam_info_subfolder)/scene_rgb_cam_wf"/> 
        <!-- Input: Camera calibration and metadata. --> 
        <remap from="rgb/camera_info" to="$(arg cam_info_subfolder)/scene_depth_camera_info"/>
        <!-- Output: XYZRGB point cloud. -->
        <remap from="depth_registered/points" to="$(arg cam_info_subfolder)/points"/>
    </node>
</launch>

But to do this I cheated: I manually changed the scene_rgb_cam_wf frame_id to the scene_depth_cam_wf frame_id, something that is not true (as I said before, they have a horizontal displacement of 4 cm). I did it because otherwise I got this error:

[ERROR] [1406735828.647852543, 4422.770000000]: Depth image frame id [/scene_depth_camera_box] doesn't match RGB image frame id [/scene_rgb_camera_box]

So, by doing this I got my depth image transformed into a PoinCloudXYZRGB, but with the colors displaced. SO, to correct this I read that the depth_image_proc has another nodelet that lets you transform a depth image into the RGB image frame_id, the depth_image_proc/register.

So, I undid the manual change of the RGB image's frame id and wrote this .launch file:

<!-- Load processing nodelets for the depth camera -->
<launch>

     <arg name="cloud_input_ns" value="some_namespace/cameras/" />
     <arg name="upm_ns"         value="/upm_perception"/>
<!--
     <arg name="cam_info_subfolder" value="$(arg upm_ns)/cam_info"/>     
-->
     <arg name="cam_info_subfolder" value="$(arg upm_ns)/cameras"/> 

     <!-- Nodelet manager for this pipeline -->
    <node pkg="nodelet" type="nodelet" args="manager" name="depth_transforms_manager" output="screen"/>

     <!-- Get the scene depth image transformed to the rgb camera frame -->
    <node pkg="nodelet" type="nodelet" name="transform_scenedepth2rgb"
    args="load depth_image_proc/register depth_transforms_manager --no-bond">

        <!-- Input: RGB camera calibration and metadata. (sensor_msgs/CameraInfo) -->
        <remap from="rgb/camera_info" to="$(arg cam_info_subfolder)/scene_rgb_camera_info"/> 
        <!-- Input: Depth camera calibration and metadata. (sensor_msgs/CameraInfo) -->
        <remap from="depth/camera_info" to="$(arg cam_info_subfolder)/scene_depth_camera_info"/> 
        <!-- Input: Rectified depth image. (sensor_msgs/Image) -->
        <remap from="depth/image_rect" to="$(arg cam_info_subfolder)/scene_depth_cam_wf"/>

        <!-- Output: Camera calibration and metadata. Same as rgb/camera_info but time-synced 
        to depth_registered/image_rect. (sensor_msgs/CameraInfo)   -->
        <remap from="depth_registered/camera_info" to="$(arg cam_info_subfolder)/scene_depth_camera_info_sync"/>
        <!-- Output: Reprojected depth image in the RGB camera frame. -->
        <remap from="depth_registered/image_rect" to="$(arg cam_info_subfolder)/scene_depth_cam_wf_2rgb"/> 
    </node>

     <!-- Convert to point cloud -->
    <node pkg="nodelet" type="nodelet" name="cloudify"
    args="load depth_image_proc/point_cloud_xyzrgb depth_transforms_manager --no-bond">

        <!-- Input: Rectified depth image, registered to the RGB camera. -->
        <remap from="depth_registered/image_rect" to="$(arg cam_info_subfolder)/scene_depth_cam_wf_2rgb"/> 
        <!-- Input: Rectified color image. -->
        <remap from="rgb/image_rect_color" to="$(arg cam_info_subfolder ...
(more)
edit retag flag offensive close merge delete

Comments

Have you solved this problem? I met the same problem...

xiaohuang gravatar imagexiaohuang ( 2017-07-13 10:00:30 -0500 )edit

me too! :) looking for a working launch file.

mojovski gravatar imagemojovski ( 2017-07-18 16:01:01 -0500 )edit

any luck now?

huangzonghao gravatar imagehuangzonghao ( 2018-07-27 14:03:29 -0500 )edit