Can not convert depth image to pointCloud2 using depth_image_proc

asked 2022-08-31 00:36:15 -0500

HossamAlzomor
5 ●3 ●4 ●6

updated 2022-09-01 23:42:54 -0500

1744 ●66 ●163 ●103 https://ravijoshi.xyz

I am trying to convert depth image generated by depth camera into pointCloud2 using depth_image_proc. I publish depth image and camera info from the my camera node and generated launcher for invoking depth_image_proc. But I can't show pointCloud2 in rviz.

Here's my launch file

<launch>
    <node pkg="nodelet" type="nodelet" name="nodelet_manager" args="manager" />

    <node pkg="nodelet" type="nodelet" name="nodelet1"
        args="load depth_image_proc/point_cloud_xyz nodelet_manager">
        <remap from="ela0895_info" to="/camera/depth/camera_info"/>
        <remap from="ela0895_depth" to="/camera/depth/image_rect_raw"/>
        <remap from="points" to="/camera/depth/points"/>
    </node>
</launch>

When I run the launcher I get these error messages

[ERROR] [1661931466.453595500]: Skipped loading plugin with error: XML Document 'c:\opt\ros\noetic\x64\share\test_nodelet/test_nodelet.xml' has no Root Element. This likely means the XML is malformed or missing..
[ERROR] [1661931466.458673700]: Skipped loading plugin with error: XML Document 'c:\opt\ros\noetic\x64\share\test_nodelet_topic_tools/test/test_nodelets.xml' has no Root Element. This likely means the XML is malformed or missing..

am I missing somthing?

edit retag flag offensive close merge delete

add a comment

answered 2022-08-31 03:33:39 -0500

ravijoshi

1744 ●66 ●163 ●103 https://ravijoshi.xyz

updated 2022-09-01 20:48:07 -0500

Just now, I tested it with an Intel RealSense D435 camera and found it working. Below are the steps I followed:

Start Intel RealSense D435 camera: roslaunch realsense2_camera rs_camera.launch
Invoke the launch file: roslaunch depth_to_pointcloud.launch

Below is the content of the launch file:

<launch>
  <node pkg="nodelet" type="nodelet" name="standalone_nodelet" args="manager" />
  <node pkg="nodelet" type="nodelet" name="point_cloud_xyzrgb" args="load depth_image_proc/point_cloud_xyz standalone_nodelet">
    <remap from="camera_info" to="/camera/depth/camera_info" />  
    <remap from="image_rect" to="/camera/depth/image_rect_raw" />
  </node>
</launch>

These are all the topics I can see at the terminal:

/camera/color/camera_info
/camera/color/image_raw
/camera/color/image_raw/compressed
/camera/color/image_raw/compressed/parameter_descriptions
/camera/color/image_raw/compressed/parameter_updates
/camera/color/image_raw/compressedDepth
/camera/color/image_raw/compressedDepth/parameter_descriptions
/camera/color/image_raw/compressedDepth/parameter_updates
/camera/color/image_raw/theora
/camera/color/image_raw/theora/parameter_descriptions
/camera/color/image_raw/theora/parameter_updates
/camera/color/metadata
/camera/depth/camera_info
/camera/depth/image_rect_raw
/camera/depth/image_rect_raw/compressed
/camera/depth/image_rect_raw/compressed/parameter_descriptions
/camera/depth/image_rect_raw/compressed/parameter_updates
/camera/depth/image_rect_raw/compressedDepth
/camera/depth/image_rect_raw/compressedDepth/parameter_descriptions
/camera/depth/image_rect_raw/compressedDepth/parameter_updates
/camera/depth/image_rect_raw/theora
/camera/depth/image_rect_raw/theora/parameter_descriptions
/camera/depth/image_rect_raw/theora/parameter_updates
/camera/depth/metadata
/camera/extrinsics/depth_to_color
/camera/realsense2_camera_manager/bond
/camera/rgb_camera/auto_exposure_roi/parameter_descriptions
/camera/rgb_camera/auto_exposure_roi/parameter_updates
/camera/rgb_camera/parameter_descriptions
/camera/rgb_camera/parameter_updates
/camera/stereo_module/auto_exposure_roi/parameter_descriptions
/camera/stereo_module/auto_exposure_roi/parameter_updates
/camera/stereo_module/parameter_descriptions
/camera/stereo_module/parameter_updates
/clicked_point
/diagnostics
/initialpose
/move_base_simple/goal
/points
/rosout
/rosout_agg
/standalone_nodelet/bond
/tf
/tf_static

The generated point cloud is visible in RViz as shown below:

image description

Update

The /camera/depth/camera_info topic looks like following:

$ rostopic echo -n 1 /camera/depth/camera_info
header: 
  seq: 0
  stamp: 
    secs: 1662082576
    nsecs: 680095911
  frame_id: "camera_depth_optical_frame"
height: 480
width: 848
distortion_model: "plumb_bob"
D: [0.0, 0.0, 0.0, 0.0, 0.0]
K: [425.5743713378906, 0.0, 421.3022766113281, 0.0, 425.5743713378906, 239.98716735839844, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [425.5743713378906, 0.0, 421.3022766113281, 0.0, 0.0, 425.5743713378906, 239.98716735839844, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi: 
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False
---

The /camera/depth/image_rect_raw looks like following:

$ rostopic echo -n 1 /camera/depth/image_rect_raw
header: 
  seq: 3
  stamp: 
    secs: 1662082604
    nsecs: 533638000
  frame_id: "camera_depth_optical_frame"
height: 480
width: 848
encoding: "16UC1"
is_bigendian: 0
step: 1696
data: [0, 255, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, text removed to keep the post short , 12, 7, 8, 7, 8, 7, 3, 7, 3, 7, 3, 7, 254, 6,0, 0, 0, 0, 0, 0, 0, 0, 0]
---

Important Point

The camera info belongs to the depth image. This is why, we can see that the frame_id, height, and width attributes are the same in both topics. If they are different, a point cloud can not be generated. Therefore we ... (more)

edit flag offensive delete link

Comments

I used it but It's not working for me , I can see a /points topic , but it seems to be empty I just changed the from name to match my camera topics. Is this every thing I need to do, just launch this launch file after running the node which publish the camera info and depth information? or is there anything else? I am running noetic on Windows10, is there any special procedures for Windows?

HossamAlzomor ( 2022-08-31 08:33:46 -0500 )edit

I updated the question with all the details. All you need is two topics, one is camera_info, and another is the corresponding depth image. Make sure both of these topics are publishing values.

ravijoshi ( 2022-08-31 22:08:09 -0500 )edit

camera info is published

header:
  seq: 1
  stamp:
    secs: 1662043280
    nsecs: 801370620
  frame_id: "Info"
height: 80
width: 260
distortion_model: "plumb_bob"
D: [0.0, 0.0, 0.0, 0.0, 0.0]
K: [0.012, 0.0, 130.0, 0.0, 0.012, 40.0, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [0.012, 0.0, 130.0, 0.0, 0.0, 0.012, 40.0, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi:
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False
---

and depth image is published

header:
  seq: 2710
  stamp:
    secs: 1662043359
    nsecs: 989972829
  frame_id: "Depth"
height: 80
width: 260
encoding: "16SC1"
is_bigendian: 0
step: 520
data: [ ....]

I can see two topics created by the launcher