Abstract

Following this tutorial, I’m trying to use ROS2 image_pipeline for point cloud creation from a stereo camera. https://jeffzzq.medium.com/ros2-image-pipeline-tutorial-3b18903e7329

Hardware Setup

ROS2 Humble on Ubuntu 22.04 and two Logitech c922 pro webcams. After fixing a permission problem by adding the user to the "video" group, the cameras are currently accessible via index 0 and 2.

Building packages

The only notice here, as mentioned in the original tutorial, is to use the corresponding branch (Foxy, Humble, etc.) for imagepipeline and imaglecommon packages.

Stereo camera calibration

In the original tutorial the indexes of the cameras are 5 and 6, but in my case: Right camera - 0 Left camera - 2

The link to 6x8 grid actually refers to 9x6 grid. Square parameter is the size of the square. On my A4 sheet is 2.4 mm

ros2 run opencv_cam opencv_cam_main \
 --ros-args --param index:=2 \
 --remap __ns:=/left \
 --remap /left/image_raw:=/left/image_rect

ros2 run opencv_cam opencv_cam_main \
 --ros-args --param index:=0 \
 --remap __ns:=/right \
 --remap /right/image_raw:=/right/image_rect

ros2 run camera_calibration cameracalibrator \
 --size=9x6 \
 --square=0.024 \
 --approximate=0.3 \
 --no-service-check \
 --ros-args --remap /left:=/left/image_rect \
 --remap /right:=/right/image_rect

Calibration screens https://ctrl.vi/i/nQ8xrbvPB https://ctrl.vi/i/rCrtCFXvH

The save button saves the samples and the configuration parameters in /tmp/calibrationdata.tar.gz tarball.

This is the content of the camera-info-left.ini The camera-info-right.ini file has the same structure. Is it correct?

# oST version 5.0 parameters


[image]

width
2304

height
1536

[narrow_stereo/left]

camera matrix
1868.566245 0.000000 1159.724198
0.000000 1858.163692 667.343211
0.000000 0.000000 1.000000

distortion
0.054167 -0.126659 -0.002232 -0.005700 0.000000

rectification
0.988207 0.014404 -0.152444
-0.016510 0.999785 -0.012560
0.152230 0.014929 0.988232

projection
2203.697874 0.000000 1581.168335 0.000000
0.000000 2203.697874 707.677567 0.000000
0.000000 0.000000 1.000000 0.000000

Create a Disparity Map

Update the both Camera nodes to use the --param --camerainfopath:={left|right}-camera.ini

ros2 run opencv_cam opencv_cam_main \
 --ros-args --param index:=2 \
 --remap __ns:=/left \
 --remap /left/image_raw:=/left/image_rect \
--param camera_info_path:=camera-info-left.ini

ros2 run opencv_cam opencv_cam_main \
 --ros-args --param index:=0 \
 --remap __ns:=/right \
 --remap /right/image_raw:=/right/image_rect \
--param camera_info_path:=camera-info-right.ini

The disparity node should create DisparityImage on /disparity topic.

ros2 run stereo_image_proc disparity_node --ros-args --params-file disparity-params.yaml

This is the content of disparity-params.yaml Could some tell is it correct or not?

disparity_node:
  ros__parameters:
    P1: 200.0
    P2: 400.0
    approximate_sync: false
    correlation_window_size: 15
    disp12_max_diff: 0
    disparity_range: 64
    full_dp: false
    min_disparity: 0
    prefilter_cap: 31
    prefilter_size: 9
    qos_overrides./disparity.publisher.depth: 1
    qos_overrides./disparity.publisher.history: keep_last
    qos_overrides./disparity.publisher.reliability: reliable
    qos_overrides./left/camera_info.subscription.depth: 5
    qos_overrides./left/camera_info.subscription.history: keep_last
    qos_overrides./left/camera_info.subscription.reliability: best_effort
    qos_overrides./left/image_rect.subscription.depth: 5
    qos_overrides./left/image_rect.subscription.history: keep_last
    qos_overrides./left/image_rect.subscription.reliability: best_effort
    qos_overrides./parameter_events.publisher.depth: 1000
    qos_overrides./parameter_events.publisher.durability: volatile
    qos_overrides./parameter_events.publisher.history: keep_last
    qos_overrides./parameter_events.publisher.reliability: reliable
    qos_overrides./right/camera_info.subscription.depth: 5
    qos_overrides./right/camera_info.subscription.history: keep_last
    qos_overrides./right/camera_info.subscription.reliability: best_effort
    qos_overrides./right/image_rect.subscription.depth: 5
    qos_overrides./right/image_rect.subscription.history: keep_last
    qos_overrides./right/image_rect.subscription.reliability: best_effort
    queue_size: 5
    speckle_range: 4
    speckle_size: 100
    stereo_algorithm: 0
    texture_threshold: 10
    uniqueness_ratio: 15.0
    use_sim_time: false
    use_system_default_qos: false

There are messages on

/left/camera_info
/left/image_rect
/right/camera_info
/right/image_rect

But not on /disparity topic.

Here is the graph https://ctrl.vi/i/Yu4LFTJAD

Now, I should view the DisparityImage using the disparity_view node:

ros2 run image_view disparity_view --ros-args --remap image:=/disparity

Nothing shown, probably because nothing is published on /disparity topic

According to the comment in the original tutorial, image_proc provides really rectified image and improves the disparity map.

So, I added this node and got rectified images, but still nothing in /disparity topic.

ros2 run opencv_cam opencv_cam_main \
 --ros-args --param index:=2 \
 --remap __ns:=/left \
--param camera_info_path:=camera-info-left.ini

ros2 run opencv_cam opencv_cam_main \
 --ros-args --param index:=0 \
 --remap __ns:=/right \
--param camera_info_path:=camera-info-right.ini

ros2 run image_proc image_proc \
--ros-args --remap __ns:=/left \
 --remap /left/image:=/left/image_color

ros2 run image_proc image_proc \
--ros-args --remap __ns:=/right \
 --remap /right/image:=/right/image_color

No changes in disparitynode and pointcloud_node. Here is the graph https://paste.pics/JY08E

Troubleshooting

The disparity_node takes 4 input and 1 configuration file (disparity-params.yaml) to provide /disparity topic.

So, let's check all the inputs.

ros2 topic echo --no-arr  /left/camera_info

ros2 topic echo --no-arr  /right/camera_info

ros2 run image_view image_view --ros-args --remap /image:=/left/image_rect

Shows the left camera image

ros2 run image_view image_view --ros-args --remap /image:=/right/image_rect

Shows the right camera image

The check the correctness of the disparity-params.yaml I use ROS2 params. For example

ros2 param get /disparity_node P1

returns: Double value is: 200.0

All inputs of the disparity_node look correct, but there are no any messages on the /disparity topic.