ROS2 image pipeline for point cloud creation using stereo camera - no disparity

edit

Abstract

Following this tutorial, I’m trying to use ROS2 image_pipeline for point cloud creation from a stereo camera. https://jeffzzq.medium.com/ros2-image...

Hardware Setup

ROS2 Humble on Ubuntu 22.04 and two Logitech c922 pro webcams. After fixing a permission problem by adding the user to the "video" group, the cameras are currently accessible via index 0 and 2.

Building packages

The only notice here, as mentioned in the original tutorial, is to use the corresponding branch (Foxy, Humble, etc.) for image_pipeline and imagle_common packages.

Stereo camera calibration

In the original tutorial the indexes of the cameras are 5 and 6, but in my case: Right camera - 0 Left camera - 2

The link to 6x8 grid actually refers to 9x6 grid. Square parameter is the size of the square. On my A4 sheet is 2.4 mm

ros2 run opencv_cam opencv_cam_main \
 --ros-args --param index:=2 \
 --remap __ns:=/left \
 --remap /left/image_raw:=/left/image_rect

ros2 run opencv_cam opencv_cam_main \
 --ros-args --param index:=0 \
 --remap __ns:=/right \
 --remap /right/image_raw:=/right/image_rect

ros2 run camera_calibration cameracalibrator \
 --size=9x6 \
 --square=0.024 \
 --approximate=0.3 \
 --no-service-check \
 --ros-args --remap /left:=/left/image_rect \
 --remap /right:=/right/image_rect

Calibration screens https://ctrl.vi/i/nQ8xrbvPB https://ctrl.vi/i/rCrtCFXvH

The save button saves the samples and the configuration parameters in /tmp/calibrationdata.tar.gz tarball.

This is the content of the camera-info-left.ini The camera-info-right.ini file has the same structure. Is it correct?

# oST version 5.0 parameters


[image]

width
2304

height
1536

[narrow_stereo/left]

camera matrix
1868.566245 0.000000 1159.724198
0.000000 1858.163692 667.343211
0.000000 0.000000 1.000000

distortion
0.054167 -0.126659 -0.002232 -0.005700 0.000000

rectification
0.988207 0.014404 -0.152444
-0.016510 0.999785 -0.012560
0.152230 0.014929 0.988232

projection
2203.697874 0.000000 1581.168335 0.000000
0.000000 2203.697874 707.677567 0.000000
0.000000 0.000000 1.000000 0.000000

Create a Disparity Map

Update the both Camera nodes to use the --param --camera_info_path:={left|right}-camera.ini

ros2 run opencv_cam opencv_cam_main \
 --ros-args --param index:=2 \
 --remap __ns:=/left \
 --remap /left/image_raw:=/left/image_rect \
--param camera_info_path:=camera-info-left.ini

ros2 run opencv_cam opencv_cam_main \
 --ros-args --param index:=0 \
 --remap __ns:=/right \
 --remap /right/image_raw:=/right/image_rect \
--param camera_info_path:=camera-info-right.ini

The disparity node should create DisparityImage on /disparity topic.

ros2 run stereo_image_proc disparity_node --ros-args --params-file disparity-params.yaml

This is the content of disparity-params.yaml Could some tell is it correct or not?

disparity_node:
  ros__parameters:
    P1: 200.0
    P2: 400.0
    approximate_sync: false
    correlation_window_size: 15
    disp12_max_diff: 0
    disparity_range: 64
    full_dp: false
    min_disparity: 0
    prefilter_cap: 31
    prefilter_size: 9
    qos_overrides./disparity.publisher.depth: 1
    qos_overrides./disparity.publisher.history: keep_last
    qos_overrides./disparity.publisher.reliability: reliable
    qos_overrides./left/camera_info.subscription.depth: 5
    qos_overrides./left/camera_info.subscription.history: keep_last
    qos_overrides./left/camera_info.subscription.reliability: best_effort
    qos_overrides./left/image_rect.subscription.depth: 5
    qos_overrides./left/image_rect.subscription.history: keep_last
    qos_overrides./left/image_rect.subscription.reliability: best_effort
    qos_overrides./parameter_events.publisher.depth: 1000
    qos_overrides./parameter_events.publisher ...