Erroneous stereo reprojection

edit

I am using a camera pair from Aptina, producing 1920x1092 px images. I created a ROS node for them and used the stereo_image_proc/camera_calibration node to create a calibration. By now everything looks fine, the calibration looks good. Then I have the following tool chain for left and right camera respectively:

• aptina (delivers left/image_raw and left/camera_info)
• bit_converter (node to convert to 8bit)
• image_proc/rectify
• image_proc/crop_decimate (cropping to 1920x880 for some project reason and reducing the resolution by a factor of 8/8 to 240x110px)

Having the same for both sides I use - stereo_image_proc/disparity

And here the problems occur. I have the following left/camera_info:

header: 
  seq: 11793
  stamp: 
    secs: 1535645408
    nsecs: 924607110
  frame_id: "camera_102_left"
height: 1092
width: 1920
distortion_model: "plumb_bob"
D: [-0.009216, -0.092237, 0.009712, -0.010848, 0.0]
K: [1843.04496, 0.0, 920.519756, 0.0, 1842.589347, 578.078866, 0.0, 0.0, 1.0]
R: [0.999581, -0.027793, 0.008056, 0.027725, 0.999579, 0.008503, -0.008289, -0.008276, 0.999931]
P: [1958.152785, 0.0, 905.844528, 0.0, 0.0, 1958.152785, 565.23769, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi: 
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False

left/image_raw:

header: 
  seq: 1
  stamp: 
    secs: 1535645468
    nsecs: 953573126
  frame_id: "camera_102_left"
height: 1092
width: 1920
encoding: "bayer_grbg16"
is_bigendian: 0
step: 3840
data: [...]

And after the cropping left/cropped_info:

header: 
  seq: 16235
  stamp: 
    secs: 1535645708
    nsecs: 867018980
  frame_id: "camera_102_left"
height: 1092
width: 1920
distortion_model: "plumb_bob"
D: [-0.009216, -0.092237, 0.009712, -0.010848, 0.0]
K: [1843.04496, 0.0, 920.519756, 0.0, 1842.589347, 578.078866, 0.0, 0.0, 1.0]
R: [0.999581, -0.027793, 0.008056, 0.027725, 0.999579, 0.008503, -0.008289, -0.008276, 0.999931]
P: [1958.152785, 0.0, 905.844528, 0.0, 0.0, 1958.152785, 565.23769, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 8
binning_y: 8
roi: 
  x_offset: 0
  y_offset: 0
  height: 880
  width: 1920
  do_rectify: True

and left/image_cropped

header: 
  seq: 17126
  stamp: 
    secs: 1535645768
    nsecs: 963487656
  frame_id: "camera_102_left"
height: 110
width: 240
encoding: "bgr8"
is_bigendian: 0
step: 720
data: [...]

The resulting disparity_info looks:

header: seq: 0 stamp: secs: 1535646144 nsecs: 380223275 frame_id: "camera_102_left" image: header: seq: 1677 stamp: secs: 1535646144 nsecs: 380223275 frame_id: "camera_102_left" height: 110 width: 240 encoding: "32FC1" is_bigendian: 0 step: 960 data: [...] f: 244.769104004 T: 0.175583317876 valid_window: x_offset: 70 y_offset: 7 height: 4294967281 width: 4294967218 do_rectify: False min_disparity: 0.0 max_disparity: 63.0 delta_d: 0.0625

And here are my questions/problems: 1. The stereo_image_proc/disparity output seems extremely slow, it produces at most one image per minute. 2. The stereo_msgs/DisparityImage shows very interesting valid_window values. How are these produced? Are these the reason for the slow algorithm? They seem to be autogenerated somewhere, but I can't find out where.

1. As last step I try to convert the stereo_msgs/DisparityImage into ...