Robotics StackExchange | Archived questions

Stereo camera standards in ROS

Hi everyone,

I am trying to use stereo cameras to use RTABMap for 3D mapping. The problem is that the driver is horrible. I have found that:

Header header

  Camera information for the left camera
  sensor_msgs/CameraInfo left_info

   # Camera information for the right camera
   sensor_msgs/CameraInfo right_info

   # Disparity-to-depth mapping matrix in 4x4 row-major format. Use this
   # matrix together with the  Reconstruct3D class from libvisiontransfer
   # to transform a disparity map to 3D points.

   float64[16] Q

   # Translation vector between the coordinate systems of the left and    
   # right camera.    
   float64[3] T_left_right

  # Rotation matrix between the coordinate systems of the left and right    
  # camera in 3x3 row-major format.    
  float64[9] R_left_right

So clearly I have to fix their driver to be able to use it... I don't know the standards as to stereo cameras in ROS so I have the following questions:

  1. The StereoCameraInfo message contains a translation vector and a rotation 3x3 row major matrix. How do I apply these to provide the necessary transforms for ROS?
  2. The camera info message has the matrix P where Txleft = 0 and Txright = -fx' * baseline. This is all 0 in their topics... Would Tx be equal to -T[0] in their message?
  3. The translation vector in their message also provides a translation for y and z, ROS assumes these to be 0 in stereo cameras. Would I include these in the P matrix?
  4. Should the rotation matrix in their message be used on the broadcast publisher?
  5. Is the translation and rotation in their message in the right frame relevant to the left frame where left is assumed to be 0 0 0 0 0 0?
  6. I read in the camerainfo message that the coordinate frame should contain +x to the right, +y downward, and +z into the image... The driver has a parameter roscoordinatesystem of which it uses ENU frame (RHR). Since it only uses a single frame for both images, should I keep this as true and then publish the transforms correct in my to be written publisher.

Furthermore, I asked a question on their GIT about the camera info to which they said they made their own since there is no support for stereo in ROS. The CameraInfo message documentation literally points to stereo specific variables and standards. So it appears to me that I am required to fix this driver myself, this is why I am asking for assistance on the ROS standards and conventions to stereo imagery.

Thank you, Grant. :)

EDIT an example message is

---
header: 
  seq: 17
  stamp: 
    secs: 1564101175
    nsecs: 102466534
  frame_id: "nerian_cam"
left_info: 
  header: 
    seq: 121
    stamp: 
      secs: 1564101175
      nsecs: 102466534
    frame_id: "nerian_cam"
  height: 592
  width: 800
  distortion_model: "plumb_bob"
  D: [-0.6593078062972767, -1.6450547223209395, -0.04670625772500081, 0.0019792053730460528, 21.998118625719002]
  K: [1168.907670056606, 0.0, 408.94118260726344, 0.0, 1164.592984644571, 348.0267389668725, 0.0, 0.0, 1.0]
  R: [0.9999020284351581, -0.007200111904734488, 0.012003829381314148, 0.007180239781997346, 0.9999727805165786, 0.0016977581145578389, -0.012015726691691313, -0.0016114014092786255, 0.9999265101484052]
  P: [1124.8368436562992, 0.0, 388.9723472595215, 0.0, 0.0, 1124.8368436562992, 330.9644031524658, 0.0, 0.0, 0.0, 1.0, 0.0]
  binning_x: 1
  binning_y: 1
  roi: 
    x_offset: 0
    y_offset: 0
    height: 0
    width: 0
    do_rectify: False
right_info: 
  header: 
    seq: 121
    stamp: 
      secs: 1564101175
      nsecs: 102466534
    frame_id: "nerian_cam"
  height: 592
  width: 800
  distortion_model: "plumb_bob"
  D: [-0.5588013819794537, 1.2027859529979525, -0.016566840464484058, -0.0016258727363515644, -0.6880879828159225]
  K: [1174.9490079100433, 0.0, 404.1205345050226, 0.0, 1181.3882142749992, 346.4888744060625, 0.0, 0.0, 1.0]
  R: [0.999805451901642, -0.012726882443029455, 0.015069333463518485, 0.01275180025738805, 0.9999174777562336, -0.0015586108582770159, -0.015048253651141309, 0.0017504687638372058, 0.999885236375234]
  P: [1124.8368436562992, 0.0, 388.9723472595215, 111.01827578491618, 0.0, 1124.8368436562992, 330.9644031524658, 0.0, 0.0, 0.0, 1.0, 0.0]
  binning_x: 1
  binning_y: 1
  roi: 
    x_offset: 0
    y_offset: 0
    height: 0
    width: 0
    do_rectify: False
Q: [1.0, 0.0, 0.0, -388.97235107421875, 0.0, 1.0, 0.0, -330.96441650390625, 0.0, 0.0, 0.0, 1124.8367919921875, 0.0, 0.0, -10.131997108459473, 0.0]
T_left_right: [0.09867802429877996, -0.0012561079883815739, 0.0014873014053397623]
R_left_right: [0.9999798760835412, 0.00557699080046342, -0.003024004225719713, -0.0055670214721956855, 0.9999790747889804, 0.003295186808497701, 0.003042318174309927, -0.0032782857999770943, 0.9999899985211552]
---

Asked by PG_GrantDare on 2019-07-25 18:30:02 UTC

Comments

Answers