How do the YAML Parameters relate to Pinhole-Model
I try to make sense of the YAML Matrixes below and so far know that
camera_matrix
holds intrinsics
distortion_coefficient
is vector for lens distortions and holds k and p parameters
projection_matrix
is for coordinate mapping world to camera.
But why is it the same as the camera matrix and what does the rectification matrix hold. According to opencv-docs this matrix does not exist. Could someone shed some light into this, I feel confused.
image_width: 640 //Region of Interest
image_height: 480
camera_name: depth_A70774V01752221A //@do Seriennummer anpassen
camera_matrix:
rows: 3
cols: 3
data: [4827.94, 0, 1223.5, 0, 4835.62, 1024.5, 0, 0, 1]
distortion_model: plumb_bob
distortion_coefficients: //Korrektur der Linsenfehler
rows: 1
cols: 5
data: [-0.41527, 0.31874, -0.00197, 0.00071, 0]
rectification_matrix:
rows: 3
cols: 3
data: [1, 0, 0, 0, 1, 0, 0, 0, 1]
projection_matrix:
rows: 3
cols: 4
data: [4827.94, 0, 1223.5, 0, 0, 4835.62, 1024.5, 0, 0, 0, 1, 0]
EDIT
So the projection_matrix
is equal to the camera_matrix
, just modified to comply with the rectification_matrix
. No Rectification means projection_matrix
equal to camera_matrix
, which would explain why both have the same entries, as the rectification matrix is the identity.
But why do the dimensions not fit and also the principal point c0 c1 is zero.
Edit
I have read through this image_proc but the rectification process still does not make sense to me. If I perform correction with respect to the distortion coeff, the intrinsics and the extrinsics matrixes, I have a perfectly fine image. Ok it reveals having a look at this. (so good!) I made a flowchart, could someone correct me if sth is wrong? ( the x,y,z is camera coord., XYZ is world coord., ud for undistorted)
Furthermore is this tatement right: The mapping from depth to 3D point is only possible with depth-frames because there I still have the info to reverse the normalisation (division by z), because the z-coordinate is the depth-entry itself. In RGB the z is missing thus no reversing like: x' * z = x(in camera)
Maybe this makes it a little easier to imagine:
CameraInfo.msg has some documentation for projection matrices that might be useful. (e.g.
/opt/ros/jade/share/sensor_msgs/msg/CameraInfo.msg
)