What is the correspondence between the parameters produced by camera_calibrator and camera_info?

edit

• Here is a link to repository code reading Yaml file github
• I have produced a video about it as well -> link

Camera_calibration package and camera_info message are tools used to calibrate and communicate about camera settings in a robotic system. They are indeed related, and understanding the relationship between them requires understanding what each of the parameters 1. Camera Matrix (K): This is the Intrinsic camera matrix, and it contains information about the intrinsic properties of the camera - that is, properties that are inherent to the camera and do not change regardless of the scene being viewed. This includes the focal length and the optical centers, represented as a 3x3 matrix.

1. Rectification Matrix (R): This is used for stereo cameras and is a rotation matrix that aligns the camera coordinate system to the ideal stereo image plane, so that epipolar lines in both stereo images are parallel. For a monocular camera, this matrix is typically the identity matrix, meaning it doesn't do anything.

2. Projection Matrix (P): This matrix includes information about the camera intrinsics (like the camera matrix), and extrinsics, which include rotation and translation vectors that describe the position and orientation of the camera in the world. In case of a monocular camera, the projection matrix is typically formed by augmenting the camera matrix with a zero column to make it a 3x4 matrix.

Regarding the confusion about P[1:3,1:3] = K, this condition might not always hold true, especially for cameras with non-zero skew or non-zero translation. The note "Normally, monocular cameras will also have R = the identity and P[1:3,1:3] = K" is a simplified statement, and exceptions can occur depending on the exact calibration of the camera.

In general, P is a 3x4 matrix, where P[1:3,1:3] is a 3x3 matrix that should correspond to K in the ideal case (no translation or skew), and P[1:3,4] corresponds to the translation of the camera. So, if your camera is translated, P[1:3,1:3] != K could happen.