How do the YAML Parameters relate to Pinhole-Model

I try to make sense of the YAML Matrixes below and so far know that
camera_matrix holds intrinsics
distortion_coefficient is vector for lens distortions and holds k and p parameters projection_matrix is for coordinate mapping world to camera. But why is it the same as the camera matrix and what does the rectification matrix hold. According to opencv-docs this matrix does not exist. Could someone shed some light into this, I feel confused.

image_width: 640 //Region of Interest
image_height: 480
camera_name: depth_A70774V01752221A //@do Seriennummer anpassen
camera_matrix:
rows: 3
cols: 3
data: [4827.94, 0, 1223.5, 0, 4835.62, 1024.5, 0, 0, 1]
distortion_model: plumb_bob
distortion_coefficients: //Korrektur der Linsenfehler
rows: 1
cols: 5
data: [-0.41527, 0.31874, -0.00197, 0.00071, 0]
rectification_matrix:
rows: 3
cols: 3
data: [1, 0, 0, 0, 1, 0, 0, 0, 1]
projection_matrix:
rows: 3
cols: 4
data: [4827.94, 0, 1223.5, 0, 0, 4835.62, 1024.5, 0, 0, 0, 1, 0]


EDIT So the projection_matrix is equal to the camera_matrix, just modified to comply with the rectification_matrix. No Rectification means projection_matrix equal to camera_matrix, which would explain why both have the same entries, as the rectification matrix is the identity.
But why do the dimensions not fit and also the principal point c0 c1 is zero.

Edit
I have read through this image_proc but the rectification process still does not make sense to me. If I perform correction with respect to the distortion coeff, the intrinsics and the extrinsics matrixes, I have a perfectly fine image. Ok it reveals having a look at this. (so good!) I made a flowchart, could someone correct me if sth is wrong? ( the x,y,z is camera coord., XYZ is world coord., ud for undistorted)

Furthermore is this tatement right: The mapping from depth to 3D point is only possible with depth-frames because there I still have the info to reverse the normalisation (division by z), because the z-coordinate is the depth-entry itself. In RGB the z is missing thus no reversing like: x' * z = x(in camera)
Maybe this makes it a little easier to imagine:

edit retag close merge delete

1

CameraInfo.msg has some documentation for projection matrices that might be useful. (e.g. /opt/ros/jade/share/sensor_msgs/msg/CameraInfo.msg)

( 2016-02-17 11:22:34 -0500 )edit

Sort by » oldest newest most voted

The projection matrix contains the intrinsic values for the rectified image. When you rectify your image (remove the distortion), you (in general) also modify the intrinsic calibration matrix (e.g. because you remove pixels at the edge of the image). So if your subscribe to the raw image, you need to use the coefficients from the camera_matrix, if you subscribe to the rect image, you use the projection_matrix.

The projection matrix can also be used in a Stereo Setup. It then contains not only the intrinsic calibration, but also the transformation between the two cameras.

more

So what your saying is the distortion coefficients modify the camera_matrix leading to the projection_matrix. In my case both are the same, which then must be false. How to get the projection matrix?

( 2016-05-08 11:40:36 -0500 )edit

I read this opposing your answer. Monocular case: *In this case, since there is no rotation, translation, or scaling from the original image, K = K' , and only the K and D elements of CameraInfo are needed.

( 2016-05-08 12:05:18 -0500 )edit

The difference between both matrices comes from the translation of the stereo setup you mentioned. The second paragraph is accepted.

( 2016-05-08 12:06:45 -0500 )edit