ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question
1

why the left 3x3 portion of Projection matrix(P) is different from the Intrinsic camera matrix(K)?

asked 2021-12-16 06:25:59 -0500

koughua gravatar image

Intrinsic camera matrix for the raw
(distorted) images. [fx 0 cx]
K = [ 0 fy cy]
[ 0 0 1]
Projects 3D points in the camera coordinate frame to 2D pixel coordinates using the focal lengths (fx, fy) and principal point (cx, cy).

Projection/camera matrix
[fx' 0 cx' Tx]
P = [ 0 fy' cy' Ty]
[ 0 0 1 0]
By convention, this matrix specifies the intrinsic (camera) matrix of the processed (rectified) image. That is, the left 3x3 portion is the normal camera intrinsic matrix for the rectified image. It projects 3D points in the camera coordinate frame to 2D pixel coordinates using the focal lengths (fx', fy') and principal point (cx', cy') - these may differ from the values in K.

given by http://docs.ros.org/en/noetic/api/sen...

inage width: 1920 image height:1080 canera nane: narrow stereo
cameramatrix: rows: 3 cols:3
data:[1963.58626 0. 978.96792 0. 1961.4452 548.41778 0. 0. 1.]
distortion_model: plumb bob
distortion coefficients: rOWS: 1 cols:5
data: [-0.554099, 0.343450,-0.000465. 0.001727, 0.000000]
rectification matrix: rows: 3 cols:3
data: [1. 0. 0. 0. 1. 0. 0.0. 1. ]
projection matrix rows. 3 cols:4
data: [1679.60999 0. 989.73115 0. 0.1875.7949 548.96323 0. 0. 0. 1. 0. ]

when i use the camera_calibration cameracalibrator.py to calibrate the camera, the result shows the left 3x3 portion of Projection matrix(P) is different from the Intrinsic camera matrix(K), why they are different?

how to caculate from Intrinsic camera matrix(K) to Projection matrix(P)?

edit retag flag offensive close merge delete

1 Answer

Sort by » oldest newest most voted
1

answered 2021-12-16 07:47:14 -0500

updated 2021-12-16 07:53:03 -0500

Have a look at this page, which is also linked in the sensor_msgs/CameraInfo comments: https://wiki.ros.org/image_pipeline/C...

Specifically this part:

4.1 Simple monocular rectification

In this case, there is a monocular device with distortion correction only. To transform a pixel in the input image into one in the output image, it is sent through the K - D - R - K' series of transformations. K - D gets to the normalized, undistorted image; the rotation R is the identity because we don't want to rotate the normalized undistorted image; and then K' converts back to pixel coordinates in the output image. In this case, since there is no need of in-image-plane translation, or scaling from the original image, so K = K' , and only the K and D elements of CameraInfo are needed. But since ROS Electric, the camera_calibration package does monocular calibration to get K' using OpenCV's getOptimalNewCameraMatrix() function with argument 'alpha'=0.0 that cause K' ≠ K.

Projection onto the output image is by the P matrix, which is formed from K' and the optional rotation and translation, in this case the identity and 0, respectively. So if K' = K, P is redundant information, and could be formed directly from K.

The important point here is that the left 3x3 portion of P is computed by getOptimalNewCameraMatrix() function with argument 'alpha'=0.0. If you undistort an image, it is no longer rectangular, and you'll get a pincushioning effect like this:

image description

Since the image has to be rectangular to be stored in memory, the remaining pixels are filled with a black color.

The getOptimalNewCameraMatrix() scales the camera matrix K and computes the new image size so that the undistorted (rectified) image either contains no invalid pixels (alpha = 0.0), or so that all original pixels are contained in the image (alpha = 1.0) or something in between. Also, the new image size for the rectified image has to be specified. The camera_calibration package uses alpha = 0 (i.e., crop the image) and output image size = input image size.

And if you're working with stereo camera pairs, it gets even more complicated (see the page I linked above), so in that case P can never be replaced with just using K.

Short rule of thumb: If you're working with rectified images, use P. If you're working with unrectified (raw) images, use K.

edit flag offensive delete link more

Comments

thats what I want, I got it ! Thank you !

koughua gravatar image koughua  ( 2021-12-16 07:54:09 -0500 )edit

Question Tools

1 follower

Stats

Asked: 2021-12-16 06:14:29 -0500

Seen: 1,307 times

Last updated: Dec 16 '21