ROS stereo camera calibration issues

I am currently trying to calibrate my stereo pinhole cameras. I am using the ros-noetic-camera-calibration package, a checkerboard with a 9x9 size (for 7x7 points), tried in different rooms with good lighting.

I have tried to calibrate the cameras 8 times now, all with bad outputs. I make sure every bar (x, y, size, skew) is at least green before clicking the calibrate button. The output vary between one camera being zoomed in where I can barely see 1/32 of the original image and is rotated with the other camera being just black. Sometimes the cameras become super warped where it isn’t even recognizable that it is a room.

Looking at the output parameters I can see some numbers being around 1 and some number going over 1000. It seems like some parameters are causing the bad calibration.

Is this behavior normal, should I just try more times or is there something I am doing wrong here?

I have tried a full day of calibration with no luck, any suggestions will be appreciated!

Asked by Tim2000 on 2023-03-22 10:46:11 UTC

Comments

Answers

Are you using checkerboard with orientation marker? One thing may be that each of your cameras detect the checkerboard in different orientations (this is especially true for square targets, since it is more symmetric), so it finds wrong point pairs. The simple thing you can try is to print rectangular calibration pattern and see if it works better.

Asked by dmonkoff on 2023-03-23 07:47:35 UTC

Comments

When viewing the checkboard recognition, I can see the points and lines lining up with the other camera. I have also tried with a different 8x6 and 5x8 board, but the results are still useless regardless. I have found that the problem is either with the stereoCalibrate function or the rectification used with the camera_calibrate. The distortion coefficient and matrixes are correct since I tested these myself.

Asked by Tim2000 on 2023-03-23 07:52:22 UTC

Do you use undistorted images? If you use cheap cameras, the undistorted ROI can be quite small, and the checkerboard should be inside that undistorted ROI of both cameras. Sadly, I did not try camera calibration package myself, only directly in openCV, so I don't know whether ROS takes care of that

Asked by dmonkoff on 2023-03-23 08:07:35 UTC

No, I do not use undistorted images. After the ROS camera calibration has finished, it gives you all the parameters together with a preview of what your new image will look like. I took only the camera matrix and distortion coefficients from it to test for myself if the undistortion is done correctly, which it is. The problem lays with the stereoCalibration and rectification.

Asked by Tim2000 on 2023-03-23 08:10:37 UTC

So did you check the image itself after single camera calibration? Again, distortions outside ROI can be quite bad, so you need to take images for stereo calib inside the ROI. Just an example how bad it can be https://imgur.com/a/wngA10J

Asked by dmonkoff on 2023-03-23 08:33:41 UTC