Stereo triangulation of points
Hello!
I need to reconstruct 3d position of a tennis ball by using a stereo camera. I am detecting the ball center positons in the rectified images and pass them to the cv::triangulatePoints(P1, P2, points1, points2, points_out). For the P matrices I use the ones obtained from the camera_info of both cameras. The reconstructed 3d position is not correct - it moves a lot when the camera is rotated or moved. It looks ok only when the ball is close to the center of the images. I use the same code for a simulated stereo camera which has no distortion and it works properly so the error is somewhere in the projectiom matrices or detected center coordinates for the real images. I have compared the matrices in the camera_info when using the simulation and the real camera and they seem ok (the baseline is correct and the rotations also).
There is one thing I have noticed when comparing the images before and after rectification - the image is cropped and resized to the full frame dimensions (done automatically by ROS).
Is it properly reflected in the matrices of the camera_info message?
If not, I suppose I have to modify the detected ball center to get the position before the crop/resize?
Can I get the image without the crop/resize? How to do it?
Maybe there is some other problem?
Best, Patryk
Have you checked if there is a difference between the camera_info message of the rectified image and the original one? This would at least tell you, whether they are adapted at all.
Have you taken a look at the arToolkit package?