# Does OpenCV's undistortPoints also rectify them?

I'm trying to determine the distance between two object by using two cameras (OCV 2.3.1, MSVC++, Win7) but can't calculate the object's objectPoints. I think this is because the image points aren't being rectified before their disparity is calculated.

I. WHAT I DO FIRST Step 1. Calibrate each camera by itself

int numSquares = numCornersHor * numCornersVer;
Size board_sz = Size(numCornersHor, numCornersVer);

Mat cameraMatrix = Mat(3, 3, CV_32FC1);
Mat distCoeffs;

vector<Mat> rvecs, tvecs;

cameraMatrix.ptr<float>(0)[0] = 1;
cameraMatrix.ptr<float>(1)[1] = 1;

calibrateCamera(object_points, image_points, image.size(),
cameraMatrix, distCoeffs, rvecs, tvecs);


Step 2. Calibrate the cameras to together.
int numCornersHor = 4; int numCornersVer = 3; const float squareSize = 1.75;

Size imageSize = Size(numCornersHor, numCornersVer);
int numSquares = numCornersHor * numCornersVer;

for(int i = 0; i < pairs; i++ )
{
for( int j = 0; j < imageSize.height; j++ )
{
for( int k = 0; k < imageSize.width; k++ )
{
objectPoints[i].push_back(Point3f(j*squareSize, k*squareSize, 0));
}
}
}

Mat R, T, E, F;

rms = stereoCalibrate(
objectPoints,
imagePoints[0],     imagePoints[1],
cameraMatrix[0],    distCoeffs[0],
cameraMatrix[1],    distCoeffs[1],
imageSize,
R,  T,  E,  F,
TermCriteria(CV_TERMCRIT_ITER+CV_TERMCRIT_EPS, 100, 1e-5),
CV_CALIB_FIX_ASPECT_RATIO +
CV_CALIB_ZERO_TANGENT_DIST +
CV_CALIB_SAME_FOCAL_LENGTH +
CV_CALIB_RATIONAL_MODEL +
CV_CALIB_FIX_K3 +   CV_CALIB_FIX_K4 +   CV_CALIB_FIX_K5
);


Step 3. Create the rectification data stereoRectify( cameraMatrix[0], cameraMatrix[1], distCoeffs[0], distCoeffs[1], imageSize, R, T, RC1, RC2, //RC1: Rotation matrix Camera 1 PC1, PC2, Q, CALIB_ZERO_DISPARITY, 1,
imageSize);

II. WHAT I BELIEVE Goal: I'm trying undistort and rectify the image points of one object in the image from camera 1 and the image from camera 2 (I do this process twice: once while the clay pigeon's on the launcher and once one frame before the clay pigeon disintegrates)

Method: I believe that I don't need to use initUndistortRectifyMap then Remap but can instead just use undistortPoints. I think undistortPoints undistorts the points of interest and rectifies them.

III. WHAT I DO SECOND You can ignore this if my beliefs aren't correct.

undistortPoints(launcherC1, launcherC1Undistorted, cameraMatrixC1, distCoeffsC1, R1, P1);
undistortPoints(launcherC2, launcherC2Undistorted, cameraMatrixC2, distCoeffsC2, R2, P2);

undistortPoints(clayPigeonC1, clayPigeonC1Undistorted, cameraMatrix1, distCoeffs1, R1, P1);
undistortPoints(clayPigeonC2, clayPigeonC2Undistorted, cameraMatrix2, distCoeffs2, R2, P2);


The input and output arrays for undistortPoints (launcherC1, launcherC1Undistorted, ... clayPigeonC2, clayPigeonC2Undistorted) are vectors of Point2f objects.

IV. DISCREPANCY BETWEEN BELIEF AND REALITY After all undistortPoints functions are run, 1. launcherC1Undisorted.y does not equal launcherC2Undistorted.y 2. clayPigeonC1Undistorted.y does not equal clayPigeonC2Undistorted.y.
They are up to 30% different.

V. QUESTIONS Q1: In addition to undistorting them does undistortPoints also rectify points? Q1.1_yes. Are the values of y supposed to be equal after rectification? Q1.1.1_yes Can you tell from the code I've included what I'm doing wrong so that they don't? Q1_no If undistortPoints doesn't rectify the points then how do I rectify them?

Thank you for any assistance. Charles++

edit retag close merge delete

Sort by » oldest newest most voted

Hi Charles,

Your question doesn't have much to do with ROS, maybe you should ask it on an OpenCV forum, but as long as you made the effort to post it so nicely let my try to answer it.

As far as I know undistortPoints do not rectify the points. The correct approach would be to call initUndistortRectifyMap and then use Remap.

If instead of remapping the whole image what you need is to undistort and rectify only some image points, then what you can do is use manually the maps (mapx and mapy) generated by initUndistortRectifyMap. Something like this (assuming that point1 is a point on the original image):

undistortedRectifiedPoint.x = mapx[point1.x, point1.y]
undistortedRectifiedPoint.y = mapy[point1.x, point1.y]


I hope this helps.

Best regards, Martin.

more

I can't read mapx & mapy. I try Mat mapxC1; mapxC1.create(sizeImage,1); initUndistortRectifyMap(cameraMatrix1, distCoeffs1, R1, camMat1UndistRect, sizeImage, 0, /* ???*/ mapxC1, mapyC1);

( 2012-05-15 09:15:44 -0600 )edit

vector<Point2f> point1Cam1UndistortedRectified; point1Cam1UndistortedRectified.at(0).x = mapxC1[PointC1x, PointC1y]; point1Cam1UndistortedRectified.at(0).y = mapxC1[PointC1x, PointC1y]; but MSVC++ says not operator "[]" matches these operands right after mapxC1 in the last two lines. Know why?

( 2012-05-15 09:22:25 -0600 )edit

Hi CharlesM, Actually the syntax "mapxC1[PointC1x, PointC1y]" is not correct, sorry I misled you with my pseudo-code. How about mapxC1.at(PointC1x,PointC1y)?

( 2012-05-15 16:51:28 -0600 )edit

You can find more info on how to access OpenCV matrices efficiently here: http://docs.opencv.org/doc/tutorials/core/how_to_scan_images/how_to_scan_images.html#howtoscanimagesopencv

( 2012-05-15 16:58:40 -0600 )edit

Hi Martin, Thank you so much for your help! I know I'll reach the answer being assisted by an OCV expert. You told me how to put exactly the data I want into maps but now I'm frustrated since I still can't get that data. I tried your suggestion and link. Results:

( 2012-05-16 05:17:32 -0600 )edit

1.) MSVC++ says about mapxC1.at(PointC1x,PointC1y) that no overloaded functn matches argumnt list and that there are too many args in functn call.
2.) I don't know OCV or C++ well enough to make enough sense of the link you sent to get at a map's data

( 2012-05-16 05:17:45 -0600 )edit

What I've done: PointC1x & PointC1y are floats that are the x & y of Keypoint objects in an image that's been cvtColor( ... CV_RGB2GRAY)

Q.) Am I even creating the Mat for the map correctly mapxC1.create(sizeImage,1); ? Charles Mennell

( 2012-05-16 05:19:11 -0600 )edit

The problem is that the parameters PointC1x and PointC1y should be integers (remember, after all, you are accessing data stored in the computer's memory which can only be indexed using integer numbers). Try: mapxC1.at(static_cast<int>(PointC1x), static_cast<int>(PointC1y))

( 2012-05-16 14:14:12 -0600 )edit

Hi Cherry, because I haven't found much about this it would be usefull if you share your results. Would you please add them? Is this rectify"thing" realy needed for triangulation? did you got any further? i tried with undistortPoints too because many people are only using this but no rectification. I get weird results so please let us now what you have done with some code like you wrote at your questions. Thanks a lot!

more