ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question
1

Does OpenCV's undistortPoints also rectify them?

asked 2012-05-04 04:07:15 -0500

I'm trying to determine the distance between two object by using two cameras (OCV 2.3.1, MSVC++, Win7) but can't calculate the object's objectPoints. I think this is because the image points aren't being rectified before their disparity is calculated.

I. WHAT I DO FIRST Step 1. Calibrate each camera by itself

int numSquares = numCornersHor * numCornersVer;
Size board_sz = Size(numCornersHor, numCornersVer);

Mat cameraMatrix = Mat(3, 3, CV_32FC1);
Mat distCoeffs;

vector<Mat> rvecs, tvecs;

cameraMatrix.ptr<float>(0)[0] = 1;
cameraMatrix.ptr<float>(1)[1] = 1;

calibrateCamera(object_points, image_points, image.size(), 
                    cameraMatrix, distCoeffs, rvecs, tvecs);

Step 2. Calibrate the cameras to together.
int numCornersHor = 4; int numCornersVer = 3; const float squareSize = 1.75;

Size imageSize = Size(numCornersHor, numCornersVer);
   int numSquares = numCornersHor * numCornersVer;

for(int i = 0; i < pairs; i++ )
{
    for( int j = 0; j < imageSize.height; j++ )
    {
        for( int k = 0; k < imageSize.width; k++ )
        {
            objectPoints[i].push_back(Point3f(j*squareSize, k*squareSize, 0));
        }
    }
}

Mat R, T, E, F;

rms = stereoCalibrate(  
    objectPoints, 
    imagePoints[0],     imagePoints[1],
    cameraMatrix[0],    distCoeffs[0],
    cameraMatrix[1],    distCoeffs[1],
    imageSize, 
    R,  T,  E,  F,
    TermCriteria(CV_TERMCRIT_ITER+CV_TERMCRIT_EPS, 100, 1e-5),
    CV_CALIB_FIX_ASPECT_RATIO +
    CV_CALIB_ZERO_TANGENT_DIST +
            CV_CALIB_SAME_FOCAL_LENGTH +
    CV_CALIB_RATIONAL_MODEL +
    CV_CALIB_FIX_K3 +   CV_CALIB_FIX_K4 +   CV_CALIB_FIX_K5
    );

Step 3. Create the rectification data stereoRectify( cameraMatrix[0], cameraMatrix[1], distCoeffs[0], distCoeffs[1], imageSize, R, T, RC1, RC2, //RC1: Rotation matrix Camera 1 PC1, PC2, Q, CALIB_ZERO_DISPARITY, 1,
imageSize);

II. WHAT I BELIEVE Goal: I'm trying undistort and rectify the image points of one object in the image from camera 1 and the image from camera 2 (I do this process twice: once while the clay pigeon's on the launcher and once one frame before the clay pigeon disintegrates)

Method: I believe that I don't need to use initUndistortRectifyMap then Remap but can instead just use undistortPoints. I think undistortPoints undistorts the points of interest and rectifies them.

III. WHAT I DO SECOND You can ignore this if my beliefs aren't correct.

undistortPoints(launcherC1, launcherC1Undistorted, cameraMatrixC1, distCoeffsC1, R1, P1);   
undistortPoints(launcherC2, launcherC2Undistorted, cameraMatrixC2, distCoeffsC2, R2, P2);   

undistortPoints(clayPigeonC1, clayPigeonC1Undistorted, cameraMatrix1, distCoeffs1, R1, P1); 
undistortPoints(clayPigeonC2, clayPigeonC2Undistorted, cameraMatrix2, distCoeffs2, R2, P2);

The input and output arrays for undistortPoints (launcherC1, launcherC1Undistorted, ... clayPigeonC2, clayPigeonC2Undistorted) are vectors of Point2f objects.

IV. DISCREPANCY BETWEEN BELIEF AND REALITY After all undistortPoints functions are run, 1. launcherC1Undisorted.y does not equal launcherC2Undistorted.y 2. clayPigeonC1Undistorted.y does not equal clayPigeonC2Undistorted.y.
They are up to 30% different.

V. QUESTIONS Q1: In addition to undistorting them does undistortPoints also rectify points? Q1.1_yes. Are the values of y supposed to be equal after rectification? Q1.1.1_yes Can you tell from the code I've included what I'm doing wrong so that they don't? Q1_no If undistortPoints doesn't rectify the points then how do I rectify them?

Thank you for any assistance. Charles++

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
1

answered 2012-05-10 00:12:57 -0500

Hi Charles,

Your question doesn't have much to do with ROS, maybe you should ask it on an OpenCV forum, but as long as you made the effort to post it so nicely let my try to answer it.

As far as I know undistortPoints do not rectify the points. The correct approach would be to call initUndistortRectifyMap and then use Remap.

If instead of remapping the whole image what you need is to undistort and rectify only some image points, then what you can do is use manually the maps (mapx and mapy) generated by initUndistortRectifyMap. Something like this (assuming that point1 is a point on the original image):

undistortedRectifiedPoint.x = mapx[point1.x, point1.y]
undistortedRectifiedPoint.y = mapy[point1.x, point1.y]

I hope this helps.

Best regards, Martin.

edit flag offensive delete link more

Comments

I can't read mapx & mapy. I try Mat mapxC1; mapxC1.create(sizeImage,1); initUndistortRectifyMap(cameraMatrix1, distCoeffs1, R1, camMat1UndistRect, sizeImage, 0, /* ???*/ mapxC1, mapyC1);

Cherry 3.14 gravatar image Cherry 3.14  ( 2012-05-15 09:15:44 -0500 )edit

vector<Point2f> point1Cam1UndistortedRectified; point1Cam1UndistortedRectified.at(0).x = mapxC1[PointC1x, PointC1y]; point1Cam1UndistortedRectified.at(0).y = mapxC1[PointC1x, PointC1y]; but MSVC++ says not operator "[]" matches these operands right after mapxC1 in the last two lines. Know why?

Cherry 3.14 gravatar image Cherry 3.14  ( 2012-05-15 09:22:25 -0500 )edit

Hi CharlesM, Actually the syntax "mapxC1[PointC1x, PointC1y]" is not correct, sorry I misled you with my pseudo-code. How about mapxC1.at(PointC1x,PointC1y)?

Martin Peris gravatar image Martin Peris  ( 2012-05-15 16:51:28 -0500 )edit

You can find more info on how to access OpenCV matrices efficiently here: http://docs.opencv.org/doc/tutorials/core/how_to_scan_images/how_to_scan_images.html#howtoscanimagesopencv

Martin Peris gravatar image Martin Peris  ( 2012-05-15 16:58:40 -0500 )edit

Hi Martin, Thank you so much for your help! I know I'll reach the answer being assisted by an OCV expert. You told me how to put exactly the data I want into maps but now I'm frustrated since I still can't get that data. I tried your suggestion and link. Results:

Cherry 3.14 gravatar image Cherry 3.14  ( 2012-05-16 05:17:32 -0500 )edit

1.) MSVC++ says about mapxC1.at(PointC1x,PointC1y) that no overloaded functn matches argumnt list and that there are too many args in functn call.
2.) I don't know OCV or C++ well enough to make enough sense of the link you sent to get at a map's data

Cherry 3.14 gravatar image Cherry 3.14  ( 2012-05-16 05:17:45 -0500 )edit

What I've done: PointC1x & PointC1y are floats that are the x & y of Keypoint objects in an image that's been cvtColor( ... CV_RGB2GRAY)

Q.) Am I even creating the Mat for the map correctly mapxC1.create(sizeImage,1); ? Charles Mennell

Cherry 3.14 gravatar image Cherry 3.14  ( 2012-05-16 05:19:11 -0500 )edit

The problem is that the parameters PointC1x and PointC1y should be integers (remember, after all, you are accessing data stored in the computer's memory which can only be indexed using integer numbers). Try: mapxC1.at(static_cast<int>(PointC1x), static_cast<int>(PointC1y))

Martin Peris gravatar image Martin Peris  ( 2012-05-16 14:14:12 -0500 )edit
0

answered 2012-09-11 21:15:42 -0500

Terry gravatar image

Hi Cherry, because I haven't found much about this it would be usefull if you share your results. Would you please add them? Is this rectify"thing" realy needed for triangulation? did you got any further? i tried with undistortPoints too because many people are only using this but no rectification. I get weird results so please let us now what you have done with some code like you wrote at your questions. Thanks a lot!

edit flag offensive delete link more

Question Tools

Stats

Asked: 2012-05-04 04:07:15 -0500

Seen: 5,746 times

Last updated: Sep 11 '12