Ask Your Question
0

Pose (rvec, tvec) from Opencv

asked 2011-03-22 01:07:56 -0500

dlmypr gravatar image

updated 2014-01-28 17:09:23 -0500

ngrennan gravatar image

Hi everyone,

Does anyone know what rvec and tvec corresponds to? I read the corresponding opencv pages but it just indicates that rvec is the rotation and tvec is the translation vector. But with respect to what?

Best regards..

References:

SolvePnp

FindExtrinsiccameraparams2

edit retag flag offensive close merge delete

4 Answers

Sort by » oldest newest most voted
1

answered 2011-03-23 13:32:17 -0500

updated 2011-03-23 14:05:46 -0500

In tod_detecting (I think we tell about this transformation) after running recognizer->match you get vector of guesses. Each guess contains PoseRT (rvec, tvec) field (aligned_pose). It is transformation from some abstract coordinate system (defined by fiducial.yaml configuration file: in this system coordinates of corners of fiducial markers must be equals described ones in config) to coordinate system of camera for test image. If you want to find projection of this guess to the test image, you should take any observation of this object from training base(recommend to choose observation with ImageId equals imageIndex from guess->imageIndices). Next you should transform 3d points related with the observation to the abstract coord. system using inverse features3d->camera->pose transformation (for example with using Tools::invert and project3dPoints). Then you should transform obtained 3d points with guess->aligned_pose and project them to the test image (for this purpose you should know intrinsics parameters of the camera, now we assume, that test and train images was photographed with the same camera; we use projectPoints function for this purpose).

Let consider a case of solvePnP or solvePnPRansac:

void solvePnP(const Mat& objectPoints, const Mat& imagePoints, const Mat& cameraMatrix, const Mat& distCoeffs, Mat& rvec, Mat& tvec, bool useExtrinsicGuess=false)

The function returns rvec and tvec. This transformation converts objectPoints to the coordinate system of the camera such a way, that re-projection error between imagePoints and projected points after transformation would be a minimal. In other words we try to find rvec-tvec pair to minimize distance between projection (u, v) of transformed point (x,y,z) calculated on (X,Y,Z) point from objectPoints and corresponding point (x, y) from imagePoints. ki and pi are from distCoeffs, fx, fy, cx, cy are from cameraMatrix.

See attached image: image description

edit flag offensive delete link more

Comments

Thank you for your detailed reply. For being sure, if I have a guess containing PoseRT and transformed it to the image plane using projectpoints and saw that my GUESS is fine. So the tvec of the guess is the TRANSLATION vector IN CENTIMETERS? from my board coordinate system to the camera coordinate system. But in terms of camera coordinate system or chessboard coordinate system?
dlmypr gravatar image dlmypr  ( 2011-03-23 20:07:44 -0500 )edit
As an example in tod_detecting we end up with a candidate object pose inside an ObjectInfo object. The output of object.tvec is as follows, tvec = [-0.0661444212290265; -0.0178236515058925; 0.01091917408303104]. So it means the object coordinate frame is moved -6 CM in x direction, -1 CM in y direction and 1 CM in z direction with respect to what? Where this translation will get me from object origin to WHICH coordinate frame's ORIGIN?
dlmypr gravatar image dlmypr  ( 2011-03-23 21:11:28 -0500 )edit
2

answered 2011-03-23 01:28:50 -0500

updated 2011-03-23 08:32:58 -0500

rvec and tvec determine transformation between coordinate systems. rvec uniquely determines rotation matrix 3x3 (let it be named R). Transformation equation (T - transpose): (x y z)T = R*(X Y Z)T + tvec

Rodrigues function from OpenCV can convert rvec to R or R to rvec. Direction of rvec vector is the same with the axis of rotation, magnitude of the vector is angle of rotation.

Also OpenCV contains solvePnPRansac function in calib3d module. If you have incorrect matches you should use it.

edit flag offensive delete link more

Comments

Thank you for your reply Alexander. But do you know from which frame to which frame does this transformation corresponds? I get correct poses and can draw them but I dont what transformation (from which frame to another?) does these vectors belong?
dlmypr gravatar image dlmypr  ( 2011-03-23 07:42:04 -0500 )edit
Like if I put a chessboard and assign a frame on it (for example using fiducial.yaml) I got 0,0,0 point as that origin and when I draw the pose it is perfectly aligned with the axis I created on the chessboard. But the transformation matrix created with rvec and tvec is from which frame to another?
dlmypr gravatar image dlmypr  ( 2011-03-23 09:52:07 -0500 )edit
0

answered 2011-03-24 01:51:35 -0500

updated 2011-03-24 01:51:51 -0500

As an example in tod_detecting we end up with a candidate object pose inside an ObjectInfo object. The output of object.tvec is as follows, tvec = [-0.0661444212290265; -0.0178236515058925; 0.01091917408303104]. So it means the object coordinate frame is moved -6 CM in x direction, -1 CM in y direction and 1 CM in z direction with respect to what? Where this translation will get me from object origin to WHICH coordinate frame's ORIGIN?

Guess contains rvec and tvec transformed some 3d points from abstract coordinate system (that set by fidycial.yaml config and rigidly attached to fidycial markers) to the coordinate system rigidly attached to test camera. It is not in centimeters. Scale of 3d translation depends on calibration parameters of your device. I don't know what is it in the case of the Kinect.

But in terms of camera coordinate system or chessboard coordinate system?

In terms of camera coordinate system.

edit flag offensive delete link more

Comments

First I thought it should be the camera frame and when I looked into the tvec (i was thinking that it was in centimeters) and thought like it can not be the camera frame. Thanks for enlightening me =) What do I need to calculate 3d translation (tvec) into centimeters? What is the formula for it? I need to know where is the camera frame is located with respect to the real world (chess board frame) in centimeters. Thanks for your replies and care..
dlmypr gravatar image dlmypr  ( 2011-03-24 02:01:23 -0500 )edit
Kinect Camera parameters camera_matrix: rows: 3 cols: 3 data: [ 526.37013657, 0.00000000, 313.68782938, 0.00000000, 526.37013657, 259.01834898, 0.00000000, 0.00000000, 1.00000000 ] distortion_coefficients: rows: 1 cols: 5 data: [ 0.18126525, -0.39866885, 0.00000000, 0.00000000, 0.00000000 ]
dlmypr gravatar image dlmypr  ( 2011-03-24 02:04:35 -0500 )edit
I know that openni depthmap gives back coordinates for each pixel in mm. But I don't know which way ros wrapper for openni changes this coords. Maybe you find some useful information here: http://groups.google.com/group/openni-dev/browse_thread/thread/478f9167acc30759/bc6f9c1cb751b72d
Alexander Shishkov gravatar image Alexander Shishkov  ( 2011-03-24 02:36:00 -0500 )edit
But apparently you are right in the case of the Kinect. Here (http://groups.google.com/group/openni-dev/browse_thread/thread/ba354b6bb1606cb3) Suat Gedikli wrote that in ROS points are in meters.
Alexander Shishkov gravatar image Alexander Shishkov  ( 2011-03-24 02:38:59 -0500 )edit
I made some experiments now with a fiducial chessboard marker and what I got as tvec is nearly same as I measured while putting the chessboard. The problem of the old experiment was probably about the wrong camera parameters. Thank you for all your replies and time you spend answering it helped me a lot. So we can conclude that if some is using kinect with the above parameters the resulting tvec of PoseRT is in meters and giving the translation of your chessboard frame origin FROM camera frame represented in CAMERA COORDINATE FRAME.
dlmypr gravatar image dlmypr  ( 2011-03-24 03:19:54 -0500 )edit
0

answered 2011-03-22 05:18:13 -0500

a) This is really an OpenCV question not a ROS question and thus answers.ros.org is probably not the best place to ask this.

b) In the links you gave there are descriptions of both of those:

rvec – The output rotation vector (see Rodrigues2) that (together with tvec) brings points from the model coordinate system to the camera coordinate system

tvec – The output translation vector

If you need help in interpreting those descriptions of rvec and tvec you should probably look to the OpenCV community.

edit flag offensive delete link more

Comments

Actually I wonder this transformation (created from a rvec, tvec pair)is from which frame to which other frame?
dlmypr gravatar image dlmypr  ( 2011-03-23 09:53:31 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2011-03-22 01:07:56 -0500

Seen: 21,642 times

Last updated: Mar 24 '11