Projecting image coordinates into world coordinates

asked 2014-08-31 08:43:00 -0600

James Diprose gravatar image

Hi

I'm using a Creative Senz3D for human-robot interaction perception tasks. I want to project an image point (u,v) into world coordinates, but where there is no corresponding point in the point cloud. E.g. the sensed pixel may be further away than the range of the depth sensor, but not the color camera.

So I'm using these equations, where DEPTH is the estimated depth of the point.

x = DEPTH * (u - cx) / fx 
y = DEPTH * (v - cy) / fy 
z = DEPTH

But I'm not sure whether I should be using the unrectifed image or the rectified image? And whether I should be using should be using cx, cy, fx and fy from K or P in CameraInfo.

I'm guessing that if I use the rectified image then the image points will be distorted away from their corresponding depth points... And then if I did cloud.at(u,v) on a point that did have a depth value, then it might not actually find the point.

Thanks

Jamie

edit retag flag offensive close merge delete

Comments

Hi! I'm corot, I did the patch for adding RGB info to the pointcloud. Hope people with better understanding will give you a good answer... because all I can tell u is that "it seems to work" as I did. But be aware that you don't have a rectified image within the softkinetic driver!

jorge gravatar image jorge  ( 2014-08-31 15:49:02 -0600 )edit

Hey Corot! Thanks for your answer. Yeah I was using the equations that you would use to rectify an image (I think) by using the variables from P in camera info, but it always seemed like the transformed (u,v) points didn't match up correctly with the point cloud.

James Diprose gravatar image James Diprose  ( 2014-09-03 10:23:04 -0600 )edit

So I'm going to try just with K which has the variables for the raw images that the softkinetic driver provides...

James Diprose gravatar image James Diprose  ( 2014-09-03 10:23:37 -0600 )edit
add a comment