ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question

conversion of depht image coordinates to world coordinates (u,v,d) to (x,y,z)

asked 2014-01-21 02:23:56 -0600

stfn gravatar image

Hi, I implemented a object detection classifier working on rgb and depth data (from openni_launch) which provides object center coordinates in the image plane (u,v). Now I want to convert those image coordinates (+depth) to world coordinates (resp. the 3D camera frame).

I know I could just use the (u,v) coordinates and the registered cloud to get the point stored at (u,v,) but sometimes there is a NaN at this point and I dont want to deal with that as it would require to look for the neighbours which biases the object center and stuff. Another drawback is that after some pcl-calculations (resampling, removing nans, calculating normals, removing major planes etc.) the cloud is not organized anymore.

Second I could use the /cameraInfo topic to get the intrinsics of my depth sensor in order to calculate the conversion by hand using the lens equation, but this is some trouble if you want to do it the correct way (considering all instrinsics of the camera matrix).

So ... as openni_launch already does this for the whole pointcloud-creation, is there a function or service offering this conversion in openni_launch, openni api, pcl api, ros_perception etc?

edit retag flag offensive close merge delete

3 Answers

Sort by » oldest newest most voted

answered 2014-01-21 05:18:57 -0600

updated 2014-01-21 05:19:39 -0600

image_geometry can do this very easily (unfortunately the API docs are broken right now).

image_geometry::PinholeCameraModel model_;
cv::Point3d point3d = projectPixelTo3dRay(point2d); //point2d is a cv::Point2d

This will give you a ray which you can then normalize and multiply by your depth to get the 3D point.

PCL probably has a way to do this too.

edit flag offensive delete link more


That should do, even though I'd expect openni_launch resp. rgbd_launch to offer that as well somewhere. Anyway, which */camera_info do I need then, rgb, or depth? And does this depend on weather depth_registered is set?

stfn gravatar image stfn  ( 2014-01-22 09:10:54 -0600 )edit

camera_info should be from whatever camera the detection came from. If it came from /camera/rgb/whatever, you should use /camera/rgb/camera_info. It's probably simplest to have depth registration on and do everything in /camera/depth_registered, but it shouldn't matter.

Dan Lazewatsky gravatar image Dan Lazewatsky  ( 2014-01-22 09:24:37 -0600 )edit

The solution you propose looks reasonable to me. But I wonder why the most of the documentation and discussion on the internet about how to get a 3D point from the depth image is different. They directly consider the depth value as z coordinate (in mm).

Pablo Iñigo Blasco gravatar image Pablo Iñigo Blasco  ( 2016-09-30 02:41:45 -0600 )edit

It looks like they are an orthogonal projection, while you are proposing a perspective projection, right?

Pablo Iñigo Blasco gravatar image Pablo Iñigo Blasco  ( 2016-09-30 05:53:58 -0600 )edit

answered 2014-01-21 03:14:47 -0600

Yuan gravatar image

updated 2014-01-21 03:15:15 -0600

Hi, [xnConvertProjectiveToRealWorld]( from openni is what you need. PCL may have wrapper for it.

edit flag offensive delete link more

answered 2015-08-28 10:53:07 -0600

Yake gravatar image

hello there, I'm using different image(the rgb and depth),but I can't get the z correctly,it always 1. and the x, y from rgb and depth to 3d is not the same.

 cv::Point2d pixel_point(98, 296);

cout << "pixel_point (2D) = " << pixel_point << endl << endl;

  cv::Point3d xyz = cam_model_.projectPixelTo3dRay(pixel_point);

cout << "point (3D) = " << xyz << endl << endl; rgb


edit flag offensive delete link more

Question Tools



Asked: 2014-01-21 02:23:56 -0600

Seen: 4,341 times

Last updated: Aug 28 '15