Ask Your Question

Simulated raytracing in rviz or gazebo

asked 2017-07-20 13:34:09 -0500

JoshMarino gravatar image

updated 2017-07-21 21:04:11 -0500


I have an application where I have a camera attached to the end of a 5-DOF robot arm (X, Y, Z, yaw, pitch). The robot arm is moving the camera around a fixed object (0,0,0) for inspection purposes. Given the position and orientation of the camera, is it possible to detect where on the object the camera is seeing? For example, I would like to know whether the axis of the camera is intersecting the object, and if so, determine the exact coordinates of the object the camera axis is intersecting.

Given the known camera axis (position/orientation), is it possible to extend a line along the +x-axis and see where it hits the fixed object (if it all)?

I should be able to calculate positions along the +x-axis moving outwards from the camera frame. However, I do not know how to check if these points are contained inside of the fixed object of interest. Would also be nice if something already existed for this, or at least something I can use to help.


edit retag flag offensive close merge delete


Are you sensing the object somehow? If so, how?

Airuno2L gravatar image Airuno2L  ( 2017-07-21 09:11:05 -0500 )edit

I know what the object is beforehand, and it is placed at world coordinates (0,0,0).

JoshMarino gravatar image JoshMarino  ( 2017-07-21 12:42:14 -0500 )edit

I'm a little confused, if you know it is at (0,0,0) why do you do say "determine the world coordinates of the object the camera is looking at"? Aren't the world coordinates of the object are going to be (0,0,0)?

Airuno2L gravatar image Airuno2L  ( 2017-07-21 17:55:00 -0500 )edit

Sorry, guess I was not clear. The object is very large wrt the camera. I would like to know where on the object I am looking at through the center of the camera, not the center of the object (0,0,0).

JoshMarino gravatar image JoshMarino  ( 2017-07-21 20:51:02 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted

answered 2017-07-21 18:09:13 -0500

Airuno2L gravatar image

updated 2017-07-24 07:39:54 -0500

Although I'm not 100% sure what you're trying to do, the image_geometry library might have some useful stuff in it.

For example, the project3dToPixel(point) and/or projectPixelTo3dRay(uv) might be useful.

If you're wondering if, at any given time, the camera's center axis is intersecting the objects bounding box, you might have to implement that by hand, or grab an example from online somewhere by searching for something like "code for ray passing through a bounding box".

Or are you trying to find the world coordinates of the point at which the camera's center axis intersects the object? If so, is the object shape arbitrary? Or is it like a cube or sphere or some other geometric primitive shape?

EDIT: I understand better now. What I would do is convert my object model to an octree and do ray tracing on that. There are two options that I know of, one is to use Octomap which has the castRay function or use the Octrees in Point Cloud Library which have the OctreePointCloudSearch functions.

In either case you'll need to convert your model into something that one of those libraries can use. Depending on what format the object is already in, here are some hints for doing it with Octomap. For Point Cloud Library, I know you can at least import from a PLY file using this class.

Then just use the camera location as the origin of the ray and the camera direction as the ray direction and bingo! For what it's worth, I think I've used PCL and Octomap to do something similar in the past and don't recall either one being easier or harder than the other, so I don't think it matters too much which you choose.

edit flag offensive delete link more


Yes to the last paragraph. The object is not a geometric primitive shape.

JoshMarino gravatar image JoshMarino  ( 2017-07-21 20:54:25 -0500 )edit

That sounds exactly what I am looking for. I'll give it a try and see if it works.

JoshMarino gravatar image JoshMarino  ( 2017-07-24 16:58:54 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools



Asked: 2017-07-20 13:34:09 -0500

Seen: 832 times

Last updated: Jul 24 '17