ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange

Simulated raytracing in rviz or gazebo

Hello,

I have an application where I have a camera attached to the end of a 5-DOF robot arm (X, Y, Z, yaw, pitch). The robot arm is moving the camera around a fixed object (0,0,0) for inspection purposes. Given the position and orientation of the camera, is it possible to detect where on the object the camera is seeing? For example, I would like to know whether the axis of the camera is intersecting the object, and if so, determine the exact coordinates of the object the camera axis is intersecting.

Given the known camera axis (position/orientation), is it possible to extend a line along the +x-axis and see where it hits the fixed object (if it all)?

I should be able to calculate positions along the +x-axis moving outwards from the camera frame. However, I do not know how to check if these points are contained inside of the fixed object of interest. Would also be nice if something already existed for this, or at least something I can use to help.

Thanks

edit retag close merge delete

Are you sensing the object somehow? If so, how?

( 2017-07-21 09:11:05 -0500 )edit

I know what the object is beforehand, and it is placed at world coordinates (0,0,0).

( 2017-07-21 12:42:14 -0500 )edit

I'm a little confused, if you know it is at (0,0,0) why do you do say "determine the world coordinates of the object the camera is looking at"? Aren't the world coordinates of the object are going to be (0,0,0)?

( 2017-07-21 17:55:00 -0500 )edit

Sorry, guess I was not clear. The object is very large wrt the camera. I would like to know where on the object I am looking at through the center of the camera, not the center of the object (0,0,0).

( 2017-07-21 20:51:02 -0500 )edit

@Airuno2L, could you please have a look at this query? thanks

( 2021-08-04 02:27:22 -0500 )edit

Sort by ยป oldest newest most voted

Although I'm not 100% sure what you're trying to do, the image_geometry library might have some useful stuff in it.

For example, the project3dToPixel(point) and/or projectPixelTo3dRay(uv) might be useful.

If you're wondering if, at any given time, the camera's center axis is intersecting the objects bounding box, you might have to implement that by hand, or grab an example from online somewhere by searching for something like "code for ray passing through a bounding box".

Or are you trying to find the world coordinates of the point at which the camera's center axis intersects the object? If so, is the object shape arbitrary? Or is it like a cube or sphere or some other geometric primitive shape?

EDIT: I understand better now. What I would do is convert my object model to an octree and do ray tracing on that. There are two options that I know of, one is to use Octomap which has the castRay function or use the Octrees in Point Cloud Library which have the OctreePointCloudSearch functions.

In either case you'll need to convert your model into something that one of those libraries can use. Depending on what format the object is already in, here are some hints for doing it with Octomap. For Point Cloud Library, I know you can at least import from a PLY file using this class.

Then just use the camera location as the origin of the ray and the camera direction as the ray direction and bingo! For what it's worth, I think I've used PCL and Octomap to do something similar in the past and don't recall either one being easier or harder than the other, so I don't think it matters too much which you choose.

more

Yes to the last paragraph. The object is not a geometric primitive shape.

( 2017-07-21 20:54:25 -0500 )edit

That sounds exactly what I am looking for. I'll give it a try and see if it works.

( 2017-07-24 16:58:54 -0500 )edit