ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question

Rviz: Display image in 3D scene

asked 2015-10-01 08:59:01 -0500

NickDP gravatar image


For a visual odometry project I'm using Rviz to display what my filter is doing. I.e. I'm displaying the current camera pose, the path the camera has taken and the features it is tracking as a point cloud. I would also like to show the current camera image in the scene, attached to the cameras pose as it moves through around.

So far I have not been able to accomplish this. I can show the image in an Image display in Rviz, which is just a separate view/window, similar to what I would get with cv::imshow(). I also played around with creating a point cloud of all my pixels and assigning grayscale intensity values to it. This technically works, but it does not look very nice and is very slow, as is to be expected.

Is there a way to do what I'm trying to accomplish?

Any help is much appreciated

edit retag flag offensive close merge delete

3 Answers

Sort by ยป oldest newest most voted

answered 2015-10-07 03:53:29 -0500

updated 2015-10-12 09:33:18 -0500

For DRC we developed a rviz Display type that takes in a Image message and shape_msgs/Mesh. It then texture maps the image on the mesh using the OGRE texturing API. So if you publish a flat rectangle mesh to it and simultaneously select the appropriate image topic, you should basically get what you're looking for. Unfortunately, the Mesh message type doesn't provide a frame_id and the plugin code currently lives in a pretty big repo, so I'd recommend copying the plugin code and implementing your own slightly modified version of the plugin.

Code here.

edit flag offensive delete link more


That looks promising. Thank you very much

NickDP gravatar image NickDP  ( 2015-10-12 08:53:00 -0500 )edit

I'm trying that customized mesh display plugin route: if I have a square mesh in the xy plane, a projector set to [0,0,1] and [ 0. , 0.70682518, 0. , 0.70738827] should I see the texture on the plane? Currently I can see my mesh but no texture. And the frame_id of the image doesn't matter?

lucasw gravatar image lucasw  ( 2016-02-08 08:02:09 -0500 )edit

IIRC the mesh is always diplayed as if it's frame_id was the currently selected frame_id in rviz. The frame_id of the image is used. Our use case was transforming the mesh into world frame and then projecting image data based on the tf chain from camera to world.

Stefan Kohlbrecher gravatar image Stefan Kohlbrecher  ( 2016-02-08 15:14:30 -0500 )edit

Hi, I'm trying to do dynamic texturing on a mesh(actually a 3D model in .dae format) does this plugin add textures only to a plane mesh or does it work for any arbitary mesh??

natsu gravatar image natsu  ( 2017-07-23 12:18:44 -0500 )edit

Works for arbitrary meshes.

Stefan Kohlbrecher gravatar image Stefan Kohlbrecher  ( 2017-08-09 09:30:06 -0500 )edit

answered 2016-10-27 09:49:00 -0500

lucasw gravatar image could do what you want though you would have to create a tf child of the camera pose at the desired distance to the camera tf. The texture would be provided via an image topic. The core code is in fact a couple steps removed from that DRC plugin @stefan-kohlbrecher links to, but I wanted a simple image topic on only a rectangle interface, maybe the general Mesh projection could be re-introduced later or be in a different plugin.

Also answered in a similar question here:

edit flag offensive delete link more

answered 2015-10-06 12:21:10 -0500

William gravatar image

I'm not aware of anyway to do this in rviz at the moment, but you could write a rviz plugin to accomplish this. To me the open question for a non-depth image is how far away from the image origin should the image be rendered? I guess it would be a fixed, configurable distance?

edit flag offensive delete link more


Exactly. The image floats in front of the camera pose at a distance proportional to the desired image size & focal length. I have no experience with rviz plugins and unfortunately I don't expect I will have a lot of time to play around with them in the timeframe of this project.

NickDP gravatar image NickDP  ( 2015-10-06 16:46:00 -0500 )edit

Question Tools



Asked: 2015-10-01 08:59:01 -0500

Seen: 5,693 times

Last updated: Oct 27 '16