Rviz: Display image in 3D scene

rviz
image

asked 2015-10-01 08:59:01 -0500

NickDP
148 ●6 ●12 ●18

Hi

For a visual odometry project I'm using Rviz to display what my filter is doing. I.e. I'm displaying the current camera pose, the path the camera has taken and the features it is tracking as a point cloud. I would also like to show the current camera image in the scene, attached to the cameras pose as it moves through around.

So far I have not been able to accomplish this. I can show the image in an Image display in Rviz, which is just a separate view/window, similar to what I would get with cv::imshow(). I also played around with creating a point cloud of all my pixels and assigning grayscale intensity values to it. This technically works, but it does not look very nice and is very slow, as is to be expected.

Is there a way to do what I'm trying to accomplish?

Any help is much appreciated

edit retag flag offensive close merge delete

add a comment

2

answered 2015-10-07 03:53:29 -0500

Stefan Kohlbrecher

24361 ●173 ●299 ●444 https://www.energy-rob...

updated 2015-10-12 09:33:18 -0500

For DRC we developed a rviz Display type that takes in a Image message and shape_msgs/Mesh. It then texture maps the image on the mesh using the OGRE texturing API. So if you publish a flat rectangle mesh to it and simultaneously select the appropriate image topic, you should basically get what you're looking for. Unfortunately, the Mesh message type doesn't provide a frame_id and the plugin code currently lives in a pretty big repo, so I'd recommend copying the plugin code and implementing your own slightly modified version of the plugin.

Code here.

edit flag offensive delete link

Comments

That looks promising. Thank you very much

NickDP ( 2015-10-12 08:53:00 -0500 )edit

I'm trying that customized mesh display plugin route: if I have a square mesh in the xy plane, a projector set to [0,0,1] and [ 0. , 0.70682518, 0. , 0.70738827] should I see the texture on the plane? Currently I can see my mesh but no texture. And the frame_id of the image doesn't matter?

lucasw ( 2016-02-08 08:02:09 -0500 )edit

2

IIRC the mesh is always diplayed as if it's frame_id was the currently selected frame_id in rviz. The frame_id of the image is used. Our use case was transforming the mesh into world frame and then projecting image data based on the tf chain from camera to world.

Stefan Kohlbrecher ( 2016-02-08 15:14:30 -0500 )edit

Hi, I'm trying to do dynamic texturing on a mesh(actually a 3D model in .dae format) does this plugin add textures only to a plane mesh or does it work for any arbitary mesh??

natsu ( 2017-07-23 12:18:44 -0500 )edit

Works for arbitrary meshes.

Stefan Kohlbrecher ( 2017-08-09 09:30:06 -0500 )edit

add a comment

1

answered 2016-10-27 09:49:00 -0500

lucasw

8729 ●137 ●239 ●256 https://github.com/lucasw

https://github.com/lucasw/rviz_textur... could do what you want though you would have to create a tf child of the camera pose at the desired distance to the camera tf. The texture would be provided via an image topic. The core code is in fact a couple steps removed from that DRC plugin @stefan-kohlbrecher links to, but I wanted a simple image topic on only a rectangle interface, maybe the general Mesh projection could be re-introduced later or be in a different plugin.

Also answered in a similar question here: http://answers.ros.org/question/22730...

edit flag offensive delete link

add a comment

0

answered 2015-10-06 12:21:10 -0500

William

17335 ●101 ●182 ●262 http://wjwwood.io/

I'm not aware of anyway to do this in rviz at the moment, but you could write a rviz plugin to accomplish this. To me the open question for a non-depth image is how far away from the image origin should the image be rendered? I guess it would be a fixed, configurable distance?

edit flag offensive delete link

Comments

Exactly. The image floats in front of the camera pose at a distance proportional to the desired image size & focal length. I have no experience with rviz plugins and unfortunately I don't expect I will have a lot of time to play around with them in the timeframe of this project.

NickDP ( 2015-10-06 16:46:00 -0500 )edit

add a comment

Rviz: Display image in 3D scene

3 Answers

Comments

Comments

Question Tools

Stats

Related questions

Rviz: Display image in 3D scene edit

3 Answers

Comments

Comments

Question Tools

Stats

Related questions

Rviz: Display image in 3D scene