Robotics StackExchange | Archived questions

How to obtain depth registered image from simulated Kinect in Gazebo ?

Hi,

I am new to using Kinect in gazebo_ros, and while I am able to publish (from Gazebo) the ROS topics given in this tutorial , however I do not understand how to obtain the depth-registered (registered with RGB image) images from the simulated Kinect.

Is there some way in ROS to get the depth registered image ?

The output of my rostopic list (after launching gazebo_ros with kinect model) is - 

/camera/depth/camera_info
/camera/depth/image_raw
/camera/depth/image_raw/compressed
/camera/depth/image_raw/compressed/parameter_descriptions
/camera/depth/image_raw/compressed/parameter_updates
/camera/depth/image_raw/compressedDepth
/camera/depth/image_raw/compressedDepth/parameter_descriptions
/camera/depth/image_raw/compressedDepth/parameter_updates
/camera/depth/image_raw/theora
/camera/depth/image_raw/theora/parameter_descriptions
/camera/depth/image_raw/theora/parameter_updates
/camera/depth/points
/camera_ir/depth/camera_info
/camera_ir/parameter_descriptions
/camera_ir/parameter_updates

Thanks!

Asked by malharjajoo on 2018-04-24 10:54:50 UTC

Comments

Answers

In my experience, whenever I've used a simulated Kinect in Gazebo the point cloud produced (in your case /camera/depth/points) is already an XYZRGB templated pointcloud; i.e. there are already depth and color fields filled out.

For example, using the turtlebot_simulator the header for a pointcloud produced on the /camera/depth/points topic is as follows:

header: 
  seq: 8148
  stamp: 
    secs: 996
    nsecs: 190000000
  frame_id: "camera_depth_optical_frame"
height: 480
width: 640
fields: 
  - 
    name: "x"
    offset: 0
    datatype: 7
    count: 1
  - 
    name: "y"
    offset: 4
    datatype: 7
    count: 1
  - 
    name: "z"
    offset: 8
    datatype: 7
    count: 1
  - 
    name: "rgb"
    offset: 16
    datatype: 7
    count: 1

Notice there are fields for position and color. For reference, the configuration for Kinect Gazebo plugin used by the turtlebot sim is here. As an additional piece of info, note that this snippet of code is where the simulated depth map is turned into a point cloud inside of the Kinect Gazebo plugin, and you can see they also add the RGB fields.

I suppose it would be possible to instead mimic something like freenect_launch and use depth_image_proc to register the RBG/depth map, and build the point cloud outside of the Gazebo plugin (if you had a need for such a thing).

Asked by jarvisschultz on 2018-04-27 09:59:39 UTC

Comments

Sorry I'm not clear. The code snippet you pointed makes sense although I don't know what double hfov = this->parentSensor->DepthCamera()->HFOV().Radian(); and related stuff does.

Regardless, are you trying to say that the pointcloud provided here is the same as a depth registered image ?

Asked by malharjajoo on 2018-04-27 17:35:30 UTC

It's not the same as a depth registered image (it's a pointcloud). However, when running a real Kinect the pointcloud contained in the depth_registered namespace (e.g. /camera/depth_registered/points) is an XYZRGB pointcloud just like this one is. If that's ultimately what you're looking for....

Asked by jarvisschultz on 2018-04-30 09:37:45 UTC

... then this topic already contains that information.

Asked by jarvisschultz on 2018-04-30 09:38:04 UTC

Hi, I am looking for a depth-registered image and not a registered pointcloud (unless there is a way to get the depth-registered image from the pointcloud). This is mainly because ROS Object Recognition Kitchen (ORK) requires this topic.

Asked by malharjajoo on 2018-04-30 09:43:23 UTC

I haven't spent much time studying the code, but in a quick test, it looks to me like /camera/depth/image_raw and the /camera/rgb/image_raw are already expressed in the same coordinate system. Perhaps you could pass those topics to ORK?

Asked by jarvisschultz on 2018-05-01 07:15:08 UTC

Hey!

@jarvisschultz, if /camera/depth/points should contain color information as well as you pointed out that it is a XYZRGB cloud, then why cannot we see color information while plotting it in RVIZ?

Asked by pravi on 2020-12-07 15:48:58 UTC

@pravi couple of thoughts:

  • The Gazebo depth camera plugins are definitely sensitive to things like the <format> tag. It's possible your plugin isn't producing colored point clouds. For example, you could be getting an XYZRGB pointcloud where the RGB fields are actually grayscale
  • Are you sure that the "Color Transformer" field in your PointCloud2 rviz display is set to RGB8?

Asked by jarvisschultz on 2020-12-07 18:24:33 UTC