ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question

Depth filter of a (Kinect) image

asked 2014-04-23 01:33:38 -0500

Poseidonius gravatar image


I run a Kinect sensor and evaluate depth and camera information to track positions of small robots. The idea is to use the pointcloud to identify all robots (2.5m < z < 3m). I used a pcl/passthrough filter for this purpose. Now I want to mask out all parts of the corresponding image which are not part of this cloud.

From my point of view I have to implement the following steps:

  1. Filter the point cloud (already available in pcl)
  2. Transfer the point cloud into an binary image (?)
  3. Mask the image (opencv?)

Other ideas? Thanks for your help.


edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2014-05-05 04:45:52 -0500

Poseidonius gravatar image

updated 2014-05-06 12:47:00 -0500


Ifound a solution for the problem and want to present them ... in order to motivate you for improvements:-) I used the

  • openni,
  • openni_launch,
  • depth_image_proc
  • pcl_ros
  • pcl_conversions

packages for ROS Hydro on Kubuntu 12.10. I used a ASUS Xtion for my setup.

1. Launch file

The solution combines 3 core elements:

  1. a nodelet, generating images and pointclouds based on the sensor
  2. a filter merging the 3D PointCloud and RGB Image information
  3. a filter that evaluates the depth values and adapts the RGB values
  4. a transormation that generates an image based on the PointCloud2.

The launch file illustrates the start up of these elements and follow at the end of the description part. I want to include some comments:

-> 1. The standard launch file $(find openni_launch)/launch/openni.launch has to be adapted related to the fact, that the following filter just accepts image and cloud data with the same frame id. I changed line 7 and 8 from the mentioned file

 <arg name="rgb_frame_id"   default="$(arg tf_prefix)/$(arg camera)_rgb_optical_frame" />
 <arg name="depth_frame_id" default="$(arg tf_prefix)/$(arg camera)_depth_optical_frame" />


 <arg name="rgb_frame_id"   default="$(arg tf_prefix)/$(arg camera)_optical_frame" />
 <arg name="depth_frame_id" default="$(arg tf_prefix)/$(arg camera)_optical_frame" />

in a new file myopenni.launch.

-> 2. The depth_image_proc package offers nodes for PointCloud2 processing. We use one of them for a fusion of point cloud data with corresponding RGB information. Take a view to the documentation on !

-> 3+4 The last two points are combined in the pointCloudToImage node. I wrote some lines of C++ to transform the ROS messages to PCL Pointcloud2 later to an pcl::PointCloud<pcl::pointcloudxyzrgb> object. It filters the RGB values according to the z coordinate. Afterwards the PointCloud2 is transformed to a ros image.

   #include <iostream>
   #include "ros/ros.h"
   #include <sensor_msgs/PointCloud2.h>
   #include <sensor_msgs/Image.h>
   #include <pcl/filters/passthrough.h>
   #include "pcl_conversions/pcl_conversions.h"

   #define Z_MIN 1.0
   #define Z_MAX 1.5

   ros::Publisher pub_cloud;
   ros::Publisher pub_image; 

   void cloudCallback(const sensor_msgs::PointCloud2 &ros_cloud)
     pcl::PCLPointCloud2 pcl_pc2;
     pcl_conversions::toPCL(ros_cloud, pcl_pc2); 
     pcl::PointCloud<pcl::PointXYZRGB>::Ptr pcl_cloud(new pcl::PointCloud<pcl::PointXYZRGB>);     
     pcl::fromPCLPointCloud2 (pcl_pc2, *pcl_cloud); 

     uint8_t r = 0;
     uint8_t g = 0;
     uint8_t b = 0;
     int32_t rgb = (r << 16) | (g << 8) | b; 

     for (size_t i = 0; i < pcl_cloud->points.size (); ++i)
       if ((pcl_cloud->points[i].z > Z_MAX) or (pcl_cloud->points[i].z < Z_MIN))
         pcl_cloud->points[i].rgb = rgb;
     pcl::toPCLPointCloud2 (*pcl_cloud, pcl_pc2);
     sensor_msgs::PointCloud2 ros_pc2;
     pcl_conversions::fromPCL(pcl_pc2, ros_pc2);

     pcl::PCLImage pcl_image;
     pcl::toPCLPointCloud2 (pcl_pc2, pcl_image);  
     sensor_msgs::Image ros_image; 
     pcl_conversions::fromPCL(pcl_image, ros_image);

   int main(int argc, char **argv)
     ros::init(argc, argv, "PointCloud2Filter");
     ros::NodeHandle n;
     ros::Subscriber sub = n.subscribe("input/", 1000, cloudCallback);
     pub_cloud = n.advertise<sensor_msgs::PointCloud2> ("output_cloud/", 30);
     pub_image = n.advertise<sensor_msgs::Image> ("output_image/", 30);
     return 0;

And the launch file ...

    <include file="$(find RoboMix)/launch/myopenni.launch" />

  <node pkg="nodelet" type="nodelet" name="pcl_manager" args="manager" output="screen ...
edit flag offensive delete link more

Question Tools



Asked: 2014-04-23 01:33:38 -0500

Seen: 2,373 times

Last updated: May 06 '14