Issues in working with pointcloud and 2D feature detector [closed]

asked 2013-11-08 10:56:43 -0500

SM gravatar image

updated 2016-10-24 09:06:49 -0500

ngrennan gravatar image

I would like to know how I can apply point cloud RGB data with SURF features. With point cloud, I am subscribing to "camera/depth_registered/points", and in SURF /camera/rgb/image_rect. The objective is to do object recognition using the color and SURF features. So I had 2 approaches in mind (Approach1) Get a clean 3D segmented portion using point cloud and then work with these and 2D FURF features. So, based on the color information and the texture information obtained from point cloud and SURF, I can do object recognition. (Approach2): After the template matching is done nad the bounding box has been constructed, I go back to the template and extract the color information.

Any suggestions on how to go about this will be greatly appreciated. I am having a tough time in integrating point cloud with SURF. Can somebody please let me know how to combine them?

cloud_cb (const sensor_msgs::PointCloud2ConstPtr& input)

    pcl::fromROSMsg (*input, *cloud);   
    // ... do data processing

     std::cout << "PointCloud before filtering has: " << cloud->points.size () << " data points." << std::endl; //*

      // Create the filtering object: downsample the dataset using a leaf size of 1cm
      pcl::VoxelGrid<pcl::PointXYZRGB> vg;
      pcl::PointCloud<pcl::PointXYZRGB>::Ptr cloud_filtered (new pcl::PointCloud<pcl::PointXYZRGB>);
  vg.setInputCloud (cloud);
  vg.setLeafSize (0.01f, 0.01f, 0.01f);
  vg.filter (*cloud_filtered);
  std::cout << "PointCloud after filtering has: " << cloud_filtered->points.size ()  << " data points." << std::endl; //*

  // Create the segmentation object for the planar model and set all the parameters


main (int argc, char** argv)
  // Initialize ROS
  ros::init (argc, argv, "euclideanclusters");
  ros::NodeHandle nh;

  // Create a ROS subscriber for the input point cloud
  ros::Subscriber sub = nh.subscribe ("camera/depth_registered/points", 1, cloud_cb);

  // Create a ROS publisher for the output point cloud
  pub = nh.advertise<sensor_msgs::PointCloud2> ("output", 1);

  // Spin
  ros::spin ();


    : it_(nh_)
    // Subscrive to input video feed and publish output video feed
    image_sub_ = it_.subscribe("/camera/rgb/image_rect", 1, 
      &ImageConverter::imageCb, this);
    image_pub_ = it_.advertise("/image_converter/output_video", 1);





  void imageCb(const sensor_msgs::ImageConstPtr& msg)

 cv_bridge::CvImage cv_img;   // training image

 cv_bridge::CvImagePtr cv_ptr_frames;  // kinect frames

 cv::Mat objectMat = imread( "juice2575.png" );

 //  imshow( WINDOW1, object );

 cv_img.header.stamp = ros::Time::now();
 cv_img.encoding = "rgb8";
 cv_img.encoding = sensor_msgs::image_encodings::RGB8;
 cv_img.image = object;

  sensor_msgs::Image im;
  cv_img.toImageMsg(im);  //Conversion of Input image to ROS image

     cv_ptr_frames = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::BGR8);
    catch (cv_bridge::Exception& e)
     ROS_ERROR("cv_bridge exception: %s", e.what());
    cv:: Mat sceneMat;
   sceneMat = cv_ptr_frames;

   // SURF Object detection

        bool objectFound = false;
            float nndrRatio = 0.7f;
        //vector of keypoints  
        vector< cv::KeyPoint > keypointsO;
        vector< cv::KeyPoint > keypointsS;   

        Mat descriptors_object, descriptors_scene;     

       //-- Step 1: Extract keypoints
        SurfFeatureDetector surf(hessianValue);

        //-- Step 2: Calculate descriptors (feature vectors)
      SurfDescriptorExtractor extractor;
      extractor.compute( sceneMat, keypointsS, descriptors_scene );
      extractor.compute( objectMat, keypointso, descriptors_object );

      //-- Step 3: Matching descriptor vectors using FLANN matcher
      FlannBasedMatcher matcher; 
      descriptors_scene.size(), keypointsO.size(), keypointsS.size());
      std::vector< vector< DMatch ...
edit retag flag offensive reopen merge delete

Closed for the following reason PCL Question: The PCL community prefers to answer questions at by tfoote
close date 2015-11-19 20:05:35.142915