Revision history [back]

What sort of object is this that you want to perform pick and place on ? You need to achieve a reliable 3D detection of this object first. If the object have a primitive shape, PCL can be handy. I have not used Kinect so I don't know how precise/accurate is the pointclouds, In any case try to hold camera as close as possible to your object scene,

If your comfortable with CNNs, you can train a CNN model to detect your object on image, and then do a sensor fusion between image and your pointcloud to "reflect" detection information to pointclouds, this way you can get object clusters on point cloud.

After you get Object clusters in point cloud, You can construct a 3D box around each cluster, which will give you final 3D detection.

You may consider to use this repo for constructing a minimal volume 3D box around each object cluster. LidarPerception/object_builders_lib.

else you may just skip CNN part and use a clustering/segmentation algorithm to clusterize your pointcloud. You can do some prepossessing such as ground plane removal to make job of clustering algo easier.

again from LidarPerception, they have a great repo for clustering algorithms targeted for pointclouds LidarPerception/segmenters_lib

After you have 3D detectionS, transform pose of objects from camera frame to base_link, and then follow moveit pick and place pipeline tutorial here, you will likely modify very few things