Model Free 3D Object Detection & tracking PCL

edit

There are some tools in the Point Cloud Library that could be useful.

If the pose of the sensor is fixed and known, you could start by throwing away any points that aren't of interest like the background or any areas that don't contain the conveyor or objects using a passthrough filter.

Then you can detect the plane that belongs to the conveyor surface and throw those points away using plane segmentation.

After that you should be left with a point cloud that (roughly) only contains points that belong to objects on the conveyor. In order to group the points of individual objects into their own smaller point clouds you could use Euclidean Cluster Extraction. If everything goes right, you should be left with a bunch of little point cloud, each of which belongs to a single object on the conveyor.

I don't know if you need to, but if you do, you could probably then try and fit a box shape to each cloud - by that I don't mean compare a bunch of box models to the clouds, I mean assume that each cloud is a box and try to figure out what the dimensions of the boxes are using some of the tools in the sample consensus library. I don't know if there is a function specialized in fitting box shapes in Point Cloud Library, but you could probably do something like try to find the planes that make up the sides of the box, and make your own function that takes a few planes as inputs and outputs the height, width, and length of boxes, and even the X,Y,Z position of the center of the box and the box's orientation. Or maybe this is as simple as finding the oriented bounding box of the small point clouds(but keep in mind the sensor wouldn't see all sides of the box), you could probably find an oriented bounding box code on the internet somewhere.