Identifying colored spherical objects from 3D point cloud from SLAM map
I am trying to identify spherical objects from a 3D point cloud generated using Localization and Mapping for Solid-State LiDAR (SSL-SLAM). For example, suppose I have a yellow ball in the map that SSL-SLAM generates. I am trying to identify this and only this object in my 3D point cloud, meaning all other points in the point cloud must be discarded.
I think this comes down to what queries I can do on a (XYZRGB) point cloud. I am working with a Python script in ROS, with the SSL-SLAM library written in C++. I did not modify any C++ nodes from SSL-SLAM; I decided to add another Python node from which I can modify the pointcloud coming from the '/map' topic and publish a modified pointcloud to a topic which I name '/map2'. I can visualize '/map2' in RViz. This '/map2' topic needs to contain the pointcloud that contains only yellow balls.
I want to use the 2D RGB frames somehow to first identify only circular yellow objects using OpenCV, then find the corresponding 3D pointcloud points, then apply a Gaussian Mixture Model to check if the GMM parameters of the 3D yellow points match that of a yellow sphere (I will need GMM parameters for a yellow sphere beforehand).
However, as I mentioned, I do not have a way of finding correspondences between a 2D RGB image and the 3D point cloud of the generated map. So I thought of the following:
Since I am working only with a (XYZRGB) point cloud, I will first project the 3D point onto the 2D camera plane. I will do the OpenCV identification mentioned before, here. I will then find the 2D camera plane points corresponding to the 3D point cloud and find the GMM parameters from there. The image below shows what I am trying to do:
The above image is from Projection, Texture-Mapping and Occlusion with Intel® RealSense™ Depth Cameras. What is the best way to find the yellow balls only from a 3D point cloud? Please help
