Order of point clouds to use while merging (ICP)
My team and I have built the following setup (view from above). It's a 3x3x3m aluminum grid with 4 Kinects placed at height 2.1~2.2m and are facing downwards (at 20~30 degrees). The angle between the neighbors is 90 degrees. At this point I need to merge point clouds from all these cameras (camera calibration, manual transformations have been applied). The question is: do I merge them to one single camera one by one, or follow clockwise/counterclockwise direction, or there is a better way to merge them?
UPDATE: if anyone is interested, I calibrated 3 cameras to 1, then followed the procedure of merging the clouds that showed the least error during the calibration process.
Asked by Mikhail K. on 2018-07-16 06:13:11 UTC
Comments
I don't understand the problem if the depth cameras are calibrated and their extrinsic poses are known then you don't need to use ICP at all! You can simply transform the points into the same reference frame and add them to a single output cloud. What am I missing?
Asked by PeteBlackerThe3rd on 2018-08-07 07:10:25 UTC
@PeteBlackerThe3rd calibration of 4 cameras at this angle turned out to be a serious problem. A couple of calibration toolboxes (MATLAB, OpenCV) gave insane errors. I had to manually align the clouds in CloudCompare software and apply ICP afterwards
Asked by Mikhail K. on 2018-08-07 07:14:02 UTC
So you're using ICP to calibrate the poses of the cameras? I would expect a poor overlap given the positions of the sensors. Have you tried using a known shape such as a large cube for calibration and registering to the known shape?
Asked by PeteBlackerThe3rd on 2018-08-07 07:26:10 UTC
I've calibrated using multiple matching points via CloudCompare and the error is now around 1-2 cm, which is more than enough for my project. As for ICP, I'm not using PCL's ICP, I've ported non-rigid one from cilantro and it works great even for rapidly moving hands. PCL's ICP performed poorly
Asked by Mikhail K. on 2018-08-07 22:33:10 UTC
Interesting. If you're scanning rapidly moving objects then there will probably by a small difference in time between each scan because the sensors aren't synchronised. This would explain why non rigid ICP is producing better results.
Asked by PeteBlackerThe3rd on 2018-08-08 13:30:59 UTC
I use rosbag to record data and then I additionally apply message_filters::sync_policies::ApproximateTime, which decreases the sync error to about 1 ms.
Asked by Mikhail K. on 2018-08-08 22:25:12 UTC
How can a message filter decrease the sync error? If they are captured at different times then the temporal offset is fixed.
Asked by PeteBlackerThe3rd on 2018-08-09 02:15:29 UTC
Timestamps of messages from 4 cameras in my rosbag recordings can differ up to 2-3 ms before applying the filter, so they are not 100% synchronized.
Asked by Mikhail K. on 2018-08-09 02:24:12 UTC