Align Point clouds from multiple kinects
Hi, I am using two Kinects to capture an object. In this scenario the cameras are rigidly attached to a surface and their poses are also fixed. I have obtained the relative camera poses using "camera_pose_calibration" package, calibrated for rgb images. I want to visualize the point clouds from both Kinects in Rviz but the point clouds i obtain are not aligned. In some cases I get one of the point clouds completely inverted with respect to the other when viewed from the /world as fixed frame in Rviz. In other cases the alignment is not satisfactory.
How can i achieve reasonable alignment between the point clouds from multiple Kinects and visualize the result in Rviz?
Additional info: I am using ROS fuerte and platform is Ubuntu 12.04. In Rviz I am displaying PointCloud2 and the depth registration is turned on so the topics are /kinect1/depth_registered/points and /kinect2/depth_registered/points respectively.
Are you able to see both the /kinect1 and /kinect2 tf frames simultaneously?
yes. i would upload the result of tf view_frames here but i dont have enough karma so I uploaded it on Dropbox here https://www.dropbox.com/s/cvn1nilsn089xyo/calibration_2Kinects.pdf
I think your problem is related to the fact that you have three separate tf trees. The point clouds are defined in terms of the depth_frame, but you only have the rgb_frames' position relative to world. It is reasonable to expect that this could be easily resolved, but unfortunately it isn't...
Ideally you would get the tf between /world and /kinectX_rgb_frame and things would connect in a graph-like form, but that's not possible currently. We can only have one parent to each child, as seen here: http://answers.ros.org/question/56646/relative-tf-between-two-cameras-looking-at-the-same-ar-m
That's true. It can also be verified. The /world is arbitrarily chosen, and sometimes during the configuration it was aligned perfectly with one of the /kinect frames. i.e. the /world and /kinect1 had the same coordinates and orientation. At the point the point clouds were perfectly aligned.
So do you have a work around? how can I align multiple point clouds using camera_pose_calibration? As I understand you are also trying to achieve similar results. Were you able to achieve that using multiple AR markers to estimate camera poses relative to each other? Any help would be appreciated.
I haven't continued in that direction yet because I have been involved in some other developments in my lab, but you should be able to write a node that reads in all clouds you are interested in merging, read all relevant transforms and publish a single cloud on a different topic.