LiMuBei's profile - activity

I'm trying to register two pointclouds gathered with Kinect. The two pointclouds are of type PointXYZRGB and are pre-aligned. From the signature of the function I assumed the following usage:

pcl::PointCloud<pcl::PointXYZRGB> A, B; // imagine these to be the input clouds, where A should be aligned to B
pcl::registration::TransformationEstimation<pcl::PointXYZRGB,pcl::PointXYZRGB> MyEstimation;
Eigen::Matrix4f T; // the result transformation
MyEstimation.estimateRigidTransformation( A, B, T );

This crashes in Ubuntu 10.10 x64 using diamondback with an assertion in Eigen:

Eigen::ProductBase<Derived, Lhs, Rhs>::ProductBase(const Lhs&, const Rhs&) [with Derived = Eigen::GeneralProduct<Eigen::Matrix<float, -0x00000000000000001, -0x00000000000000001, 0, -0x00000000000000001, -0x00000000000000001>, Eigen::Transpose<Eigen::Matrix<float, -0x00000000000000001, -0x00000000000000001, 0, -0x00000000000000001, -0x00000000000000001> >, 5>, Lhs = Eigen::Matrix<float, -0x00000000000000001, -0x00000000000000001, 0, -0x00000000000000001, -0x00000000000000001>, Rhs = Eigen::Transpose<Eigen::Matrix<float, -0x00000000000000001, -0x00000000000000001, 0, -0x00000000000000001, -0x00000000000000001> >]: Assertion `lhs.cols() == rhs.rows() && "invalid matrix product" && "if you wanted a coeff-wise or a dot product use the respective explicit functions"' failed.

So I'm not sure what's going wrong? Am I using the registration wrong? The pointclouds are okay, as I can concatenate and display them fine without this fine registration.

I'm using the RGBDSLAM from http://www.ros.org/wiki/rgbdslam to create a map with my Kinect. The general ICP implementation provided is a bit buggy so I wanted to replace that with the ICP from PCL. Used the tutorial and everything compiles fine. But when executed the ICP does not converge but outputs this error message:

[pcl::IterativeClosestPoint::computeTransformation] Unable to find a nearest neighbour in the target dataset for point 3 in the source!

I'd like to narrow down the cause for this, so if anyone has any experience with the parameters of the PCL ICP, I'd really appreciate if you share them! Just so I have an idea what parameters should work for this scenario. If I can rule out wrong parameters (especially max iterations and distance) I can dig into the pointclouds themselves to see if I am somehow messing up target and input cloud or have the wrong transformations or whatever.

I'm using ROS diamondback on Ubuntu 10.10 32bit with the provided PCL.

UPDATE: I tried several sets of parameters, though nothing seems to impact this. I have saved out two input pointclouds that I try to align with the icp, you can download them here if you like. I'm pretty much out of ideas on this. Unfortunately the documentation is rather lacking...

To further clarify what I'm doing:

pcl::IterativeClosestPoint<pcl::PointXYZRGB,pcl::PointXYZRGB> icp;
icp.setInputCloud(inputcloud.makeShared());
icp.setInputTarget(targetcloud.makeShared());
pcl::PointCloud<pcl::PointXYZRGB> final;
icp.align(final);

The above download contains inputcloud and targetcloud as they are saved right before icp.align().

UPDATE2: Using the ICP test sample program and the provided sample files bun0.pcd and bun4.pcd I found out that the problem seems to be specific to my pointclouds as those samples work fine. Are there any preconditions that the pointclouds must fulfill for ICP to work?

Let's say I have a series of images and want to create a bag file out of them to use as input for the camera calibration.

Is it even necessary to create the bag for the calibration or is there another way to use the images?

If not, can a bag file be created from an image sequence? I guess using some python-fu it should be possible. Would that be a good idea?

UPDATE: Using the bag file for calibration seems not feasible as the standard calibration app insists on talking back to the camera. So to use that one will need to hack the script. The other possibility that might be working (testing atm) is a not documented script "tarfile_calibration.py" which can read a tar file containing the image sequence. Simply create a tar file with numbered images in format "left%d.png" and "right%d.png" and you're good to go:

rosrun camera_calibration tarfile_calibration.py MyImages.tar --square 0.108 --size 8x6

The material errors you are getting is because Ogre (rendering engine of rviz) cannot render the pointcloud material on your graphics hardware. Maybe try upgrading your driver or your hardware. Alternatively you can try to change the material definition, maybe you find something that works with your hardware.

Check the Ogre.log file, it should contain more information on the error.

Ok, since I can also answer my own question, I'll do just that. Apparently the pointclouds I got from the Kinect contained some NaN or Inf values, which the ICP doesn't like. It's kinda obvious that the nearest neighbour search with those values cannot really succeed, a better error message in that case would've saved some time :-)

Anyway, to fix that one can apply a simple passthrough filter to the Kinect pointcloud like this:

pcl::PassThrough<pcl::PointXYZRGB> pass; // can do this without parameters
pass.setInputCloud( input );
pass.filter( output );

You'll have to do this for both, input and target cloud. Then give the filtered clouds to the ICP.

Alternatively one can use pcl::removeNaNFromPointCloud.

If someone knows a more elegant way to get rid of the NaN or Inf points than that - feel free to edit this post :-)