The right way to use CameraInfo to correct barrel distortion
Environment: Kinetic, on Ubuntu 16, kernel 4.13.0-31-generic
OpenCV: 3.3.0
C++ code base
I have a USB camera which is easily publishing the raw images and the default camera_info topics, as generated by the image_transport helper methods. Looking at the image on RVIZ, there's clearly a barrel distortion. That's fine - that's what camera_calibration cameracalibrator.py is for. I ran through that, and at the end I saw a feed of the distortion-corrected frames. I was able to generate a calibration file. That's a YAML file that has the width and height, and then a camera matrix (3x3), a distortion (1x5), rectification (3x3 identity) and projection (3x4). Great. I can get this file into the camera_info topic and now I'm publishing all of the information needed to correct barrel distortion.
Now, how do I correct barrel distortion?
The "obvious" answer (because I'm new at ROS and ignorant of many packages) was to use cv_bridge and OpenCV. After some time passed, I ran into the same problem as the stack overflow poster linked here, where the remove fisheye distortion function was, instead, creating unusably distorted images. I have adopted his solution, which is to carry out each step that the remove-fisheye-distortion function allegedly wraps, and I now have an image, but with a different fisheye distortion.
Now that I've gotten that out of my system, it seems like there's something I'm missing here. Since there's a standard camera_info channel that carries the information you need to correct distortion, it seems like there should be a standard way to correct distortion that doesn't involve hand-wiring OpenCV. This is where my googling is coming up empty.
What is the "right" way to remove fisheye distortion in ROS? Is there a node that you just point your camera to, that subscribes to the /camera_info and /image_raw nodes and issues an /image_corrected topic? Is there a wrapper library that depends on OpenCV but will make the transformation for you if you provide the ImageConstPtr and CameraInfoConstPtr? Or is it the case that the best way to do this is using OpenCV yourself?
If you're aware of tutorials or example code for this kind of problem, I'd also be glad to see those.
