Implementing RegionOfInterest and do_rectify in camera driver for use with image_proc?
I'd like to implement support for RegionOfInterest for my camera driver but I'm not sure how to do it correctly. I'd like image_proc to be able to subscribe to the roi image and operate correctly, though it's not clear what correct operation would look like.
There is some helpful description at http://www.ros.org/reps/rep-0104.html... 'Raw and Rectified ROI' section but it isn't always clear where responsibility lies for implementing a feature, and what numbers should be published in CameraInfo for use by image_proc.
If do_rectify is true, does that mean the associated roi is in rectified coordinates? Should I calculate a much bigger raw roi to send to the camera using the bounding box of the distorted desired roi boundary to make sure there are no black pixels when the undistort happens later, but then publish the original desired roi on the camera_info topic for use by image_proc which will cause image_proc to select a window within the undistorted image that has no border? But then how would image_proc know where to place the distorted image in the distorted coordinate frame? It seems like there isn't enough information present (unless image_proc is running it's own distort/undistort to get a mapping).
Right now I'm trying to set the camera_info roi to be the distorted roi sent to the camera, but for some reason I have to divide the camera_info roi x and y offset by 2 otherwise the image will get increasingly offscreen for larger x and y offset (could be a bug on my part), and I'm still not eliminating the undistort border- that could also be me improperly generating the distorted roi, but I still suspect I'm publishing the wrong thing to camera_info.
The way I like to imagine the process is that image_proc would create a large black image the size of the sensor width and height, place the subwindow image from the camera into it using distorted coordinates, then perform an undistort on the whole image, then window out a new roi image where the user originally requested and publish that. But again it seems like two sets of xy offsets have to change hands from camera driver to image_proc to accomplish that- or is the responsibility of some other node downstream to compute the window within the image_proc generated rectified image to match the desires of the original roi request with do_rectify set to true?
Should I be able to toggle do_rectify on and off and see the same parts of the image that are visible stay put, while the distortion border moves around?
