# How can I speed up image rectification?

I hope to increase the rectified frame rate that I'm getting from image_proc. I'm feeding it a 64 fps 704x456 video stream, with that 704x456 rectangle being the region of interest sampled from a 1600x1200 sensor. The camera calibration is specified for the full-sensor size, and I'm using the Diamondback version of image_proc.

$rostopic hz /camera/image_raw average rate: 64.023$ rostopic hz /camera/image_color
average rate: 64.113
$rostopic hz /camera/image_rect_color average rate: 27.558$ top
767 user      20   0  587m 109m  11m S  *115*  0.9 132:37.80 image_proc


Using earlier versions of image_proc and camera1394, I was able to get 38 fps rectified by choosing an MxN subrectangle [slightly larger than the current ROI] and calibrating with that. In that case, the Image and CameraInfo messages would both specify MxN pixels. Now, though, camera_info lists the full 1624x1224, and I suspected that something inside the rectify nodelet was copying my ROI onto a full-size image before undistorting anything.

$rostopic echo /camera/image_raw height: 456 width: 704 encoding: bayer_rggb8 is_bigendian: 0 step: 704  However, when I remove the ROI the undistorted framerate drops further: $ rostopic hz /camera/image_color
average rate: 29.145
\$ rostopic hz /camera/image_rect_color
average rate: 19.497


so I'm unsure of the cause of the slowdown. Can anyone see what's going on?

edit retag close merge delete

Sort by » oldest newest most voted

Note that the behavior for rectifying ROI images has changed in Diamondback. The Cturtle behavior was actually somewhat broken, because it did not account for the fact that the chosen ROI in the raw image may correspond to a different (perhaps not even overlapping!) ROI in the rectified image coordinates due to distortion (ticket #4206). The new, regrettably more complicated behavior is documented in REP 104. image_geometry (used by image_proc) now calculates a distinct rectified ROI that maps well onto the pixels in the raw ROI image.

So the key question is: what is the resolution of /camera/image_rect_color? If it is larger than 704x456, that may explain some drop in frame rate.

In any case, can you open a ticket against image_pipeline and attach to it a small bag of the /camera/image_raw and /camera/camera_info topics? I can double-check if there has been any performance regression, and it will be nice to have test data from another camera.

more

What about using the new nodelets instead of different processes for your image pipeline, it should increase the processing speed as no serialization will be required?

more