# down sample camera image on driver or image_proc level

In addition to producing higher resolution images than the wge100 cameras, the PR2's Prosilica also puts out images with much better color quality. In the perfect world, I would have a camera on the PR2 that can spew out images at the same frame rate of the wge100 cameras but with the image quality of the Prosilica.

I was wondering if there is a way to down sample the images from the Prosilica camera on the driver level or at least have it done by the image_proc node. We obviously down sample the received images in code but we are getting them at 2fps and would like to get them at a faster frame rate and at a lower resolution. If the images were to be down sampled by the driver or by the image_proc node, maybe the travel time would be shorter and we would get a faster frame rate? I don't actually know how fast the camera can spit out images.

Just to be clear, I know that you can crop the images using dynamic_reconfigure but I'm wondering whether it can down sample the entire image somehow. I'm a vision noob so I think this is probably feasible - I just don't know how to do it.

edit retag close merge delete

Sort by » oldest newest most voted

Your most immediate problem is likely that you're exposure-limited. At full resolution, the PR2's Prosilica can stream at 15fps. Since you're only getting 2fps, I'm guessing the environment is dark and auto-exposure is ballooning to ~0.5s to achieve an acceptably bright image.

Try playing with the exposure and gain settings - you can access these through dynamic_reconfigure (look for the /prosilica_driver group). Turn off auto_exposure and set the exposure time to 0.066 or less (for 15fps). Then adjust the gain level to get a sufficiently bright image, or let auto_gain do it for you. Gain is digital amplification; it gets you a brighter image at the cost of increased noise.

The downsampling can be done either in hardware or software.

#### Hardware Downsampling

This is called binning. When you use MxN binning, the camera combines each MxN block of pixels into one super-pixel. You can enable binning with the binning_x and binning_y parameters.

Besides reducing the image size, binning can increase frame rate in a couple of ways:

• For each frame there's less data going over the wire. At full resolution, the frame rate is bandwidth limited, so your max FPS goes up. For the PR2's Prosilica, with 2x2 binning you can achieve 25fps, and at 4x4 binning (612x512 image) you can get ~39fps.
• A pixel is basically an electron bucket, and by pooling adjacent buckets in hardware you get better sensitivity. So you can reduce exposure time and/or gain in low-light conditions.

There is one big drawback to hardware binning: you lose color. Color cameras generally work by tesselating a 2x2 color filter called a Bayer pattern over the sensor. Each pixel only detects one RGB channel. One of image_proc's duties is to interpolate (debayer) the raw image to get a full RGB image. But when you apply binning, each 2x2 Bayer element gets averaged into one monochrome super-pixel.

#### Software Downsampling

Downsampling in software gives you none of the frame rate and sensitivity advantages of hardware binning. You should only bother if you need a smaller image and at least one of the following is true:

• Your camera doesn't support hardware binning.
• You require color.

In ROS Electric, the image_proc package contains a new crop_decimate nodelet that applies software binning/ROI. It is not loaded as part of the standard image_proc node. You can try it out by doing:

rosrun nodelet nodelet standalone image_proc/crop_decimate camera:=prosilica camera_out:=prosilica_downsampled

rosrun image_view image_view image:=/prosilica_downsampled/image_raw


Then open /image_proc_crop_decimate in reconfigure_gui and set decimation_x and decimation_y to 2.

#### Conclusion

You may not be able to get everything you want from the PR2 Prosilica. If you don't need color, use hardware binning. If you need color but can live with 15fps, use software downsampling. If you absolutely need color at 30fps, the GC2450C is simply the wrong camera. Prosilica does have cheaper cameras at VGA resolution.

more

Thanks a lot Patrick! I really appreciate the detailed response. I do need color because I'm doing color blob detection so I'll try the software downsampling.
( 2011-09-22 09:20:44 -0500 )edit