Ask Your Question

How to rectify downsampled stereo images using binning?

asked 2013-07-11 22:50:30 -0500

timster gravatar image

updated 2014-01-28 17:17:14 -0500

ngrennan gravatar image

Hi there!

This is my first post! (So far I found all the answers I needed, which is awesome!)

I am writing my own node for rectifying stereo images. The reason I do this is to be able to rectify/undistort only every n-th frame and also to downsample the images right at the beginning, both to reduce computational cost.

So far, I manage to rectify images at the full resolution and then downsample them. I suspect that first downsampling and then rectifying them would be faster, which I was trying to get to work. Looking through the code of PinholeCameraModel, it seemed to me that when I would set the binning_x and binning_y fields in the camera_info to let's say 2, the PinholeCameraModel would compute the rectification map for downsampled images. Unfortunately, this gave weirdly warped images. What am I missing? What else needs to be set?

Thanks in advance!

Here is my callback function:

void process(const sensor_msgs::ImageConstPtr& l_image_msg, const sensor_msgs::ImageConstPtr& r_image_msg, const sensor_msgs::CameraInfoConstPtr& l_info_msg, const sensor_msgs::CameraInfoConstPtr& r_info_msg){
if (count_ == 0) // The count_ stuff is there not to process every frame but to skip some in between
  // Prepare images
  int32_t l_step, r_step;
  cv_bridge::CvImageConstPtr l_cv_ptr, r_cv_ptr;
  l_cv_ptr = cv_bridge::toCvShare(l_image_msg, sensor_msgs::image_encodings::MONO8);
  l_step = l_cv_ptr->image.step[0];
  r_cv_ptr = cv_bridge::toCvShare(r_image_msg, sensor_msgs::image_encodings::MONO8);
  r_step = r_cv_ptr->image.step[0];

  ROS_ASSERT(l_step == r_step);
  ROS_ASSERT(l_image_msg->width == r_image_msg->width);
  ROS_ASSERT(l_image_msg->height == r_image_msg->height);

  cv_bridge::CvImagePtr l_image_rect(new cv_bridge::CvImage(l_cv_ptr->header, l_cv_ptr->encoding));
  cv_bridge::CvImagePtr r_image_rect(new cv_bridge::CvImage(r_cv_ptr->header, r_cv_ptr->encoding));

  // Downsample if desired
  if (config_.downsample)
    // Copy cam info to be able to modify
    sensor_msgs::CameraInfo l_info_msg_new(*l_info_msg);
    sensor_msgs::CameraInfo r_info_msg_new(*r_info_msg);

    // Modify cam info
    l_info_msg_new.binning_x = 2;
    l_info_msg_new.binning_y = 2;
    r_info_msg_new.binning_x = 2;
    r_info_msg_new.binning_y = 2;

    // Update camera models

    // Downsample
    cv_bridge::CvImagePtr l_image_ds(new cv_bridge::CvImage(l_cv_ptr->header, l_cv_ptr->encoding));
    cv_bridge::CvImagePtr r_image_ds(new cv_bridge::CvImage(r_cv_ptr->header, r_cv_ptr->encoding));
    cv::resize(l_cv_ptr->image, l_image_ds->image, cv::Size(0, 0), 0.5, 0.5, CV_INTER_LINEAR);
    cv::resize(r_cv_ptr->image, r_image_ds->image, cv::Size(0, 0), 0.5, 0.5, CV_INTER_LINEAR);

    // Rectify
    l_model_.rectifyImage(l_image_ds->image, l_image_rect->image, CV_INTER_LINEAR); 
    r_model_.rectifyImage(r_image_ds->image, r_image_rect->image, CV_INTER_LINEAR); 
    // Update camera models

    // Rectify
    l_model_.rectifyImage(l_cv_ptr->image, l_image_rect->image, CV_INTER_LINEAR); 
    r_model_.rectifyImage(r_cv_ptr->image, r_image_rect->image, CV_INTER_LINEAR); 

  // Publish

else if (count_ >= config_.frames_to_skip) // >= because config_.frames_to_skip can change
  count_ = 0;

Best Tim

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2013-07-12 03:07:33 -0500

timster gravatar image

It seems that this could in fact be a bug in PinholeCameraModel:

If binning is set to 2 from the start, rectifying the downsampled image works as you would expect it to. If you switch binning to 1, rectifying the original (non-downsampled) image works as well. If you then switch binning back to 2, you get crazy stuff.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower


Asked: 2013-07-11 22:50:30 -0500

Seen: 601 times

Last updated: Jul 12 '13