Ask Your Question
0

depth data retrieved by image_transport has confusing format

asked 2019-02-08 08:46:51 -0500

Ferguth gravatar image

updated 2019-02-14 03:01:12 -0500

  • OS: Ubuntu 16.04 LTS
    • ROS distro: kinetic

Hey, I am reading depth data from a Microsoft Kinect using the freenect package.

My Node:

#include <ros/ros.h>
#include <iostream>
#include <sensor_msgs/CompressedImage.h>
#include <image_transport/image_transport.h>
#include <cv_bridge/cv_bridge.h>
#include <typeinfo>
#include <vector>

void depthCallback(const sensor_msgs::Image::ConstPtr& msg)
  {
  sensor_msgs::ImagePtr msg_c(new sensor_msgs::Image);

  msg_c->data = msg->data;
  msg_c->encoding = msg->encoding;
  msg_c->header = msg->header;
  msg_c->height = msg->height;
  msg_c->width = msg->width;
  msg_c->is_bigendian = msg->is_bigendian;
  msg_c->step = msg->step;

  std::cout << msg_c->encoding << std::endl;
  std::cout << msg_c->height << std::endl;
  std::cout << msg_c->width << std::endl;
  std::cout << msg_c->step << std::endl;
  std::cout << msg_c->data.size() << std::endl;
  std::cout << typeid(msg_c->data).name() << std::endl;
  }

int main(int argc, char **argv)
  {
  ros::init(argc, argv, "depth");
  ros::NodeHandle nh;
  image_transport::ImageTransport it(nh);
  image_transport::Subscriber sub = it.subscribe("/camera/depth_registered/image_raw/", 1,depthCallback, image_transport::TransportHints("compressed"));
  //image_transport::Subscriber sub = it.subscribe("/camera/depth_registered/image_raw/", 1,depthCallback);
  ros::spin();

  return 0;
  }
  • Encoding is 16UC1, as expected after http://wiki.ros.org/freenect_camera.
  • Height and width are 480 and 640, as expected.
  • Step returns 640.
  • data.size() returns only 307200, I expected 614400 since 2 bytes should code for one float value per pixel.
  • typeid(msg_c->data).name()) returns St6vectorIhSaIhEE. It being a vector is confirmed by trying to access an index > 307199 with .at(), which throws an 'out of bounds' exception.

Question: What is the content of my data field representing at this moment? Cause it doesn't seem to be 16UC1 coded float values.


EDIT:

I used cv_bridge and checked the encoding of the resulting cvImage pointer:

cv_ptr = cv_bridge::toCvCopy(msg_c, sensor_msgs::image_encodings::TYPE_16UC1);
std::cout << "Encoding: " << cv_ptr->encoding << std::endl;

This throws following exception:

terminate called after throwing an instance of 'cv_bridge::Exception'
  what():  Image is wrongly formed: step < width * byte_depth * num_channels  or  640 != 640 * 2 * 1

It looks like my missing bytes are here, but not in a format i expected. Will look into this tomorrow.


EDIT 2:

Going through cv_bridge.cpp shows that byte_depth is calculated in sensor_msgs::image_encodings given the encoding of the source image. As mentioned in the comments by PeteBlackerThe3rd, the step size needs to be twice as big. Probably need to dig into the freenect code.

edit retag flag offensive close merge delete

Comments

I agree with your conclusion so far. Another way to investigate this would be to use an existing ROS node that processes depth images and see if that produces errors when using the same messages. This would confirm that it's a problem with that version of the freenect node.

PeteBlackerThe3rd gravatar imagePeteBlackerThe3rd ( 2019-02-14 03:27:48 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
2

answered 2019-02-08 09:19:11 -0500

16UC1 is not a floating point format. It is an unsigned 16 bit integer format, and by convention for depth images they are in units of mm. The step and data size should be twice what they are at first glance though.

I would highly recommend using the cv_bridge package to extract the content of sensor_msgs/Image messages. It will take care of all this for you so you don't need to worry about how the data is encoded.

edit flag offensive delete link more

Comments

1

I would highly recommend using the cv_bridge package to extract the content of sensor_msgs/Image messages. It will take care of all this for you so you don't need to worry about how the data is encoded.

yes.

+1 to this.

gvdhoorn gravatar imagegvdhoorn ( 2019-02-08 09:21:38 -0500 )edit

Thank you for your responses! I update my post.

Ferguth gravatar imageFerguth ( 2019-02-13 09:14:39 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2019-02-08 08:46:51 -0500

Seen: 99 times

Last updated: Feb 14