depth data retrieved by image_transport has confusing format
- OS: Ubuntu 16.04 LTS
- ROS distro: kinetic
Hey, I am reading depth data from a Microsoft Kinect using the freenect package.
My Node:
#include <ros/ros.h>
#include <iostream>
#include <sensor_msgs/CompressedImage.h>
#include <image_transport/image_transport.h>
#include <cv_bridge/cv_bridge.h>
#include <typeinfo>
#include <vector>
void depthCallback(const sensor_msgs::Image::ConstPtr& msg)
{
sensor_msgs::ImagePtr msg_c(new sensor_msgs::Image);
msg_c->data = msg->data;
msg_c->encoding = msg->encoding;
msg_c->header = msg->header;
msg_c->height = msg->height;
msg_c->width = msg->width;
msg_c->is_bigendian = msg->is_bigendian;
msg_c->step = msg->step;
std::cout << msg_c->encoding << std::endl;
std::cout << msg_c->height << std::endl;
std::cout << msg_c->width << std::endl;
std::cout << msg_c->step << std::endl;
std::cout << msg_c->data.size() << std::endl;
std::cout << typeid(msg_c->data).name() << std::endl;
}
int main(int argc, char **argv)
{
ros::init(argc, argv, "depth");
ros::NodeHandle nh;
image_transport::ImageTransport it(nh);
image_transport::Subscriber sub = it.subscribe("/camera/depth_registered/image_raw/", 1,depthCallback, image_transport::TransportHints("compressed"));
//image_transport::Subscriber sub = it.subscribe("/camera/depth_registered/image_raw/", 1,depthCallback);
ros::spin();
return 0;
}
- Encoding is 16UC1, as expected after http://wiki.ros.org/freenect_camera.
- Height and width are 480 and 640, as expected.
- Step returns 640.
- data.size() returns only 307200, I expected 614400 since 2 bytes should code for one float value per pixel.
- typeid(msg_c->data).name()) returns St6vectorIhSaIhEE. It being a vector is confirmed by trying to access an index > 307199 with .at(), which throws an 'out of bounds' exception.
Question: What is the content of my data field representing at this moment? Cause it doesn't seem to be 16UC1 coded float values.
EDIT:
I used cv_bridge and checked the encoding of the resulting cvImage pointer:
cv_ptr = cv_bridge::toCvCopy(msg_c, sensor_msgs::image_encodings::TYPE_16UC1);
std::cout << "Encoding: " << cv_ptr->encoding << std::endl;
This throws following exception:
terminate called after throwing an instance of 'cv_bridge::Exception'
what(): Image is wrongly formed: step < width * byte_depth * num_channels or 640 != 640 * 2 * 1
It looks like my missing bytes are here, but not in a format i expected. Will look into this tomorrow.
EDIT 2:
Going through cv_bridge.cpp shows that byte_depth is calculated in sensor_msgs::image_encodings given the encoding of the source image. As mentioned in the comments by PeteBlackerThe3rd, the step size needs to be twice as big. Probably need to dig into the freenect code.
I agree with your conclusion so far. Another way to investigate this would be to use an existing ROS node that processes depth images and see if that produces errors when using the same messages. This would confirm that it's a problem with that version of the freenect node.