Ask Your Question
4

What is the actual size of point cloud from a kinect.

asked 2011-05-08 23:05:48 -0500

enddl22 gravatar image

updated 2016-10-24 09:07:11 -0500

ngrennan gravatar image

Hello, all.

As far as I know a kinect provides 640x480x3 size of 2D image data (widthxheightxRGB) and 640x480x(sizeof a point in 3D).

Is there anyone who knows the actual size of a point cloud?

640x480x4x3 ? widthxheightx(float)x(x,y,z) ?

Cheers.

edit retag flag offensive close merge delete

5 Answers

Sort by » oldest newest most voted
13

answered 2011-05-09 20:42:29 -0500

updated 2011-06-08 21:20:40 -0500

The data payload is 640 x 480 x 8 x sizeof(float) bytes = 9830400 bytes

Plus some bytes for auxiliary information like origin, timestamp etc.

A good way to check out the bandwidth required for transmission is rostopic bw

$ rostopic bw /camera/rgb/points
subscribed to [/camera/rgb/points]
(...)
average: 295.37MB/s
    mean: 9.83MB min: 9.83MB max: 9.83MB window: 100

To find the size in memory consider the following analysis:

I store them as pcl::pointcloud. The data for one point is described in this header file, although it is somewhat cryptic. PointXYZRGB is described as follows:

00204 struct PointXYZRGB
00205 {
00206   PCL_ADD_POINT4D;  // (...)
00207   union
00208   {
00209     struct
00210     {
00211       float rgb;
00212     };
00213     float data_c[4];
00214   };
00215   EIGEN_MAKE_ALIGNED_OPERATOR_NEW
00216 } EIGEN_ALIGN16;

PCL_ADD_POINT4D and the union below will add 4 floats each. While that is overkill memorywise, it is required to benefit from some cpu optimizations. Therefore for a pcl::pointcloud<pcl::PointXYZRGB> from the kinect, you store 640 x 480 x 8 x sizeof(float) bytes.

I wonder why rgb is in a separate struct though. Does anybody know this.

For a PointCloud2 you can look at the message with rostopic echo -n 1 /camera/rgb/points| less. The interesting parts are:

height: 480
width: 640
point_step: 32

Therefore, also here you have 640 x 480 x 32, where 32 is 8 x sizeof(float)

Optimization Hack

A colleague of mine modified the openni_camera driver and rgbdslam to communicate the color value via the unused forth dimension of the pointXYZ. This essentially cuts the required memory to half the size. It is a little hackish, but if anyone wants the patch, you are welcome to contact me. Keep in mind though, that linear algebra operations with Eigen might be affected by the color value or overwrite it. Haven't experienced that yet though. The bandwidth reduction shown by rostopic bw

average: 148.35MB/s
        mean: 4.92MB min: 4.92MB max: 4.92MB window: 100
edit flag offensive delete link more
6

answered 2011-05-10 06:59:00 -0500

It's probably worth pointing out that the Kinect itself does not output point clouds, but rather a depth map. That is, you can think of the Kinect output as being a 640x480 array (that's 307,200 entries) where each entry contains the Z-axis value of the point in the world corresponding to that entry. IIRC this Z-value is encoded with 11 bits but stored in 16 bits, so the size of the (depth, excluding colour) dataset coming off of the sensor is 307,200*2 = 614,400 bytes. At 30 fps that's about 18 MB/sec. However, when you project these depth values to 3D to get 3 floats per entry in the array the size of the dataset of course expands greatly.

The point is, depending on your application, you might not need all of the depth information projected to 3D coordinates, at the full 30 fps. I don't know what your application is but here's a made-up example. Say you want to grab 3d models of faces that are within the Kinect's view. Your first stage is to use the (RGB) camera data to detect bounding boxes of faces. Then you want to find out the 3D points that are associated with those bounding boxes. This means that you really only want to project the depth data that is associated with the bounding boxes, not all of it.

BTW with the openni_camera driver there is a way to do this; in the current (Diamondback) driver you can set the use_indices parameter, and then publish pcl/PointIndices messages to camera/depth/indices. The pointclouds output on camera/depth/points will then only contain projected points corresponding to the indices you asked for (i.e. the ones that correspond to the bounding boxes in the example above). This is what I did for my quadrotor kinect demo, to cut down the amount of 3D data to a reasonable level so that the onboard CPU could handle the calculations at the full 30 Hz.

The newer openni_camera driver (openni_camera_unstable) looks like it provides more flexibility still -- you can subscribe directly to the raw depth information and do whatever you want with it.

edit flag offensive delete link more

Comments

Were you using the AscTec Atom board in the Kinect altitude hold experiment (could you please provide some more info regarding the onboard computer used when not)? How much data did you have to leave out in order to achieve real-time operation and what resolution of depth / RGB data were you using?
tom gravatar imagetom ( 2011-05-11 01:50:49 -0500 )edit
Yes, the Atom board. Full resolution of depth data -- RGB data not used at all. Used about 10% of the overall point cloud (at least in the initial working implementation where mask was just a rectangle, later was more complex and I'd have to look into how many points actually get used). HTH, Cheers.
Patrick Bouffard gravatar imagePatrick Bouffard ( 2011-05-11 03:54:55 -0500 )edit
Patrick, I am trying to use the use_indices method to cut the bandwidth, but cannot get it to work. Would you be so kind as to have a look at my question at http://answers.ros.org/question/2343/how-to-use_indices-with-kinectasus ? Much appreciated!
prp gravatar imageprp ( 2011-10-04 01:07:24 -0500 )edit
0

answered 2011-05-09 23:32:34 -0500

enddl22 gravatar image

updated 2011-05-18 23:58:45 -0500

Thank you Felix.

I can understand very easily.

Almost 9Mbyte per frame...

It's quite big data and need many computational power.

9.83*30=294MBytes / sec...

Thank you again.

PS: How can you put source code on the web page?

edit flag offensive delete link more

Comments

For code markup, there is a button above the text field, showing 0s and 1s. You could put the same data into 4 floats instead of 8, I don't know why that is not done by pcl, though.
Felix Endres gravatar imageFelix Endres ( 2011-05-10 03:36:42 -0500 )edit
It's best either to edit your original question or use comments when willing to add / request more info or just thank someone. Answers are being ordered by thumbs up and correctness, so with time it will not necessarily be clear who you were talking to otherwise. Also please accept best answer.
tom gravatar imagetom ( 2011-05-11 01:46:19 -0500 )edit
Oh. Next time I will put request using comments. I made a mistake because I am not used to it. Thank you tom.
enddl22 gravatar imageenddl22 ( 2011-05-19 00:01:27 -0500 )edit
You still haven't accepted any answer, though :).
tom gravatar imagetom ( 2011-05-20 05:14:01 -0500 )edit
tom/ Could you please tell me how can i accept answers? I can't find it.
enddl22 gravatar imageenddl22 ( 2011-05-20 06:29:48 -0500 )edit
@enddl22: Next to each answer, under the thumbs up / down there is a check (saying "mark this answer as favorite" when you drag your mouse over it). Just click on it. Obviously this only applies to questions you ask. Cheers.
tom gravatar imagetom ( 2011-05-23 00:15:19 -0500 )edit
0

answered 2011-05-24 02:57:44 -0500

enddl22 gravatar image

updated 2011-05-24 03:00:48 -0500

Could you please tell me how can I access color data?

Is this right?

pct cloud;
rgb.float_value=cloud.points[y*cloud.height+x].data[3];
            ptr[3*x+1] = rgb.Red;
            ptr[3*x+2] = rgb.Green;
            ptr[3*x+3] = rgb.Blue;

where pt,rgb,ptr and image1 are defined the following.

typedef pcl::PointXYZ        pt; 
typedef pcl::PointCloud<pt> pct;

typedef union
{
  struct /*anonymous*/
  {
    unsigned char Blue;
    unsigned char Green;
    unsigned char Red;
    unsigned char Alpha;
  };
  float float_value;
  long long_value;
} RGBValue;

IplImage *Image1;
uchar* ptr = (uchar*) (Image1->imageData + y * Image1->widthStep);
edit flag offensive delete link more

Comments

seems right. If it doesn't work, you can have a look at rgbdslams GLViewer::pointCloud2GLStrip(...), the conversion is done there.
Felix Endres gravatar imageFelix Endres ( 2011-05-24 03:30:27 -0500 )edit
Thank you. I fixed it.
enddl22 gravatar imageenddl22 ( 2011-05-24 04:01:51 -0500 )edit
0

answered 2011-05-23 23:22:38 -0500

enddl22 gravatar image

updated 2011-05-24 00:43:03 -0500

Thank you Felix.

I could get a this bandwidth by merging openni_nodelet.cpp given 320x240 configuration.

The code is about changing from PointXYZRGB to PointXYZ and some modification of it.

rgbd code that you sent me has compile error on node.cpp

/opt/ros/freiburg_tools/rgbdslam/src/node.cpp:820: error: ‘class ParameterServer’ has no member named ‘getMinMatches’

It looks simple but cut down half size of pcl.

rostopic bw /camera/rgb/points
subscribed to [/camera/rgb/points]
average: 35.81MB/s
    mean: 1.23MB min: 1.23MB max: 1.23MB window: 21
average: 29.98MB/s
    mean: 1.23MB min: 1.23MB max: 1.23MB window: 42

I would like to ask one more question.

Why the size of PointXYZ is not 3float(x,y,z, 12bytes) but 4float(16bytes)?

Is this because of 16bytes alignment?

Thank you again.

Cheers.

edit flag offensive delete link more

Comments

"Is this because of 16bytes alignment?" yes. it is required to benefit from some cpu optimizations.
Felix Endres gravatar imageFelix Endres ( 2011-05-24 00:21:01 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

3 followers

Stats

Asked: 2011-05-08 23:05:48 -0500

Seen: 6,246 times

Last updated: Jun 08 '11