# Determine depth of a pixel via PointCloud2

Hello,

I subscribe to a Point Cloud2 message.

Here is my callback:

void callback_depth_image_d435 (const sensor_msgs::msg::PointCloud2::SharedPtr point_cloud2_msgs)
{
//sensor_msgs::msg::PointCloud2& cloud = ;
pcl::PointCloud<pcl::PointXYZ> point_cloud;

// convert point clouds2 msgs data to points
pcl::fromROSMsg(*point_cloud2_msgs, point_cloud);

// resize vector every callback
point_cloud2_points.resize(point_cloud.points.size());

for(unsigned long i = 0; i < point_cloud.points.size(); i++)
{
// get points
point_cloud2_points.at(i).x = point_cloud.points.at(i).x;
point_cloud2_points.at(i).y = point_cloud.points.at(i).y;
point_cloud2_points.at(i).z = point_cloud.points.at(i).z;
}
}


I need one point for each pixel of my camera image for my object detection. I have set my camera image to 480 x 640. Now I actually expected to get a point for each pixel with the following function.

   pcl::fromROSMsg(*point_cloud2_msgs, point_cloud);


but since the size of the "point_cloud" is constantly changing, this means that I constantly have a different number of pixels?

Can anyone explain the reason for this?

edit retag close merge delete

During a pointcloud generation there is a matching between the 2 images. On every step, every pixel (or set of pixels) from the left part are searched a matching element in right image. The distance between theses 2 points in the image is stored as depth and with this, a depth image is generated. From here is where a pointcloud appears.

Understanding this, you can imagine that even with the best matching algorithm ever and with the best camera, always will be sets of points that, by any reason, are notfound in one of both images and this creates a hole. This hole can be represented by two ways in the pointcloud :

1. removing it.
2. considering the point As NaN.

This is decided with the optinon is_dense (True option 1, False 2)

What i think is probably happening you is that you are asking the pointcloud to be dense, this is ...(more)

( 2021-01-04 00:37:09 -0500 )edit

btw once the image is dense, there is no way to revert. The order and size of the image is lost (and yes, what a pitty)

( 2021-01-04 00:41:52 -0500 )edit