Getting depth data from (x,y) position of image using Realsense RGB-D Camera

Hi there,

I'm currently working with a Realsense RGB-D camera on ROS.

I've noted that latest builds of intel-ros' realsense has fixed the alignment issue and is able to publish aligned depth frames to colour/infra-red.

However, may I know how I can then use just the (x,y) position of the image stream to get the depth data at that particular pixel?

Thank you.

Asked by mattso on 2018-03-21 09:18:02 UTC

Comments

Answers

Hello mattso,

I am not sure if I understood the question entirely, but I will try to give you the best possible answer.

I am guessing that you are using registered depth images which means that both depth and color images are in the same coordinate system. This is important because, in that case, you can match the pixels between color and depth images - meaning, a pixel at (x1,y1) in color image has a depth encoded at pixel (x1,y1) in depth image.

That is true for every pixel in the image. However, you need to have in mind that depth sensors are not perfect and not all pixels will show the real depth. Some will just have noise added and others will end up as NaNs or zeros because the sensor was not able to determine the depth (or it was not certain enough - depends on the RealSense presets you are using).

One more thing to pay attention to is the depth image encoding. You need to check how are the depth values encoded - in millimeters or in meters. The easiest way to do it is by checking the value of the encoding field in your sensor_msgs::Image (Image message). It can be either float(in m) or 16-bit unsigned integer (in mm). You can read more about it here.

Moreover, if you are interested in creating a point cloud based on the input depth image check out the depth_image_proc package - especially this part.

If you have any further questions, feel free to ask.

Asked by dljubic on 2018-05-23 15:23:35 UTC

Comments

@dljubic I have further questions. as explained by you, we will work with two different topics /camera/color/image_raw and /camera/aligned_depth_to_color/image_raw in two different callback.as in first callback will find(x,y), but how to integrate second callback into first one(as both are two different topics from camera) so we can get depth d at that particular (x,y)?

Asked by nd on 2019-07-21 06:00:43 UTC

Actually, you would have one callback where you would synchronize both color and depth image topics. You can read more about it here. By using the synchronized callback, you ensure that the color image has a corresponding depth image.

Furthermore, if you are interested in a way to extract the depth for a particular pixel in a color image, it is already implemented in depth_image_proc. Take a look at the convert function here.

Asked by dljubic on 2019-07-21 07:50:00 UTC

@mattso Hi I'm also experiencing the same problem, have you solved this?

Asked by abc71485 on 2021-09-02 09:20:24 UTC