Revision history [back]

So basically you have a bounding box of a 2d image, and you need distance or physical coordinates of that object. This can be done by following these 2 steps.

1. First you need to find that object in 3d space. For that you need to fuse your depth information with camera so you need lidar-camera fusion. Here is a good starting point, it works with melodic and is easy enough to use. This step can be done in 2 ways. Fuse whole image with depth image, or fuse part of it, just the bounding box.
2. You look at specific pixel on object and find corresponding value from depth image. Note that bounding box will also include points that are not part of object so it might be good idea to apply a clustering algorithm beforehand and maybe take mean of the left points, or take the point that is closest (might not give correct result always).

Also note that you have depth-image and this package doesn't work with depth-images but with pointcloud, you can either modify it to work with depth image or you can use this to convert your depth image into pointcloud. Hope this helps :)

So basically you have This is more of a bounding box of a 2d image, and you need distance or physical coordinates of that object. This can be done by following these 2 steps.question of whether your raw image and depth image are in same frame, if they are then, you just look at corresponding pixel value on depth image and you get the distance, if they are not then you can do something like following.

1. First you need to find that object in 3d space. For that you You need to fuse your depth information with camera so you need lidar-camera fusion. Here is a good starting point, it works with melodic and is easy enough to use. Also note that you have depth-image and this package doesn't work with depth-images but with pointcloud, you can either modify it to work with depth image (not sure how easy it is or is it possible) or you can use this to convert your depth image into pointcloud This step can be done in 2 ways. Fuse whole image with depth image, or fuse part of it, just the bounding box.
2. You look at specific pixel on object and find corresponding value from depth image. Note that bounding box will also include points that are not part of object so it might be good idea to apply a clustering algorithm beforehand and maybe take mean of the left points, or take the point that is closest (might not give correct result always).always), reduce size of bounding box to have less points that are not part of object, get creative with it.

Also note that you have depth-image and this package doesn't work with depth-images but with pointcloud, you can either modify it to work with depth image or you can use this to convert your depth image into pointcloud. Hope this helps :)