Revision history [back]

YOLO is only going to give you a bounding box around the thing you want (of varying quality). You must do the remainder yourself based on the information you have. Lets think about what you have access to in this situation:

• D435 depth
• D435 RGB / bounding boxes
• D435 Extrinsic to robot (presumably)

So, how I'd start this is to take the bounding boxes in the RGB frame and project them into the depth frame so that you now have a patch of the depth image corresponding to the detected object. From a set of points in a pointcloud/depth image you should be able to make a rough estimate of distance based on either naiively averaging or model fitting. This should give you distances. Height / width of the door you'll need to fit a model to it (plane should work fine, maybe with a segmentation based on color so you can remove wall that might have been added).

Now you want to visualize this all in RVIZ. You'll have to look at the rviz-able data types (or create your own) and think about ways to manipulate what you have into those.