Ask Your Question
0

ROS YOLO object dimensions measure

asked 2019-05-22 06:58:02 -0500

k5519995 gravatar image

updated 2019-05-22 06:58:39 -0500

Hello everyone,

I am using ubuntu 16.04 with ROS kinetic. For my project work I have detected the door and door handle with YOLO darknet. For that I am using intel realsense d435 camera.

Now my questions are as follow :

  1. How can I measure the distance between door and the camera.

  2. How can I measure the height and width of the door.

  3. How can I add the detected door in rviz 3d.

edit retag flag offensive close merge delete

2 Answers

Sort by » oldest newest most voted
3

answered 2019-05-22 09:03:15 -0500

YOLO is only going to give you a bounding box around the thing you want (of varying quality). You must do the remainder yourself based on the information you have. Lets think about what you have access to in this situation:

  • D435 depth
  • D435 RGB / bounding boxes
  • D435 Extrinsic to robot (presumably)

So, how I'd start this is to take the bounding boxes in the RGB frame and project them into the depth frame so that you now have a patch of the depth image corresponding to the detected object. From a set of points in a pointcloud/depth image you should be able to make a rough estimate of distance based on either naiively averaging or model fitting. This should give you distances. Height / width of the door you'll need to fit a model to it (plane should work fine, maybe with a segmentation based on color so you can remove wall that might have been added).

Now you want to visualize this all in RVIZ. You'll have to look at the rviz-able data types (or create your own) and think about ways to manipulate what you have into those.

edit flag offensive delete link more
1

answered 2019-05-22 10:45:10 -0500

@stevemacenski makes a lot of good points, this is a very difficult set of tasks.

How can I measure the distance between door and the camera.

What exactly does this mean? the distance between the centre of the door and the camera, the distance to the closest point on the door, or the distance to the plane the door is in? For all of these you will need to estimate the shape and pose of the door, YOLO isn't doing this for you, so you will have to build something to do this.

How can I measure the height and width of the door.

For this you will need to estimate the location of the two vertical edges of the door and the two horizontal edges of the door, this seems obvious I know, but this is the real question here.

One approach would be to use a NN to segment the door once it has been detected by YOLO. This would give you an estimate of each of the 3D points from the depth camera that actually make up part of the door, this estimate will probably be a bit noisy but should be good enough to make a start. You'll be able to use this set of 3D points and find the best fit plane which should lie fairly accurately over the door.

Secondly you'll want to have some estimate of the orientation of your camera, if it's mounted on a platform with no roll or pitch that's great you can assume it's vertical. If not you'll need to have some TFs or visual estimate so you can know roughly which way is up.

Now you know which way is up in the point cloud and the plane of the door, it is now relatively straight forward to remove the outliers and estimate the edges of the door.

What you're asking is possible, but it is by no means a simple task.

edit flag offensive delete link more

Comments

2

NN to segment the door

Yes, that's a good point. If you're already going to use YOLO, just swap it out for semantic segmentation and just use the pixels for the door/handle directly and bypass the detector.

stevemacenski gravatar imagestevemacenski ( 2019-05-22 10:50:55 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2019-05-22 06:58:02 -0500

Seen: 176 times

Last updated: May 22