| ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange |
Hello everyone,
I am using ubuntu 16.04 with ROS kinetic. For my project work I have detected the door and door handle with YOLO darknet. For that I am using intel realsense d435 camera.
Now my questions are as follow :
How can I measure the distance between door and the camera.
How can I measure the height and width of the door.
How can I add the detected door in rviz 3d.
YOLO is only going to give you a bounding box around the thing you want (of varying quality). You must do the remainder yourself based on the information you have. Lets think about what you have access to in this situation:
So, how I'd start this is to take the bounding boxes in the RGB frame and project them into the depth frame so that you now have a patch of the depth image corresponding to the detected object. From a set of points in a pointcloud/depth image you should be able to make a rough estimate of distance based on either naiively averaging or model fitting. This should give you distances. Height / width of the door you'll need to fit a model to it (plane should work fine, maybe with a segmentation based on color so you can remove wall that might have been added).
Now you want to visualize this all in RVIZ. You'll have to look at the rviz-able data types (or create your own) and think about ways to manipulate what you have into those.
@stevemacenski makes a lot of good points, this is a very difficult set of tasks.
How can I measure the distance between door and the camera.
What exactly does this mean? the distance between the centre of the door and the camera, the distance to the closest point on the door, or the distance to the plane the door is in? For all of these you will need to estimate the shape and pose of the door, YOLO isn't doing this for you, so you will have to build something to do this.
How can I measure the height and width of the door.
For this you will need to estimate the location of the two vertical edges of the door and the two horizontal edges of the door, this seems obvious I know, but this is the real question here.
One approach would be to use a NN to segment the door once it has been detected by YOLO. This would give you an estimate of each of the 3D points from the depth camera that actually make up part of the door, this estimate will probably be a bit noisy but should be good enough to make a start. You'll be able to use this set of 3D points and find the best fit plane which should lie fairly accurately over the door.
Secondly you'll want to have some estimate of the orientation of your camera, if it's mounted on a platform with no roll or pitch that's great you can assume it's vertical. If not you'll need to have some TFs or visual estimate so you can know roughly which way is up.
Now you know which way is up in the point cloud and the plane of the door, it is now relatively straight forward to remove the outliers and estimate the edges of the door.
What you're asking is possible, but it is by no means a simple task.
NN to segment the door
Yes, that's a good point. If you're already going to use YOLO, just swap it out for semantic segmentation and just use the pixels for the door/handle directly and bypass the detector.
Asked: 2019-05-22 06:58:02 -0600
Seen: 2,035 times
Last updated: Mar 11 '20
ROS Answers is licensed under Creative Commons Attribution 3.0 Content on this site is licensed under a Creative Commons Attribution Share Alike 3.0 license.
What data set have you trained YOLO on to recognize doors and door handles? Thanks!!
I am facing nearly the same problem did you find anything ?