How to map bounding boxes from object detector to a map ?

asked 2019-05-14 11:27:04 -0500

argon2300 gravatar image

Hello,

I"m starting to understand ROS but I'm still a bit confused. I started learning ROS with a turtlebot, my goal is to recognize a track made of traffic cones, find a closed loop and drive around.

At this moment I'm able to run slam and recognize the cones with an object detector but I'm stuck at combining the two. Is it possible to show to the position of the cones (bounding box) on the map created with SLAM ? I'm using the Kinect camera, at this moment I run gmapping and object detection separately.

I'm really stuck with this one so any help would be really appreciated

Thanks in advance,

edit retag flag offensive close merge delete

Comments

Can you be more specific about the object detector you are using and its ouput please ? You can find an example here in the case you are able to get the object frame and then you just do a tf lookup transform to have the pose in the map frame. The example was taken from the find_object_2d package wiki that could be useful for you.

Delb gravatar image Delb  ( 2019-05-15 03:55:38 -0500 )edit

Thanks for your reply, at this moment I”m using this package to detect cones with kinect. ( https://github.com/fdiazgon/cone-dete... ) My next step would be to locate the cones on a map for mavigation pruposes.

argon2300 gravatar image argon2300  ( 2019-05-15 16:37:38 -0500 )edit

I'm not sure your package is the one needed because it only returns the position of a cone within an image (meaning only { x; y}). If you could get the depth too then you would be able to use the tf to add your cones within your map but your object detec tor package doesn't seem to deal with depths.

You could "merge" the package I gave as example in my previous comment.with your cone-detector to get things working.

Delb gravatar image Delb  ( 2019-05-16 02:45:44 -0500 )edit

Thanks again for the reply, Things are getting clearer for me. The only questions I have left :

  1. How am I able to merge two different packages? I do not have any experiences and couldn't find any examples.

  2. I"m still a bit confused about the map building progress, are there any examples to add coordinates to a map in ROS ?

Thanks in advance

argon2300 gravatar image argon2300  ( 2019-05-17 12:56:15 -0500 )edit
1

hi i have the same problem, used YOLO darknet for object detection but the package only outputs a 2d pixel coordinate of the bounding box around the object of interest. I was wondering if you figured out how to convert this 2d pixel coordinate into 3d pose stamped msg (x y z) coordinate?

marcs gravatar image marcs  ( 2020-02-13 07:40:26 -0500 )edit

YOLO darknet will only give 2d Bounding box , in order to find the pose , Point cloud for correspondence pixel coordinate is needed or depth information is required . I have figure out the estimate distance of detected object in camera frame ,but i am not sure about the proper lookup transform for converting this distance w.r.t map frame . as the odom to map transform in gmapping is not accurate . can anyone tell me how to get a proper transform between the odom and map frame as of now i have the distance of each detected object only in the base_link frame .

Badal gravatar image Badal  ( 2021-09-28 01:48:40 -0500 )edit

Hello, I also have this part in my project. Did you find a project or code about it?

gokce gravatar image gokce  ( 2023-06-25 11:07:03 -0500 )edit
add a comment