ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question

Object detection using fixed camera and robot arm

asked 2019-10-24 01:49:46 -0600

Mano gravatar image

I have built a simple 4 dof robot arm, I'm trying to add object detection, So how to get the coordinates from the camera and send it to the end effector ? I think getting tf from camera and then transform it into base_link and then to end-effector. But I have no clear Idea, can someone guide me? also how to add camera to the rviz urdf ?

edit retag flag offensive close merge delete


Are you trying to make a manipulator based on object detection? Move your arm toward the found object?

Choco93 gravatar image Choco93  ( 2019-10-24 05:46:03 -0600 )edit

Trying to move the arm towards the object

Mano gravatar image Mano  ( 2019-10-31 00:10:01 -0600 )edit

2 Answers

Sort by ยป oldest newest most voted

answered 2019-10-31 02:29:44 -0600

Choco93 gravatar image

First of all you can detect object with camera, but you cannot get coordinates of the object since camera is a 2D sensor, you need laserscan, pointcloud or stereo-camera. But if you want to do it with just camera, take a look here. You can add the distance you measure as a new transform and give it's coordinates to end effector to move towards. Here is a URDF tutorial that will help you to add a new link.

edit flag offensive delete link more


I would like to do like this in this video

I think I only need x and y coordinates in a 2D plane considering a fixed height(z-axis).

Mano gravatar image Mano  ( 2019-11-06 00:50:57 -0600 )edit

Simplest way I can think of is using thresholding in opencv, that should give you all the objects based on your color threshold and you start moving towards the one nearest to you until it's right in front of you, and detection seems like a simple classification task.

Choco93 gravatar image Choco93  ( 2019-11-06 01:42:36 -0600 )edit

answered 2019-10-31 03:02:32 -0600

duck-development gravatar image

I thin it is not ease to get the exact position of on generic object in x y z, mith an 2d Camera you may use some 2d codes on you object to get the missing information. Otherwise you need the get a 3d camera

edit flag offensive delete link more

Question Tools



Asked: 2019-10-24 01:49:46 -0600

Seen: 1,129 times

Last updated: Oct 31 '19