Object detection using fixed camera and robot arm
I have built a simple 4 dof robot arm, I'm trying to add object detection, So how to get the coordinates from the camera and send it to the end effector ? I think getting tf from camera and then transform it into base_link and then to end-effector. But I have no clear Idea, can someone guide me? also how to add camera to the rviz urdf ?
Asked by Mano on 2019-10-24 01:49:46 UTC
Answers
First of all you can detect object with camera, but you cannot get coordinates of the object since camera is a 2D sensor, you need laserscan, pointcloud or stereo-camera. But if you want to do it with just camera, take a look here. You can add the distance you measure as a new transform and give it's coordinates to end effector to move towards. Here is a URDF tutorial that will help you to add a new link.
Asked by Choco93 on 2019-10-31 02:29:44 UTC
Comments
I would like to do like this in this video
https://www.youtube.com/watch?v=TllM3kLrq-I
I think I only need x and y coordinates in a 2D plane considering a fixed height(z-axis).
Asked by Mano on 2019-11-06 01:50:57 UTC
Simplest way I can think of is using thresholding in opencv, that should give you all the objects based on your color threshold and you start moving towards the one nearest to you until it's right in front of you, and detection seems like a simple classification task.
Asked by Choco93 on 2019-11-06 02:42:36 UTC
I thin it is not ease to get the exact position of on generic object in x y z, mith an 2d Camera you may use some 2d codes on you object to get the missing information. Otherwise you need the get a 3d camera
Asked by duck-development on 2019-10-31 03:02:32 UTC
Comments
Are you trying to make a manipulator based on object detection? Move your arm toward the found object?
Asked by Choco93 on 2019-10-24 05:46:03 UTC
Trying to move the arm towards the object
Asked by Mano on 2019-10-31 00:10:01 UTC