Ask Your Question

Pick and Place using 2 hands

asked 2018-07-31 13:02:50 -0600

DebasmitaGhose gravatar image

updated 2018-08-01 01:48:27 -0600

gvdhoorn gravatar image

I want to do a bimanual pick and place task for an object at a known position of a known shape using Moveit. I have been using this tutorial for reference.

But, since this tutorial is for one hand, I am unable to understand how to correctly specify the end effector in the srdf, since there will be 2 end effectors in this case, with different parent links.

Also, I am unable to find any example code where something similar has been successfully implemented. It would be great if somebody could suggest a good reference for this.

Edit: Thank you for a detailed answer.

I have already created 4 move groups and created the SRDF file such that there is no collision between the joints. What I dont understand is how do I write the open_gripper and closed_gripper function. Should it just contain the position of the end effectors or the complete robot?

Also, while populating the grasp message with the position of the end effector while picking and placing, how do I specify the coordinates of the goal, since there are 2 hands approaching it?

I have modified the tutorial code like this, but it is not working.

It would be great if you could tell me how do I modify the code. This is the link to the SRDF file.

edit retag flag offensive close merge delete


A more detailed explanation of what I have done and the problems I am facing can be found here

DebasmitaGhose gravatar image DebasmitaGhose  ( 2018-08-01 06:18:13 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted

answered 2018-07-31 17:05:45 -0600

Geoff gravatar image

updated 2018-07-31 17:09:30 -0600

I can give you a partial answer that will at least allow you to control the two arms as separate move groups, which might give you a start.

  • In the setup assistant, specify four planning groups, one for each arm and one for each end effector. Make sure to give them names that allow you to know which end effector is on which arm.

  • In the setup assistant, label your end effector planning groups as end effectors. Again, give them names that let you know which end effector is on which arm.

  • Generate the files again.

  • In your code, do the same as that tutorial specifies, but do it twice, once for each end effector. When specifying the allowable grasps, you will need to ensure that the end effectors will not collide with each other, so set the grasps to be on the opposite sides of the object or similar.

See this tutorial for a working dual-arm setup.

Unfortunately, moving the arms together without tearing the thing you're holding apart is beyond my MoveIt! knowledge. I don't know if MoveIt! even supports this without a custom planner of some kind, because it needs to ensure that the trajectories for both arms will maintain the grippers in the same relative position. This answer discusses closed chains, which is effectively what you have when you are holding something in two hands, and so the information there may be helpful. Most industrial dual-arm use cases seem to focus on using the two arms separately. Hopefully one of the ROS-Industrial people will have a solution.

edit flag offensive delete link more


Afaik there is currently no simple way to achieve what the OP asks for with MoveIt. It simply wasn't designed for this. The infrastructure can be used, but custom development will be needed.

It could be that planning environments like OpenRave (or possibly Dart) have support for this.

gvdhoorn gravatar image gvdhoorn  ( 2018-08-01 01:55:54 -0600 )edit

Some previous discussions can be found in the moveit-users google group. Planning for dual arm manipulation in object space would be an example.

gvdhoorn gravatar image gvdhoorn  ( 2018-08-01 01:56:41 -0600 )edit

If I remember right, these guys set one robot as master and had the other robot follow the grasped object. It's not MoveIt, but extending the planner like this is definitely possible.

fvd gravatar image fvd  ( 2018-08-01 08:41:47 -0600 )edit

Yes, as I wrote -- and the student did in the Google group discussions I linked -- it's all possible, just not something that will work out-of-the-box. It will require some work.

gvdhoorn gravatar image gvdhoorn  ( 2018-08-02 02:54:18 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools



Asked: 2018-07-31 13:02:50 -0600

Seen: 470 times

Last updated: Aug 01 '18