Finding pose in base coordinate when given input point on camera image

Hello,

I'm a beginner to ROS, and I'm using ROS Indigo in Ubuntu 14.04.

With mobile manipulator, I'm going to implement a teleoperation system which user can operate from a distance. Especially in manipulation part, I'm thinking of manipulation which user can see robot's head camera view (from remote PC) and determine the pose of end-effector.

For example, if user gives specific input point (with a mouse input) to an image from head camera, system calculates the pose of input point at the base coordinate. (So that it makes to move the end-effector to the pose.)

Therefore, I would like to ask about the validity of this plan, and if possible, can you recommend the most appropriate method? (Or any package that I can apply.)

Thank you very much.

Asked by jfet94 on 2018-07-13 09:55:58 UTC

Comments

Answers

Using a monocular camera in the general case this is not possible. The problem is that a 2D point on a camera image does not define a point in 3D space relative to the camera.

You can solve this problem in two ways; restrict the problem or get a 3D camera system.

You could restrict the problem to points on a horizontal surface, then if the pose of the camera relative to this surface was known then point in 3D space could be determined.

You could use either a stereo camera or RGB-D camera instead which would provide you with depth information. This additional information means that a point on an image then does define a point in 3D space.

I hope this helps to get you started.

Asked by PeteBlackerThe3rd on 2018-07-14 05:11:07 UTC

Comments

I didn't mention that the robot's head camera is RGB-D camera! As long as this method is possible, I'm going to implement in this way. (I'm thinking of using openCV.)

Your answer was really helpful, thank you very much for your reply.

Asked by jfet94 on 2018-07-16 00:14:08 UTC