Revision history [back]

You are describing a very good approach in your question: put markers on the end-effector of the robot, and observe those markers with the kinect camera. It sounds like you have some difficulties with the code to recognize the markers, but that does not make the approach less valuable. We apply the exact same approach when calibrating the PR2 cameras relative to the PR2 arm. We use checkerboard as markers; opencv has good methods to compute the pose of a checkerboard relative to a camera frame.