Revision history [back]

Since you don't have encoders and will have to rely completely on the camera to determine the future position of the object I foresee the one issue to be extrinsic camera calibration, that is knowing where the camera and the robot exactly are in relation to one another or to a third frame. For that I can recommend you to take a look at the industrial extrinsic calibration package.

Another issue I can imagine is grasping which can be more or less complex depending on the geometry and position of the incoming object (for example if it is LEGO pieces it might be ok with having one or two predefined grasps but if it is a cup of tea that can come upside down then it is necessary to both increase processing and maybe number of sensors). You can start by taking a look at this.

Good luck, sounds like an awesome project!

Since you don't have encoders and will have to rely completely on the camera to determine the future position of the object I foresee the one issue to be extrinsic camera calibration, that is knowing where the camera and the robot exactly are in relation to one another or to a third frame. For that I can recommend you to take a look at the industrial extrinsic calibration package.

Another issue I can imagine is grasping which can be more or less complex depending on the geometry and position of the incoming object (for example if it is LEGO pieces it might be ok with having one or two predefined grasps but if it is a cup of tea that can come upside down then it is necessary to both increase processing and maybe number of sensors). You can start by taking a look at this.

Good luck, sounds like an awesome project!