3D point cloud reconstruction from movable camera

edit

Hi folks,

after a while I'm back with a new project, a camera system that's purpose is to scan and classify an object by somehow moving around it. I only have a single RGB camera that I can move in 4 degrees of freedom (x, y, pitch, yaw). I also already can move the camera in a circle around the object based on these 4 DOF while it permanently points towards (looks at) the object.

I would like to find a way to create a 3D point cloud from the pan around the object. How do I do that?

I know that I have to calibrate the camera and use rectified images for this purpose. So assume that I already have the images rectified. Where do I go from here? What do I need?

Thanks a lot!

Cheers,
Hendrik