I want to map the thermal image of the Flir with the depth image of the XTion. As the depth image is calculatet from Xtions IR camera I want to map the Flir with Xtions IR image.

Therefor I placed both cameras on one plane close to each other (about 7 cm in x, 1 cm in y and 3 cm in z).

Then I used ROS Indigo and openCV 2.4.9 to:

• Set the Flir Focus to a fix on (no autofocus)
• Get both images synchronized.
• Resize the Xtion IR image from 640x480 to 320x240 pixels as the Flir image
• Calculate the intrinsic camera parameters for both cameras. (Flir + Xtion IR)
• Calculate the extrinsic parameters
• Remap both images to get the rectified images

I now have the two rectified images but still an offset in X (horizontal direction).

If I understand that correctly, I have the offset due to the different focal lengths and field of views (Flir with objective: 45° H x 33.8° V and 9.66mm focal length, XTion: 58° H x 45° V) and could solve the problem with a perspective transform but I don't have both focal lengths in mm.

The datasheets:

1. http://support.flir.com/DsDownload/Assets/48001-0101_en_40.pdf
2. https://www.imc-store.com.au/v/vspfiles/assets/images/1196960_en_51.pdf
3. http://www.asus.com/us/Multimedia/Xtion_PRO_LIVE/specifications/

I had the idea to get the focal lengths with cv::calibrationMatrixValues but I dont know the apertureWith and Heigth.

Cross-post: http://stackoverflow.com/questions/29966080/how-to-map-thermal-image-flir-a325sc-with-asus-xtion-pro-live-ir-image-under-o

How could I solve this problem?