Large black areas on rectified stereo images

edit

I'm using two RGB cameras. They are rotated, one clock-wise and the other counter clock-wise. I'm calibrating them with Ros Stereo Calibration node which is based on OpenCv. The output I get the camera matrix, projection matrix and R and T from cv::stereoCalibrate. The stereo_image_proc node outputs the rectified images that you can see below:

They seem rectified, they are both "facing" the right way but there is a HUGE area of only black pixels......why is that? Does it have to do with some parameter in cv::stereoRectify that I should change?

Does it matter if I hold the calibration patter horizontally or vertically when calibrating? Could that be the issue?

Thanks for the help.

ps: This is a continuation of this question (when I was having completely black rectified images because I was doing the calibration wrong).