Ask Your Question
0

completely black rectified images (Kind of solve....?) [closed]

asked 2015-12-09 08:36:39 -0600

Ariel gravatar image

updated 2016-01-12 07:59:34 -0600

Hello,

I just got a couple of cameras for stereo vision. I calibrated them with camera_calibration. I got the ost.txt file which looks like this:

# oST version 5.0 parameters


[image]

width
640

height
480

[stereo/left]

camera matrix
2936.789087 0.000000 583.498064
0.000000 2865.839793 254.251806
0.000000 0.000000 1.000000

distortion
-0.224767 -2.059623 -0.000719 0.023231 0.000000

rectification
0.779954 0.219443 -0.586102
-0.092957 0.966743 0.238258
0.618894 -0.131348 0.774414

projection
7412.660530 0.000000 6128.045624 0.000000
0.000000 7412.660530 -855.670516 0.000000
0.000000 0.000000 1.000000 0.000000

# oST version 5.0 parameters


[image]

width
640

height
480

[stereo/right]

camera matrix
2822.855469 0.000000 113.685666
0.000000 2746.930572 199.606104
0.000000 0.000000 1.000000

distortion
-0.986390 4.581731 0.017456 0.010306 0.000000

rectification
0.146495 0.250593 -0.956944
-0.488950 0.859289 0.150169
0.859923 0.445898 0.248409

projection
7412.660530 0.000000 6128.045624 -86524.314248
0.000000 7412.660530 -855.670516 0.000000
0.000000 0.000000 1.000000 0.000000

That file I separated left and right .ini and .yaml:

left.ini: # oST version 5.0 parameters

[image]

width
640

height
480

[nstereo/left]

camera matrix
2936.789087 0.000000 583.498064
0.000000 2865.839793 254.251806
0.000000 0.000000 1.000000

distortion
-0.224767 -2.059623 -0.000719 0.023231 0.000000

rectification
0.779954 0.219443 -0.586102
-0.092957 0.966743 0.238258
0.618894 -0.131348 0.774414

projection
7412.660530 0.000000 6128.045624 0.000000
0.000000 7412.660530 -855.670516 0.000000
0.000000 0.000000 1.000000 0.000000

right.ini:

# oST version 5.0 parameters


[image]

width
640

height
480

[stereo/right]

camera matrix
2822.855469 0.000000 113.685666
0.000000 2746.930572 199.606104
0.000000 0.000000 1.000000

distortion
-0.986390 4.581731 0.017456 0.010306 0.000000

rectification
0.146495 0.250593 -0.956944
-0.488950 0.859289 0.150169
0.859923 0.445898 0.248409

projection
7412.660530 0.000000 6128.045624 -86524.314248
0.000000 7412.660530 -855.670516 0.000000
0.000000 0.000000 1.000000 0.000000

And with rosrun camera_calibration_parsers convert left.ini left.yaml and rosrun camera_calibration_parsers convert right.ini right.yaml I obtained the .yaml.

Once I launched the node with the embedded drivers for my cameras ueye_cam node I echoed the camera_info topic from both cameras and I got every data according to the calibration.

Then I ran ROS_NAMESPACE=stereo rosrun stereo_image_proc stereo_image_proc and the images obtained (raw_image directly from the driver's node, rectified and mono from stereo_image_proc) are shown below:

images

First of all, master camera is rotated 90 deg clockwise and slave is rotated 90 counter clockwise. The calibration was with the chessboard pattern farther from the two objects I'm showing in ... (more)

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by Ariel
close date 2016-01-15 07:20:27.042058

Comments

You could try running undistortPoints on some sample points like the center of the image, and the four corners, to see if those calibrations are reasonable. Is the calibration process stereo-aware? Maybe try calibrating a single camera and rectifying without any stereo stuff running.

lucasw gravatar imagelucasw ( 2015-12-10 13:20:38 -0600 )edit

I'm out for holidays now. I'll calibrate them separately when I get back. But shouldn't I have something to project both images to a common plane for the sterevision? And what about the extrinsic parameters then?

Ariel gravatar imageAriel ( 2015-12-13 13:38:52 -0600 )edit
1

In my opinion, your calibration looks wrong. The cx and cy terms should be close to the half of the resolution. Try to attach the calibration pattern to a rigid surface (wood) and repeat the calibration. Don't move the cameras, move the pattern instead, and focus the cameras.

Miquel Massot gravatar imageMiquel Massot ( 2015-12-28 05:00:12 -0600 )edit

2 Answers

Sort by ยป oldest newest most voted
1

answered 2016-01-12 03:30:19 -0600

Ariel gravatar image

updated 2016-01-14 04:31:46 -0600

I did the calibration again, this time separately, one camera at a time. They were fixed on the table and I moved the calibration pattern, which is attached to a rigid surface. I also changed the resolution of the cameras so as to have the biggest resolution they can. The results seem similar for the camera matrix, but the distortion values are not all the same and the rectification matrix is always the identity......is that correct.....?. Here are the new values:

left:

# oST version 5.0 parameters


[image]

width
1280

height
1024

[narrow_stereo]

camera matrix
2295.015272 0.000000 640.541295
0.000000 2303.682687 471.554511
0.000000 0.000000 1.000000

distortion
-0.465980 -0.188907 0.003472 0.003497 0.000000

rectification
1.000000 0.000000 0.000000
0.000000 1.000000 0.000000
0.000000 0.000000 1.000000

projection
2199.656494 0.000000 642.325188 0.000000
0.000000 2244.262207 470.208280 0.000000
0.000000 0.000000 1.000000 0.000000

right:

# oST version 5.0 parameters


[image]

width
1280

height
1024

[narrow_stereo]

camera matrix
2339.253648 0.000000 634.067723
0.000000 2347.476009 479.539465
0.000000 0.000000 1.000000

distortion
-0.444712 -0.188235 0.000477 0.002607 0.000000

rectification
1.000000 0.000000 0.000000
0.000000 1.000000 0.000000
0.000000 0.000000 1.000000

projection
2250.488770 0.000000 634.664472 0.000000
0.000000 2291.515381 477.643991 0.000000
0.000000 0.000000 1.000000 0.000000

Thanks again for the help!

EDIT 1: Seems to be ok. These are the rectified images and the disparity image I get after running:

ROS_NAMESPACE=stereo rosrun stereo_image_proc stereo_image_pcoc
rosrun image_view stereo_view stereo:=stereo image:=image_rect_color

And here is the result: image description http://oi68.tinypic.com/9plshk.jpg

Is the disparity image correct?

As one camera is rotated clockwise and the other counter-clockwise, what would be the best way to have the images rotated? Should they be rotated 'before' or 'after' the stereo_proc node? I mean, should I rotate the raw images I get directly from the cameras to be inputs of stereo_proc node? Again, thanks a lot for the help!

EDIT 1: I rotated the cameras by subscribing to the original /stereo/{left,right}/image_raw and publishing them on another topic like this:

#include <ros/ros.h>
#include <image_transport/image_transport.h>
#include <sensor_msgs/image_encodings.h>
#include <sensor_msgs/Image.h>

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <cv_bridge/cv_bridge.h>

cv::Mat dst;

/* Subscribe and Publish class: http://answers.ros.org/question/59725/publishing-to-a-topic-via-subscriber-callback-function/ */
/* Publish image: http://answers.ros.org/question/192959/how-to-publish-and-sub-camera-in-opencv/ */

class SubscribeAndPublish
{
    public:
        SubscribeAndPublish()
        {
            //Topic to Subscribe
            subMasterImage_ = n_.subscribe("stereo/left/image_raw", 1, &SubscribeAndPublish::callbackMaster, this);
            subSlaveImage_ = n_.subscribe("stereo/right/image_raw", 1, &SubscribeAndPublish::callbackSlave, this);

            //Topic to Publish
            pubMasterImage_ = n_.advertise<sensor_msgs::Image>("stereorotated/left/image_raw", 1);
            pubSlaveImage_ = n_.advertise<sensor_msgs::Image>("stereorotated/right/image_raw", 1);
        }

        void callbackMaster(const sensor_msgs::ImageConstPtr& input)
        {
            cv_bridge::CvImage rotatedimage;
            try ...
(more)
edit flag offensive delete link more

Comments

There ought to be a standard image flip/rotate90/180/270 nodelet. (image_rotate isn't right for that when arbitrary rotations aren't needed) It would be great to get one added to image_proc.

lucasw gravatar imagelucasw ( 2016-01-18 11:20:13 -0600 )edit

The two separate calibrations look reasonable, I should have been clearer before but that is mainly a sanity check - if you can calibrate the cameras individually and run image rectification on them then you have a much better chance of doing stereo, but you still have to do a stereo calibration.

lucasw gravatar imagelucasw ( 2016-01-18 11:21:38 -0600 )edit

If I calibrate them individually and the do the stereo calibration, how do I 'merge' the result?

Ariel gravatar imageAriel ( 2016-01-19 02:03:38 -0600 )edit

Other thing is that I can physically measure that the baseline is 10cm. Therefore, if I use the values from right P I should get the same number (4319.6/4243.1=1.01), right?(which I don't...). Considering (-right_.Tx/right_.fx). Therefore, I can check that the stereo calibration is not correct?

Ariel gravatar imageAriel ( 2016-01-19 02:06:38 -0600 )edit

Can you run opencv decomposeProjectionMatrix on the right P matrix to get the translation calculated for you? (I'm not sure if there are ros vs. opencv P matrix differences) It ought to give you consistent units with however you defined the chessboard intersections.

lucasw gravatar imagelucasw ( 2016-01-20 17:24:17 -0600 )edit
1

answered 2016-01-12 08:26:31 -0600

Looking at the images it appears you have two cameras with long focal length/high zoom. What is also readily apparent is that they have only a very small area of overlap, as the small "IDS box" is on the far right edge of the left camera image and on the far left edge of the right camera image. Valid distance (and calibration) data can only be estimated in the area of overlap, so stereo calibration might fail because of this small overlap (I'm actually surprised that it allowed you to optimize at all, as normally the "calibrate" button only becomes enabled when enough image area has been sampled).

I'm not sure if and how stereo calibration can cope with rotated cameras, so I'd recommend to rotate them before feeding them to calibration (or rectification later).

Note that calibrating both cameras standalone allows for rectifying their images, but not for proper stereo computation as there is no way to know the transform between both cameras in that case.

edit flag offensive delete link more

Comments

As you said, the focal length is 12mm.It gives a small area of overlap. Could have been a non flat calibration patter the issue before? I'm in the process of writing a node to rotate the images, following the image transport tutorials and cv transpose and flip functions.Any better idea?Thanks again.

Ariel gravatar imageAriel ( 2016-01-12 08:54:10 -0600 )edit
1

Yes, non-flat calibration patterns will result in larger calibration error when using calibration patterns that use flat checkerboard calibration targets. Re rotation: This should be a good start: https://github.com/ros-perception/ima...

Stefan Kohlbrecher gravatar imageStefan Kohlbrecher ( 2016-01-12 09:10:18 -0600 )edit

According to its documentation, it says: It is not recommended to use the output from this node for further computation, as it interpolates the source image, introduces black borders, and does not output a camera_info. I don't know then if I should use that....

Ariel gravatar imageAriel ( 2016-01-12 09:22:22 -0600 )edit

What I meant is using the code (camera callback, nodelet structure etc.), throwing out all the complicated tf/rotation stuff and adding a simple cv transpose/flip at the right place :)

Stefan Kohlbrecher gravatar imageStefan Kohlbrecher ( 2016-01-12 09:29:34 -0600 )edit

Right! My bad! I'll get to it and come back with the results after stereo calibrating with the images rotated!

Ariel gravatar imageAriel ( 2016-01-12 09:37:27 -0600 )edit

I rotated the images using this. Originally I had stereo/{L,R}/{image_raw,camera_info}. I read the image and publish it on rotated/{L,R}/image_raw but still have issues with camera_info now. Can i remap?

Ariel gravatar imageAriel ( 2016-01-13 06:52:12 -0600 )edit

Uhm yes, you probably need to remap the CameraInfos, because the resolution will be mixed up if you don't do so. OTOH if you calibrate using the rotated images and update the CameraInfo, things should start working.

Stefan Kohlbrecher gravatar imageStefan Kohlbrecher ( 2016-01-13 07:03:08 -0600 )edit

I remapped the topic in the launch file of the camera, but not the service. Is it possible to do so without renaming the whole NAMESPACE?

Ariel gravatar imageAriel ( 2016-01-13 07:07:20 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2015-12-09 08:36:39 -0600

Seen: 836 times

Last updated: Jan 14 '16