Improper Disparity map obtained for stereo scene reconstruction

I have done stereo calibration using the Stereo Camera Calibrator App available in MATLAB R2014b.

When I obtain the results of calibration, the maximum reprojection error showed on the graph is around 0.55 (little less than 0.6). However, the translation of Camera 2 wrt Camera 1 obtained is correct (I know that the baseline is 200cm. The app detects it as 199.98cm, which I guess is fine). Also, it is only from this parameter that I believe the various other parameters obtained after calibration is right. (Of course, apart from the reprojected points which should match with the detected corners.) Is there any other way I can check if my calibration results are proper (in this stage)?

I have attached the stereo parameters ( stereoParams ). Using this, I want to calculate the x, y, z distance of any object (1m to 2m away from baseline, as this is where checkerboard too was placed for calibration). I did the following:

1. using stereoParams, I found the distance to checkerboard (as object). I did this using detectCheckerboardPoints (to both left and right images) and triangulate.I got the correct results.
2. I placed a wheel (object) and found disparity. Mine is a 10MP image, so I used DisparityRange = [400, 640]; BlockSize = 55. Why are there two objects (wheels) detected? I am getting an error in the values of distances too.

What are the other parameters I can tune to get a better disparity output?

Any help will be appreciated.

Attachments (left and right images from Camera A and B respectively and stereo parameters) are uploaded on Dropbox .

Answer 1

[Dima Lisin]

He Meghana,

Your calibration looks fine. The problem here is that the wheel is too close to the cameras. The overlap between the rectified images is too small, and the disparity is too large. You should get much better results if you either move the wheel farther away from the cameras, or move the cameras closer together.

Also, you may want to try pre-processing the images with histogram equalization using histeq and low pass filtering using imgaussfilt before computing disparity.

[Meghana Dinesh]

Hello Dima,

Thanks for your input. I calibrated again and tried with my baseline as 158 mm. The disparity is still very high (it's around 400 when manually calculated). If I move my object away from the camera, it's not visible clearly. I cannot decrease the baseline further due to hardware constraints.

1. Should I rotate the two cameras slightly towards each other? (It will have relatively high FOV overlap though principle axis will not be parallel. Is it fine?)
2. Should I use a different sensor of a lesser resolution? Will this decrease the disparity?

Are there any other ways by which disparity can be decreased?

[Dima Lisin]

I would still try to move the object farther away, just as a sanity check. Rotating the cameras to verge slightly is a good idea. Rectification will take care of the optical axes not being parallel. Also, I think more light in your scene will help.

[Meghana Dinesh]

I would like to add that using the same set of stereo parameters, I could successfully find the distance of a checkerboard (using detectCheckerboardPoints followed by triangulate ).

1. I was wondering if I can find corresponding points (similar to how I used detectCheckerboardPoints to find corresponding points in both images) for the present object (wheel) too?
2. What does the parameter DistanceThreshold refer to? Is is the same as MaxDisparity (in DisparityRange)?
3. Does an entire block (of dimension mentioned as BlockSize ) have the same disparity?
4. Does the background lighting during calibration and during runtime have to be same?

I have attached the stereo parameters along with images taken from Camera A and Camera B (A1r and B1r are stereo pairs) on dropbox. I have also attached the 3D point cloud I obtained. Why is there so much variation in the z-axis? (The entire object (wheel) seems like it is distributed along a wide range in z plane while actually, it is simply leaning against a wall)

I have decreased my baseline to 90mm (cannot go below this). What other parameters can I adjust? Let me know if you require any other details from my algorithm

[kashif khalid]

Mail me the code.