Trouble getting worldToImage to reproduce the reprojection points shown in the Stereo Camera Calibrator

The stereo camera calibrator is showing and reporting a mean error of 0.84 pixels and a max under 1.5 when reprojecting the checkerboard, however when I try reproducing those results using worldToImage I am experiencing much larger discrepancies. I have tried adjusting parameters in undistortImage, worldToImage, and detectCheckerboardPoints but cannot get my results to match.

Is there another method the calibrator uses to get such high accuracy?

Included an image of my reprojection using default parameters (using the code below). This particular point is at least 4 pixels off, but the calibrator shows and reports under 1.

Calibrator result of same location:

Code below:

imgL_temp = imread('C:\***\stereo_calibration\L\L24.png');
imgCalCheck = imgL_temp;

cameraParams = stereoParams.CameraParameters1;

imUndistorted = undistortImage(imgCalCheck,cameraParams);

[imagePoints,boardSize] = detectCheckerboardPoints(imUndistorted);

squareSize = 101.6; % in millimeters
worldPoints = generateCheckerboardPoints(boardSize,squareSize);

[R,t] = extrinsics(imagePoints,worldPoints,cameraParams);

zCoord = zeros(size(worldPoints,1),1);
worldPoints = [worldPoints zCoord];

projectedPoints = worldToImage(cameraParams,R,t,worldPoints);

imshow(imUndistorted)

hold on
plot(projectedPoints(:,1),projectedPoints(:,2),'r+','LineWidth',2,'MarkerSize',20);
legend('Projected points');
plot(projectedPoints(:,1),projectedPoints(:,2),'g-');
hold off

title(sprintf('L24 imUndistorted, Default Reprojection'));

Answer 1

[Giridharan Kumaravelu]

The major differences between the workflow in your code snippet and that of in the calibrator app are mentioned below. In the calibrator app,

1. the corner points in the checkerboard are detected on the raw calibration images without undistorting them and
2. the world points are projected to the image by applying distortion.

To reproduce the same reprojection error in your code snippet, you can try the following:

imgL_temp = imread('C:\***\stereo_calibration\L\L24.png');
imgCalCheck = imgL_temp;

cameraParams = stereoParams.CameraParameters1;

[imagePoints,boardSize] = detectCheckerboardPoints(imgCalCheck);

squareSize = 101.6; % in millimeters
worldPoints = generateCheckerboardPoints(boardSize,squareSize);

[R,t] = extrinsics(imagePoints,worldPoints,cameraParams);

zCoord = zeros(size(worldPoints,1),1);
worldPoints = [worldPoints zCoord];

projectedPoints = worldToImage(cameraParams,R,t,worldPoints, 'ApplyDistortion', true);

imshow(imgCalCheck)

hold on
plot(projectedPoints(:,1),projectedPoints(:,2),'r+','LineWidth',2,'MarkerSize',20);
legend('Projected points');
plot(projectedPoints(:,1),projectedPoints(:,2),'g-');
hold off

title(sprintf('L24 imUndistorted, Default Reprojection'));

[Yufan Wang]

Hi

I have the same problem when l apply the lntrinsics and Extinsics(rotation vectors and translation vectors) of fisheyeParameters(from mono camera calbration)to get the reprojected points in the original image, the reprojected points are different from those in fisheyearameters.reprojectedPoints. I've tried the way you suggested, applying distortion to the world points when projecting them, but it has no help. I also fix the intrinsics and estimate the extrinsics. My code is as following.

cameraExtrinsics_ori =rigidtform3d(cameraParams.RotationVectors(i,:)/pi*180,cameraParams.TranslationVectors(i,:)) ;
projectPoints_ori = world2img([worldPoints,ones(size(worldPoints,1),1)],cameraExtrinsics_ori,cameraParams.Intrinsics,'ApplyDistortion',true);

cameraExtrinsics_ori2 = estimateExtrinsics(imagePoints_ori,worldPoints,cameraParams.Intrinsics);
projectPoints_ori2 = world2img([worldPoints,ones(size(worldPoints,1),1)],cameraExtrinsics_ori2,cameraParams.Intrinsics,'ApplyDistortion',true);

The result shows as following. The blue circles are the original points detected in the original image. Purple stars are fisheyearameters.reprojectedPoints. Yellow and red circles are projectPoints_ori and projectPoints_ori2. Please tell me how to produce the same results as Purple stars. Looking forward for your reply.