How can I align an image according to another image?

I am trying to align an image according to a base image and according to the position of 2 circles on the left of each page.

The link for base image: http://tinypic.com/view.php?pic=nmi053&s=6

The link for unregistered image: http://tinypic.com/view.php?pic=2zsbmuu&s=6

The code I have so far is listed below but the only thing I have a problem with is how to use imtransfrom function to register the image.

[Vic]

I1 = imread('C:\Users\Victoras\Desktop\answertest.bmp');

J1= imread('C:\Users\Victoras\Desktop\studenttest.bmp');

I2= rgb2gray(I1); %convert colour images to grayscale

J2= rgb2gray(J1);

BWI = im2bw(I2);% convert grayscale images to binary

BWJ = im2bw(J2);

IMI = imcomplement(BWI); % invert colours, black to white and white to black

IMJ = imcomplement(BWJ);

%imshow(IMI);

%imshow(IMJ);

BWIremove = bwareaopen(IMI, 5000); % remove objects less that 4500 pixels

BWJremove = bwareaopen(IMJ, 5000);

%imshow(BWIremove);

%imshow(BWJremove);

[LI, numI] = bwlabel(BWIremove);

[LJ, numJ] = bwlabel(BWJremove);

centroidsI = regionprops(LI,'Centroid');

centroidsJ = regionprops(LJ,'Centroid');

% Create a transformation structure for an affine % transformation.

t_concord = cp2tform(centroidsI,centroidsJ,'affine');

% Get the width and height of the orthophoto and perform % the transformation.

info = imfinfo(I1);

registered = imtransform(IMJ,t_concord,'XData',[1 info.Width], 'YData',[1 info.Height]);

figure, imshow(registered)

Answer 1

[Image Analyst]

Have you tried imregister()? It will condense all that code down into a single line of code. Go ahead, make it easy on yourself.

[Matt J]

I'd be surprised if IMREGISTER worked on these images. You'd have to be sure the big circles overlapped in some way, and you would have to mask out all the multiple choice boxes, at the very least.

[Vic]

I did not know that but as I can see from the Matlab example of registering 2-D images from same capture device, I am not sure that's what I need. I tried the other day using the imregister() unsuccessfully. Any solution to on my current code? You can help me on the imregister() if you insist!!

Thank you!!!

[Image Analyst]

I don't know how you went wrong. I just adapted the example in the help and it worked beautifully. Maybe I just have a magic touch. Here is my code:

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
workspace;  % Make sure the workspace panel is showing.
fontSize = 20;
format compact;
% Read in a standard MATLAB gray scale demo image.
folder = 'C:\Users\Vic\Documents\Temporary';
baseFileName = 'straight_sheet.jpg';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
  % File doesn't exist -- didn't find it there.  Check the search path for it.
  fullFileName = baseFileName; % No path this time.
  if ~exist(fullFileName, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
straightImage = imread(fullFileName);
% Get the dimensions of the image.  
% numberOfColorBands should be = 1.
[rows columns numberOfColorBands] = size(straightImage)
% Display the original gray scale image.
subplot(2, 2, 1);
imshow(straightImage, []);
title('Original Color Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
% Give a name to the title bar.
set(gcf,'name','Demo by ImageAnalyst','numbertitle','off') 

baseFileName = 'tilted_sheet.jpg';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
tiltedImage = imread(fullFileName);
% Get the dimensions of the image.  
% numberOfColorBands should be = 1.
[rows2 columns2 numberOfColorBands2] = size(tiltedImage)
% Display the original gray scale image.
subplot(2, 2, 2);
imshow(tiltedImage, []);
title('Tilted Color Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
% Give a name to the title bar.
set(gcf,'name','Demo by ImageAnalyst','numbertitle','off') 

% View the misaligned images.
subplot(2, 2, 3);
% figure;
imshowpair(straightImage, tiltedImage, 'Scaling', 'joint');
drawnow;

% imregister works on gray scale images so we need to do it one channel at a time.
% Extract the individual red, green, and blue color channels.
straightRedChannel = straightImage(:, :, 1);
straightGreenChannel = straightImage(:, :, 2);
straightBlueChannel = straightImage(:, :, 3);
tiltedRedChannel = tiltedImage(:, :, 1);
tiltedGreenChannel = tiltedImage(:, :, 2);
tiltedBlueChannel = tiltedImage(:, :, 3);

promptMessage = sprintf('Now I will alight them.\nThis will take about 50 sweconds.\nDo you want to Continue processing,\nor Cancel to abort processing?');
titleBarCaption = 'Continue?';
button = questdlg(promptMessage, titleBarCaption, 'Continue', 'Cancel', 'Continue');
if strcmpi(button, 'Cancel')
  return;
end

% Create a configuration suitable for registering imagesfrom the same sensor. 
[optimizer, metric] = imregconfig('monomodal')
% Now modify the optimizer configuration to get more precision.
optimizer.MaximumIterations = 300;
optimizer.MinimumStepLength = 5e-4;

% Perform the registration.
alignedRedChannel = imregister(tiltedRedChannel, straightRedChannel, 'rigid',optimizer,metric);
% View registered images.
figure
imshowpair(straightRedChannel, alignedRedChannel,'Scaling','joint');
title('Aligned image superimposed with straight image');
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
helpdlg('Done with demo.  Now do the same with the green and blue channels.');

[Vic]

That is very helpful my friend. But I kind of need the tilted image to be aligned and displayed (only that image), not overlap with the straight image. Any ideas??

Thank you very much for this solution as well!!

[Image Analyst]

Then just use imshow(alignedRedChannel) instead of imshowpair().

[Matt J]

Done with demo. Now do the same with the green and blue channels.

The problem is, I don't think you will be able to register the blue channels because the blue channels in each image do not differ by an rigid/affine transformation, unless there is a mistake in the images posted. Note that in each image, the questionaire is filled out differently with different boxes filled out in blue.

Unfortunately also, IMREGISTER doesn't return a tform for the derived transfomration. If it did, you could take the tform derived from the red channel and apply to the other channels...

[Vic]

Matt, are you talking about the demo with imregister? What about the code I have provided about my problem with imtransform?

[Image Analyst]

Good point. However imregister doesn't require that the images be totally the same except for spatial transform. It is somewhat tolerant of differences in the images. If the images are totally different then of course it won't make much sense. I actually tried the green and blue channels of Vic's images and it actually does quite an admirable job despite the tilted image not having the same boxes filled out. I think he'll be very happy with the results.

[Vic]

That is true Image Analyst. It does a very good job and I am quite happy for your help. I didn't get any help for my current code but it is great to have this solution.. Thank you again!!!

[Image Analyst]

That said, if you could figure out a transform, tformfwd() would probably be faster than calling imregister() three times.

[Matt J]

Seems like a fluke to me. What if all the 'A' answers had been filled out in the first image and all the 'D' Answers had been filled out in the second? The alignment of the blue channels would have been heavily inconsistent with the other channels.

Answer 2

[Matt J]

Here's a modification of the original approach using ABSOR, available here

    [LI, numI] = bwlabel( imfill(BWIremove,'holes') );

    [LJ, numJ] = bwlabel( imfill(BWJremove,'holes')  );

    centroidsI = regionprops(LI,'Centroid');

    centroidsJ = regionprops(LJ,'Centroid');

    % Create a transformation structure for an affine % transformation.

     cI=vertcat(centroidsI.Centroid);
     cJ=vertcat(centroidsJ.Centroid);

     S=absor(cJ.', cI.');
     t_concord=maketform('affine', S.M(1:2,:).');

     registered = imtransform(IMJ,t_concord,....);

Because you're only using 2 landmarks to perform the registration, the alignment will probably not be as perfect as with imregister, although as I've said, I have my doubts about how well imregister would work across a larger population of questionnaires.

There are things you could do to improve the tilt angle estimate though, like find more landmarks, or use regionprops(...,'Orientation') on the answer boxes.