Combining two matrices into one

Hi all,

I have a question regarding matrix manipulation in MATLAB.

My scenario is as follows:

I have a set of GPS coordinates, which I have converted into relative meters using an algorithm. These coordinates correspond to locations on a farm which I have gathered data for using a video camera, and roughly correspond to a standard "go up one row, go down the next" path.

For each coordinate, I also have an image at that coordinate, which I am using to identify the location of weeds.

From the GPS coordinates, which are simple X-Y points, I can generate a matrix. The matrix would likely be a roughly 2000x2000 size matrix, where a cell would have a value of 1 if there was a GPS point identifying that the tractor had been on that spot.

The images are 800x600x3 color images.

What I want to be able to do, is take the images, and, using the matrix I made from the GPS coordinates, somehow combine all images together into one large image.

If the images were distinct, then this would not be as big of an issue, as I could generate an 800x600x3 matrix at each cell of the 2000x2000 matrix. This would be a rather large matrix, however, it would accomplish the task.

However, the GPS coordinates are such that an image might overlap to a certain extent with the images adjacent to it.

Can anyone suggest any ways I can accomplish what I am trying to do? The end result simply needs to be a large image I can look at, which will show me the entirety of my farm land, using the images which I have taken.

Answer 1

[Vidhi Agarwal]

Hi @Kiran

I understand you have a query regarding a method to generate a one large image by combining all the images using matrix made up of GPS coordinates. Combining multiple overlapping images into a single large image is a problem often referred to as "image stitching" or "mosaicking". Following steps can help you to lead towards the solution:

• Load all the images and extract features using Scale-Invariant Feature Transform (SIFT) or Oriented FAST and Rotated BRIEF (ORB) to identify key points and descriptors in each image.
• Use Fast Library for Approximate Nearest Neighbors to find matching key points between overlapping images and apply Random Sample Consensus to filter out incorrect matches and compute the homography matrix (transformation matrix).
• Use the homography matrices to warp the images into a common coordinate system. This involves transforming the perspective of each image so that they align correctly with each other.
• Use blending techniques like multi-band blending or feathering to blend the edges of overlapping images smoothly to avoid visible seams.
• Create a large canvas (matrix) to hold the final stitched image and place each warped image onto the canvas using the coordinates from the GPS-derived matrix.

Below is the sample code that can help in getting started:

% Load Images
imageDir = 'path_to_your_images';
imageFiles = dir(fullfile(imageDir, '*.jpg'));
numImages = length(imageFiles);

% Initialize the large canvas
canvasHeight = 2000;
canvasWidth = 2000;
canvas = zeros(canvasHeight, canvasWidth, 3, 'uint8');

% Initialize features and points
points = cell(numImages, 1);
features = cell(numImages, 1);
tforms(numImages) = projective2d(eye(3));

% Read the first image and detect features
I = imread(fullfile(imageDir, imageFiles(1).name));
grayImage = rgb2gray(I);
points{1} = detectSURFFeatures(grayImage);
[features{1}, points{1}] = extractFeatures(grayImage, points{1});

% Iterate over remaining images
for n = 2:numImages
    % Read the next image
    I = imread(fullfile(imageDir, imageFiles(n).name));
    grayImage = rgb2gray(I);
    
    % Detect and extract features
    points{n} = detectSURFFeatures(grayImage);
    [features{n}, points{n}] = extractFeatures(grayImage, points{n});
    
    % Match features between the current and the previous image
    indexPairs = matchFeatures(features{n}, features{n-1}, 'Unique', true);
    matchedPoints1 = points{n}(indexPairs(:, 1), :);
    matchedPoints2 = points{n-1}(indexPairs(:, 2), :);
    
    % Estimate the transformation between the current and previous image
    tforms(n) = estimateGeometricTransform(matchedPoints1, matchedPoints2, 'projective', 'Confidence', 99.9, 'MaxNumTrials', 2000);
    
    % Compute the transformation for the current image relative to the first image
    tforms(n).T = tforms(n).T * tforms(n-1).T;
end

% Compute the output limits for each transform
for i = 1:numImages
    [xlim(i,:), ylim(i,:)] = outputLimits(tforms(i), [1 size(I, 2)], [1 size(I, 1)]);
end

% Find the minimum and maximum output limits
xMin = min([1; xlim(:)]);
xMax = max([canvasWidth; xlim(:)]);
yMin = min([1; ylim(:)]);
yMax = max([canvasHeight; ylim(:)]);

% Width and height of panorama
width  = round(xMax - xMin);
height = round(yMax - yMin);

% Initialize the panorama
panorama = zeros([height width 3], 'like', I);

% Create a 2-D spatial reference object defining the size of the panorama
xLimits = [xMin xMax];
yLimits = [yMin yMax];
panoramaView = imref2d([height width], xLimits, yLimits);

% Create the panorama by overlaying all images
for i = 1:numImages
    I = imread(fullfile(imageDir, imageFiles(i).name));
    
    % Transform I into the panorama
    warpedImage = imwarp(I, tforms(i), 'OutputView', panoramaView);
    
    % Overlay the warpedImage onto the panorama
    panorama = max(panorama, warpedImage);
end

% Display the panorama
imshow(panorama)
title('Stitched Farm Image')

% Save the final stitched image
imwrite(panorama, 'stitched_farm_image.jpg');

For better understanding of concept like SIFT, “image stitching” and function like “estimateGeometricTransform”and “detectSURFFeatures”, refer to the following documentation:

• https://www.mathworks.com/help/vision/ug/feature-based-panoramic-image-stitching.html
• https://www.mathworks.com/help/vision/ref/detectsiftfeatures.html
• https://www.mathworks.com/help/vision/ref/detectsurffeatures.html
• https://www.mathworks.com/help/vision/ref/estimategeometrictransformation.html

Hope that helps!