Detect all rectangles from an image

Question

Anoy Chowdhury on 7 Apr 2023

0
Link

Direct link to this question

https://ch.mathworks.com/matlabcentral/answers/1943479-detect-all-rectangles-from-an-image

Commented: Anoy Chowdhury on 9 Apr 2023

I am trying to make a program which plays Minesweeper - online

The first step is to recognize the squares that are part of the game, then identify each square as a ROI for OCR detection.

I did an

roi1=imcrop(I);
roi2=imcrop(I);

And then a loop like:

roi3_hat(1)=roi2(1)+roi2(1)-roi1(1); %First corner X with horizontal difference
roi3_hat(2)=roi1(2); % First corner H is in the same line
roi3_hat(3:4)=28; %The squares have fixed sizes

Where I get the measurments and compute in a sloppy way the distribution of the remaining ones to finally achieve the 64 rois.

But there must be an easier and reliable way! Any ideas?

0 Comments
Show -2 older commentsHide -2 older comments

Sign in to comment.

Sign in to answer this question.

Answer 1

DGM on 8 Apr 2023

1
Link

Direct link to this answer

https://ch.mathworks.com/matlabcentral/answers/1943479-detect-all-rectangles-from-an-image#answer_1211859

Edited: DGM on 8 Apr 2023

Open in MATLAB Online

The image you have is messed up somehow. The tiles are actually not consistently sized due to whatever display aliasing is going on. If the tiles were reliably and uniformly spaced, you could use simple operations to isolate the playing field region and then split it directly into an integer number of tiles. Because the tiles are not consistently sized, I'm going to detile the image by addressing a variable-sized window for each tile in the nonuniform grid. It's more complicated, but that's what happens when you're dealing with bad images.

That said, this example doesn't use your nonuniform image, since I wanted to get images with more numbers. It was easier to synthesize test images that way.

Start with a new game image that can be used as a template. Use it to find the information needed to address any other working image.

% the blank field image (the template)

tpict = imread('mswtemplate.png');

% create mask

tilefacecolor = [1 1 1]*0.74;

mask = all(abs(im2double(tpict) - permute(tilefacecolor,[1 3 2]))<0.05,3);

% get rid of bridges and spurs

mask = imopen(mask,ones(3));

% isolate tiles from everything else based on area

P = regionprops(mask,'area');

tilearea = median([P.Area]);

mask = bwpropfilt(mask,'area',tilearea*(1+[-1 1]*0.08));

% dilate the masks so that we catch some of the bevel

mask = imdilate(mask,ones(3));

imshow(mask)

% get the extents of each tile face as subscript vectors

% use centroid ordering to find tiling

P = regionprops(mask,'centroid','subarrayidx');

C = vertcat(P.Centroid);

p = diff(C,1,1);

tiling = nnz(p(:,1)>10)+1; % number of columns

tiling = [numel(P)/tiling tiling]; % [rows cols]

Now you have information that you can use to detile or otherwise process other images. I doubt OCR is even necessary. You can probably just use color information to determine tile state. In this example, I'm just going to create a state array that gives the tile state as a number representing the number of neighbor mines. An unchecked tile is NaN.

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% read the working image
inpict = imread('msw123.png');
% this is a map of the approximate H,S and V
% for each number that may appear. 
hsvmap = [240 1 1;
    120 1 0.5;
    0   1 1;
    240 1 0.5;
    0   1 0.5;
    180 1 0.5;
    0   0 0;
    0   0 0.5];
% build state array without detiling
mask = all(abs(im2double(inpict) - permute(tilefacecolor,[1 3 2]))<0.05,3);
hsvpict = rgb2hsv(inpict);
state = NaN(tiling); % preallocate the output
for k = 1:prod(tiling)
    % get this tile
    ridx = P(k).SubarrayIdx{1};
    cidx = P(k).SubarrayIdx{2};
    [H S V] = imsplit(hsvpict(ridx,cidx,:));
    
    if range(V(2:end-1,2:end-1)) < 0.01
        % tile is either unchecked or empty if its interior has low variance
        % tile is empty if entire tile has low variance
        if range(V(:)) < 0.01
            state(k) = 0;
        end
    else
        % tile has something in its interior (e.g. a number)
        thismask = ~mask(ridx,cidx);
        meanH = circmean(H(thismask)*360);
        meanS = mean(S(thismask));
        meanV = mean(V(thismask));
        [~,state(k)] = min(sum(abs([meanH meanS meanV] - hsvmap),2));
    end
end
state % show the result
state = 9×9
   NaN     1     0     1   NaN   NaN   NaN   NaN   NaN
   NaN     1     0     1     2   NaN   NaN   NaN   NaN
     1     1     0     0     1   NaN   NaN   NaN   NaN
     0     0     0     0     1     2     3   NaN   NaN
     0     0     0     0     0     0     1     1     1
     0     1     1     2     1     1     0     0     0
     0     1   NaN   NaN   NaN     2     1     1     0
     0     1     2   NaN   NaN   NaN   NaN     1     0
     0     0     1   NaN   NaN   NaN   NaN     1     0

This works for other images as well: