Extracting external Boundary from Binary Image generated from noisy RGB

Hi there,

I am trying to extract the following outermost boundary from the following binary image.

This image was generated from an rgb image using kmeans clustering. Unfortunately the background is highly noisy and the object depicted includes holes such that the background colour is visible within it. Due to the colour scheme of the object mainly in the greens to dark blues, it is near indistinguishable from the actual object other than through the border edge which is yellow. I therefore went into hsv to adapthisteq the hue at first then latter after seeing a couple of papers the saturation as well. This allows for the usage of kmeans clustering which correctly identifies the surrounding background, the internal structure and the border region into three seperate clusters.

Now after turning the clustered image into binary as seen above, I applied thed bwboundaries function which did not yield any real workable result. This most probably is due to the entire middle part of fuzzyness. Next I tried to bwareafilt, but as the background is inside as well this reduces at the thinner edges the boundary depending on size of pixel to keep. Lastly, I used the boundarymask funtion. This at least yields the below image. The external boundary has sometimes holes in it but seems to be traceable once a starting point is found. This can be done by using the corner points along the diagonals as to be orthogonal to the boundary.

As you may imagine this is only part of the image to illustrate the issue. The complete object is at best described as ellipitcal in shape though with the boundary being ridge as shown above. I have looked at ellipitcal growing and reducing algos as well but they just do not perform. Any hints as to what algorithm and implementations would be best suited for this issue are highly welcome. Thank you in advance for your time!

EDIT: As the issue seems to require the entire image, given that the complete image has properties which differ from the section shown. Although I hazard a guess it is the section in the centre of the image which causes the issue. Here is the complete image.

EDIT 2:

As it seems that the segmentation is inadequate, the original RGB image was added. Essentially, the feature extraction uses Kmeans Clustering to extract the clusters after some optimisation of colour in the hsv space. As my current computing setup only allows 15 GB, this is a trial resized image. Ideally if segmentation works this can be up scaled.

Answer 1

[Image Analyst]

What do you think of this:

% Demo by Image Analyst.
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clearvars;
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 16;
fprintf('Beginning to run %s.m ...\n', mfilename);

%-----------------------------------------------------------------------------------------------------------------------------------
% Read in image.  This is a horrible image.  NEVER use JPG format for image analysis.  Use PNG, TIFF, or BMP instead.
folder = [];
baseFileName = 'green.png';
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
	% The file doesn't exist -- didn't find it there in that folder.
	% Check the entire search path (other folders) for the file by stripping off the folder.
	fullFileNameOnSearchPath = baseFileName; % No path this time.
	if ~exist(fullFileNameOnSearchPath, 'file')
		% Still didn't find it.  Alert user.
		errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
		uiwait(warndlg(errorMessage));
		return;
	end
end
rgbImage = imread(fullFileName);
[rows, columns, numberOfColorChannels] = size(rgbImage)
% Display the test image.
subplot(3, 2, 1);
imshow(rgbImage, []);
axis('on', 'image');
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.
caption = sprintf('Image : "%s"', baseFileName);
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

% Set up figure properties:
% Enlarge figure to full screen.
hFig1 = gcf;
hFig1.Units = 'Normalized';
hFig1.WindowState = 'maximized';
% Get rid of tool bar and pulldown menus that are along top of figure.
% set(gcf, 'Toolbar', 'none', 'Menu', 'none');
% Give a name to the title bar.
hFig1.Name = 'Demo by Image Analyst';

% Create a binary image of the purple spots using code generated from the Color Thresholder app.
[binaryImage, maskedRGBImage] = createMask(rgbImage);

% Display the images.
subplot(3, 2, 2);
imshow(binaryImage, []);
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.
axis('on', 'image');
title('Initial Mask Image', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;

% Fill Holes
binaryImage = imfill(binaryImage, 'holes');
% Take largest one.
binaryImage = bwareafilt(binaryImage, 1);
% Show the image.
subplot(3, 2, 3);
imshow(binaryImage, []);
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.
axis('on', 'image');
title('Final Binary Image', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;

% Get the areas of the blobs from the initial mask image.
props = regionprops(binaryImage, 'Area');
allAreas = sort([props.Area]) % There are a bunch of areas less than 30 and 5 areas greater than 4199.

% Show the final masked image.
% Mask the image using bsxfun() function to multiply the mask by each channel individually.  Works for gray scale as well as RGB Color images.
maskedRGBImage = bsxfun(@times, rgbImage, cast(binaryImage, 'like', rgbImage));
subplot(3, 2, 4);
% Multiply by 5 so we can see it better.
imshow(5 * maskedRGBImage, []);
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.
axis('on', 'image');
title('Final Masked Image', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;

% Histogram of the image
subplot(3, 2, 5:6);
grayImage = rgb2gray(maskedRGBImage);
imhist(grayImage);
grid on;
xlabel('Gray Scale', 'FontSize', fontSize);
ylabel('Count', 'FontSize', fontSize);
% Get the area(s) of the blob(s) from the final mask image.
props = regionprops(binaryImage, 'Area');

msgbox('Done!');

function [BW,maskedRGBImage] = createMask(RGB)
%createMask  Threshold RGB image using auto-generated code from colorThresholder app.
%  [BW,MASKEDRGBIMAGE] = createMask(RGB) thresholds image RGB using
%  auto-generated code from the colorThresholder app. The colorspace and
%  range for each channel of the colorspace were set within the app. The
%  segmentation mask is returned in BW, and a composite of the mask and
%  original RGB images is returned in maskedRGBImage.

% Auto-generated by colorThresholder app on 18-Feb-2021
%------------------------------------------------------


% Convert RGB image to chosen color space
I = rgb2ycbcr(RGB);

% Define thresholds for channel 1 based on histogram settings
channel1Min = 34.000;
channel1Max = 230.000;

% Define thresholds for channel 2 based on histogram settings
channel2Min = 0.000;
channel2Max = 255.000;

% Define thresholds for channel 3 based on histogram settings
channel3Min = 0.000;
channel3Max = 255.000;

% Create mask based on chosen histogram thresholds
sliderBW = (I(:,:,1) >= channel1Min ) & (I(:,:,1) <= channel1Max) & ...
	(I(:,:,2) >= channel2Min ) & (I(:,:,2) <= channel2Max) & ...
	(I(:,:,3) >= channel3Min ) & (I(:,:,3) <= channel3Max);
BW = sliderBW;

% Initialize output masked image based on input image.
maskedRGBImage = RGB;

% Set background pixels where BW is false to zero.
maskedRGBImage(repmat(~BW,[1 1 3])) = 0;
end

[Maximilian Oremek]

Thank you for your help. Basically, I should have tried to explore all the colour spaces. As thresholding in RGB was quite though, I decided to move into HSV and LAB.

Answer 2

[KALYAN ACHARJYA]

bwImage2=~bwareafilt(~bwImage,1); % Largest Blob
se=strel('line',10,10); 
% Note one strel element, you may require to compensate the extra pixels
bw_roi=bwImage & ~imerode(bwImage2,se); 
figure,imshow([bwImage,bw_roi]);

[Maximilian Oremek]

Hi,

Thanks for the answer. Does not work on the entire image though. But it does work on the image sliced into quartes as the above excerpt.

There are some missing pixels. I will play around with strel size to deal with that. Any ideas why it should't work on the entire image?

[KALYAN ACHARJYA]

"Does not work on the entire image though"

What does the whole image mean here? I have answered based on the image provided with the question, which works perfectly in my case.

[Maximilian Oremek]

Hi,

Thanks for the quick reply. The image represents a quarter of the acutal image to illustrate the issue as stated in the original question.

As the original image is rather large and I have only been testing with downsampled data. I think the main issue is to fill in the area behind the boundary... I will update the question with a complete image.

[KALYAN ACHARJYA]

Hope now you can modify the code and get the more accurate result

se=strel('line',10,10); 
bwImage1=imdilate(bwImage,se);
bwImage2=bwareafilt(~bwImage1,1); % Largest Blob
se=strel('line',15,15); 
bw_roi=bwImage & imdilate(bwImage2,se); 
figure,imshow([bwImage,bw_roi]); 

Note one strel elements, you may require to compensate all those extra pixels

[Maximilian Oremek]

Thanks for the answer! The approach seems to require optimising the strel function with trying to optimise erode and dilation. I reckon I'll have to write some optimisation function, if i do not want to do it by hand?.

[KALYAN ACHARJYA]

As I have already shared, you may need some effort, but precise segmentation of the image is possible. You can get the exact boundary in the entire image by scanning first and last while pixels in individual columns (Issue: but yes, it will take a little longer).

[Maximilian Oremek]

Thank you for your help! I greatly appreciate it.

Answer 3

[Image Analyst]

If you just want the top pixel of the quarter image, you can simply scan the image column by column looking for the first pixel in each column using find().

If you want the whole image, and you want to close that gap at the bottom, then you can close all gaps using bwconvhull()

mask = bwconvhull(mask);

Of course that will also fill in small indentations as well as the giant gap at the bottom. If you want to keep the exact shape and just fill in the giant gap then there are things you can do by combining the original image and the convex hull image in clever ways.

Or you might just look into using a better segmentation routine so that you start with a better binary image.

[Maximilian Oremek]

Ideally, I would like to trace the exact shape as accurately as possible. Herein lies the problem. I have played around with the segmentation functions in the toolboxes. In the end, the background blur of the original image which is not clearly seperateable by colour from the data seeked has massively confused anything other than clustering algorithms. Here Kmeans has shown to be more robust than hierarchical clustering. Next closest was using the ORB detection.

Therefore, either some clever combination of these two or the usage of a pretrained Deep Learning approach is necessary. With the latter I fear neither the amount of data required to train nor the computational ressources to train it are available to me.

[Image Analyst]

Could you share your original RGB image?

[Maximilian Oremek]

I have edited the original question.