how can i segment characters from the image with lot of white pixels and similar background like text ?

i need to segment the character part of image and then each individual characters as a seperate image. i am having problem to do in matlab. Here is the image attached. P.S. : I have tried earlier answers for character segmentation, but this image seems a bit difficult. could anybody please help :)

i want to segment the portion containing characters into a seperate image. so my aim is not to just recognize text but to segment it as an image from above image. Finally i want to compare the segmented image of hello world from this printed board with the reference image which is made in paint and have just hello world witten in it. So main aim is Kinda quality comparison of printed image with 100 percent accurate image, so basically how good each character it is printing.

[KSSV]

Have you checked with ocr?

[Avinash Kumar]

yes, ocr doesnt seem to work. As background is quite similar to the text here. this is what ocr return instead of hello world : ocrText with properties:

                      Text: 'efloflwofldi↵↵'
    CharacterBoundingBoxes: [11×4 double]
      CharacterConfidences: [11×1 single]
                     Words: {'efloflwofldi'}
         WordBoundingBoxes: [47 2 302 55]
           WordConfidences: 0.1218

Moreover i want the segmented result as image as i want to kinda compare quality of printing on this board with original reference image of hello world.

Answer 1

[Image Analyst]

You just need to subtract the images:

diffImage = abs(double(grayImage) - double(refImage));

Then see if you have a significant number of pixels above zero. If you do, then the test image does not say Hello World.

[Avinash Kumar]

Hello sir, my purpose is how good letters are printed, not to check whether it contains hello world or not. So my approach was to first segment all characters from image and then segment individual character from that and compare it with reference image. As example, i need to extract "H" from above image and the compare it with "H" from reference image attached below and compare how much similar they are. It is basically a measure how accurate robot is printing it. Reference image can be any image made in paint and written hello world on it.

For example this can be my reference image. Now i need to compare each character from printed and each character from reference, how similar they are. Hope now i am clear.

[Avinash Kumar]

Can anybody please gimme a pointer or help with this ? :) i wud really appreciate that.

[Image Analyst]

I'd probably do an edge detection, like a Sobel or something. Then threshold and find the area of the edges. The total for your reference image will be different (hopefully lower) than for your (possibly corrupted and bad) "test" letters. For example if there is lots of debris around the "bad" letters, the edge area will be greater because of that.

[Image Analyst]

First of all, erase everything outside the known region where your letters will be. You should have your camera permanently mounted and your part perfectly positioned so you will know the bounding box. Then erase that. Then get the area like this:

area = sum(binaryImage(:))
% or
area = bwarea(binaryImage);

[Avinash Kumar]

hello again sir, thanks for your wonderful suggestion. i did this :

temp1 = imread('Aluminium.png'); temp1 = rgb2gray(temp1); temp1 = temp1(573:750,420:1200); temp1= temp1>80; area = sum(temp1(:)) imshow(temp1)

the area comes to be 9763 Now how shud i use area to compare from reference image. i didnt understand this point ? Also, now how to segment each character individually from this image. as i can see there are lot of other white pixels. I want to segment each character from here and now compare from each corresponding character of reference image. Thanks again.

MAIN POINT : WHICH SIMILARTIY METRIC I SHUD CHOOSE TO COMPARE THE IMAGE OR CHARACTERS : SHUD IT BE STRUCTURAL SIMILARITY (SSIM) or MSE?

[Image Analyst]

I don't think you need to segment out each character. Why do you think you'd need to do that? Just get the area for this image and for the reference image and compare them. If you want to try ssim() or immse(), you can do that too. Try them all and see which correlates best with the ground truth.

[Avinash Kumar]

I wanted to see characterwise comparison, like sometimes robot print only specifically one character wrong, i wanted to know which one. Btw image which has more area is worse than reference image right ? P.S. - i shud calculate area on binary image or the bounding box directly obtained from grayscale image which is not binary ? The binary image depends upon thresold which i choose thus the area will depend upon the thresold which i choose. What to do in that ? Also, i have not done edge detection , i just chose a thresold and got binary image and got area on that.

[Image Analyst]

If the font is fixed width, then you can just segment based on the known columns that define the separation columns. If you want you don't have to threshold - you can just sum up the gradient/edge image. Again, reference and test images should have similar values if the character is okay and high values if they are different.

[Avinash Kumar]

Hello sir, thanks for great advice. Yes, it is better. Now the only thing is i need to align my two images for perfect comparison (ssim). my extracted image for interest is

and my reference image of same size is :

Now ssim is giving result as 0.005 because of different alignments of letter in both images. So i need to extract only letters from my first image. i used ocr for creating bounding box but it is not giving perfect bounding box, what it gives is this :

How can i get the perfect bounding box from this now ? Many many thanks for your help always sir.

[Avinash Kumar]

Hello sir, can u gimme some pointer about this. I will close this thread after this last question. Thanks for your so much help.

[Image Analyst]

What would be "perfect"? Do you mean that you don't want the small one in the upper left? Just use bwareafilt() to throw out small blobs before getting bounding boxes.

[Avinash Kumar]

Perect means, yes i dont want the upper left small one and also you can see, the "world" bounding box is larger than it should be.

[Image Analyst]

OK, then I gave you the answer.

[Avinash Kumar]

i tried your method by this : BW2 = bwareafilt(temp1,18);

One problem is if i take top 10 characters, the dot below punctuation marks also disapper. But i need that. if i take top 18 characters, there are some blobs in the lower and right corner that are coming. How to remove that ?

[Image Analyst]

bwareafilt() will not introduce any additional blobs, like those in the corner. You can delete blobs touching the border with imclearborder().

[Avinash Kumar]

yes, this works :D Thanks a lot for all your help. If i have any question again, i will open this thread.