unit test in image segmentation

Hi,

I am working on medical image segmentation (images acquired with several machines, several patients) and I have questions on how to do unit testing in Matlab: So far, I found this https://www.mathworks.com/help/matlab/class-based-unit-tests.html

but it is very general and i'd like to understand the procedure for image segmentation.

I searched for examples and I found this response:

https://stackoverflow.com/questions/1283147/is-it-possible-to-use-tdd-with-image-processing-algorithms

who says that: "TDD in image processing only makes sense for deterministic problems like:

image arithmetic histogram generation and so on.. However TDD is not suitable for feature extraction algorithms like:

edge detection segmentation corner detection ... since no algorithm can solve this kind of problems for all images perfectly."

Do you agree with the above statement ?

I'd like to know how do you perform unit testing or TDD in your image segmentation.

Thanks a lot, Maia

Answer 1

[Ashish Uthama]

Maia, The stackoverflow link you found has a lot of relevant detail, especially the one from user "jilles de wit".

I would create some manual segmentation (for e.g using the image segmenter app) and then use that mask as the 'golden data' to compare against whatever new automated approach I am trying to develop. You could use one of these functions: jaccard, bfscore or dice metric to compare the results of the ground truth/golden data against the algorithm you are developing with some allowance/tolerance to account for variability.

[Maia]

Thank you very much.

[Maia]

Hi,

Thank you again for your answers.

I'd like to know how would you perform unit tests with several image samples, i.e., testing a single function on several images, which have different expected values. I tried these codes, but I am not sure if this is the way to go (especially the for ... end). Also, I want to track/know for which image the test didn't pass, but verifyEqual does not return a value to test that.

So far, I created a subfolder for each image test where I put data like expected value for that image.

===

for iCase = 1:length(testCase.TestData.data)

data = testCase.TestData.data(iCase);

actSol = functionToBeTested(); expSol = data.value; verifyEqual(testCase, actSol, expSol);

end

Answer 2

[CJ]

I second Ashish's comment. It sounds to me that the main concern here is the ever-changing test set and the non-deterministic nature of the algorithm.

TDD and unit test can mean a lot of things, but in general we want things to be fixed and deterministic so we can test them. In that case one needs to try to eliminate variables from the system. The ground truth approach suggested by Ashish is one way of doing that. Another example would be incorporating statistics into your expected results. Let's say your algorithm may produce variable results on a single/small set of images, then perhaps increase the sample size and obtain a fixed lower/higher bound according to your requirement.

[Maia]

Thank you very much.