Main Content

Create Labeled Blocked Image from ROIs and Masks

This example shows how to create a labeled blocked image consisting of categorical labels.

There are several ways to specify categorical label data for an image. This example shows two approaches. One approach uses polygonal ROI objects that store the coordinates of the boundaries of tumor and normal tissue. The other approach uses a mask to indicate a binary segmentation of the image into tissue and background. The example combines the information in the polygon coordinates and mask representations to create a single labeled blocked image.

Create a blocked image using a modified version of an image from the CAMELYON16 data set, a training image of a lymph node containing tumor tissue. The modified image has three coarse resolution levels. The spatial referencing has been adjusted to enforce a consistent aspect ratio and to register features at each level.

bim = blockedImage('tumor_091R.tif');

Get the spatial referencing and pixel extent of the blocked image at the desired output level.

pixelExtent = bim.Size(1,:)./(bim.WorldEnd(1,:) - bim.WorldStart(1,:));

Load Label Data

The CAMELYON16 data set provides labels of tumor and normal regions as a set of coordinates specifying manually annotated region boundaries with respect to the finest resolution level. When pixels exist within the boundaries of both a normal region and a tumor region, the correct label for those pixels is normal tissue.

Load label data for the blocked image. This example uses a modified version of labels of the "tumor_091.tif" image from the CAMELYON16 data set. The original labels are stored in XML format. The modified labels are resampled and saved as a MAT file.

roiPoints = load('labelledROIs.mat')
roiPoints = struct with fields:
       cancerRegions: {[344×2 double]  [53×2 double]  [539×2 double]  [247×2 double]  [37×2 double]  [161×2 double]}
    nonCancerRegions: {[46×2 double]}

Create polygonal ROI objects that store the coordinates of the tumor boundaries and normal tissue boundaries.

tumorPolys = cellfun(@(position) images.roi.Polygon( ...
    'Position',position,'Visible','on','Color','r'), ...
    roiPoints.cancerRegions);
normalPolys = cellfun(@(position) images.roi.Polygon( ...
    'Position',position,'Visible','on','Color','g'), ...
    roiPoints.nonCancerRegions);

Represent Labels as ROI Objects

Display the image overlaid with the annotated ROIs. The ROIs have the same coordinate system as the image, so changing the resolution levels of the displayed image still renders the ROIs accurately.

figure
h = bigimageshow(bim);
set(tumorPolys,'Parent',gca);
set(normalPolys,'Parent',gca);
title(['Resolution Level:' num2str(h.ResolutionLevel)]);

Zoom in to one ROI.

xlim([3940 4290])
ylim([2680 3010])
title(['Resolution Level:' num2str(h.ResolutionLevel)]);

Represent Labels as Masks

Create a mask at the coarsest resolution level for the stained region, which includes both tumor and normal tissue. The mask is 1 (true) for pixels whose grayscale value is less than 130. Fill small holes in the mask by performing morphological closing using the bwmorph function.

tissueMask = apply(bim, ...
    @(bs)bwmorph(rgb2gray(bs.Data)<130,'close'),"Level",3);
bigimageshow(tissueMask);

Create Combined Labeled Blocked Image

Combine the information in the polygon coordinates and mask representations to create a single labeled blocked image.

To store the labeled image, create a writeable blocked image with data type categorical. Specify the required class names and the corresponding numeric pixel label ID values. Assign the label 0 to the 'Background' class.

blockSize = [512 512]; 
destination = []; % Save the result 'in memory'
sizeOfLabeled = bim.Size(1,1:2); % Labelled image is MxN (2D)
labelIDs = [0,1,2];
labelClasses = ["Background","Normal","Tumor"];
initialvalue = categorical(NaN, labelIDs, labelClasses);
bLabeled = blockedImage(destination,sizeOfLabeled,blockSize,initialvalue,...
    'WorldStart', bim.WorldStart(1,1:2),...
    'WorldEnd', bim.WorldEnd(1,1:2),...
    'Mode','w')
bLabeled = 
  blockedImage with properties:

   Read only properties
             Source: []
            Adapter: [1×1 images.blocked.InMemory]
               Size: [5000 5358]
       SizeInBlocks: [10 11]
    ClassUnderlying: "categorical"
          BlockSize: [512 512]

   Settable properties
       InitialValue: <undefined>

Loop through each output blocked image, one block at a time. Determine the label of each block, then set the pixel data of the block accordingly.

To determine the label of each block, start with the mask of all tissue. Pixel values of 0 in the mask correspond to background, which matches the pixel label ID of the 'Background' class. Pixel values of 1 in the mask correspond to all tissue, which matches the pixel label ID of the 'Normal' class. Convert the polygon coordinates of tumor tissue to a mask by using the poly2mask function, then replace those pixels with the pixel label ID of the 'Tumor' class, 2.

If you have Parallel Computing Toolbox™, then you can run the loop in parallel by replacing the for statement with a parfor statement.

for cInd =  1:bLabeled.SizeInBlocks(2)
    for rInd = 1:bLabeled.SizeInBlocks(1)
        blockSub = [rInd, cInd];
        blockStart = bLabeled.blocksub2sub(blockSub);
        blockEnd = blockStart + bLabeled.BlockSize - 1;
        % Clamp to image end
        blockEnd = min(blockEnd, bLabeled.Size);
        
        % Read the corresponding region from the tissue mask. Since the
        % mask is at a different level, convert coordinates to world and
        % back.
        blockStartEndInWorld = bLabeled.sub2world([blockStart; blockEnd]);
        blockStartEndMaskSub = tissueMask.world2sub(blockStartEndInWorld);
        maskBlock = getRegion(tissueMask, blockStartEndMaskSub(1,:), blockStartEndMaskSub(2,:));
        
        % Scale up the mask region
        maskBlock = imresize(maskBlock, [blockEnd-blockStart+1]);
        
        % Find the pixel coordinates of healthy tissue then convert the
        % polygon coordinates to a mask.
        roiBlock = false(size(maskBlock));
        xyStart = fliplr(blockStart);
        for ind = 1:numel(roiPoints.cancerRegions)
            vertices = roiPoints.cancerRegions{ind};
            % Transform coordinates to local block pixel locations
            vertices = (vertices-xyStart);
            isTumor = poly2mask(vertices(:,1),vertices(:,2), ...
                size(roiBlock,1), size(roiBlock,2));
            roiBlock = roiBlock|isTumor;
        end
        
        % Some healthy tissue ROIs are enclosed within a tumor ROI. Find
        % the pixel coordinates of healthy tissue then convert the polygon
        % coordinates to a mask.
        for ind = 1:numel(roiPoints.nonCancerRegions)
            vertices = roiPoints.nonCancerRegions{ind};
            % Transform coordinates to local block pixel locations
            vertices = (vertices-xyStart);
            isHealthy = poly2mask(vertices(:,1),vertices(:,2), ...
                size(roiBlock,1), size(roiBlock,2));
            roiBlock = roiBlock & ~isHealthy;
        end         
        
        % Set the value of pixels in tumor regions as the corresponding
        % pixel label ID, |2|.
        blockNumeric = uint8(maskBlock);
        blockNumeric(roiBlock) = 2;
        
        % Create a categorical image from the block data.
        blockCategorical = categorical(blockNumeric,...
            labelIDs, labelClasses);
        
        % Set the block of the categorical blocked image as the categorical
        % block image.
        setBlock(bLabeled, blockSub ,blockCategorical);
        
    end
end

bLabeled.Mode = 'r';

Display Overlay of Labels and Original Data

Display the image data, then display the labeled image data in the same axes. The three labels (normal, tumor, and background) appear in three different colors. Make the labels partially transparent so that you can distinguish the image content underneath.

figure
hbim = bigimageshow(bim);
hla = axes;
hbl  = bigimageshow(bLabeled,'Parent',hla);
hbl.AlphaData = 0.7;
hla.Visible = 'off';

Zoom in to a ROI. Increase the label transparency so that you can more clearly distinguish the image content underneath.

linkaxes(findall(gcf,'Type','axes'));

xlim([3940 4290])
ylim([2680 3010])
hbl.AlphaData = 0.5;

Save Labels for Use Later

For most data sets, you can create labels once and then reuse the labels for multiple training sessions. The labeled blocked image, bLabeled, is backed by temporary files that do not exist across MATLAB® sessions. To reuse the labels in a different session of MATLAB, write bLabeled to a persistent location.

imageDir = 'Labels';
if exist(imageDir,'dir')
    rmdir('Labels','s');
end
labelDir = fullfile(imageDir,'labelled');
write(bLabeled,labelDir);

In a fresh session of MATLAB, you can reload the labeled blocked image by creating a new blockedImage object. When loading a labeled blocked image, you must also specify 'Classes', 'PixelLabelIDs', and 'UndefinedID'.

bLabeled = blockedImage(labelDir);

See Also

| |

Related Topics