Image segmentation is a commonly used technique in digital image processing and analysis to partition an image into multiple parts or regions, often based on the characteristics of the pixels in the image. Image segmentation could involve separating foreground from background or clustering regions of pixels based on similarities in color or shape. For example, a common application of image segmentation in medical imaging is to detect and label pixels in an image or voxels of a 3D volume that represent a tumor in a patient’s brain or other organs.
Why Image Segmentation Is Important
Several algorithms and techniques for image segmentation have been developed to effectively solve segmentation problems using domain-specific knowledge. These applications include medical imaging, autonomous driving, video surveillance, and machine vision.
Medical Imaging
Segmentation and analysis have important applications for clinical diagnosis, treatment planning, and medical research. Use segmentation to label and analyze organs, tumors, cells, implants, or other regions of interest. For example, you can use radiomics to classify a tumor as cancerous or benign, and then measure tumor growth in follow-up scans. Alternatively, you can track the position and morphology of cells in segmented microscopy images, or analyze bone density in a pharmaceutical trial. Segmentation masks can also be used to generate 3D anatomical models for applications such as 3D printing and finite element analysis, which are useful for patient-specific surgical planning.
Examples of segmentation using Medical Imaging Toolbox™, including (clockwise from upper left) lungs in a CT scan, a brain in an MRI scan, cells in a microscopy image, and the left ventricle in a cardiac MRI.
Autonomous Driving
When designing perception for autonomous vehicles, such as self-driving cars, semantic segmentation is often applied to enable the system to identify and locate vehicles and other objects on the road.
Using semantic segmentation in Computer Vision Toolbox™ to associate each pixel of the image with a class label (such as car, road, sky, pedestrian, or bike). (See documentation.)
How Image Segmentation Works
The first step in image segmentation involves converting an image into a collection of regions of pixels that are represented by a mask or a labeled image. By dividing an image into segments, you can select and then process only the important segments of the image instead of processing the entire image.
A common technique is to look for abrupt discontinuities in pixel values, which typically indicate edges that define a region.
Using the adaptthresh function in Image Processing Toolbox for adaptive image thresholding. Converting to a binary image improves the legibility of the text in an image. (See MATLAB code.)
Another common approach is to detect similarities in the regions of an image. Some techniques that follow this approach are region growing, clustering, and thresholding.
Segmenting regions based on color values, shapes, or texture using Image Processing Toolbox. (See documentation.)
Keep Exploring This Topic
Image Segmentation with MATLAB
With MATLAB®, you can:
- Use apps to interactively explore different segmentation techniques
- Simplify image analysis workflows using built-in image segmentation algorithms
- Perform deep learning for image segmentation
Using Apps to Interactively Threshold Images
Image Segmenter App
Using the interactive Image Segmenter app, you can iteratively try several methods to segment an image to achieve the desired result. For example, you can use the app to segment and further refine the images of a car.
Using the Image Segmenter app to interactively apply different segmentation techniques.
Color Thresholder App
The Color Thresholder app lets you apply thresholding to color images by manipulating the color of the images interactively, based on different color spaces. For example, with the Color Thresholder app you can create a binary mask using point cloud controls for a color image.
Using a Variety of Image Segmentation Techniques
With functions in MATLAB and Image Processing Toolbox™, you can experiment and build expertise in different image segmentation techniques, including thresholding, clustering, graph-based segmentation, and region growing, as well as deep learning techniques, such as with the Segment Anything Model.
Thresholding
To create a binary image, you can use the imbinarize function to perform thresholding on a 2D or 3D grayscale image. To produce a binary image from an RGB color image, use the rgb2gray function to first convert it to a grayscale image.
Using the imbinarize function in Image Processing Toolbox to convert a color image to a binary image. (See MATLAB code.)
Clustering
This technique lets you create a segmented labeled image using a specific clustering algorithm. Using k-means clustering–based segmentation, imsegkmeans segments an image into clusters within a color space.
Using an imsegkmeans k-means clustering technique to separate out the patterned background on the floor. (See MATLAB code.)
Graph-Based Segmentation
Graph-based segmentation techniques like lazy snapping enable you to segment an image into foreground and background regions. MATLAB lets you perform this segmentation on your image either programmatically (lazysnapping) or interactively using the Image Segmenter app.
Using the lazysnapping function in Image Processing Toolbox to separate the foreground and background regions. (See MATLAB code.)
Using the Image Segmenter app in Image Processing Toolbox to interactively apply graph-based segmentation.
Region Growing
Region growing is a simple region-based (also classified as a pixel-based) image segmentation method. A popularly used algorithm is activecontour, which examines neighboring pixels of initial seed points and determines iteratively whether the pixel neighbors should be added to the region. You can also perform this segmentation on images using the Image Segmenter app.
Deep Learning for Image Segmentation
Using convolutional neural networks (CNNs), a deep learning technique called semantic segmentation lets you associate every pixel of an image with a class label. Applications for semantic segmentation include autonomous driving, industrial inspection, robotics, medical imaging, and satellite image analysis. See the Semantic Segmentation Using Deep Learning example to learn more.
Schematic of the semantic segmentation technique.
You can design and train semantic segmentation networks with a collection of images and their corresponding labeled images, and then use the trained network to label new images. To label the training images, you can use the Image Labeler, Video Labeler, or Ground Truth Labeler apps.
Using the Image Labeler app to perform semantic segmentation.
Image segmentation is a technique in digital image processing that partitions an image into multiple parts or regions based on characteristics of the pixels, such as separating foreground from background or clustering regions by color or shape.
Image segmentation enables focused processing on important segments rather than entire images, with critical applications in medical imaging for tumor detection, autonomous driving for object identification, visual inspection for anomaly detection, video surveillance, and machine vision.
Image segmentation converts an image into regions of pixels represented by a mask or labeled image using techniques like detecting abrupt discontinuities in pixel values; identifying similarities through region growing, clustering, and thresholding; or utilizing deep learning segmentation networks.
The Image Segmenter app is an interactive tool in MATLAB that lets you iteratively try several segmentation methods, apply techniques like thresholding or active contours, and refine results to achieve desired segmentation outcomes.
MATLAB offers thresholding with imbinarize, k-means clustering with imsegkmeans, graph-based methods like lazy snapping, region growing with activecontour, and deep learning techniques, including the Segment Anything Model.
You can use semantic segmentation with convolutional neural networks (CNNs) to associate every pixel with a class label, designing and training networks with collections of images and their corresponding labeled images using apps like Image Labeler, Medical Image Labeler, and Video Labeler.
The Color Thresholder app lets you apply thresholding to color images by interactively manipulating colors based on different color spaces, enabling you to create binary masks using point cloud controls.
Yes, MATLAB supports medical image segmentation for labeling organs, tumors, cells, and other regions of interest in X-rays, CT scans, MRI scans, and microscopy images for clinical diagnosis, treatment planning, and research.
Resources
Expand your knowledge through documentation, examples, videos, and more.
Documentation
