Main Content

Get Started with the Image Labeler

The Image Labeler app provides an easy way to mark rectangular region of interest (ROI) labels, polyline ROI labels, pixel ROI labels, and scene labels in a video or image sequence. This example gets you started using the app by showing you how to:

  • Manually label an image frame from an image collection.

  • Automatically label across image frames using an automation algorithm.

  • Export the labeled ground truth data.

ROI and Scene Label Definitions

  • An ROI label corresponds to either a rectangular, polyline, or pixel region of interest. These labels contain two components: the label name, such as "cars," and the region you create.

  • A Scene label describes the nature of a scene, such as "sunny." You can associate this label with a frame.

Load Unlabeled Data

Open the app and load a collection of images. You can load images stored in a datastore, from a folder, or load a previous labeler session. The images must be readable by imread.

imageFolder = fullfile(toolboxdir('vision'),'visiondata','stopSignImages')
imds = imageDatastore(imageFolder)
imageLabeler(imds)
imageFolder = fullfile(toolboxdir('vision'),'visiondata','stopSignImages')
imageLabeler(imageFolder)

Alternatively, open the app from the Apps tab, under Image Processing and Computer Vision. Then, from the Load menu, load an images data source.

Create Label Definitions

Define the labels you intend to draw. In this example, you define labels directly within the app. To define labels from the MATLAB® command line instead, use the labelDefinitionCreator.

Create ROI Labels

An ROI label is a label that corresponds to a region of interest (ROI). You can define these types of ROI labels.

ROI LabelDescriptionExample: Driving Scene
RectangleDraw rectangular ROI labels (bounding boxes) around objects.

Vehicles, pedestrians, road signs

Projected cuboidDraw cuboidal ROI labels (3-D bounding boxes).

cuboid containing image of car

LineDraw linear ROI labels to represent lines. To draw a polyline ROI, use two or more points.

Lane boundaries, guard rails, road curbs

Pixel labelAssign labels to pixels for semantic segmentation. You can label pixels manually using polygons, brushes, or flood fill. For more on pixel labeling, see Label Pixels for Semantic Segmentation.

Vehicles, road surface, trees, pavement

In this example, you define a vehicle group for labeling types of vehicles, and then create a Rectangle ROI label for a Car and a Truck. Optionally, you can use the Show ROI Labels drop-down menu to select On Hover, Always, or Never to control how the ROI label names appear during labeling. By default, the names will appear when you hover on an ROI.

  1. In the ROI Labels pane on the left, click Label.

  2. Create a Rectangle label named Car.

  3. Optionally, change the label color by clicking the preview color.

  4. From the Group drop-down menu, select New Group and name the group Vehicle

  5. Click OK.

    The Vehicle group name appears in the ROI Labels pane with the label Car created. You can move a label in the list to a different position or group in the list by left-clicking and dragging the label up or down.

  6. Add a second label. Click Label. Name the label Truck and make sure the Vehicle group is selected. Click OK.

  7. Use the mouse to draw rectangular Car ROIs around the two vehicles.

Create Sublabels

A sublabel is a type of ROI label that corresponds to a parent ROI label. Each sublabel must belong to, or be a child of, a specific label defined in the ROI Labels pane. For example, in a driving scene, a vehicle label might have sublabels for headlights, license plates, or wheels.

Define a sublabel for headlights.

  1. In the ROI Labels pane on the left, click the Car label.

  2. Click Sublabel.

  3. Create a Rectangle sublabel named headlight and optionally write a description. Click OK.

    The headlight sublabel appears in the ROI Labels pane. The sublabel is nested under the selected ROI label, Car, and has the same color as its parent label.

    You can add multiple sublabels under a label. You can also drag-and-drop the sublabels to reorder them in the list. Right-click any label for additional edits.

  4. In the ROI Labels pane, select the headlight sublabel.

  5. In the image frame, select the Car label. The label turns yellow when selected. You must select the Car label (parent ROI) before you can add a sublabel to it.

    Draw headlight sublabels for each of the cars.

  6. Repeat the previous steps to label the headlights of the other car. To draw the labels more precisely, use the pan and zoom options located in the upper-right corner of the labeling window.

Sublabels can only be used with rectangular or polyline ROI labels and cannot have their own sublabels. For more details on working with sublabels, see Use Sublabels and Attributes to Label Ground Truth Data.

Create Attributes

An attribute provides further categorization of an ROI label or sublabel. Attributes specify additional information about a drawable label. For example, in a driving scene, attributes might include the type or color of a vehicle.

You can define these types of attributes.

Attribute TypeSample Attribute DefinitionSample Default Values

Numeric Value

String

Logical

List

Add an attribute for the vehicle type.

  1. In the ROI Labels pane on the left, select the Car label and click Attribute.

  2. In the Attribute Name box, type carType. Set the attribute type to List.

  3. In the List Items section, type different types of cars, such as Sedan, Hatchback, and Wagon, each on its own line. Optionally give the attribute a description, and click OK.

  4. In the first frame of the video, select a Car ROI label. In the Attributes and Sublabels pane, select the appropriate carType attribute value for that vehicle.

  5. Repeat the previous step to assign a carType attribute to the other vehicle.

You can also add attributes to sublabels. Add an attribute for the headlight sublabel that tells whether the headlight is on.

  1. In the ROI Labels pane on the left, select the headlight sublabel and click Attribute.

  2. In the Attribute Name box, type isOn. Set the attribute type to Logical. Leave the Default Value set to Empty, optionally write a description, and click OK.

  3. Select a headlight in the video frame. Set the appropriate isOn attribute value, or leave the attribute value set to Empty.

  4. Repeat the previous step to set the isOn attribute for the other headlights.

To delete an attribute, right-click an ROI label or sublabel, and select the attribute to delete. Deleting the attribute removes attribute information from all previously created ROI label annotations.

Create Scene Labels

A scene label defines additional information for the entire scene. Use scene labels to describe conditions, such as lighting and weather, or events, such as lane changes.

Create a scene label to use in the video.

  1. Select the Scene Labels tab on the left.

  2. Click the Define new scene label button, and create a scene label named sunny. Make sure Group is set to None. Click OK.

    The Scene Labels pane shows the scene label definition.

  3. You can apply the label to just the current frame or to an interval of frames. With the sunny scene label definition still selected in the Scene Labels pane, select Time Interval.

  4. Click the Add Label. A checkmark appears for the sunny scene label indicating that the label now applies to all frames in the time interval.

  5. To edit or delete a scene label, right-click on the label and select either Edit Label or Delete Label.

Label Ground Truth

So far, you have labeled only one frame in the video. To label the remaining frames, choose one of these options.

Label Ground Truth Manually

When you click the right arrow key to advance to the next frame, the ROI labels from the previous frame do not carry over. Only the sunny scene label applies to each frame, because this label was applied over the entire time interval.

Advance frame by frame and draw the label and sublabel ROIs manually. Also update the attribute information for these ROIs.

Label Ground Truth Using Automation Algorithm

To speed up the labeling process, you can use an automation algorithm within the app. You can either define your own automation algorithm, see Create Automation Algorithm for Labeling or use a built-in automation algorithm. In this example, you label the ground truth using a built-in algorithm.

After using an automation algorithm you can manually label the remaining frames with sublabel and attribute information.

To further evaluate your labels, you can view a visual summary of the labeled ground truth. From the app toolstrip, select View Label Summary. Use this summary to compare the frames, frequency of labels, and scene conditions. For more details, see View Summary of Ground Truth Labels. This summary does not support sublabels or attributes.

Export Labeled Ground Truth

You can export the labeled ground truth to a MAT-file or to a variable in the MATLAB workspace. In both cases, the labeled ground truth is stored as a groundTruth object. You can use this object to train a deep-learning-based computer vision algorithm. For more details, see Training Data for Object Detection and Semantic Segmentation.

Note

If you export pixel data, the pixel label data and ground truth data are saved in separate files but in the same folder. For considerations when working with exported pixel labels, see How Labeler Apps Store Exported Pixel Labels.

In this example, you export the labeled ground truth to the MATLAB workspace. From the app toolstrip, select Export Labels > To Workspace. The exported MATLAB variable is gTruth.

Display the properties of the exported groundTruth object. The information in your exported object might differ from the information shown here.

gTruth
gTruth = 

  groundTruth with properties:

          DataSource: [1×1 groundTruthDataSource]
    LabelDefinitions: [2x6 table]
           LabelData: [531×3 timetable]

Data Source

DataSource is a groundTruthDataSource object containing the path to the images or video and timestamps. Display the properties of this object.

gTruth.DataSource
ans = 

groundTruthDataSource for a video file with properties

        Source: ...matlab\toolbox\vision\visiondata\visiontraffic.avi
    TimeStamps: [531×1 duration]

Label Definitions

LabelDefinitions is a table containing information about the label definitions. This table does not contain information about the labels that are drawn on the video frames. To save the label definitions in their own MAT-file, from the app toolstrip, select Save > Label Definitions. You can then import these label definitions into another app session by selecting Import Files.

Display the label definitions table. Each row contains information about an ROI label definition or a scene label definition. If you exported pixel label data, the LabelDefinitions table also includes a PixelLabelID column containing the ID numbers for each pixel label definition.

gTruth.LabelDefinitions
ans =
  3×6 table

      Name         Type        LabelColor        Group       Description     Hierarchy  
    _________    _________    ____________    ___________    ___________    ____________

    {'Car'  }    Rectangle    {1×3 double}    {'Vehicle'}    {0×0 char}     {1×1 struct}
    {'Truck'}    Rectangle    {1×3 double}    {'Vehicle'}    {0×0 char}     {0×0 double}
    {'Sunny'}    Scene        {1×3 double}    {'Weather'}    {0×0 char}     {0×0 double} 

Within LabelDefinitions, the Hierarchy column stores information about the sublabel and attribute definitions of a parent ROI label.

Display the sublabel and attribute information for the Car label.

gTruth.LabelDefinitions.Hierarchy{1}
ans = 

  struct with fields:
       numDoors: [1×1 struct]
          color: [1×1 struct]
       inMotion: [1×1 struct]
        carType: [1×1 struct]
      headlight: [1×1 struct]
           Type: Rectangle
    Description: ''

Display information about the headlight sublabel.

gTruth.LabelDefinitions.Hierarchy{1}.headlight
ans = 

  struct with fields:
           Type: Rectangle
    Description: ''
          Color: [0.5862 0.8276 0.3103]
           isOn: [1×1 struct]

Display information about the carType attribute.

gTruth.LabelDefinitions.Hierarchy{1}.carType
ans = 

  struct with fields:

      ListItems: {3×1 cell}
    Description: ''

Save App Session

From the app toolstrip, select Save and save a MAT-file of the app session. The saved session includes the data source, label definitions, and labeled ground truth. It also includes your session preferences, such as the layout of the app. To change layout options, select Layout.

At any time during a session, you can select New Session to start a new session. You have the option of saving the current session or cancelling.

The app session MAT-file is separate from the ground truth MAT-file that is exported when you select Export > From File. To share labeled ground truth data, as a best practice, share the ground truth MAT-file containing the groundTruth object, not the app session MAT-file. For more details, see Share and Store Labeled Ground Truth Data.

See Also

Apps

Objects

Related Topics