Main Content

Get Started with the Image Labeler

The Image Labeler app provides an easy way to interactively create a variety of shapes to mark as region of interest (ROI) labels. You can create rectangular, polyline, pixel, and polygon ROI labels and scene labels in an image or image sequence.

You can use labeled data to validate or train algorithms such as image classifiers, object detectors, and semantic and instance segmentation networks. Consider your application when choosing a labeling drawing tool to create ROI labels. For more details on how to select the right label type and drawing tool for your application, see ROI Labels, Sublabels, and Attributes.

This example gets you started using the app by showing you how to:

  • Load unlabeled data

  • Manually label an image frame from an image collection.

  • Automatically label across image frames using an automation algorithm.

  • Export the labeled ground truth data.

ROI and Scene Label Definitions

  • An ROI label corresponds to either a rectangular, polyline, pixel, or polygon region of interest. These labels contain two components: the label name, such as "cars," and the region you create.

  • A Scene label describes the nature of a scene, such as "sunny." You can associate this label with a frame.

Load Unlabeled Data

You can load images stored in a datastore, from a folder, or load a previous labeler session. The images must be readable by imread.

Load Data Programmatically

To programmatically load images from a datastore:

  1. Create a datastore from a folder of images:

    imageFolder = fullfile(toolboxdir('vision'),'visiondata','boats')
    imds = imageDatastore(imageFolder)
    

  2. Load the datastore:

    imageLabeler(imds)

To programmatically load a folder of images:

imageFolder = fullfile(toolboxdir('vision'),'visiondata','boats')
imageLabeler(imageFolder)

Load Data Directly from the App

Open the app from the Apps tab, under Image Processing and Computer Vision. Click Import to load images or click Open Session to open a saved session.

Create Label Definitions

Define the labels you intend to draw. In this example, you define labels directly within the app. To define labels from the MATLAB® command line instead, use the labelDefinitionCreator.

Create ROI Label

An ROI label is a label that corresponds to a region of interest (ROI) in an image. You can define these ROI label types.

  • Rectangle — Draw 2-D rectangular bounding box labels around objects in an image, such as vehicles, boats, buildings.

  • Projected cuboid — Draw 3-D bounding box labels around objects in an image, such as vehicles, boats, buildings.

  • Line — Draw linear ROIs to label lines, such as lane boundaries.

  • Pixel label — Draw pixels to label various classes, such as road or sky, for semantic segmentation. For more information about pixel labeling, see Label Pixels for Semantic Segmentation.

  • Polygon — Draw polygon labels around objects. You can label distinct instances of the same class. For more information on drawing polygon ROI labels for instance and semantic segmentation networks, see Label Objects Using Polygons

For more details about these ROI label definitions, see ROI Labels, Sublabels, and Attributes.

In this example, you define a Boat group for labeling types of boats, and then create a Rectangle ROI label for a Sailboat and a Tanker. To control showing the ROI label names during labeling, select On Hover, Always, or Never from the Show ROI Labels drop-down menu.

  1. In the ROI Labels pane on the left, click Label.

  2. Create a Rectangle label type named Sailboat.

  3. Optionally, change the label color by clicking the preview color.

  4. From the Group drop-down menu, select New Group ... and name the group Boats

  5. Click OK.

    The Boats group name appears in the ROI Labels pane with the label Sailboat created. You can move a label in the list to a different position or group in the list by left-clicking and dragging the label up or down.

  6. To add a second type of Boats label, select the group Boats, then click Label. Name the label Tanker. Click OK.

  7. Select the Sailboat sublabel, then use the mouse to draw a rectangular ROI around the a sailboat. Select the Tanker sublabel to draw an ROI around the tanker ship.

    Sailboat in the foreground and tanker ship in the background

Create Sublabels

A sublabel is a type of ROI label that corresponds to a parent ROI label. Each sublabel must belong to, or be a child of, a specific label defined in the ROI Labels pane. For example, in the ocean scene, a sailboat label might have sublabels for sails. For more details about sublabels, see ROI Labels, Sublabels, and Attributes.

Define a sublabel for sails.

  1. In the ROI Labels pane on the left, click the Sailboat label.

  2. Click Sublabel.

  3. Create a Rectangle sublabel named sail and optionally write a description. Click OK.

    The sail sublabel appears in the ROI Labels pane. The sublabel is nested under the selected ROI label, Sailboat, and by default has the same color as its parent label.

    You can add multiple sublabels under a label. You can also drag-and-drop the sublabels to reorder them in the list and right-click any label for additional edits.

  4. In the ROI Labels pane, select the sail sublabel.

  5. In the image frame, select the Sailboat label. The label turns yellow when selected. You must select the Sailboat label (parent ROI) before you can draw a sublabel for it.

    Draw sail sublabels for each of the sails.

Sublabels cannot have their own sublabels and they can only be used with rectangular or polyline ROI labels.

Create ROI Attribute

An ROI attribute specifies additional information about an ROI label or sublabel. For example, in a driving scene, attributes might include the type or color of a vehicle. In the ocean scene, attributes might include the type of sailboat or number of sails. You can define ROI attributes of these types.

  • Numeric Value — Specify a numeric scalar attribute, such as the number of doors on a labeled vehicle or the number of sails on a sailboat.

  • String — Specify a string scalar attribute, such as the color of a vehicle or boat.

  • Logical — Specify a logical true or false attribute, such as whether a vehicle is in motion or a boat is making way.

  • List — Specify a drop-down list attribute of predefined strings, such as make or model of a vehicle or boat.

For more details about these attribute types, see ROI Labels, Sublabels, and Attributes.

Add an attribute to a Label.

  1. In the ROI Labels pane on the left, select the Sailboat label and click Attribute.

  2. In the Attribute Name box, type sailboatType. Set the attribute type to List.

  3. In the List Items section, type different types of sailboats, such as Monohull, Catamaran, and Multi-hull, each on its own line. Optionally give the attribute a description, and click OK.

  4. Select a Sailboat ROI label. In the Attributes and Sublabels pane, select the appropriate sailboatType attribute value for that vehicle.

You can also add attributes to sublabels. Add an attribute for the sail sublabel to indicate whether it is a foresail or a mainsail.

  1. In the ROI Labels pane on the left, select the sail sublabel and click Attribute.

  2. In the Attribute Name box, type sailRaised. Set the attribute type to Logical. Leave the Default Value set to Empty, optionally write a description, and click OK.

  3. Select a sail in the scene. In the right-hand pane, set the appropriate isOn attribute value, or leave the attribute value set to Empty.

    The "Attributes and Sublabels" pane showing the sailRaised attribute with "True" selected

To delete an attribute, right-click an ROI label or sublabel, and select the attribute to delete. Deleting the attribute removes attribute information from all previously created ROI label annotations.

Create Scene Labels

A scene label defines additional information for the entire scene. Use scene labels to describe conditions, such as lighting and weather, or events, such as lane changes or point of sail.

Create a scene label to apply to an image.

  1. In the left pane of the app, select the Scene Labels next to the ROI Labels tab.

  2. Click Define new scene label, and in the Label Name box, enter a scene label named daytime.

  3. Change the color of the label definition to light blue to reflect the nature of the scene label. Under the Color parameter, click the color preview and select the standard light blue colors. Then, click OK to close the color selection window.

  4. Leave the Group parameter set to the default of None and click OK. The Scene Labels pane shows the scene label definition.

  5. Click Apply to Image to apply the daytime label to the scene. A checkmark appears for the scene label.

  6. To edit or delete a scene label, right-click on the label and select either Edit Label or Delete Label.

Label Ground Truth

You can label images manually, use a suitable built-in automation algorithm, create a new algorithm, or import an algorithm.

Label Ground Truth Manually

The ROI and Scene labels are defined for your entire session and all images, but you must draw the labels, and sublabels for each image and update the attribute information.

Label Ground Truth Using an Automation Algorithm

To speed up the labeling process, you can use an automation algorithm to label the remainder of your images. Select one of these types of automation algorithms from the app toolstrip Automate Labeling > Select Algorithm section.

  • A built-in automation algorithm — Select a suitable algorithm. Follow the steps that appear in the right pane.

  • Add algorithm — Create an automation. For details on how to define your own automatino algorithm, see Create Automation Algorithm for Labeling.

  • Import algorithm — Import a custom automation algorithm. For more details about importing an automation algorithm, see Create Automation Algorithm for Labeling.

After using an automation algorithm you can manually label the remaining frames with sublabel and attribute information.

To further evaluate your labels, you can view a visual summary of the labeled ground truth. From the app toolstrip, select View Label Summary. Use this summary to compare the frames, frequency of labels, and scene conditions. For more details, see View Summary of Ground Truth Labels. This summary does not show sublabels or attributes.

Export Labeled Ground Truth

You can export the labeled ground truth to a MAT-file or to a variable in the MATLAB workspace. In both cases, the labeled ground truth is stored as a groundTruth object. You can use this object to train a deep-learning-based computer vision algorithm. For more details, see Training Data for Object Detection and Semantic Segmentation.

Note

If you export pixel data, the pixel label data and ground truth data are saved in separate files but in the same folder. For considerations when working with exported pixel labels, see How Labeler Apps Store Exported Pixel Labels.

In this example, you export the labeled ground truth to the MATLAB workspace. From the app toolstrip, select Export Labels > To Workspace. The exported MATLAB variable is gTruth.

Display the properties of the exported groundTruth object. The information in your exported object might differ from the information shown here.

gTruth
gTruth = 

  groundTruth with properties:

          DataSource: [1×1 groundTruthDataSource]
    LabelDefinitions: [2x6 table]
           LabelData: [531×3 timetable]

Data Source

DataSource is a groundTruthDataSource object containing the path to the images or video and timestamps. Display the properties of this object.

gTruth.DataSource
ans = 

groundTruthDataSource for a video file with properties

        Source: ...matlab\toolbox\vision\visiondata\visiontraffic.avi
    TimeStamps: [531×1 duration]

Label Definitions

LabelDefinitions is a table containing information about the label definitions. This table does not contain information about the labels that are drawn on the video frames. To save the label definitions in their own MAT-file, from the app toolstrip, select Save > Label Definitions. You can then import these label definitions into another app session by selecting Import Files.

Display the label definitions table. Each row contains information about an ROI label definition or a scene label definition. If you exported pixel label data, the LabelDefinitions table also includes a PixelLabelID column containing the ID numbers for each pixel label definition.

gTruth.LabelDefinitions
ans =
  3×6 table

      Name         Type        LabelColor        Group       Description     Hierarchy  
    _________    _________    ____________    ___________    ___________    ____________

    {'Car'  }    Rectangle    {1×3 double}    {'Vehicle'}    {0×0 char}     {1×1 struct}
    {'Truck'}    Rectangle    {1×3 double}    {'Vehicle'}    {0×0 char}     {0×0 double}
    {'Sunny'}    Scene        {1×3 double}    {'Weather'}    {0×0 char}     {0×0 double} 

Within LabelDefinitions, the Hierarchy column stores information about the sublabel and attribute definitions of a parent ROI label.

Display the sublabel and attribute information for the Car label.

gTruth.LabelDefinitions.Hierarchy{1}
ans = 

  struct with fields:
       numDoors: [1×1 struct]
          color: [1×1 struct]
       inMotion: [1×1 struct]
        carType: [1×1 struct]
      headlight: [1×1 struct]
           Type: Rectangle
    Description: ''

Display information about the headlight sublabel.

gTruth.LabelDefinitions.Hierarchy{1}.headlight
ans = 

  struct with fields:
           Type: Rectangle
    Description: ''
          Color: [0.5862 0.8276 0.3103]
           isOn: [1×1 struct]

Display information about the carType attribute.

gTruth.LabelDefinitions.Hierarchy{1}.carType
ans = 

  struct with fields:

      ListItems: {3×1 cell}
    Description: ''

Save App Session

From the app toolstrip, select Save and save a MAT-file of the app session. The saved session includes the data source, label definitions, and labeled ground truth. It also includes your session preferences, such as the layout of the app. To change layout options, select Layout.

At any time during a session, you can select New Session to start a new session. You have the option of saving the current session or cancelling.

The app session MAT-file is separate from the ground truth MAT-file that is exported when you select Export > From File. To share labeled ground truth data, as a best practice, share the ground truth MAT-file containing the groundTruth object, not the app session MAT-file. For more details, see Share and Store Labeled Ground Truth Data.

See Also

Apps

Objects

Related Topics