Create Automation Algorithm Function for Labeling

This topic shows how to implement a custom automation algorithm using a function-based interface to accelerate ground truth labeling across labeling apps such as Image Labeler, Video Labeler, Lidar Labeler (Lidar Toolbox), and Ground Truth Labeler (Automated Driving Toolbox). The labeling apps enable you to label ground truth for a variety of data sources and automatically label your data by creating and importing a custom automation algorithm.

To build a custom automation algorithm for labeling apps, you can use either a function-based or class-based interface. The apps provide templates for both interface types. The table below compares their capabilities to help you choose the right interface for your workflow:

Automation Algorithm Interface Type	Description	Use Case & Capabilities
Function-based	Define automation logic using a standalone function with parameter tuning.	Ideal for quick setup and prototyping simple automation tasks Useful for migrating existing functional code Lightweight and fast to implement
Class-based	Create a custom class inheriting from vision.labeler.AutomationAlgorithm, which offers full control over labeling behavior and app interaction.	Suitable for complex workflows requiring customization Provides access to temporal information using vision.labeler.mixin.Temporal class. Supports automating blockedImage labels using vision.labeler.mixin.BlockedImageAutomation class. Provides customized initialization and termination steps. Enables implementation of customized settings dialog. Enables customized algorithm state management across image frames. Supports multiple signals or point cloud signals, such as what you might use with the Ground Truth Labeler (Automated Driving Toolbox). Customized name, description algorithm instructions.

For more information about the class-based automation algorithm interface, see Create Custom Automation Algorithm for Labeling.

How to Specify an Automation Function in an App

To create an automation algorithm using the built-in function template:

1. On the app toolstrip, click Select Algorithm > Custom Automation Function.

2. In the BrowserPanelDisplay pane, select the images for which to automate labeling, then click Automate in the toolbar.

3. On the app toolstrip, select Settings, and then specify the algorithm function. If you need to create a new function, you can click the blue information icon to open the automation function template.

The app invokes the automation algorithm on each image selected for automation. The app returns the labels created by the automation algorithm in an autoLabels structure. To automate pixel labeling, the autoLabels structure must be a categorical matrix. Otherwise, autoLabels must be a structure or a table.

Use a Function to Automate Labeling with Your Custom Detector

The labeler app built-in algorithms may not work to explicitly detect the features unique to your data. Therefore, you can train a detector using your data, and then create a custom algorithm using the function template provided within the app. The function requires a minimum set of parameters, which are related to the type of labels suited to your detector. Specifying a function handle within the app enables you to quickly test different automation algorithms and change the parameters of your algorithm.

This is an example of a function that creates an algorithm to use with a labeling app. It runs a pretrained aggregate channel features (ACF) object detector to label people in the input image. The function returns the predicted labels autoLabels, which is a structure array that contains the Name, Type, and Position fields.

 Example Using a Custom Detector

function autoLabels = exampleAutomationAlgorithmFunction(I)
 
% One-time initialization of the detector. A one-time initialization saves
% time on subsequent runs.
persistent detector
if isempty(detector)
    detector = peopleDetectorACF();
end
 
% Run the detector on the input image, I.
bboxes = detect(detector,I);
 
% Create and fill the autoLabels structure with the predicted bounding box
% locations. The Name and Type of ROI returned by the automation function
% must match one of the labels defined in the labeling app.
autoLabels = struct("Name",{},"Type",{},"Position",{});
for i = 1:size(bboxes,1)
    autoLabels(i).Name = "people";
    autoLabels(i).Type = labelType.Rectangle;
    autoLabels(i).Position = bboxes(i,:);
end

Create an Automation Algorithm Function

The function template contains descriptions for the fields in autoLabels and an example of how to set the fields. The template also specifies where to insert your custom algorithm function by name, or by specifying a function handle. Use a function handle to pass additional inputs to your function, if required. To access the template, select Settings, and then click the blue information icon in the Custom Automation Function Settings dialog box. The template contains this information:

 Template for Automation Function

function autoLabels = myAutomationFunction(I)
% Your automation function is invoked on each image, I, chosen for
% automation in the labeler app. Implement your automation algorithm below
% and return automated labels in autoLabels. autoLabels must be a
% categorical matrix for automating pixel labeling. Otherwise, autoLabels
% must be a struct or table with fields Type, Name, Position and optionally
% Attributes. The Attributes field is valid only when labels with
% attributes are defined in the app.
%
% The fields of the autoLabels struct array are described below:
%
%     Type        A labelType enumeration that defines the type of label.
%                 Type can have values Rectangle, Line, Polygon, Projected
%                 cuboid, or Scene.
%
%     Name        A character vector specifying a label name. Only
%                 existing label names previously defined in the
%                 labeler app can be used.
%
%     Position    Positions of the labels. The type of label determines
%                 the format of the position data. For more information,
%                 see the doc page for vision.labeler.AutomationAlgorithmFunction.
%
%     Attributes  An array of structs representing the attributes
%                 contained by the automated labels. Each attribute
%                 is specified as a field of the struct, with the
%                 name of the field representing the name of the
%                 attribute and the value of the field representing
%                 the value of the attribute.
%
% Below is an example of how to specify an autoLabels structure for an
% algorithm that detects a car, finds a lane, and classifies the
% scene as sunny.
%
% % Rectangle labeled 'Car' positioned with top-left at (20,20)
% % with width and height equal to 50.
% autoLabels(1).Name      = 'Car';
% autoLabels(1).Type      = labelType('Rectangle');
% autoLabels(1).Position  = [20 20 50 50];
%
% % Line labeled 'LaneMarker' with 3 points.
% autoLabels(2).Name      = 'LaneMarker';
% autoLabels(2).Type      = labelType('Line');
% autoLabels(2).Position  = [100 100; 100 110; 110 120];
%
% % Scene labeled 'Sunny'
% autoLabels(3).Name      = 'Sunny';
% autoLabels(3).Type      = labelType('Scene');
% autoLabels(3).Position  = true;

%--------------------------------------------------------
% Place your algorithm code here
%--------------------------------------------------------