multiObjectTracker

Track objects using GNN assignment

Description

The multiObjectTracker System object™ initializes, confirms, predicts, corrects, and deletes the tracks of moving objects. Inputs to the multi-object tracker are detection reports generated by an objectDetection object, drivingRadarDataGenerator object, or visionDetectionGenerator object. The multi-object tracker accepts detections from multiple sensors and assigns them to tracks using a global nearest neighbor (GNN) criterion. Each detection is assigned to a separate track. If the detection cannot be assigned to any track, based on the AssignmentThreshold property, the tracker creates a new track. The tracks are returned in a structure array.

A new track starts in a tentative state. If enough detections are assigned to a tentative track, its status changes to confirmed. If the detection is a known classification (the ObjectClassID field of the returned track is nonzero), that track can be confirmed immediately. For details on the multi-object tracker properties used to confirm tracks, see Algorithms.

When a track is confirmed, the multi-object tracker considers that track to represent a physical object. If detections are not added to the track within a specifiable number of updates, the track is deleted.

The tracker also estimates the state vector and state vector covariance matrix for each track using a Kalman filter. These state vectors are used to predict a track's location in each frame and determine the likelihood of each detection being assigned to each track.

To track objects using a multi-object tracker:

1. Create the multiObjectTracker object and set its properties.

2. Call the object with arguments, as if it were a function.

To learn more about how System objects work, see What Are System Objects?

Creation

Syntax

tracker = multiObjectTracker

tracker = multiObjectTracker(Name,Value)

Description

tracker = multiObjectTracker creates a multiObjectTracker System object with default property values.

tracker = multiObjectTracker(Name,Value) sets properties for the multi-object tracker using one or more name-value pairs. For example, multiObjectTracker('FilterInitializationFcn',@initcvukf,'MaxNumTracks',100) creates a multi-object tracker that uses a constant-velocity, unscented Kalman filter and maintains a maximum of 100 tracks. Enclose each property name in quotes.

Properties

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the release function unlocks them.

If a property is tunable, you can change its value at any time.

For more information on changing property values, see System Design in MATLAB Using System Objects.

TrackerIndex — Unique tracker identifier

0 (default) | nonnegative integer

Unique tracker identifier, specified as a nonnegative integer. This property is used as the SourceIndex in the tracker outputs, and distinguishes tracks that come from different trackers in a multiple-tracker system. You must specify this property as a positive integer to use the track outputs as inputs to a track fuser.

Example: 1

FilterInitializationFcn — Kalman filter initialization function

@initcvkf (default) | function handle | character vector | string scalar

Kalman filter initialization function, specified as a function handle or as a character vector or string scalar of the name of a valid Kalman filter initialization function.

Automated Driving Toolbox™ supplies several initialization functions that you can use to specify FilterInitializationFcn.

Initialization Function	Function Definition
initcvekf	Initialize constant-velocity extended Kalman filter.
initcvkf	Initialize constant-velocity linear Kalman filter.
initcvukf	Initialize constant-velocity unscented Kalman filter.
initcaekf	Initialize constant-acceleration extended Kalman filter.
initcakf	Initialize constant-acceleration linear Kalman filter.
initcaukf	Initialize constant-acceleration unscented Kalman filter.
initctekf	Initialize constant-turnrate extended Kalman filter.
initctukf	Initialize constant-turnrate unscented Kalman filter.

You can also write your own initialization function. The input to this function must be a detection report created by objectDetection. The output of this function must be a Kalman filter object: trackingKF, trackingEKF, or trackingUKF. To guide you in writing this function, you can examine the details of the supplied functions from within MATLAB^®. For example:

type initcvkf

Data Types: function_handle | char | string

AssignmentThreshold — Detection assignment threshold

30*[1 Inf] (default) | positive scalar | 1-by-2 vector of positive values

Detection assignment threshold (or gating threshold), specified as a positive scalar or a 1-by-2 vector of [C₁,C₂], where C₁≤C₂. If specified as a scalar, the specified value, val, will be expanded to [val, Inf].

Initially, the tracker executes a coarse estimation for the normalized distance between all the tracks and detections. The tracker only calculates the accurate normalized distance for the combinations whose coarse normalized distance is less than C₂. Also, the tracker can only assign a detection to a track if their accurate normalized distance is less than C₁. See the distance function used with tracking filters (for example, trackingEKF) for an explanation of the distance calculation.

Tips:

• Increase the value of C₂ if there are combinations of track and detection that should be calculated for assignment but are not. Decrease it if cost calculation takes too much time.

• Increase the value of C₁ if there are detections that should be assigned to tracks but are not. Decrease it if there are detections that are assigned to tracks they should not be assigned to (too far away).

Tunable: Yes

MaxNumTracks — Maximum number of tracks

200 (default) | positive integer

Maximum number of tracks that the tracker can maintain, specified as a positive integer.

Data Types: double

MaxNumSensors — Maximum number of sensors

20 (default) | positive integer

Maximum number of sensors that can be connected to the tracker, specified as a positive integer. When you specify detections as input to the multi-object tracker, MaxNumSensors must be greater than or equal to the highest SensorIndex value in the detections cell array of objectDetection objects used to update the multi-object tracker. This property determines how many sets of ObjectAttributes fields each output track can have.

Data Types: double

MaxNumDetections — Maximum number of detections

Inf (default) | positive integer

Maximum number of detections that the tracker can take as inputs, specified as a positive integer.

Data Types: single | double

OOSMHandling — Handle out-of-sequence measurement (OOSM)

'Terminate' (default) | 'Neglect'

Handle out-of-sequence measurement (OOSM), specified as 'Terminate' or 'Neglect'. Each detection has a timestamp associated with it, t_d, and the tracker has it own timestamp, t_t, which is updated in each call. The tracker considers a measurement as an OOSM if t_d < t_t.

When the property is specified as

• 'Terminate' — The tracker stops running when it encounters any out-of-sequence measurements.

• 'Neglect' — The tracker neglects any out-of-sequence measurements and continue to run.

Tunable: Yes

ConfirmationThreshold — Threshold for track confirmation

[2 3] (default) | two-element vector of non-decreasing positive integers

Threshold for track confirmation, specified as a two-element vector of non-decreasing positive integers, [M N], where M is less than or equal to N. A track is confirmed if it receives at least M detections in the last N updates.

• When setting M, take into account the probability of object detection for the sensors. The probability of detection depends on factors such as occlusion or clutter. You can reduce M when tracks fail to be confirmed or increase M when too many false detections are assigned to tracks.

• When setting N, consider the number of times you want the tracker to update before it makes a confirmation decision. For example, if a tracker updates every 0.05 seconds, and you allow 0.5 seconds to make a confirmation decision, set N = 10.

Example: [3 5]

Data Types: double

DeletionThreshold — Threshold for track deletion

[5 5] (default) | two-element vector of positive non-decreasing integers

Threshold for track deletion, specified as a two-element vector of positive non-decreasing integers [P Q], where P is less than or equal to Q. If a confirmed track is not assigned to any detection P times in the last Q tracker updates, then the track is deleted.

• Decrease Q (or increase P) if tracks should be deleted earlier.

• Increase Q (or decrease P) if tracks should be kept for a longer time before deletion.

Example: [3 5]

Data Types: single | double

HasCostMatrixInput — Enable cost matrix input

false (default) | true

Enable a cost matrix as input to the multiObjectTracker System object or to the updateTracks function, specified as false or true.

Data Types: logical

HasDetectableTrackIDsInput — Enable input of detectable track IDs

false (default) | true

Enable the input of detectable track IDs at each object update, specified as false or true. Set this property to true if you want to provide a list of detectable track IDs. This list tells the tracker of all tracks that the sensors are expected to detect and, optionally, the probability of detection for each track.

Data Types: logical

StateParameters — Parameters of track state reference frame

struct([]) (default) | struct array

Parameters of the track state reference frame, specified as a structure or a structure array. The tracker passes its StateParameters property values to the StateParameters property of the generated tracks. You can use these parameters to define the reference frame in which the track is reported or other desirable attributes of the generated tracks.

For example, you can use the following structure to define a rectangular reference frame whose origin position is at [10 10 0] meters and whose origin velocity is [2 -2 0] meters per second with respect to the scenario frame.

Field Name	Value
Frame	"Rectangular"
Position	[10 10 0]
Velocity	[2 -2 0]

Tunable: Yes

Data Types: struct

NumTracks — Number of tracks maintained by multi-object tracker

nonnegative integer

This property is read-only.

Number of tracks maintained by the multi-object tracker, specified as a nonnegative integer.

Data Types: double

NumConfirmedTracks — Number of confirmed tracks

nonnegative integer

This property is read-only.

Number of confirmed tracks, specified as a nonnegative integer. The IsConfirmed fields of the output track structures indicate which tracks are confirmed.

Data Types: double

Usage

To update tracks, call the created multi-object tracker with arguments, as if it were a function (described here). Alternatively, update tracks by using the updateTracks function, specifying the multi-object tracker as an input argument.

Syntax

confirmedTracks = tracker(detections,time)

[confirmedTracks,tentativeTracks] = tracker(detections,time)

[confirmedTracks,tentativeTracks,allTracks] = tracker(detections,time)

[___] = tracker(detections,time,costMatrix)

[___] = tracker(___,detectableTrackIDs)

Description

confirmedTracks = tracker(detections,time) creates, updates, and deletes tracks in the multi-object tracker and returns details about the confirmed tracks. Updates are based on the specified list of detections, and all tracks are updated to the specified time. Each element in the returned confirmedTracks corresponds to a single track.

[confirmedTracks,tentativeTracks] = tracker(detections,time) also returns tentativeTracks containing details about the tentative tracks.

[confirmedTracks,tentativeTracks,allTracks] = tracker(detections,time) also returns allTracks containing details about all the confirmed and tentative tracks. The tracks are returned in the order by which the tracker internally maintains them. You can use this output to help you calculate the cost matrix, an optional input argument.

[___] = tracker(detections,time,costMatrix) specifies a cost matrix, returning any of the outputs from preceding syntaxes.

To specify a cost matrix, set the HasCostMatrixInput property of the multiObjectTracker System object to true.

[___] = tracker(___,detectableTrackIDs) also specifies a list of expected detectable tracks given by detectableTrackIDs. This argument can be used with any of the previous input syntaxes.

To enable this syntax, set the HasDetectableTrackIDsInput property to true.

Input Arguments

detections — Detection list

cell array of objectDetection objects

Detection list, specified as a cell array of objectDetection objects. The Time property value of each objectDetection object must be less than or equal to the current time of update, time, and greater than the previous time value used to update the multi-object tracker.

time — Time of update

real scalar

Time of update, specified as a real scalar. The tracker updates all tracks to this time. Units are in seconds.

time must be greater than or equal to the largest Time property value of the objectDetection objects in the input detections list. time must increase in value with each update to the multi-object tracker.

Data Types: double

costMatrix — Cost matrix

N_T-by-N_D matrix

Cost matrix, specified as a real-valued N_T-by-N_D matrix, where N_T is the number of existing tracks, and N_D is the number of current detections. The rows of the cost matrix correspond to the existing tracks. The columns correspond to the detections. Tracks are ordered as they appear in the list of tracks in the allTracks output argument of the previous update to the multi-object tracker.

In the first update to the multi-object tracker, or when the tracker has no previous tracks, assign the cost matrix a size of [0, N_D]. The cost must be calculated so that lower costs indicate a higher likelihood that the tracker assigns a detection to a track. To prevent certain detections from being assigned to certain tracks, use Inf.

Dependencies

To enable specification of the cost matrix when updating tracks, set the HasCostMatrixInput property of the tracker to true

Data Types: double

detectableTrackIDs — Detectable track IDs

real-valued M-by-1 vector | real-valued M-by-2 matrix

Detectable track IDs, specified as a real-valued M-by-1 vector or M-by-2 matrix. Detectable tracks are tracks that the sensors expect to detect. The first column of the matrix contains a list of track IDs that the sensors report as detectable. The optional second column contains the detection probability for the track. The detection probability is either reported by a sensor or, if not reported, obtained from the DetectionProbability property.

Tracks whose identifiers are not included in detectableTrackIDs are considered as undetectable. The track deletion logic does not count the lack of detection as a 'missed detection' for track deletion purposes.

Dependencies

To enable this input argument, set the detectableTrackIDs property to true.

Data Types: single | double

Output Arguments

confirmedTracks — Confirmed tracks

array of objectTrack objects | array of structures

Confirmed tracks, returned as an array of objectTrack objects in MATLAB, and returned as an array of structures in code generation. In code generation, the field names of the returned structure are same with the property names of objectTrack.

A track is confirmed if it satisfies the confirmation threshold specified in the ConfirmationThreshold property. In that case, the IsConfirmed property of the object or field of the structure is true.

Data Types: struct | object

tentativeTracks — Tentative tracks

array of objectTrack objects | array of structures

Tentative tracks, returned as an array of objectTrack objects in MATLAB, and returned as an array of structures in code generation. In code generation, the field names of the returned structure are same with the property names of objectTrack.

A track is tentative if it does not satisfy the confirmation threshold specified in the ConfirmationThreshold property. In that case, the IsConfirmed property of the object or field of the structure is false.

Data Types: struct | object

allTracks — All tracks

array of objectTrack objects | array of structures

All tracks, returned as an array of objectTrack objects in MATLAB, and returned as an array of structures in code generation. In code generation, the field names of the returned structure are same with the property names of objectTrack. All tracks consists of confirmed and tentative tracks.

Data Types: struct | object

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named obj, use this syntax:

release(obj)

Specific to multiObjectTracker

updateTracks	Update multi-object tracker with new detections
initializeTrack	Initialize new track
confirmTrack	Confirm tentative track
deleteTrack	Delete existing track
getTrackFilterProperties	Obtain filter properties of track from multi-object tracker
setTrackFilterProperties	Set filter properties of track from multi-object tracker
predictTracksToTime	Predict track state

Common to All System Objects

step	Run System object algorithm
release	Release resources and allow changes to System object property values and input characteristics
clone	Create duplicate System object
isLocked	Determine if System object is in use
reset	Reset internal states of System object

Examples

Track Single Object Using Multi-Object Tracker

Create a multiObjectTracker System object™ using the default filter initialization function for a 2-D constant-velocity model. For this motion model, the state vector is [x;vx;y;vy].

tracker = multiObjectTracker('ConfirmationThreshold',[4 5], ...
    'DeletionThreshold',10);

Create a detection by specifying an objectDetection object. To use this detection with the multi-object tracker, enclose the detection in a cell array.

dettime = 1.0;
det = { ...
    objectDetection(dettime,[10; -1], ...
    'SensorIndex',1, ...
    'ObjectAttributes',{'ExampleObject',1}) ...
    };

Update the multi-object tracker with this detection by using the updateTracks function. The time at which you update the multi-object tracker must be greater than or equal to the time at which the object was detected.

updatetime = 1.25;
[confirmedTracks,tentativeTracks,allTracks] = updateTracks(tracker,det,updatetime);

Create another detection of the same object and update the multi-object tracker, this time by calling the tracker itself instead of using updateTracks. The tracker maintains only one track.

dettime = 1.5;
det = { ...
    objectDetection(dettime,[10.1; -1.1], ...
    'SensorIndex',1, ...
    'ObjectAttributes',{'ExampleObject',1}) ...
    };
updatetime = 1.75;
[confirmedTracks,tentativeTracks,allTracks] = tracker(det,updatetime);

Determine whether the track has been verified by checking the number of confirmed tracks.

numConfirmed = tracker.NumConfirmedTracks

numConfirmed = 
0

Examine the position and velocity of the tracked object. Because the track has not been confirmed, get the position and velocity from the tentativeTracks structure.

positionSelector = [1 0 0 0; 0 0 1 0];
velocitySelector = [0 1 0 0; 0 0 0 1];
position = getTrackPositions(tentativeTracks,positionSelector)

position = 1×2

   10.1426   -1.1426

velocity = getTrackVelocities(tentativeTracks,velocitySelector)

velocity = 1×2

    0.1852   -0.1852

Confirm and Delete Track in Multi-Object Tracker

Create a sequence of detections of a moving object. Track the detections using a multiObjectTracker System object™. Observe how the tracks switch from tentative to confirmed and then to deleted.

Create a multi-object tracker using the initcakf filter initialization function. The tracker models 2-D constant-acceleration motion. For this motion model, the state vector is [x;vx;ax;y;vy;ay].

tracker = multiObjectTracker('FilterInitializationFcn',@initcakf, ...
    'ConfirmationThreshold',[3 4],'DeletionThreshold',[6 6]);

Create a sequence of detections of a moving target using objectDetection. To use these detections with the multiObjectTracker, enclose the detections in a cell array.

dt = 0.1;
pos = [10; -1];
vel = [10; 5];
for detno = 1:2
    time = (detno-1)*dt;
    det = { ...
        objectDetection(time,pos, ...
        'SensorIndex',1, ...
        'ObjectAttributes',{'ExampleObject',1}) ...
        };
    [confirmedTracks,tentativeTracks,allTracks] = tracker(det,time);
    pos = pos + vel*dt;
    meas = pos;
end

Verify that the track has not been confirmed yet by checking the number of confirmed tracks.

numConfirmed = tracker.NumConfirmedTracks

numConfirmed = 
0

Because the track is not confirmed, get the position and velocity from the tentativeTracks structure.

positionSelector = [1 0 0 0 0 0; 0 0 0 1 0 0];
velocitySelector = [0 1 0 0 0 0; 0 0 0 0 1 0];
position = getTrackPositions(tentativeTracks,positionSelector)

position = 1×2

   10.6669   -0.6665

velocity = getTrackVelocities(tentativeTracks,velocitySelector)

velocity = 1×2

    3.3473    1.6737

Add more detections to confirm the track.

for detno = 3:5
    time = (detno-1)*dt;
    det = { ...
        objectDetection(time,pos, ...
        'SensorIndex',1, ...
        'ObjectAttributes',{'ExampleObject',1}) ...
        };
    [confirmedTracks,tentativeTracks,allTracks] = tracker(det,time);
    pos = pos + vel*dt;
    meas = pos;
end

Verify that the track has been confirmed, and display the position and velocity vectors for that track.

numConfirmed = tracker.NumConfirmedTracks

numConfirmed = 
1

position = getTrackPositions(confirmedTracks,positionSelector)

position = 1×2

   13.8417    0.9208

velocity = getTrackVelocities(confirmedTracks,velocitySelector)

velocity = 1×2

    9.4670    4.7335

Let the tracker run but do not add new detections. The existing track is deleted.

for detno = 6:20
    time = (detno-1)*dt;
    det = {};
    [confirmedTracks,tentativeTracks,allTracks] = tracker(det,time);
    pos = pos + vel*dt;
    meas = pos;
end

Verify that the tracker has no tentative or confirmed tracks.

isempty(allTracks)

ans = logical
   1

Algorithms

When you pass detections into a multi-object tracker, the System object:

• Attempts to assign the input detections to existing tracks, based on the AssignmentThreshold property of the multi-object tracker.

• Creates new tracks from unassigned detections.

• Updates already assigned tracks and possibly confirms them, based on the ConfirmationThreshold property of the tracker.

• Deletes tracks that have no assigned detections, based on the DeletionThreshold property of the tracker.

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

• See System Objects in MATLAB Code Generation (MATLAB Coder).

• All the detections used with the tracker must have fields with the same sizes and types.

• The objectDetection structure must have an ObjectAttributes field. The value of this field can be an empty structure, a structure, or a cell containing a structure. The structure for all detections must have the same fields and the values in these fields must always have the same size and type. The form of the structure cannot change during simulation.

• The first update to the tracker must contain at least one detection.

• When the filter initialization function returns a trackingEKF or trackingUKF object and when the MaxNuMDetections property is specified as a finite integer, the tracker supports non-dynamic memory allocation code generation.

Version History

Introduced in R2017a

R2020a: Track output format changed

Starting from R2020a, the track output format of multiObjectTracker changes from track structure to objectTrack. As a result, when you load a multiObjectTracker created in an earlier version of MATLAB, you need to release the tracker first so that it can allow objectTrack as the track output format.