Incremental Learning

Fit linear model for regression to streaming data and track its performance

Incremental learning, or online learning, involves processing incoming data from a data stream, possibly given little to no knowledge of the distribution of the predictor variables, aspects of the objective function, and whether the observations are labeled. Incremental learning is often used in on-device learning applications, where models are trained on low-power edge devices using locally generated streaming data. Incremental learning problems contrast with traditional machine learning methods, in which enough labeled data is available to fit to a model, perform cross-validation to tune hyperparameters, and infer the predictor distribution characteristics.

Incremental learning requires a configured incremental model. You can create and configure most incremental models directly by calling an object, for example incrementalRegressionLinear, or you can convert a supported, traditionally trained model to an incremental learner by using incrementalLearner. After configuring a model and setting up a data stream, you can fit the incremental model to the incoming chunks of data, track the predictive performance of the model, or perform both actions simultaneously.

For more details, see Incremental Learning Overview.

Statistics and Machine Learning Toolbox™ also provides functions that allow you to:

  • Normalize streaming data incrementally, by creating a normalizer model using incrementalNormalizer, and updating the model using the incremental fit function. Use fit or transform to return normalized data.

  • Compute the principal component coefficients and variances of streaming data incrementally, by creating a principal component analysis (PCA) model using incrementalPCA, and updating the model using the incremental fit function. Use fit or transform to return principal component scores.

  • Monitor for drift incrementally in concept data, such as regression loss, by creating a drift detector using incrementalConceptDriftDetector. After setting up a data stream, update the drift detector and check for any drift using detectdrift.

For more information, see the reference pages.

Blocks

IncrementalRegressionLinear PredictPredict responses using incremental linear regression model (Since R2023b)
IncrementalRegressionLinear FitFit incremental linear regression model (Since R2023b)
IncrementalRegressionKernel FitFit incremental kernel regression model (Since R2024b)
IncrementalRegressionKernel PredictPredict responses using incremental kernel regression model (Since R2024b)
Detect DriftUpdate drift detector states and drift status with new data (Since R2024b)
Update MetricsUpdate performance metrics in incremental learning model given new data (Since R2023b)

Functions

expand all

Create Incremental Normalizer Model

incrementalNormalizerInstantiate incremental data normalizer (Since R2026a)

Incrementally Fit and Normalize Data

fitFit incremental normalizer model to streaming data (Since R2026a)
transformNormalize streaming data (Since R2026a)

Reset Normalizer Model

resetReset incremental normalizer model (Since R2026a)

Create Incremental Drift-Aware Model

incrementalDriftAwareLearnerConstruct drift-aware model for incremental learning (Since R2022b)

Incrementally Fit and Track Performance

fitTrain drift-aware learner for incremental learning with new data (Since R2022b)
updateMetricsUpdate performance metrics in incremental drift-aware learning model given new data (Since R2022b)
updateMetricsAndFitUpdate performance metrics in incremental drift-aware learning model given new data and train model (Since R2022b)

Other Model Operations

lossRegression or classification error of incremental drift-aware learner (Since R2022b)
perObservationLossPer observation regression or classification error of incremental drift-aware learner (Since R2022b)
predictPredict responses for new observations from incremental drift-aware learning model (Since R2022b)
resetReset incremental drift-aware learner (Since R2022b)

Create Incremental Model

incrementalRegressionKernelKernel regression model for incremental learning (Since R2022a)
incrementalLearnerConvert kernel regression model to incremental learner (Since R2022a)

Incrementally Fit and Track Performance

fitTrain kernel model for incremental learning (Since R2022a)
updateMetricsUpdate performance metrics in kernel incremental learning model given new data (Since R2022a)
updateMetricsAndFitUpdate performance metrics in kernel incremental learning model given new data and train model (Since R2022a)

Other Model Operations

predictPredict responses for new observations from kernel incremental learning model (Since R2022a)
lossLoss of kernel incremental learning model on batch of data (Since R2022a)
perObservationLossPer observation regression error of model for incremental learning (Since R2022a)
resetReset incremental regression model (Since R2022a)

Create Incremental Model

incrementalRegressionLinearLinear regression model for incremental learning
incrementalLearnerConvert support vector machine (SVM) regression model to incremental learner
incrementalLearnerConvert linear regression model to incremental learner

Incrementally Fit and Track Performance

fitTrain linear model for incremental learning
updateMetricsUpdate performance metrics in linear incremental learning model given new data
updateMetricsAndFitUpdate performance metrics in linear incremental learning model given new data and train model

Other Model Operations

predictPredict responses for new observations from linear incremental learning model
lossLoss of linear incremental learning model on batch of data
perObservationLossPer observation regression error of model for incremental learning (Since R2022a)
resetReset incremental regression model (Since R2022a)

Create Concept Drift Detector

incrementalConceptDriftDetectorInstantiate incremental concept drift detector (Since R2022a)

Detect Drift and Reset Model

detectdriftUpdate drift detector states and drift status with new data (Since R2022a)
resetReset incremental concept drift detector (Since R2022a)

Objects

expand all

ZScoreNormalizerSimple-weighted normalizer (Since R2026a)
ExponentiallyWeightedNormalizerExponentially weighted normalizer (Since R2026a)
DriftDetectionMethodIncremental drift detector that utilizes Drift Detection Method (DDM) (Since R2022a)
HoeffdingDriftDetectionMethodIncremental concept drift detector that utilizes Hoeffding's Bounds Drift Detection Method (HDDM) (Since R2022a)

Topics