Main Content

nssTrainingSGDM

SGDM training options object for neural state-space systems

Since R2022b

    Description

    SGDM options set object to train an idNeuralStateSpace network using nlssest.

    Creation

    Create a nssTrainingSGDM object using nssTrainingOptions and specifying "sgdm" as input argument.

    Properties

    expand all

    Solver used to update network parameters, returned as a string. This property is read-only. Use nssTrainingOptions("sgdm") to return an options set object for the SGDM solver instead.

    Example: SGDM

    Contribution of the parameter update step of the previous iteration to the current iteration of stochastic gradient descent with momentum, specified as a scalar from 0 to 1.

    A value of 0 means no contribution from the previous step, whereas a value of 1 means maximal contribution from the previous step. The default value works well for most tasks.

    For more information, see TrainingOptionsSGDM (Deep Learning Toolbox).

    Example: 0.9

    Type of function used to calculate loss, specified as one of the following:

    • "MeanAbsoluteError" — use the mean value of the absolute error.

    • "MeanSquaredError" — using the mean value of the squared error.

    Example: MeanSquaredError

    Option to plot the value of the loss function during training, specified as one of the following:

    • true — plot the value of the loss function during training.

    • false — do not plot the value of the loss function during training

    Example: false

    Learning rate used for training, specified as a positive scalar. If the learning rate is too small, then training can take a long time. If the learning rate is too large, then training might reach a suboptimal result or diverge.

    For more information, see TrainingOptionsSGDM (Deep Learning Toolbox).

    Example: 0.01

    Maximum number of epochs to use for training, specified as a positive integer. An epoch is the full pass of the training algorithm over the entire training set.

    For more information, see TrainingOptionsSGDM (Deep Learning Toolbox).

    Example: 400

    Size of the mini-batch to use for each training iteration, specified as a positive integer. A mini-batch is a subset of the training set that is used to evaluate the gradient of the loss function and update the weights.

    If the mini-batch size does not evenly divide the number of training samples, then nlssest discards the training data that does not fit into the final complete mini-batch of each epoch.

    For more information, see TrainingOptionsSGDM (Deep Learning Toolbox).

    Example: 200

    ODE solver options to integrate continuous-time neural state-space systems, specified as an nssDLODE45 object

    Use dot notation to access properties such as the following:

    • Solver — Solver type, set as "dlode45". This is a read-only property.

    • InitialStepSize — Initial step size, specified as a positive scalar. If you do not specify an initial step size, then the solver bases the initial step size on the slope of the solution at the initial time point.

    • MaxStepSize — Maximum step size, specified as a positive scalar. It is an upper bound on the size of any step taken by the solver. The default is one tenth of the difference between final and initial time.

    • AbsoluteTolerance — Absolute tolerance, specified as a positive scalar. It is the largest allowable absolute error. Intuitively, when the solution approaches 0, AbsoluteTolerance is the threshold below which you do not worry about the accuracy of the solution since it is effectively 0.

    • RelativeTolerance — Relative tolerance, specified as a positive scalar. This tolerance measures the error relative to the magnitude of each solution component. Intuitively, it controls the number of significant digits in a solution, (except when it is smaller than the absolute tolerance).

    For more information, see odeset.

    Example: 200

    Input interpolation method, specified as one of the following strings:

    • "zoh" — Use zero-order hold interpolation method.

    • "foh" — Use first-order hold interpolation method.

    • "cubic" — Use cubic interpolation method.

    • "makima" — Use modified Akima interpolation method.

    • "pchip" — Use shape-preserving piecewise cubic interpolation method.

    • "spline" — Use spline interpolation method.

    This is the interpolation method used to interpolate the input when integrating continuous-time neural state-space systems. For more information, see interpolation methods in interp1.

    Example: "foh"

    Examples

    collapse all

    Use nssTrainingOptions to return an options set object to train an idNeuralStateSpace system.

    adamOpts = nssTrainingOptions("sgdm")
    adamOpts = 
      nssTrainingSGDM with properties:
    
            UpdateMethod: "SGDM"
                Momentum: 0.9500
                 LossFcn: "MeanAbsoluteError"
             PlotLossFcn: 1
               LearnRate: 1.0000e-03
               MaxEpochs: 100
           MiniBatchSize: 100
        ODESolverOptions: [1x1 idoptions.nssDLODE45]
        InputInterSample: 'spline'
    
    

    Use dot notation to access the object properties.

    adamOpts.LearnRate = 0.01;

    You can now use sgdmOpts as the value of a name-value pair input argument to nlssest to specify the training options for the state (StateOptions=sgdmOpts) or the output (OutputOptions=sgdmOpts) network of an idNeuralStateSpace object.

    Version History

    Introduced in R2022b