Error using dlarray/dlgradient : Value to differentiate is non-scalar. It must be a traced real dlarray scalar.

Hello, I want to train a Seq_to_one regression problem using a mae loss function found at "https://www.mathworks.com/help/deeplearning/ug/define-custom-regression-output-layer.html"

Input are :

X_train1 is 9x27 double array

Y_train2 is 9x3 double array

data_0.xlsx and mae loss function m file are attached

error occurred :

Error using trainNetwork

Error using 'backwardLoss' in Layer maeRegressionLayer. The function threw an error and could not be executed.

Caused by:

Error using dlarray/dlgradient

Value to differentiate is non-scalar. It must be a traced real dlarray scalar.

numChannels = 1;
numResponses = 3;
numHiddenUnits2 = 3;
X_train1 = xlsread('data_0.xlsx',1,'A2:AA10');
X_train2  = num2cell(X_train1,2);
Y_train2 = xlsread('data_0.xlsx',2,'A2:C10');
layers = [ ...
    sequenceInputLayer(numChannels,Normalization="zscore") 
    gruLayer(numHiddenUnits2,OutputMode="last")
    fullyConnectedLayer(3)
    maeRegressionLayer('mae')];
opts = trainingOptions('adam',...     
                       'MaxEpochs',3000000,...
                        'GradientThreshold',0.1,...
                        'InitialLearnRate',0.01,...
                        'MiniBatchSize',27,...
                        'ResetInputNormalization',false, ...
                        'VerboseFrequency',50, ...
                        'Plots','training-progress');
[net1,info] = trainNetwork(X_train2, Y_train2, layers, opts);
save net1;

Answer 1

[Venu]

Hi @Hossemeddine Saidi

Try making these 2 modifications:

1. Use "mean(meanAbsoluteError, 'all')" to calculate the MAE. This simplifies the calculation and ensures that the loss is aggregated across all dimensions, resulting in a scalar value. This change is crucial because the training function expects a single scalar loss value to optimize, not an array of losses.

2. In custom layers, especially for regression tasks, both forwardLoss and backwardLoss methods need to be properly defined. Add "backwardLoss" method which calculates the gradient of the loss with respect to the predictions, which is essential for backpropagation during training.

classdef maeRegressionLayer < nnet.layer.RegressionLayer ...
        & nnet.layer.Acceleratable
    % Example custom regression layer with mean-absolute-error loss.

    methods
        function layer = maeRegressionLayer(name)
            % layer = maeRegressionLayer(name) creates a
            % mean-absolute-error regression layer and specifies the layer
            % name.

            % Set layer name.
            layer.Name = name;

            % Set layer description.
            layer.Description = 'Mean absolute error';
        end

        function loss = forwardLoss(~, Y, T)
            % Calculate MAE.
            meanAbsoluteError = abs(Y-T);

            % Take mean over all elements.
            loss = mean(meanAbsoluteError, 'all');
        end

        function dLdY = backwardLoss(~, Y, T)
            % dLdY = backwardLoss(layer, Y, T) returns the derivatives of the
            % MAE loss with respect to the predictions Y.

            % Calculate gradients
            R = size(Y,3);
            N = size(Y,4);
            dLdY = sign(Y - T) / (R * N);
        end
    end
end

Hope this helps!

[Hossemeddine Saidi]

Thanks, It was very helpful