DLnetwork definition and training problem with outputLayer

I need to use dlnetwork (in order to export it to ONNX format).

I have defined its layers as follows:

layers = [

sequenceInputLayer(202,"Name","sequence")

fullyConnectedLayer(25,"Name","fc")

fullyConnectedLayer(2,"Name","fc_1")

softmaxLayer("Name","softmax")];

and then I defined the net:

nett = dlnetwork(layers);

so until here everything is fine.

When I am trying to train the network:

[net,tr] = trainNetwork(DATA,Alllabels, layers, options);

I get the following error:

Network: Missing output layer. The network must have at least one output layer.

Layer 'softmax': Unconnected output. Each layer output must be connected to the input of another

layer.

So I redefined my layers to include an ouput layer:

layers = [

sequenceInputLayer(202,"Name","sequence")

fullyConnectedLayer(25,"Name","fc")

fullyConnectedLayer(2,"Name","fc_1")

softmaxLayer("Name","softmax")

classificationLayer()];

but now the definition of the network as before (nett = dlnetwork(layers) ) returns the following error:

Layer 'classoutput': Detected output layer. The network must not have output layers.

So this is kind of a catch-22 problem - I need to add an output layer to train the network, but I must remove an output layer for defining it!!

Does someone have clue?

Thanks

Answer 1

[Abhinav Aravindan]

I understand that you are trying to train a deep learning network and export the trained network as an “ONNX” file but facing an issue in defining and training the network. While utilizing “dlnetwork”, the layers cannot contain an output layer.

The network input to “exportONNXNetwork” need not be a “dlnetwork” object. “layers” array can be utilized directly as an input to “trainNetwork” for training the neural network. The output of “trainNetwork” can then be exported as an “ONNX” file as given below. Kindly note that this is with respect to MATLAB R2023a.

% Define your network layers 
layers = [ 
    sequenceInputLayer(202,"Name","sequence") 
    fullyConnectedLayer(25,"Name","fc") 
    fullyConnectedLayer(2,"Name","fc_1") 
    softmaxLayer("Name","softmax") 
    classificationLayer()];  

% Replace with your data and required training options 
net = trainNetwork(data,layers,options);  

% Export trained network as ONNX file 
exportONNXNetwork(net, filename.onnx) 

Please find below the documentation links for your reference.

dlnetwork: https://www.mathworks.com/help/releases/R2023a/deeplearning/ref/dlnetwork.html?s_tid=doc_ta#mw_daaa19ab-3b04-4a1a-b5e8-38095a7cac38:~:text=The%20network%20layers%20must%20not%20contain%20output%20layers.

trainNetwork: https://www.mathworks.com/help/releases/R2023a/deeplearning/ref/trainnetwork.html

exportONNXNetwork: https://www.mathworks.com/help/releases/R2023a/deeplearning/ref/exportonnxnetwork.html?s_tid=doc_ta#mw_844c3879-ae5e-4704-8595-20250a86a2ef

I hope this helps resolve your issue!

Answer 2

[Lucas García]

You should use the trainnet function instead of trainNetwork to train dlnetwork objects. That said, you can convert the Series or DAG network trained with trainNetwork if you are using R2024a by using dag2dlnetwork .

Answer 3

[SIMON]

Hi yes I have the same problem however I am not wishing to train my Model with trainNetwork or trainnet as I am using a bespoke loss function in order to train an VAE. Now when I convert my layers to a dlnetwork I have no errors. However when i analyse the lgraph it shows that the softmax is not connected to an output - I presume this error doesnt affecct the dlnetwork which has no need for outputs. How are you to proceeed if you need a dlnetwork to use a custom loss function - I am having trouble with gradients:

function loss = computeTotalLoss(dlX, dlY, encoderNet, decoderNet, classifierNet, sequenceLength)
    % All operations inside this function are traced
    [mu, logvar] = encodeLatents(dlX, encoderNet);
    z = sampleLatents(mu, logvar);  % Sampling with reparameterization
    loss = computeLoss(dlX, dlY, z, mu, logvar, decoderNet, classifierNet, sequenceLength);
end

function [mu, logvar] = encodeLatents(dlX, encoderNet)
    mu = forward(encoderNet, dlX, 'Outputs', 'fc_mu');
    logvar = forward(encoderNet, dlX, 'Outputs', 'fc_logvar');
end

function z = sampleLatents(mu, logvar)
    eps = dlarray(randn(size(mu), 'like', mu));  % Traced random noise
    z = mu + exp(0.5 * logvar) .* eps;
end

function loss = computeLoss(dlX, dlY, z, mu, logvar, decoderNet, classifierNet, sequenceLength)
    zRepeated = repmat(z, 1, 1, sequenceLength);
    dlZSeq = dlarray(zRepeated, 'CBT');
    reconOut = forward(decoderNet, dlZSeq, 'Outputs', 'fc_recon');
    classOut = forward(classifierNet, dlarray(z, 'CB'), 'Outputs', 'softmax_out');

    reconLoss = mse(reconOut, dlX);
    classLoss = crossentropy(classOut, dlY);
    klLoss = -0.5 * sum(1 + logvar - mu.^2 - exp(logvar), 'all');

    loss = reconLoss + classLoss + klLoss;
end


for epoch = 1:numEpochs
    idx = randperm(numel(XTrain));
    totalEpochLoss = 0;
    numBatches = 0;

    for i = 1:miniBatchSize:numel(XTrain)
        batchIdx = idx(i:min(i+miniBatchSize-1, numel(XTrain)));
        XBatch = XTrain(batchIdx);
        YBatch = YTrain(batchIdx, :);

        % === Format Inputs ===
        XCat = cat(3, XBatch{:});
        XCat = permute(XCat, [1, 3, 2]);
        dlX = dlarray(XCat, 'CBT');
        dlY = dlarray(YBatch', 'CB');

        % === Forward Pass: Encoder ===
        mu = forward(encoderNet, dlX, 'Outputs', 'fc_mu');
        logvar = forward(encoderNet, dlX, 'Outputs', 'fc_logvar');

        % === Latent Sampling ===
        eps = randn(size(mu), 'like', mu);
        z = mu + exp(0.5 * logvar) .* eps;
        dlZ = dlarray(z, 'CB');

        T = sequenceLength;  % Number of time steps to reconstruct
        zRepeated = repmat(z, 1, 1, T);  % [latentDim × batchSize × T]
        dlZSeq = dlarray(zRepeated, 'CBT');

        % === Forward Pass: Decoder ===
        reconOut = forward(decoderNet, dlZSeq, 'Outputs', 'fc_recon');
        reconOut = dlarray(reconOut, 'CBT');  % Match input format
       
        % === Forward Pass: Classifier ===
        classOut = forward(classifierNet, dlZ, 'Outputs', 'softmax_out');

        % === Losses ===
        reconLoss = mse(reconOut, dlX);
        classificationLoss = crossentropy(classOut, dlY);
        klLoss = -0.5 * sum(1 + logvar - mu.^2 - exp(logvar), 'all');

        totalLoss = reconLoss + classificationLoss + klLoss;

        totalLoss = dlfeval(@computeTotalLoss, dlX, dlY, encoderNet, decoderNet, classifierNet, sequenceLength);
        gradients = dlgradient(totalLoss, encoderNet.Learnables);

        Error using dlarray/dlgradient (line 105)
Value to differentiate is not traced. It must be a traced real dlarray scalar. Use dlgradient inside a function called by dlfeval to trace the variables.

Hope you can help.