Equal outputs using customized regression output layer
2 views (last 30 days)
Show older comments
Hello everyone. I defined a customized regression output layer using the provided template and used the MSE for the Loss function:
classdef mseRegressionLayer < nnet.layer.RegressionLayer
methods
function layer = mseRegressionLayer(name)
% Set layer name.
layer.Name = name;
% Set layer description.
layer.Description = 'Mean square error';
end
function loss = forwardLoss(layer, Y, T)
% the predictions Y and the training targets T.
loss= mse(Y,T,'DataFormat','T')/size(Y,2);
end
end
end
Then I used this script to a trial dataset of arrays, the same one that has been trained in the example of training shallow neural networks here https://it.mathworks.com/help/deeplearning/ug/train-and-apply-multilayer-neural-networks.html , but now I wanted to build myself the Loss function, so I tried with this:
layer = mseRegressionLayer('mse');
load bodyfat_dataset % dataset with bodyfatInputs and bodyfatTargets
layers = [
sequenceInputLayer(13)
lstmLayer(100,'OutputMode',"last") % Output the last time step of the sequence
fullyConnectedLayer(1)
mseRegressionLayer('mse')];
%options = trainingOptions('sgdm');
options = trainingOptions('adam', ...
'MaxEpochs',100, ...
'GradientThreshold',1, ...
'InitialLearnRate',0.001, ...
'LearnRateSchedule','piecewise', ...
'LearnRateDropPeriod',100, ...
'LearnRateDropFactor',0.1,...
'Verbose',1, ...
'Plots','training-progress');
N = 100; %number of sequences
cellArrTrain = cell(N,1);
for i = 1:N
seq = bodyfatInputs(:,i);
seq = num2cell(seq,1);
cellArrTrain(i) = seq;
end
net = trainNetwork(cellArrTrain,bodyfatTargets(1:N)',layers,options);
YPred = predict(net,cellArrTrain);
predictionError = YPred - bodyfatTargets;
The problem is that the outputs in this way are all basically the same number... I don't think that is caused by the particular loss function because also using the mae error, on the standard template provided in the documentation, it does the same. The mini batch Loss acceptable, but outputs are too similiar. What can be the reason for it?
Thank you for your help!
0 Comments
Answers (1)
Shashank Gupta
on 14 Oct 2020
Hi Fabrizio,
On the first look, the code does look good and it should work, did you check increasing the number of epoch and decreasing the learning rate decay, it seems like, it should converge but learning rate decay is super high which is making loss to converge at higher value than expected and the model is not able to learn. Try removing the decay and check if the training is smooth. I can't find any flag anywhere in the code atleast at the first look.
I hope my suggestion helps you out, Update me if you find anything interesting.
Cheers.
0 Comments
See Also
Categories
Find more on Image Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!