Customized Regression output layer

7 views (last 30 days)
Fabrizio Bernardi
Fabrizio Bernardi on 7 Sep 2020
Hello everyone, following the example of https://it.mathworks.com/help/deeplearning/ug/define-custom-regression-output-layer.html I tried to build a regression output layer using the mse error. Using the provided script, I did this for the loss function:
function loss = forwardLoss(layer, Y, T)
loss = mse(Y,T);
end
But trying with a data set in matlab net = trainNetwork(bodyfatInputs,bodyfatTargets,layers,options);
it gave me
Error using trainNetwork (line 170)
Error using 'forwardLoss' in Layer mseRegressionLayer. The function threw an error and could not be executed.
I buit layers
layers = [
sequenceInputLayer(13)
lstmLayer(100)
fullyConnectedLayer(1)
mseRegressionLayer('mse')];
What did I do wrong?
Thanks for your help
  7 Comments
Mohammad Sami
Mohammad Sami on 7 Sep 2020
It seems the layer should be valid. Maybe something else is wrong. Try using the built-in regression layer ( which also uses mse) to verify that there is nothing else wrong.

Sign in to comment.

Answers (1)

Uday Pradhan
Uday Pradhan on 10 Sep 2020
Hi,
I tried to implement your network on my end and found two problems. One, when using "mse" as the loss function, it is advisable to mention the 'DataFormat' argument as well, for example, see this page. So, modify the line (in your definition of 'mseRegressionLayer.m')
loss = mse(Y,T);
%change to
loss = mse(Y,T,'DataFormat','T'); %for sequences
Coming to the problem you are trying to solve:
The "bodyfat_dataset" consists of two important vectors X and T where X is of size 13 - by - 252 and targets T is 1 - by - 252. From my understanding, you would like to create a LSTM network which accepts a sequence of 13 features and predicts the body fat percentage. This is a sequence to one regression problem and as advised here, I redesigned your network as such:
layers = [
sequenceInputLayer(13)
lstmLayer(100,'OutputMode',"last") % Output the last time step of the sequence
fullyConnectedLayer(1)
mseRegressionLayer('mse')];
However, in this output mode the input must be in cell array format. To do this you may use the following:
N = 240; %number of sequences
cellArrTrain = cell(N,1);
for i = 1:N
seq = xtrain(:,i);
seq = num2cell(seq,1);
cellArrTrain(i) = seq;
end
% ------ FOR TRAINING PURPOSES -------%
net = trainNetwork(cellArrTrain,Ytrain,layers,options); %be cautious of the dimensions of
% cellArrTrain and Ytrain, they should match.
% Similarly convert the test data into a cell array too
Hope this helps!
  5 Comments
Fabrizio Bernardi
Fabrizio Bernardi on 4 Oct 2020
Now the loss seems more fair:
Training on single CPU.
|========================================================================================|
| Epoch | Iteration | Time Elapsed | Mini-batch | Mini-batch | Base Learning |
| | | (hh:mm:ss) | RMSE | Loss | Rate |
|========================================================================================|
| 1 | 1 | 00:00:01 | 20.06 | 201.3 | 0.0010 |
| 50 | 50 | 00:00:02 | 15.77 | 124.4 | 0.0010 |
| 100 | 100 | 00:00:04 | 13.13 | 86.2 | 0.0010 |
| 150 | 150 | 00:00:05 | 12.89 | 83.1 | 0.0001 |
| 200 | 200 | 00:00:06 | 12.55 | 78.7 | 0.0001 |
| 250 | 250 | 00:00:07 | 12.51 | 78.2 | 1.0000e-05 |
| 300 | 300 | 00:00:09 | 12.48 | 77.9 | 1.0000e-05 |
| 350 | 350 | 00:00:11 | 12.48 | 77.9 | 1.0000e-06 |
| 400 | 400 | 00:00:12 | 12.48 | 77.9 | 1.0000e-06 |
| 450 | 450 | 00:00:13 | 12.48 | 77.8 | 1.0000e-07 |
| 500 | 500 | 00:00:14 | 12.48 | 77.8 | 1.0000e-07 |
|========================================================================================|
However , using
YPred = predict(net,bodyfatInputs);
The predictions are basically all the same... In general I saw that changing some parts of the networks, Ypred changes but again all the values are basically the same, definitely not in accord with the Targets... What could be the cause of it?
Again, thank you for all the time you are giving me :)

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!