Neural Network options Output net work, 'best-validation-loss'

I use simple feed forward neural network only 2 layers with miniBatchSize, and inside the neural network options, I use adam for solver option, and 'OutputNetwork' is 'best-validation-loss'. But the error message says outputNetwork is not an option for solver Adam. Then I tried sgdm, and rsprop, the rest of 2 options Matlab offered for solver. The error message still there and says outputnetwork is not an option fo solver sgdm, and rsprop respectively. I wonder how to set outputnetwork to be best-validation-loss since it fails under all solver options ?

[KSSV]

Show us your whole code which you have tried.

[Miao]

%TTr is training table, reponse variable is last column named exRet.

numFeatures = size(TTr,2)-1; % all numeric data, no categorical.

layers = [

featureInputLayer(numFeatures)

fullyConnectedLayer(32)

batchNormalizationLayer

reluLayer

fullyConnectedLayer(16)

batchNormalizationLayer

reluLayer

fullyConnectedLayer(1)

regressionLayer

];

miniBatchSize = 100;

num = height(TTr)/miniBatchSize; % total training observation/batch size = number of batch

options = trainingOptions('adam',...

'MiniBatchSize',miniBatchSize,...

'InitialLearnRate',0.001,...

'ValidationData',TVa,...

'ValidationPatience',5,...

'ValidationFrequency',round(num),...

'OutputNetwork','best-validation-loss',...

'Shuffle','every-epoch',...

'MaxEpochs',500,...

'L2Regularization',0.001,...

'Plots','training-progress', ...

'Verbose',false);

net = trainNetwork(TTr,'exRet',layers,options);

When I run the code above, error message is below, the version my matlab is 2021a, should I update to 2021b then maybe it work ok?

Error using nnet.cnn.TrainingOptionsADAM (line 132)

'OutputNetwork' is not an option for solver 'adam'.

Error in trainingOptions (line 317)

opts = nnet.cnn.TrainingOptionsADAM(varargin{:});

Error in tryneural (line 25)

options = trainingOptions('adam',...

[Taylor Chu]

I'm getting the same error but only when I'm running it on cluster which I assume is running linux. I've tried running the same script (my script) on windows and it runs fine with the 'OutputNetwork', 'best-validation-loss' training option. Removing that option works on cluster but it does not really solve the problem

Answer 1

[Niccolò Dal Santo]

Hi Miao,

The 'OutputNetwork', 'best-validation-loss' training option is available in R2021b or later versions. Which MATLAB version are you using? You can obtain the version with the following command:

>> version 
ans =
    '9.11.0.1751886 (R2021b)'

Cheers,

Niccolò