How to change learning rate and momentum in 'trainlm' neural network?

124 views (last 30 days)
I have a feed forward neural network trained with trainlm function.Now i want to change the learning rate and momentum.Is there a specific default range for learning rate or Do I need to find the optimum range for learning rate?

Accepted Answer

Wolfgang Gross
Wolfgang Gross on 2 Nov 2018
Edited: Wolfgang Gross on 2 Nov 2018
use
options = trainingOptions('sgdm',...
'MaxEpochs',60, ...
'LearnRateDropFactor',0.1,...
'LearnRateDropPeriod',1,...
'LearnRateSchedule','piecewise',...
'ValidationData',{valImages,valLabels},...
'ValidationFrequency',50,...
'Momentum',0.85,...
'ValidationPatience',5,...
'InitialLearnRate',0.1,...
'MiniBatchSize',128,...
'Verbose',true,...
'executionenvironment','cpu',...
'Shuffle','every-epoch',...
'Plots','training-progress');
net = trainNetwork(...,...,...,options);
and choose the parameters accordingly.
For default values, see the docs of trainingOptions.
In terms of the learning rate and momentum, I typically start with a large one just to test the general behaviour and then I drop the learning rate to get more accurate results. Its difficult to give you some concrete advice because the optimal learning rate depends on your specific problem.

More Answers (0)

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!