How to change learning rate and momentum in 'trainlm' neural network?

I have a feed forward neural network trained with trainlm function.Now i want to change the learning rate and momentum.Is there a specific default range for learning rate or Do I need to find the optimum range for learning rate?

Answer 1

[Wolfgang Gross]

use

options = trainingOptions('sgdm',...
    'MaxEpochs',60, ...
    'LearnRateDropFactor',0.1,...
    'LearnRateDropPeriod',1,...
    'LearnRateSchedule','piecewise',...
    'ValidationData',{valImages,valLabels},...
    'ValidationFrequency',50,...
    'Momentum',0.85,...
    'ValidationPatience',5,...
    'InitialLearnRate',0.1,...
    'MiniBatchSize',128,...
    'Verbose',true,...
    'executionenvironment','cpu',...
    'Shuffle','every-epoch',...
    'Plots','training-progress');

net = trainNetwork(...,...,...,options);

and choose the parameters accordingly.

For default values, see the docs of trainingOptions.

In terms of the learning rate and momentum, I typically start with a large one just to test the general behaviour and then I drop the learning rate to get more accurate results. Its difficult to give you some concrete advice because the optimal learning rate depends on your specific problem.

[Avanthi Saumyamala]

Thank you Wolfgang.It was a great help.

[Pendar Arzandeh]

Could you explain about trainlm? How to change learning rate? matlab allow changing mu and Gradient

Thanks