Can someone explain to me the meaning of these numbers is this line of code?

In relation to deep learning setting the weight and bias learn rate factors both to 20 means learn faster in the new layers not the updated layers. Does the 20 represent faster learning from layer 20 onward or something else?

Code:

fullyConnectedLayer(numClasses,'Name','fc','WeightLearnRateFactor',20,'BiasLearnRateFactor',20)

Answer 1

[Prajit T R]

Hi Nathan

I think the documentation of the function 'fullyConnectedLayer' explains this accurately.

'WeightLearnRateFactor' is the factor by which the learning rate of the weights in the layer is greater than that of the global learning rate.

'BiasLearnRateFactor' is the factor by which the learning rate of the biases in the layer is greater than that of the global learning rate.

Let us take your code as an example to explain this: The 'WeightLearnRateFactor' in this layer you are creating is 20 times that of the global learning rate. That would mean that this layer hugely influences the model as the learning rate is much higher. Similar is the case for the 'BiasLearnRateFactor'.

As you suggested, this does represent faster learning, but has nothing to do with layer 20. The faster learning corresponds to the fully connected layer that you currently generated.

Hope this helps

Cheers