Problems training deep neural network
1 view (last 30 days)
Show older comments
I have a custom neural network with 3 hidden layers and I'm having trouble initializing it. The problem is that when training stops I get a very small gradient so no much learning in the last epochs but my error is still huge even for the training set! I've investigated the problem and it come out that my last layer of sigmas is saturated and there is no remarquable change for the weights of the early layers. I've red that this problem of vanishing gradient can be solved using the Relu transfer functions but now I have a huge error due to enormous weights. I'm stuck for 2 weeks tryings Relu and its variants (Leaky RElu and Elu ) with different training functions (trainrp and trainscg) with no success. I doubt that I need a good initialization but I can't find anything in Matlab other the NW and I thinks that's strange for Matlab. Can anyone help me or at least put me on the right truck ?! Thanks
Regards
Amine
1 Comment
Greg Heath
on 5 Jan 2020
How are your inputs and targets normalized? i,e, what are the boundaries of each component?
Greg
Answers (1)
See Also
Categories
Find more on Deep Learning Toolbox in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!