why is the neural network performance is not improving
2 views (last 30 days)
Show older comments
I am using a feedforward neural network with 7 input vectors which are each matrices of 4 values and a target dataset of 4,600 values for one single output matrix. Keeping aside the output or simulation, my question is why is only my training performance reaching minimal value and not validation and testing performance, these two are remaining almost at a constant level.![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/1142680/image.png)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/1142680/image.png)
2 Comments
Walter Roberson
on 3 Oct 2022
If you were to over-train then the mse for the training data could drop a lot, but the test and validation might still have large errors.
I would wonder about whether the training data is full rank.
Answers (1)
Animesh Gupta
on 12 Oct 2022
Hello,
It is my understanding that validation and testing mean square error (MSE) are significantly higher compared to training MSE in this case.
When there is a significant difference between training and testing error, like this one, it is a case of over-fitting. There are many reasons for over-fitting. Some of them are:
- Not enough training data
- Complex model
- The training data is biased
- Continue training overshooting the generalization point (6th epoch in this case)
There are several methods to avoid over-fitting. One of them is early stopping. In this technique, when the validation error increases for a specified number of iterations, the training is stopped, and the weights and biases at the minimum of the validation error are returned.
Please refer the following documentation for more techniques to improve generalisation (training) and avoid overfitting - https://www.mathworks.com/help/deeplearning/ug/improve-neural-network-generalization-and-avoid-overfitting.html
I hope it helps.
0 Comments
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!