What is Validation data in deep learning?
110 views (last 30 days)
Hi, I am new to deep learning and I am struggling to grasp the concept of validation and its interpretation in training progress plots.
As far I understand after some Googling, it is a check on how the model is behaving during training before deploying on test data.
My question is what is validation data and what it validation accuracy and how to interpret. I would appreciate if some help me to understand in simple words.
Philip Brown on 21 Jun 2021
The "validation data" is a set of data held separate from your training data. It's used during the training process to see how the network would perform on data it hasn't been directly trained on.
The "training data" is what's used in the process of updating the layer weights via backpropagation. Training data is fed through the network every iteration, the loss is computed, and the layer weights are updated via backpropagation to reduce the loss for that iteration. If the training is going well, every iteration updates the weights of the network so it becomes better and better at predicting on the training data.
However, there's a danger the network becomes too good at predicting on the training data. It can learn very specific features of the training set, rather than generally useful features which would be helpful for predicting on new data. This is called "overfitting". To check for that, we can use "validation data". The validation data is not used directly to train the network. It's instead used to see how the network is performing. In your training plot, these validation checks happen every 50th iteration. Results from the validation data are not used to update the network weights.
As you're seeing in the training plot, the validation accuracy is a little bit below the training accuracy - the network is better at predicting on the data it's been directly trained on. This is quite common. If you saw the validation accuracy start to drop substantially as you train further, that would be more evidence for overfitting: your network would be learning specific features of the training set, rather than general features also useful for the validation set.
More Answers (1)
Lei Liu on 29 May 2021
I'm also a newcomer to neural network learning,perhaps some of this page will help you: