Neural network validation checks net.TrainParam.max_fail <- is a bigger or a smaller number better?

59 views (last 30 days)
While trying to improve my neural network I wondered, whether I should increase or decrease
TrainParam.max_fail
(default value is 6)
Training stops when any of these conditions occurs:
  • "Validation performance has increased more than max_fail times since the last time it decreased (when using validation)."
which I interpret as: if validation error decreases more than 6 times -> early stopping
When the validation error increases for a specified number of iterations (net.trainParam.max_fail), the training is stopped, and the
weights and biases at the minimum of the validation error are returned.
which I interpret as: if validation error increases more than 6 times -> early stopping
So what is the purpose of the net.TrainParam.max_fail?
____________________________________________________________________________________
Second question in the same post:
When my Trainratio/Validationratio/Testratio is 70/25/5.
After how many Train-epochs is there an Validation-Epoch?
Thank you very much in advance!

Accepted Answer

Anshika Chaurasia
Anshika Chaurasia on 8 Oct 2021
Hi,
1. Training stops when any of these conditions occurs:
  • "Validation performance has increased more than max_fail times since the last time it decreased (when using validation)."
In above lines, "Validation Performance" means validation error. Hence, the interpretation of above line will be:
if validation error increases more than 6 times -> early stopping
To understand the terminology refer to following documents:
2. After each training epoch validation will occur. Or, in an epoch first training will be done then validation.
  1 Comment
NightStalker
NightStalker on 21 Oct 2021
Edited: NightStalker on 21 Oct 2021
Hi Anshika Chaurasia,
this was the very solution to my first question. Thank you!
Your answer to my second question lead to more questions, which I really hope you can answer me:
2.a) What will happen, if the testset has 1000 samples but the validationset only 500? will the validation-procedures be evenly distributed over the test-procedures or will some validation samples be used more than 1 time and force the validation-procedure after every test-procedure?
2.b) After one test-procedure, how does MATLAB pick/choose the sample within the validationset to validate this test-procedure?
Thanks in advance!

Sign in to comment.

More Answers (3)

pathakunta
pathakunta on 26 Jan 2024
1. Training stops when any of these conditions occurs: "Validation performance has increased more than max_fail times since the last time it decreased (when using validation)." In above lines, "Validation Performance" means validation error. Hence, the interpretation of above line will be: if validation error increases more than 6 times -> early stopping To understand the terminology refer to following documents: Calculate network performance - MATLAB perform (mathworks.com) https://www.mathworks.com/help/deeplearning/ug/neural-network-object-properties.html#bss4hk6-52 2. After each training epoch validation will occur. Or, in an epoch first training will be done then validation.

pathakunta
pathakunta on 26 Jan 2024
1. Training stops when any of these conditions occurs: "Validation performance has increased more than max_fail times since the last time it decreased (when using validation)." In above lines, "Validation Performance" means validation error. Hence, the interpretation of above line will be: if validation error increases more than 6 times -> early stopping To understand the terminology refer to following documents: Calculate network performance - MATLAB perform (mathworks.com) https://www.mathworks.com/help/deeplearning/ug/neural-network-object-properties.html#bss4hk6-52 2. After each training epoch validation will occur. Or, in an epoch first training will be done then validation.

pathakunta
pathakunta on 26 Jan 2024
1. Training stops when any of these conditions occurs: "Validation performance has increased more than max_fail times since the last time it decreased (when using validation)." In above lines, "Validation Performance" means validation error. Hence, the interpretation of above line will be: if validation error increases more than 6 times -> early stopping To understand the terminology refer to following documents: Calculate network performance - MATLAB perform (mathworks.com) https://www.mathworks.com/help/deeplearning/ug/neural-network-object-properties.html#bss4hk6-52 2. After each training epoch validation will occur. Or, in an epoch first training will be done then validation.

Categories

Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!