Cross validation settings for Classification Discriminant models
1 view (last 30 days)
Show older comments
I foumd the 'KFold' in classification models can not be controlled when 'HyperparameterOptimizationOptions' is enabled. Here is a snippet of my codes:
discrMdl = fitcdiscr(X,y,'discrimType','linear',...
'OptimizeHyperparameters', VariableDescriptions,'HyperparameterOptimizationOptions',...
struct('ShowPlots',0,'Verbose',0,'Kfold',5));
After training, typing
discrMdl.crossval.KFold
gives a number of 10, implying 10-fold crosss validations had been perforemd, although 5-fold was defined in the function. I also tried other numbers, but it is alwasy 10-fold. It does not make sense according to the documentation athttps://www.mathworks.com/help/stats/fitcdiscr.html#bt6d86x-1_sep_shared-HyperparameterOptimizationOptions, unless the hyparameter optimization is only done in a 10-fold cross-validation fashion?
If the 'KFold' is put outside the strucured options, an error message will occur:
When optimizing parameters, validation arguments may only appear in the
'HyperparameterOptimizationOptions' argument.
Also, the resulting model is a ClassificationDiscriminant model rather than ClassificationPartitionedModel. Should the model be the latter given the cross validation was enabled in the hyperparameter optimization, according to the documentation at https://www.mathworks.com/help/stats/fitcdiscr.html#bt6d86x-1-Mdl.
Did I interpret something wrong?
0 Comments
Answers (1)
Aditya Patil
on 14 Jul 2020
When you use the crossval function, it creates a partioned model with KFold set to default value of 10. Currently, it's not possible to read the value of Kfold for ClassificationDiscriminant class. I have brought this issue to the concerned people and it might be considered in any future release.
0 Comments
See Also
Categories
Find more on Discriminant Analysis in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!