kfoldfun

Cross-validate function for regression

Description

example

vals = kfoldfun(CVMdl,fun) cross-validates the function fun by applying fun to the data stored in the cross-validated model CVMdl. You must pass fun as a function handle.

Examples

collapse all

Train a regression tree model, and then cross-validate it using a custom k-fold loss function.

Load the imports-85 data set. Train a regression tree using a subset of the data.

Mdl = fitrtree(X(:,[4 5]),X(:,16),...
'PredictorNames',{'Length','Width'},...
'ResponseName','Price');

Cross-validate the regression tree, and obtain the mean squared error.

CVMdl = crossval(Mdl);
L = kfoldLoss(CVMdl)
L = 1.9167e+07

Examine the error when you use a simple averaging of training responses instead of predictions in the calculation.

f = @(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)...
mean((Ytest-mean(Ytrain)).^2)
f = function_handle with value:
@(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)mean((Ytest-mean(Ytrain)).^2)

mean(kfoldfun(CVMdl,f))
ans = 6.3586e+07

Input Arguments

collapse all

Cross-validated model, specified as a RegressionPartitionedModel object, RegressionPartitionedEnsemble object, RegressionPartitionedGAM object, or RegressionPartitionedSVM object.

Cross-validated function, specified as a function handle. fun has the syntax:

testvals = fun(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)
• CMP is a compact model stored in one element of the CVMdl.Trained property.

• Xtrain is the training matrix of predictor values.

• Ytrain is the training array of response values.

• Wtrain are the training weights for observations.

• Xtest and Ytest are the test data, with associated weights Wtest.

• The returned value testvals must have the same size across all folds.

Data Types: function_handle

Output Arguments

collapse all

Cross-validation results, returned as a numeric matrix. vals contains the arrays of testvals output, concatenated vertically over all folds. For example, if testvals from every fold is a numeric vector of length N, kfoldfun returns a KFold-by-N numeric matrix with one row per fold.

Data Types: double

Extended Capabilities

Introduced in R2011a