Regularize an ensemble of bagged trees.

Generate sample data.

You can create a bagged classification ensemble of 300 trees from the sample data.

bag = fitrensemble(X,Y,'Method','Bag','NumLearningCycles',300);

`fitrensemble`

uses a default template tree object `templateTree()`

as a weak learner when `'Method'`

is `'Bag'`

. In this example, for reproducibility, specify `'Reproducible',true`

when you create a tree template object, and then use the object as a weak learner.

Regularize the ensemble of bagged regression trees.

Starting lasso regularization for Lambda=0.001. Initial MSE=0.109923.
Lasso regularization completed pass 1 for Lambda=0.001
MSE = 0.086912
Relative change in MSE = 0.264768
Number of learners with nonzero weights = 15
Lasso regularization completed pass 2 for Lambda=0.001
MSE = 0.0670602
Relative change in MSE = 0.296029
Number of learners with nonzero weights = 34
Lasso regularization completed pass 3 for Lambda=0.001
MSE = 0.0623931
Relative change in MSE = 0.0748019
Number of learners with nonzero weights = 51
Lasso regularization completed pass 4 for Lambda=0.001
MSE = 0.0605444
Relative change in MSE = 0.0305348
Number of learners with nonzero weights = 70
Lasso regularization completed pass 5 for Lambda=0.001
MSE = 0.0599666
Relative change in MSE = 0.00963517
Number of learners with nonzero weights = 94
Lasso regularization completed pass 6 for Lambda=0.001
MSE = 0.0598835
Relative change in MSE = 0.00138719
Number of learners with nonzero weights = 105
Lasso regularization completed pass 7 for Lambda=0.001
MSE = 0.0598608
Relative change in MSE = 0.000379227
Number of learners with nonzero weights = 113
Lasso regularization completed pass 8 for Lambda=0.001
MSE = 0.0598586
Relative change in MSE = 3.72856e-05
Number of learners with nonzero weights = 115
Lasso regularization completed pass 9 for Lambda=0.001
MSE = 0.0598587
Relative change in MSE = 6.42954e-07
Number of learners with nonzero weights = 115
Lasso regularization completed pass 10 for Lambda=0.001
MSE = 0.0598587
Relative change in MSE = 4.53658e-08
Number of learners with nonzero weights = 115
Completed lasso minimization for Lambda=0.001.
Resubstitution MSE changed from 0.109923 to 0.0598587.
Number of learners reduced from 300 to 115.
Starting lasso regularization for Lambda=0.1. Initial MSE=0.109923.
Lasso regularization completed pass 1 for Lambda=0.1
MSE = 0.104917
Relative change in MSE = 0.0477191
Number of learners with nonzero weights = 12
Lasso regularization completed pass 2 for Lambda=0.1
MSE = 0.0851031
Relative change in MSE = 0.232821
Number of learners with nonzero weights = 30
Lasso regularization completed pass 3 for Lambda=0.1
MSE = 0.081245
Relative change in MSE = 0.0474877
Number of learners with nonzero weights = 40
Lasso regularization completed pass 4 for Lambda=0.1
MSE = 0.0796749
Relative change in MSE = 0.0197067
Number of learners with nonzero weights = 53
Lasso regularization completed pass 5 for Lambda=0.1
MSE = 0.0788411
Relative change in MSE = 0.0105746
Number of learners with nonzero weights = 64
Lasso regularization completed pass 6 for Lambda=0.1
MSE = 0.0784959
Relative change in MSE = 0.00439793
Number of learners with nonzero weights = 81
Lasso regularization completed pass 7 for Lambda=0.1
MSE = 0.0784429
Relative change in MSE = 0.000676468
Number of learners with nonzero weights = 88
Lasso regularization completed pass 8 for Lambda=0.1
MSE = 0.078447
Relative change in MSE = 5.24449e-05
Number of learners with nonzero weights = 88
Completed lasso minimization for Lambda=0.1.
Resubstitution MSE changed from 0.109923 to 0.078447.
Number of learners reduced from 300 to 88.

`regularize`

reports on its progress.

Inspect the resulting regularization structure.

ans = *struct with fields:*
Method: 'Lasso'
TrainedWeights: [300x2 double]
Lambda: [1.0000e-03 0.1000]
ResubstitutionMSE: [0.0599 0.0784]
CombineWeights: @classreg.learning.combiner.WeightedSum

Check how many learners in the regularized ensemble have positive weights. These are the learners included in a shrunken ensemble.

Shrink the ensemble using the weights from `Lambda = 0.1`

.

cmp =
CompactRegressionEnsemble
ResponseName: 'Y'
CategoricalPredictors: []
ResponseTransform: 'none'
NumTrained: 88
Properties, Methods

The compact ensemble contains `87`

members, less than 1/3 of the original `300`

.