Change the range of potential hyperparameters when optimizing hyperparameter choice using fitrensemble

5 views (last 30 days)
Hi,
I am experimenting with randon search and grid search optimizers when optimizing hyperparameters for a boosted trees regression model, using the fitrensemble function.
To speed up the process, and also avoid overfitting, I want to specify non-default ranges to explore for the hyperparamters - however after reading documentation, I still don't understand how to do this.
For example, in the code below:
Mdl_ls = fitrensemble(X,Y,'Learners','tree', ...
'OptimizeHyperparameters',{'NumLearningCycles','LearnRate','MaxNumSplits'},...
'HyperparameterOptimizationOptions',struct(...
'Optimizer','randomsearch', 'MaxObjectiveEvaluations', 500, 'ShowPlots', false, 'Verbose', 1 ));
How would I go about changing the search range for the LearnRate to between 0.001 and 0.2 - rather than the defaul range of [1e-3,1].

Accepted Answer

Alan Weiss
Alan Weiss on 20 May 2020
Looking in the documentation for fitrensemble I find this:
----
Set nondefault parameters by passing a vector of optimizableVariable objects that have nondefault values. For example,
load carsmall
params = hyperparameters('fitrensemble',[Horsepower,Weight],MPG,'Tree');
params(4).Range = [1,20];
Pass params as the value of OptimizeHyperparameters.
----
Is that clear to you? For more information, see https://www.mathworks.com/help/stats/hyperparameters.html#bvdtepr-1
Alan Weiss
MATLAB mathematical toolbox documentation
  6 Comments
Machine Learning Enthusiast
I am getting following error:
params(1).Range = {'Bag'};
Error using optimizableVariable/checkRange (line 172)
'Range' value must be a string array, a cell array of character vectors, or a numeric vector of length 2.
Alan Weiss
Alan Weiss on 27 Jul 2021
Sorry, my mistake. It seems that if you want to have a single value there you have to repeat it.
params(1).Range = {'Bag' 'Bag'};
params(1).Optimize = false;
Maybe this is bad advice somehow, I don't know, but it seems to work.
Alan Weiss
MATLAB mathematical toolbox documentation

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!