Random Forest in Classification Learner App: what the inputs are

Hello everyone,

I'm about to use Random Forest (Bagged Trees) in the classification learner app to train a set of 350 observations with 27 features.

I'm not a machine learning expert, and so far I understand that RF requires two inputs: - Number of decision trees, and - Number of predictor variables.

However in the app I have two other inputs: - Max number of splits and - Number of learners.

I need help understanding (in simple terms whenever possible) what this two last inputs are so I can set them properly in my training process.

Thanks in advance for any help.

Ame

Answer 1

[Yash Ubale]

Hi Ame,

I understand that you want to know the significance of the two parameters. I am explaining the significance of "Number of learners" and "Maximum number of splits" below:

Number of learners - It basically means the number of decision trees in your random forest. Greater the value, greater will be the number of subsets your data will be divided into to train every decision tree.

Maximum number of splits - With decision trees, you can choose a splitting variable at every tree depth using which the data will be split. It basically defines the depth of your decision tree. Very high number may cause overfitting and very low number may cause underfitting.

TreeBagger - https://www.mathworks.com/help/stats/treebagger.html

[Ame ZL]

HI Yash,

Thanks very much for your reply.

Yes, I want to understand what the Machine Learning App parameters for Random Forest are, so thanks for your reply. But also how are these setting parameters related to 'number of decision trees' and 'number of predictor variables', used by the Random Forest algorithm.

I'm replicating studies using these last parameters, and I'm finding difficult to replicate the results using Max number of splits instead.

So far, it seems number of learners will be equivalent to number of desicion trees. But that Max number of splits is not equivalent to Number of predictor variables (or is it?). How are they related? or how can i set the number of predictor variables to be randomly selected in each iteration. I understand that if all predictor variables were used then is not the Random Forest algorithm.

Please correct me if I'm wrong.

Looking forward for your comments.

Many thanks,

Ame

[Yash Ubale]

Hi Ame,

I understand that you want to understand the algorithm used and the parameters that can be tuned to replicate your results.

The algorithm used by "Classification Learner" is Breiman's 'random forest' algorithm. "Number of predictor variables" is different from "Maximum number of splits" in a sense that the later is any number up to the maximum limit that you have set and the previous one corresponds to the exact number. They can be same if "Number of predictor variables" is equal to the "Maximum number of splits".

As of now you cannot set the number of predicted variables to be randomly selected in each iteration in the "Classification Learner". You have to run it manually and compare the results and then finally choose parameters that best replicate your results. You can write a script and do everything programmatically.

Secondly, if you wish to go through all the parameters that can be tuned to build a robust model kindly go through this link.

https://www.mathworks.com/help/releases/R2018a/stats/choose-a-classifier.html

Answer 2

[Ame ZL]

Hi Yash,

Thanks very much for your thorough reply. It is clearer now. I will go trough the link you have suggested.

Best wishes,

Ame