sequentialfs
Sequential feature selection using custom criterion
Syntax
Description
selects a subset of features in tf
= sequentialfs(fun
,X
,y
)X
that are important for predicting
y
. The function defines a random nonstratified partition for 10-fold
cross-validation using X
and y
, and then
sequentially selects features based on the cross-validate prediction criterion values
computed by the fun
function. The initial feature set includes no
features. sequentialfs
adds one feature to the set at each iteration,
until adding a feature does not decrease the criterion value by greater than the termination
tolerance value. The output tf
is a logical vector that indicates the
selected features. For more details, see Algorithms.
specifies options using one or more name-value arguments in addition to any of the input
argument combinations in the previous syntaxes. For example, specify
tf
= sequentialfs(___,Name,Value
)"Direction","backward"
to perform recursive feature elimination (RFE).
The initial feature set includes all features. sequentialfs
removes one
feature from the set at each iteration, until removing a feature does not decrease the
prediction criterion.
Examples
Input Arguments
Output Arguments
More About
Algorithms
sequentialfs
sequentially selects features in X
by performing these steps:
Define a random nonstratified partition for 10-fold cross-validation on
n
observations, wheren
is the number of observations inX
.Initialize the selected feature set
S
as an empty set.For each feature xi in
X
, compute the cross-validated criterion value using thefun
function.Add the feature with the smallest criterion value to
S
.For each feature xi in
X\S
, define a candidate feature setC
i asS
∪{xi}. Compute the cross-validated criterion value usingfun
forC
i.Among the candidate sets (
C
is), select the set that reduces the criterion value the most, compared to the criterion value forS
. Add the feature corresponding to the selected candidate set toS
.Repeat steps 5 and 6 until adding a feature does not decrease the criterion value by greater than the termination tolerance value.
To customize the feature selection process, use the name-value arguments of
sequentialfs
.
You can specify cross-validation options by using the
CV
andMCReps
name-value arguments.For wrapper type feature selection, specify the arguments to cross-validate the criterion value for each candidate feature set. You can define the
fun
function to train a model and return a criterion value for the trained model. For an example, see Forward Feature Selection.For filter type feature selection, which does not involve cross-validation, specify
CV
as"none"
and use thefun
function to measure characteristics of the input data, such as correlation. For an example, see Filter Type Feature Selection.
To perform backward feature selection, or recursive feature elimination (RFE), specify the
Direction
name-value argument as"backward"
.sequentialfs
initializes the selected feature setS
as a set with all features, and then removes one feature at a time from the set.You can specify which features to always include or exclude, the number of features in the final selected feature set, and whether to consider a model with no features as a valid option. For details, see the
KeepIn
,KeepOut
,NFeatures
, andNullModel
name-value arguments.Use the
Options
name-value argument to specify options for the iterations and parallel computation. For example,Options,statset("TolFun",1e-2)
sets the iteration termination tolerance on the criterion value to1e-2
.
Extended Capabilities
Version History
Introduced in R2008a