# ClassificationECOC

Multiclass model for support vector machines (SVMs) and other classifiers

## Description

`ClassificationECOC`

is an error-correcting output codes (ECOC)
classifier for multiclass learning, where the classifier consists of multiple binary
learners such as support vector machines (SVMs). Trained `ClassificationECOC`

classifiers store training data, parameter values, prior probabilities, and coding matrices.
Use these classifiers to perform tasks such as predicting labels or posterior probabilities
for new data (see `predict`

).

## Creation

Create a `ClassificationECOC`

object by using `fitcecoc`

.

If you specify linear or kernel binary learners without specifying cross-validation
options, then `fitcecoc`

returns a `CompactClassificationECOC`

object instead.

## Properties

After you create a `ClassificationECOC`

model object, you can use dot
notation to access its properties. For an example, see Train Multiclass Model Using SVM Learners.

### ECOC Properties

`BinaryLearners`

— Trained binary learners

cell vector of model objects

Trained binary learners, specified as a cell vector of model objects. The number of binary
learners depends on the number of classes in `Y`

and the coding
design.

The software trains `BinaryLearner{j}`

according to the binary problem
specified by `CodingMatrix`

`(:,j)`

. For example, for
multiclass learning using SVM learners, each element of
`BinaryLearners`

is a `CompactClassificationSVM`

classifier.

**Data Types: **`cell`

`BinaryLoss`

— Binary learner loss function

`'binodeviance'`

| `'exponential'`

| `'hamming'`

| `'hinge'`

| `'linear'`

| `'logit'`

| `'quadratic'`

Binary learner loss function, specified as a character vector representing the loss function name.

The default `BinaryLoss`

value depends on the score ranges returned by the
binary learners. This table identifies what some default `BinaryLoss`

values are when you use the default score transform (`ScoreTransform`

property of the model is `'none'`

).

Assumption | Default Value |
---|---|

All binary learners are any of the following: Classification decision trees Discriminant analysis models *k*-nearest neighbor modelsLinear or kernel classification models of logistic regression learners Naive Bayes models
| `'quadratic'` |

All binary learners are SVMs or linear or kernel classification models of SVM learners. | `'hinge'` |

All binary learners are ensembles trained by
`AdaboostM1` or
`GentleBoost` . | `'exponential'` |

All binary learners are ensembles trained by
`LogitBoost` . | `'binodeviance'` |

You specify to predict class posterior probabilities by setting
`'FitPosterior',true` in `fitcecoc` . | `'quadratic'` |

Binary learners are heterogeneous and use different loss functions. | `'hamming'` |

To check the default value, use dot notation to display the `BinaryLoss`

property of the trained model at the command line.

To potentially increase accuracy, specify a binary loss function other than the
default during a prediction or loss computation by using the
`BinaryLoss`

name-value argument of `predict`

or `loss`

. For more information, see Binary Loss.

**Data Types: **`char`

`BinaryY`

— Binary learner class labels

numeric matrix

Binary learner class labels, specified as a numeric matrix. `BinaryY`

is a
`NumObservations`

-by-*L* matrix, where
*L* is the number of binary learners
(`length(Mdl.BinaryLearners)`

).

Elements of `BinaryY`

are `–1`

, `0`

, or
`1`

, and the value corresponds to a dichotomous class assignment.
This table describes how learner `j`

assigns observation
`k`

to a dichotomous class corresponding to the value of
`BinaryY(k,j)`

.

Value | Dichotomous Class Assignment |
---|---|

`–1` | Learner `j` assigns observation `k` to a negative
class. |

`0` | Before training, learner `j` removes observation `k` from the data set. |

`1` | Learner `j` assigns observation `k` to a positive
class. |

**Data Types: **`double`

`BinEdges`

— Bin edges for numeric predictors

cell array of numeric vectors | `[]`

This property is read-only.

Bin edges for numeric predictors, specified as a cell array of *p* numeric vectors, where *p* is the number of predictors. Each vector includes the bin edges for a numeric predictor. The element in the cell array for a categorical predictor is empty because the software does not bin categorical predictors.

The software bins numeric predictors only if you specify the `'NumBins'`

name-value argument as a positive integer scalar when training a model with tree learners.
The `BinEdges`

property is empty if the `'NumBins'`

value is empty (default).

You can reproduce the binned predictor data `Xbinned`

by using the
`BinEdges`

property of the trained model
`mdl`

.

```
X = mdl.X; % Predictor data
Xbinned = zeros(size(X));
edges = mdl.BinEdges;
% Find indices of binned predictors.
idxNumeric = find(~cellfun(@isempty,edges));
if iscolumn(idxNumeric)
idxNumeric = idxNumeric';
end
for j = idxNumeric
x = X(:,j);
% Convert x to array if x is a table.
if istable(x)
x = table2array(x);
end
% Group x into bins by using the
````discretize`

function.
xbinned = discretize(x,[-inf; edges{j}; inf]);
Xbinned(:,j) = xbinned;
end

`Xbinned`

contains the bin indices, ranging from 1 to the number of bins, for numeric predictors.
`Xbinned`

values are 0 for categorical predictors. If
`X`

contains `NaN`

s, then the corresponding
`Xbinned`

values are `NaN`

s.
**Data Types: **`cell`

`CodingMatrix`

— Class assignment codes

numeric matrix

Class assignment codes for the binary learners, specified as a numeric matrix.
`CodingMatrix`

is a *K*-by-*L*
matrix, where *K* is the number of classes and *L* is
the number of binary learners.

The elements of `CodingMatrix`

are `–1`

,
`0`

, and `1`

, and the values correspond to
dichotomous class assignments. This table describes how learner `j`

assigns observations in class `i`

to a dichotomous class corresponding
to the value of `CodingMatrix(i,j)`

.

Value | Dichotomous Class Assignment |
---|---|

`–1` | Learner `j` assigns observations in class `i` to a negative
class. |

`0` | Before training, learner `j` removes observations
in class `i` from the data set. |

`1` | Learner `j` assigns observations in class `i` to a positive
class. |

**Data Types: **`double`

| `single`

| `int8`

| `int16`

| `int32`

| `int64`

`CodingName`

— Coding design name

character vector

Coding design name, specified as a character vector. For more details, see Coding Design.

**Data Types: **`char`

`LearnerWeights`

— Binary learner weights

numeric row vector

Binary learner weights, specified as a numeric row vector. The length of
`LearnerWeights`

is equal to the
number of binary learners
(`length(Mdl.BinaryLearners)`

).

`LearnerWeights(j)`

is the sum of the observation weights that binary learner
`j`

uses to train its classifier.

The software uses `LearnerWeights`

to fit posterior probabilities by
minimizing the Kullback-Leibler divergence. The software ignores
`LearnerWeights`

when it uses the
quadratic programming method of estimating posterior
probabilities.

**Data Types: **`double`

| `single`

### Other Classification Properties

`CategoricalPredictors`

— Categorical predictor indices

vector of positive integers | `[]`

Categorical predictor
indices, specified as a vector of positive integers. `CategoricalPredictors`

contains index values indicating that the corresponding predictors are categorical. The index
values are between 1 and `p`

, where `p`

is the number of
predictors used to train the model. If none of the predictors are categorical, then this
property is empty (`[]`

).

**Data Types: **`single`

| `double`

`ClassNames`

— Unique class labels

categorical array | character array | logical vector | numeric vector | cell array of character vectors

Unique class labels used in training, specified as a categorical or
character array, logical or numeric vector, or cell array of
character vectors. `ClassNames`

has the same
data type as the class labels `Y`

.
(The software treats string arrays as cell arrays of character
vectors.)
`ClassNames`

also determines the class
order.

**Data Types: **`categorical`

| `char`

| `logical`

| `single`

| `double`

| `cell`

`Cost`

— Misclassification costs

square numeric matrix

This property is read-only.

Misclassification costs, specified as a square numeric matrix. `Cost`

has
*K* rows and columns, where *K* is the number of
classes.

`Cost(i,j)`

is the cost of classifying a point into class
`j`

if its true class is `i`

. The order of the
rows and columns of `Cost`

corresponds to the order of the classes in
`ClassNames`

.

**Data Types: **`double`

`ExpandedPredictorNames`

— Expanded predictor names

cell array of character vectors

Expanded predictor names, specified as a cell array of character vectors.

If the model uses encoding for categorical variables, then
`ExpandedPredictorNames`

includes the names that describe the
expanded variables. Otherwise, `ExpandedPredictorNames`

is the same
as `PredictorNames`

.

**Data Types: **`cell`

`ModelParameters`

— Parameter values

object

Parameter values, such as the name-value pair argument values, used to train the
ECOC classifier, specified as an object. `ModelParameters`

does not
contain estimated parameters.

Access properties of `ModelParameters`

using dot notation. For
example, list the templates containing parameters of the binary learners by using
`Mdl.ModelParameters.BinaryLearner`

.

`NumObservations`

— Number of observations

positive numeric scalar

Number of observations in the training data, specified as a positive numeric scalar.

**Data Types: **`double`

`PredictorNames`

— Predictor names

cell array of character vectors

Predictor names in order of their appearance in the predictor data
`X`

, specified as a cell array of
character vectors. The length of
`PredictorNames`

is equal to the
number of columns in `X`

.

**Data Types: **`cell`

`Prior`

— Prior class probabilities

numeric vector

This property is read-only.

Prior class probabilities, specified as a numeric vector. `Prior`

has as
many elements as the number of classes in
`ClassNames`

, and the order of
the elements corresponds to the order of the classes in
`ClassNames`

.

`fitcecoc`

incorporates misclassification
costs differently among different types of binary learners.

**Data Types: **`double`

`ResponseName`

— Response variable name

character vector

Response variable name, specified as a character vector.

**Data Types: **`char`

`RowsUsed`

— Rows used in fitting

`[]`

(default) | logical vector

Rows of the original training data used in fitting the `ClassificationECOC`

model,
specified as a logical vector. This property is empty if all rows are used.

**Data Types: **`logical`

`ScoreTransform`

— Score transformation function to apply to predicted scores

`'doublelogit'`

| `'invlogit'`

| `'ismax'`

| `'logit'`

| `'none'`

| function handle | ...

Score transformation function to apply to predicted scores, specified as a function name or function handle.

To change the score transformation function to * function*, for
example, use dot notation.

For a built-in function, enter this code and replace

with a value in the table.`function`

Mdl.ScoreTransform = '

*function*';Value Description `"doublelogit"`

1/(1 + *e*^{–2x})`"invlogit"`

log( *x*/ (1 –*x*))`"ismax"`

Sets the score for the class with the largest score to 1, and sets the scores for all other classes to 0 `"logit"`

1/(1 + *e*^{–x})`"none"`

or`"identity"`

*x*(no transformation)`"sign"`

–1 for *x*< 0

0 for*x*= 0

1 for*x*> 0`"symmetric"`

2 *x*– 1`"symmetricismax"`

Sets the score for the class with the largest score to 1, and sets the scores for all other classes to –1 `"symmetriclogit"`

2/(1 + *e*^{–x}) – 1For a MATLAB

^{®}function or a function that you define, enter its function handle.Mdl.ScoreTransform = @

*function*;must accept a matrix (the original scores) and return a matrix of the same size (the transformed scores).`function`

**Data Types: **`char`

| `function_handle`

`W`

— Observation weights

numeric vector

Observation weights used to train the ECOC classifier, specified as a numeric
vector. `W`

has `NumObservations`

elements.

The software normalizes the weights used for training so that
`sum(W,'omitnan')`

is `1`

.

**Data Types: **`single`

| `double`

`X`

— Unstandardized predictor data

numeric matrix | table

Unstandardized predictor data used to train the ECOC classifier, specified as a numeric matrix or table.

Each row of `X`

corresponds to one observation, and each column
corresponds to one variable.

**Data Types: **`single`

| `double`

| `table`

`Y`

— Observed class labels

categorical array | character array | logical vector | numeric vector | cell array of character vectors

Observed class labels used to train the ECOC classifier, specified as a
categorical or character array, logical or numeric vector, or cell array of character
vectors. `Y`

has `NumObservations`

elements and has the same data type as the input argument `Y`

of
`fitcecoc`

. (The software treats string arrays as cell arrays of character
vectors.)

Each row of `Y`

represents the observed classification of the
corresponding row of `X`

.

**Data Types: **`categorical`

| `char`

| `logical`

| `single`

| `double`

| `cell`

### Hyperparameter Optimization Properties

`HyperparameterOptimizationResults`

— Cross-validation optimization of hyperparameters

`BayesianOptimization`

object | table

This property is read-only.

Cross-validation optimization of hyperparameters, specified as a `BayesianOptimization`

object or a table of hyperparameters and associated
values. This property is nonempty if the `'OptimizeHyperparameters'`

name-value pair argument is nonempty when you create the model. The value of
`HyperparameterOptimizationResults`

depends on the setting of the
`Optimizer`

field in the
`HyperparameterOptimizationOptions`

structure when you create the
model.

Value of `Optimizer` Field | Value of `HyperparameterOptimizationResults` |
---|---|

`'bayesopt'` (default) | Object of class `BayesianOptimization` |

`'gridsearch'` or `'randomsearch'` | Table of hyperparameters used, observed objective function values (cross-validation loss), and rank of observations from lowest (best) to highest (worst) |

## Object Functions

`compact` | Reduce size of multiclass error-correcting output codes (ECOC) model |

`compareHoldout` | Compare accuracies of two classification models using new data |

`crossval` | Cross-validate multiclass error-correcting output codes (ECOC) model |

`discardSupportVectors` | Discard support vectors of linear SVM binary learners in ECOC model |

`edge` | Classification edge for multiclass error-correcting output codes (ECOC) model |

`gather` | Gather properties of Statistics and Machine Learning Toolbox object from GPU |

`incrementalLearner` | Convert multiclass error-correcting output codes (ECOC) model to incremental learner |

`loss` | Classification loss for multiclass error-correcting output codes (ECOC) model |

`margin` | Classification margins for multiclass error-correcting output codes (ECOC) model |

`partialDependence` | Compute partial dependence |

`plotPartialDependence` | Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots |

`predict` | Classify observations using multiclass error-correcting output codes (ECOC) model |

`resubEdge` | Resubstitution classification edge for multiclass error-correcting output codes (ECOC) model |

`lime` | Local interpretable model-agnostic explanations (LIME) |

`resubLoss` | Resubstitution classification loss for multiclass error-correcting output codes (ECOC) model |

`resubMargin` | Resubstitution classification margins for multiclass error-correcting output codes (ECOC) model |

`resubPredict` | Classify observations in multiclass error-correcting output codes (ECOC) model |

`shapley` | Shapley values |

`testckfold` | Compare accuracies of two classification models by repeated cross-validation |

## Examples

### Train Multiclass Model Using SVM Learners

Train a multiclass error-correcting output codes (ECOC) model using support vector machine (SVM) binary learners.

Load Fisher's iris data set. Specify the predictor data `X`

and the response data `Y`

.

```
load fisheriris
X = meas;
Y = species;
```

Train a multiclass ECOC model using the default options.

Mdl = fitcecoc(X,Y)

Mdl = ClassificationECOC ResponseName: 'Y' CategoricalPredictors: [] ClassNames: {'setosa' 'versicolor' 'virginica'} ScoreTransform: 'none' BinaryLearners: {3x1 cell} CodingName: 'onevsone' Properties, Methods

`Mdl`

is a `ClassificationECOC`

model. By default, `fitcecoc`

uses SVM binary learners and a one-versus-one coding design. You can access `Mdl`

properties using dot notation.

Display the class names and the coding design matrix.

Mdl.ClassNames

`ans = `*3x1 cell*
{'setosa' }
{'versicolor'}
{'virginica' }

CodingMat = Mdl.CodingMatrix

`CodingMat = `*3×3*
1 1 0
-1 0 1
0 -1 -1

A one-versus-one coding design for three classes yields three binary learners. The columns of `CodingMat`

correspond to the learners, and the rows correspond to the classes. The class order is the same as the order in `Mdl.ClassNames`

. For example, `CodingMat(:,1)`

is `[1; –1; 0]`

and indicates that the software trains the first SVM binary learner using all observations classified as `'setosa'`

and `'versicolor'`

. Because `'setosa'`

corresponds to `1`

, it is the positive class; `'versicolor'`

corresponds to `–1`

, so it is the negative class.

You can access each binary learner using cell indexing and dot notation.

`Mdl.BinaryLearners{1} % The first binary learner`

ans = CompactClassificationSVM ResponseName: 'Y' CategoricalPredictors: [] ClassNames: [-1 1] ScoreTransform: 'none' Beta: [4x1 double] Bias: 1.4492 KernelParameters: [1x1 struct] Properties, Methods

Compute the resubstitution classification error.

error = resubLoss(Mdl)

error = 0.0067

The classification error on the training data is small, but the classifier might be an overfitted model. You can cross-validate the classifier using `crossval`

and compute the cross-validation classification error instead.

### Inspect Binary Learner Properties of ECOC Classifier

Train an ECOC classifier using SVM binary learners. Then, access properties of the binary learners, such as estimated parameters, by using dot notation.

Load Fisher's iris data set. Specify the petal dimensions as the predictors and the species names as the response.

```
load fisheriris
X = meas(:,3:4);
Y = species;
```

Train an ECOC classifier using SVM binary learners and the default coding design (one-versus-one). Standardize the predictors and save the support vectors.

t = templateSVM('Standardize',true,'SaveSupportVectors',true); predictorNames = {'petalLength','petalWidth'}; responseName = 'irisSpecies'; classNames = {'setosa','versicolor','virginica'}; % Specify class order Mdl = fitcecoc(X,Y,'Learners',t,'ResponseName',responseName,... 'PredictorNames',predictorNames,'ClassNames',classNames)

Mdl = ClassificationECOC PredictorNames: {'petalLength' 'petalWidth'} ResponseName: 'irisSpecies' CategoricalPredictors: [] ClassNames: {'setosa' 'versicolor' 'virginica'} ScoreTransform: 'none' BinaryLearners: {3x1 cell} CodingName: 'onevsone' Properties, Methods

`t`

is a template object that contains options for SVM classification. The function `fitcecoc`

uses default values for the empty (`[]`

) properties. `Mdl`

is a `ClassificationECOC`

classifier. You can access properties of `Mdl`

using dot notation.

Display the class names and the coding design matrix.

Mdl.ClassNames

`ans = `*3x1 cell*
{'setosa' }
{'versicolor'}
{'virginica' }

Mdl.CodingMatrix

`ans = `*3×3*
1 1 0
-1 0 1
0 -1 -1

The columns correspond to SVM binary learners, and the rows correspond to the distinct classes. The row order is the same as the order in the `ClassNames`

property of `Mdl`

. For each column:

`1`

indicates that`fitcecoc`

trains the SVM using observations in the corresponding class as members of the positive group.`–1`

indicates that`fitcecoc`

trains the SVM using observations in the corresponding class as members of the negative group.`0`

indicates that the SVM does not use observations in the corresponding class.

In the first SVM, for example, `fitcecoc`

assigns all observations to `'setosa'`

or `'versicolor'`

, but not `'virginica'`

.

Access properties of the SVMs using cell subscripting and dot notation. Store the standardized support vectors of each SVM. Unstandardize the support vectors.

L = size(Mdl.CodingMatrix,2); % Number of SVMs sv = cell(L,1); % Preallocate for support vector indices for j = 1:L SVM = Mdl.BinaryLearners{j}; sv{j} = SVM.SupportVectors; sv{j} = sv{j}.*SVM.Sigma + SVM.Mu; end

`sv`

is a cell array of matrices containing the unstandardized support vectors for the SVMs.

Plot the data, and identify the support vectors.

figure gscatter(X(:,1),X(:,2),Y); hold on markers = {'ko','ro','bo'}; % Should be of length L for j = 1:L svs = sv{j}; plot(svs(:,1),svs(:,2),markers{j},... 'MarkerSize',10 + (j - 1)*3); end title('Fisher''s Iris -- ECOC Support Vectors') xlabel(predictorNames{1}) ylabel(predictorNames{2}) legend([classNames,{'Support vectors - SVM 1',... 'Support vectors - SVM 2','Support vectors - SVM 3'}],... 'Location','Best') hold off

You can pass `Mdl`

to these functions:

`predict`

, to classify new observations`resubLoss`

, to estimate the classification error on the training data`crossval`

, to perform 10-fold cross-validation

### Cross-Validate ECOC Classifier

Cross-validate an ECOC classifier with SVM binary learners, and estimate the generalized classification error.

Load Fisher's iris data set. Specify the predictor data `X`

and the response data `Y`

.

load fisheriris X = meas; Y = species; rng(1); % For reproducibility

Create an SVM template, and standardize the predictors.

`t = templateSVM('Standardize',true)`

t = Fit template for classification SVM. Alpha: [0x1 double] BoxConstraint: [] CacheSize: [] CachingMethod: '' ClipAlphas: [] DeltaGradientTolerance: [] Epsilon: [] GapTolerance: [] KKTTolerance: [] IterationLimit: [] KernelFunction: '' KernelScale: [] KernelOffset: [] KernelPolynomialOrder: [] NumPrint: [] Nu: [] OutlierFraction: [] RemoveDuplicates: [] ShrinkagePeriod: [] Solver: '' StandardizeData: 1 SaveSupportVectors: [] VerbosityLevel: [] Version: 2 Method: 'SVM' Type: 'classification'

`t`

is an SVM template. Most of the template object properties are empty. When training the ECOC classifier, the software sets the applicable properties to their default values.

Train the ECOC classifier, and specify the class order.

Mdl = fitcecoc(X,Y,'Learners',t,... 'ClassNames',{'setosa','versicolor','virginica'});

`Mdl`

is a `ClassificationECOC`

classifier. You can access its properties using dot notation.

Cross-validate `Mdl`

using 10-fold cross-validation.

CVMdl = crossval(Mdl);

`CVMdl`

is a `ClassificationPartitionedECOC`

cross-validated ECOC classifier.

Estimate the generalized classification error.

genError = kfoldLoss(CVMdl)

genError = 0.0400

The generalized classification error is 4%, which indicates that the ECOC classifier generalizes fairly well.

## More About

### Error-Correcting Output Codes Model

An *error-correcting output codes (ECOC) model* reduces
the problem of classification with three or more classes to a set of binary classification
problems.

ECOC classification requires a coding design, which determines the classes that the binary learners train on, and a decoding scheme, which determines how the results (predictions) of the binary classifiers are aggregated.

Assume the following:

The classification problem has three classes.

The coding design is one-versus-one. For three classes, this coding design is

$$\begin{array}{cccc}& \text{Learner1}& \text{Learner2}& \text{Learner3}\\ \text{Class1}& 1& 1& 0\\ \text{Class2}& -1& 0& 1\\ \text{Class3}& 0& -1& -1\end{array}$$

You can specify a different coding design by using the

`Coding`

name-value argument when you create a classification model.The model determines the predicted class by using the loss-weighted decoding scheme with the binary loss function

*g*. The software also supports the loss-based decoding scheme. You can specify the decoding scheme and binary loss function by using the`Decoding`

and`BinaryLoss`

name-value arguments, respectively, when you call object functions, such as`predict`

,`loss`

,`margin`

,`edge`

, and so on.

The ECOC algorithm follows these steps.

Learner 1 trains on observations in Class 1 or Class 2, and treats Class 1 as the positive class and Class 2 as the negative class. The other learners are trained similarly.

Let

*M*be the coding design matrix with elements*m*, and_{kl}*s*be the predicted classification score for the positive class of learner_{l}*l*. The algorithm assigns a new observation to the class ($$\widehat{k}$$) that minimizes the aggregation of the losses for the*B*binary learners.$$\widehat{k}=\underset{k}{\text{argmin}}\frac{{\displaystyle \sum}_{l=1}^{B}\left|{m}_{kl}\right|g\left({m}_{kl},{s}_{l}\right)}{{\displaystyle \sum}_{l=1}^{B}\left|{m}_{kl}\right|}.$$

ECOC models can improve classification accuracy, compared to other multiclass models [1].

### Coding Design

The *coding design* is a matrix whose elements direct
which classes are trained by each binary learner, that is, how the multiclass problem is
reduced to a series of binary problems.

Each row of the coding design corresponds to a distinct class, and each column corresponds to a binary learner. In a ternary coding design, for a particular column (or binary learner):

A row containing 1 directs the binary learner to group all observations in the corresponding class into a positive class.

A row containing –1 directs the binary learner to group all observations in the corresponding class into a negative class.

A row containing 0 directs the binary learner to ignore all observations in the corresponding class.

Coding design matrices with large, minimal, pairwise row distances based on the Hamming measure are optimal. For details on the pairwise row distance, see Random Coding Design Matrices and [2].

This table describes popular coding designs.

Coding Design | Description | Number of Learners | Minimal Pairwise Row Distance |
---|---|---|---|

one-versus-all (OVA) | For each binary learner, one class is positive and the rest are negative. This design exhausts all combinations of positive class assignments. | K | 2 |

one-versus-one (OVO) | For each binary learner, one class is positive, one class is negative, and the rest are ignored. This design exhausts all combinations of class pair assignments. |
| 1 |

binary complete | This design partitions the classes into all binary
combinations, and does not ignore any classes. That is, all class
assignments are | 2^{K – 1} – 1 | 2^{K – 2} |

ternary complete | This design partitions the classes into all ternary
combinations. That is, all class assignments are
| (3 | 3^{K – 2} |

ordinal | For the first binary learner, the first class is negative and the rest are positive. For the second binary learner, the first two classes are negative and the rest are positive, and so on. | K – 1 | 1 |

dense random | For each binary learner, the software randomly assigns classes into positive or negative classes, with at least one of each type. For more details, see Random Coding Design Matrices. | Random, but approximately 10
log | Variable |

sparse random | For each binary learner, the software randomly assigns classes as positive or negative with probability 0.25 for each, and ignores classes with probability 0.5. For more details, see Random Coding Design Matrices. | Random, but approximately 15
log | Variable |

This plot compares the number of binary learners for the coding designs with
increasing *K*.

## Algorithms

### Random Coding Design Matrices

For a given number of classes *K*, the software generates random coding
design matrices as follows.

The software generates one of these matrices:

Dense random — The software assigns 1 or –1 with equal probability to each element of the

*K*-by-*L*coding design matrix, where $${L}_{d}\approx \lceil 10{\mathrm{log}}_{2}K\rceil $$._{d}Sparse random — The software assigns 1 to each element of the

*K*-by-*L*coding design matrix with probability 0.25, –1 with probability 0.25, and 0 with probability 0.5, where $${L}_{s}\approx \lceil 15{\mathrm{log}}_{2}K\rceil $$._{s}

If a column does not contain at least one 1 and one –1, then the software removes that column.

For distinct columns

*u*and*v*, if*u*=*v*or*u*= –*v*, then the software removes*v*from the coding design matrix.

The software randomly generates 10,000 matrices by default, and retains the matrix with the largest, minimal, pairwise row distance based on the Hamming measure ([2]) given by

$$\Delta ({k}_{1},{k}_{2})=0.5{\displaystyle \sum}_{l=1}^{L}\left|{m}_{{k}_{1}l}\right|\left|{m}_{{k}_{2}l}\right|\left|{m}_{{k}_{1}l}-{m}_{{k}_{2}l}\right|,$$

where
*m _{kjl}* is an element of
coding design matrix

*j*.

### Support Vector Storage

By default and for efficiency, `fitcecoc`

empties the `Alpha`

, `SupportVectorLabels`

,
and `SupportVectors`

properties
for all linear SVM binary learners. `fitcecoc`

lists `Beta`

, rather than
`Alpha`

, in the model display.

To store `Alpha`

, `SupportVectorLabels`

, and
`SupportVectors`

, pass a linear SVM template that specifies storing
support vectors to `fitcecoc`

. For example,
enter:

t = templateSVM('SaveSupportVectors',true) Mdl = fitcecoc(X,Y,'Learners',t);

You can remove the support vectors and related values by passing the resulting
`ClassificationECOC`

model to
`discardSupportVectors`

.

## Alternative Functionality

You can use these alternative algorithms to train a multiclass model:

Classification ensembles—see

`fitcensemble`

and`ClassificationEnsemble`

Classification trees—see

`fitctree`

and`ClassificationTree`

Discriminant analysis classifiers—see

`fitcdiscr`

and`ClassificationDiscriminant`

*k*-nearest neighbor classifiers—see`fitcknn`

and`ClassificationKNN`

Naive Bayes classifiers—see

`fitcnb`

and`ClassificationNaiveBayes`

## References

[1] Fürnkranz, Johannes. “Round Robin
Classification.” *J. Mach. Learn. Res.*, Vol. 2, 2002, pp.
721–747.

[2] Escalera, S., O. Pujol, and P. Radeva. “Separability of ternary codes for sparse designs of error-correcting output codes.” *Pattern Recog. Lett.*, Vol. 30, Issue 3, 2009, pp. 285–297.

## Extended Capabilities

### C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

When you train an ECOC model by using

`fitcecoc`

, the following restrictions apply.All binary learners must be either SVM classifiers or linear classification models. For the

`Learners`

name-value argument, you can specify:`'svm'`

or`'linear'`

An SVM template object or a cell array of such objects (see

`templateSVM`

)A linear classification model template object or a cell array of such objects (see

`templateLinear`

)

When you generate code using a coder configurer for

`predict`

and`update`

, the following additional restrictions apply for binary learners.If you use a cell array of SVM template objects, the value of

`Standardize`

for SVM learners must be consistent. For example, if you specify`'Standardize',true`

for one SVM learner, you must specify the same value for all SVM learners.If you use a cell array of SVM template objects, and you use one SVM learner with a linear kernel (

`'KernelFunction','linear'`

) and another with a different type of kernel function, then you must specify

for the learner with a linear kernel.`'SaveSupportVectors'`

,true

For details, see

`ClassificationECOCCoderConfigurer`

. For information on name-value arguments that you cannot modify when you retrain a model, see Tips.Code generation limitations for SVM classifiers and linear classification models also apply to ECOC classifiers, depending on the choice of binary learners. For more details, see Code Generation of the

`CompactClassificationSVM`

class and Code Generation of the`ClassificationLinear`

class.

For more information, see Introduction to Code Generation.

### GPU Arrays

Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.

Usage notes and limitations:

The following object functions fully support GPU arrays:

The following object functions offer limited support for GPU arrays:

For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).

## Version History

**Introduced in R2014b**

## Open Example

You have a modified version of this example. Do you want to open this example with your edits?

## MATLAB Command

You clicked a link that corresponds to this MATLAB command:

Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.

# Select a Web Site

Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .

You can also select a web site from the following list:

## How to Get Best Site Performance

Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.

### Americas

- América Latina (Español)
- Canada (English)
- United States (English)

### Europe

- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)

- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)