Loss function with fitcecoc and posteriors

Hello,

I'm not sure to understand why the loss function has to be a quadratic one using fitecoc with fitPosteriors=true while training (using SVM as binary learner). This information is simply mentioned in the description of fitecoc (https://fr.mathworks.com/help/stats/fitcecoc.html#).

Why can't the others loss function be used for this specific case ? Is it due to the fact that posterior probabilities are limited to [0,1] ? It also suprising because the Hinge loss is massively used for SVM.

This loss function has also the caracteristic of penalizing samples that are considered to be well classified and without ambiguity. In term of probability this phenomena should not happend.

Best regards.

Answer 1

[Neha]

Hi Dostya,

I understand that you have some queries regarding the ECOC model which uses SVM binary classifiers. In the ECOC framework, binary classifiers (such as Support Vector Machines) are trained to distinguish between pairs of classes. The outputs of these binary classifiers are combined to make multiclass predictions. The "fitecoc" function in MATLAB uses a quadratic loss function because it is mathematically consistent with the ECOC approach and ensures that the combination of binary classifiers produces accurate and reliable multiclass predictions.

Using other loss functions, such as the hinge loss commonly used in SVM, may not be suitable in the ECOC framework. The hinge loss is designed for binary classification and may not provide the same level of confidence and reliability in the multiclass predictions generated by the ECOC approach.

Regarding your observation about the quadratic loss function penalizing samples that are considered to be well classified and without ambiguity, this can happen because the quadratic loss function tries to maximize the margin between classes, which can lead to overfitting when there are few training examples or when the data is noisy.

Hope this helps!