Plotting ROC Curve for Multiclass

Hi, I'm trying to make a multiclass ROC Curve and i think i'm close to it but it doesn't work. I've spent quite a while to figure out where the problem is, yet it came to nothing. I'm curious could someone who had done it before can help me ?

Firstly I have the classes below.

template=templateSVM('Standardize',true,'KernelFunction','gaussian')
MultiClassSVM=fitcecoc(winedataset,'quality','Learners',template,'FitPosterior',true)
yPredicted=predict(MultiClassSVM,winedataset)
fprintf(sprintf('Training accuracy: %0.2f%%', 100*mean(yPredicted==winedataset.quality)));
confusionchart(winedataset.quality,yPredicted,'RowSummary','row-normalized')

MultiClassSVM.ClassNames
ans =

     3
     4
     5
     6
     7
     8   

This is my confusion matrix. Furthermore, I'm using the code below to create a ROC curve.

[~,score_svm] = resubPredict(MultiClassSVM);

diffscore=zeros;
for i=1:size(score_svm,1)
    temp=score_svm(i,:); 
    diffscore(i,:)=temp(2)-max([temp(1),temp(3),temp(4),temp(5),temp(6)]); 
end

[X,Y,T,~,OPTROCPT,suby,subnames] = perfcurve(winedataset.quality,diffscore,4)
plot(X,Y)
hold on
plot(OPTROCPT(1),OPTROCPT(2),'ro')  
xlabel('False positive rate') 
ylabel('True positive rate')
title('ROC for Classification by SVM')
fprintf("Area Under the Curve (AUC) = %f",AUCsvm) 
hold off

Well the curve is supposed to have a value like 0.94 yet it shows 1. I probably made a mistake while calculating diffscore but i'm not quite sure of it. I would appreciate if anyone who has done it before could help.

Answer 1

[sanidhyak]

Hello @Gorkem Akgul,

I understand that you are trying to plot a multiclass ROC curve in MATLAB using “fitcecoc” with SVM learners. From the code you shared, I noticed that you are manually calculating a variable named “diffscore” to pass into the “perfcurve” function. This is not needed and is the reason why your ROC curve appears incorrect (always reaching 1).

The “resubPredict” function with the “FitPosterior”, true option already provides posterior probabilities for each class, which can be directly used with “perfcurve”. To correctly generate ROC curves for a multiclass problem, you need to apply the “one-vs-all” strategy, i.e., compute and plot an ROC curve for each class individually.

Kindly refer to the below corrected code:

% Train multiclass SVM with posterior probabilities
template = templateSVM('Standardize',true,'KernelFunction','gaussian');
MultiClassSVM = fitcecoc(winedataset,'quality','Learners',template,'FitPosterior',true);

% Get posterior scores
[~,score_svm] = resubPredict(MultiClassSVM);

% Get class names
classNames = MultiClassSVM.ClassNames;

% Plot ROC for each class (one-vs-all)
figure;
hold on;
legendEntries = {};

for i = 1:numel(classNames)
    % Binary labels for current class vs rest
    trueClass = (winedataset.quality == classNames(i));
    
    % Use posterior scores for this class
    [X,Y,~,AUC] = perfcurve(trueClass, score_svm(:,i), true);
    
    % Plot ROC
    plot(X,Y,'LineWidth',1.5);
    legendEntries{end+1} = sprintf('Class %d (AUC = %.2f)', classNames(i), AUC);
end

xlabel('False Positive Rate');
ylabel('True Positive Rate');
title('Multiclass ROC Curve (One-vs-All)');
legend(legendEntries,'Location','Best');
grid on;
hold off;

With these corrections, now you will be able to visualize a separate ROC curve for each class “(3, 4, 5, 6, 7, 8)” and obtain the correct AUC values.

For further details, you may refer to the following MATLAB documentation on “perfcurve”:

• https://www.mathworks.com/help/stats/perfcurve.html

I hope this is helpful!