How to get the mean of ROC curves using Matlab?
4 views (last 30 days)
Show older comments
I met a problem to plot the mean ROC curve of the 10-fold cross-validation using Matlab.
I run the code cvPartition = cvpartition(dataSize,'k', 10); to get 10 fold of training and testing. However, as it randomly choose the number of training and testing. The ROC curve I got from each fold is with different size. In addition, I want to plot the mean ROC of these ten ROC curves I got from the cross-validation. Anyone knows how to do this? I read another post using Python perfectly solve the problem using 1D interpolation. Not sure how to do this in Matlab.
All the FPR and TPR values:
FPR_All =
Columns 1 through 9
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0.2500 0.2000 0 0.1667 0.1667 0.1429 0.3333 0.2000 0
0.5000 0.4000 0.2500 0.3333 0.3333 0.2857 0.6667 0.4000 0.3333
0.7500 0.6000 0.5000 0.5000 0.5000 0.4286 1.0000 0.6000 0.6667
1.0000 0.8000 0.7500 0.6667 0.6667 0.5714 NaN 0.8000 1.0000
NaN 1.0000 1.0000 0.8333 0.8333 0.7143 NaN 1.0000 NaN
NaN NaN NaN 1.0000 1.0000 0.8571 NaN NaN NaN
NaN NaN NaN NaN NaN 1.0000 NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN
Column 10
0
0
0.1429
0.2857
0.4286
0.5714
0.7143
0.8571
1.0000
NaN
TPR_All =
Columns 1 through 9
0 0 0 0 0 0 0 0 0
1.0000 1.0000 0.8333 1.0000 1.0000 1.0000 1.0000 1.0000 0.8571
1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 NaN 1.0000 1.0000
NaN 1.0000 1.0000 1.0000 1.0000 1.0000 NaN 1.0000 NaN
NaN NaN NaN 1.0000 1.0000 1.0000 NaN NaN NaN
NaN NaN NaN NaN NaN 1.0000 NaN NaN NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN
Column 10
0
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
NaN
Answers (2)
Erik
on 1 Sep 2016
I guess the matrix rows are the ROC-curves, each a number of elements in every column (ans a few NaN after the 1). Then you could take the mean of every row while replacing NaNs using
matrix(isnan(matrix)) = 1; % replace NaN with 1
meanROC = mean(matrix,2); % mean of rows
which will tell MATLAB to take the mean along the 2nd dimension of the matrix, i.e. its rows. The NaNs must be replaces with ones, because otherwise the mean function would return NaN on every row with a NaN. The result is a column vector with the mean ROC-value.
1 Comment
Ilya
on 1 Sep 2016
Use perfcurve. Take a look at this piece of documentation. Pass true labels and predicted scores as cell arrays, one element per fold. You will get the mean curve and confidence intervals.
5 Comments
Ilya
on 5 Sep 2016
Yes, here you are using vertical averaging. I have trouble telling where red is, but if red is the smoothest line going midway, it looks sensible.
See Also
Categories
Find more on Detection in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!