How to obtain a ROC curve through cross validation on the training data?

Hi all...I have raw EEG data of size ( 1025 x 1 ) .Out of that I used data of size (820 x 1) for training and remaining (205 x 1) for testing. Now I want a find a threshold (varied b/w 0.1:0.1:1) by obtaining ROC through cross validation (5-fold) on the training data. How to obtain a ROC curve through cross validation on the training data? Find the code below:

    threshold = 0.1:0.1:1;; numfolds = 5;
    for ithreshold = 1:length(threshold);
        cv = cvpartition(train_data,'Kfold',numfolds);
        %cvFolds = crossvalind('Kfold',train_data,numfolds);
        for ifold = 1:numfolds;
            trIdx = cv.training(ifold);teIdx = cv.test(ifold);
            train_data = norm_temp(trIdx,:);test_data = norm_temp(teIdx,:);
            [tpr,fpr,ithreshold] = roc(train_data,teIdx);

Answer 1

[Mudambi Srivatsa]

In Statistics Toolbox, [X,Y] = perfcurve(labels,scores,posclass) computes a ROC curve for a vector of classifier predictions scores given the true class labels.

For more information on the "perfcurve" function, refer to the following link:

http://www.mathworks.com/help/stats/perfcurve.html

The following links provide some examples of using "perfcurve" function:

http://www.mathworks.com/matlabcentral/answers/96334-how-can-i-create-an-roc-plot-from-the-set-of-cross-validation-models-using-statistics-toolbox-7-5-r

http://stackoverflow.com/questions/20267563/obtaining-roc-through-cross-validation-on-the-training-data

[Jessica Schrouff]

How do people do when they do not have the statistics toolbox? I'm trying to use the interpolation as scikit-learn does, but an error is thrown:

Error using griddedInterpolant
The grid vectors must contain unique points.

The only solution for this error was to do a unique, but quite a few points of the curve would be lost. It would also tend to be under-estimating as it would take only the first occurrence of each x.