Clear Filters
Clear Filters

Is this ROC curve and AUC correct?

3 views (last 30 days)
Win Sheng Liew
Win Sheng Liew on 13 Oct 2018
Edited: KALYAN ACHARJYA on 13 Oct 2018
Hi, I have extracted the features for 2 set image datasets (normal & abnormal) through the deep learning. May I know is my code correct? (Cause my AUC is 0.999 which is almost equal to 1). Please advise, thank you.
rootFolder = fullfile('c:\', 'Users', 'Ws Liew', 'Documents', 'MATLAB', 'Dataset');
categories = {'normal', 'abnormal'};
imds = imageDatastore(fullfile(rootFolder, categories), 'LabelSource','foldernames');
% Notice that each set now has exactly the same number of images.
countEachLabel(imds)
% Find the first instance of an image for each category
normal = find(imds.Labels == 'normal', 1);
abnormal = find(imds.Labels == 'abnormal', 1);
% Load pretrained network
net = alexnet;
featureLayer = 'fc8';
% View the CNN architecture
net.Layers
% Inspect the first layer
net.Layers(1)
% Inspect the last layer
net.Layers(end)
% Number of class names for ImageNet classification task
numel(net.Layers(end).ClassNames)
%Prepare Training and Test Image Sets
[trainingSet, testSet] = splitEachLabel(imds, 0.65, 'randomized');
% Create augmentedImageDatastore from training and test sets to resize
% images in imds to the size required by the network.
imageSize = net.Layers(1).InputSize;
augmentedTrainingSet = augmentedImageDatastore(imageSize(1:2), trainingSet, 'ColorPreprocessing', 'gray2rgb');
augmentedTestSet = augmentedImageDatastore(imageSize(1:2), testSet, 'ColorPreprocessing', 'gray2rgb');
% Get the network weights for the second convolutional layer
w1 = net.Layers(2).Weights;
% Scale and resize the weights for visualization
w1 = mat2gray(w1);
w1 = imresize(w1,5);
trainingFeatures = activations(net, augmentedTrainingSet, featureLayer, 'OutputAs', 'rows');
trainingFeatures = double(trainingFeatures);
testFeatures = activations(net, augmentedTestSet, featureLayer, 'OutputAs', 'rows');
testFeatures = double(testFeatures);
% Get training and test labels from the trainingSet and testSet
trainingLabels = trainingSet.Labels;%YTrain
trainingLabels = cellstr(trainingLabels);
testLabels = testSet.Labels; %YTest
testLabels = cellstr(testLabels);
rng(1);
t = templateSVM('Standardize',1)
Md1 = fitcecoc(double(trainingFeatures), cellstr(trainingLabels),'Learners',t, 'FitPosterior',1, 'ClassNames',{'normal','abnormal'});
CVMdl = crossval(Md1);
loss = kfoldLoss(CVMdl)
predictedLabels = predict(Md1, testFeatures);
[predictedLabels,~,~,Posterior] = predict(Md1,testFeatures);
[X,Y,T,AUC,OPTROCPT,SUBY,SUBYNAMES] = perfcurve(testLabels, Posterior(:,2), 'abnormal');
figure(1)
plot(X,Y);
xlabel('False positive rate')
ylabel('True positive rate')
title('ROC for Classification CNN')
AUC
  1 Comment
KALYAN ACHARJYA
KALYAN ACHARJYA on 13 Oct 2018
Edited: KALYAN ACHARJYA on 13 Oct 2018
It seems perfectly ideal, are you sure? I never saw any such type of perfect curve in experimentation results.
Though the AUC is not 1, near about 0.999, it may appear in the number of cases. Classification of?
You can check the curve in maths way (little tedious) also, do it and confirm.

Sign in to comment.

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!