KNN classifier with ROC Analysis

10 views (last 30 days)
Aaronne
Aaronne on 19 Mar 2013
Hi Smart guys,
I wrote following codes to get a plot of ROC for my KNN classifier:
load fisheriris;
features = meas;
featureSelcted = features;
numFeatures = size(meas,1);
%%Define ground truth
groundTruthGroup = species;
%%Construct a KNN classifier
KNNClassifierObject = ClassificationKNN.fit(featureSelcted, groundTruthGroup, 'NumNeighbors', 3, 'Distance', 'euclidean');
% Predict resubstitution response of k-nearest neighbor classifier
[KNNLabel, KNNScore] = resubPredict(KNNClassifierObject);
% Fit probabilities for scores
groundTruthNumericalLable = [ones(50,1); zeros(50,1); -1.*ones(50,1)];
[FPR, TPR, Thr, AUC, OPTROCPT] = perfcurve(groundTruthNumericalLable(:,1), KNNScore(:,1), 1);
Then we can plot the FPR vs TPR to get the ROC curve.
However, the FPR and TPR is different from what I got using my own implementation that the one above will not display all the points, actually, the codes above display only three points on the ROC. The codes I implemented will dispaly 151 points on the ROC as the size of the data is 150.
patternsKNN = [KNNScore(:,1), groundTruthNumericalLable(:,1)];
patternsKNN = sortrows(patternsKNN, -1);
groundTruthPattern = patternsKNN(:,2);
POS = cumsum(groundTruthPattern==1);
TPR = POS/sum(groundTruthPattern==1);
NEG = cumsum(groundTruthPattern==0);
FPR = NEG/sum(groundTruthPattern==0);
FPR = [0; FPR];
TPR = [0; TPR];
May I ask how to tune '`perfcurve`' to let it output all the points for the ROC? Thanks a lot.
A.
  1 Comment
Alessandro
Alessandro on 20 Mar 2013
Edited: Alessandro on 20 Mar 2013
try adding 'xvals','all' [FPR, TPR, Thr, AUC, OPTROCPT] = perfcurve(groundTruthNumericalLable(:,1), KNNScore(:,1), 1,'xvals','all');

Sign in to comment.

Accepted Answer

Ilya
Ilya on 19 Mar 2013
For 3 neighbors, the posterior probability has at most 4 distinct values, namely (0:3)/3. Likely less for the Fisher iris data because the classes are well separated. With 4 distinct score values, you won't see more than 4 points on the ROC curve. Your implementation does not account for such ties.
  2 Comments
Aaronne
Aaronne on 20 Mar 2013
Hi Ilya,
Thanks for your reply. Does that mean my implementation is wrong?
Why we we can't have more than 4 points on the ROC curve if there are 4 distinct score values? I thought the number of points on the ROC curve is defined as size of the data plus one.
A.
Ilya
Ilya on 20 Mar 2013
Yes, it does mean that your implementation is wrong. As I said, you can't have more points on a ROC curve than distinct threshold values. This is actually quite simple - you just need to think about it.

Sign in to comment.

More Answers (0)

Categories

Find more on ROC - AUC in Help Center and File Exchange

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!