MATLAB Answers

ROC curve for the validation set

14 views (last 30 days)
Uerm on 13 Jan 2020
Commented: Silvia on 19 Feb 2020
I am working on a classification problem, where I use 10-fold cross-validation. I have made the code for the ROC curve for the training part of the data. I want to do the same for the validation set. How can I do that? My code is below:
indices = crossvalind('Kfold',trainingData(:,end),10);
for i = 1:10
test = (indices == i);
train = ~test;
% Linear SVM
classificationLinearSVM128 = fitcsvm(...
trainingData(train,end), ...
'KernelFunction', 'linear', ...
'PolynomialOrder', [], ...
'KernelScale', 'auto', ...
'BoxConstraint', 1, ...
'Standardize', true, ...
'ClassNames', [0; 1]);
% Training
[predsLinSVM128train,~] = predict(classificationLinearSVM128,trainingData(train,1:end-1));
targetsLinSVM128train = trainingData(train,end);
[~,scoresLinSVM128] = resubPredict(fitPosterior(classificationLinearSVM128));
[xLinSVM128,yLinSVM128,~,aucLinSVM128] = perfcurve(trainingData(train,end),scoresLinSVM128(:,2),1);
% Validation
[predsLinSVM128test,~] = predict(classificationLinearSVM128,trainingData(test,1:end-1));
targetsLinSVM128test = trainingData(test,end);
[~,scoresLinSVM128test] = resubPredict(fitPosterior(classificationLinearSVM128));
%[xLinSVM128test,yLinSVM128test,~,aucLinSVM128test] = perfcurve(trainingData(test,end),scoresLinSVM128test(:,2),1);
title('Linear SVM, training')
title('Linear SVM, validation')
xlabel('False Positive Rate')
ylabel('True Positive Rate')
title('ROC, Linear SVM')
I have tried the line that I have commented, but it does not work. Can anyone help with this? The positive class is 1.

Answers (1)

Shashank Gupta
Shashank Gupta on 21 Jan 2020
Hi Uerm,
I don’t see any point, why “perfcurve" function should not work. You even used the same function in your training part and it seems to be working there. I am not really sure why it is not working. Also you can verify it by calculating FPR(False positive rate) and TPR(True positive rate) manually or through ROC function, plot the FPR vs TPR and see if it matches with the perfcurve function output.
Also let me know if you find something interesting. It is supposed to work perfectly. If you are getting any error while using the function post that as well.
Silvia on 19 Feb 2020
Hi Uerm,
How did you solve the problem?
Thank you

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!