Uniform class probabilities vs. Empirical class probabilities

4 views (last 30 days)
Hi;
I found on one Matlab example of Uniform class probabilities and Empirical class probabilities.
Empirical class probabilities is calculated as follows:
svmStruct = fitcsvm(X,Y); % X is training data and Y are classes
%%10-fold cross-validation
cvm = crossval(svmStruct);
%%Accuracy on cross-validated data
[yhatcv,S] = kfoldPredict(cvm);
% cross-validated error with empirical class probabilities
empirical_error=mean(Y~=yhatcv)
Uniform class probabilities is calculated as follows:
% cross-validated error with uniform class probabilities
uniform_error=kfoldLoss(cvm)
Could you pleas give me a formal definition of those 2 errors types?

Accepted Answer

Ilya
Ilya on 2 Dec 2015
If you are still looking for an answer, there is only one definition for error. In each case, you form a confusion matrix and then take a weighted sum of off-diagonal elements. This code snippet should explain it:
load ionosphere
prior = [1 3]'/4;
m = fitcsvm(X,Y,'prior',prior,'kfold',5,'stand',1);
Yhat = m.kfoldPredict;
C = confusionmat(Y,Yhat,'order',m.ClassNames)
Coff = C;
Coff(1:3:end) = 0
sum(sum(Coff,2).*prior./sum(C,2))
m.kfoldLoss
  1 Comment
Diver
Diver on 16 May 2016
@Ilya Thank you .. I think in this case I better use kfoldLoss error.

Sign in to comment.

More Answers (0)

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!