What parameters to use to show performance of a classifer

Hi. I want to know what are the parameters which best show the performance of a classifier? My data is divided into 3 classes and I have used KNN and ANN to classify it. Now I want to compare the two classifiers and wanted to know which parameters I could use. I calculated Sensitivity, Specificity, accuracy and precision for KNN but when I was searching for calculation of these parameters for ANN I found that they are best suited for binary (2-class) classification. So, what parameters can show the performance in case of 3 class classifier? And if the above mentioned parameters are only for binary classification then what are the values that classperf gives for KNN? And lastly if Sensitivity, Specificity, accuracy and precision can be used to show the performance of 3 class classifier then how do I calculate them for ANN? Thank you.

[Greg Heath]

4x4 count and percent confusion matrices containing row and column sums are good summaries of performance.

Not sure what else is available given those summaries. Summarizing the results in terms of 3 binary classifiers doesn't light my fire.

Greg

Answer 1

[MHN]

There is not any "best" performance evaluation, specifically for N-class classifier (N>2) . It depends on your problem in hand. Usually some of the parameters are more important than the others based on your problem. For example, you can see the following paper for some performance evaluation parameters when N=3 : Patel, Amit C., and Mia K. Markey. "Comparison of three-class classification performance metrics: a case study in breast cancer CAD." Medical imaging. International Society for Optics and Photonics, 2005.

If you would like to find the confusion matrix and then calculate the parameters, this is an example :

load ionosphere
tc = fitctree(X,Y);
Y_hat = predict(tc,X);
[C,order] = confusionmat(Y, Y_hat);