# How to find probability of classification in boosted tree (AdaboostM2)

8 views (last 30 days)
Sal on 30 Dec 2015
Answered: Ruben Fernandez on 31 Jan 2019
Hello, I am using boosted tree for multi-class classification (which uses fitensemble with AdaboostM2, script generated by classification app). I am getting nearly 92% training accuracy with this settings while bagged tree is giving me nearly 82%. However, I need the probability of each class apart from the final class outcome. When I am using [class,score] = predict(...), what I am getting is NOT the probability (which is inline with the documentation), rather the averaged count among selected trees and therefore, they are not confined within [0,1]. To get the probability, I am currently summing up each row and divide each elements in the corresponding row with the summation. I understand this is not the correct way to get the probability, but I am out of idea here.
I need the probability as the competition requires me to submit probability only. Is there any way to get the class probability for the boosting method?
##### 2 CommentsShowHide 1 older comment
Sal on 30 Dec 2015
Hi Brendan,
Thanks. I know AdaboostM2 does not provide probabilities. But I was still interested to get some way to transform those scores to probabilities as I am getting almost 92% training accuracy with this method. The next best method from both MATLAB and R (using a variety of algorithms and tuning effort) is slightly less than 82%.

Ruben Fernandez on 31 Jan 2019
Any solution for this problem?
Simply summing up each row and divide each elements in the corresponding row with the summation it's statiscally correct to transform scores from a boosted tree to probablities?
Thanks

Ilya on 30 Dec 2015
For AdaBoostM1 you can convert scores into probabilities by assigning string 'doublelogit' to the ScoreTransform property of the ensemble object. For AdaBoostM2 there is no simple transformation.
What you do may be good enough for the competition. If classification accuracy is used to determine the winner, you just need a monotone transformation to [0,1]. If they really want probabilities, isotonic regression has been explored in the literature for mapping scores onto probabilities. I can give you some pointers if you'd like, but the approach would be somewhat involved. First, you would need to compute scores for a dataset with known labels not used for training (using either an independent test set or cross-validation), then you would have to fit isotonic regression on that dataset to find probability for each score and then you would need to fit a non-parametric curve of your choice mapping those scores onto those probabilities.
##### 2 CommentsShowHide 1 older comment
Ilya on 31 Dec 2015
You said "To get the probability, I am currently summing up each row and divide each elements in the corresponding row with the summation." That is a monotone transformation to [0,1].
One weak learner is one tree. Look at the Trained property.