What functionality does MATLAB offer for Gradient Boosting that is equivalent to XGBoost?

262 views (last 30 days)
What functionality does MATLAB offer for Gradient Boosting that is equivalent to XGBoost?
XGBoost is a popular machine learning package available for both R and Python.

Accepted Answer

MathWorks Support Team
MathWorks Support Team on 29 Aug 2019
Edited: MathWorks Support Team on 29 Aug 2019
Please note that XGBoost is an efficient implementation for gradient boosting and not a machine learning algorithm. Gradient boosting technique has been supported in MATLAB since R2011a.
Implementations of the gradient boosting technique in MATLAB are:
a) AdaBoostM1, GentleBoost and LogitBoost in ‘fitcensemble’ for classification
b) LSBoost in ‘fitrensemble’ for regression.
MATLAB supports Gradient Boosting for specific forms of loss functions:
a) Mean squared error (MSE) through the 'LSBoost' method.
b) Exponential loss through 'AdaBoostM1' and 'GentleBoost' methods.
c) Binomial deviance through the 'LogitBoost' method.
MATLAB does not provide an inbuilt function to perform gradient boosting for an arbitrary loss function.
Here is a comparison between our software and XGBoost for your reference:
MATLAB:
1. Provides better support for multiclass problems via the 'fitcensemble' and 'fitcecoc' functions.
2. Provides better support for Random Forest via the 'fitrensemble' and 'fitensemble' functions.
3. MATLAB functions also support additional boosting techniques, such as AdaBoostM2 for multiclass problems, RUSBoost for binary and multiclass learning on imbalanced data, and RobustBoost for learning in the presence of label noise.
4. The ‘fitcecoc’ function in MATLAB supports various multiclass-to-binary reduction schemes, while XGBoost supports only one-vs-all.
5. MATLAB also supports categorical predictors and surrogate splits to handle missing values.
XGBoost:
1. It is faster but not by orders of magnitude as prior to MATLAB R2019a*.
2. It includes support for other regression methods such as Cox regression models for survival data.
3. It provides support for boosting an arbitrary loss function supplied by the user.
(*)Until R2019a, the MATLAB implementation of gradient boosted trees was much slower than XGBoost, by about an order of magnitude. With R2019a, we are also growing the trees on binned predictors like XGBoost. You can specify the number of bins by using the 'NumBins' name-value pair argument when you train a classification model using ‘fitctree’, ‘fitcensemble’, and ‘fitcecoc’ or a regression model using ‘fitrtree’ and ‘fitrensemble’.
For additional information, kindly refer to the below links:
fitrensemble:
fitcensemble:
fitcecoc:
XGBoost:
Release Notes:
  1 Comment
Bernhard Suhm
Bernhard Suhm on 28 Dec 2020
Thank you for pointing out this video. We may disagree whether variants in splitting criteria of boosting techniques are sufficient to call them a new machine learning algorithm. MATLAB's gradient boosting supports a few splitting criteria, including RUSboost that handles imbalanced data sets. The similarity score described in the video for xgboost squares the sum of residuals, whereas standard gradient boosting computes sums of squared residuals.

Sign in to comment.

More Answers (0)

Products


Release

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!