Superclasses: CompactRegressionSVM
Support vector machine regression model
RegressionSVM
is a support vector machine (SVM) regression model.
Train a RegressionSVM
model using fitrsvm
and the sample data.
RegressionSVM
models store data, parameter values, support vectors,
and algorithmic implementation information. You can use these models to:
Estimate resubstitution predictions. For details, see resubPredict
.
Predict values for new data. For details, see predict
.
Compute resubstitution loss. For details, see resubLoss
.
Compute the mean square error or epsiloninsensitive loss. For details, see
loss
.
Create a RegressionSVM
object by using fitrsvm
.
Alpha
— Dual problem coefficientsDual problem coefficients, specified as a vector of numeric values. Alpha
contains m elements, where m is the number of
support vectors in the trained SVM regression model. The dual problem introduces two
Lagrange multipliers for each support vector. The values of Alpha
are
the differences between the two estimated Lagrange multipliers for the support vectors.
For more details, see Understanding Support Vector Machine Regression.
If you specified to remove duplicates using RemoveDuplicates
,
then, for a particular set of duplicate observations that are support vectors,
Alpha
contains one coefficient corresponding to the entire set.
That is, MATLAB^{®} attributes a nonzero coefficient to one observation from the set of
duplicates and a coefficient of 0
to all other duplicate observations
in the set.
Data Types: single
 double
Beta
— Primal linear problem coefficients'[]'
Primal linear problem coefficients, stored as a numeric vector of length p, where p is the number of predictors in the SVM regression model.
The values in Beta
are the linear coefficients for the
primal optimization problem.
If the model is obtained using a kernel function other than
'linear'
, this property is empty
('[]'
).
The predict
method computes predicted
response values for the model as YFIT = (X/S)×Beta +
Bias
, where S
is the value of the kernel
scale stored in the KernelParameters.Scale
property.
Data Types: single
 double
Bias
— Bias termBias term in the SVM regression model, stored as a scalar value.
Data Types: single
 double
BoxConstraints
— Box constraints for dual problem coefficientsBox constraints for dual problem alpha coefficients, stored as a numeric
vector containing n elements, where n
is the number of observations in X
(Mdl.NumObservations
).
The absolute value of the dual coefficient Alpha
for
observation i cannot exceed
BoxConstraints(i)
.
If you specify removing duplicates using 'RemoveDuplicates'
, then for a given set of duplicate
observations, MATLAB sums the box constraints, and then attributes the sum to one
observation and box constraints of 0
to all other
observations in the set.
Data Types: single
 double
CacheInfo
— Caching informationCaching information, stored as a structure with the following fields.
Field  Description 

'Size'  Positive scalar value indicating the cache size (in MB)
that the software reserves to store entries of the Gram
matrix. Set the cache size by using the
'CacheSize' namevalue pair argument
in fitrsvm . 
'Algorithm'  Character vector containing the name of the algorithm
used to remove entries from the cache when its capacity is
exceeded. Currently, the only available caching algorithm is
'Queue' . You cannot set the caching
algorithm. 
Data Types: struct
CategoricalPredictors
— Indices of categorical predictorsCategorical predictor
indices, specified as a vector of positive integers. CategoricalPredictors
contains index values corresponding to the columns of the predictor data that contain
categorical predictors. If none of the predictors are categorical, then this property is empty
([]
).
Data Types: single
 double
ConvergenceInfo
— Convergence informationConvergence information, stored as a structure with the following fields.
Field  Description 

Converged  Logical flag indicating whether the algorithm converged.
A value of 1 indicates
convergence. 
ReasonForConvergence  Character vector indicating the criterion the software used to detect convergence. 
Gap  Scalar feasibility gap between the dual and primal objective functions. 
GapTolerance  Scalar tolerance for the feasibility gap. You can set
this tolerance using the 'GapTolerance'
namevalue pair argument in fitrsvm . 
DeltaGradient  Scalar gradient difference between upper and lower violators. 
DeltaGradientTolerance  Scalar tolerance for the gradient difference. You can set
this tolerance using the
DeltaGradientTolerance namevalue
pair argument in fitrsvm . 
LargestKKTViolation  Maximal scalar KarushKuhnTucker (KKT) violation value. 
KKTTolerance  Scalar tolerance for the largest KKT violation. You can
set this tolerance using the
'KKTTolerance' namevalue pair
argument in fitrsvm . 
History  Structure containing convergence information recorded at periodic intervals during the model training process. This structure contains the following fields:

Objective  Numeric value of the dual objective. 
Data Types: struct
Epsilon
— Half the width of the epsiloninsensitive bandHalf the width of the epsiloninsensitive band, stored as a nonnegative scalar value.
Data Types: single
 double
ExpandedPredictorNames
— Expanded predictor namesExpanded predictor names, stored as a cell array of character vectors.
If the model uses encoding for categorical variables, then
ExpandedPredictorNames
includes the names that
describe the expanded variables. Otherwise,
ExpandedPredictorNames
is the same as
PredictorNames
.
Data Types: cell
Gradient
— Gradient values in training dataGradient values in training data, stored as a numeric vector containing 2n elements, where n is the number of observations in the training data.
Element i of Gradient
contains the
gradient value for the Alpha
coefficient that corresponds
to the upper boundary of the epsiloninsensitive band at observation
i at the end of the optimization.
Element i + NumObservations of
Gradient
contains the gradient value for the
Alpha
coefficient that corresponds to the lower
boundary of the epsiloninsensitive band at observation i
at the end of the optimization.
Data Types: single
 double
HyperparameterOptimizationResults
— Description of crossvalidation optimization of hyperparametersBayesianOptimization
object  tableDescription of the crossvalidation optimization of hyperparameters, specified as a
BayesianOptimization
object or a table of
hyperparameters and associated values. This property is nonempty if the
'OptimizeHyperparameters'
namevalue pair argument is nonempty
when you create the model. The value of
HyperparameterOptimizationResults
depends on the setting of the
Optimizer
field in the
HyperparameterOptimizationOptions
structure when you create the
model, as described in this table.
Value of Optimizer Field  Value of HyperparameterOptimizationResults 

'bayesopt' (default)  Object of class BayesianOptimization 
'gridsearch' or 'randomsearch'  Table of hyperparameters used, observed objective function values (crossvalidation loss), and rank of observations from lowest (best) to highest (worst) 
IsSupportVector
— Flag indicating whether observation is support vectorFlag indicating whether an observation is a support vector, stored as an
nby1 logical vector. n is the
number of observations in X
(see
NumObservations
). A value of 1
indicates that the corresponding observation in the training data is a
support vector.
If you specify removing duplicates using RemoveDuplicates
, then for a given set of duplicate
observations that are support vectors, IsSupportVector
flags only one as a support vector.
Data Types: logical
KernelParameters
— Kernel function parametersKernel function parameters, stored as a structure with the following fields.
Field  Description 

Function
 Kernel function name (a character vector). 
Scale  Numeric scale factor used to divide predictor values. 
You can specify values for KernelParameters.Function
and KernelParameters.Scale
by using the
KernelFunction
and KernelScale
namevalue pair arguments in fitrsvm
, respectively.
Data Types: struct
ModelParameters
— Parameter valuesParameter values used to train the SVM regression model, stored as a model
parameter object. Access the properties of
ModelParameters
using dot notation. For example,
access the value of Epsilon
used to train the model as
Mdl.ModelParameters.Epsilon
.
Mu
— Predictor means'[]'
Predictor means, stored as a vector of numeric values.
If the predictors are standardized, then Mu
is a
numeric vector of length p, where p is
the number of predictors used to train the model. In this case, the predict
method centers predictor
matrix X
by subtracting the corresponding element of
Mu
from each column.
If the predictors are not standardized, then Mu
is
empty ('[]'
).
If the data contains categorical predictors, then Mu
includes elements for the dummy variables for those predictors. The
corresponding entries in Mu
are 0 because dummy variables
are not centered or scaled.
Data Types: single
 double
NumIterations
— Number of iterations required for convergenceNumber of iterations required for the optimization routine to reach convergence, stored as a positive integer value.
To set a limit on the number of iterations, use the
'IterationLimit'
namevalue pair argument of
fitrsvm
.
Data Types: single
 double
NumObservations
— Number of observationsNumber of observations in the training data, stored as a positive integer value.
Data Types: single
 double
PredictorNames
— Predictor namesPredictor names, stored as a cell array of character vectors containing
the name of each predictor in the order they appear in X
.
PredictorNames
has a length equal to the number of
columns in X
.
Data Types: cell
OutlierFraction
— Expected fraction of outliersExpected fraction of outliers in the training set, stored as a scalar
value in the range [0,1]. You can specify the expected fraction of outliers
using the 'OutlierFraction'
namevalue pair argument in
fitrsvm
.
Data Types: double
ResponseName
— Response variable nameResponse variable name, stored as a character vector.
Data Types: char
ResponseTransform
— Response transformation function'none'
 function handleResponse transformation function, specified as 'none'
or a function handle.
ResponseTransform
describes how the software transforms raw
response values.
For a MATLAB function, or a function that you define, enter
its function handle. For example, you can enter Mdl.ResponseTransform
= @function
, where function
accepts
a numeric vector of the original responses and returns a numeric vector
of the same size containing the transformed responses.
Data Types: char
 function_handle
ShrinkagePeriod
— Number of iterations between reductions of active setNumber of iterations between reductions of the active set during optimization, stored as a nonnegative integer value.
You can set the shrinkage period by using the
'ShrinkagePeriod'
namevalue pair argument in
fitrsvm
.
Data Types: single
 double
Sigma
— Predictor standard deviations'[]'
Predictor standard deviations, stored as a vector of numeric values.
If the predictors are standardized, then Sigma
is a
numeric vector of length p, where p is
the number of predictors used to train the model. In this case, the predict
method scales the
predictor matrix X
by dividing each column by the
corresponding element of Sigma
, after centering each
element using Mu
.
If the predictors are not standardized, then Sigma
is
empty ('[]'
).
If the data contains categorical predictors, Sigma
includes elements for the dummy variables for those predictors. The
corresponding entries in Sigma
are 1, because dummy
variables are not centered or scaled.
Data Types: single
 double
Solver
— Name of solver algorithmName of the solver algorithm used to solve the optimization problem, stored as a value in this table.
Value  Description 

'SMO'  Sequential Minimal Optimization 
'ISDA'  Iterative Single Data Algorithm 
'L1QP'  L1 softmargin minimization by quadratic programming (requires an Optimization Toolbox™ license). 
SupportVectors
— Support vectorsSupport vectors, stored as an mbyp matrix
of numeric values. m is the number of support vectors
(sum(Mdl.IsSupportVector)
), and p is
the number of predictors in X
.
If you specified to remove duplicates using RemoveDuplicates
,
then for a given set of duplicate observations that are support vectors,
SupportVectors
contains one unique support vector.
Data Types: single
 double
W
— Observation weightsObservation weights used to train the model, stored as a numeric vector
containing NumObservation
number of elements.
fitrsvm
normalizes the weights used for training so
that they sum to 1.
Data Types: single
 double
X
— Predictor valuesPredictor values used to train the model, stored as a matrix of numeric
values if the model is trained on a matrix, or a table if the model is
trained on a table. X
has size
nbyp, where n
is the number of rows and p is the number of predictor
variables or columns in the training data.
Data Types: single
 double
 table
Y
— Observed response valuesObserved response values, stored as a numeric vector containing
NumObservations
number of elements.
Data Types: single
 double
compact  Compact support vector machine regression model 
crossval  Crossvalidated support vector machine regression model 
resubLoss  Resubstitution loss for support vector machine regression model 
resubPredict  Predict resubstitution response of support vector machine regression model 
resume  Resume training support vector machine regression model 
discardSupportVectors  Discard support vectors 
loss  Regression error for support vector machine regression model 
predict  Predict responses using support vector machine regression model 
Value. To learn how value classes affect copy operations, see Copying Objects (MATLAB).
This example shows how to train a linear support vector machine (SVM) regression model using sample data stored in matrices.
Load the carsmall
data set.
load carsmall rng default % for reproducibility
Specify Horsepower
and Weight
as the predictor variables (X
) and MPG
as the response variable (Y
).
X = [Horsepower,Weight]; Y = MPG;
Train a linear SVM regression model.
Mdl = fitrsvm(X,Y)
Mdl = RegressionSVM ResponseName: 'Y' CategoricalPredictors: [] ResponseTransform: 'none' Alpha: [75x1 double] Bias: 43.2943 KernelParameters: [1x1 struct] NumObservations: 93 BoxConstraints: [93x1 double] ConvergenceInfo: [1x1 struct] IsSupportVector: [93x1 logical] Solver: 'SMO' Properties, Methods
The Command Window shows that Mdl
is a trained RegressionSVM
model and a list of its properties.
Check the model for convergence.
Mdl.ConvergenceInfo.Converged
ans = logical
0
0
indicates that the model did not converge.
MdlStd = fitrsvm(X,Y,'Standardize',true)
MdlStd = RegressionSVM ResponseName: 'Y' CategoricalPredictors: [] ResponseTransform: 'none' Alpha: [77x1 double] Bias: 22.9131 KernelParameters: [1x1 struct] Mu: [109.3441 2.9625e+03] Sigma: [45.3545 805.9668] NumObservations: 93 BoxConstraints: [93x1 double] ConvergenceInfo: [1x1 struct] IsSupportVector: [93x1 logical] Solver: 'SMO' Properties, Methods
Check the model for convergence.
MdlStd.ConvergenceInfo.Converged
ans = logical
1
1
indicates that the model did converge.
Compute the resubstitution mean squared error for the new model.
lStd = resubLoss(MdlStd)
lStd = 17.0256
Train a support vector machine regression model using the abalone data from the UCI Machine Learning Repository.
Download the data and save it in your current folder with the name 'abalone.csv'
.
url = 'https://archive.ics.uci.edu/ml/machinelearningdatabases/abalone/abalone.data'; websave('abalone.csv',url);
Read the data into a table. Specify the variable names.
varnames = {'Sex'; 'Length'; 'Diameter'; 'Height'; 'Whole_weight';... 'Shucked_weight'; 'Viscera_weight'; 'Shell_weight'; 'Rings'}; Tbl = readtable('abalone.csv','Filetype','text','ReadVariableNames',false); Tbl.Properties.VariableNames = varnames;
The sample data contains 4177 observations. All the predictor variables are continuous except for Sex
, which is a categorical variable with possible values 'M'
(for males), 'F'
(for females), and 'I'
(for infants). The goal is to predict the number of rings (stored in Rings
) on the abalone and determine its age using physical measurements.
Train an SVM regression model, using a Gaussian kernel function with an automatic kernel scale. Standardize the data.
rng default % For reproducibility Mdl = fitrsvm(Tbl,'Rings','KernelFunction','gaussian','KernelScale','auto',... 'Standardize',true)
Mdl = RegressionSVM PredictorNames: {1×8 cell} ResponseName: 'Rings' CategoricalPredictors: 1 ResponseTransform: 'none' Alpha: [3635×1 double] Bias: 10.8144 KernelParameters: [1×1 struct] Mu: [1×10 double] Sigma: [1×10 double] NumObservations: 4177 BoxConstraints: [4177×1 double] ConvergenceInfo: [1×1 struct] IsSupportVector: [4177×1 logical] Solver: 'SMO'
The Command Window shows that Mdl
is a trained RegressionSVM
model and displays a property list.
Display the properties of Mdl
using dot notation. For example, check to confirm whether the model converged and how many iterations it completed.
conv = Mdl.ConvergenceInfo.Converged iter = Mdl.NumIterations
conv = logical 1 iter = 2759
The returned results indicate that the model converged after 2759 iterations.
[1] Nash, W.J., T. L. Sellers, S. R. Talbot, A. J. Cawthorn, and W. B. Ford. The Population Biology of Abalone (Haliotis species) in Tasmania. I. Blacklip Abalone (H. rubra) from the North Coast and Islands of Bass Strait, Sea Fisheries Division, Technical Report No. 48, 1994.
[2] Waugh, S. Extending and benchmarking CascadeCorrelation, Ph.D. thesis, Computer Science Department, University of Tasmania, 1995.
[3] Clark, D., Z. Schreter, A. Adams. A Quantitative Comparison of Dystal and Backpropagation, submitted to the Australian Conference on Neural Networks, 1996.
[4] Lichman, M. UCI Machine Learning Repository, [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
Usage notes and limitations:
When you train an SVM regression model by using fitrsvm
, the following restrictions apply.
Code generation does
not support categorical predictors. You cannot supply training data in a table that contains a
logical vector, character array, categorical array, string array, or cell array of character
vectors. Also, you cannot use the 'CategoricalPredictors'
namevalue pair
argument.
To include categorical predictors in a model, preprocess the
categorical predictors by using dummyvar
before fitting the model.
The value of the 'ResponseTransform'
namevalue pair argument cannot be an anonymous function.
For more information, see Introduction to Code Generation.
A modified version of this example exists on your system. Do you want to open this version instead?
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
Select web siteYou can also select a web site from the following list:
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.