RegressionSVM class
Superclasses: CompactRegressionSVM
Support vector machine regression model
Description
RegressionSVM
is a support vector machine (SVM) regression model. Train a RegressionSVM
model using fitrsvm
and the sample data.
RegressionSVM
models store data, parameter values, support vectors, and algorithmic implementation information. You can use these models to:
Estimate resubstitution predictions. For details, see
resubPredict
.Predict values for new data. For details, see
predict
.Compute resubstitution loss. For details, see
resubLoss
.Compute the mean square error or epsiloninsensitive loss. For details, see
loss
.
Construction
Create a RegressionSVM
object by using fitrsvm
.
Properties
Alpha
— Dual problem coefficients
vector of numeric values
Dual problem coefficients, specified as a vector of numeric values. Alpha
contains m elements, where m is the number of
support vectors in the trained SVM regression model. The dual problem introduces two
Lagrange multipliers for each support vector. The values of Alpha
are
the differences between the two estimated Lagrange multipliers for the support vectors.
For more details, see Understanding Support Vector Machine Regression.
If you specified to remove duplicates using RemoveDuplicates
,
then, for a particular set of duplicate observations that are support vectors,
Alpha
contains one coefficient corresponding to the entire set.
That is, MATLAB^{®} attributes a nonzero coefficient to one observation from the set of
duplicates and a coefficient of 0
to all other duplicate observations
in the set.
Data Types: single
 double
Beta
— Primal linear problem coefficients
vector of numeric values  '[]'
Primal linear problem coefficients, stored as a numeric vector of length p, where p is the number of predictors in the SVM regression model.
The values in Beta
are the linear coefficients for the primal optimization problem.
If the model is obtained using a kernel function other than 'linear'
, this property is empty ('[]'
).
The predict
method computes predicted response values for the model as YFIT = (X/S)×Beta + Bias
, where S
is the value of the kernel scale stored in the KernelParameters.Scale
property.
Data Types: single
 double
Bias
— Bias term
scalar value
Bias term in the SVM regression model, stored as a scalar value.
Data Types: single
 double
BoxConstraints
— Box constraints for dual problem coefficients
vector of numeric values
Box constraints for dual problem alpha coefficients, stored as a numeric vector containing n elements, where n is the number of observations in X
(Mdl.NumObservations
).
The absolute value of the dual coefficient Alpha
for observation i cannot exceed BoxConstraints(i)
.
If you specify removing duplicates using 'RemoveDuplicates'
, then for a given set of duplicate observations, MATLAB sums the box constraints, and then attributes the sum to one observation and box constraints of 0
to all other observations in the set.
Data Types: single
 double
CacheInfo
— Caching information
structure
Caching information, stored as a structure with the following fields.
Field  Description 

'Size'  Positive scalar value indicating the cache size (in MB) that the software reserves to store entries of the Gram matrix. Set the cache size by using the 'CacheSize' namevalue pair argument in fitrsvm . 
'Algorithm'  Character vector containing the name of the algorithm used to remove entries from the cache when its capacity is exceeded. Currently, the only available caching algorithm is 'Queue' . You cannot set the caching algorithm. 
Data Types: struct
CategoricalPredictors
— Indices of categorical predictors
vector of positive integers  []
Categorical predictor
indices, specified as a vector of positive integers. CategoricalPredictors
contains index values indicating that the corresponding predictors are categorical. The index
values are between 1 and p
, where p
is the number of
predictors used to train the model. If none of the predictors are categorical, then this
property is empty ([]
).
Data Types: single
 double
ConvergenceInfo
— Convergence information
structure
Convergence information, stored as a structure with the following fields.
Field  Description 

Converged  Logical flag indicating whether the algorithm converged. A value of 1 indicates convergence. 
ReasonForConvergence  Character vector indicating the criterion the software used to detect convergence. 
Gap  Scalar feasibility gap between the dual and primal objective functions. 
GapTolerance  Scalar tolerance for the feasibility gap. You can set this tolerance using the 'GapTolerance' namevalue pair argument in fitrsvm . 
DeltaGradient  Scalar gradient difference between upper and lower violators. 
DeltaGradientTolerance  Scalar tolerance for the gradient difference. You can set this tolerance using the DeltaGradientTolerance namevalue pair argument in fitrsvm . 
LargestKKTViolation  Maximal scalar KarushKuhnTucker (KKT) violation value. 
KKTTolerance  Scalar tolerance for the largest KKT violation. You can set this tolerance using the 'KKTTolerance' namevalue pair argument in fitrsvm . 
History  Structure containing convergence information recorded at periodic intervals during the model training process. This structure contains the following fields:

Objective  Numeric value of the dual objective. 
Data Types: struct
Epsilon
— Half the width of the epsiloninsensitive band
nonnegative scalar value
Half the width of the epsiloninsensitive band, stored as a nonnegative scalar value.
Data Types: single
 double
ExpandedPredictorNames
— Expanded predictor names
cell array of character vectors
Expanded predictor names, stored as a cell array of character vectors.
If the model uses encoding for categorical variables, then ExpandedPredictorNames
includes the names that describe the expanded variables. Otherwise, ExpandedPredictorNames
is the same as PredictorNames
.
Data Types: cell
Gradient
— Gradient values in training data
vector of numeric values
Gradient values in training data, stored as a numeric vector containing 2n elements, where n is the number of observations in the training data.
Element i of Gradient
contains the gradient value for the Alpha
coefficient that corresponds to the upper boundary of the epsiloninsensitive band at observation i at the end of the optimization.
Element i + NumObservations of Gradient
contains the gradient value for the Alpha
coefficient that corresponds to the lower boundary of the epsiloninsensitive band at observation i at the end of the optimization.
Data Types: single
 double
HyperparameterOptimizationResults
— Crossvalidation optimization of hyperparameters
BayesianOptimization
object  table
This property is readonly.
Crossvalidation optimization of hyperparameters, specified as a BayesianOptimization
object or a table of hyperparameters and associated
values. This property is nonempty if the 'OptimizeHyperparameters'
namevalue pair argument is nonempty when you create the model. The value of
HyperparameterOptimizationResults
depends on the setting of the
Optimizer
field in the
HyperparameterOptimizationOptions
structure when you create the
model.
Value of Optimizer Field  Value of HyperparameterOptimizationResults 

'bayesopt' (default)  Object of class BayesianOptimization 
'gridsearch' or 'randomsearch'  Table of hyperparameters used, observed objective function values (crossvalidation loss), and rank of observations from lowest (best) to highest (worst) 
IsSupportVector
— Flag indicating whether observation is support vector
logical vector
Flag indicating whether an observation is a support vector, stored as an nby1 logical vector. n is the number of observations in X
(see NumObservations
). A value of 1
indicates that the corresponding observation in the training data is a support vector.
If you specify removing duplicates using RemoveDuplicates
, then for a given set of duplicate observations that are support vectors, IsSupportVector
flags only one as a support vector.
Data Types: logical
KernelParameters
— Kernel function parameters
structure
Kernel function parameters, stored as a structure with the following fields.
Field  Description 

Function
 Kernel function name (a character vector). 
Scale  Numeric scale factor used to divide predictor values. 
You can specify values for KernelParameters.Function
and KernelParameters.Scale
by using the KernelFunction
and KernelScale
namevalue pair arguments in fitrsvm
, respectively.
Data Types: struct
ModelParameters
— Parameter values
model parameter object
Parameter values used to train the SVM regression model, stored as a model parameter object. Access the properties of ModelParameters
using dot notation. For example, access the value of Epsilon
used to train the model as Mdl.ModelParameters.Epsilon
.
Mu
— Predictor means
vector of numeric values  '[]'
Predictor means, stored as a vector of numeric values.
If the predictors are standardized, then Mu
is a numeric vector of length p, where p is the number of predictors used to train the model. In this case, the predict
method centers predictor matrix X
by subtracting the corresponding element of Mu
from each column.
If the predictors are not standardized, then Mu
is empty ('[]'
).
If the data contains categorical predictors, then Mu
includes elements for the dummy variables for those predictors. The corresponding entries in Mu
are 0 because dummy variables are not centered or scaled.
Data Types: single
 double
NumIterations
— Number of iterations required for convergence
positive integer value
Number of iterations required for the optimization routine to reach convergence, stored as a positive integer value.
To set a limit on the number of iterations, use the 'IterationLimit'
namevalue pair argument of fitrsvm
.
Data Types: single
 double
NumObservations
— Number of observations
positive integer value
Number of observations in the training data, stored as a positive integer value.
Data Types: single
 double
PredictorNames
— Predictor names
cell array of character vectors
Predictor names, stored as a cell array of character vectors containing the name of each predictor in the order they appear in X
. PredictorNames
has a length equal to the number of columns in X
.
Data Types: cell
OutlierFraction
— Expected fraction of outliers
scalar value in the range [0,1]
Expected fraction of outliers in the training set, stored as a scalar value in the range [0,1]. You can specify the expected fraction of outliers using the 'OutlierFraction'
namevalue pair argument in fitrsvm
.
Data Types: double
ResponseName
— Response variable name
character vector
Response variable name, stored as a character vector.
Data Types: char
ResponseTransform
— Response transformation function
'none'
 function handle
Response transformation function, specified as 'none'
or a function handle.
ResponseTransform
describes how the software transforms raw
response values.
For a MATLAB function or a function that you define, enter its function handle. For
example, you can enter Mdl.ResponseTransform =
@function
, where
function
accepts a numeric vector of the
original responses and returns a numeric vector of the same size containing the
transformed responses.
Data Types: char
 function_handle
RowsUsed
— Rows of original training data stored
logical vector  []
Rows of the original training data stored in the model, specified as a
logical vector. This property is empty if all rows are stored in
X
and Y
.
Data Types: logical
ShrinkagePeriod
— Number of iterations between reductions of active set
nonnegative integer value
Number of iterations between reductions of the active set during optimization, stored as a nonnegative integer value.
You can set the shrinkage period by using the 'ShrinkagePeriod'
namevalue pair argument in fitrsvm
.
Data Types: single
 double
Sigma
— Predictor standard deviations
vector of numeric values  '[]'
Predictor standard deviations, stored as a vector of numeric values.
If the predictors are standardized, then Sigma
is a numeric vector of length p, where p is the number of predictors used to train the model. In this case, the predict
method scales the predictor matrix X
by dividing each column by the corresponding element of Sigma
, after centering each element using Mu
.
If the predictors are not standardized, then Sigma
is empty ('[]'
).
If the data contains categorical predictors, Sigma
includes elements for the dummy variables for those predictors. The corresponding entries in Sigma
are 1, because dummy variables are not centered or scaled.
Data Types: single
 double
Solver
— Name of solver algorithm
character vector
Name of the solver algorithm used to solve the optimization problem, stored as a value in this table.
Value  Description 

'SMO'  Sequential Minimal Optimization 
'ISDA'  Iterative Single Data Algorithm 
'L1QP'  L1 softmargin minimization by quadratic programming (requires an Optimization Toolbox™ license). 
SupportVectors
— Support vectors
matrix of numeric values
Support vectors, stored as an mbyp matrix
of numeric values. m is the number of support vectors
(sum(Mdl.IsSupportVector)
), and p is
the number of predictors in X
.
If you specified to remove duplicates using RemoveDuplicates
,
then for a given set of duplicate observations that are support vectors,
SupportVectors
contains one unique support vector.
Data Types: single
 double
W
— Observation weights
vector of numeric values
Observation weights used to train the model, stored as a numeric vector containing NumObservation
number of elements. fitrsvm
normalizes the weights used for training so that they sum to 1.
Data Types: single
 double
X
— Predictor values
matrix of numeric values  table of numeric values
Predictor values used to train the model, stored as a matrix of numeric values if the model is trained on a matrix, or a table if the model is trained on a table. X
has size nbyp, where n is the number of rows and p is the number of predictor variables or columns in the training data.
Data Types: single
 double
 table
Y
— Observed response values
vector of numeric values
Observed response values, stored as a numeric vector containing NumObservations
number of elements.
Data Types: single
 double
Object Functions
compact  Compact support vector machine regression model 
crossval  Crossvalidated support vector machine regression model 
discardSupportVectors  Discard support vectors for linear support vector machine (SVM) regression model 
gather  Gather properties of Statistics and Machine Learning Toolbox object from GPU 
incrementalLearner  Convert support vector machine (SVM) regression model to incremental learner 
lime  Local interpretable modelagnostic explanations (LIME) 
loss  Regression error for support vector machine regression model 
partialDependence  Compute partial dependence 
plotPartialDependence  Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots 
predict  Predict responses using support vector machine regression model 
resubLoss  Resubstitution loss for support vector machine regression model 
resubPredict  Predict resubstitution response of support vector machine regression model 
resume  Resume training support vector machine regression model 
shapley  Shapley values 
Copy Semantics
Value. To learn how value classes affect copy operations, see Copying Objects.
Examples
Train Linear Support Vector Machine Regression Model
This example shows how to train a linear support vector machine (SVM) regression model using sample data stored in matrices.
Load the carsmall
data set.
load carsmall rng default % for reproducibility
Specify Horsepower
and Weight
as the predictor variables (X
) and MPG
as the response variable (Y
).
X = [Horsepower,Weight]; Y = MPG;
Train a linear SVM regression model.
Mdl = fitrsvm(X,Y)
Mdl = RegressionSVM ResponseName: 'Y' CategoricalPredictors: [] ResponseTransform: 'none' Alpha: [75x1 double] Bias: 57.3800 KernelParameters: [1x1 struct] NumObservations: 94 BoxConstraints: [94x1 double] ConvergenceInfo: [1x1 struct] IsSupportVector: [94x1 logical] Solver: 'SMO'
The Command Window shows that Mdl
is a trained RegressionSVM
model and a list of its properties.
Check the model for convergence.
Mdl.ConvergenceInfo.Converged
ans = logical
0
0
indicates that the model did not converge.
MdlStd = fitrsvm(X,Y,'Standardize',true)
MdlStd = RegressionSVM ResponseName: 'Y' CategoricalPredictors: [] ResponseTransform: 'none' Alpha: [77x1 double] Bias: 22.9131 KernelParameters: [1x1 struct] Mu: [109.3441 2.9625e+03] Sigma: [45.3545 805.9668] NumObservations: 94 BoxConstraints: [94x1 double] ConvergenceInfo: [1x1 struct] IsSupportVector: [94x1 logical] Solver: 'SMO'
Check the model for convergence.
MdlStd.ConvergenceInfo.Converged
ans = logical
1
1
indicates that the model did converge.
Compute the resubstitution mean squared error for the new model.
lStd = resubLoss(MdlStd)
lStd = 16.8551
Train Support Vector Machine Regression Model
Train a support vector machine regression model using the abalone data from the UCI Machine Learning Repository.
Download the data and save it in your current folder with the name 'abalone.csv'
.
url = 'https://archive.ics.uci.edu/ml/machinelearningdatabases/abalone/abalone.data'; websave('abalone.csv',url);
Read the data into a table. Specify the variable names.
varnames = {'Sex'; 'Length'; 'Diameter'; 'Height'; 'Whole_weight';... 'Shucked_weight'; 'Viscera_weight'; 'Shell_weight'; 'Rings'}; Tbl = readtable('abalone.csv','Filetype','text','ReadVariableNames',false); Tbl.Properties.VariableNames = varnames;
The sample data contains 4177 observations. All the predictor variables are continuous except for Sex
, which is a categorical variable with possible values 'M'
(for males), 'F'
(for females), and 'I'
(for infants). The goal is to predict the number of rings (stored in Rings
) on the abalone and determine its age using physical measurements.
Train an SVM regression model, using a Gaussian kernel function with an automatic kernel scale. Standardize the data.
rng default % For reproducibility Mdl = fitrsvm(Tbl,'Rings','KernelFunction','gaussian','KernelScale','auto',... 'Standardize',true)
Mdl = RegressionSVM PredictorNames: {'Sex' 'Length' 'Diameter' 'Height' 'Whole_weight' 'Shucked_weight' 'Viscera_weight' 'Shell_weight'} ResponseName: 'Rings' CategoricalPredictors: 1 ResponseTransform: 'none' Alpha: [3635×1 double] Bias: 10.8144 KernelParameters: [1×1 struct] Mu: [0 0 0 0.5240 0.4079 0.1395 0.8287 0.3594 0.1806 0.2388] Sigma: [1 1 1 0.1201 0.0992 0.0418 0.4904 0.2220 0.1096 0.1392] NumObservations: 4177 BoxConstraints: [4177×1 double] ConvergenceInfo: [1×1 struct] IsSupportVector: [4177×1 logical] Solver: 'SMO' Properties, Methods
The Command Window shows that Mdl
is a trained RegressionSVM
model and displays a property list.
Display the properties of Mdl
using dot notation. For example, check to confirm whether the model converged and how many iterations it completed.
conv = Mdl.ConvergenceInfo.Converged
conv = logical
1
iter = Mdl.NumIterations
iter = 2759
The returned results indicate that the model converged after 2759 iterations.
References
[1] Nash, W.J., T. L. Sellers, S. R. Talbot, A. J. Cawthorn, and W. B. Ford. "The Population Biology of Abalone (Haliotis species) in Tasmania. I. Blacklip Abalone (H. rubra) from the North Coast and Islands of Bass Strait." Sea Fisheries Division, Technical Report No. 48, 1994.
[2] Waugh, S. "Extending and Benchmarking CascadeCorrelation: Extensions to the CascadeCorrelation Architecture and Benchmarking of Feedforward Supervised Artificial Neural Networks." University of Tasmania Department of Computer Science thesis, 1995.
[3] Clark, D., Z. Schreter, A. Adams. "A Quantitative Comparison of Dystal and Backpropagation." submitted to the Australian Conference on Neural Networks, 1996.
[4] Lichman, M. UCI Machine Learning Repository, [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
Extended Capabilities
C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.
Usage notes and limitations:
To integrate the prediction of an SVM regression model into Simulink^{®}, you can use the RegressionSVM Predict block in the Statistics and Machine Learning Toolbox™ library or a MATLAB Function block with the
predict
function.When you train an SVM regression model by using
fitrsvm
, the following restrictions apply.The value of the
ResponseTransform
namevalue argument cannot be an anonymous function. For fixedpoint code generation, the value must be'none'
(default).For fixedpoint code generation, the value of the
KernelFunction
namevalue argument must be'gaussian'
,'linear'
, or'polynomial'
.Fixedpoint code generation and code generation with a coder configurer do not support categorical predictors (
logical
,categorical
,char
,string
, orcell
). You cannot use theCategoricalPredictors
namevalue argument. To include categorical predictors in a model, preprocess them by usingdummyvar
before fitting the model.
For more information, see Introduction to Code Generation.
GPU Arrays
Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™. (since R2023a)
Usage notes and limitations:
The following object functions fully support GPU arrays:
The object functions execute on a GPU if any of the following apply:
The model was fitted with GPU arrays.
The predictor data that you pass to the object function is a GPU array.
The response data that you pass to the object function is a GPU array.
For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
Version History
Introduced in R2015bR2023b: Model stores observations with missing predictor values
Starting in R2023b, training observations with missing predictor values are included in the X
, Y
, and W
data properties. The RowsUsed
property indicates the training observations stored in the model, rather than those used for training. Observations with missing predictor values continue to be omitted from the model training process.
In previous releases, the software omitted training observations that contained missing predictor values from the data properties of the model.
R2023a: GPU array support for object functions (requires Parallel Computing Toolbox)
Starting in R2023a, you can fit a RegressionSVM
object on a GPU by
using fitrsvm
. Most
RegressionSVM
object functions now support GPU array input
arguments so that they can execute on a GPU. The object functions that do not
support GPU array inputs are incrementalLearner
, lime
, and
shapley
.
Open Example
You have a modified version of this example. Do you want to open this example with your edits?
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
 América Latina (Español)
 Canada (English)
 United States (English)
Europe
 Belgium (English)
 Denmark (English)
 Deutschland (Deutsch)
 España (Español)
 Finland (English)
 France (Français)
 Ireland (English)
 Italia (Italiano)
 Luxembourg (English)
 Netherlands (English)
 Norway (English)
 Österreich (Deutsch)
 Portugal (English)
 Sweden (English)
 Switzerland
 United Kingdom (English)