RegressionLinear

R2026b

Linear regression model for high-dimensional data

Description

RegressionLinear is a trained linear model object for regression; the linear model is a support vector machine regression (SVM) or linear regression model. fitrlinear fits a RegressionLinear model by minimizing the objective function using techniques that reduce computation time for high-dimensional data sets (for example, stochastic gradient descent). The regression loss plus the regularization term compose the objective function.

Unlike other regression models, and for economical memory usage, RegressionLinear model objects do not store the training data. However, they do store, for example, the estimated linear model coefficients, estimated coefficients, and the regularization strength.

You can use trained RegressionLinear models to predict responses for new data. For details, see predict.

Creation

Create a RegressionLinear object by using fitrlinear.

Properties

Linear Regression Properties

Epsilon — Half of width of epsilon-insensitive band

Read-only: nonnegative scalar

This property is read-only.

Half of the width of the epsilon insensitive band, returned as a nonnegative scalar.

If Learner is not 'svm', then Epsilon is an empty array ([]).

Data Types: single | double

Lambda — Regularization term strength

Read-only: nonnegative scalar | vector of nonnegative values

This property is read-only.

Regularization term strength, returned as a nonnegative scalar or vector of nonnegative values.

Data Types: double | single

Learner — Linear regression model type

Read-only: 'leastsquares' | 'svm'

This property is read-only.

Linear regression model type, returned as 'leastsquares' or 'svm'.

In this table, $$f\left(x\right)=x\beta +b.$$

• β is a vector of p coefficients.

• x is an observation from p predictor variables.

• b is the scalar bias.

Value	Algorithm	Loss Function	FittedLoss Value
'svm'	Support vector machine regression	Epsilon insensitive: $$\ell \left[y,f\left(x\right)\right]=\max \left[0,\left|y-f\left(x\right)\right|-\epsilon \right]$$	'epsiloninsensitive'
'leastsquares'	Linear regression through ordinary least squares	Mean squared error (MSE): $$\ell \left[y,f\left(x\right)\right]=\frac{1}{2}{\left[y-f\left(x\right)\right]}^2$$	'mse'

Beta — Linear coefficient estimates

Read-only: numeric vector

This property is read-only.

Linear coefficient estimates, returned as a numeric vector with length equal to the number of expanded predictors (see ExpandedPredictorNames).

Data Types: double

Bias — Estimated bias term

Read-only: numeric scalar

This property is read-only.

Estimated bias term or model intercept, returned as a numeric scalar.

Data Types: double

FittedLoss — Loss function used to fit the linear model

Read-only: 'epsiloninsensitive' | 'mse'

This property is read-only.

Loss function used to fit the model, returned as 'epsiloninsensitive' or 'mse'.

Value	Algorithm	Loss Function	Learner Value
'epsiloninsensitive'	Support vector machine regression	Epsilon insensitive: $$\ell \left[y,f\left(x\right)\right]=\max \left[0,\left|y-f\left(x\right)\right|-\epsilon \right]$$	'svm'
'mse'	Linear regression through ordinary least squares	Mean squared error (MSE): $$\ell \left[y,f\left(x\right)\right]=\frac{1}{2}{\left[y-f\left(x\right)\right]}^2$$	'leastsquares'

ModelParameters — Parameters used for training model

Read-only: structure

This property is read-only.

Parameters used for training the RegressionLinear model, returned as a structure.

Access fields of ModelParameters using dot notation. For example, access the relative tolerance on the linear coefficients and the bias term by using Mdl.ModelParameters.BetaTolerance.

Data Types: struct

Regularization — Complexity penalty type

Read-only: 'lasso (L1)' | 'ridge (L2)'

This property is read-only.

Complexity penalty type, returned as 'lasso (L1)' or 'ridge (L2)'.

The software composes the objective function for minimization from the sum of the average loss function (see FittedLoss) and a regularization value from this table.

Value	Description
'lasso (L1)'	Lasso (L1) penalty: $$\lambda {\displaystyle \sum _{j=1}^p\left|{\beta }_j\right|}$$
'ridge (L2)'	Ridge (L2) penalty: $$\frac{\lambda }{2}{\displaystyle \sum _{j=1}^p{\beta }_j^2}$$

λ specifies the regularization term strength (see Lambda).

The software excludes the bias term (β₀) from the regularization penalty.

Data Types: char

Data Properties

CategoricalPredictors — Categorical predictor indices

Read-only: vector of positive integers | []

This property is read-only.

Categorical predictor indices, returned as a vector of positive integers. CategoricalPredictors contains index values indicating that the corresponding predictors are categorical. The index values are between 1 and p, where p is the number of predictors used to train the model. If none of the predictors are categorical, then this property is empty ([]).

Data Types: single | double

PredictorNames — Predictor names

Read-only: cell array of character vectors

This property is read-only.

Predictor names in order of their appearance in the predictor data, returned as a cell array of character vectors. The length of PredictorNames is equal to the number of variables in the training data X or Tbl used as predictor variables.

Data Types: cell

ExpandedPredictorNames — Expanded predictor names

Read-only: cell array of character vectors

This property is read-only.

Expanded predictor names, returned as a cell array of character vectors.

If the model uses encoding for categorical variables, then ExpandedPredictorNames includes the names that describe the expanded variables. Otherwise, ExpandedPredictorNames is the same as PredictorNames.

Data Types: cell

ResponseName — Response variable name

Read-only: character vector

This property is read-only.

Response variable name, returned as a character vector.

Data Types: char

ResponseTransform — Response transformation function

"none" | function handle

Response transformation function, specified as "none" or a function handle. ResponseTransform describes how the software transforms raw response values.

For a MATLAB^® function or a function that you define, enter its function handle. For example, you can enter Mdl.ResponseTransform = @function, where function accepts a numeric vector of the original responses and returns a numeric vector of the same size containing the transformed responses.

Data Types: char | string | function_handle

Object Functions

incrementalLearner	Convert linear regression model to incremental learner
lime	Local interpretable model-agnostic explanations (LIME)
loss	Regression loss for linear regression models
partialDependence	Compute partial dependence
plotPartialDependence	Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots
predict	Predict response of linear regression model
selectModels	Select fitted regularized linear regression models
shapley	Shapley values
update	Update model parameters for code generation

Examples

Train Linear Regression Model

Train a linear regression model using SVM, dual SGD, and ridge regularization.

Simulate 10000 observations from this model

$$y=x_{100}+2x_{200}+e.$$

• $$X=x_1,...,x_{1000}$$ is a 10000-by-1000 sparse matrix with 10% nonzero standard normal elements.

• e is random normal error with mean 0 and standard deviation 0.3.

rng(1) % For reproducibility
n = 1e4;
d = 1e3;
nz = 0.1;
X = sprandn(n,d,nz);
Y = X(:,100) + 2*X(:,200) + 0.3*randn(n,1);

Train a linear regression model. By default, fitrlinear uses support vector machines with a ridge penalty, and optimizes using dual SGD for SVM. Determine how well the optimization algorithm fit the model to the data by extracting a fit summary.

[Mdl,FitInfo] = fitrlinear(X,Y)

Mdl = 
  RegressionLinear
         ResponseName: 'Y'
    ResponseTransform: 'none'
                 Beta: [1000×1 double]
                 Bias: -0.0056
               Lambda: 1.0000e-04
              Learner: 'svm'


  Properties, Methods

FitInfo = struct with fields:
                    Lambda: 1.0000e-04
                 Objective: 0.2725
                 PassLimit: 10
                 NumPasses: 10
                BatchLimit: []
             NumIterations: 100000
              GradientNorm: NaN
         GradientTolerance: 0
      RelativeChangeInBeta: 0.4907
             BetaTolerance: 1.0000e-04
             DeltaGradient: 1.5816
    DeltaGradientTolerance: 0.1000
           TerminationCode: 0
         TerminationStatus: {'Iteration limit exceeded.'}
                     Alpha: [10000×1 double]
                   History: []
                   FitTime: 0.0626
                    Solver: {'dual'}

Mdl is a RegressionLinear model. You can pass Mdl and the training or new data to loss to inspect the in-sample mean-squared error. Or, you can pass Mdl and new predictor data to predict to predict responses for new observations.

FitInfo is a structure array containing, among other things, the termination status (TerminationStatus) and how long the solver took to fit the model to the data (FitTime). It is good practice to use FitInfo to determine whether optimization-termination measurements are satisfactory. In this case, fitrlinear reached the maximum number of iterations. Because training time is fast, you can retrain the model, but increase the number of passes through the data. Or, try another solver, such as LBFGS.

Predict Responses Using Linear Regression Model

Create a linear regression model. Use the model to predict responses for test set observations.

Simulate 10000 observations from this model

$$y=x_{100}+2x_{200}+e.$$

• $$X=\{x_1,...,x_{1000}\}$$ is a 10000-by-1000 sparse matrix with 10% nonzero standard normal elements.

• e is random normal error with mean 0 and standard deviation 0.3.

rng(1) % For reproducibility
n = 1e4;
d = 1e3;
nz = 0.1;
X = sprandn(n,d,nz);
Y = X(:,100) + 2*X(:,200) + 0.3*randn(n,1);

Reserve 5% of the data for testing

rng(1) % For reproducibility
cvp = cvpartition(n,Holdout=0.05)

cvp = 
Hold-out cross validation partition
    NumObservations: 10000
        NumTestSets: 1
          TrainSize: 9500
           TestSize: 500
           IsCustom: 0
          IsGrouped: 0
       IsStratified: 0


  Properties, Methods

cvp is a CVPartition object that defines the random partition of n observations into training and test sets.

Train a linear regression model using the training set. For faster training time, orient the predictor data matrix so that the observations are in columns.

idxTrain = training(cvp);
X = X';
Mdl = fitrlinear(X(:,idxTrain),Y(idxTrain), ...
    ObservationsIn="columns");

Predict responses for the test set observations, and compute the mean squared error (MSE).

idxTest = test(cvp);
yHat = predict(Mdl,X(:,idxTest),ObservationsIn="columns");
L = loss(Mdl,X(:,idxTest),Y(idxTest),ObservationsIn="columns")

L = 
0.1851

The test set MSE is approximately 0.19.

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

• The predict and update functions support code generation.

• When you train a linear regression model by using fitrlinear, the following restrictions apply.

  • If the predictor data input argument value is a matrix, it must be a full, numeric matrix. Code generation does not support sparse data.

  • You can specify only one regularization strength, either "auto" or a nonnegative scalar for the Lambda name-value argument.

  • The value of the ResponseTransform name-value argument cannot be an anonymous function.

  • Code generation with a coder configurer does not support categorical predictors (logical, categorical, char, string, or cell). You cannot use the CategoricalPredictors name-value argument. To include categorical predictors in a model, preprocess them by using dummyvar before fitting the model.

For more information, see Introduction to Code Generation for Statistics and Machine Learning Functions.

GPU Code Generation

Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Refer to the usage notes and limitations in the C/C++ Code Generation section. The same usage notes and limitations apply to GPU code generation.

GPU Arrays

Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.

• The following object functions fully support GPU arrays:

  • loss

  • selectModels

  • partialDependence

  • plotPartialDependence

  • predict

• The object functions execute on a GPU if at least one of the following applies:

  • The model was fitted with GPU arrays.

  • The predictor data that you pass to the object function is a GPU array.

  • The response data that you pass to the object function is a GPU array.

• The shapley object function partially supports GPU arrays. For more information, see GPU Arrays.

Version History

Introduced in R2016a

R2026a: Specify GPU arrays when computing Shapley values using the Linear SHAP algorithm (requires Parallel Computing Toolbox)

The shapley object function accepts GPU array input arguments when the machine learning model is a RegressionLinear object, and the shapley function uses an interventional algorithm (Method="interventional").

R2024a: Specify GPU arrays for RegressionLinear object functions (requires Parallel Computing Toolbox)

You can fit a RegressionLinear object with GPU arrays by using fitrlinear. Most RegressionLinear object functions now support GPU array input arguments so that they can execute on a GPU. The object functions that do not support GPU array inputs are incrementalLearner, lime, shapley, and update.