**Class: **RegressionLinear

Regression loss for linear regression models

uses
any of the previous syntaxes and additional options specified by one
or more `L`

= loss(___,`Name,Value`

)`Name,Value`

pair arguments. For example,
specify that columns in the predictor data correspond to observations
or specify the regression loss function.

`Mdl`

— Linear regression model`RegressionLinear`

model objectLinear regression model, specified as a `RegressionLinear`

model
object. You can create a `RegressionLinear`

model
object using `fitrlinear`

.

`X`

— Predictor datafull matrix | sparse matrix

Predictor data, specified as an *n*-by-*p* full or sparse matrix. This orientation of `X`

indicates that rows correspond to individual observations, and columns correspond to individual predictor variables.

If you orient your predictor matrix so that observations correspond to columns and specify `'ObservationsIn','columns'`

, then you might experience a significant reduction in computation time.

The length of `Y`

and the number of observations
in `X`

must be equal.

**Data Types: **`single`

| `double`

`Y`

— Response datanumeric vector

Response data, specified as an *n*-dimensional
numeric vector. The length of `Y`

and the number
of observations in `X`

must be equal.

**Data Types: **`single`

| `double`

Specify optional
comma-separated pairs of `Name,Value`

arguments. `Name`

is
the argument name and `Value`

is the corresponding value.
`Name`

must appear inside quotes. You can specify several name and value
pair arguments in any order as
`Name1,Value1,...,NameN,ValueN`

.

`'LossFun'`

— Loss function`'mse'`

(default) | `'epsiloninsensitive'`

| function handleLoss function, specified as the comma-separated pair consisting of
`'LossFun'`

and a built-in loss function name or
function handle.

The following table lists the available loss functions. Specify one using its corresponding value. Also, in the table, $$f\left(x\right)=x\beta +b.$$

*β*is a vector of*p*coefficients.*x*is an observation from*p*predictor variables.*b*is the scalar bias.

Value Description `'epsiloninsensitive'`

Epsilon-insensitive loss: $$\ell \left[y,f\left(x\right)\right]=\mathrm{max}\left[0,\left|y-f\left(x\right)\right|-\epsilon \right]$$ `'mse'`

MSE: $$\ell \left[y,f\left(x\right)\right]={\left[y-f\left(x\right)\right]}^{2}$$ `'epsiloninsensitive'`

is appropriate for SVM learners only.Specify your own function using function handle notation.

Let

*n*be the number of observations in`X`

. Your function must have this signaturewhere:`lossvalue =`

(Y,Yhat,W)`lossfun`

The output argument

`lossvalue`

is a scalar.You choose the function name (

).`lossfun`

`Y`

is an*n*-dimensional vector of observed responses.`loss`

passes the input argument`Y`

in for`Y`

.`Yhat`

is an*n*-dimensional vector of predicted responses, which is similar to the output of`predict`

.`W`

is an*n*-by-1 numeric vector of observation weights.

Specify your function using

`'LossFun',@`

.`lossfun`

**Data Types: **`char`

| `string`

| `function_handle`

`'ObservationsIn'`

— Predictor data observation dimension`'rows'`

(default) | `'columns'`

Predictor data observation dimension, specified as the comma-separated
pair consisting of `'ObservationsIn'`

and `'columns'`

or `'rows'`

.

If you orient your predictor matrix so that observations correspond
to columns and specify `'ObservationsIn','columns'`

,
then you might experience a significant reduction in optimization-execution
time.

`'Weights'`

— Observation weightsnumeric vector of positive values

Observation weights, specified as the comma-separated pair consisting
of `'Weights'`

and a numeric vector of positive values.
If you supply weights, `loss`

computes the weighted
classification loss.

Let * n* be the number of observations
in

`X`

.`numel(Weights)`

must be.`n`

By default,

`Weights`

is`ones(`

.,1)`n`

**Data Types: **`double`

| `single`

`L`

— Regression lossesnumeric scalar | numeric row vector

If `Mdl.FittedLoss`

is `'mse'`

,
then the loss term in the objective function is half of the MSE. `loss`

returns
the MSE by default. Therefore, if you use `loss`

to
check the resubstitution (training) error, then there is a discrepancy
between the MSE and optimization results that `fitrlinear`

returns.

Simulate 10000 observations from this model

$$y={x}_{100}+2{x}_{200}+e.$$

$$X={x}_{1},...,{x}_{1000}$$ is a 10000-by-1000 sparse matrix with 10% nonzero standard normal elements.

*e*is random normal error with mean 0 and standard deviation 0.3.

```
rng(1) % For reproducibility
n = 1e4;
d = 1e3;
nz = 0.1;
X = sprandn(n,d,nz);
Y = X(:,100) + 2*X(:,200) + 0.3*randn(n,1);
```

Train a linear regression model. Reserve 30% of the observations as a holdout sample.

```
CVMdl = fitrlinear(X,Y,'Holdout',0.3);
Mdl = CVMdl.Trained{1}
```

Mdl = RegressionLinear ResponseName: 'Y' ResponseTransform: 'none' Beta: [1000x1 double] Bias: -0.0066 Lambda: 1.4286e-04 Learner: 'svm' Properties, Methods

`CVMdl`

is a `RegressionPartitionedLinear`

model. It contains the property `Trained`

, which is a 1-by-1 cell array holding a `RegressionLinear`

model that the software trained using the training set.

Extract the training and test data from the partition definition.

trainIdx = training(CVMdl.Partition); testIdx = test(CVMdl.Partition);

Estimate the training- and test-sample MSE.

mseTrain = loss(Mdl,X(trainIdx,:),Y(trainIdx))

mseTrain = 0.1496

mseTest = loss(Mdl,X(testIdx,:),Y(testIdx))

mseTest = 0.1798

Because there is one regularization strength in `Mdl`

, `mseTrain`

and `mseTest`

are numeric scalars.

Simulate 10000 observations from this model

$$y={x}_{100}+2{x}_{200}+e.$$

$$X={x}_{1},...,{x}_{1000}$$ is a 10000-by-1000 sparse matrix with 10% nonzero standard normal elements.

*e*is random normal error with mean 0 and standard deviation 0.3.

rng(1) % For reproducibility n = 1e4; d = 1e3; nz = 0.1; X = sprandn(n,d,nz); Y = X(:,100) + 2*X(:,200) + 0.3*randn(n,1); X = X'; % Put observations in columns for faster training

Train a linear regression model. Reserve 30% of the observations as a holdout sample.

CVMdl = fitrlinear(X,Y,'Holdout',0.3,'ObservationsIn','columns'); Mdl = CVMdl.Trained{1}

Mdl = RegressionLinear ResponseName: 'Y' ResponseTransform: 'none' Beta: [1000x1 double] Bias: -0.0066 Lambda: 1.4286e-04 Learner: 'svm' Properties, Methods

`CVMdl`

is a `RegressionPartitionedLinear`

model. It contains the property `Trained`

, which is a 1-by-1 cell array holding a `RegressionLinear`

model that the software trained using the training set.

Extract the training and test data from the partition definition.

trainIdx = training(CVMdl.Partition); testIdx = test(CVMdl.Partition);

Create an anonymous function that measures Huber loss ($$\delta $$ = 1), that is,

$$L=\frac{1}{\sum {w}_{j}}\sum _{j=1}^{n}{w}_{j}{\ell}_{j},$$

where

$$\begin{array}{l}\\ {\ell}_{j}=\{\begin{array}{c}0.5{\underset{}{\overset{\u02c6}{{e}_{j}}}}^{2};\\ \left|\underset{}{\overset{\u02c6}{{e}_{j}}}\right|-0.5;\phantom{\rule{0.2777777777777778em}{0ex}}\phantom{\rule{0.2777777777777778em}{0ex}}\end{array}\begin{array}{c}\phantom{\rule{0.2777777777777778em}{0ex}}\phantom{\rule{0.2777777777777778em}{0ex}}\left|\underset{}{\overset{\u02c6}{{e}_{j}}}\right|\le 1\\ \phantom{\rule{0.2777777777777778em}{0ex}}\phantom{\rule{0.2777777777777778em}{0ex}}\left|\underset{}{\overset{\u02c6}{{e}_{j}}}\right|>1\end{array}.\end{array}$$

$\underset{}{\overset{\u02c6}{{e}_{j}}}$ is the residual for observation *j*. Custom loss functions must be written in a particular form. For rules on writing a custom loss function, see the `'LossFun'`

name-value pair argument.

```
huberloss = @(Y,Yhat,W)sum(W.*((0.5*(abs(Y-Yhat)<=1).*(Y-Yhat).^2) + ...
((abs(Y-Yhat)>1).*abs(Y-Yhat)-0.5)))/sum(W);
```

Estimate the training set and test set regression loss using the Huber loss function.

eTrain = loss(Mdl,X(:,trainIdx),Y(trainIdx),'LossFun',huberloss,... 'ObservationsIn','columns')

eTrain = -0.4186

eTest = loss(Mdl,X(:,testIdx),Y(testIdx),'LossFun',huberloss,... 'ObservationsIn','columns')

eTest = -0.4010

Simulate 10000 observations from this model

$$y={x}_{100}+2{x}_{200}+e.$$

$$X=\{{x}_{1},...,{x}_{1000}\}$$ is a 10000-by-1000 sparse matrix with 10% nonzero standard normal elements.

*e*is random normal error with mean 0 and standard deviation 0.3.

```
rng(1) % For reproducibility
n = 1e4;
d = 1e3;
nz = 0.1;
X = sprandn(n,d,nz);
Y = X(:,100) + 2*X(:,200) + 0.3*randn(n,1);
```

Create a set of 15 logarithmically-spaced regularization strengths from $$1{0}^{-4}$$ through $$1{0}^{-1}$$.

Lambda = logspace(-4,-1,15);

Hold out 30% of the data for testing. Identify the test-sample indices.

```
cvp = cvpartition(numel(Y),'Holdout',0.30);
idxTest = test(cvp);
```

Train a linear regression model using lasso penalties with the strengths in `Lambda`

. Specify the regularization strengths, optimizing the objective function using SpaRSA, and the data partition. To increase execution speed, transpose the predictor data and specify that the observations are in columns.

X = X'; CVMdl = fitrlinear(X,Y,'ObservationsIn','columns','Lambda',Lambda,... 'Solver','sparsa','Regularization','lasso','CVPartition',cvp); Mdl1 = CVMdl.Trained{1}; numel(Mdl1.Lambda)

ans = 15

`Mdl1`

is a `RegressionLinear`

model. Because `Lambda`

is a 15-dimensional vector of regularization strengths, you can think of `Mdl1`

as 15 trained models, one for each regularization strength.

Estimate the test-sample mean squared error for each regularized model.

mse = loss(Mdl1,X(:,idxTest),Y(idxTest),'ObservationsIn','columns');

Higher values of `Lambda`

lead to predictor variable sparsity, which is a good quality of a regression model. Retrain the model using the entire data set and all options used previously, except the data-partition specification. Determine the number of nonzero coefficients per model.

Mdl = fitrlinear(X,Y,'ObservationsIn','columns','Lambda',Lambda,... 'Solver','sparsa','Regularization','lasso'); numNZCoeff = sum(Mdl.Beta~=0);

In the same figure, plot the MSE and frequency of nonzero coefficients for each regularization strength. Plot all variables on the log scale.

figure; [h,hL1,hL2] = plotyy(log10(Lambda),log10(mse),... log10(Lambda),log10(numNZCoeff)); hL1.Marker = 'o'; hL2.Marker = 'o'; ylabel(h(1),'log_{10} MSE') ylabel(h(2),'log_{10} nonzero-coefficient frequency') xlabel('log_{10} Lambda') hold off

Select the index or indices of `Lambda`

that balance minimal classification error and predictor-variable sparsity (for example, `Lambda(11)`

).

idx = 11; MdlFinal = selectModels(Mdl,idx);

`MdlFinal`

is a trained `RegressionLinear`

model object that uses `Lambda(11)`

as a regularization strength.

Calculate with arrays that have more rows than fit in memory.

This function fully supports tall arrays. For more information, see Tall Arrays (MATLAB).

A modified version of this example exists on your system. Do you want to open this version instead?

You clicked a link that corresponds to this MATLAB command:

Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.

Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .

Select web siteYou can also select a web site from the following list:

Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.

- América Latina (Español)
- Canada (English)
- United States (English)

- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)

- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)