Main Content

Each regression function has a specific operation. This section
shows how to use these functions to perform specific types of regressions.
To illustrate use of the functions for various regressions, “typical”
usage is shown with optional arguments kept to a minimum. For a typical
regression, you estimate model parameters and residual covariance
matrices with the `mle`

functions and estimate the
standard errors of model parameters with the `std`

functions.
The regressions “without missing data” essentially ignore
samples with any missing values, and the regressions “with
missing data” ignore samples with every value missing.

Multivariate normal regression, or MVNR, is the “standard” implementation of the regression functions in Financial Toolbox™ software.

Estimate parameters using `mvnrmle`

:

[Parameters, Covariance] = mvnrmle(Data, Design);

Estimate standard errors using `mvnrstd`

:

StdParameters = mvnrstd(Data, Design, Covariance);

Estimate parameters using `ecmmvnrmle`

:

[Parameters, Covariance] = ecmmvnrmle(Data, Design);

Estimate standard errors using `ecmmvnrstd`

:

StdParameters = ecmmvnrstd(Data, Design, Covariance);

Least-squares regression, or LSR, sometimes called ordinary least-squares or multiple linear regression, is the simplest linear regression model. It also enjoys the property that, independent of the underlying distribution, it is a best linear unbiased estimator (BLUE).

Given *m* = `NumSamples`

observations,
the typical least-squares regression model seeks to minimize the objective
function

$$\sum _{k=1}^{m}{\left({Z}_{k}-{H}_{k}b\right)}^{T}\left({Z}_{k}-{H}_{k}b\right)},$$

which, within the maximum likelihood framework of the multivariate normal regression routine
`mvnrmle`

, is equivalent to a
single-iteration estimation of just the parameters to obtain
`Parameters`

with the initial covariance matrix
`Covariance`

held fixed as the identity matrix. In the case of
missing data, however, the internal algorithm to handle missing data requires a
separate routine `ecmlsrmle`

to do least-squares
instead of multivariate normal regression.

Estimate parameters using `mvnrmle`

:

[Parameters, Covariance] = mvnrmle(Data, Design, 1);

Estimate standard errors using `mvnrstd`

:

StdParameters = mvnrstd(Data, Design, Covariance);

Estimate parameters using `ecmlsrmle`

:

[Parameters, Covariance] = ecmlsrmle(Data, Design);

Estimate standard errors using `ecmmvnrstd`

:

StdParameters = ecmmvnrstd(Data, Design, Covariance);

Given* m *= `NUMSAMPLES`

observations,
the typical covariance-weighted least squares, or CWLS, regression
model seeks to minimize the objective function

$$\sum _{k=1}^{m}{\left({Z}_{k}-{H}_{k}b\right)}^{T}{C}_{0}\left({Z}_{k}-{H}_{k}b\right)$$

with fixed covariance *C*_{0}.

In most cases, *C*_{0} is
a diagonal matrix. The inverse matrix $$W={C}_{0}^{-1}$$ has diagonal elements that can
be considered relative “weights” for each series. Thus,
CWLS is a form of weighted least squares with the weights applied
across series.

Estimate parameters using `mvnrmle`

:

[Parameters, Covariance] = mvnrmle(Data, Design, 1, [], [], [], Covar0);

Estimate standard errors using `mvnrstd`

:

StdParameters = mvnrstd(Data, Design, Covariance);

Estimate parameters using `ecmlsrmle`

:

[Parameters, Covariance] = ecmlsrmle(Data, Design, [], [], [], [], Covar0);

Estimate standard errors using `ecmmvnrstd`

:

StdParameters = ecmmvnrstd(Data, Design, Covariance);

An *ad hoc* form of least squares that has surprisingly good properties for
misspecified or nonnormal models is known as feasible generalized least squares, or
FGLS. The basic procedure is to do least-squares regression and then to do
covariance-weighted least-squares regression with the resultant residual covariance
from the first regression.

Estimate parameters using `mvnrmle`

:

[Parameters, Covariance] = mvnrmle(Data, Design, 2, 0, 0);

or (to illustrate the FGLS process explicitly)

[Parameters, Covar0] = mvnrmle(Data, Design, 1); [Parameters, Covariance] = mvnrmle(Data, Design, 1, [], [], [], Covar0);

Estimate standard errors using `mvnrstd`

:

StdParameters = mvnrstd(Data, Design, Covariance);

Estimate parameters using `ecmlsrmle`

:

[Parameters, Covar0] = ecmlsrmle(Data, Design); [Parameters, Covariance] = ecmlsrmle(Data, Design, [], [], [], [], Covar0);

Estimate standard errors using `ecmmvnrstd`

:

StdParameters = ecmmvnrstd(Data, Design, Covariance);

Given a multivariate normal regression model in standard form
with a `Data`

matrix and a `Design`

array,
it is possible to convert the problem into a seemingly unrelated regression
(SUR) problem by a simple transformation of the `Design`

array.
The main idea of SUR is that instead of having a common parameter
vector over all data series, you have a separate parameter vector
associated with each separate series or with distinct groups of series
that, nevertheless, share a common residual covariance. It is this
ability to aggregate and disaggregate series and to perform comparative
tests on each design that is the power of SUR.

To make the transformation, use the function `convert2sur`

, which converts a
standard-form design array into an equivalent design array to do SUR with a
specified mapping of the series into `NUMGROUPS`

groups. The
regression functions are used in the usual manner, but with the SUR design array
instead of the original design array. Instead of having `NUMPARAMS`

elements, the SUR output parameter vector has `NUMGROUPS`

of
stacked parameter estimates, where the first `NUMPARAMS`

elements
of `Parameters`

contain parameter estimates associated with the
first group of series, the next `NUMPARAMS`

elements of
`Parameters`

contain parameter estimates associated with the
second group of series, and so on. If the model has only one series, for example,
`NUMSERIES`

= `1`

, then the SUR design array
is the same as the original design array since SUR requires two or more series to
generate distinct parameter estimates.

Given `NUMPARAMS`

parameters and `NUMGROUPS`

groups
with a parameter vector (`Parameters`

) with ```
NUMGROUPS
* NUMPARAMS
```

elements from any of the regression routines,
the following MATLAB^{®} code fragment shows how to print a table
of SUR parameter estimates with rows that correspond to each parameter
and columns that correspond to each group or series:

fprintf(1,'Seemingly Unrelated Regression Parameter Estimates\n'); fprintf(1,' %7s ',' '); fprintf(1,' Group(%3d) ',1:NumGroups); fprintf(1,'\n'); for i = 1:NumParams fprintf(1,' %7d ',i); ii = i; for j = 1:NumGroups fprintf(1,'%12g ',Param(ii)); ii = ii + NumParams; end fprintf(1,'\n'); end fprintf(1,'\n');

Form a SUR design using `convert2sur`

:

DesignSUR = convert2sur(Design, Group);

Estimate parameters using `mvnrmle`

:

[Parameters, Covariance] = mvnrmle(Data, DesignSUR);

Estimate standard errors using `mvnrstd`

:

StdParameters = mvnrstd(Data, DesignSUR, Covariance);

Form a SUR design using `convert2sur`

:

DesignSUR = convert2sur(Design, Group);

Estimate parameters using `ecmmvnrmle`

:

[Parameters, Covariance] = ecmmvnrmle(Data, DesignSUR);

Estimate standard errors using `ecmmvnrstd`

:

StdParameters = ecmmvnrstd(Data, DesignSUR, Covariance);

Without missing data, you can estimate the mean of your `Data`

with the
function `mean`

and the covariance with the function
`cov`

. Nevertheless, the function `ecmnmle`

does this for you if it
detects an absence of missing values. Otherwise, it uses the ECM algorithm to handle
missing values.

Estimate parameters using `ecmnmle`

:

[Mean, Covariance] = ecmnmle(Data);

Estimate standard errors using `ecmnstd`

:

StdMean = ecmnstd(Data, Mean, Covariance);

`convert2sur`

| `ecmlsrmle`

| `ecmlsrobj`

| `ecmmvnrfish`

| `ecmmvnrfish`

| `ecmmvnrmle`

| `ecmmvnrobj`

| `ecmmvnrstd`

| `ecmmvnrstd`

| `ecmnfish`

| `ecmnhess`

| `ecmninit`

| `ecmnmle`

| `ecmnobj`

| `ecmnstd`

| `mvnrfish`

| `mvnrmle`

| `mvnrobj`

| `mvnrstd`