goodnessOfFit
Goodness of fit between test and reference data for analysis and validation of identified models
Description
goodnessOfFit
returns fit values that represent the error
norm between test and reference data sets. If you want to compare and visualize simulated
model output with measurement data, see also compare
.
returns the goodness of fit between the test data fit
= goodnessOfFit(x
,xref
,cost_func
)x
and the reference
data xref
using the cost function cost_func
.
fit
is a quantitative representation of the closeness of
x
to xref
. To perform multiple test-to-reference
fit comparisons, you can specify x
and xref
as
cell arrays of equal size that contain multiple test and reference data sets. With cell
array inputs, fit
returns an array of fit values.
Examples
Calculate Goodness of Fit Between Estimated and Measured Data
Find the goodness of fit between measured output data and the simulated output of an estimated model.
Obtain the measured output.
load iddata1 z1 yref = z1.y;
z1
is an iddata
object containing measured input-output data. z1.y
is the measured output.
Estimate a second-order transfer function model and simulate the model output y_est
.
sys = tfest(z1,2); y_est = sim(sys,z1(:,[],:));
Calculate the goodness of fit, or error norm, between the measured and estimated outputs. Specify the normalized root mean squared error (NRMSE) as the cost function.
cost_func = 'NRMSE';
y = y_est.y;
fit = goodnessOfFit(y,yref,cost_func)
fit = 0.2943
Alternatively, you can use compare
to calculate the fit. compare
uses the NRMSE cost function, and expresses the fit percentage using the one's complement of the error norm. The fit relationship between compare
and goodnessOfFit
is therefore ${\mathrm{fit}}_{\mathrm{compare}}=\left(1-{\mathrm{fit}}_{\mathrm{gof}}\right)*100$. A compare
result of 100% is equivalent to a goodnessOfFit
result of 0.
Specify an initial condition of zero to match the initial condition that goodnessOfFit
assumes.
opt = compareOptions('InitialCondition','z'); compare(z1,sys,opt);
The fit results are equivalent.
Goodness of Fit for Multiple Data Sets
Find the goodness of fit between measured and estimated outputs for two models.
Obtain the input-output measurements z2
from iddata2
. Copy the measured output into reference output yref
.
load iddata2 z2 yref = z2.y;
Estimate second-order and fourth-order transfer function models using z2
.
sys2 = tfest(z2,2); sys4 = tfest(z2,4);
Simulate both systems to get estimated outputs.
y_sim2 = sim(sys2,z2(:,[],:)); y2 = y_sim2.y; y_sim4 = sim(sys4,z2(:,[],:)); y4 = y_sim4.y;
Create cell arrays from the reference and estimated outputs. The reference data set is the same for both model comparisons, so create identical reference cells.
yrefc = {yref yref}; yc = {y2 y4};
Compute fit
values for the three cost functions.
fit_nrmse = goodnessOfFit(yc,yrefc,'NRMSE')
fit_nrmse = 1×2
0.1429 0.1342
fit_nmse = goodnessOfFit(yc,yrefc,'NMSE')
fit_nmse = 1×2
0.0204 0.0180
fit_mse = goodnessOfFit(yc,yrefc,'MSE')
fit_mse = 1×2
1.0811 0.9539
A fit value of 0 indicates a perfect fit between reference and estimated outputs. The fit value rises as fit goodness decreases. For all three cost functions, the fourth-order model produces a better fit than the second-order model.
Input Arguments
x
— Data to test
matrix (default) | cell array
Data to test, specified as a matrix or cell array.
For a single test data set, specify an N_{s}-by-N matrix, where N_{s} is the number of samples and N is the number of channels. You must specify
cost_fun
as'NRMSE'
or'NMSE'
to use multiple-channel data.For multiple test data sets, specify a cell array of length N_{d}, where N_{d} is the number of test-to-reference pairs and each cell contains one data matrix.
x
must not contain any NaN
or
Inf
values.
xref
— Reference data
matrix (default) | cell array
Reference data with which to compare x
, specified as a matrix
or cell array.
For a single reference data set, specify an N_{s}-by-N matrix, where N_{s} is the number of samples and N is the number of channels.
xref
must be the same size asx
. You must specifycost_fun
as'NRMSE'
or'NMSE'
to use multiple-channel data.For multiple reference data sets, specify a cell array of length N_{d}, where N_{d} is the number of test-to-reference pairs and each cell contains one reference data matrix. As with the individual data matrices, the cell array sizes for
x
andxref
must match. Each ith element offit
corresponds to the pairs of the ith cells ofx
andxref
.
xref
must not contain any NaN
or
Inf
values.
cost_func
— Cost function
'MSE'
| 'NRMSE'
| 'NMSE'
Cost function to determine goodness of fit, specified as one of the following values. In the equations, the value fit applies to a single pairing of test and reference data sets.
Value | Description | Equation | Notes |
---|---|---|---|
'MSE' | Mean squared error |
$$fit=\frac{{\Vert x-xref\Vert}^{2}}{Ns}$$ where N_{s} is the number of samples and ‖ indicates the 2-norm of a vector. | fit is a scalar. |
'NRMSE' | Normalized root mean squared error |
$$fit(i)=\frac{\Vert xref(:,i)-x(:,i)\Vert}{\Vert xref(:,i)-mean(xref(:,i))\Vert}$$ where ‖ indicates the 2-norm of a vector.
| fit is a row vector. |
'NMSE' | Normalized mean squared error |
$$fit(i)=\frac{{\Vert xref(:,i)-x(:,i)\Vert}^{2}}{{\Vert xref(:,i)-mean(xref(:,i))\Vert}^{2}}$$
| fit is a row vector. |
Output Arguments
fit
— Goodness of fit
scalar | row vector | cell array
Goodness of fit between test and reference data pairs, returned as a scalar, a row vector, or a cell array.
For a single test and reference data set pair,
fit
is returned as a scalar or row vector.If
cost_fun
is'MSE'
, thenfit
is a scalar.If
cost_fun
is'NRMSE'
or'NMSE'
, thenfit
is a column vector of length N, where N is the number of channels.
For multiple test and data set and reference pairs, where
x
andxref
are cell arrays of length N_{D},fit
is returned as a vector or a matrix.If
cost_fun
is'MSE'
, thenfit
is a row vector of length N_{D}.If
cost_fun
is'NRMSE'
or'NMSE'
, thenfit
is a matrix of size N-by- N_{d}, where N is the number of channels (data columns) and N_{d} represents the number of test pairs. Each element offit
contains the goodness of fit values for the corresponding test data and reference pair.
Each element of
fit
contains the goodness of fit values for the corresponding test data and reference pair.
Possible values for individual fit elements depend on the selection of
cost_func
.
If
cost_func
is'MSE'
, eachfit
value is a positive scalar that grows with the error between test and reference data. Afit
value of0
indicates a perfect match between test and reference data.If
cost_func
is'NRMSE'
or'NMSE'
,fit
values vary between -Inf
and 1.0
— Perfect fit to reference data (zero error)-
Inf
— Bad fit1
—x
is no better than a straight line at matchingxref
Version History
Introduced in R2012aR2020a: goodnessOfFit
: Fit result represents the error norm for all three cost functions, with a value of zero indicating a perfect fit
Behavior changed in R2020a
goodnessOfFit
now returns the error norm E as
the fit value for all three cost functions (MSE, NRMSE, and NMSE). Previously,
goodnessOfFit
returned the one's complement of the error norm,
1-E, for fit values that used the NRMSE or NMSE cost functions. This
change allows consistent fit-value interpretation across the three cost functions, with the
ideal fit value of zero representing a perfect fit.
Previously computed NRMSE and NMSE fit values are the one's complements of the fit
values computed with the current software. Similarly, the NRMSE fit value is now the one's
complement of the fit used in the percentage value that compare
computes. For example, if the previous
goodnessOfFit
fit value was 0.8, the current fit value is 0.2. A
goodnessOfFit
fit value of 0.2 is equivalent to a
compare
fit percentage of 80%.
Open Example
You have a modified version of this example. Do you want to open this example with your edits?
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)