Neuro-Adaptive Learning and ANFIS
Suppose that you want to apply fuzzy inference to a system for which you already have a collection of input/output data that you would like to use for modeling, model-following, or some similar scenario. Also, assume that you do not necessarily have a predetermined model structure based on the characteristics of variables in your system. In some modeling situations, discerning membership functions parameters by looking at data can be difficult or impossible. In these cases, rather than choosing the parameters associated with a given membership function arbitrarily, you can tailor the membership function parameters to the input/output data. Using Fuzzy Logic Toolbox™ software, you can tune Sugeno fuzzy inference systems using neuro-adaptive learning techniques similar to those used for training neural networks.
Using ANFIS training methods, you can train Sugeno systems with the following properties:
Weighted average defuzzification
First or zeroth order system; that is, all output membership functions must be the same type, either
No rule sharing. Different rules cannot use the same output membership function; that is, the number of output membership functions must equal the number of rules.
Unity weight for each rule.
No custom membership functions or defuzzification methods.
To create such a fuzzy system in the MATLAB® workspace, you can:
genfisfunction. When using this method, you can create your system using either grid partitioning or subtractive clustering. Grid partitioning can produce a large number of rules when the number of inputs reaches four or five. To reduce the number of rules, consider using the subtractive clustering method.
Use the Fuzzy Logic Designer app, and export the FIS to the MATLAB workspace.
Load a system from a file using the
When training your system using the
anfis function, specify
the initial structure by creating an
anfisOptions option set
and setting the
InitialFIS property. If you do not specify this
anfis function derives the FIS structure using
When using Neuro-Fuzzy Designer, in the Generate FIS section, you can create your FIS by:
Loading from a file (select Load from file)
Loading from the MATLAB workspace (select Load from worksp)
Using grid partitioning (select Grid partition)
Using subtractive clustering (select Sub. clustering)
To train a fuzzy system using neuro-adaptive methods, you must collect input/output training data using experiments or simulations of the system you want to model. In general, ANFIS training works well if the training data is fully representative of the features of the data that the trained FIS is intended to model.
To specify your training data, you can:
Create an array in the MATLAB workspace. Each row contains a data point, with the final column containing the output value and the remaining columns containing input values. You can then pass this data to the
trainingDatainput argument of the
anfisfunction or load it into the Neuro-Fuzzy Designer app.
Load the data from a
.datfile. Each line of the file contains a data point with values separated by white space. The final value on each line is the output, and the remaining values are the inputs.
When using the
anfis function, create or load the input data
and pass it to the
trainingData input argument.
When using Neuro-Fuzzy Designer, in the Load data section, select Training, and then:
To load data from a file, select file.
To load data from the MATLAB workspace, select worksp.
anfis and Neuro-Fuzzy Designer allow you to
adjust the optimization method, number of training epochs, and training error goal.
anfis provides additional training options to control
the training step size.
|Option||Neuro-Fuzzy Designer Setting|
|Optimization Method||In the Train FIS section, specify Optim. Method.|
|Number of training epochs||In the Train FIS section, specify Epochs.|
|Training error goal||In the Train FIS section, specify Error Tolerance.|
|Initial step size||Not available|
|Step-size decrease rate|
|Step-size increase rate|
To train a fuzzy system using ANFIS, the Fuzzy Logic Toolbox software uses a back-propagation algorithm either alone or in combination with a least-squares algorithm. This training process tunes the membership function parameters of a FIS such that the system models your input/output data.
The following table shows the two methods that both
and Neuro-Fuzzy Designer use for updating membership function
|Optimization Method||Neuro-Fuzzy Designer Setting|
|Backpropagation for all parameters (a steepest-descent method)||In the Train FIS section, under
Optim. Method, select
|Hybrid method consisting of backpropagation for the parameters associated with the input membership functions, and least squares estimation for the parameters associated with the output membership functions||In the Train FIS section, under
Optim. Method, select
When training using the
anfis function, you can adjust the
training step size options. During training, the software updates the step-size
according to the following rules:
If the error undergoes four consecutive reductions, increase the step size by multiplying it by a constant (
StepSizeIncreaseRate) greater than one.
If the error undergoes two consecutive combinations of one increase and one reduction, decrease the step size by multiplying it by a constant (
StepSizeDecreaseRate) less than one.
Ideally, the step size increases at the start of training, reaches a maximum,
and then decreases for the remainder of the training. To achieve this step size
profile, adjust the initial step size (
step-size increase rate, and step-size decrease rate.
When training using the
anfis function, you can specify what
training progress information to display in the MATLAB Command Window. Using an
anfisOptions option set,
you can set the following display options.
DisplayANFISInformation— Display ANFIS information at the start of training
DisplayErrorValues— Display the training error at each epoch
DisplayStepSize— Display the step-size each time it changes.
DisplayFinalResults— Display the final training error and validation error
Neuro-Fuzzy Designer does not provide user-specified display options. Instead, it displays the training progress as a plot.
Validation data lets you check the generalization capability of your trained fuzzy inference system. The validation data should fully represent the features of the data the FIS is intended to model, while also being sufficiently different from the training data to test training generalization. The software uses this data set to cross-validate the fuzzy inference model by applying the validation data to the model and seeing how well the model responds to this data.
Model validation is useful in the following situations:
Noisy data — In some cases, data is collected using noisy measurements, and the training data is unable to represent all the features of the data the FIS is intended to model.
Overfitting — Since the model structure used for ANFIS is fixed with a large number of parameters, there is a tendency for the model to overfit the data on which it is trained, especially when using a large number of training epochs. If overfitting does occur, the trained FIS may not generalize well to other independent data sets.
The idea behind using a checking data set for model validation is that, after a certain point in the training process, the model begins overfitting the training data set. In principle, the model error for the checking data set decreases up to the point that overfitting begins. After this point, the model error for the checking data increases. Overfitting is accounted for by testing the trained FIS against the checking data, and choosing the membership function parameters to be those associated with the minimum checking error if these errors indicate model overfitting.
Usually, the training and checking data sets are collected based on observations of the target system and are then stored in separate files. To specify validation data when using the:
anfisfunction, create an
anfisOptionsobject, and set the
Neuro-Fuzzy Designer, in the Load data section, select Checking.
The array and file formats for the checking data are the same as those for the training data.
When you train your fuzzy system using the
you can obtain the following trained fuzzy systems:
fisoutput argument is the fuzzy system for which the training error is minimum. This system is always returned by the
anfisfunction, and corresponds to the FIS returned by Neuro-Fuzzy Designer when you do not specify checking data.
chkFISoutput argument is the fuzzy system for which the validation error is minimum. This system is returned only when you specify validation data using
anfisOptions, and corresponds to the FIS returned by Neuro-Fuzzy Designer when you specify checking data. This FIS object is the one that you should use for further calculation if checking data is used for cross-validation.
You can obtain the error associated with each of the trained fuzzy systems. In each case, the returned error is the root mean squared error (RMSE), and is returned as a vector. Each element of the vector is the RMSE error value at each training epoch.
Training error — Difference between the training data output value and the output of the fuzzy inference system for the corresponding training data input values.
Validation error — Difference between the checking data output value and the output of the fuzzy inference system for the corresponding checking data input values. This error is returned only when you specify validation data using
During training, the Neuro-Fuzzy Designer app plots the training and checking error for each training epoch. Exporting training and checking error from the app is not supported. To obtain the training error, you must retrain the system from the command-line. For an example, see Save Training Error Data to MATLAB Workspace.
To further test your trained fuzzy system, you can use an additional set of testing data that you did not use for training or validation. To do so:
When training a system at the command-line, use the
When using Neuro-Fuzzy Designer, in the Load data section, select Testing, and click Load Data. To evaluate the trained system for any loaded data set, in the Test FIS section, select a data set, and click Test Now.
Training Algorithm Differences
The Neuro-Fuzzy Designer app manages training epochs in a manner
different from the
anfis function. This difference produces
variations in the training results.
To train a system for N epochs at the command line, you call
anfis function one time, specifying the number of epochs as
N. However, the Neuro-Fuzzy Designer app calls the
anfis function N times, specifying the
number of epochs as
2 each time.
For a command-line example that demonstrates the Neuro-Fuzzy Designer training algorithm, see Save Training Error Data to MATLAB Workspace.
 Jang, J.-S. R., "Fuzzy Modeling Using Generalized Neural Networks and Kalman Filter Algorithm," Proc. of the Ninth National Conf. on Artificial Intelligence (AAAI-91), pp. 762-767, July 1991.
 Jang, J.-S. R., "ANFIS: Adaptive-Network-based Fuzzy Inference Systems," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 23, No. 3, pp. 665-685, May 1993.
 Jang, J.-S.R., and N. Gulley. ‘Gain Scheduling Based Fuzzy Controller Design’. In NAFIPS/IFIS/NASA ’94. Proceedings of the First International Joint Conference of The North American Fuzzy Information Processing Society Biannual Conference. The Industrial Fuzzy Control and Intelligent Systems Conference, and the NASA Joint Technology Wo, 101–5. San Antonio, TX, USA: IEEE, 1994.
 Jang, J.-S.R. and Chuen-Tsai Sun. ‘Neuro-Fuzzy Modeling and Control’. Proceedings of the IEEE 83, no. 3 (March 1995): 378–406.
 Jang, Jyh-Shing Roger, Chuen-Tsai Sun, and Eiji Mizutani. Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence. MATLAB Curriculum Series. Upper Saddle River, NJ: Prentice Hall, 1997.
 Wang, Li-Xin. Adaptive Fuzzy Systems and Control: Design and Stability Analysis. Englewood Cliffs, NJ: PTR Prentice Hall, 1994.
 Widrow, B. and D. Stearns, Adaptive Signal Processing, Prentice Hall, 1985.