## Neuro-Adaptive Learning and ANFIS

Suppose that you want to apply fuzzy inference to a system for which you already have a collection of input/output data that you would like to use for modeling, model-following, or some similar scenario. Also, assume that you do not necessarily have a predetermined model structure based on the characteristics of variables in your system. In some modeling situations, discerning membership functions parameters by looking at data can be difficult or impossible. In these cases, rather than choosing the parameters associated with a given membership function arbitrarily, you can tailor the membership function parameters to the input/output data. Using Fuzzy Logic Toolbox™ software, you can tune Sugeno fuzzy inference systems using neuro-adaptive learning techniques similar to those used for training neural networks.

Fuzzy Logic Toolbox software provides a command-line function (`anfis`

) and an interactive app (**Neuro-Fuzzy Designer**) for training an adaptive neuro-fuzzy inference system
(ANFIS).

### FIS Structure

Using ANFIS training methods, you can train Sugeno systems with the following properties:

Single output

Weighted average defuzzification

First or zeroth order system; that is, all output membership functions must be the same type, either

`'linear'`

or`'constant'`

.No rule sharing. Different rules cannot use the same output membership function; that is, the number of output membership functions must equal the number of rules.

Unity weight for each rule.

No custom membership functions or defuzzification methods.

To create such a fuzzy system in the MATLAB^{®} workspace, you can:

Use the

`genfis`

function. When using this method, you can create your system using either grid partitioning or subtractive clustering. Grid partitioning can produce a large number of rules when the number of inputs reaches four or five. To reduce the number of rules, consider using the subtractive clustering method.Use the

**Fuzzy Logic Designer**app, and export the FIS to the MATLAB workspace.Use the

`sugfis`

function.Load a system from a file using the

`readfis`

function.

When training your system using the `anfis`

function, specify
the initial structure by creating an `anfisOptions`

option set
and setting the `InitialFIS`

property. If you do not specify this
property, the `anfis`

function derives the FIS structure using
grid partitioning.

When using **Neuro-Fuzzy Designer**, in the **Generate
FIS** section, you can create your FIS by:

Loading from a file (select

**Load from file**)Loading from the MATLAB workspace (select

**Load from worksp**)Using grid partitioning (select

**Grid partition**)Using subtractive clustering (select

**Sub. clustering**)

### Training Data

To train a fuzzy system using neuro-adaptive methods, you must collect input/output training data using experiments or simulations of the system you want to model. In general, ANFIS training works well if the training data is fully representative of the features of the data that the trained FIS is intended to model.

To specify your training data, you can:

Create an array in the MATLAB workspace. Each row contains a data point, with the final column containing the output value and the remaining columns containing input values. You can then pass this data to the

`trainingData`

input argument of the`anfis`

function or load it into the**Neuro-Fuzzy Designer**app.Load the data from a

`.dat`

file. Each line of the file contains a data point with values separated by white space. The final value on each line is the output, and the remaining values are the inputs.

When using the `anfis`

function, create or load the input data
and pass it to the `trainingData`

input argument.

When using **Neuro-Fuzzy Designer**, in the **Load data**
section, select **Training**, and then:

To load data from a file, select

**file**.To load data from the MATLAB workspace, select

**worksp**.

### Training Options

Both `anfis`

and **Neuro-Fuzzy Designer** allow you to
adjust the optimization method, number of training epochs, and training error goal.
However, `anfis`

provides additional training options to control
the training step size.

Option | `anfisOptions` Property | Neuro-Fuzzy Designer Setting |
---|---|---|

Optimization Method | `OptimizationMethod` | In the Train FIS section, specify
Optim. Method. |

Number of training epochs | `EpochNumber` | In the Train FIS section, specify
Epochs. |

Training error goal | `ErrorGoal` | In the Train FIS section, specify
Error Tolerance. |

Initial step size | `InitialStepSize` | Not available |

Step-size decrease rate | `StepSizeDecreaseRate` | |

Step-size increase rate | `StepSizeIncreaseRate` |

#### Optimization Method

To train a fuzzy system using ANFIS, the Fuzzy Logic Toolbox software uses a back-propagation algorithm either alone or in combination with a least-squares algorithm. This training process tunes the membership function parameters of a FIS such that the system models your input/output data.

The following table shows the two methods that both `anfis`

and **Neuro-Fuzzy Designer** use for updating membership function
parameters.

Optimization Method | `anfisOptions` Setting | Neuro-Fuzzy Designer Setting |
---|---|---|

Backpropagation for all parameters (a steepest-descent method) | ```
OprimizationMethod =
'backpropagation'
``` | In the Train FIS section, under
Optim. Method, select
`backpropa` . |

Hybrid method consisting of backpropagation for the parameters associated with the input membership functions, and least squares estimation for the parameters associated with the output membership functions | `OprimizationMethod = 'hybrid'` | In the Train FIS section, under
Optim. Method, select
`hybrid` . |

#### Step Size

When training using the `anfis`

function, you can adjust the
training step size options. During training, the software updates the step-size
according to the following rules:

If the error undergoes four consecutive reductions, increase the step size by multiplying it by a constant (

`StepSizeIncreaseRate`

) greater than one.If the error undergoes two consecutive combinations of one increase and one reduction, decrease the step size by multiplying it by a constant (

`StepSizeDecreaseRate`

) less than one.

Ideally, the step size increases at the start of training, reaches a maximum,
and then decreases for the remainder of the training. To achieve this step size
profile, adjust the initial step size (`InitialStepSize`

),
step-size increase rate, and step-size decrease rate.

### Display Options

When training using the `anfis`

function, you can specify what
training progress information to display in the MATLAB Command Window. Using an `anfisOptions`

option set,
you can set the following display options.

`DisplayANFISInformation`

— Display ANFIS information at the start of training`DisplayErrorValues`

— Display the training error at each epoch`DisplayStepSize`

— Display the step-size each time it changes.`DisplayFinalResults`

— Display the final training error and validation error

**Neuro-Fuzzy Designer** does not provide user-specified display options.
Instead, it displays the training progress as a plot.

### Training Validation

Validation data lets you check the generalization capability of your trained fuzzy inference system. The validation data should fully represent the features of the data the FIS is intended to model, while also being sufficiently different from the training data to test training generalization. The software uses this data set to cross-validate the fuzzy inference model by applying the validation data to the model and seeing how well the model responds to this data.

Model validation is useful in the following situations:

Noisy data — In some cases, data is collected using noisy measurements, and the training data is unable to represent all the features of the data the FIS is intended to model.

Overfitting — Since the model structure used for ANFIS is fixed with a large number of parameters, there is a tendency for the model to overfit the data on which it is trained, especially when using a large number of training epochs. If overfitting does occur, the trained FIS may not generalize well to other independent data sets.

The idea behind using a checking data set for model validation is that, after a certain point in the training process, the model begins overfitting the training data set. In principle, the model error for the checking data set decreases up to the point that overfitting begins. After this point, the model error for the checking data increases. Overfitting is accounted for by testing the trained FIS against the checking data, and choosing the membership function parameters to be those associated with the minimum checking error if these errors indicate model overfitting.

Usually, the training and checking data sets are collected based on observations of the target system and are then stored in separate files. To specify validation data when using the:

`anfis`

function, create an`anfisOptions`

object, and set the`ValidationData`

option.**Neuro-Fuzzy Designer**, in the**Load data**section, select**Checking**.

The array and file formats for the checking data are the same as those for the training data.

### Training Results

When you train your fuzzy system using the `anfis`

function,
you can obtain the following trained fuzzy systems:

The

`fis`

output argument is the fuzzy system for which the training error is minimum. This system is always returned by the`anfis`

function, and corresponds to the FIS returned by**Neuro-Fuzzy Designer**when you do not specify checking data.The

`chkFIS`

output argument is the fuzzy system for which the validation error is minimum. This system is returned only when you specify validation data using`anfisOptions`

, and corresponds to the FIS returned by**Neuro-Fuzzy Designer**when you specify checking data. This FIS object is the one that you should use for further calculation if checking data is used for cross-validation.

You can obtain the error associated with each of the trained fuzzy systems. In each case, the returned error is the root mean squared error (RMSE), and is returned as a vector. Each element of the vector is the RMSE error value at each training epoch.

Training error — Difference between the training data output value and the output of the fuzzy inference system for the corresponding training data input values.

Validation error — Difference between the checking data output value and the output of the fuzzy inference system for the corresponding checking data input values. This error is returned only when you specify validation data using

`anfisOptions`

.

During training, the **Neuro-Fuzzy Designer** app plots the training and
checking error for each training epoch. Exporting training and checking error from
the app is not supported. To obtain the training error, you must retrain the system
from the command-line. For an example, see Save Training Error Data to MATLAB Workspace.

To further test your trained fuzzy system, you can use an additional set of testing data that you did not use for training or validation. To do so:

When training a system at the command-line, use the

`evalfis`

function.When using

**Neuro-Fuzzy Designer**, in the**Load data**section, select**Testing**, and click**Load Data**. To evaluate the trained system for any loaded data set, in the**Test FIS**section, select a data set, and click**Test Now**.

### Training Algorithm Differences

The **Neuro-Fuzzy Designer** app manages training epochs in a manner
different from the `anfis`

function. This difference produces
variations in the training results.

To train a system for *N* epochs at the command line, you call
the `anfis`

function one time, specifying the number of epochs as
*N*. However, the **Neuro-Fuzzy Designer** app calls the
`anfis`

function *N* times, specifying the
number of epochs as `2`

each time.

For a command-line example that demonstrates the **Neuro-Fuzzy Designer**
training algorithm, see Save Training Error Data to MATLAB Workspace.

## References

[1] Jang, J.-S. R., "Fuzzy
Modeling Using Generalized Neural Networks and Kalman Filter Algorithm,"
*Proc. of the Ninth National Conf. on Artificial Intelligence
(AAAI-91)*, pp. 762-767, July 1991.

[2] Jang, J.-S. R., "ANFIS:
Adaptive-Network-based Fuzzy Inference Systems," *IEEE Transactions on
Systems, Man, and Cybernetics*, Vol. 23, No. 3, pp. 665-685, May
1993.

[3] Jang, J.-S.R., and N.
Gulley. ‘Gain Scheduling Based Fuzzy Controller Design’. In *NAFIPS/IFIS/NASA ’94. Proceedings of the First International Joint Conference
of The North American Fuzzy Information Processing Society Biannual Conference.
The Industrial Fuzzy Control and Intelligent Systems Conference, and the NASA
Joint Technology Wo*, 101–5. San Antonio, TX, USA: IEEE,
1994.

[4] Jang, J.-S.R. and
Chuen-Tsai Sun. ‘Neuro-Fuzzy Modeling and Control’. *Proceedings of the IEEE* 83, no. 3 (March 1995):
378–406.

[5] Jang, Jyh-Shing Roger,
Chuen-Tsai Sun, and Eiji Mizutani. *Neuro-Fuzzy and Soft
Computing: A Computational Approach to Learning and Machine
Intelligence*. MATLAB Curriculum Series. Upper Saddle River, NJ:
Prentice Hall, 1997.

[6] Wang, Li-Xin. *Adaptive Fuzzy Systems and Control: Design and Stability
Analysis*. Englewood Cliffs, NJ: PTR Prentice Hall,
1994.

[7] Widrow, B. and D. Stearns,
*Adaptive Signal Processing*, Prentice Hall,
1985.