Fuzzy Logic Toolbox

More on ANFIS and the ANFIS Editor GUI

The command anfis takes at least two and at most six input arguments. The general format is

• [fismat1,trnError,ss,fismat2,chkError] = ...
      anfis(trnData,fismat,trnOpt,dispOpt,chkData,method);
  

where trnOpt (training options), dispOpt (display options), chkData (checking data), and method (training method), are optional. All of the output arguments are also optional. In this section we discuss the arguments and range components of the command line function anfis, as well as the analogous functionality of the ANFIS Editor GUI.

When the ANFIS Editor GUI is invoked using anfisedit, only the training data set must exist prior to implementing anfis.In addition, the step-size will be fixed when the adaptive neuro-fuzzy system is trained using this GUI tool.

Training Data

The training data, trnData, is a required argument to anfis, as well as to the ANFIS Editor GUI. Each row of trnData is a desired input/output pair of the target system to be modeled. Each row starts with an input vector and is followed by an output value. Therefore, the number of rows of trnData is equal to the number of training data pairs, and, since there is only one output, the number of columns of trnData is equal to the number of inputs plus one.

Input FIS Structure

The input FIS structure, fismat, can be obtained either from any of the fuzzy editors: the FIS Editor, the Membership Function Editor, and the Rule Editor from the ANFIS Editor GUI, (which allows an FIS structure to be loaded from the disk or the workspace), or from the command line function, genfis1 (for which you only need to give numbers and types of membership functions). The FIS structure contains both the model structure, (which specifies such items as the number of rules in the FIS, the number of membership functions for each input, etc.), and the parameters, (which specify the shapes of membership functions). There are two methods that anfis learning employs for updating membership function parameters: backpropagation for all parameters (a steepest descent method), and a hybrid method consisting of backpropagation for the parameters associated with the input membership functions, and least squares estimation for the parameters associated with the output membership functions. As a result, the training error decreases, at least locally, throughout the learning process. Therefore, the more the initial membership functions resemble the optimal ones, the easier it will be for the model parameter training to converge. Human expertise about the target system to be modeled may aid in setting up these initial membership function parameters in the FIS structure.

Note that genfis1 produces an FIS structure based on a fixed number of membership functions. This invokes the so-called curse of dimensionality, and causes an explosion of the number of rules when the number of inputs is moderately large, that is, more than four or five. The Fuzzy Logic Toolbox offers a method that will provide for some dimension reduction in the fuzzy inference system: you can generate an FIS structure using the clustering algorithm discussed in Subtractive Clustering. From the ANFIS Editor GUI, this algorithm is selected with a radio button before the FIS is generated. This subtractive clustering method partitions the data into groups called clusters, and generates an FIS with the minimum number rules required to distinguish the fuzzy qualities associated with each of the clusters.

Training Options

The ANFIS Editor GUI tool allows you to choose your desired error tolerance and number of training epochs.

Training option trnOpt for the command line anfis is a vector that specifies the stopping criteria and the step-size adaptation strategy:

• trnOpt(1): number of training epochs, default = 10.
• trnOpt(2): error tolerance, default = 0.
• trnOpt(3): initial step-size, default = 0.01.
• trnOpt(4): step-size decrease rate, default = 0.9.
• trnOpt(5): step-size increase rate, default = 1.1.

If any element of trnOpt is an NaN or missing, then the default value is taken. The training process stops if the designated epoch number is reached or the error goal is achieved, whichever comes first.

Usually we want the step-size profile to be a curve that increases initially, reaches some maximum, and then decreases for the remainder of the training. This ideal step-size profile is achieved by adjusting the initial step-size and the increase and decrease rates (trnOpt(3) - trnOpt(5)). The default values are set up to cover a wide range of learning tasks. For any specific application, you may want to modify these step-size options in order to optimize the training. However, as we mentioned previously, there are no user-specified step-size options for training the adaptive neuro fuzzy inference system generated using the ANFIS Editor GUI.

Display Options

Display options apply only to the command-line function anfis.

For the command line anfis, the display options argument, dispOpt, is a vector of either ones or zeros that specifies what information to display, (print in the MATLAB command line window), before, during, and after the training process. One is used to denote print this option, whereas zero denotes don't print this option:

• dispOpt(1): display ANFIS information, default = 1.
• dispOpt(2): display error (each epoch), default = 1.
• dispOpt(3): display step-size (each epoch), default = 1.
• dispOpt(4): display final results, default = 1.

The default mode displays all available information. If any element of dispOpt is NaN or missing, the default value will be taken.

Method

Both the ANFIS Editor GUI and the command line anfis apply either a backpropagation form of the steepest descent method for membership function parameter estimation, or a combination of backpropagation and the least-squares method to estimate membership function parameters. The choices for this argument are hybrid or backpropagation. These method choices are designated in the command line function, anfis, by 1 and 0, respectively.

Output FIS Structure for Training Data

fismat1 is the output FIS structure corresponding to a minimal training error. This is the FIS structure that you will use to represent the fuzzy system when there is no checking data used for model crossvalidation. This data also represents the FIS structure that is saved by the ANFIS Editor GUI when the checking data option is not used.

When the checking data option is used, the output saved is that associated with the minimum checking error.

Training Error

The training error is the difference between the training data output value, and the output of the fuzzy inference system corresponding to the same training data input value, (the one associated with that training data output value). The training error trnError records the root mean squared error (RMSE) of the training data set at each epoch. fismat1 is the snapshot of the FIS structure when the training error measure is at its minimum. The ANFIS Editor GUI will plot the training error versus epochs curve as the system is trained.

Step-Size

You cannot control the step-size options with the ANFIS Editor GUI. Using the command line anfis, the step-size array ss records the step-size during the training. Plotting ss gives the step-size profile, which serves as a reference for adjusting the initial step-size and the corresponding decrease and increase rates. The step-size (ss) for the command-line function anfis is updated according to the following guidelines:

• If the error undergoes four consecutive reductions, increase the step-size by multiplying it by a constant (ssinc) greater than one.
• If the error undergoes two consecutive combinations of one increase and one reduction, decrease the step-size by multiplying it by a constant (ssdec) less than one.

The default value for the initial step-size is 0.01; the default values for ssinc and ssdec are 1.1 and 0.9, respectively. All the default values can be changed via the training option for the command line anfis.

Checking Data

The checking data, chkData, is used for testing the generalization capability of the fuzzy inference system at each epoch. The checking data has the same format as that of the training data, and its elements are generally distinct from those of the training data.

The checking data is important for learning tasks for which the input number is large, and/or the data itself is noisy. In general we want a fuzzy inference system to track a given input/output data set well. Since the model structure used for anfis is fixed, there is a tendency for the model to overfit the data on which is it trained, especially for a large number of training epochs. If overfitting does occur, we cannot expect the fuzzy inference system to respond well to other independent data sets, especially if they are corrupted by noise. A validation or checking data set can be useful for these situations. This data set is used to crossvalidate the fuzzy inference model. This crossvalidation is accomplished by applying the checking data to the model, and seeing how well the model responds to this data.

When the checking data option is used with anfis, either via the command line, or using the ANFIS Editor GUI, the checking data is applied to the model at each training epoch. When the command line anfis is invoked, the model parameters that correspond to the minimum checking error are returned via the output argument fismat2. The FIS membership function parameters computed using the ANFIS Editor GUI when both training and checking data are loaded are associated with the training epoch that has a minimum checking error.

The use of the minimum checking data error epoch to set the membership function parameters assumes:

• The checking data is similar enough to the training data that the checking data error will decrease as the training begins
• The checking data increases at some point in the training, after which data overfitting has occurred.

As discussed in ANFIS Editor GUI Example 2: Checking Data Doesn't Validate Model, depending on the behavior of the checking data error, the resulting FIS may or may not be the one you should be using.

Output FIS Structure for Checking Data

The output of the command line anfis, fismat2, is the output FIS structure with the minimum checking error. This is the FIS structure that should be used for further calculation if checking data is used for cross validation.

Checking Error

The checking error is the difference between the checking data output value, and the output of the fuzzy inference system corresponding to the same checking data input value, (the one associated with that checking data output value). The checking error chkError records the RMSE for the checking data at each epoch. fismat2 is the snapshot of the FIS structure when the checking error is at its minimum. The ANFIS Editor GUI will plot the checking error vs. epochs curve as the system is trained.

ANFIS from the Command Line

Fuzzy Clustering