Fuzzy Control of Multilayer Backpropagation Neural Network

Implementation of Fuzzy Control of Multilayer Backpropagation Neural Network

Fuzzy Control of Multilayer Backpropagation Neural Network with momentum and any number of input units, hidden layers and output units and any number of neurons in hidden layers.

Fuzzy inference system is used as a solution to speed up convergence of the Multilayer Backpropogation Neural Network with Momentum. A Fuzzy controller is used to tune the learning rate parameter automatically based on a heuristic solution and depending on the shape of the error surface.

The central idea behind the fuzzy control of backpropagation is the implementation of heuristics in the form of fuzzy if then rules. Which is done using the error and change in error as variables and further associating them with classes based on value of the error surface.Several membership functions for the fuzzy controller are implemented which classify the error and change in error accordingly to tune the update to the learning rate parameter for the next iteration leading to fast convergence.

Implementation is based on the concepts presented in the following papers:
[1]. Fuzzy control of backpropagation
Payman Arabshahi, Jai J Choi, RJ Marks, Thomas P Caudell
IEEE International Conference on Fuzzy Systems 1992, Pages 967-972

[2]. Fuzzy parameter adaptation in neural systems
Jai J Choi, Payman Arabshahi, RJ Marks, TP Caudell
International Joint Conference on Neural Networks 1992,Pages 232-238

[3]. Fuzzy parameter adaptation in optimization: Some neural net training examples
Payman Arabshahi, Jai J Choi, Robert J Marks, Thomas P Caudell
IEEE Computational Science and Engineering Vol. 3, Issue 1, Pages 57-65,1996

Make sure that the program converges before evaluating it using the test function which is the same as that of MLBPN.

Can be used as a tutorial example for FIS in Backpropagation for convergence speed up. You can also try this by disabling the update to the learning rate parameter alpha by deltaAlpha by setting DISABLEFIS = 0; At this point it will just become the backpropagation algorithm.

Momentum parameter is also implemented to speed up convergence of the backpropagation algorithm and should be used carefully with FIS or seperately for comparison.

There are a total of four m-files:
FuzzyControl_of_MLBPN_Train.m is used for building and training the multilayer network on a desired input pattern and incorporates the implementation of Fuzzy Inference System for control of MLBPN for fast convergence.
MLBPN_Test.m is used for testing the trained neural network.
DefinePattern2.m is used to provide patterns for training the net.
SGN.m is used internally for sign evaluation.

The code provides you the ability to modify the forward and back propagation stages individually as well as the Fuzzy Control mechanism to allow for fast convergence on complicated training data.