NARX Recurrent Neural Network

Nonlinear AutoRegressive eXogenous (NARX) Recurrent Neural Network used for solving differential equations

This script implements a simple recurrent neural network (RNN) with a single hidden layer trained using a simplified recurrent architecture that processes sequences step-by-step with immediate weight updates. This script uses a simplified recurrent architecture that processes sequences step-by-step with immediate weight updates, but without true temporal backpropagation. For this reason, the RNN in this script is similar to a nonlinear autoregressive exogenous model (NARX), since it relates the current value of a time series (or sequence) to both the past values of the same series and the current and past values of the driving (exogenous) series, i.e. the externally determined series that influences the series of interest. The RNN approximates the dynamic response in the time domain of a Single Degree of Freedom (SDOF) oscillator with lumped mass, stiffness and damping, after being trained on the response of the same oscillator under a different excitation. The application of RNN in the present code is organized in local functions in such a way so that it can be easily generalized for any differential equation in the time domain (or in any other discretized domain, since in this application time plays the role of an independent variable only).

This script serves as a reference implementation to help users understand how a NARX RNN can be implemented and modify it for more advanced applications. As a generalization to this script, different types of differential equations can be solved using the RNN.

No MATLAB toolbox is required to run this code, which is particularly useful for educational RNN prototypes or if you want fine-grained control over weight updates, learning rate, activation functions, etc. No dependencies on MATLAB’s Deep Learning Toolbox or any other toolboxes exist, therefore it can run on any MATLAB version. It is transparent and easy to extend (ideal for RNN research and learning). The local functions used in this script are listed alphabetically at the end of the main script.

See the file readMe.pdf included in this package for a detailed description.

Note: The results produced by the code may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. For consistency purposes and without loss of generality, the random number generator is appropriately initialized at the beginning of the code. If this option is removed, consider running the example a few times and compare the average outcome.

This code is licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This license allows reusers to distribute, remix, adapt, and build upon the material in any medium or format, even for commercial purposes. It requires that reusers give credit to the creator.