How can I train multiple sequences in neural network using feedforwardnet?

Hello,

I have 5 pairs (input-output) of time series. I know the way to train multiple time series using a dynamic neural network ( http://www.mathworks.com/help/nnet/ug/multiple-sequences-with-dynamic-neural-networks.html ), however I'd like to use a static neural network (feedforwardnet function) instead. How could I do that?

Thanks,

Ghazi

Answer 1

[Greg Heath]

 close all, clear all, clc
 [ X0 T0 ] = simplenarx_dataset;
 X(1,:) = X0(1:end-2); X(2,:) = X0(2:end-1);
 X(3,:) = T0(1:end-2); X(4,:) = T0(2:end-1);
 T      = T0(3:end);
 x= cell2mat(X); t = cell2mat(T);
 vart = var(t,1)

 net = feedforwardnet;
 for i =1:10
    net = configure(net, x, t);
   [ net tr y e ] = train(net,x,t);
   NMSE(i,1) = mse(e)/vart;
 end
  NMSE = NMSE

For your 5 series problem with 4 default 1:2 input and feedback delays,

 size(X,1) = 5*4 = 20   and size(T,1) = 5*1 = 5

Hope this helps.

Thank you for formally accepting my answer

Greg

[Ghazi Binarandi]

The sample code really helps! However, I am still confused with several parts:

1. In 'configure' function (line 10), it changes the number of input and output to 1. But, why do they interpret it that way? Since we have 4xN input matrix and 1xN target matrix.
2. Why did you make a 'for' loop from 1 to 10?
3. By '4 default,' does that the same with '4 elements'?
4. What does 1:2 input mean?
5. Why does the feedback being included in the feedforward network?

[Greg Heath]

% The sample code really helps! However, I am still confused with several parts:

% 1.In 'configure' function (line 10), it changes the number of input and output to 1. But, why do they interpret it that way?

 Because there is one input and 1 output.

% Since we have 4xN input matrix and 1xN target matrix.

 The N input vectors are 4-dimensional

% 2.Why did you make a 'for' loop from 1 to 10?

 To mitigate getting a bad combination of random datadivision AND random 
initial weights

% 3.By '4 default,' does that the same with '4 elements'? % 4.What does 1:2 input mean?

 For your 5 series problem you have 4 input series and 1 target 
series. The default input and feedback delays of narxnet are 
both 1:2. To simulate that with feedforward net, you have to 
increase the input dimension from 4 to 20.

% 5.Why does the feedback being included in the feedforward network?

 Because you wanted to simulate narxnet with feedforwardnet.

[Ghazi Binarandi]

Thanks for the explanations! I just need to confirm one more thing.

So in the 'for' loop that you made, do you basically 'retrain' the network 9 times (so the network is trained 10 times) with the same input and output data set? That is why the NMSE value that you put on the code is getting less as the 'retraining' process continues (or in other words, performance is getting better)?

[Greg Heath]

I do retrain the net 9 times. HOWEVER, in order to get 10 INDEPENDENT designs, I need to use CONFIGURE to reinintialize BOTH the random trn/val/tst data division AND the random weight initialization.

You can keep the current best net through the loop, or save the current RNG state corresponding to the best net for a later redesign.

In general, I use a double loop approach with the outer loop to determine the smallest value of number of hidden nodes, H that will yield the desired goal.

Search both NEWSGROUP and ANSWERS using

 for h = Hmin:dH:Hmax

[Ghazi Binarandi]

Thank you! It really helps.

Answer 2

[Dennis]

Do you mean the "input output and curve fitting" neural network type from nnstart? Since in that network, no delayed information about previous inputs/outputs is used, you can simply add all your datasets in one big matrix:

Example input u and output y (where 1 and 2 are two seperately measured datasets):

u1 = [1 2 3 4 5]         y1 = [1 1 1 2 1]
u2 = [2 2 3 2 3]         y2 = [3 3 3 4 3]

X  = [u1 u2]             T  = [y1 y2]

and train the nn with input X and output T. Be aware that you still need to convert these matrices to a nn dataset with tonndata(). Watch the dimensions of your data, e.g. if it is a vertical vector instead of the shown horizontal vector. If you are not familiar with the training commands, type in nnstart and let the tool help you.

However, if you want to use a nn as mathematical model for a dynamic system (engineering background, machinery, some electrical filter), then you must use the dynamic ones from the nnstart toolbox "dynamic time series". Since this kind of nn reacts to past events, multiple datasets for training cannot be simply put together - it would look like your physical system would "jump" every time a new dataset it coming. Instead, for dynamic nn you must use catsample to make the training algorithm acknowledge that there are multiple, separated datasets. You can see the effect of catsample in the following screenshot:

[Ghazi Binarandi]

I am intending to train a multiple time series using a feedforward neural network. I couldn't put them together into a single big matrix because it is consisted of several separate sequences.