neural network??

hi Guys! i have no experience with the neuronal toolbox. But i hope that i can solve these problem: My problem sounds not so difficult:

i have different testing resaults. - 1 channel: enginge speed - 2 channel: preassure - 3 channel: temperature trend of the cooling medium

my unknown signal is an other temperature trend. First of all i installed a temperature-senor on the place to get the unknown signal to study the behavior of heating. I tried 7 testing with different load cases. now i have 7 different behaviors of the temperatur trends dependent on the 3 channels.

by the way, in the future i have to deinstall the sensor..... is it possible to solve this problem with the neural network toolbox? i want to create a kind of forecast of the temperature-trend with the help of the testing resaults.

who can help???

[Markus]

my idea is to create a net on one testing case. than i want to train this net with my other testing resaults????

Answer 1

[Greg Heath]

The construction of the net is basic. See the documentation, examples and demos.

The basic problem is that with N = 7 you probably do not have enough training data for an accurate I-H-O = 3-H-1 network. If you use all the defaults of FITNET or NEWFF, H = 10, Ntrn ~ 0.7*N = 5, Nval = 1 and Ntst = 1.

The number of unknown weights is Nw = (I+1)*H+(H+1)*O = 40+11 = 51

However, the number of training equations is only Neq = Ntrn*O = 5 << Nw

Possible solutions which can be combined:

 1. Obtain more data. 
 2. Decrease H
 3. Regularized training with TRAINBR
 4. Simulate new data by adding noise to the current inputs (but not to outputs)
 5. Average the outputs of multiple nets selected from 
     a. Many different weight initializations and data divisions.

6. I suggest you begin with

 a. Standardize all variables to zero-mean/unit-variance
 b. Calculate the 4-variable correlation coefficient matrix 
 c. Plot the 3 outputs vs the input
 d. Ponder the results of c and d
 e. Vary H with Ntrials(=20) weight-initializations for each 
    candidate value of H.

Hope this helps.

Greg

[omar belhaj]

Ntrn ~ 0.7*N

what's mean Ntrn (number of training= number of epochs)!!!!???

and why 0.7 ????

what's diffrent between FITNET and NEWFF???

Vary H with Ntrials

what's mean Ntrails ???? it is number of trials ie number of epochs???

best regards

thank you very much Greg

Answer 2

[Markus]

Hi Greg! do you have any example for your solution? is it possible to use the GUI of this problem, or you think it is besser to create a programm with the editor??

i don't understand: I can train the network for ONE case. The computer build me exacly the "unknown" temperature trend. But if i put in an other case of my testing series, the result doesnt match. i want one network that works and agree vor all 7 testings..........

What do u mean with ponder results of c and d??

greets

Answer 3

[Greg Heath]

Training has to be done with a subset that has the dominant characteristics of all the data. One case is not sufficient. If training to convergence without a validation set you should require Neq >= Nw but desire Neq >> Nw. Otherwise the net is too sensitive to measurement errors and noise resulting in poor performance on nontraining data.

Ponder the results of b and c so that you are somewhat familiar with the relationships between the variables.

If you use Nval = Ntst = 1, there are 7*6 = 42 possible data divisions. For each singleton test set, there are 6 possible singleton validation sets.

For each data division you can try numH (=10?) values for the number of hidden nodes, H.

For each value of H you can design Ntrials (=10?) nets with different random initial conditions.

Since this is a very small data set it shouldn't take long to run 10s of trials. Just be sure that you remember the initial random number seed and keep track of how many random number calls are made before each design is made.

It is not unusual for me to design with a single, but much larger data set, 10 random weight initialization trials for each of 10 different values for H. Search the Newsgroup with

heath close clear newff Ntrials

Hope this helps.

Greg

Answer 4

[Markus]

is it possible that u give me an example for a system with 3 Inputs and 1 output trained on other cases. I can't imagine how it works.

[Greg Heath]

There are a plethora of demos and examples in the documentation, the code is similar for all dimensions.

If you have problems, include code and error messages with your post.

Greg

Answer 5

[Markus]

Hi Greg!

i beef uped my signals. Now i have like 8 Inputs and one output. Now i have this code

x= [known102']; %8 Inputs t=[T_neu102'];% 1 Output

[ I N] = size(x);

[O N] =size(t);

% Naives Konstants Output Model (output = Mittelwert,Durchschnitt(t))

y00 = mean(t)

MSE00 = mse(t-y00) % 0.222 Mittlerer quadratischer Fehler

Nerr00 = numel(find(round(y00)~=t)) % 3 Anzahl der Elemente im Feld oder indizierter Array-Ausdruck

PctErr00 = 100*Nerr00/N % 33.3

% Linear Modell

W = t/[ones(1,N);x]

y0 = W*[ones(1,N);x]

MSE0=mse(t-y0) % 0.0098

R20 = 1-MSE0/MSE00 % 0.96 Want R^2 >= 0.99

Nerr0 = numel(find(round(y00)~=t)) % 3

PctErr0 = 100*Nerr0/N % 33.3

% Neurales Netzd Modell

Neq = N*O % No. Der Gleichungen

%Nw = (I+1)*H+(H+1)*O Anzahl der unbekannten Gewichte

Hub = floor((N-1)*O/(I+O+1)) % =1 Maximun H fuer Neq >= Nw

Ntrials = 10

rand(0) % In case you want to duplicate the run

for j = 1:10 % Validation set stopping allows H > Hub

H = j

Nw = (I+1)*H+(H+1)*O

for i = 1:Ntrials

%net = newff(minmax(x),[H O]); OBSOLETE

net = newff(x,t,H);

net.trainParam.goal = 0.01*MSE00; %(R^2 >= 0.99)

[net tr Y E] = train(net,x,t);

Nepochs(i,j) = tr.epoch(end);

MSE = tr.perf(end);

R2(i,j) = 1-MSE/MSE00;

Nerr = numel(find(round(Y)~=t));

PctErr(i,j) = 100*Nerr/N;

end

end

% Tabulations

format short

Nepochs = Nepochs

R2 = R2

format bank

PctErr = PctErr

S = sim(net, x);

it works very well for these data "102". but now i want to know how i can adapt this net to other datas? u have an idea? is it possible to contact u in ICQ oder something else?

Answer 6

[Greg Heath]

>it works very well for these data "102". but now i want to know how i can adapt this net to other datas?

>u have an idea? is it possible to contact u in >ICQ oder something else?

Very simple:

Depending on version

newoutput = sim(net,newinput); or newoutput = net(newinput);

Hope this helps.

Greg

Answer 7

[Markus]

Hi Greg! No that is not what i mean! That is the same like to simulate it on simulink.

Ok i will try it again: I run the programm with the datasheet 102. 102 Its one out of ten trails. I get this result:

http://imageshack.us/photo/my-images/94/temp102.png/

the yellow line is the meassured line. The pink line is the simulation. The result of this datasheet 102 is ok.

So now the problem! i put for example the dataset 105 in. (105=another set of my trails) than i get this result. i'am dissatisfied with it.

http://imageshack.us/photo/my-images/696/temp105.png/

i need one model for all trails. i need to train or adapt some datasheets together, to cover as much as possible of the behavior. Later i want to put in a unknown datasheet (without the yellow line). My model should be able to estimate the yellow line based of past experience.

u know what i mean?

Markus

[Greg Heath]

All data subsets (trn/val/tst) must be able to be interpreted as random samples from the same probability distribution.

If you overlay scatterplots of all of the data, the training data must be representative of all of the data.

Although Neural Nets tend to be excellent interpolators, they tend to be poor extrapolators.

Therefore you should train on a random sampling of all of the data.

[Greg Heath]

> Ntrn ~ 0.7*N >> what's mean Ntrn (number of training= number of epochs)!!!!??? >> and why 0.7 ????

 The default data division is [Ntrn,Nval,Ntst]= [0.7,0.15,0.15]*N

>what's diffrent between FITNET and NEWFF???

Compare the results of

 help newff   % obsolete
 help fitnet  % current

and

 doc newff
 doc fitnet

Basically

newff is obsolete. It takes (x,t,H) as inputs and automatically assigns initial weights

 - newfit(obsolete) is a special case for regression and curvefitting. It calls newff
 - newpr(obsolete) is a special case for classification and pattern-recognition. It also calls newff

feedforwardnet is current. it takes (H) as input and does not assign initial weights. Weights can be assigned at any time by configure(net,x,t). However, train(net,x,t) will automatically assign initial weights to weightless nets before training.

 -fitnet(current) is a special case for regression and curvefitting. It calls feedforwardnet.
 -patternnet(current) is a special case for classification and pattern-recognition it also calls feedforwardnet

>>Vary H with Ntrials

>what's mean Ntrails ???? it is number of trials ie number of epochs???

The best value for H is, typically, not known AND random initial weights may not result in a good design. Therefore I design Ntrials = 10 designs for each value of H that I try.

 If [I N ] = size(x) and [I O ] = size(t), the number of default training 
equations is Ntrneq = Ntrn*O ~0.7*N*O and the number of unknown weights is Nw = (I+1)*H+(H+1)*O = O+(I+O+1)*H. If Nw >~ 0.5*Ntrneq you should use a validation set or use trainbr.

Before you ask about functions, check with the help and doc commands.

AND, if you need additional examples, search NEWSGROUP and ANSWERS using greg as an additional search word. For example

 greg newff

 greg fitnet

 greg Ntrials

Hope this helps.

Greg

[omar belhaj]

thank you very much Greg for these answers