Output of neural network is offset and scaled.. help!?

6 views (last 30 days)
I am trying to simulate the outputs for a neural network myself for later translation to java so i can run it on a mobile device. For this i generated the following simulation code for a network with two hidden layers and tangent-sigmoid nonlinear function at all layers:
function [ Results ] = sim_net( net, input )
y1 = tansig(net.IW{1} * input + net.b{1});
y2 = tansig(net.LW{2} * y1 + net.b{2});
Results = tansig(net.LW{6} * y2 + net.b{3});
end
The sim_net function is then held up against matlab's own functions using the following code:
clc
clear all
net = feedforwardnet([20 20]);
net.divideParam.trainRatio = 75/100; % Adjust as desired
net.divideParam.valRatio = 15/100; % Adjust as desired
net.divideParam.testRatio = 10/100; % Adjust as desired
net.inputs{1}.processFcns = {}; % no preprocessing
net.outputs{2}.processFcns = {};
net.layers{1}.transferFcn = 'tansig';
net.layers{2}.transferFcn = 'tansig';
net.layers{3}.transferFcn = 'tansig';
% Train and Apply Network
[x,t] = simplefit_dataset;
[net,tr] = train(net,x,t);
for i=1:length(x)
disp(i) = sim_net(net,x(i));
disp2(i) = sim(net,x(i));
end
plot(disp)
hold on
plot(disp2)
legend('our code','matlabs code')
the plot of the two outputs:
however, a quick inspection using the following edit reveals that matlabs results are offset by a factor of 5, and scaled by a factor of 5 also
plot(disp)
hold on
plot((disp2-5)/5+0.1)
legend('our code','matlabs code')
However, matlab's net function shouldn't even be able to give values above 1 when using tansig as the last activation function?
  1 Comment
Greg Heath
Greg Heath on 28 Apr 2015
One hidden layer is sufficient since it is also a universal approximator.
The fewer weights that are used, the more robust the design.

Sign in to comment.

Accepted Answer

Greg Heath
Greg Heath on 28 Apr 2015
Edited: Greg Heath on 28 Apr 2015
The function input should be weights, instead of the net.
Is LW{6} a typo?
One hidden layer is sufficient since it is a universal approximator.
Since the function is smooth with four local extrema, only four hidden nodes are necessary.
The fewer weights that are used, the more robust the design. Also, this will make your java coding much easier.
Please use the notation h to denote the output from a hidden layer.
Your output can have any scale because of the default normalization/denormalization used within train
Code should be faster if you use dummy variables like IW = net.IW{1,1}, b1 = , b2 = , LW = ..
Thank you for formally accepting my answer
Greg

More Answers (1)

Søren Jensen
Søren Jensen on 29 Apr 2015
Thank you for taking the time to look through my code!
first, the training data is just an example. when i export it to java i need to use another network and more complicated data, so it really needs to work with two layers!!
second, net.LW{6} is not a typo (i can see how it would look that way). However, matlab has a weird way of organizing the layer and input weights:
BUT the problem is now solved, and thank you for pointing out matlabs normalization/denormalization. somehow there was a factor in the neural net that i did't reproduce in my sim_net function.
To anyone who might be stuck on this problem later here are the lines that solved it:
[xn,xs] = mapminmax(x);
[tn,ts] = mapminmax(t);
[net,tr] = train(net,xn,tn);
thereby the normalization is not run during training, and i avoid getting the weird scaling factor I don't know how to access in my network.
thanks very much to Greg Heath for his answer!

Categories

Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!