29 views (last 30 days)

Hello,

I have a problem with some outputs from a trained custom neural network. I am using MATLAB 2014 with NN ToolBox ver 8.2.

I have created a simple feedforward NN for classification. I have used some Inputs and Targets, trained the NN, and tried to simulate Outputs given Inputs from the same range. The NN I created gives me incorrect outputs. First it returns outputs only with 0,1 range while the targets are in rage of -6 ... 3 and when I add a process function 'mapminimax' to input and output, the results are wrong: 1. when targets are 0 ... 1, there is an offset of 0.5 such that the outputs are 0.5 and 1 2. when targets are e.g. -6 ... 3, the outputs are around -3 ... 3

I am trying to understand what I am doing wrong.

PS. I have already asked this question in SO and also I provided some more details code: http://stackoverflow.com/questions/36449224/mapminmax-process-function-causes-that-nn-incorrectly-simulates-outputs

The code to test NN

clear all; close all; clc;

net = NNPatRec;

%net.inputs{1}.processFcns = {'mapminmax'};

%net.outputs{2}.processFcns = {'mapminmax'};

Inputs = -10:10;

%Targets = [-6*ones(1,11) 3*ones(1,10)];

Targets = [zeros(1,11) ones(1,10)];

[net,tr] = train(net,Inputs,Targets);

net(-10:10)

The code to create NN:

function net = NNPatternRecognition

net = nntest;

end

function net = nntest

net = network;

net.numInputs = 1;

net.numLayers = 2;

net.biasConnect = [1 1]';

net.inputConnect = [1; 0];

net.layerConnect = [0 0; 1 0];

net.outputConnect = [0 1];

% Inputs

%net.inputs{1}.processFcns = {'mapminmax'};

net.inputWeights{1}.learnFcn = 'learngdm';

% layers 1 (at input)

net.layers{1}.initFcn = 'initnw';

net.layers{1}.netInputFcn = 'netsum';

net.layers{1}.transferFcn = 'tansig';

net.layers{1}.size = 3;

% layers 2 (hidden)

net.layers{2}.initFcn = 'initnw';

net.layers{2}.netInputFcn = 'netsum';

net.layers{2}.transferFcn = 'purelin';

net.layers{2}.size = 1;

% Network functions

net.adaptFcn = 'adaptwb';

net.derivFcn = 'defaultderiv';

net.divideFcn = 'dividerand'; %'divideblock';

net.initFcn = 'initlay';

net.performFcn = 'crossentropy';

net.trainFcn = 'trainscg';

% Outputs

%net.outputs{2}.processFcns = {'mapminmax'};

%net.outputs{2}.exampleOutput = [0 1];

net.trainParam.showWindow = false;

net.trainParam.showCommandLine = true;

end

Greg Heath
on 8 Apr 2016

close all, clear all, clc, plt=0, tic

x = -10:10; N = length(x)

trueind = 1 + [zeros(1,11) ones(1,10)];

t = full(ind2vec(trueind))

plt = plt+1, figure(plt), hold on

plot( x( 1:11), trueind( 1:11) ,'o' )

plot( x(12:21), trueind(12:21),'ro' )

axis([ -11 11 0 3 ])

title('CLASS INDICES')

rng('default')

net = patternnet;

[ net tr y e ] = train( net, x, t );

outind = vec2ind(y)

plot( x( 1:11), outind( 1:11) ,'x' ,'LineWidth',2)

plot( x(12:21), outind(12:21),'rx' ,'LineWidth',2)

err = outind~=trueind;

Nerr = sum(err) % 1

PctErr = 100*Nerr/N % 4.7619

Hope this helps.

For details, remove the semicolon to get

net = net

ALSO, for a trn/val/tst breakdown use

tr = tr

Hope this helps.

Thank you for formally accepting my answer

Greg

Brendan Hamm
on 12 Apr 2016

Sign in to comment.

Brendan Hamm
on 6 Apr 2016

You would likely have better luck if you just started with the patternnet which is meant for NN classification.

What I see that is wrong with your current implementation is you have a linear transferFcn for your second layer. For classification purposes this should really be a softmax function. That is change

net.layers{2}.transferFcn = 'purelin';

to

net.layers{2}.transferFcn = 'softmax';

There are also 2 functions used for the processFcns for the the input and output:

net.outputs{2}.processFcns == {'removeconstantrows', mapminmax};

net.inputs{1}.processFcns == {'removeconstantrows', mapminmax};

Brendan Hamm
on 7 Apr 2016

mapminmax will map the output values to a minimum and maximum value. By default this is set to the range [-1 1]. So, I suppose I missed this part above, we would need to change the range to be [0,1]. This is controlled in the processParams:

p.outputs{2}.processParams{1}.max_range = 0; % params for remove constant rows

p.outputs{2}.processParams{2}.ymin = 0; % Params for mapminmax

p.outputs{2}.processParams{2}.ymax = 1; % Params for mapminmax

Furthermore with a classification problem the output should really be a probability of belonging to each class. For this reason for a binary classification you would expect 2 outputs for every sample, the probability of class1 and the probability of class2.

So, You really want your targets to be a matrix of ones and zeros where 1 indicates belonging to that class (i.e. you know it belongs to class1 so the probability is 1). You can get this result by calling:

dummyvar(Targets+1)';

I use the +1 as dummyvar expects all inputs to be positive integers (or alternatively categorical or strings) So all in all this could have been accomplished with the following:

Inputs = -10:10;

Targets = [ones(1,11) 2*ones(1,10)]; % I'll Increment by 1 here (Class name is irrelevant)

p = patternnet(10); % For classification problems

Targets = dummyvar(Targets)'; % Make this into an indicator matrix

p = train(p,Inputs,Targets); % train the NN

Tpred = p(Inputs) % Verify the results with our training set.

Tpred =

Columns 1 through 3

1.0000 1.0000 1.0000

0.0000 0.0000 0.0000

Columns 4 through 6

1.0000 1.0000 1.0000

0.0000 0.0000 0.0000

Columns 7 through 9

1.0000 1.0000 1.0000

0.0000 0.0000 0.0000

Columns 10 through 12

1.0000 1.0000 0.0000

0.0000 0.0000 1.0000

Columns 13 through 15

0.0000 0.0000 0.0000

1.0000 1.0000 1.0000

Columns 16 through 18

0.0000 0.0000 0.0000

1.0000 1.0000 1.0000

Columns 19 through 21

0.0000 0.0000 0.0000

1.0000 1.0000 1.0000

Brendan Hamm
on 7 Apr 2016

Sign in to comment.

Sign in to answer this question.

Opportunities for recent engineering grads.

Apply Today
## 0 Comments

Sign in to comment.