# Why neural network sometimes performs worse when additional input variables are used?

6 views (last 30 days)
Kamuran Turksoy on 7 Jul 2017
Commented: Greg Heath on 8 Jul 2017
Why neural network sometimes performs worse when additional input variables are used?
I would expect a neural network trained with n+1 input variables not to perform worse than a neural network (exactly same structured) trained with n input variables (the n inputs are the same). If the added input variable has no value then during the training step shouldn't all the weights associated to it be 0 and the overall performance not be worse. Is my expectation wrong? If not, why I am getting worse performance sometimes (even with seeded random generator)?

Greg Heath on 7 Jul 2017
1. You don't say what n is
2. How is the net suppose to know that n inputs is sufficient?
3. All it does is start somewhere amongst a bunch of hills. Now since it is blind and can only feel within it's reach to determine where down is, it's lucky to get anywhere.
4. If you were kidnapped and dumped in a 4-dimensional mountain range do you think you would do better if you were in 3-D??
Hope this helps.
Greg tries to roll down in the steepest direction
##### 2 CommentsShowHide 1 older comment
Greg Heath on 8 Jul 2017
There is a dreaded NN phenomenom called
OVERTRAINING AN OVERFIT NET.
An overfit net is a net that has more unknown parameters than equations used to solve those parameters. The result is that, without other constraints to satisfy, there are extra degrees of freedom that can do what ever they want. As a result, the answer between input data points is somewhat arbitrary.
Simple case: Solve
x1 + 2*x2 + 3*x3 = 6
3*x1 + 2*x2 + x3 = 6
Whatever answer you give, it is not the one I wanted... UNLESS you chose the one with x2 = 0.
Hope I got the idea across
Greg