MATLAB neural network classification different results

Hello,

i used MATLAB function "patternet" to create 1 layer (10 neurons) neural network classifier to classify data into 3 classes with default attributes (training function, initializatio and ect.). Suppose have matrix - NxM with rows corresponding to observations and columns are classification features. I found that when i use different combination of same features i get different classification results. For example, using matrix(:,[1 2 3]) gives different classification results (and also different weight values of whole network) compared with matrix(:,[1 3 2]).

Can somebody explain why it is so? Is this drawback is fundamentally related to neural networks classification algorithm or some implementation features?

[Gediminas]

Forgot to mention that when i use the same matrix several times with "patternet" (e.g. matrix(:,[1 2 3])) i get identical weight and classification values.

Answer 1

[Greg Heath]

When making multiple designs in a loop, use rng to initialize the random number generator BEFORE the loop. If the training function uses batch learning, the results will be independent of the order of the columns.

[Gediminas]

Thanks for the answer. By saying multiple designs you mean for example when i use cross-validation with the model? I create independent neural network training and testing function and i use it with external cross-validation, i use rng('shuffle') inside my function now, not in the external loops and it works fine, when i will use rng('shuffle') before the external cross-validation will it work the same? Because previously when i use rng('shuffle') and i train/test neural network several times i get unique result first time, and later the results are the same (weights of the network are the same as the first time).

[Image Analyst]

Gediminas's "Answer" moved to here:

I found that the problem is related to initialization of neural network weights. It gives the identical results every time i use same column configuration, but if i use rng('shuffle') each time i train and test network it gives different results even with the same column configuration. So the problem to me is now how to initialize randomly all weights without previous repetition without each time calling rng('shuffle')?

[Greg Heath]

When training in a loop only initialize the RNG once: just before the outer loop

Answer 2

[Greg Heath]

Remember that the state of the RNG changes every time it is called. Now,

 1. Net creation is different for the obsolete functions newfit and newpr which call newff than the corresponding new functions fitnet and patternnet which call feedforwardnet.

 2. Obsolete 
    a. Random weight initialization occurs at net creation 
    b. Random data division occurs at the beginning of training

 3. Current
    a. Weights are no longer assigned at creation
    b. Weights can be assigned before training using configure
.   c. Random data division occurs at the beginning of training
    d. Train will only assign initial weights to a weightless net

Therefore, if you are training multiple nets of the current version in a loop, you have to use configure to initialize weights at the beginning of the loop.

Hope this helps

Greg

Answer 3

[Hamza]

@Gediminas@Star Strider@Image Analyst@Greg Heath

Hello everyone, I'm facing the same issue on CNN. When I shuffled the features, I obtained different accuracy values, which ideally should remain consistent. Do you have any suggestions on how to resolve this issue? I am using Malba 2023.