The effect of rng() on neural network training result

I use rng('default') to control random number generation, so each neural network has the same divide data (train, validation, test) and initial weight/bias, then I change node size and compare the error to decide the best node size. But when I change rng('default') to rng(3,'twister'), I find the best node size is different from the previous one. Two rng() function gives the different answers, which one should I choose?

rng function (eg. rng('default'), rng(0,'twister'), rng('sshuffle')...) does not produce unique result for neural network training!

[Greg Heath]

I am not familiar with the different types. I just stick with the old ones: rng(0), rng(4151941) and rng('default'). It's hard to believe that one way of generating pseudo random numbers will consistently result in better designs.

Answer 1

[Steven Lord]

The pseudorandom number generators will eventually (after generating a LARGE number of values) repeat themselves. The 'twister' generator, the Mersenne Twister, has an approximate period of 2^19937-1. When you use RNG with a seed value, you're basically specifying where in the period to start generating numbers. [Before you ask, it's not as simple as "the second number generated after starting at seed value 1 is the first number generated after starting at seed value 2."]

So you're starting at different points in the period when you use rng('default') and rng(3, 'twister'). There's nothing inherently "better" about either one; they're just different. As an example, if I want to get to New York City, New York, USA, then a seed value that started me in Boston, Massachusetts, USA would be better than one that started me in Paris, France. But if I wanted to get to Berlin, Germany, the Paris starting point may be better.

[Greg Heath]

This is not an answer to your question. However, it is an interesting point to ponder.

 Most of the NN functions have a default of 10 hidden nodes. For a trained SISO 1-10-1 MLP, how many ways can the weighted nodes be reordered without changing the output?