If you have data with a high number of variables, it is important that the number of constraints to be multiple times the number of weight/bias values being optimized, especially for noisy data.
In this case with only a low number of samples or constraints, this would be difficult. So, it is preferable to have a low number of neurons in the middle layers.
A solution would be to reduce the number of variables. There are two ways to do this. (1) You could make a decision as to which variables are most likely to contain useful information. And (2) the PROCESSPCA preprocessing function can be used as the first input processing function as follows:
net.inputs{1}.processFcns = [{'processpca'} net.inputs{i}.processFcns];
net.inputs{1}.processParams{1}.maxfrac = 0.01;
PROCESSPCA removes redundant information from input data by orthogonalizing information in order of information content, and then dropping information which contributes less than a specific fraction of the information (maxfrac). Maxfrac can be increased to filter out more low quality information.
For such data, TRAINBR may be the best training function to use.
TRAINBR uses Bayesian Regularization, which means it attempts to find a good solution with small of weight values. Solutions with small weights, like solutions with fewer weights, are more likely to generalize well and less likely to over fit for any reason, including noisy data.
