Why do i obtain good results but poor input error cross correlation plot?
1 view (last 30 days)
Show older comments
Hi,
May i know why do i obtain a low MSE and good predicted outputs but poor input error cross correlation plot?The correlations do not fall within the confidence limit. I am using neural networks time series tool. Why does the confidence limit varies for different MSE results? Currently mine my confidence limit is at +- 2000. Other MSE results can give me confidence limit of +- 4000 but with higher MSE. May i know the reason for this?
Thank you
0 Comments
Accepted Answer
Greg Heath
on 6 Nov 2014
If the input/error cross-correlation is large, then a different set of weights could use that correlation to further improve performance.
The best result would be an error that is not at all correlated with either input or output.
Hope this helps.
Thank you for formally accepting my answer
Greg
3 Comments
Greg Heath
on 7 Nov 2014
How are you obtaining the MSE confidence limit?
Why not use normalized MSE values (divide by average target variance, MSE00) to remove the obscuration of target scaling?
Different weights associated with a different design.
a. divideblock instead of dividerand
b. Better choice of delays (See my correlation function method of choosing delays). Search NEWSGROUP and ANSWERS using
greg ncorr
c. Better combination of hidden nodes and random initial weights See my double for loop method of optimizing number of hidden nodes and initial weights):
greg narxnet Ntrials
Hope this helps.
Thank you for formally accepting my answer
Greg
Greg Heath
on 7 Nov 2014
To be clear:
Training results are biased because the data used to obtain performance summaries are the same data used to design the model. However, if the number of training equations is larger than the number of unknown weights, a degree of freedom correction is useful.
Validation results are also biased because the data is used to determine when to stop the design. However, the bias is much smaller. Therefore, the val performance should be used to choose among competing designs.
Therefore, test performance which, in no way is involved in design, is the only UNBIASED prediction of performance on unseen data.
Hope this helps.
Greg
More Answers (0)
See Also
Categories
Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!