The best Ann configuration
4 views (last 30 days)
I have run Ann for prediction with hidden nodes from 2-17 about 50 times. My question is which criteria I should rely on to select the best Ann? Should I choose R squerd of Test or Mse of the Ann or validation performance?
My favorite technique:
1. Accept all default parameters except for the number of hidden nodes, H.
2. Minimize H subject to the constraint that the degree-of-freedom adjusted
mean-square-error of the training data is less than 1% of the average training target variance.
3. Design and test Ntrials >= 10 nets for each value of H in a range less than the upper
bound Hub (determined by not having more unknown weights Nw than training equations Ntrneq).
The untrained nets only differ by the random trn/val/tst data division AND the random initial weights.
4. Rank the nets via their slightly biased performance on validation data.
5. Obtain unbiased performance estimates on the nets using test data.
6. Statistically significant differences in performance can be estimated using the standard
deviation of the performance estimates.
I have posted zillions of examples in both the NEWSGROUP and ANSWERS using the same notation. Therefore searching with
greg fitnet Ntrials
should dig up enough references to clarify what I have written. If not just post a comment.
Hope this helps.
Thank you for formally accepting my answer
More Answers (2)
Walter Roberson on 9 Feb 2016
What is the best way of getting around "inner London" (United Kingdom)? Is it taxi, personal automobile, Tube (subway) -- or even bicycle (ha ha ha)?
Did you decide yet? So did you measure "best" by convenience, cost, health benefits, or speed? Or did you construct a carefully weighted measure of all of those, such as being willing to trade 1 minute longer travel time for each 2000 Calories of fat burned? Or should it be 1.5 minutes and 1000 Calories? What scientific study did you use to decide the trade-offs?
Oh, by the way: multiple tests have shown that during a typical work-day afternoon, the lowest cost way of getting around inner London is by bicycle, but the healthiest way of getting around inner London is instead by bicycle; and on the third hand, the fastest way of getting around inner London is... by bicycle.
So when you are selecting an ANN, what do you mean by "best" ?
Insuffient information and explanation:
size(input) ? size(target) ?
If I guess both are [ 1 N ] , Ntrn ~ 0.7*N and Hub = 114, then
(Ntrn-1)/(1+1+1) = 114
Ntrn = (3*114+1) % 343
N = Ntrn/0.7 % 490
NMSEgoal <= 0.01 with Hub = 114 % Probably don't need 0.005
why in the world are you even considering
NMSE = 0.25 @ H = 16
NMSE = 0.51 @ H= 17
instead of increasing H???