Problems with outputs in Neural Networks (in Matlab NN toolbox)

Hi, I trained my ANN (in Matlab) with 652500 data points, and in another blind test (652100 data points -for completely new input data sets) the output is excellent (as i want). But the problem occurs when i insert very less amount of data (for example below 50 data points). The output is quite unexpected,i checked it many times.

To be more precise, The training phase contains 10% data for training, 45 % for validation and 45% for testing. The training is quite successful, and for large amount of new input data it works very well. The problem is when very limited data (Compared to training data points) are inserted in the NN, it shows quite unrealistic output, beyond the range on what it was trained.

Why is this so? Could anyone light some sheds on this please?.

Also mention please, is there any strict (hard and fast) rules on training and final testing data points. For example: what percent of training data should be / must be introduced in the new input data sets. I guess the problem is my network overestimate or underestimate the output as very less percentage of data it receives as compared to training phase.

Answer 1

[Greg Heath]

You are experiencing the result of overtraining an overfit network.

Training, Validation and Test subsets should, each, be large enough to adequately represent the dominant characteristics of the population.

For I-dimensional inputs, O-dimensional outputs and a single hidden layer net with H nodes, there will be

Ntrneq = Ntrn*O training equations to solve for Nw = (I+1)*H+(H+1)*O unknown weights

If Ntrn < Nw/O, then any solution is not unique. Consequently, the resulting net should not be expected to perform well on nontraining data. Unraveling the inequality, Ntrn >= Nw/O if and only if

 H <= Hub  = -1 + ceil( (Ntrneq-O) / (I+O+1) )  % upper bound

 I have posted many, many posts using this logic to successfully design nets. Search on

 greg Hub

Thank you for formally accepting my answer

Greg