Neural Network Time Series Prediction - changing the inital state
6 views (last 30 days)
Show older comments
Hello, I'm working currently with prediction-problems for dynamical systems, e.g. single pendulum with friction. At the moment I'm testing neural networks for time series predictions, although my knowledge is very basic. My understanding of neural networks in light of dynamical systems is that they are working like a flexible state-space-model. Training the neural network with some testdata should result in an accurate state-space-model, which can be used for predictions, am I right?
Lets say, I split my testdata into two sections. The first one will be used for training purpose and the second one for validation (in reference to my attached file). The prediction gives good results on the validation data, going forward, we are using the same net, but vary the inital state, here the inital angle of the pendulum. Is it even possible to vary the inital state? Does the net just predict on a one-off basis of the training data?
Regards, Michael
0 Comments
Accepted Answer
Greg Heath
on 10 Sep 2015
You are using inappropriate terminology for NNs:
Neural network data division is threefold: training, validation and testing:
total = design + test
design = train + validate
non-design = test
non-training = validate + test
TRAINING: Given input(x)/target(t) training data and parameters, estimate the network weights and biases to minimize the error between the output(y) and the target(t).
Training error calculations are BIASED because the data used to calculate the error is the same data that is used to estimate the weights. If the number of estimated weights, Nw, is smaller than the number of training equations, Ntrneq, then the bias can be reduced by calculating the number of degrees of freedom, Ndof that result after the Nw weights are estimated using the Ntrneq training equations:
Ndof = Ntrneq - Nw
If the error and sum-squared error of the training data are given by
etrn = ttrn - ytrn; SSEtrn = sum( etrn.^2),
then the biased and degree-of-freedom-adjusted estimates of the training data mean-squared-error are
MSEtrn = SSEtrn/Ntrneq and MSEtrna = SSEtrn/Ndof
VALIDATION: Although validation data is used for design, it is not used to to directly estimate weights. It's purpose is to make sure that the net will perform well on non-training data. Therefore it's error is continuously monitored during training. If the error monotonically increases for a specified number of epochs (e.g., the default is 6) , then the training is stopped.
Although validation is part of the design process, the error estimates for the validation data are only SLIGHTLY biased. Therefore, no effort is made to estimate the bias or adjust the estimate.
TESTING: Although the test subset error is also continuously calculated during training, it is in no way involved in design. Therefore it's error is an UNBIASED estimate of the networks performance on non-training data (e.g., validation, testing and unseen).
Although MATLAB NN functions automatically use random data-division in the trn/val/tst proportion 0.7/0.15/0.15. For time-series I usually advise the use of the data-division option 'divideblock' that retains the data's time sequential order with the validation data placed between the training and test data.
Please look up some of my time-series posts using "greg" with TIMEDELAYNET, NARNET OR NARXNET. However, first look up the help and doc documentation. For example:
help narxnet
doc narxnet
Hope this helps.
Greg
0 Comments
More Answers (1)
Michael
on 11 Sep 2015
Edited: Michael
on 22 Dec 2015
2 Comments
Greg Heath
on 11 Sep 2015
WHEN I CLICK ON THE LINK I GET
404. That’s an error.
The requested URL /http%3A%2F%2Flab.fs.uni-lj.si%2Flasin%2Fwp%2FIMIT_files%2Fneural%2FNN-examples.pdf was not found on this server. That’s all we know
See Also
Categories
Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!