Neural Networks toolbox - How to choose which 'divideFcn' to use for time series prediction?

13 views (last 30 days)
David
David on 24 Jun 2014
Commented: Calvin on 18 Jul 2014
I'm using a layer-recurrent network for time series prediction (predicting joint angles from EMG recordings). My inputs are data from four EMG channels, formatted as a 4-by-N cell array for the four channels across N time steps (target signal is a 1-by-N cell array). The layer delays make my ANN an infinite-impulse response filter (right?), and thus the output depends on the order that data is fed into it. Based on that, it seems to me I should set
net.divideFcn = 'divideblock';
net.divideMode = 'time';
so that training data is presented in sequence. However, after training a few networks with 'divideblock' and a few with 'dividerand', I got better results for 'dividerand' (I have not tried different architectures, just one with layerDelays=1:2 and hiddenLayerSize = 5). My best guess right now is that 'dividerand' can still take some advantage of temporal structures because I have a 1 kHz sampling rate but most of the input signals are below 100 Hz and most of the target signal is below 20 Hz (that is, it doesn't matter if you skip a few data points).
I thought I had a reasonable understanding of how artificial neural nets work both mathematically and in the Matlab toolbox, but this isn't clicking for me. I'm curious if there's something I don't understand about the training process, about how to format the inputs/outputs (and am doing something wrong), or if everything is working as it ought to and I'm just slightly confused. (I'm also not 100% clear on when one ought to use multiple inputs vs. one input of size > 1 for time series, but I don't think that's the issue.)
To summarize, the question is whether 'divideblock' is the appropriate choice for a recurrent network doing time series prediction, and, if not, why it's okay to use 'dividerand' given that input order matters.
If anyone has words of wisdom to help clarify, I'd greatly appreciate it. Thanks much!

Accepted Answer

Greg Heath
Greg Heath on 25 Jun 2014
1. Standardize all signals to have zero-mean/unit-variance
2. Find the significant positive lags of the target autocorrelation function
3. Find the significant nonnegative lags of EACH input/target crosscorrelation function
4. For examples search the NEWSGROUP and ANSWERS using the keywords
greg nncorr
5. Use the results of 2 and 3 to determine
a. Which inputs to use
b. Which timeseries function to use
c. Which combination of delays to use
5. Unfortunately, delays for all inputs must be the same. However, it may be possible to improve performance by combining outputs of multiple nets with different input/delay combinations.
6. In order to maintain temporal correlations I typically use multiple designs with divideblock to determine both the number of hidden nodes and corresponding weights.
7. In your case where dividerand appeared to work better than divideblock, try divideind or divideint.
Hope this helps.
Thank you for formally accepting my answer
Greg
  1 Comment
David
David on 25 Jun 2014
Thanks for your response. So in short, divideFcn choice is important in ANNs that predict data based on temporal patterns?
Re: 5b - by "timeseries function," do you mean narxnet vs. narnet vs. layrecnet, etc?
Thanks again.

Sign in to comment.

More Answers (1)

Shashank Prasanna
Shashank Prasanna on 24 Jun 2014
You can use any approach that works best for you. For time-series data, the toolbox uses lagged observations to create new predictors for your network. This means it preserves the temporal structure of the data for the number of lags you choose. Once this is done, any random sampling of the data for validation and testing is appropriate. Hope that helps.
  4 Comments
Calvin
Calvin on 18 Jul 2014
This is good dialogue and I hope it can continue so some of us can learn. Perhaps the random data partitioning (dividerand) affect(?) on the temporal structure of the input & target data can be framed by this question: http://www.mathworks.com/matlabcentral/answers/140637-does-dividerand-really-destroy-time-series-data-autocorrelations Please guys, share your wisdom!

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!