serial classification of timeseries "phases" with neural network

Hello,

I'm just getting acquainted with methods of machine learning, and am experimenting with neural networks at the moment. I need some advice how to approach a specific task of classification where serial ordering is important as well.

My current goal is to examine a data timeseries, and classify it into successive phases. I have not seen any examples of classificaiton where there is a serial dependence between classes, and a fixed order that must be satisfied.

To better explain the data set, and the task scenario, the data is acquired from a motion analysis system. There are 7 motion-tracked markers on someone's hand/arm, and the dataseries is captured in 3D, i.e. 21 features total.

There are multiple repetitions of the following behavioural task: 1. trial begins with hand at rest 2. after short delay, a the hand is lifted and moves to a target location to pick up an object 3. the object is lifted (from target-0), and then set down at another location (target-1) 4. the object then picked up from target-1 and set down at target-2

In other words the phases I need to classify would be: phase 1 would be from the beginning of the timeseries until the hand is lifted phase 2 the relatively motionless period during grasping the object at target-0 until lifting the object at target-0 phase 3 the time spent in motion from lifting the object at target-0 to setting down the object at target-1 phase 4 the relatively motionless period during setting the object at target-1, to lifting it from target-1 phase 5 the time spent in motion from lifting the object at target-1 to setting down the object at target-2

My goal is to train a neural network to recognize when the hand is initially lifted from the neutral position, when the hand reaches the target-0 to lift the object, when the hand sets down the object at target-1, and when the hand lifts from target-1 and then sets down object at target-2.

I can manually create the correct classifications for training data very easily, as the data stream is easy to classify through a visual inspection of the plotted timeseries. My use for applying a solution with a neural network is to infer when each movement phase starts/ends for many separate iterations (over a thousand) of this pattern of movement.

Any suggestions how to approach this problem?

If the info provided isn't adequate to understand my data or classification challenge, I'll post a reply with links to some plots.

Thanks, Brian

p.s. I'm not so much worried about data preprocessing as I am about how to construct, train and use an appropriate neural network model.

Answer 1

[Greg Heath]

Inputs to both both static and dynamic nets have a fixed number of inputs. There may be problems if the significant temporal length of feature vectors is variable.

If you are considering a stream of multivariable data with variable length feature regions separated by variable length non-feature regions, it will be very, very difficult.

The success of a this type of classifier depends on preprocessing. Therefore you have to have a clear idea of

 1. How to recognize the presence of target data

 2. How to extract the data from the steam

Answer 2

[Dan]

You may wish to consider lstm's ( long short term memory networks ( or recurrent nn's in general ) . https://github.com/yechengxi/LightNet Has a simple implementation with an example for predicting the next letter in a sequence ( a letter is really a class in this case) . Good luck Dan

Answer 3

[Greg Heath]

1. Obtain N training pairs of I-dimensional inputs and corresponding O-dimensional targets

2. If the task is classification between c mutually exclusive categories, the targets can be columns of the c-dimensional unit matrix eye(c). Then outputs can be interpreted as estimates of class conditional posterior probabilities, conditional on the input.

3. For an arbitrary input, the input is assigned to the category corresponding to the maximum output component.

4. Form input and target matrices with dimensions [ I N ] and [ O N ], respectively.

5. Use patternnet

 help patternnet
 doc patternnet

6. Practice on one or more of the MATLAB classification/pattern-recognition data sets

help nndatsets

7. Search through the NEWSGROUP and ANSWERS for posts containing the keyword PATTERNNET.

Hope this helps.

Greg

[Brian]

great! thanks for the info. I'll check out patternnet. Thus far I have only toyed with nnstart and haven't much knowledge of the specific nnet toolbox functions. will report back and accept this answer if all works out.

[Brian]

Perhaps I am missing a crucial detail, but when reading your answer again I understand the steps you have suggested to implement a standard pattern classifier, not one that has any ability to represent serial ordering of the labelled classes.

One 'hack' to make it more appropriate for my purpose would be to include a 'sample number' as one of the features, but some degree of ambiguity in resolving the proper class remains.

Consider the following: the motionless period during grasping of an object at target-0, or target-1, or target-2 will all have similar features (e.g. low velocity for a short duration). Whether the algorithm can resolve this as being at target-0,1, or 2 is then dependent on other features. If these grasps always occurred around the same time during a data time series (say 500th sample, 1000th, and 1500th), a 'sample number' features would likely successfully resolve each grasp as being classified into different classes. However, if one trial recording has someone who only grasps the object at target-0 near the 1000th sample, this is likely going to be wrongly classified as being a grasp to target-1.

Consequently, the system lacks a true representation of serial ordering of the successive phases of the data timeseries. So, perhaps I have missed a detail in your answer, but my initial question remains. I am not so much worried about how to script a pattern classifier, but I need one that operates on a time series with consideration of the history of the time series.

Just as a NARX integrates previous data points to generate its output, I need a pattern classifier (not a timeseries predictor) that operates in the same manner.

EDIT: target-0,1,2 are three sequential targets, to different locations... i.e. one trial might go target-A,target-C,target-B, the next target-B,target-A, target-C...so absolute spatial information would not completely resolve the ambiguity of determining target-0,1,2 as they appear sequentially in time.