NARX model training in the Neural Network Tool Box

I have Two questions concerning NARX neural network model when training is done using the early stopping technique:

1. It is true that NARX model is trained with a static configuration?
2. If yes, which one of the errors is used to stop training : error (between actual data/predicted data) calculated using a feed back or without feed back like the classic MLP static network? (please see figure below)

Thank your in advance for your response.

Answer 1

[Greg Heath]

Narnet and narxnet have two modes: OPENLOOP and CLOSELOOP.

The training of these and most of the MATLAB NNs is stopped via te FIRST of 6 conditions. For example,

net         = narxnet            % NO SEMICOLON
TrainParams = net.trainParam     % NO SEMICOLON

% Partial results

         trainParam: .showWindow, .showCommandLine, .show, .epochs,
                    .time, .goal, .min_grad, .max_fail, .mu, .mu_dec,
                    .mu_inc, .mu_max

TrainParams =

    Function Parameters for 'trainlm'

    Maximum Epochs                      epochs: 1000
    Maximum Training Time                 time: Inf
    Performance Goal                      goal: 0
    Minimum Gradient                  min_grad: 1e-005
    Maximum Validation Checks         max_fail: 6
    Maximum mu                          mu_max: 10000000000

The OPENLOOP mode is static and only used for training with target data fed into the feedback delay nodes. However, when it is converted to CLOSELOOP for operational use, the accuracy can be worse than if it had been directly trained in the CLOSELOOP mode.

The CLOSELOOP configuration is dynamic and is used for both training and use in the operational mode. However, CLOSELOOP training is very slow compared to OPENLOOP training and the final results may not match well with the target data.

Therefore, there are three approaches to training an operational CLOSELOOP net.

 1. Train and operate CLOSELOOP

 2. Train OPENLOOP and convert to CLOSELOOP for operational use

 3. Train OPENLOOP, convert to CLOSELOOP to train further and use for operation.

The third method tends to yield the best results.

Hope this helps

• Thank you for formally accepting my answer*

Greg

Answer 2

[Platon]

Thank you Greg.

So, when I use this code :

net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
[net,tr] = train(net,inputs,targets,inputStates,layerStates);

Training is done following which approache? it seems to me that is the second one (Train OPENLOOP and convert to CLOSELOOP for operational use). How could I train the net in OPENLOOP, convert to CLOSELOOP to train further and use for operation? is there a specific functions in Mathlab to do that or I have to code it?

Answer 3

[Greg Heath]

See the examples in

help preparets

help closeloop

Also see if the doc explanations and examples are different.

Hope this helps.

Thank you for formally accepting my answer

Greg

P.S. Change names when converting to the closeloop configuration

netc = closeloop(net);

[Platon]

I get it, to train in close loop I have to put the closed NN in the train function :

 net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
 netc = closeloop(net);  
[netc,tr] = train(netc,inputs,targets,inputStates,layerStates);

But how can I do to train it successively in open loop and then in the closed loop ?

 net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
[net,tr] = train(net,inputs,targets,inputStates,layerStates);
 netc = closeloop(net);   
[netc,tr] = train(netc,inputs,targets,inputStates,layerStates);

??

[Greg Heath]

Almost!

You forgot to use preparets before training and to use the outputs of preparets in train.

Go back and see

 help preparets
 help closeloop

[Platon]

net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
[inputs,inputStates,layerStates,targets] = preparets(net,inputSeries,{},targetSeries);
[net,tr] = train(net,inputs,targets,inputStates,layerStates);
 netc = closeloop(net); 
[inputs,inputStates,layerStates,targets] = preparets(netc,inputSeries,{},targetSeries);  
[netc,tr] = train(netc,inputs,targets,inputStates,layerStates);

It is correct now?

[Greg Heath]

OK. Now try it.

I always compare the close-loop and open-loop responses before close-loop training to get an appreciation for what the additional training really did.

And sometimes I train close-loop from the beginning to see how much pre-training with open-loop really helps w.r.t. training time.