NARX - Multi step prediction

I am working on my thesis in which I have 2 objectives.

Create a neural network and make predictions;

To control that network (impose a constant output and find out which input values make that possible);

This way I created a NARX network. In open-loop the mse is low but in closed-loop it is around 0.05. I would like to know if it was possible to decrease more the error in closed-loop since for what I want it is insufficient.

The second question is that if I put only one input value and make the prediction and then change that input value for another, the output is always the same. That is, every time I change the input the output is the same. I did this in order to find the input values that were closest to the reference value I wanted. However I can't determine why the output is always the same.

The code I used was this:

X = tonndata(input,true,false);

T = tonndata(target,true,false);

trainFcn = 'trainlm'; % Levenberg-Marquardt backpropagation.

inputDelays = 1:4;

feedbackDelays = 1:4;

hiddenLayerSize = 25;

net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize,'open',trainFcn);

% Prepare the Data for Training and Simulation

[x,xi,ai,t] = preparets(net,X,{},T);

% Setup Division of Data for Training, Validation, Testing

net.divideParam.trainRatio = 70/100;

net.divideParam.valRatio = 15/100;

net.divideParam.testRatio = 15/100;

% Train the Network

[net,tr] = train(net,x,t,xi,ai);

% Test the Network

y = net(x,xi,ai);

e = gsubtract(t,y);

performance = perform(net,t,y)

% Closed Loop Network

netc = closeloop(net);

netc.name = [net.name ' - Closed Loop'];

[xc,xic,aic,tc] = preparets(netc,X,{},T);

yc = netc(xc,xic,aic);

closedLoopPerformance = perform(net,tc,yc)

% New input

entrada = tonndata(input3,true,false);

saida = tonndata(target3,true,false);

% Prediction

next_value = sim(netc, entrada);

entrada = entrada(1:numel(next_value));

% Plot results

plot(1:numel(entrada), cell2mat(saida), 'b', 1:numel(entrada), cell2mat(next_value), 'r')

I appreciate any help

Answer 1

[Prasanna]

Hi Vladislav,

To reduce the error in closed-loop mode, you can try the following steps:

• Increase the training data by providing a large diverse dataset that can improve the model’s ability to generalise. Also, allowing the network to train for more epochs can reduce the error obtained.
• Optimize the network parameters by modifying the network layer size, delays and the training function. While ‘trainlm’ is a good choice, other functions like ‘trainbr’ (Bayesian Regularization) or ‘trainscg’ (Scaled conjugate gradient) might yield better results.
• Perform data normalization and regularization to avoid overfitting which becomes significant in closed-loops.

To address the output consistency with different inputs, refer the following steps:

• Ensure that the network is properly initialized each time you test with new inputs. Random initialization of weights can sometimes lead to unexpected results.
• Double-check that the new input data (‘input3’) is being prepared correctly and matches the format of the training data. Ensure that the variables ‘entrada’ and ‘saida’ are properly aligned and preprocessed.
• In closed-loop mode, the network uses its own predictions as inputs for future predictions. Ensure that this feedback mechanism is correctly set up and that the network isn't stuck in a loop with constant output due to feedback issues.
• If the network output is saturated (e.g., due to activation function limits), this could cause constant outputs. Consider changing the activation functions if necessary.

You can also visualize immediate outputs, perform verbose training and experiment with the hyperparameters by using grid search to debug the network better and find the best network configuration. By carefully examining these areas, you should be able to improve the closed-loop performance and resolve the issue with constant outputs. For more information regarding the NARX networks, refer the following documentations:

• Narxnet Multistep forecast within the time series: https://www.mathworks.com/matlabcentral/answers/279654-narxnet-multistep-forecast-within-the-time-series
• Multi Step Ahead Prediction: https://www.mathworks.com/matlabcentral/answers/14970-neural-network-multi-step-ahead-prediction
• NARX Feedback neural networks: https://www.mathworks.com/help/deeplearning/ug/design-time-series-narx-feedback-neural-networks.html
• trainbr: https://www.mathworks.com/help/deeplearning/ref/trainbr.html
• trainscg: https://www.mathworks.com/help/deeplearning/ref/trainscg.html

Hope this helps!