Is it possible to apply upper and lower bounds to predictions in an LSTM?
3 views (last 30 days)
Show older comments
My predictions, which have been running very well as late, have hit an issue, where they are forecasting well below what should be the minimum possible value, i.e. forecasting at ~200, when the minimum should be 0. Is there anyway of simply applying a constraint/bounds, or indeed using the historical data to apply this?
0 Comments
Answers (1)
Ben
on 13 Mar 2023
I believe the default LSTM has outputs bounded in (-1,1) due to the activation functions used.
In any case you can try using activations to add constraints to your model, e.g. a tanhLayer has outputs bounded in (-1,1) and you can add a fixed linear transform to modify that to an arbitrary interval (a,b) as follows:
a = 0; b = 10;
% since tanh(x) is in (-1,1), (tanh(x) + 1)/2 is in (0,1).
% then scale to (0,b-a) and translate to (a,b);
boundToIntervalLayer = functionLayer(@(x) a + (b-a)*(tanh(x)+1)/2);
layers = [
sequenceInputLayer(1)
lstmLayer(1)
fullyConnectedLayer(1)
boundToIntervalLayer];
net = dlnetwork(layers);
% test it out
seqLen = 10;
x = dlarray(randn(1,seqLen),"CT");
y = forward(net,x);
% you can check y is in (a,b)
Alternatively if you use a custom training loop you could add a "soft constraint" by adding a penalty term to the loss function that is large when the network predicts values outside of your bounds.
0 Comments
See Also
Categories
Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!