Predict for Deep neural networks very slow

Hi all together,

I have a trained LSTM and want to use it now in my productive area. What I could see is that it is quite slow in terms of execution time

I created a simple script to compare the simulation time

net = load('lstmNN.mat');

nnInput = zeros(10,55);

%%%%%

% 1 'sequenceinput' Sequence Input Sequence input with 10 dimensions

% 2 'lstm_1' LSTM LSTM with 256 hidden units

% 3 'dropout_1' Dropout 30% dropout

% 4 'lstm_2' LSTM LSTM with 128 hidden units

% 5 'dropout_2' Dropout 30% dropout

% 6 'lstm_3' LSTM LSTM with 64 hidden units

% 7 'fc' Fully Connected 2 fully connected layer

% 8 'regressionoutput' Regression Output mean-squared-error with response 'Response'

%%%%

tic

for i=1:500

predict(net.net, {nnInput});

end

toc

%Elapsed time is 22.540738 seconds.

tic

for i=1:500

net.net.predict({nnInput}, 'SequenceLength', 'shortest');

end

toc

%Elapsed time is 21.369452 seconds.

%%%% Possible usage with standard net no DNN

% tic

% for i=1:500

% net.net({nnInput});

% end

% toc

%

I can see a slight difference in the different calls but all in all it takes quite long for predicting (and I know yes the net is quite deep but nonetheless I guess the execution could be faster).

So my question would be if there is any possibility to speed up the execution time

Additionally when having a look into the profile viewer I stumbled upon the fact that the DAGNetwork.predict() function was called 1000 times (as expected) but the LSTM.forward() function was called 3000 times which actually doesn´t make sense for me (see attachement).

Thanks a lot for your support! :)