How to retrain a deep neural network with another dataset which is already trained using a particular dataset?

I have designed a deep neural network and trained it using dataset-1. I have saved the network as my best model. Now I want to retrain this network with another dataset-2. In the following code "Net, trailingAvg, trailingAvgSq" used inside "adamupdate" is the network, average gradient and squared average gradient respectively. These three were already saved during training process using dataset-1. Is this is right way to retrain the network? Do I need to keep the dataset-1 with with the new dataset-2 to retrain (due to the chance of forgetting the old weight)? Provided the dataset is 1-D.

[Net,trailingAvg,trailingAvgSq] = adamupdate(Net,gradients,trailingAvg,trailingAvgSq,iteration,learnRate,decay);

Answer 1

[Ganesh]

Hi @BIPIN SAMUEL,

The implementation you are trying to achieve is the same as a "Transfer Learning system". The right way to do this would be load the weights and retrain the model using the new dataset. In order to ensure that the original model is not majorly affected, kindly choose an appropriate "Learning Rate" (generally, a lower learning rate to the prior training)

"Transfer Learning" is achievable with the "Deep Learning Toolbox", kindly refer to the link below:

https://in.mathworks.com/help/deeplearning/gs/get-started-with-transfer-learning.html