MATLAB Answers

Mini-Batch loss and accuracy trends

40 views (last 30 days)
VaL V6
VaL V6 on 9 Oct 2019
Answered: Kritika Bansal on 6 Nov 2019
I'm training a Fast RCNN detector for the first time. I've got a data set of 3000 images with about 3 label per image.
The training gives me the following results:
I would ask why the Mini-batch loss and the Mini-batch accuracy have trands that go up and down sharply and can't settle around fix values.
Below my training options:
opts = trainingOptions('adam',...
'InitialLearnRate', 0.000001, ...
'LearnRateSchedule', 'piecewise', ...
'LearnRateDropFactor', 0.1, ...
'LearnRateDropPeriod', 8, ...
'MaxEpochs', 10, ...
'MiniBatchSize', 32, ...
'Verbose', true);
Could it be the initial learning rate too high or the number of epoch too low?
Thank you!

Answers (1)

Kritika Bansal
Kritika Bansal on 6 Nov 2019
As you said you’re using Fast R-CNN for the first time, the example given on page can help you understand the role of various inputs going into trainingOptions function. There can be many probable reasons behind your network not getting converged.
Since you are training your network using adam solver, consider using the other training options associated with it like GradientDecayFactor and SquaredGradientDecayFactor. Also, consider training your network using other methods available in MATLAB R2016a and above, like sgdm and rmsprop. You can learn more about the solver options available to you in the link below:
Also, check to see if the images in your training data do not contain excessive NAN values, as too many NaNs may lead to propagation of NaNs through the network and hence the training will fail to converge.
As far as your initial learning rate in considered, 10^-6 seems to be very small, and with every 10 epochs it is further decreasing. An initial learning rate of 10^-3 may be of help in your case.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!