Neural network for regression that has mulitple outputs of different order
1 view (last 30 days)
Show older comments
byungchan
on 13 Nov 2022
Commented: byungchan
on 25 Nov 2022
Hello, I am asking to see if there is any advice to create the neural net that has multiple outputs (for regression) with differnt order(say one of output has order of 10^-5, while the other output has order of 10^5).
Is there any tips for this? Shall I have to make seperate neural nets for each of those outputs?
Thank you very much!
0 Comments
Accepted Answer
Varun Sai Alaparthi
on 16 Nov 2022
I understand that you want to effectively train a regression model with multiple output heads and the outputs are of different orders.
My advice would be to use weights while adding L2 regression losses. This would normalise the magnitude effect caused due to difference in order.
For example:
Loss = Loss1*W1+Loss2*W2;
% If Loss1 s order > Loss2’s order W1<W2 (example: W1 = 0.001, W2=1)
Here W1 and W2 can be treated as hyper parameters and can be tuned accordingly for effective training.
Please refer to the following link for implementing custom training loop and loss function with weights for multiple outputs:
Please try this and reach out to us for any further issues.
Sincerely
Varun
More Answers (0)
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!