- Set an initial guess for the mean in the bias of the “fullyConnectedLayer” in the mean calculation.
- However, because of the non-linearities like the “tanhLayer”, directly setting the bias to achieve a specific mean after scaling and non-linear transformations can be complex.
How to set initial estimate for mean in PPO actor critic network
1 view (last 30 days)
Show older comments
I am using a PPO actor critic network. I created the actor following this example
https://www.mathworks.com/help/reinforcement-learning/ref/rl.function.rlcontinuousgaussianactor.html
How can I set an initial guess for the mean? Currently the actor always starts with an intial mean at time zero of zero.
0 Comments
Answers (1)
Aneela
on 22 May 2024
Hi Jason Butler,
To set an initial guess for the mean in PPO actor network, modify the initial weights or biases of the layers that contribute to calculating the mean.
Assuming the desired initial mean as 5, here’s a workaround:
desiredInitialMean = 5; % Adjust this value as needed
% Since you have 3 actions, create a bias vector with 3 elements
biasForDesiredMean = repmat(desiredInitialMean / actInfo.UpperLimit, [prod(actInfo.Dimension), 1]);
% Modify the meanPath definition to include the bias initialization as a vector
meanPath = [
tanhLayer(Name="tanhMean");
fullyConnectedLayer(prod(actInfo.Dimension), ...
'Bias', biasForDesiredMean, ...
Name="fcMean");
scalingLayer(Name="scale", ...
'Scale', actInfo.UpperLimit)
];
For more information on “Bias” in the “fullyConnectedLayer”, refer to the following MathWorks documentation: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.fullyconnectedlayer.html?s_tid=doc_ta#:~:text=Layer%20biases%2C%20specified,single%20%7C%20double
0 Comments
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!