How to set boundaries for action in reinforcement leaning?

3 views (last 30 days)
There are 3 actions in my environment and the boundaries of them are from [1; 1; 0] to [5; 5; 1]. The codes are as follows:
function this = myEnvClass()
% Initialize Observation settings
ObservationInfo = rlNumericSpec([9 1]);
ObservationInfo.Name = 'ASV States';
%ObservationInfo.Description = 'x, dx, theta, dtheta';
ObservationInfo.Description = 'dx, dy, dz,dl,vx,vy,vz,phi,theta';
% Initialize Action settings
ActionInfo = rlNumericSpec([3 1 1], 'LowerLimit',[1;1;0], 'UpperLimit',[5;5;1]);
ActionInfo.Name = 'ASV Action';
ActionInfo.Description = 'rho,sigma,theta';
% The following line implements built-in functions of RL env
this = this@rl.env.MATLABEnvironment(ObservationInfo,ActionInfo);
% Initialize property values and pre-compute necessary values
updateActionInfo(this);
% this.State = [400 400 -50 0 0 0 0 0 0]';
end
and the codes of updateActionInfo function are as follow:
function updateActionInfo(this)
% this.ActionInfo.Elements = this.MaxAngle*[-1 1];
this.ActionInfo = rlNumericSpec([3 1 1], 'LowerLimit',[1;1;0], 'UpperLimit',[5;5;1]);
this.ActionInfo.Name = 'ASV Action';
this.ActionInfo.Description = 'rho,sigma,theta';
end
But when I trained the agent(PPO), the actions in step fucntion were always far greater or far less than the boundary value. For example, action = [144, 152, -63], action = [1608, -1463, -598].
I attached my myEnvClass.m, would someone please help me?

Answers (0)

Products


Release

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!