Clear Filters
Clear Filters

Mismatch between rlCreateEnvTemplate() and rlTable()

2 views (last 30 days)
If rlTable() requires scalar inputs then why can't I change the ObservationInfo in the environment template from rlNumericSpec([4 1]); to rlFiniteSetSpec([1 2 3 4])?
I created an environment from template, like:
rlCreateEnvTemplate("myEnvClass");
env = myEnvClass;
The environment validates fine, using
validateEnvironment(env);
The problem comes when setting up sarsaAgent, which first requires creation of an rlTable object as an input parameter:
>> qTable = rlTable(getObservationInfo(myenv), getActionInfo(myenv));
Error using rlTable/validateInput (line 131)
Input must be a scalar rlFiniteSetSpec.
Error in rlTable (line 51)
validateInput(obj, ObservationInfo)
So then I figured changing these lines (from myEnvClass.m):
function this = myEnvClass()
% Initialize Observation settings
ObservationInfo = rlNumericSpec([4 1]);
to:
function this = myEnvClass()
% Initialize Observation settings
ObservationInfo = rlFiniteSetSpec([1 2 3 4]);
would do the trick. By now the environment won't validate:
>> env = myEnvClass;
>> validateEnvironment(env);
Error using rl.env.MATLABEnvironment/validateEnvironment (line 28)
Environment 'ObservationInfo' does not match observation output from reset function.
Check the data type, dimensions, and range.
And I don't see how to change the reset function to make it work. Here it is from the template generated environment class:
% Reset environment to initial state and output initial observation
function InitialObservation = reset(this)
% Theta (+- .05 rad)
T0 = 2 * 0.05 * rand - 0.05;
% Thetadot
Td0 = 0;
% X
X0 = 0;
% Xdot
Xd0 = 0;
InitialObservation = [T0;Td0;X0;Xd0];
this.State = InitialObservation;
% (optional) use notifyEnvUpdated to signal that the
% environment has been updated (e.g. to update visualization)
notifyEnvUpdated(this);
end
This doesn't make sense because this same reset() function returns a 4 row column vector:
>> reset(env)
ans =
-0.0108
0
0
0
like the one expected from the class definition:
>> getObservationInfo(env)
ans =
rlFiniteSetSpec with properties:
Elements: [4×1 double]
Name: "CartPole States"
Description: "x, dx, theta, dtheta"
Dimension: [1 1]
DataType: "double"
>> ans.Elements
ans =
1
2
3
4
And I'd like to point out that changing the datatype set within the reset() function, like this:
InitialObservation = rlFiniteSetSpec([T0 Td0 X0 Xd0]);
this.State = InitialObservation;
doesn't help:
>> env = myEnvClass;
>> validateEnvironment(env);
Error using rl.env.MATLABEnvironment/validateEnvironment (line 22)
There was an error evaluating the reset function.
Caused by:
Error using myEnvClass/set.State (line 165)
Expected State to be one of these types:
double, single, uint8, uint16, uint32, uint64, int8, int16, int32, int64
Instead its type was rl.util.rlFiniteSetSpec.

Answers (0)

Products


Release

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!