rlTable

Value table or Q table

Description

Value tables and Q-tables are one of the approximation models that can be used within value functions and Q-value functions, respectively. Value tables store rewards for a finite set of observations. Q tables store rewards for corresponding finite observation-action pairs.

To create a value function approximator using an rlTable object, use an rlValueFunction, rlQValueFunction, or rlVectorQValueFunction object.

Creation

Syntax

T = rlTable(obsinfo)

T = rlTable(obsinfo,actinfo)

Description

T = rlTable(obsinfo) creates a value table for the given discrete observations.

T = rlTable(obsinfo,actinfo) creates a Q table for the given discrete observations and actions.

Input Arguments

obsinfo — Observation specification

rlFiniteSetSpec object

Observation specification, specified as an rlFiniteSetSpec object.

actinfo — Action specification

rlFiniteSetSpec object

Action specification, specified as an rlFiniteSetSpec object.

Properties

Table — Reward table

array

Reward table, returned as an array. When Table is a:

• Value table, it contains N_O rows, where N_O is the number of finite observation values.

• Q table, it contains N_O rows and N_A columns, where N_A is the number of possible finite actions.

Object Functions

rlValueFunction	Value function approximator object for reinforcement learning agents
rlQValueFunction	Q-Value function approximator with a continuous or discrete action space reinforcement learning agents
rlVectorQValueFunction	Vector Q-value function approximator with hybrid or discrete action space for reinforcement learning agents

Examples

Create a Value Table

Create a finite set observation specification object (or alternatively use the getObservationInfo function to extract the specification object from an environment with a discrete observation space). For this example, define the observation space as a finite set consisting of four possible values 1, 3, 5 and 7.

obsInfo = rlFiniteSetSpec([1 3 5 7]);

Use rlTable to create a value table object from the observation specification.

vTable = rlTable(obsInfo);

The table is a column vector in which each entry stores the value of the corresponding observation, under the given policy. You can access the table using the Table property of the vTable object. The initial value of each element is zero.

vTable.Table

ans = 4×1

     0
     0
     0
     0

You can also initialize the table to any value, for example, initialize the table to a random four-dimensional vector.

vTable.Table = rand(4,1);

You can now use your table as an approximation model for a value function for an agent with a discrete observation space. For more information, see rlValueFunction.

Create a Q Table

Create a finite set observation specification object (or alternatively use the getObservationInfo function to extract the specification object from an environment with a discrete observation space). For this example define the observation space as a scalar belonging to a finite set with three possible values.

obsInfo = rlFiniteSetSpec([0 -1 1]);

Create a finite set action specification object (or alternatively use the getActionInfo function to extract the specification object from an environment with a discrete action space). For this example define the action space as a finite set with two possible values.

actInfo = rlFiniteSetSpec([0 1]);

Use rlTable to create table object from the observation and action specifications.

qTable = rlTable(obsInfo,actInfo);

The table stores a value (representing the expected cumulative long term reward) for each possible observation-action pair. Each row corresponds to an observation and each column corresponds to an action. You can access the table using the Table property of the vTable object. The initial value of each element is zero.

qTable.Table

ans = 3×2

     0     0
     0     0
     0     0

You can initialize the table to any value. For example, initialize the table to a random array.

qTable.Table=rand(3,2);

You can now use your table as an approximation model for a Q-value function for an agent with finite (and low-dimensional) action and observation spaces. For more information, see rlQValueFunction.

Version History

Introduced in R2019a