MATLAB Answers


Reinforcement Learning Toolbox - Experience Buffer Samples

The simulation I'm running has a fixed-step solver with a fixed-step-size of 5e-4. The sample time of my DQN-Agent (and the corresponding S-function for the reward-signal) is 0.25.
How is it possible that after a simulation time of 20 seconds I have a BufferLength of ~1600 samples? I hope you can enlighten me...
Bonus question:
Is it possible to look into the ExperienceBuffer? As impressed as I am by the RL-Toolbox, I would really prefer it not to be such a blackbox in most cases.


Sign in to comment.

1 Answer

Answer by Raunak Gupta on 20 Sep 2019
 Accepted Answer

Since it is mentioned that DQN Agent is used, I am assuming that rlDQNAgentOptions is used for setting up the agent properties. The ExperienceBufferLength can be specified for storing that many experiences from training the agent. Also, there is a parameter SaveExperienceBufferWithAgent which can be set to true for saving the Experience buffer while training. The experience upto the limit of ExperienceBufferLength will be stores in a rlDQNAgent Object.
You may look for other Training Options here.

  1 Comment

Thank you for your answer!
I found a bug in my simulation setup so that the Buffer was filled way to fast. I fixed this with a delay block inheriting a certain sample-time.
Yet it is not pissible to look inside the buffer to view what kind of (s,a,r,s')-touples are stored in there.

Sign in to comment.