MATLAB Answers

How can I plot extra RL training data while using parallel computing?

1 view (last 30 days)
I'm working on a RL project using the Reinforcement Learning Designer, the PPO agent, and a custom environment. To help with diagnostics during the training process I set up some extra plots that track some statistics related to my agent's performance (separate from the RL designer's built in plot that shows reward, average reward, and q0). These plots are intended to capture statistics about the given episode regarding the final state of the environment as well as important moves made during the episode. This data is saved to environment properties inside the step() function during an episode, and then written to plots at the end of an episode in the reset() function.
For example,
classdef custEnv < rl.env.MATLABEnvironment
properties
numKeyMoves = 0;
numTotalMoves = 0;
numEpisodes = 0;
Figure1
end
methods
function [Observation, Reward, IsDone, LoggedSignals] = step(this, Action)
if 1 % if the move was a key move
this.numKeyMoves = this.numKeyMoves + 1;
end
this.numTotalMoves = this.numTotalMoves + 1;
end
function InitialObservation = reset(this)
ha = gca(this.Figure1);
% plot the percentage of key moves against episode num to track
% progress
plot(ha, this.numKeyMoves / this.numTotalMoves, this.episodeNum)
this.numKeyMoves = 0;
this.numTotalMoves = 0;
this.episodeNum = this.episodeNum + 1;
end
function plot(this)
this.Figure1 = figure('Visible', 'on', 'HandleVisibility', 'off');
end
end
end
This has worked well to date, but now I'd like to use parallel computing to speed up future training sessions. However it seems like the parallel workers aren't interacting with the step() and reset() functions in the same way, so the plots I've created are never getting populated with any data once parallel training is enabled. Is there a way to recreate the funcitonality above with parallel training, or any other stable way to similarly track environment properties as episodes progress?

Accepted Answer

Shadaab Siddiqie
Shadaab Siddiqie on 4 Aug 2021
Form my understanding you are not getting correct plot when using parallel computing. I have heard that this issue is known, and the concerned parties may be investigating further.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!