How to force Matlab to start reinfocement learning training from a certain initial state of my simulink environment
3 views (last 30 days)
Show older comments
My reinforcement learning (RL) environment, which is build in simulink, needs some initial time to settle numrically (t_settle). So, in the time span from simulation start t=0 sec till t=t_settle of my environment all signals, especially the reward and observation signals, which are the inputs of my RL_agent, are not representative, because my model of a physical hydraulic system is still numerically not settled. Hence the agent would learn some numerical issues, rather than to learn to performe well on my modelled hydraulic system. Furthermore, this time span from t=0 sec till t=t_settle is computationally very expensive, which prolongs my training-duration drastically. Hence, due to both reasons I would like to start to train my RL_agent from a certain simulation time, where all states are already numerically settled.
I tried to two approaches:
1) Firstly, where I have stored and reloaded some initial states manually, like described under the following link Save State Information - MATLAB & Simulink - MathWorks Deutschland.
2) Secondly, where I have stored and reloaded some initial states automatically, with the command "setInitialState()" like described in Creates SimulationInput objects to make changes to a model for multiple or individual simulations - MATLAB - MathWorks Deutschland
With both approaches I was able to run my system without errors, but it seems that Matlab is not considering my RL_agent. During training the reward is always reward=0. When the training was finished and I tried to simulate my system with "environment = sim(env,agent,simOpts)", the environment variable was empty. When I repeated this procedure without any initial states, hence "in the normal way starting from t=0 sec", everything has worked fine.
Is there a procedure how I can train my RL_agent together with my environment starting always at some initial state?
0 Comments
Answers (0)
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!