Reinforcement Learning does not work with SimEvents.

I was told by Theresa Hubscher-Younger of Mathworks that "Reinforcement Learning" does NOT work with "SimEvents." When will this be fixed? When will release R2020a occur? Thank you.

Averill M. Law

520-795-6265

[e-mail]

[Emmanouil Tzorakoleftherakis]

Hi Averill,

Is there a specific application you have in mind? That would be helpful to see if there are any workarounds.

Answer 1

[Averill Law]

Hi Emmanouil,

I'm confused about how technical support works at Mathworks. At most software companies, you call a particular technical support number. Is this possible with Mathworks?

I purchased the Reinforcement Learning (RL) toolbox with the expectation that it would work with any discrete-event simulation (i.e., all state changes occur at a countable number of separated points in times, namely, where events occur). Will the RL toolbox be fixed so that this will, in fact be the case? Will the fixes be in R2020a? Thank you very much for your assistance.

Averill M. Law

[Image Analyst]

Yes, if you've bought the product you can call them and talk to a real live person. If needed the person can do a webex session with you to see your computer and solve the problem. Call the main MATLAB phone number. Click the phone icon at the top right of this page to get the Contact Us page. Or you can do it through emails to [e-mail] though that will take longer than a phone call.

[Emmanouil Tzorakoleftherakis]

+1 on the tech support part.

In terms of whether Reinforcement Learning Toolbox can work with SimEvents, I have personally created an example that uses both, but in my case learning was time-based and not based on events. I am not sure how you would set up the problem for event-based learning given that these can occur at random time steps - maybe LSTMs or using the Agent block from a trigerred subsystem would work. I think that was probably what Teresa was hinting to.

[Thorsten]

Dear Emmanouil,

can you send me your example, because I am very interested in that topic.

Best regards

Thorsten

[e-mail]

Answer 2

[Averill Law]

Hi Emmanouill,

Will the Reinforcement Learning (RL) toolbox work with ANY discrete-event simulation (DES) provided that the decision points are equal-spaced in time, or will you still need workarounds? Theresa H. said not to count on the toolbox being fixed to work with random decision points any time in the near future? Since the error has been known about for months, why is no one apparently working on fixing the problem? I bought the RL toolbox and SIMULINK specifically to use with DES. Thank you for your assistance.

Averill M. Law

[e-mail]

[Emmanouil Tzorakoleftherakis]

Hi Averill,

I am pretty sure you will not see any error if you try to use the two products together, it's just that some workflows may not be supported yet. This is because the first few releases are more focused on controls problems that have fixed sample time/decision points.

I think putting together a simple example and calling tech support would be the best way to solve any issues you may face. The development team will eventually get involved if necessary.

Answer 3

[Maria del Mar Morejón de Girón Albelda]

Dear Averill,

Did you get an answer? Can both toolboxes be mixed in a simulation?

Thanks in advance and hope you could get further with your project.