# Using MATLAB Live Scripts to Teach Optimal Control and Dynamic Programming Online

By Duarte J. Antunes, Eindhoven University of Technology

When my course Optimal Control and Dynamic Programming transitioned to an online model during the COVID-19 pandemic, MATLAB® live scripts proved invaluable. They enabled me to teach complex concepts in a way that was interactive, engaging, and better suited to online learning than a traditional lecture-with-slides approach.

With live scripts I can combine formatted text, images, hyperlinks, equations, and code (Figure 1). Because the scripts are live, students can tune parameters or modify the code and immediately see how their changes affect the output. The process engages students in active learning and encourages exploration of the material. In a post-course student survey, 97% of the respondents reported that live scripts either “helped” or were “very useful” in completing assignments.

Figure 1. A live script used in Optimal Control and Dynamic Programming.

## Limitations of Lectures-with-Slides

When I taught Optimal Control and Dynamic Programming in the past, my lectures incorporated slides in which I introduced a new concept, described how it could be implemented, and then showed some sample output. For example, in a lesson on linear-quadratic regulators (LQRs) I would start with a discussion of the relevant equations as well as special cases and assumptions, show the MATLAB implementation, and then show the output of the code for various parameter values (Figure 2). Later, I would assign exercises that asked students to solve a set of optimal control problems using MATLAB. The exercises included a sample solution with a snapshot of MATLAB code.

Figure 2. Slide series on LQR. Clockwise from top left: introduction, sample application, the corresponding MATLAB code, and plots of the results.

I always felt that this approach was a clumsy way of presenting material. The code on the slide was just a snapshot of the implementation, and I had to include numerous plots to show the effect of different parameter values on the output. Motivated by these issues, I decided to convert all the course material involving MATLAB to live scripts.

## From Slides to Live Scripts

Altogether, I converted 16 lectures, 30 MATLAB examples, 30 MATLAB exercises, and 10 graded assignments. For each course topic, I copied the introductory discussion text into the live script and structured it with headings, bullet points, and hyperlinks to background material. I used the Live Editor to type in LaTeX commands and preview the corresponding equation before placing the equation in the live script (Figure 3).

Figure 3. Inserting a LaTeX equation into a Live Script and previewing the output.

In some cases, the equations were quite long, and I did not have the LaTeX code. In these cases I used Mathpix OCR software to scan written or printed equations and generate the LaTeX for me.

Once I had created the live scripts, I shared them with students via MATLAB Drive cloud-based storage. Students downloaded the scripts for each lecture and ran them either on their laptops or in a browser using MATLAB Online (Figure 4). With MATLAB Online, students can run and edit scripts from any computer or tablet, even machines that do not have MATLAB installed.

Figure 4. The live script for the LQR control lesson running in a browser with MATLAB Online. Students can change parameter values with the slider on line 29 and see the resulting effects in the plots on the right.

## Live Script Assignments and Automatic Grading

During lectures, students often experiment with the corresponding live scripts, varying parameter values and checking the results. I encourage this kind of experimentation because it helps them to absorb the new material.

Live scripts are equally valuable in helping students with assignments. One assignment, for example, involves developing a control algorithm for a buzz wire game. In the real-world version of the game, the objective is to pass a ring along a curved wire without letting the ring touch the wire at any point. For the assignment, I reduced it to a two-dimensional problem in which the algorithm must control the ring’s trajectory by calculating the offset and angle of the ring for each time step as it moved down the wire (Figure 5) so that it reaches the end of the wire in a minimum time.

Figure 5. A visualization of the buzz wire assignment in which a ring (blue lines) must pass along a wire (red curve) without touching it.

Students tend to have a lot of questions when I explain this assignment. When I provide them with a live script of a sample implementation, they are able to answer many of these questions themselves; they can run the script offline, change control parameters, see exactly what constitutes a touch of the ring making contact with wire, and so on.

To simplify grading, I show the students exactly the function they needed to develop within the live script to complete the assignment, with clearly defined inputs and outputs. I then use this specific function to automatically grade the students’ solution with MATLAB Grader, which is integrated with Canvas, our learning management system.

## Planned Improvements and Other Uses for Live Scripts

I continue to add enhancements as I develop new live scripts for my courses. These enhancements include incorporating more animations. I did this for an assignment in which the students had to develop an optimal control policy for the ghosts chasing a Pac-Man™-like character in a pursuit-evasion game (Figure 6). I found that the animation helped the students understand the assignment much better than my textual description could do.

Figure 6. Pursuit-evasion game in which students develop optimal control algorithms for the ghost (red) to pursue the main character (yellow).

In response to student feedback, I’m also adding more in-line comments to the MATLAB code. I’ve noticed that today’s students depend on seeing the code, and code comments, to cement their understanding. In the past, I tended to de-emphasize implementation details, but more and more students understand the language of programming and find it easier to grasp certain concepts when they are expressed in commented code rather than in our natural language. One student noted, “It really helps to see how a problem/solution changes when you decide to change a certain parameter or piece of code. Normally, examples are limited by the one variation that is shown. Besides that, the actual implementation of some of the theory is hard to imagine sometimes, while such a piece of code can help that process immensely.”

One of the principal advantages of live scripts is that they are easy to learn and use. I’ve found that most students are up and running after about five minutes of looking at the documentation. In another course that I teach on signals and systems, I asked students to submit a report on their projects. Many decided on their own to create their final reports as live scripts and use live scripts to explain their project through a video