How to make Matlab MEX application to be real time one on SMP general purpose multicore platform

Background:

Developing, for example, a wireless modem is a complex multi skill know-how set. It starts from creating algorithms simulation model, where Matlab is often used as a tool. The ultimate target is to create a chip that functionally does all as the simulation, but it is small peace of HW, it does the job quicker and consumes less power.

There are a number of ways to get that end goal. For one of them, there is a strong requirement to do and to test that simulation model in real time, on non-necessarily embedded platform, in first iteration, but on some general purpose multicore SMP platform. Only after making the simulation model to work in real time on that kind non embedded platform –we look in details what is the ideal embedded platform required to fit the application. This kind of process have multiple advantages. It allows the algorithm people, who do not necessarily have real time embedded systems profile, to build the real time model relatively quickly and to give quick answer to number of most important questions. Basically the 1st stage provides high level of confidence that the development process is on the right track that major unknows have been answered, so the 2nd stage is a steady low risk path.

Matlab already provides some pathway towards this process of development:

• The Coder allows building MEX application where generally slow Matlab control code gets compiled in much faster C/C++ application.
• The Embedded Coder supports SIMD instruction set so that the DSP code that gets vectorized on Matlab, gets also vectorized in MEX application, since the basic Coder itself does not generate SIMD code
• The Fixed Point Designer allows Fixed Point arithmetic that is common requirement for an embedded platform.

However, the area where I do not clearly see direction is how to make the MEX application as multithreading/multicore application, which is precondition that the application can run on real time. Only if we use multicores in optimal way we can achieve to process the natural incoming data rate in real time.

The Parallel Computing Toolbox offers a number of mechanism to use the parallelism (https://uk.mathworks.com/help/parallel-computing/choosing-a-parallel-computing-solution.html). But I cannot find:

1. How that parallelism get transferred to the MEX application
2. How the MEX application gets transferred to multicore platform

For No.1, I see that Matlab offers generation of MEX application to multithreads: https://uk.mathworks.com/help/dsp/ref/dspunfold.html.

However, I am not sure how much is that helpful, since if I do the architecture of the application, I need to control that rather than letting Matlab to do it under the hood. Hence, I expect that I need to break the application to multiple MEX files by specifying the functions I need to create the MEX files for.

But No.2 is the main question mark.

The scheduling mechanism https://uk.mathworks.com/help/parallel-computing/how-parallel-computing-products-run-a-job.html#f3-6782, seems the closest match, but not sure that the scheduling is the right thing for the real time application. The real time application threads, that are supposed to be entry to the Scheduler, need to have assigned strict processing deadlines that the scheduler is obligated to meet, but looking into the Job function interfaces, I cannot see that the processing deadline is passed to the Scheduler. It is not clear how the Scheduler breaks the Jobs to the Tasks.

• So, my conclusion is that the Parallel Toolbox Scheduler is sort of “Fair Scheduler” rather than real time scheduler? In other words, the Parallel Toolbox is not designed for real time applications?
• The question is also - are the Job interface functions supported by the Coder (Embedded Coder) in first place? The aim is to run the MEX application in real time (I do not expect the Matlab code, as scripting code, can be run quick enough for real time – at least I do not see that my code is close to that).

Please is there any way to run Matlab is real time on multicore powerful SMP platform?

Ideal solution, is my view, would be to support so called “Processor Affinity” like Linux does where we can chose which thread to get statically mapped to which core, but I have not seen that something like that is supported.

I see there is an option to replace Matlab Scheduler with 3rd party scheduler https://uk.mathworks.com/help/matlab-parallel-server/integrate-matlab-with-third-party-schedulers.html, but that looks complicated operation to do, requires reinstallation of Matlab. But surely since Matlab knows that is installed on a Linux machine for example, it should be possible adding API that wraps directly Linux OS part related to scheduling and simple enable flag to use it. Hence, using Linux Scheduler rather than Matlab own scheduler would be a dynamic easy option. That could enable the path to use Matlab in real time?

Anyway, there could be already some solution that would allow running Matlab (MEX) application in real time that I could not find - so I would apreciate some input please.

Answer 1

[Walter Roberson]

Mathwork's only real-time product is Simulink Real-TIme (which is designed to run on Speedgoat hardware.)

https://www.mathworks.com/company/newsletters/articles/concurrent-execution-with-simulink-real-time-and-multicore-target-hardware.html

MATLAB itself, and Parallel Computing Toolbox in particular are not designed for real-time.

[Dusko Vujadinovic]

I emhasised that I do not expect that Matlab itself works in real time. However, the MEX applications as generated C code should work in real time - why not?

[Walter Roberson]

There are a small number of ways a program can become real time:

• it can be analyzed by a dedicated real-time system to try to figure out the critical paths, and be compiled entirely in terms of real-time library calls, to produce a program that runs on an operating system that is dedicated to real-time work, such as on speedgoat hardware -- not Linux or Windows or MacOS
• it can be analyzed by a compiler for a generate multitasking operating system, with the compiler linking against libraries of real-time extensions to the general multitasking operating -- that is, the multi-tasking operating system has to have real-time extensions designed into it; a small number of Linux versions have this, and some other operating systems do as well -- not Windows
• It can be built with regular tools for a multitasking operating system, with it being given privileges to run with dedicated resources at higher priority than anything else on the system, on an operating system that offers this as an option, such as SGI IRIX used to offer -- not Windows

Just linking against a threaded library does not make a program work in real time. Threaded libraries just mean that the program gets more efficient access to switch tasks when the operating system decides to schedule you.

For example when a real-time program asks the operating system to read from disk, and the hardware meets the minimum requirements, then a real-time operating system would guarantee that the data would be returned within a bounded amount of time (or be told that it does not exist.) Whereas a merely threaded program might have to wait 15 or 20 minutes while a Disk Defragmenter happened to run, or might have to wait several hours while the operating system swapped in and out a program that was using hard drives as virtual memory.

[Dusko Vujadinovic]

I am very familiar with general factors associated with real time, but that was not my question here. The problem is, that does not seem, that there is a link with the Parallel Computing Toolbox and the Code generation, i.e. it looks that the Parallel Computing Toolbox can be used with Matlab only.

If we want to use Matlab Coder - we cannot use Parallel Computing Toolbox?

If I use Matlab Coder (Embedded Coder), that means that I have aspiration to run the code in real time and the natural step is also to use the Parallel Computing Toolbox to speed up the execution of the generated code further, but as far as I can see, that is not possible?

[Walter Roberson]

MATLAB Coder and Embedded Coder do not support any real-time systems. Simulink Real-time is the only Mathworks product that supports real-time systems, and that is entirely within the context of a real-time operating system.

Simulink Real-time is already designed to analyze Simulink blocks to group computations into tasks that can be run independently at different rates with bounded times and with methods already in place to control what should be done if a task exceeds its time allotment. Simulink Real-time already has methods to estimate time requirements of a task.

The Parallel Computing Toolbox has no methods to analyze time requirements or prioritize computations or deal with missed deadlines or deal with multiple rates. It cannot, for example, know anything about starting a computation on a processor but suspending it after a time interval to turn the processor over to dealing with a sample having become available, then resuming the computation — which is a typical operation for a real-time system as real-time systems pre-plan based upon knowledge of resource requirements.

The Parallel Computing Toolbox was never designed for real-time operations. The Parallel Computing Toolbox was designed for multi-core operations each with independent flow of control and low interactions.

[Dusko Vujadinovic]

"MATLAB Coder and Embedded Coder do not support any real-time systems"

With a bit of experience how to write Matlab code that suits MATLAB Coder - the Coder generates relatively efficeint code that can be run on Real time, according my experience. The only problem is that it does not generate the vectorised code, but as far as I understand, the Embedded Coder is there to address that problem (have not tried that one yet). If something is for Embedded environment - it has to be surely for real time to?

Thanks for the info for "Simulink Real-time". I will have a look.

[Walter Roberson]

Embedded is not the same as real-time.

I have access to the logs of an access door that can be remotely operated. The control system is an embedded system. Occasionally the logs it generates are not written for an hour or more: it may be embedded but that function is not real-time.

[Dusko Vujadinovic]

"Embedded is not the same as real-time."

I understand that is strictly speaking true if looked through strict rule of law definition. I am not going to hold anybody, you want to officially declare that the MATLAB Coder and Embedded Coder are not for real time, so that if customers like me have problems you simply say - sorry we do not guarantee that. I have got the point.

But I just need to see how I can pragmatically move forward. I have put lots of efforts to generate the code, to be able to afford to simply abandon the approach because Mathworks officially clams that MATLAB coder and Embedded Coder are not "Real time".

There is no strict definition of "Real time". For what I am trying, the MATLAB Coder should be generating decent good enough "real time" code with the exception that it does not explore SIMD capabilities, which makes it slow for DSP code. As I have said - the generating code does not have huge real time penalties if you pay attention how you write your Matlab code.

The embedded coder has ben advertised that it offers the SIMD instructions set for number of processor platforms - which is to speed up the code, so in my view, that is primarily real time accelerator on high scale, isn't it?

Here, it has been stated: https://uk.mathworks.com/help/dsp/simd-code-generation.html

"Certain algorithms in the DSP System Toolbox, such as FIR filter and LMS filter, are computationally intensive. The Intel AVX2 SIMD intrinsics significantly improve the performance of the code generated using these algorithms on Intel platforms, in most cases meeting or exceeding the simulation performance."

Hence, it do not seem right to me that you say that Embedded Coder is not targeting real time area,

My estimation is, but I cannot be sure before I try, that the Embedded Coder should provide enough real time boost so that my real time application should work. At the moment, my understanding is that Matlab works quicker in the DSP area since it does utilise the SIMD, but the control generated code is quicker than than appropriate Matlab code, so my MEX application is not quicker than Matlab one. So, it seems to me that if I use embedded coder, my MEX application should work much quicker- hopefully in real time.

I understood already that the Parallel Processing Toolbox is not useful for me, which is unfortunate. If you generate the code you should have covered the Parallel Processing Toolbox and everything should be simply "real time". Otherwise you put lots of faith in something and you get disapointed.

[Walter Roberson]

you want to officially declare

I believe you have overlooked the fact that I do not work for Mathworks, and so cannot make any kind of official declaration. I did, though, do some real-time work in the early 1980s.

Earlier you talked about generating MEX, but you also talk about Embedded Coder. Embedded Coder is for embedded systems, which either have no operating system or have some kind of custom or real-time operating system.

But "A MEX function is a MATLAB executable." -- that is, it is to be called from inside a full MATLAB session, or from inside something built with MATLAB Compiler (and so is using MCR to provide the execution engine.) MATLAB itself and MATLAB Compiler are restricted to generating code for use on systems that MATLAB itself can execute on -- which is to say Linux, MacOS, or Windows.

"The embedded coder has ben advertised that it offers the SIMD instructions set for number of processor platforms - which is to speed up the code, so in my view, that is primarily real time accelerator on high scale, isn't it?"

NO!

Suppose you have a set of cooks (processors), and you have a series of meals to make, some due "as soon as practical" (get the appetizers in front of the people quickly), some due in groups (entres for a table should ideally all be ready at the same time), and some due according to schedules (catering business.)

The Parallel Computing Toolbox just gets told the total number of meals to make, and is not given any information about deadlines or how long each one takes to make, and is responsible for handing out work to each chef. If it hands chefs a list of items to prepare, the chef finishes everything on the list before going back for more work... regardless of whether the meals were ladles of soup, or making an entire roast hog. The same chef might happen to get assigned the roast hog and the all-day spaghetti-sauce, and the Parallel Computing toolbox doesn't know or care, and does not offer any way for the chefs to say "Hold on, give this to someone else, I'm going to be awhile on what I am doing." PCT itself doesn't care. Each individual chef cooks as quickly as they have resources for, but there is no inherent coordination or attempt to allocate resources to meet deadlines.

A real-time system, on the other hand, receives the deadlines and cooking time information, and is responsible for figuring out how to make everything work. It might be given more tasks over time, and it is responsible for looking through the work it knows it has to do and either accepting the task (and then ensuring it gets finished on time), or else rejecting it as not feasible. A real-time system does not try to make each chef continuously busy: it cares about the outcomes, about the meals being ready on time... and will, if necessary, interrupt a chef to do something higher priority.

The two situations are not the same.

Which one do you need? Well, you mentioned a wireless modem. On a wireless modem, anything to do with bit timing is Real-Time work. And there are hard-deadlines for some kinds of responses, after which the protocol might assume that your node is dead (or the link is blocked.) But if you are using something like TCP protocol, you are more often dealing with soft deadlines, where the consequence for not replying in time can be that the other node re-sends the packet. If your time to create a response and have it received by the other node is greater than the retry interval then you run the risk of having the reply work pile up... but it is still more towards the side of "soft real time".

But... if you want to maximize bandwidth, then you start getting back towards "hard real-time" work...

[Dusko Vujadinovic]

I appreciate your time and the conversation, even more if you do not work for Mathworks....

You actually say that the code generated by the Embedded Coder cannot be used in MEX application??? That is very odd restriction in my view and I do not understand where that restriction comes from?

I am aware that SIMD is not a magic concept but it relies on having specific processor architecture that support multiple registers to be able to run multiple data in single instruction cycle. In my example, we have actually a non embedded SMP Intel processor which supports the SIMD AVX instruction set. So that fact that SIMD is supported as part of Embedded Coder only, does not seem it matters? It matters that we have processor that supports the SIMD AVX instruction set which our non embedded platform supports. The embedded coder should not have any dependencies to (real time) operating system as you implicate here:

"Embedded Coder is for embedded systems, which either have no operating system or have some kind of custom or real-time operating system.".

It should care only about specific processor architecture - irrespective being embedded or not as the Matlab Coder does. Just Matlab Coder provides the code that is portable to any general purpose processor whereas the Embedded Coder need to vectorise code for specific processor architecture that supports some of SIMD concepts.

"You can generate single instruction, multiple data (SIMD) code from certain MATLAB functions by using Intel SSE and, if you have Embedded Coder®, Intel AVX technology. SIMD is a computing paradigm in which a single instruction processes multiple data. "

So again I logically cannot see why the Embedded Coder would not be possible for me to use. I would appreciate if you could provide inside to the restrictions that still might exist.

- I have been in wireless modem area and hence hard real time for almost 22 years, so I know very well what hard real time is, difference between embedded and real time, how the scheduling works etc. As far as I can see the discussion about that, just adds to noise to our main discussion - where I am trying to simply get an answer - is it possible to use the Embedded Coder:

1. To produce the code that can me MEX-ed on laptop machine that supports SIMD AVX instruction set - on which I run Matlab.

2. To produce the code that can be run on Intel industrial PC platform that supports SIMD AVX instruction set.

- Please ignore also the Parallel Computing Toolbox - I realised already that I cannot get anything useful from there that I need. I do not think that the requirement there were right. So, again less noise helps to get to the bottom of what I need.

- Also the wireless modem I talk about is not a real modem by modem application run on very powerful general purpose SMP platform - I handle that part well - I do not tnink it is relevant for my questions.

Thanks a lot for your time Walter

[Mikhail]

Hi Dusko,

Let me put it simple for you.

For your use case - forget about Parallel Computing Toolbox, forget about MEX. They are not relevant (I could explain more, but this would just burst into another lengthy comment).

You need to use Embedded Coder, probably with Code Replacement (read about it in docs). What you need then is to take generated code and integrate into your environment. Be it real-time or not, it only depends on you and on your provided (hand-coded) main harness/scheduler etc.

Answer 2

[Dusko Vujadinovic]

Hi Mikhail,

Thanks for trying to help.

I should had narrowed the scope of the question - to prevent that the discussion goes to different directions.

The MEX is absolutely central for what I want. To reiterate, to speed up the Matlab code execution when testing it, since it is very slow (I do not think my code is not efficient though - simply it is too much to do) I need to use MEXing.

Hence it is absolutely essential that I run everything from Matlab environment (forget about the target) . I develop the code in Matlab but when I want to run the regression tests, I run them by running the MEXed code rather than Matlab code itself.

The MEX is the way that I can glue the C code produced by the Matlab Coder that should have been potentially faster than the Matlab code. However that is only true for the control code. For DSP code the bottleneck is that the generated code is not vectorised, so as consequence the MEX application is even a bit slower than when running Matlab code.

One way is to write that code on my own in C and put it to be part as Code Replacement Library as you suggested. However, I do not want to waste my time on that. I would rather use the Embedded Coder to vectorise the code for my host PC that supports SIMD AVX instruction set.

According my understanding that would radically speed up my MEX application. The real time I mentioned here is just an ideal stretch goal, taking into account that the Matlab code that runs the MEX files is really light - but I should not have mentioned that - it just moved the discussion to another direction and made it difficult to follow. So simply ingore any real time.

As I have mentioned - I am aware that I cannot speed up the code by using the Parallel Computing Toolbox - I have to do someting on my own - so should not have mentioned it in first place. However, I did not expect that the Embdeed coder cannot produce the code that can be MEX-ed as Walter suggested - so my original focus was to Parallel Computing Toolbox.

But the question now is very simple.

Can I use code generated from the Embedded Coder to be run as part of MEX application?

Maybe I should open new question stated as simple as that and close this one.