Parallel processing to update plot

Hello all and thanks in advance for reading this question.

I need to run a simulation in a simulink loop. The loop involves that at each iteration a plot is updated according to the results. This makes the simulation to run to slow. I want to use parallel processing to update the plot according to the simulation results by using another core of my CPU. For this purpose, I have been playing around using the batch command and parfor command without much success.

I wonder if anybody can give me some hints on what is the best way to address this. It would be like a separate process that at each iteration reads some numerical values from the simulink loop, and updates the plot using another core. I guess the communication between processes could be achieved with some global variables. So basically it will be a separate script to update the plot according to these variables. And this script needs to run in such a way that it wont not slow down noticeably the simulink loop and vice-versa.

Thanks a lot, Mindra

[mindra]

Just an additional comment: some of you may give me the advice of not refreshing the plot at each iteration to make things faster or to check on some simulink optimization tips for speed performance. I have already done that but the improvement was not enough.

Answer 1

[Edric Ellis]

It's possible that you might be able to use parfeval to achieve this. You cannot use global variables for the communication, as pointed out by Walter. parfeval is very similar to batch, except that it uses workers from your parallel pool - and therefore the overheads of running stuff are typically much lower. For instance, you could keep the Simulink model open on the worker between runs.

Following on from Walter's suggestion, you need a function that calls sim. Then, on your MATLAB client, you call parfeval to invoke that function on a worker. The client then waits for completion of the work - perhaps using fetchNext, and then retrieves the results. The client must update the plot.

There's an example here in the doc of this sort of thing.

[mindra]

Thanks a lot Eldric. I will try this as well.

[mindra]

Hi again Eldric. I have tried your approach and I have managed to make simulink run through parfevel allowing the client to proceed doing other stuff. This theoretically would solve the problem, but I don't see that there is any way to communicate between client and worker until the worker finishes. So there is no way to update the plot in real time. fetchNext assumes that one worker has already stopped. If I would 'split' the simulink task into different parfeval calls, still each of the calls would have to wait until the previous one finishes. So it would not help either, because the client will be waiting and not plotting.

Do you have any further suggestion regarding this? Or maybe I did not understand fully what needs to be done.

Thanks again!

[Walter Roberson]

parfeval and fetchNext does not make any assumption about the order the iterations terminate in, you just get the next available result. You would update your plot in scattered order. The result of the chunk should be whatever information you need to plot the iteration.

[Walter Roberson]

fetchNext does not wait for the worker to end: it waits until a Future is available and each future becomes available when the requested function call finishes. The worker that finished the evaluation transfers the result back and then proceeds to the next function call that is queued. So you would loop placing one call to your function that calls sim() for every parameter combination that you want to compute and then you loop harvesting and plotting the results in whatever order they come back.

[mindra]

Ohh I see what you mean now. The problem is that my simulink simulation has to be executed linearly. I just need to plot what happens in real time in this simulation. So I can't divide it in different parts unless they are executed sequentially.

[Walter Roberson]

Make the first parfeval call. Enter a loop of waiting for a result, copying off whatever info you will need to plot with, start another parfeval for the next iteration, then while it is executing update the plot with the results just recorded.

Unless your plot is very dense or takes a lot of calculation after the sim() call has returned, chances are that the overhead of using the parallel worker will exceed the amount of work needed for the plotting and that you will find this arrangement to be slower than if you had not done parallel work at all.

[mindra]

Yes I think you are right. I will have to find another approach or to use a faster computer for this task.

Thanks a lot to both of you for your time!

Answer 2

[Walter Roberson]

You cannot use global variables to communicate between workers. Global variables are not transferred to the workers or back from the workers.

Workers cannot update the graphics display.

See https://www.mathworks.com/matlabcentral/newsreader/view_thread/258469

Batch... that might work.

[mindra]

Thank you for your suggestion. So do you suggest that I open a batch process to update the plot and run simulink in normal mode, or do you suggest two batch processes one to update the plot and another to run the simulink loop?

Thanks.

[Walter Roberson]

The batch processes would run sim() and return results (pulling them out of the function workspace and returning them if necessary.) The process that created the batch would be monitoring the returns and plotting the values as they become available.

I have not looked at this much, but I do know that it is possible to examine the returns before the entire batch is complete. It might take me a while to find appropriate documentation (I have not read much about batch.)

[mindra]

Thank you very much Walter.

Answer 3

[kendur]

I think one possible solution you can apply is that you use two processes: First one takes data and save into files with names followed by order numbers such as "name_00.dat, name_01.dat, ..." after every data block transferred, the other one checks if the order number is updated by finding the file name and refresh the plot.

Answer 4

[Emory Salberg]

Workaround for passing variables. Look up "matlab memory mapping". Works well for communicating between applications. May help you too.

I was struggling for a while on a similar problem and this was the best solution. Only takes ~1ms to grab data.

%%%%%%%%%%

            disp("IN PID2...");
            tempv(1,3:5) = [R2_phidot_error(pidIdx2,3:5)];
            out_msg = sprintf("%.1f," , tempv(1,:));
            out_msg = strip(out_msg,'right',','); % strip final comma
            out_msg = strcat("<"+out_msg,">");
            disp('PID2_out: '+out_msg);        
            fileID = fopen('fromMatlab2.txt','w');
            fwrite(fileID, out_msg,'char');
            fclose(fileID);
            disp("EXIT PID2...");

%%%%%%%%%%%%%

function msgReceived2(msg)
    try
        global readIdx2 finishR2 sOfVector
        global R2_SumT R2_ENC R2_E1 R2_E2 R2_E3 R2_Q3 R2_Q4 R2_Q5 R2_FM
        global Mymsg

        msgin = msg;
        Mymsg = msgin;

        msgin2 = char(extractBetween(msgin,"<",">"));
        msginCell = str2double(strsplit(msgin2,','));
        R2_SumT(readIdx2) = msginCell(1);
        R2_E1(readIdx2) = msginCell(2);
        R2_E2(readIdx2) = msginCell(3);
        R2_E3(readIdx2) = msginCell(4);
        R2_ENC(readIdx2,1:3) = [R2_E1(readIdx2),R2_E2(readIdx2),R2_E3(readIdx2)];
        R2_Q3(readIdx2) = msginCell(5);
        R2_Q4(readIdx2) = msginCell(6);
        R2_Q5(readIdx2) = msginCell(7);
        R2_FM(readIdx2) = msginCell(8);
        
        if(readIdx2>=sOfVector)
            finishR2 = 1;
        end
        readIdx2 = readIdx2 + 1;
    catch
        wut = 0;
    end
end

[Emory Salberg]

You will be essentially writing a vecotor with known delimiters to a .txt file and then reading in values from another file. The file is never 'opened' so it saves a lot of time as you are directly reading/writing from the memory.

Also, forgot to enclude how to read the files. Its usuful to keep all of your files within the same workspace as well. Note, i was passing data to and from a python terminal running in Visual Studious, hence the 'fromPython2.txt'.

msgin=textread('fromPython2.txt','%s');

Best,