Best way to run several scripts at the same time

Hello :)

I want to run the same script but with some differences in some parameters to compare results, and I don't know what is the best way to do this. My initial thought is to open several sessions of MatLab and run the scripts in each session, but I'm not sure if that is the optimum way. I have a PC with Windows and i9 12 core, I want to make sure I’m taking the most advantage out of those darn cores.

So, the question is. How can is run several scripts making sure I’m taking the most advantage out of the processor's cores?

Thanks in advance for your replies.

[Karim]

Hi, not certain if this is the proper way to do it, but perhaps you could use a dummy trick via the parallel computing toolbox :) see below for the concept. As mentioned, this is probaply not the way to go. But it could give you a start.

Also have a look at parfeval.

Good luck!

numFunctions = 12;

parfor i = 1:numFunctions
    switch i
        case 1; [Out1] = CallFunc1(Input);
        case 2; [Out2] = CallFunc2(Input);
        case 3; [Out3] = CallFunc3(Input);
        case 4; [Out4] = CallFunc4(Input);
        case ...
    end
end

[Raymond Norris]

To begin with, it will help if you have the Parallel Computing Toolbox. Working off of @Karim's idea, part of this depends on what you want to do with the output (assuming there is any).

parfeval

parpool(size)
for idx = 1:N
    f(idx) = parfeval(@fcn,numout,X1,..,Xm);
end

You'll want to tweak X1..Xm to the specific input arguments to fcn that differentiates one iteration from the next.

The slight drawback is that to aggregate your results, you need another loop to do so

for idx = 1:N
    [cidx, A(idx)] = f.fetchNext;
end

parfor

An advantage of parfor is that it's a bit easier to assign the data back to the MATLAB client

parpool(size)
parfor idx = 1:N
    A(idx) = fcn(X1,..Xm);
end

I'd be best to post an example of what you're trying to run in order to give a better answer.

[Contoso Donoso]

Hello Raymond. Thanks for your reply.

Actually, I'm not the code owner, so I'm not sure I can share it here, which is also rather large (Sorry about that). But basically, the way it works is that the MatLab code couples with another software. This is: the software solves a mathematical model, and the MatLab uses the results from the software, it solves the equations, and the set of results is fed back to the software to run a new iteration, and this happens for each time-step, meaning that there is an output from the MatLab code at each time-step, and the output file contains multiple cell arrays corresponding to the time-steps within the MatLab code.

Is this information of any help? Do you have any other suggestions? Otherwise, ill try to search more on how to use the parallel computing box and @Karim's idea.

[Walter Roberson]

With a new enough matlab you would be able to use "background threads" to run in parallel instead of using the parallel toolbox. However you would need a more recent version than you have marked.

[Contoso Donoso]

"Background threads" means to open different sessions of MatLab?

[Walter Roberson]

No, it is a feature newer than your release https://www.mathworks.com/help/matlab/background-processing.html

Answer 1

[Raymond Norris]

@Contoso Donoso coming back to this.

As I understand you had

1. External program solve a mathematical model,
2. Which MATLAB uses the results from to solve equations,
3. And the results are feed back to external program

Does the external program launch MATLAB to solve the equations? You mention the ability to launch lots of MATLABs, so I'm gathering these are separate, discrete steps.

How are the results of the solved equations fed back to the external program?

There are several ways MATLAB can make use of the cores on your machine

1. multi-threading (which automagically happens)
2. multiprocessing (which requires a pool of at least one worker)

If you explain a bit more of the workflow, we can provide a better solution.

1. how are "jobs" kicked off (programmatically, with human intervention)
2. is this a closed loop (does everything feed back into itself)
3. at the core of MATLAB, is there a massive for-loop you'd like sped up
4. are jobs serial or parallel (could all MATLABs run at the same time, or do you need to process one-step before processing the next)