Difference between simple matlabpool parfor and batch parfor.

12 views (last 30 days)
I see a huge difference between apparently similar approaches and would like to understand what's happening and why the discrepancy.
I just created this M-file
% code
parfor i=1:10024
A(i) = sin(i*2*pi/1024);
And tried to run it using 2 different approaches:
matlabpool open 100
%connected to 100 labs.
Elapsed time is 0.652927 seconds.
2. batch('mywave','matlabpool',100)
MaximumNumberOfWorkers : 101 MinimumNumberOfWorkers : 101 Elapsed time is 166.722475 seconds
My configuration is with 132 processors (workers) passing all the validatiions. connected to 100 labs.
With resource list parameter as : -l walltime=12:00:00 -l nodes=11:ppn=12
(i have a linux cluster with 12 nodes & 12 ppn , with shared file system)
Thanks in advance.

Accepted Answer

Edric Ellis
Edric Ellis on 31 Jul 2012
The amount of computation that your PARFOR loop contains is far too small to benefit from launching such a large matlabpool. Launching 100 workers and having them all set up communications etc. takes a long time - much longer than it takes for MATLAB to run the body of your FOR loop. You can only expect PARFOR to speed up your program when the amount of work to be done exceeds the overheads of setting up the workers and launching the parallel loop.
nah on 2 Aug 2012
Thanks Edric and Walter for your answers. Yes, I understand the overhead could be more than the gain one could get from parfor (it's parallelization 101). But, my problem is more with distributing the calculation. I just have some 1000s of short calculations to do & been wondering how I could achieve it. Parfor is the easiet & obvious solution, but it does behave oddly. My question was more about the differences between the same matlabpool job (similar overheads) but having different execution times when one is submitted as a batch job. And the answers here do not yet address those differences.

Sign in to comment.

More Answers (0)


Find more on Cluster Configuration in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!