Why parfor cannot handle this

7 views (last 30 days)
Hello all,
I have faced an issue when using parfor, and I would like to know why parfor cannot handle such logic.
My aim from this question is to just know the cause of the issue, which would help me to not do it again in the future. My code is huge to be pasted here, so I will write a toy example:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% A toy example
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
clear allocatedVar
myFlag = 0;
loopMax = 4;
if( myFlag == 1)
% huge memory allocation goes here, many variables, etc
allocatedVar = cell(loopMax, 1);
for i = 1 : loopMax
allocatedVar{i, 1} = ones(10, 1);
end
end
parfor i = 1 : loopMax
% some code, bla bla bla
if( myFlag == 1)
b = zeros(10, 1);
for j = 1 : 10
b(j) = allocatedVar{i, 1}(3, 1);
% some code to save some results
end
end
% some code, bla bla bla
c = 4;
end
The error is
Error: Invalid syntax for calling function 'allocatedVar' on the path. Use a valid syntax or explicitly
initialize 'allocatedVar' to make it a variable.
So, the idea is that when the input "myFlag = 1", I will execute a specific code section. While when it does not equal 1, I do not want to even allocate the variables needed to execute this code section; I'm using this to save memory space.
When using a for loop, there is no issue, while when using parfor, MATLAB seems to not accept such a thing.
I can solve the issue by simply commenting the code section when I do not want to execute it, but then this is not practical (commenting and not commenting when I want to run the simulation). So, using a flag would definitely be a better solution.
Regards,

Accepted Answer

Raymond Norris
Raymond Norris on 24 Sep 2020
Hi Hussein,
All MATLAB tokens must either be functions or variables. Keep in mind, regardless of whether or not code will be executed, MATLAB needs to serialize the entire parfor block to each of the workers. When it sees allocatedVars, it needs to inform the workers if it's a function or variable. If a function, it might also be serialized and passed down (or it could be on the worker's path already).
Because MATLAB doesn't see allocatedVar as a variable, it assumes it's a function. But calling a function with {} is invalid (hence the first part of the error string). MATLAB then suggests initialize it to be a variable if that's what you meant (which it is), overriding the default assumption of a function (though the suggestion won't work here).
I wonder if using parallel.pool.Constant could help to avoid generating the data on the client to begin with. From the doc
Make parallel.pool.Constant from Composite
This example shows how to build large data sets as a Composite on pool workers inside an spmd block, and then use that data as a parallel.pool.Constant inside a parfor-loop.
spmd
if labindex == 1
x = labBroadcast(1,rand(5000));
else
x = labBroadcast(1);
end
end
xc = parallel.pool.Constant(x);
parfor idx = 1:10
s(idx) = sum(xc.Value(:,idx));
end
Thanks,
Raymond

More Answers (0)

Categories

Find more on Parallel for-Loops (parfor) in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!