Slow indexing into local parts of distributed cell arrays in parallel computing toolbox

I have a code similar in structure to the following example:

p = 3;
N = p*10000;

if ~matlabpool('size'),matlabpool('open','local',p);end

spmd,
  dist = codistributor('1d',1);
  XX = codistributed.cell(N,1,dist);
  g_indices = globalIndices(XX,1);
end

tic
spmd,
  for k=drange(1:N),
    % perform some work
    XX(k,1) = {1};
  end
end
toc

tic
spmd,
  for k = g_indices,
    % work
    XX(k,1) = {1};
  end
end
toc

on my computer i get the following output:

Elapsed time is 26.234271 seconds.
Elapsed time is 30.065625 seconds.

I am aware that there is significant overhead associated with spmd, and distributed arrays, but this performance is far worse than I personally expected. Am I doing this indexing wrong, or is there some reason why this is so slow? The reason I expected this to be faster is that I am only indexing the local parts of each array. (Unless I there is an error somewhere?)

I would greatly appreciate all input, or any suggestions on this.

Regards, Anders

Answer 1

[Konrad Malkowski]

Hi Anders,

The best way of achieving high performance in your example is to operate directly on the LocalParts of the codistributed array as demonstrated in the attached code example below.

Note that additional performance benefits are achieved by moving the code from a script to a function, and there is some additional performance gains by moving the code inside of spmd into function calls as well.

However, most performance is obtained by simply operating on local parts and then building the new codistributed array that contains updated data.

function questionCode
p = 2;
N = p*10000;
if ~matlabpool('size'),matlabpool('open','local',p);end
spmd,
  dist = codistributor('1d',1);
  XX = codistributed.cell(N,1,dist);
  g_indices = globalIndices(XX,1);
end
tic
spmd,
   func1(XX, N)
end
toc
tic
spmd,
   func2(XX, g_indices)
end
toc
tic
spmd
    LP = getLocalPart(XX);
    dist = getCodistributor(XX);
    for i = 1:N/numlabs
        LP(i) = {1};
    end
    XX = codistributed.build(LP, dist, 'noCommunication');
end
toc
end

function func1(XX, N)
for k=drange(1:N),
    % perform some work
    XX(k,1) = {1};
  end
end

function func2(XX, g_indices)
  for k = g_indices,
    % work
    XX(k,1) = {1};
  end
end

On my machine I get the following results:

Elapsed time is 35.154233 seconds.
Elapsed time is 37.656905 seconds.
Elapsed time is 0.106672 seconds.

[Anders Hoff]

this works really well. thanks a lot!

[rzh huang]

Great!