How to store 2D arrays in a for loop inside a parfor loop?

1 view (last 30 days)
Mitsu on 2 Nov 2021
Commented: Mitsu on 4 Nov 2021
I compute e.g. a 3x1 array inside the inner for-loop. In each iteration of this loop, the 3x1 array is saved in a matrix.
At the end of each run of the inner for loop, I wish to then store the nx3 array inside a bigger A array of size (n*m)x3, in order.
Therefore, if total = n*m, for each i , the nx3 array would be saved in the position A(total*(i-1)+1:total*i,:).
Example that is not valid:
n = 1e2;
m = 1e6;
total = n*m;
A = zeros(total,3);
parfor i = 1:n
A_temp = zeros(m,3);
for j = 1:m
A_temp(j,:) = rand(1,3);
end
A(total*(i-1)+1:total,:) = A_temp;
end
Why is this not valid? I think that for any combination of i and j, there wouldn't be an overlap.
What is the correct way to do this?
I have tried using a cell as follows, but then I have to "unwrap" the cell and save it as a matrix, and that takes as much time as the parfor for large values of m.
A_cell = cell(n); % rows of 3 columns
parfor i = 1:n
A_temp = zeros(m,3);
for j = 1:m
A_temp(j,:) = rand(1,3);
end
A_cell(i) = A_temp;
end

Edric Ellis on 4 Nov 2021
I think you can do this simply by essentially "reshaping" the problem to satisfy the parfor requirement that the index expression uses the loop index in a sliced manner. (Unfortunately, parfor isn't smart enough to be able to prove that your code doesn't perform illegal indexing, and so you must work around). Some extra tweaking is required to get A in the original form. (Also, I adjusted your original indexing expression, it wasn't quite right)
n = 2;
m = 4;
total = n*m;
% `for` implementation
A = zeros(total,3);
for i = 1:n
A_temp = zeros(m,3);
for j = 1:m
A_temp(j,:) = (1:3) + j*100 + i*1000;
end
A((m*(i-1)+1):(m*i),:) = A_temp;
end
% Permuted `parfor` implementation
A2 = zeros(m, n, 3);
parfor i = 1:n
A_temp = zeros(m,3);
for j = 1:m
A_temp(j,:) = (1:3) + j*100 + i*1000;
end
A2(:,i,:) = A_temp;
end
A3 = reshape(A2, total, 3);
isequal(A,A3)
ans = logical
1
Mitsu on 4 Nov 2021
Thank you. This seems to be a good workaround, it didn't occur to me to use matrices in this way.
Nonetheless, for large values of n and m, the approach with a cell seems to be notably faster.

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!