Array changes size when passed to a reduction function in the parfor loop?
Show older comments
I ran into a weird problem when I used numerical arrays or structure arrays inside a parfor loop. Below is my main function:
comm_struct=struct('field1',0,'field2',zeros(10,1),'field3',sparse(zeros(10,10)));
a(1:100,1)=comm_struct;
b(1:10,1)=comm_struct;
disp(size(a))
disp(size(b))
parfor ct=1:12
b=max_select_2(b,a);
end
where max_select_2 is
function y=max_select_2(b,a)
if size(b,1)~=10||size(a,1)~=100
error('size of a is %d,size of b is %d',size(a,1),size(b,1)),
end
d(1:110,1)=struct('field1',0,'field2',zeros(10,1),'field3',sparse(zeros(10,10)));
d(1:10,1)=b;
d(11:110,1)=a;
[~,sort_idx]=sort([d.field1],'descend');
y=d(sort_idx(1:10));
If I replace the parfor with for and run on my local matlab, no problem, but if I use parfor and run the code on a cluster, disp(size(a)) and disp(size(b)) generates output
100 1
10 1
which is expected but I also got the error message
"size of a is 100,size of b is 100"
which means for some reason when b is passed to the function max_select_2, its size changed from 10 to 100? I doubt that it has something to do with how matlab handles the structure array but I don't really know what's going on and how to fix it...
For further information, I tweaked the code to help to diagnose.
If I change a(1:100,1)=comm_struct to a(1:10,1)=comm_struct;, and max_select_2 becomes
function y=max_select_2(b,a)
disp(size(b,1));
d(1:20,1)=struct('field1',0,'field2',zeros(10,1),'field3',sparse(zeros(10,10)));
d(1:10,1)=b;
d(11:20,1)=a;
[~,sort_idx]=sort([d.field1],'descend');
y=d(sort_idx(1:10));
the output from disp(size(a)) and disp(size(b)) in the main body are
10 1
10 1
and disp(size(b,1)) in max_select_2 generates 10 which is the expected result!
If I set the size of b to 100 and size of a to 10 in the main function and max_select_2 becomes
function y=max_select_2(b,a)
if size(b,1)~=100||size(a,1)~=10
error('size of a is %d,size of b is %d',size(a,1),size(b,1)),
end
d(1:110,1)=struct('field1',0,'field2',zeros(10,1),'field3',sparse(zeros(10,10)));
d(1:100,1)=b;
d(101:110,1)=a;
[~,sort_idx]=sort([d.field1],'descend');
y=d(sort_idx(1:100));
the outputs from disp(size(a)) and disp(size(b)) in the main function are
10 1
100 1
But I get the error message
size of a is 10,size of b is 10
If I modify the parfor loop in the main function as
parfor ct=1:12
c=max_select_2(b,a);
end
so that b is no longer a reduction variable, I don't get the error message anymore.
So apparently the size of the structure array b changes to that of a when b is a reduction variable. Is that expected? Could it be a bug?
Also, I had the same problem even if a and b are just regular arrays. i.e., I got the error message
size of a is 10,size of b is 10
when the main function is
a=zeros(10,1);
b=zeros(100,1);
disp(size(a));
disp(size(b));
parfor ct=1:12
b=max_select_array(b,a);
end
and max_select_array is
function y=max_select_array(b,a)
if size(b,1)~=100||size(a,1)~=10
error('size of a is %d,size of b is %d',size(a,1),size(b,1)),
end
d=zeros(110,1);
d(1:100,1)=b;
d(101:110,1)=a;
[~,sort_idx]=sort(d,'descend');
y=d(sort_idx(1:100));
Anybody has encountered similar problems? I'm using Matlab 2015a. Thanks!
Li
Answers (0)
Categories
Find more on Signal Integrity Kits for Industry Standards in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!