read tiff image in parfor

Hi,

Below is the code I am trying to run. I can only run it with for loop. I alwyas get an "Invalid TIFF file ID" error when trying to use parfor loop even I tried to use 1 worker. I am using Matlab 2020a.

filename = 'myfile.tif';

warning('off','all')

tstack = Tiff(filename);

[I,J] = size(tstack.read());

K = length(imfinfo(filename));

data = zeros(I,J,K);

data(:,:,1) = tstack.read();

parfor n = 2:100

disp(int2str(n));

tstack.setDirectory(n)

data(:,:,n) = tstack.read();

end

warning('on','all')

[Walter Roberson]

Note that reading an image file in parallel will tend to encounter drive contention for files larger than the file system cache, unless the frame processing is relatively long.

All image file types supported by MATLAB store their data as a single file. Typically that would be on one drive of one controller, but on clusters it is plausible that the drives are RAID for increased performance. But even that is nearly always a single controller (drives crosswired to two controllers do exist, for redundancy if nothing else, but they are rare.)

Answer 1

[Raymond Norris]

Hi Wenjie,

The file descriptor in the process of your MATLAB client (outside of the parfor) does not transfer to the processes of the workers running in the parfor. You'll need to open the file locally on each of the workers.

Raymond

[Raymond Norris]

Thinking more about this, you can probably do this with spmd

filename = 'myfile.tif';

% Store old warning
W = warning('off','all');

% Initialize data
spmd
  tstack = Tiff(filename);
  [I,J] = size(tstack.read());
  K = length(imfinfo(filename));
  data = zeros(I,J,K);
  data(:,:,1)  = tstack.read();
end

spmd
  for n = drange(2:K)
    disp(int2str(n))
    tstack.setDirectory(n)
    data(:,:,n) = tstack.read();
  end
end

% Gather the results from the Workers
gdata = gather(data{:});

% Reset warning
warning(W)

You can do this all in one spmd, I just broke it up as you had it. You might need to gather the data (gdata) at some point, but check if whatever you're doing can operate on just data instead. Instead of 100, I used K, but maybe you meant 100.

[Wenjie Li]

Thanks for the answer! Actually using spmd didn't make this running fast (slower than using for loop).

I also tried to add tstack = Tiff(filename); in the parfor loop, which works. But this creats the issue of repeatedly reading the tiff file and also makes the job running slower than just using the for loop.

Is there anyway that I could pass the tstack information to each worker just once in the parfor loop?

Answer 2

[Zhiheng Zhao]

Try this.

filename = 'myfile.tif';
num_of_workers = 6;
warning('off','all') 
tstack  = Tiff(filename);
[I,J] = size(tstack.read());
K = length(imfinfo(filename));
data = zeros(I,J,K);
data(:,:,1)  = tstack.read();
mission_per_worker = ceil(K/num_of_workers);
tic
parfor pf = 1:num_of_workers
    tstack  = Tiff(filename);
    if pf * mission_per_worker > K
        for n = (pf-1)*mission_per_worker:K
            disp(int2str(n));
            frameRead(data,tstack,n);
        end
    else
        for n = (pf-1)*mission_per_worker:pf*mission_per_worker
            disp(int2str(n));
            frameRead(data,tstack,n);
        end
    end
    
end
toc
warning('on','all')
function  frameRead(data,tstack,n)
tstack.setDirectory(n)
data(:,:,n) = tstack.read();    
end

[Raymond Norris]

Hi Zhiheng,

I don't see how this would work because frameRead is passed a matrix (not an object), so how is the assignment in frameRead sticking? And even if it did, within the parfor, it's a broadcast variable, not a sliced or reduction. That said, I didn't test this, just suprised it worked.

Raymond

[Wenjie Li]

I tested this and it actual doesn't work...

[Zhiheng Zhao]

There was one minor error in the for loop, should be "(pf-1)*mission_per_worker + 1:pf*mission_per_worker". However, the overall speed is not as expected.