Open multiple files in Matlab

Hi everyone, I have multiple files that I want to open using fopen. The files have a similar pattern, I have tried to use a for loop as follows, but it does not work. Any ideas how to open each file. Thanks in advance

for ii = 0:12
file = fprintf('population_%d.dat', ii); % -----> File names
generations_fid = fopen(file); % Question ???
matrix = {};  
while ~feof(generations_fid)
   generations = cell2mat(textscan(generations_fid, repmat('%f', 1, (3))));
   if isempty(generations)
       fgetl(generations_fid);
   else
       matrix{end+1} = generations; 
   end
end
end

Answer 1

[Ameer Hamza]

Do something like the following. It will automatically all the files with extension .dat and open them one by one.

files = dir('*.dat');
for i=1:length(files)
    generations_fid = fopen(files(i).name);
    .... your code
    fclose(generations_fid);
end

[Yro]

Thanks for your reply. I did what you recommend, but with this code I can only store the data from the last file (.dat). Any ideas how to store the data of all files in an array.

Thanks in advance.

files = dir('*.dat');
for i=1:length(files)
generations_fid = fopen(files(i).name);
matrices = {};  
while ~feof(generations_fid)
   generations = cell2mat(textscan(generations_fid, repmat('%f', 1, (3))));
   if isempty(generations)
       fgetl(generations_fid);
   else
       matrices{end+1} = generations; 
   end
end
end
fclose(generations_fid);

[Ameer Hamza]

You are initializing the variable matrices each iteration. You need to move it outside the for loop. I cannot know for sure without having an example data, but the following code might work as expected.

files = dir('*.dat');
matrices = {};
for i=1:length(files)
    generations_fid = fopen(files(i).name);
    
    while ~feof(generations_fid)
        generations = cell2mat(textscan(generations_fid, repmat('%f', 1, (3))));
        if isempty(generations)
            fgetl(generations_fid);
        else
            matrices{end+1} = generations;
        end
    end
    
    fclose(generations_fid);
end

I also changed the position of fclose so that every opened file get closed.

[Walter Roberson]

All_mat{i} = matrices;

[Yro]

Thanks for your help but it doesn't work i keep getting only data from last file. This is the file (.dat) structure:

generations:

-2.77700899786000	-4.93992382659000	-16.7733637870000
-2.77700899786000	-4.91070963520000	-16.6909250060000
-2.77700899786000	-4.86867920101000	-16.5732361280000
-2.77700899786000	-4.85338387006000	-16.5306747510000
...

There are 94 files in total, I want to make an arrangement to store all the data. Thanks

[Walter Roberson]

N = 13;
All_mat = cell(N,1);
for ii = 0:N-1
    file = fprintf('population_%d.dat', ii); % -----> File names
    generations_fid = fopen(file); % Question ???
    matrix = {};  
    while ~feof(generations_fid)
       generations = cell2mat(textscan(generations_fid, repmat('%f', 1, (3))));
       if isempty(generations)
           fgetl(generations_fid);
       else
           matrix{end+1} = generations; 
       end
    end
    All_mat{ii+1} = matrix;
end

This would give you a cell array of cell arrays. I do not just use a single cell array because I can tell from your code that you expect there to be multiple blocks for numbers in the code that are separated by a text line and that you want to store the blocks separately but together for each file.

[Ameer Hamza]

Walter, but will not the textscan read the entire file in a single go? Is there a situation when the while loop needs to run for more than one iteration.

Yrobel, you mentioned the output of generations variable. What is the output of matrices variable?

[Walter Roberson]

Imagine a file that had multiple "generations:" headers. The textscan %f format would stop reading when it hit the "g" because "g" is not valid in a number. The data up to there would be returned by textscan and stored by the code. The loop would say it is not end of file so textscan would be called again. This time the "g" is the first thing in the buffer so textscan returns no data. The loop would detect the isempty and would do an fgetl(), reading in the "generations" line and throwing it away. The loop would then continue, potentially reading another block of data.

Thus what the code does is to read as much data as possible and store it, then throw away the rest of the line that stopped it from reading, and then start reading again, storing the new group separately.

The code is thus correct to initialize "matrix" in the place it does, because matrix is responsible for all the numeric data from one file, as blocks rather than put together into a single numeric matrix. The problem with the code was that it was not storing those cell arrays for each input file.

[Ameer Hamza]

Walter, Thanks for clarifying. I wasn't fully aware of the working of the textscan.