How to reformat data from multiple files in a folder

I have written code to open a command window allowing the user to choose a folder. Then The files from this folder will be opened. I have this part of the code working. Now I need to include a code I had previously written to reformat the data in each file.

myFolder = uigetdir('C:\Users\c13459232\Documents\MATLAB');
if ~isdir(myFolder)
errorMessage = sprintf('Error: the following folder does not exist: \n%s', myFolder);
uiwait(warndlg(errorMessage));
return;
end
filePattern = fullfile(myFolder, '*.asc');
Files = dir(filePattern);
for k = 1 : length(Files)
baseFileName = Files(k).name;
FileName = fullfile(myFolder, baseFileName);
fprintf(1, 'Now reading %s\n', FileName);

This is the first section of the code I have written to open the files

Cell = textscan( FileName, '%d', 'delimiter', ';');
Data = cell2mat(Cell);
N = 1024;
Finish = reshape(Data, N, [])';
end

This code reformats the data the way I want it. I know I will need to write the code to create 50 new files (e.g. Finish_01, Finish_02, etc). I'm not sure how to compose this. Any tips would be greatly appreciated.

Answer 1

[Frank Macias-Escriva]

Before going to the generation of the new files, let's fix a detail with the reading. I don't know if it is working for you, but the textscan function expects a file identifier or a string. In your case, you want to read a file, so you should use a file identifier, I mean, you should replace your line:

 Cell = textscan( FileName, '%d', 'delimiter', ';');

with the following 3 lines:

fid = fopen(FileName);
Cell = textscan( fid, '%d', 'delimiter', ';');
fclose(fid);

Now, for the matter of saving several files in an output folder, you should add this before the for loop:

 outFolder = fullfile(myFolder, 'output');
 mkdir(outFolder);

then use that created folder for saving files. For that you should add these 3 lines as the last lines of your for loop:

newFileName = fullfile(outFolder, ['finish_' sprintf('%03d', k) '.asc']); % finish_001.asc, finish_002.asc, ...
dlmwrite(newFileName, Finish, ';');
disp([':: Now writing ' newFileName]);

The full (merged) code would be:

myFolder = uigetdir('C:\Users\c13459232\Documents\MATLAB');
if ~isdir(myFolder)
  errorMessage = sprintf('Error: the following folder does not exist: \n%s', myFolder);
  uiwait(warndlg(errorMessage));
  return;
end
outFolder = fullfile(myFolder, 'output');
mkdir(outFolder);
filePattern = fullfile(myFolder, '*.asc');
Files = dir(filePattern);
for k = 1 : length(Files)
  baseFileName = Files(k).name;
  FileName = fullfile(myFolder, baseFileName);
  fprintf(1, 'Now reading %s\n', FileName);
  fid = fopen(FileName);
  Cell = textscan( fid, '%d', 'delimiter', ';');
  fclose(fid);
  Data = cell2mat(Cell);
  N = 1024;
  Finish = reshape(Data, N, [])';
  newFileName = fullfile(outFolder, ['finish_' sprintf('%03d', k) '.asc']); % finish_001.asc, finish_002.asc, ...
  dlmwrite(newFileName, Finish, ';');
  disp([':: Now writing ' newFileName]);
end

I'm attaching a code compare image, your code on Left, the new code on Right, so you can easily view the differences:

Hope this help,

Frank

[Aaron Smith]

Thanks Frank, that works perfectly. I was wondering, If I wanted to save each of the new files to the workspace as a Matrix, what output function should I use. Would the amount of data be too big?

[Frank Macias-Escriva]

Your welcome!

For storing the data in your workspace you can add this line before closing de for loop:

 eval(['finish_' sprintf('%03d', k) ' = Finish;']);

If you don't like eval (I don't like it), then you can have all the data in a cell matrix with this statement instead:

 finishCell{k} = Finish;

But don't forget to create the cell matrix before the for loop in order to avoid that cell matrix changing size on every loop cycle. You can do that with this line:

 finishCell = cell(length(Files), 1);

Hope this answers your second question, I'm happy to help. If the original answer or any other one solved your issue, please mark it as accepted. This helps keep the focus on older MATLAB Central questions which still don't have answers. Thanks!

[Aaron Smith]

Hey, Thanks. Both of these codes work well. It saves all 50 matrices in a single cell array. I have been looking at ways to navigate through files in a GUI window while they are all open in the workspace (after using the eval line) but would it be possible to open the matrices using contourf and then navigate through them in a GUI window. Contourf works with the files when using the eval version of the code, though it is very slow. Binning the data before plotting makes the plot appear much more quickly. Anyway, back to my question; Is it possible to plot the data from the cell array? Thanks