Import range of files based on file name
1 view (last 30 days)
Show older comments
I have a folder containing around 300,000 files. I don't need to import all the files.
Problem: How can I import a range of files based on specific file name?
This is an extension on the problem I asked in another post, link: https://www.mathworks.com/matlabcentral/answers/587738-import-files-based-on-file-name#answer_488336
Problem example:
In the picture below I have a section of the files, which are all in the same folder. I only want to import the .data files. But I don't need all the .data files to be import only the last 5 of every serie.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/355705/image.jpeg)
% Select only .data file from the last time step of each simulation
Folder = uigetdir;
files = dir(fullfile(Folder,'*.data*'));
spl = regexp({files.name},'\.data\.','split','once');
spl = vertcat(spl{:});
vec = str2double(spl(:,2));
[~,idx] = sort(vec);
[~,idy,idz] = unique(spl(idx,1),'last');
out = {files(idx(idy)).name};
expData = cell(length(out),1);
for i = 1:length(out)
fid = fopen(fullfile(Folder,out{i}),'r');
%% Reading the data
% Read all the data from the file
dataRead = textscan(fid,'%f %f %f %f %f %f %f %f %f %f %f %f %f %f','HeaderLines',1);
end
This code only extracts the last .data file of every serie, but as I explained I want to have the last 5 files of every serie.
0 Comments
Accepted Answer
Cris LaPierre
on 4 Sep 2020
Acknowledging there are a lot of ways to do this, here's how I thought to do it. It borrows from the code you've shared.
% Select only .data file from the last time step of each simulation
Folder = "datafiles";
files = dir(fullfile(Folder,'*.data*'));
spl = regexp({files.name},'\.data\.','split','once')';
% Convert the cell to a table so that you can sort and group two different data types
tbl = splitvars(cell2table(spl),1,"NewVariableNames",["Name","Ext"]);
tbl.Ext = str2double(tbl.Ext);
% this gives you the unique experiments and the last extension in the series
exp = groupsummary(tbl,"Name","max","Ext");
for n = 1:height(exp)
for i = 4:-1:0
% Use the Name and Ext values to rebuild the file names
fid = fopen(fullfile(Folder,exp.Name(n)+".data." + string(exp.max_Ext(n)-i)))
% Read all the data from the file
dataRead = textscan(fid,'%f %f %f %f %f %f %f %f %f %f %f %f %f %f','HeaderLines',1);
end
end
13 Comments
Cris LaPierre
on 9 Sep 2020
Edited: Cris LaPierre
on 9 Sep 2020
Ah, I misunderstood how the code ran.
How to modify it? Replace the code that opens every file with the code I shared already that opens the last 5 in every series.
rhoPart = 2500;
files = dir(fullfile(uigetdir,'*.data*'));
spl = regexp({files.name},'\.data\.','split','once')';
% Convert the cell to a table so that you can sort and group two different data types
tbl = splitvars(cell2table(spl),1,"NewVariableNames",["Name","Ext"]);
tbl.Ext = str2double(tbl.Ext);
% this gives you the unique experiments and the last extension in the series
exp = groupsummary(tbl,"Name","max","Ext");
c=1;
for n = 1:height(exp)
for i = 4:-1:0
% Use the Name and Ext values to rebuild the file names
fid = fopen(fullfile(Folder,exp.Name(n)+".data." + string(exp.max_Ext(n)-i)))
%% Reading the data
dataRead = textscan(fid,'%f %f %f %f %f %f %f %f %f %f %f %f %f %f','HeaderLines',1);
frewind(fid);
% Write headerline N, time, xmin, ymin, zmin, xmax, ymax, zmax
runData{c} = strsplit(fgetl(fid), {' ','\t'});
% Write only the x, y, and z components of the particles, particle radius,
% z component+ particle radius and volume of the particle
expData{c} = [dataRead{1}(:,1) dataRead{2}(:,1) dataRead{3}(:,1) dataRead{7}(:,1) dataRead{3}(:,1)+dataRead{7}(:,1) rhoPart*(4/3)*pi*(dataRead{7}(:,1).^3)];
% Write only the vx,vy,vz of the particles and magnitude
velData{c} = [dataRead{4}(:,1) dataRead{5}(:,1) dataRead{6}(:,1) sqrt(dataRead{4}(:,1).^2 + dataRead{5}(:,1).^2 + dataRead{6}(:,1).^2)];
fclose(fid);
c=c+1;
end
end
More Answers (0)
See Also
Categories
Find more on Ordinary Differential Equations in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!