How can I import multiple .CSV files in MATLAB and process the data of each file?

45 views (last 30 days)
Hi all. I have a foulder containing 1488 csv files of the form: 2016-05-day-T-hour-h-minute.csv (that have 3 columns and 2304 rows with just a numerical value in each cell). I wanted Matlab to read each csv file and each cell so I can process and analyse these numbers. I have a problem with the variable F, as a result when I check what F is I can see that it displays the name of the last file in the foulder. As a consequence, the analysis of the data is (eta,north and west) is carried out only using the numbers in the last file.
After running the code, by typing F i get the following:
F=
'Files\2016\May\2016-05-31T23h36.csv'
which is my last file. How can I make matlab to read the cells in ALL the files so that I can create a vector that I can use to do my analysis?
In other words, I need to apply the eta,north,west equations to all files and not just the last one.
Thanks
P = 'Files\2016\May';
S = dir(fullfile(P,'*.csv'));
for k = 1:numel(S)
F = fullfile(P,S(k).name);
S(k).data = csvread(F,1);
end
%This analyses all the data
eta = S(k).data(:,1)/100; % Displacement 1
north = S(k).data(:,2)/100; % Displacement 2
west = S(k).data(:,3)/100; % Displacement 3
end

Accepted Answer

Jon
Jon on 22 Jan 2021
Edited: Jon on 22 Jan 2021
It looks like you are on the right track with your looping on the file names.
It is a little confusing adding the data as an additional field to your existing file list structure S, but it isn't wrong.
I'm not sure what csvread(F,1) does, in the documentation they describe using either just one argument, or additional arguments for starting row and column. I don't know what it does with only two arguments.
After the loop completes, k will equal the number of files, so your lines, eta = S(k).data ... west = S(k).data will only compute values for the last file.
You could either use a loop to assign those or I think this would work
% put all of the data into a n by 3 matrix
data = [vertcat(S(:).data)]
% extract the column data
eta = data(:,1)/100
north = data(:,2)/100
west = data(:,3)/100
  11 Comments
Christian Scalia
Christian Scalia on 25 Jan 2021
P = 'Files\2016\May';
S = dir(fullfile(P,'*.csv'));
for k = 1:numel(S)
F = fullfile(P,S(k).name);
M = csvread(F,1);
eta = M(:,1)/100; % Displacement 1
north = M(:,2)/100; % Displacement 2
west = M(:,3)/100; % Displacement 3
... whatever you want to do with these arrays.
end
@Stephen Cobeldick this only gives me the results for the last file contained in the folder. By looking at the workspace I can see that
F='Files\2016\May\2016-05-01T01h06.csv'
which is the last file contained in the folder.
Stephen23
Stephen23 on 25 Jan 2021
Edited: Stephen23 on 25 Jan 2021
"...this only gives me the results for the last file contained in the folder."
Yes, because you have not made any attempt to store your data in the loop. My last comment explained that.
"By looking at the workspace I can see that ... which is the last file contained in the folder."
That is exactly what is expected: on each loop iteration F contains the name of the current file being processed. After the loop has completed, it contains the name of the last file processed. That is how loops work.
It is irrelevant to the issue that you are having, which is that you are not storing your data during the loop.
Have a look at the code in your original question. Note how the index k is used to store the imported file data on each loop iteration. Thus after the loop all of the data is stored, can be used later.
Question: does your code do anything like that with its output/results? (hint: no).

Sign in to comment.

More Answers (0)

Categories

Find more on Large Files and Big Data in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!