How to combine hundreds of CSV files in Matlab?

Hi all,

I have LOTS of data in CSV files and I'd like to create one massive master CSV file. How can I combine all of the CSV files into one? Each CSV file has a header with time, type of data, etc. They all have the same number of columns but differet numbers of rows.

Answer 1

[Voss]

Here's an example of how to do it (with 2 files):

input_path = '.'; % location of .csv files
output_file = 'result.csv'; % name of file containing combined data

% read each .csv file into a table stored in a cell array of tables 
% called 'all_data':
file_info = dir(fullfile(input_path,'*.csv'));
full_file_names = fullfile(input_path,{file_info.name});
n_files = numel(file_info);
all_data = cell(1,n_files);
for ii = 1:n_files
    all_data{ii} = readtable(full_file_names{ii});
end
% check the tables:
all_data{:}

ans = 4×2 table
    time    type 
    ____    _____

     0      {'A'}
     1      {'B'}
     2      {'C'}
     3      {'B'}

ans = 3×2 table
    time    type 
    ____    _____

     0      {'B'}
     1      {'A'}
     2      {'C'}

% concatenate all the tables into one big table, and write it to
% output_file:
writetable(cat(1,all_data{:}),output_file);

% check that the resulting output file exists:
dir('*.csv')

data_1.csv  data_2.csv  result.csv  

% check the contents of the resulting output file:
readtable(output_file)

ans = 7×2 table
    time    type 
    ____    _____

     0      {'A'}
     1      {'B'}
     2      {'C'}
     3      {'B'}
     0      {'B'}
     1      {'A'}
     2      {'C'}

[SETIAWATI]

I try to use your example for my exercise but once I run the code. The matlab can not read the "writetable(cat(1,all_data{:}),output_file);" part. Can you tell me why? Because I already try everything but I have not figure it out.

[Walter Roberson]

That line would potentially generate an error message if not all of the tables had the same set of variables (in a possibly different order.)

For example if the format of the tables was generally two columns but with possible annotations on the second column, then by chance some of the files might not happen to have any annotations and so would be read as having two columns, whereas the ones with annotations would be read as having three columns or more. The cat(1,all_data{:}) step is going to fail trying to put those various tables together.