Loop files, create a timetable and add file names to variable names

4 views (last 30 days)
Robert
Robert on 8 Sep 2017
Commented: Robert on 9 Sep 2017
Hi,
Wondering if anyone is familiar with how adding file names to the variable names in a loop when creating a timetable?
I have 800 files, each of them have the same number of columns but a different number of rows. Rows are time steps. Columns are variables.
When trying to run the below script I get an error ‘Duplicate variable name’, this happens when building up the final timetable as the variable names are the same. What I need is to add the file name to each column of the timetable, at each loop, plus the variable name Data1, Data2, Data3…. To get something like this…
Timetable…
Time (column 0) %First column in timetable
FileName1_Data1 (Variable Name column 1)
FileName1_Data2 (Variable Name column 2)
FileName1_Data3 (Variable Name column 3)
FileName2_Data1 (Variable Name column 4)
FileName2_Data2 (Variable Name column 5)
FileName2_Data3 (Variable Name column 4)
FileName3_Data1 (Variable Name column 6)
FileName3_Data2 (Variable Name column 7)
FileName3_Data3 (Variable Name column 8)
.
FileName800_Data1 (Variable Name column ...)
FileName800_Data2 (Variable Name column ...)
FileName800_Data3 (Variable Name column ...)
Here is what I have done… …
filePattern = fullfile(myFolder, '*.csv');
theFiles = dir(filePattern); %800 files
FinalM=[]; %Empty matrix
%Create timetable T for synchronization
t1 = datetime(2002,3,10,0,0,0); t2 = datetime(2017,12,31,0,0,0);
Time_All = (t1:t2)';
column=(1:1:5776)';
T = timetable(Time_All,column);
for k = 1 : length(theFiles)
baseFileName = theFiles(k).name;
fullFileName = fullfile(myFolder, baseFileName);
fprintf(1, 'Now reading %s\n', fullFileName);
%Create FileName which I do not know how to use to include in the Variables %(columns) name in the Final timetable
FileName=strcat(theFiles(k).name(1:end-4));
aa=csvread(fullFileName, 2,0);
%Create time from first 6 columns (Year, month, …)
Time=datetime(aa(:,1),aa(:,2),aa(:,3),aa(:,4),aa(:,5),aa(:,6));
%Create Data1, Data2, and Data3
Data1=aa(:,7);
Data2=aa(:,8);
Data3=aa(:,9);
Tab=timetable(Time,Data1, Data2, Data3);
%Obtain some statistics
Dmean=rmmissing(retime(Tab,'Daily',@nanmean));
Drange=rmmissing(retime(Tab,'Daily',@range));
%Synchronize the data based on the largest time series ‘Time_All’ created above
FT=synchronize(Dmean, Drange,Time_All,'firstvalue');
%Create final timetable
FinalM=[FinalM innerjoin(FT,FT)];
end
Any suggestions or comments are welcome.

Accepted Answer

More Answers (1)

Jose Marques
Jose Marques on 9 Sep 2017
Hello Robert.
If I understand correctly, you may try:
FileName=[theFiles(k).name(1) theFiles(k).name(2) theFiles(k).name(3) theFiles(k).name(4)];

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!