How to average 'data' column of [Latitude, Longitude, Data] data from multiple file and save the average of 'data' column with using the same [Latitude, Longitude]

Hi, I have multiple .txt file with consist of column matrix [Latitude, Longitude, Data], where all the Latitude and Longitude is same for all the files. I just want to average all the 'Data' column from each files, and then save it into one new .asc file that consist of [Latitude, Longitude, Data(average)]. I have coding, but I stuck during the average process and could not get the file into [Latitude, Longitude, Data(average)].

clc
clear
format short
outpath = 'C:\Users\mniza\Desktop\SMOS_SSS_CODING\3_DataGridding_SSS_SMOS_Climatology';
[filename,pathName] = uigetfile('*.txt','Select the text-files', 'MultiSelect','on');

filename=cellstr(filename);
C=cell(size(filename));
for k=1:length(filename)
    C{k}=textread(fullfile(pathName,filename{k}));
end

average(k)=mean(C(:,3));

if ~exist (outpath)
    mkdir (outpath)
end
save('DataAverage.asc', 'average(k)','-ASCII');

Answer 1

[Cris LaPierre]

Is the firle format the same in all your files? If so, I would use a filedatastore to load all the data into a single table, and then groupsummary to compute averages by lat/lon.You can see an example of how to use one to do this in this video from the Data Processing with MATLAB specialization on Coursera.

Here is the final code from that example. You can modify this to work for your data.

myDataStore = fileDatastore("Example*.txt","ReadFcn",@readtable,"UniformRead",true);
data = readall(myDataStore);
data.Properties.VariableNames = ["Lat","Lon","Data"]

data = 87702×3 table
    Lat    Lon      Data 
    ___    ____    ______

     0       95       NaN
     0     95.1    33.723
     0     95.2    33.763
     0     95.3    33.802
     0     95.4    33.734
     0     95.5     33.62
     0     95.6     33.49
     0     95.7    33.313
     0     95.8    33.136
     0     95.9    33.123
     0       96    33.151
     0     96.1    33.183
     0     96.2    33.223
     0     96.3    33.264
     0     96.4    33.207
     0     96.5    33.137

LtLnAvg = groupsummary(data,["Lat","Lon"],"mean","Data")

LtLnAvg = 43851×4 table
    Lat    Lon     GroupCount    mean_Data
    ___    ____    __________    _________

     0       95        2             NaN  
     0     95.1        2          33.662  
     0     95.2        2          33.642  
     0     95.3        2          33.621  
     0     95.4        2          33.582  
     0     95.5        2          33.534  
     0     95.6        2          33.479  
     0     95.7        2          33.403  
     0     95.8        2          33.326  
     0     95.9        2          33.408  
     0       96        2           33.53  
     0     96.1        2          33.609  
     0     96.2        2            33.6  
     0     96.3        2          33.592  
     0     96.4        2          33.532  
     0     96.5        2          33.464  

writetable(data,'DataAverage.txt')

This is more a template than a working example, so please adapt the code to work for your data files.

[MAT NIZAM UTI]

Thanks for the help. I tried with the code you provide. My only concern, do I have to put all the filename into the "Example*.txt" in myDataStore = fileDatastore("Example*.txt","ReadFcn",@readtable,"UniformRead",true);

Because the average data is still the same as the example1 data.

[Cris LaPierre]

The code is written with the expectation that all the text files you want to import start with "Examples". You will just need to update the format to match your actual file naming convention.

The code is written expecting the files to be in your current folder. You can update that by prepending the folder path to the file name. You can use fullfile for that.