Read only specific columns and rows of text file with various length

Hey there,

I am currently trying to set up a script in which I can read in data of different types from txt files. My textfiles are outputs of a Resecs Geoelectrics Software with header and a predefined formatting.

The only varying part is the length of the test, hence the amount of rows in the different text files.

I would now like to be able to read in any of the data files and get the necessary data no matter how long the array will get.

The formatting looks as follows:

Injection timings : 
on  off  delay_ui  delay_m  interval_m  subinterval_m  per_meas  smpl_rate
128  128  64  24  48  1  1  1 
Type  C1(x)  C1(y)  C1(z)  C2(x)  C2(y)  C2(z)  P1(x)  P1(y)  P1(z)  P2(x)  P2(y)  P2(z)  MP(x)  MP(y)  MP(z)  K  a  b  I  U  Rho  Sp  M  Z  P  D  M0  M1  M2  M3  M4  Inj.U  Time
MCF  -2.00  0.00  0.00  2.00  0.00  0.00  -1.00  0.00  0.00  1.00  0.00  0.00  0.00  0.00  0.00  1.00  0.00  0.00  34.76  117.95  3.39  -214.23  -0.08  3.42  19.19  0.88  -0.08  0.00  0.00  0.00  0.00  60  18.07.2017 20:22:40
MCF  -3.00  0.00  0.00  3.00  0.00  0.00  -1.00  0.00  0.00  1.00  0.00  0.00  0.00  0.00  0.00  1.00  0.00  0.00  26.22  33.44  1.27  -212.38  0.24  1.28  9.57  3.32  0.24  0.00  0.00  0.00  0.00  60  18.07.2017 20:22:42
...

Each row starts with a string (MCF), followed by data. The last column is in Date format.

So far I couldn't find a way to get the data columns/rows only.

What I tried so far:

• dlmread and skip the header lines

A = dlmread('VES.txt','',5,1);

However, for some reason this doesn't read the full text file but only 20 of 36 rows. In addition, it inserts a row of zeros after each data row.

I also tried to look into textscan but couldn't figure out how to get the data in a non-cell format.

If anybody could help me, I'd be happy.

Answer 1

[dpb]

dlmread isn't able to handle the file with character data in the first column correctly -- that's where the blank line is coming from every other record.

It's a pet peeve of mine about textscan knowing nothing but cell arrays, but that's easy enough to work around; it's just an annoyance to have to always do so...

You might have luck with just importdata but I'm not positive on its behavior for mixed data types either; I just never think about it so haven't used it enough...

fmt=['%s' repmat('%f',1,30) '{dd.MMM.yyyy HH.mm.SS}D'];  % format string for textscan
c=textscan(fid,fmt,'headerlines',4,'collectoutput',1);   % read the file

You'll have a cellstr array for the first column type that's string, a cell array Nx30 for the data and a subsequent cell array of datetime values for the dates. It is easy enough to

data=cell2mat(c(2));

to recast the numeric cell data to a regular double array.

If the file data aren't too long, you might consider using readtable which would take the same format string but you could then use the header row to build named variables although with 30 different variables you'd probably want to read and build the table using the various x,y,z coordinate vectors as Nx3 arrays of the generic name rather than three separate variables.