How to delete headers from different row positions other than at the top - MATLAB

I have a text file that contains multiple headers. It looks like this:

Date,time,*10ms,%%,%%,%%,%%,%%,%%,%%,%%,%%,%%,DETAILS.txt;D;%%;10 11/08/19,13:19:28,03,446,0,545.75,0,6,0,0,0,14,0 11/08/19,13:19:29,05,446,0,549.25,1.9,6,102,1,0,0,0 11/08/19,13:19:30,07,446,0,549.5,1.9,6,102,1,0,0,0 11/08/19,13:19:31,09,446,0,548.75,1.9,6,102,1,0,0,0 
.
.
.
Date,time,*10ms,%%,%%,%%,%%,%%,%%,%%,%%,%%,%%,DETAILS.txt;D;%%;10 11/08/19,13:19:28,03,446,0,545.75,0,6,0,0,0,14,0 11/08/19,13:19:29,05,446,0,549.25,1.9,6,102,1,0,0,0 11/08/19,13:19:30,07,446,0,549.5,1.9,6,102,1,0,0,0 11/08/19,13:19:31,09,446,0,548.75,1.9,6,102,1,0,0,0
.
.
and so on...

I would like to write a code that deletes these header-rows and stores the rest of the data to a new file.

Could anyone help with this?

Kind regards,

Tamara

[Daniel Shub]

What differentiates the header from the non-header? Do all headers start with Date,time, ...? Do all data start with 11/08/19 or a date? Are there substantially more data lines than header lines? How many lines are you talking about (1e3, 1e6, 1e10)?

[Fangjun Jiang]

Take a look at the post again. Does each line match the line in your text file? Apply {}code format to it and make sure each line matches each line in your text file. That's important for the right solution.

[Tamara]

@Daniel: To answer your questions: All headers start with Date, time,.... and yes all data start with a date. There are certainly more data lines than header lines, header lines range from 3-8 per file. In total the amount of lines can go from about 2000-5000 or so...

Answer 1

[Fangjun Jiang]

fid=fopen('test.txt','rt');
a=textscan(fid,['%s%s',repmat('%f',1,11)],'delimiter',',','commentstyle','Date,time,');
fclose(fid)

>> a{1}

ans = 

      '11/08/19'
      '11/08/19'
      '11/08/19'
      '11/08/19'
      '11/08/19'
      '11/08/19'

>> a{13}

ans =

       0
       0
       0
       0
       0
       0

[Tamara]

Thank you Fangjun Jiang, this was what I was looking for!

But can you do one more favor :D...could you like explain to me what repmat() and commentstyle are actually doing?

Because I looked up repmat() and did not understand the explanation Matlab-help gave me...

[Fangjun Jiang]

b=repmat(a,m,n) creates a large matrix b by repeating matrix a m times in row and n times in column. Run a few examples to see the effect. It works on char array too.

a=[1 2;3 4];

b=repmat(a,3,1)

c=repmat(a,1,3)

repmat('abc',1,3)

repmat('abc',3,1)

[Fangjun Jiang]

The 'commentstyle' parameter of the textscan() function is used to specify what lines in the text files are treated as comments thus ignore them. Type help textscan and find the explanation of different way to specify comment style. In this example, the leading keyword 'Date,time,' is used to identify any line that starts with that keyword is treated as comments.

[Tamara]

Thank you so much for the explaination Fangjun Jiang!!