Read a file with textscan.

I have a text file that I'm trying to read with textscan. The top of my file looks like

######################################################################
Test File Name: MNTFEPLANC-122-202203171547   
Timestamp: 202203171547   
Test Type: FILL Valve Selction: BOTH Valve Cycle Time: 2

######################################################################
20220317154734,  0.0,  0.0,  0.0,  0.0,17714,17449,20464,17041,17028, 0.000000,0.000000,0.000000
20220317154734,  0.0,  0.0,  0.0,  0.0,17714,17449,20464,17041,17028, 0.000000,0.000000,0.000000
20220317154734,  0.0,  0.0,  0.0,  0.0,17714,17449,20464,17041,17028, 0.000000,0.000000,0.000000
20220317154734,  0.0,  0.0,  0.0,  0.0,17714,17449,20464,17041,17028, 0.000000,0.000000,0.000000
20220317154734,  0.0,  0.0,  0.0,  0.0,17714,17449,20464,17041,17028, 0.000000,0.000000,0.000000
20220317154734,  0.0,  0.0,  0.0,  0.0,17714,17449,20464,17041,17028, 0.000000,0.000000,0.000000
20220317154734,  0.0,  0.0,  0.0,  0.0,17714,17449,20464,17041,17028, 0.000000,0.000000,0.000000
20220317154734,  0.0,  0.0,  0.0,  0.0,17714,17449,20464,17041,17028, 0.000000,0.000000,0.000000

I open the file using

raw_data_file = fopen(file_name,'rt') % The full path of "file_name" has been ommitted for brevity.

which yields

raw_data_file =

    17

I read the file using

fe_data_read = textscan(raw_data_file,'%f%f%f%f%f%f%f%f%f%f%f%f%f','Delimiter',',');   

which yields

fe_data_read =

  1×13 cell array

  Columns 1 through 10

    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}

  Columns 11 through 13

    {0×1 double}    {0×1 double}    {0×1 double}

I tried to create a variable with values being the first column using

test_date_time = fe_data_read{1}

which yields

test_date_time =

  0×1 empty double column vector

Why is "test_date_time" not being populated with the values in the first column of my text file? What am I doing wrong?

Answer 1

[Jan]

The file starts with this line:

######################################################################

This does not match the format string '%f%f%f%f%f%f%f%f%f%f%f%f%f' or the Delimiter=',' in any way. Therefore textscan replies an empty output.

You have to tell textscan, that the first 6 lines are a header and have to be skipped. Use the 'Headerlines" argument.

By the wya, do you have a reason to open the file in the slower text mode?

[Allen Hammack]

Thank you for your feedback. I added "HeaderLines" like you suggested,

fe_data_read = ...
        textscan(raw_data_file,'%f%f%f%f%f%f%f%f%f%f%f%f%f','HeaderLines',6,'Delimiter',',')

which produces the same output:

fe_data_read =

  1×13 cell array

  Columns 1 through 8

    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}

  Columns 9 through 13

    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}    {0×1 double}

I don't see what I'm doing wrong. Do you notice anything?

I'm using the slower text mode because this code has worked for me in the past. Do you have a suggestion for a different way?

[Allen Hammack]

The problem I'm running into seems to be related to the carriage returns in my input file, which aren't translated when I copied a section of my input file in my original question. I made some changes to that file, and your recommendation works perfectly. Thank you!

I am still interested in knowing how to read this file other than the "slower text mode" you referred to in your original response.

[Stephen23]

"The problem I'm running into seems to be related to the carriage returns in my input file, which aren't translated when I copied a section of my input file in my original question."

@Allen Hammack: If you want us to check your original file, then you need to provide us with your original file.

Please upload your original file by clicking the paperclip button.

[Allen Hammack]

@Stephen Here is my data file.

[Stephen23]

That file has a mix of newline characters, but Walter Roberson's code imports it without any problems:

fnm = 'orig_data_trim.txt';
opt = detectImportOptions(fnm, 'HeaderLines', 6);
opt = setvartype(opt, 1, 'datetime');
opt = setvaropts(opt, 1, 'InputFormat', 'uuuuMMddHHmmss');
T = readtable(fnm, opt)

T = 29×13 table
         Var1         Var2    Var3    Var4    Var5    Var6     Var7     Var8     Var9     Var10    Var11    Var12    Var13
    ______________    ____    ____    ____    ____    _____    _____    _____    _____    _____    _____    _____    _____

    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17713    17450    20463    17042    17028      0        0        0  
    20220317154734     0       0       0       0      17712    17450    20462    17043    17028      0        0        0  
    20220317154734     0       0       0       0      17712    17450    20462    17043    17028      0        0        0  
    20220317154734     0       0       0       0      17712    17450    20462    17043    17028      0        0        0  
    20220317154734     0       0       0       0      17712    17450    20462    17043    17028      0        0        0  
    20220317154734     0       0       0       0      17712    17450    20462    17043    17028      0        0        0  

Answer 2

[Stephen23]

opt = {'Delimiter',',','Headerlines',6,'CollectOutput',true};
fmt = ['%{uuuuMMddHHmmss}D',repmat('%f',1,12)];
fid = fopen('new.txt','rt');
tmp = textscan(fid,fmt,opt{:});
fclose(fid);
tmp{1}

ans = 8×1 datetime array
   20220317154734
   20220317154734
   20220317154734
   20220317154734
   20220317154734
   20220317154734
   20220317154734
   20220317154734

tmp{2}

ans = 8×12

0 0 0 0 17714 17449 20464 17041 17028 0 0 0 0 0 0 0 17714 17449 20464 17041 17028 0 0 0 0 0 0 0 17714 17449 20464 17041 17028 0 0 0 0 0 0 0 17714 17449 20464 17041 17028 0 0 0 0 0 0 0 17714 17449 20464 17041 17028 0 0 0 0 0 0 0 17714 17449 20464 17041 17028 0 0 0 0 0 0 0 17714 17449 20464 17041 17028 0 0 0 0 0 0 0 17714 17449 20464 17041 17028 0 0 0

[Walter Roberson]

filename = 'https://www.mathworks.com/matlabcentral/answers/uploaded_files/941499/new.txt';
opt = detectImportOptions(filename, 'HeaderLines', 6);
opt = setvartype(opt, 1, 'datetime');
opt = setvaropts(opt, 1, 'InputFormat', 'uuuuMMddHHmmss');

T = readtable(filename, opt)

T = 8×13 table
         Var1         Var2    Var3    Var4    Var5    Var6     Var7     Var8     Var9     Var10    Var11    Var12    Var13
    ______________    ____    ____    ____    ____    _____    _____    _____    _____    _____    _____    _____    _____

    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0  
    20220317154734     0       0       0       0      17714    17449    20464    17041    17028      0        0        0