How to extract specific rows & columns from a text file

I need to extract all the net names and toggle rates from a text file. total 179456 nets. Below screenshot is the few row and columns shown as example

[Akira Agata]

Seems that readtable function can do that. If possible, could you upload a small example of your text data?

[Farhan K]

I provided a screenshot as example

[Farhan K]

Here is an example

Answer 1

[per isakson]

See Read formatted data from text file or string and try

fileID = fopen(filename);
net_names = textscan( fileID, '%s%*f%*f%f%*f', 'Headerlines',3 );
fclose(fileID);

Added later. The script

%%
filename = 'd:\m\cssm\Example.txt';
fileID = fopen(filename);
net_names = textscan( fileID, '%s%*f%*f%f%*f', 'Headerlines',4 );
fclose(fileID);
%%
net_names{1}(1:3,1)
net_names{2}(1:3,1)

outputs

ans =
  3×1 cell array
    {'AES/r8/t0/t3/s4/n885'}
    {'AES/r3/p21[25]'      }
    {'AES/r7/z0[6]'        }
ans =
       0.0001
       0.0001
       0.0001

[Farhan K]

Its working. But is there any way to show both result side by side similar to below:

              Net               ToggleRate
    ________________________    __________

    {'AES/r8/t0/t3/s4/n885'}      0.0001  
    {'AES/r3/p21[25]'      }      0.0001  
    {'AES/r7/z0[6]'        }      0.0001  
    {'AES/r6/p31[6]'       }      0.0001  
    {'AES/r6/p31[23]'      }      0.0001  
    {'AES/s7[102]'         }      0.0001  
    {'AES/r9/z1[30]'       }      0.0001  
    {'AES/r7/t1/t0/s0/n872'}      0.0001  
    {'AES/r7/t1/t0/s4/n863'}      0.0001  
    {'AES/s9[94]'          }      0.0001  
    {'AES/r6/t3/t1/s0/N869'}      0.0001  
    {'AES/r6/t3/t1/s4/N870'}      0.0001  

[per isakson]

"is there any way to show both result side by side?" Not really. However, with a little bit of programming

Convert the cell array to a table

T = table( net_names{1}, net_names{2} ...
        , 'VariableNames',{'Net','Togglerates'} );

and display the table.

Or use tprintf, which I uploaded to the File Exchange

tprintf( 1, '%-20s  %f', net_names{:} )

Or write your own display function

[Farhan K]

none of them works :(

[per isakson]

What error messages do you see? "none of them works :(" isn't very helpful.

Both works here!

>> T = table( net_names{1}, net_names{2} ...
            , 'VariableNames',{'Net','Togglerates'} );
>> disp(T)
             Net              Togglerates
    ______________________    ___________
    'AES/r8/t0/t3/s4/n885'      0.0001   
    'AES/r3/p21[25]'            0.0001   
    'AES/r7/z0[6]'              0.0001   
    ...
>> 
>> tprintf( 1, '%-20s  %f\n', net_names{:} )
AES/r8/t0/t3/s4/n885  0.000100
AES/r3/p21[25]        0.000100
AES/r7/z0[6]          0.000100
AES/r6/p31[6]         0.000100
...

[Farhan K]

It works with Example.txt

But with my original file, where I have total 179456 nets, it does not work

clc
filename = 'C:\Users\kabirm\Downloads\Synthesis\switching_prob_report.txt';
fileID = fopen(filename);
net_names = textscan( fileID, '%s%*f%*f%f%*f', 'Headerlines',612);
fclose(fileID);
net_names = table( net_names{1}, net_names{2} ...
            , 'VariableNames',{'Net','Togglerates'} );
disp(net_names)

Shows below error

Error using table (line 232)
All table variables must have the same number of rows.

Error in T3 (line 6)
net_names = table( net_names{1}, net_names{2} ...

[per isakson]

"All table variables must have the same number of rows." tells me that textscan() fails to read your large file to its end. The reason is most likely that there is a line in your data file that doesn't match the format string. (I think readtable() will encounter the same problem. Might differ between releases. I run R2018b)

I reproduced your error by modifying line 44 of Example.txt to

AES/s3[124]     16.927  0.496   whats   wrong

Next step is to add 'ReturnOnError' to the call of textscan()

net_names = textscan( fileID, '%s%*f%*f%f%*f'   ...
                    , 'Headerlines',4           ...
                    , 'ReturnOnError', false    ...    
                );

Now I get the error message

Error using textscan
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 40, field number 4) ==> whats       wrong\n
Error in cssm (line 4)
net_names = textscan( fileID, '%s%*f%*f%f%*f'   ...

which tells me that the two words, 'whats' and 'wrong', cause the error.

The cure to this error is to add 'TreatAsEmpty' to the call of textscan()

net_names = textscan( fileID, '%s%*f%*f%f%*f'               ...
                    , 'Headerlines',    4                   ...
                    , 'ReturnOnError',  false               ...
                    , 'TreatAsEmpty',   {'whats','wrong'}   ...
                );            

Problem solved. textscan reads the modified Example.txt without problems. The words, 'whats' and 'wrong', are converted to nan.

.

So far I've missed your attachment.

When browsing switching_prob_report.txt in the free editor notepad++ , I quickly spotted problematic lines at the end of the file. (There might be more.)

...
Trigger/n7                 2.780    0.000    0.49e-7  9.772e-08  
AES/*Logic0*             288.088    0.000    0.0000      0.0000  d
AES/*Logic1*              71.396    1.000    0.0000      0.0000  d
Tj_Trig                    7.086    0.000    0.0000      0.0000  
...
Trojan/lfsr/n25            2.489    0.000    0.0000      0.0000  
--------------------------------------------------------------------------------
Total (179456 nets)                                         24.1493 uW
1

We can solve the problem with the extra column containing the letter 'd' by modifying the format string to skip to the end of line after the 'Togglerates' column.

%%
filename = 'd:\m\cssm\switching_prob_report.txt';
fileID = fopen(filename);
net_names = textscan( fileID, '%s%*f%*f%f%*[^\n]'   ...
                    , 'Headerlines',    612         ...
                    , 'ReturnOnError',  true        ...
                );
fclose(fileID);

reads all data lines and fails (as expected) on the three last lines.

>> net_names{1}(end-4:end)
ans =
  5×1 cell array
    {'Trojan/lfsr/n21'                                                                 }
    {'Trojan/lfsr/n24'                                                                 }
    {'Trojan/lfsr/n25'                                                                 }
    {'--------------------------------------------------------------------------------'}
    {'Total'                                                                           }
>> net_names{2}(end-4:end)
ans =
     0
     0
     0
     0
   NaN
>>

The first column has one element more than the second. Without exactly understanding this result, I delete the elements, which originate in the footer.

%%
net_names{1}(end-1:end)= [];
net_names{2}(end) = [];

[Farhan K]

wow! This is why I preferred to use MATLAB rather than using Python. Excellent help

Answer 2

[Akira Agata]

Thank you for uploading an example. How about the following?

% Read from text data
T = readtable('Example.txt','HeaderLines',4,'Format','%s%f%f%f%f');
T.Properties.VariableNames = {'Net','NetLoad','StaticProb','ToggleRate','SwPower'};

% Extract NetLoad and ToggleRate
T = T(:,{'Net','ToggleRate'});

[Akira Agata]

To find out a root-cause, more details is needed. Is it possible to upload your original data file or more large sample file (such as ~1000 lines) ?

[Farhan K]

Please check attached.

[Farhan K]

In this file, if you go down, then you can find the Net and Toggle rate section with total 179456 nets

[Akira Agata]

Thank you for attaching the data.

OK. Looking at your data, I found some irregular lines and needs to some pre-processing.

How about the following?

% Read from text data
C = readcell('switching_prob_report.txt','Delimiter','\r\n','NumHeaderLines',612);

% Delete the last 3 lines (because they don't contain data)
C(end-2:end) = [];

% Detect irregular lines (which does not end with number)
idx = ~endsWith(C,compose('%d',0:9));

% Delete characters at the end of the detected lines
C(idx) = regexprep(C(idx),'\s*[^\d]$','');

% Split the line
D = split(C);

% Extract 1st and 4th column
Net = D(:,1);
ToggleRate = str2double(D(:,4));