Import from data from messy text file

3 views (last 30 days)
Zekeftw on 28 Jul 2016
Edited: dpb on 31 Jul 2016
Hi, I have looked through the questions and answers that are already available for this topic but wasn't able to apply those to my situation.
For the attached file, I need to extract the data for each "Specimen." The data that i need are the three columns corresponding to the 500+ rows for each specimen. If possible, I would like to save it in to different arrays. One for each specimen.

Accepted Answer

dpb on 28 Jul 2016
Edited: dpb on 28 Jul 2016
Pretty straightforward, just use the features in the file to find the wanted's quite regular in its construction so don't even have to do very much looking for what want--should get you started:
fid=fopen('Sample_DATA.TXT','r'); % open file
N=cell2mat(textscan(fid,'Number of specimens: %d %*[^\n]','headerlines',11));
for i=1:N % loop over number specimens in file
l=fgetl(fid); % initialize string to find start of section
while isempty(strfind(l,'Specimen:')),l=fgetl(fid);end % find the Specimen: n start
while isempty(strfind(l,'_')), l=fgetl(fid);end % then line break before data
d(i)=textscan(fid,'','delimiter',',','collectoutput',1); % the nth specimen data set
Result will be a cell array of N nx3 data arrays. Above at command line here leaves:
> whos d
Name Size Bytes Class Attributes
d 1x3 39564 cell
>> d
d =
[532x3 double] [561x3 double] [548x3 double]
Looks about right...
If the sections of auxiliary inputs, etc., are always a fixed length could use line counts in 'headerlines' parameter and avoid the while loops looking for section start, but the above will work for different sizes of those elements as long as the sections remain the same. For files of this size, that won't be that much overhead to process on a record-by-record basis anyway...
dpb on 31 Jul 2016
Edited: dpb on 31 Jul 2016
Under help for cell arrays in Data Types section is "Access the contents of cells by indexing with curly braces, {}. For more information, see <Access Data in a Cell Array>. Also note cellfun to do operations on cell arrays w/o explicitly dereferencing them which would serve for step 1 above although plot will need need to have the arrays and while cellfun will work given plot as the function, it doesn't have enough granularity to let you set hold on in between or create a new axes unless you write a complementary function to use that can handle the details so "use the curlies, Luke!"

Sign in to comment.

More Answers (1)

Shameer Parmar
Shameer Parmar on 28 Jul 2016
Here is that can help you..
Data = textread('SAMPLE_DATA.txt', '%s', 'delimiter', '');
newData = {};
for i=1:length(Data)
if ~isempty(strfind(Data{i},','))
newData{count,1} = Data{i};
Specimen = {};
for i=1:length(newData)
linedata = newData{i};
if ~isempty(strfind(linedata(1:2),'1,'))
Specimen{count1,count2} = linedata;
Specimen{count1,count2} = linedata;
to check the output

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!