How to import around 1000 text files and then convert them from cnv files to matlab files and then organize them in a structure with their original file name?

Hello Community,

This question has multiple parts in it, so I will break it down.

     -Import multiple cnv files.
     -Convert the cnv files into multiple arrays for each file (I have a function called cnv2mat.m that converts a cnv file to 6 arrays called (lat,lon,gtime,data,names,sensors)
note: the files have to be characters when they are used in this function so 
 FILE='05070601.cnv';
 [lat,lon,gtime,data,names,sensors]=cnv2mat(FILE); 

     -Finally I want to create a structure that we'll call data, that has a number associated for each cnv file and then 6 arrays for each of those files.

For a reference, one of the names of the files is 00092600.cnv (which references year 2000,mon 9, day 26, and 00 means first cast on that day)

Any help would be greatly appreciated. Thanks in advanced.

Andrew

Answer 1

[Joseph Cheng]

It looks like you already have something that converts a single cnv file which you can utilize. If all the cnv files are in the same folder the dir() command would work to get the file names into MATLAB. If you look up the help on dir() you can see it'll populate a structure of the specified folder where files(i).name would contain the string you'd feed into your cnv2mat() function. you could alternatively populate a text file of each file you want and use that to get the list of filenames.

Now that the file names are stored as a variable in MATLAB a simple for loop should suffice. example: %using dir() to get the file names into FILES

FILES(1:2) = []; 
%gets rid of the entries "." and ".." which are references to current and parent directory and not really the files you are looking for.  Additionally this is assuming you do not have any other files and folders in the folder (you'll have to filter it out yourself. 
for i=1:length(FILES)
[lat,lon,gtime,data,names,sensors]=cnv2mat(FILE(i).name);  %might need a cell2mat in there
data(i).Filename = FILE(i).name;
data(i).lat = lat;
data(i).long = long;
%etc. 
%if i will not suffice as the number associated for each cnv file
data(i).FileIdentifier = someUniqueNumberingScheme;
end

[Andrew]

I apologize for the late reply,

This is what I have, I'm pretty sure I might be missing something.

 FILES = dir('*.cnv');

 for i=1:length(FILES)
 [lat,lon,gtime,data,names,sensors]=cnv2mat(FILES(i).name);  %might need a cell2mat in there
 data(i).Filename = FILES(i).name;
 data(i).lat = lat;
 data(i).long = long;
 data(i).gtime = gtime;
 data(i).data = data;
 data(i).names = names;
 data(i).sensors = sensors;

 %if i will not suffice as the number associated for each cnv file
 %data(i).FileIdentifier = someUniqueNumberingScheme;
 end

The error i receive is

 Structure assignment to non-structure object.

 Error in datascript (line 16)
 data(i).Filename = FILES(i).name;

Does this relate to your last comment of "if i will not suffice as the number associated for each cnv file"

[Joseph Cheng]

It could possibly be that data is already assigned? http://www.mathworks.com/matlabcentral/newsreader/view_thread/100067.

This link has the similar error where data has been used and is not a structured object. Perhaps try define data at the beginning to set data as a structure.

data = struct('Filename',[],'lat',[],'long',[])

[Andrew]

I tried defining, but the error still comes back the same and looking at the link, I already have 'clear all' as the first line in the script.

[Joseph Cheng]

I have some dumb but sometimes relevant questions.

1. Which iteration of "i" do you get your error?
2. can you give an example of the FILES(i).name it has trouble putting in?
3. Have you tried putting a break point at that line and try to insert a string into data(i).Filename using the command line?

[Andrew]

The for loop will load the very first text file correctly. So I have six variables that are accurate when I look back at the original text file. The error then, would be when it tries to load the second text file. I assume it has trouble putting it into a structure. Also the variable Filename is nonexistent.

Not sure if that clears up anything.

[Joseph Cheng]

So the very first iteration you do get a valid data(1).Filename, data(1).lat, etc? I would put a breakpoint at your cnv2mat and step through the code. If it has trouble loading the second text file it would error out at your cnv2mat() function.

Perhaps lets try this, with a breakpoint at cnv2mat and you perform a single step. Before the data(i).Filename is executed see if you get a valid [lat,lon,gtime,data,names,sensors]. Also at this point see what FILES(i).name is, Also in the command window try to set data(i).Filename to something 'TEST.cnv'.

if "Structure assignment to non-structure object." is still occurring I do not know if i can help more without seeing the script. Perhaps someone in the department of ODU can supply some some real time support?

[Andrew]

Finally got it, was a simple fix, after defining the structure as data. I only needed to change my output of data to something else.

Final script looks like this:

 clear all

 FILES = dir('*.cnv');
 data = struct('Filename',[],'lat',[],'long',[],'gtime',[],'data',[],'names',[],'sensors',[]);

 for i=1:length(FILES)
    [lat,lon,gtime,D,names,sensors]=cnv2mat(FILES(i).name);  
    data(i).Filename = FILES(i).name; 
    data(i).lat = lat;
    data(i).lon = lon;
    data(i).gtime = gtime;
    data(i).data = D;
    data(i).names = names; 
    data(i).sensors = sensors;
 end

Thank you so much!

Answer 2

[per isakson]

Allocate the structure with this statement before the for-loop

    data = struct('Filename',{FILES.name},'lat',[],'long',[]);

then you may overwrite the names in the loop.

.

Here are some links on debugging in Matlab

• Debugging in MATLAB
• Debugging session
• Debugging Approaches