Tackling the right structure type for my file2struct function of mixed data types.

Evening, I am a big fan of mfcsvread function, however it did not do mixed data where you can have dates, times, and numbers. I decided to take it upon my cell to try this on my own. There are a few hurdles I need help with. The first is that it reads all data as strings instead of reading in a mix of strings and numbers or DATE/TIME and numbers. I also don't like the structures in the format I have them. They output as outputData(x).fieldName where x is the row in the source file. I would rather have outputData.fieldName(x) where x is the row in the source file for that fieldName. It makes manipulation of the entire field for unit conversions easy. Especially when trying to convert from str2num. Or am I missing something all together?

Thanks!

V/R, Frank

Here is sample CSV data:

DATE, TIME, ID, ALT, ROLL (radians)
1/1/2014, 13:51.000, 1500, 0.151
1/1/2014, 13:51.013, 1500, 0.155

Here is the function below:

function outputData = file2struct(fileName)
fileID=fopen(fileName); 

% read the first line 
textLine = fgetl(fileID); 
textLine = regexprep(textLine,'[(]','_');
textLine = regexprep(textLine,'[^a-zA-Z0-9_,]','');
commaLocations = findstr(',',textLine); % find the commas
fieldNames = cell(1,(length(commaLocations)+1));
cursor=1;
for columnIndex = 1:length(commaLocations)
    fieldNames{columnIndex} = textLine(cursor:commaLocations(columnIndex)-1);
    cursor = commaLocations(columnIndex)+1;
end

fieldNames{columnIndex+1} = textLine(cursor:end);

textLine = fgetl(fileID);
i = 1;
while ~isequal(textLine,-1)
    textLine = textscan(textLine,'%s','Delimiter',',');
    for m = 1:length(fieldNames)
        outputData(i).(fieldNames{m}) = textLine{1}{m};
    end
    i = i + 1;
    textLine = fgetl(fileID);
end
fclose(fileID);

[per isakson]

"13:51.000," . This "concatenation" of TIME and ID seems weird. Is this really the case in your data files?

Answer 1

[per isakson]

I'm not exactly sure what your question is. However, this might help. It is the standard way to read and parse your file. Try this first step. Remains to assign the result to a struture.

    >> [ hdr, sdn, id, num ] = cssm()
    hdr =
    DATE, TIME, ID, ALT, ROLL (radians)
    sdn =
       1.0e+05 *
        7.3560
        7.3560
    id =
         0
        13
    num =
       1.0e+03 *
        1.5000    0.0002
        1.5000    0.0002
    >>

where

    function   varargout = cssm()
        fid = fopen( 'cssm.txt' );
        hdr = fgetl( fid );
        cac = textscan( fid, '%s%s%f%f'         ...
                    ,   'CollectOutput', true   ...
                    ,   'Headerlines', 0        ...
                    ,   'Delimiter', ','        );
        fclose( fid );
        dti = cell2mat( cac{1} );
        id  = str2num( dti(:,[end-2:end]) );
        sdn = datenum( dti(:,[1:end-4]), 'mm/dd/yyyyHH:MM' );

        varargout = { hdr, sdn, id, cac{2} };
    end

and where cssm.txt contains the sample you provided. I'm guessing regarding the ID

[XLT_Frank]

Ok, let me try this. I want to make a mixed data set version of this function: http://www.mathworks.com/matlabcentral/fileexchange/6012-multi-format-csv-reader/content/mfcsvread.m

The data I work with tends to be 100 different columns and about 50k lines. I like to read them into a structure such that they read like this...

dataStructure.parameter(x) where x is each line for that parameter in the file.