textscan problem 7.4.0.287 (R2007a) versus 7.11.0.587(R2010b)

i have a file (myfile.csv) that contains thousands double quoted text, comma delimited columns

Example: "Name001","The Lazy Brown Dog Jumps over the quick red fox [003]","tag01","","x",... {25 total columns}

I use

>>fi = fopen('myfile.csv','rt);
>>c = textscan(fi,'%q%q%q%q%q%q%q...25 times','Headerlines',1,'Delimiter',',');

the text column will sometime contain a TAB "\t" separation after the word Fox. example "The Lazy ... Fox TAB [001]"

7.4.0.287 (R2007a) has no problem reading the file and accurately returns the information

7.11.0.587(R2010b) reads the tab as and additional delimiter; therefore breaking my expected output.

7.4.0.287 (R2007a) id = c{1};

gives me a nx1 aray 'name01' 'name02' 'name02' 'name03'

7.11.0.587(R2010b) returns nx1 aray name01 'text from another field' 'text from other column' 'something useless' ... 'name40' %when it finally catches up

when i remove the tab i get 'name01' 'name02' ... 'name999' 'Text from another field' %this means that name 999 also had a tab.

how do i reconcile the differences? i want to use the new MATLAB is there something I can add so that i can have %q comma delimited ignore the tabs when inside the double quoted text "\t"

Answer 1

[Fangjun Jiang]

I suggest you first report this issue to TMW (attach your .csv file). It sounds to me like a bug if the delimiter is specifically specified as ','.

I could suggest a work-around. Read in the whole .csv text string and then use regexp() to extract the data.

a='"Name001","The Lazy Brown Dog Jumps over the quick red fox [003]","tag01","","x"';
b=regexp(a,'"[^"]*"','match');
celldisp(b);
b{1} =
"Name001"
b{2} =
"The Lazy Brown Dog Jumps over the quick red fox [003]"
b{3} =
"tag01"
b{4} =
""
b{5} =
"x"

Answer 2

[Walter Roberson]

Please cross-check whether it is taking \t as a Delimiter or as Whitespace. That is, possibly setting Whitespace to '' will fix the behavior.