MATLAB Answers

0

How to increase reading speed from a Gigabyte large file ?

Asked by farzad on 17 Jun 2019
Latest activity Commented on by farzad on 20 Jun 2019
Hi all
how do I increase reading speed from an Excel file that contains rows and columns with a volume of some GigaBytes?

  18 Comments

I created a format with repmat of '%f' 50 times. I fopen and then
datacell = textscan(fid, fmt, 'collectoutput', 1);
Because this puts everything into a single cell the overhead to extract the array is trivial.
The timing with collectoutput 0 without joining the columns after, was a hair higher but not statistically significant.
Yeah, that's kinda' what I suspected, thanks for confirming, Walter.
I still find it more than strange that there's 30% reduction over fscanf -- what are they doing wrong with it then is the question that there's that much room for improvement?
These timings couldn't possibly be related to caching issues, I presume; you're too careful for that! :)
Thank you all guys ! It was a great answer ! thank you

Sign in to comment.

0 Answers