Speed up this fread loop

18 views (last 30 days)
T on 1 Apr 2017
Edited: Carl on 4 Apr 2017
I need your help! I am going to read a lot of big files with fread. Any ideas on how to speed this up? Also if there is a way to estimate N from the length of the file before allocating B1 to B8? Every advice is greatly appreciated
fid = fopen(Filename,'r');
B1=zeros(m, N,'single');
B2=zeros(m, N,'single');
B3=zeros(m, N,'single');
B4=zeros(m, N,'single');
B5=zeros(m, N,'single');
B6=zeros(m, N,'single');
B7=zeros(m, N,'single');
B8=zeros(m, N,'single');
fseek(fid,headerlength, 'bof');
fseek(fid, offset, 0);
for k = 1 : N
tag(k)= fread(fid,1,'int64');
disp('End of file')
Idontneedthis = fread(fid,2,'uint64');
Idontneedthis= fread(fid,1,'int');
Idontneedthis = fread(fid,1,'int');
A = fread(fid,m*8,'short','ieee-be');
A = reshape(A,m,8);
disp('End of file')
B1(:,k) = A(:,1);
B2(:,k) = A(:,2);
B3(:,k) = A(:,3);
B4(:,k) = A(:,4);
B5(:,k) = A(:,5);
B6(:,k) = A(:,6);
B7(:,k) = A(:,7);
B8(:,k) = A(:,8);
Thanks a lot!!!

Answers (2)

Carl on 4 Apr 2017
Edited: Carl on 4 Apr 2017
Hi T,
  1. If you are creating the files yourself, you can store the value of N at the beginning of the file.
  2. You can replace the assignments to 'Idontneedthis' with a call to fseek.
  3. You can also look into using the Parallel Computing Toolbox for doing the read in parallel. Here is an example of doing so with the 'parfor' loop: http://www.mathworks.com/matlabcentral/newsreader/view_thread/307594#835402

Jan on 4 Apr 2017
Edited: Jan on 4 Apr 2017
I simply added the bytes per block and divided the file size by this considering the "headerlength" and the "offset". This seems to be trivial maths.
m = 16; % Guessed
FileInfo = dir(Filename);
N = (FileInfo.bytes - headerlength - offset) / (3 * 8 + 4 + 4 + m * 8 * 2);
fid = fopen(Filename, 'r', 'ieee-be');
if fid == -1 % I've seen too many code failing silently...
error('Cannot open file: %s', Filename);
B = zeros(m, 8, N, 'single');
tag = zeros(1, N, 'uint64');
fseek(fid, headerlength + offset, 'bof');
for k = 1:N
tag(k) = fread(fid, 1, 'int64');
fread(fid, 3, 'uint64'); % Skip 24 bytes
B(:, :, k) = fread(fid, [m, 8], 'int16=>single');
Do not create 8 different variables with an index in the name. Prefer a 3D array instead. Please test this - due to the missing of test data I cannot run this.


Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!