MATLAB Answers

how can I gracefully exit a while loop, when reading from a data file of variable length, and the file may terminate abruptly and incompletely when written

6 views (last 30 days)
here is some sample code with comments on what is happening... (actual code reads huge file with huge messages... shortened here to simplify)
% create sample data
A = [0 0 2 1 2];
B = [0 0 5 1 2 3 4 5];
C = [0 0 7 1 2 3 4 5 6 7];
D = [0 0 ]; % D is incomplete data.... not following protocol described below...
% create a file with sample data, assume you don't know what order...
fileID = fopen('test.bin','w');
fwrite(fileID,B);
fwrite(fileID,C);
fwrite(fileID,A);
fwrite(fileID,D);
fclose(fileID);
% read a file of sample data
fileID = fopen('test.bin');
ALL = fread(fileID).'
fclose(fileID)
% But, suppose the file has a huge amount (over a gigabyte) of data...
% don't want to read all that data in at the beginning,
% want to read in a block of data at a time and process it,
% then move to the next block...
% Imagine that a block of data is very very large...
% for this example A = 5 bytes, B = 8 bytes, and C = 10 bytes !!! much simplification
% suppose the first 2 bytes in the block are 0 0
% and suppose the 3rd byte in the block contains the number of remaining bytes in the
% block you are trying to read...
fileID = fopen('test.bin');
while(1)
'in while loop'
DATA1_1 = fread(fileID,3);
DATA1_2 = fread(fileID,DATA1_1(3));
DATA1_ALL = cat(1,DATA1_1,DATA1_2).'
'pausing now, press any key...'
pause
end
% imagine that this is remaining code...
% how can we exit the while loop gracefully
% and still execute these lines of code !? :-)
' this is remaining code.... want to see this line...'
' this is remaining code.... want to see this line...'
' this is remaining code.... want to see this line...'
Output will look something like this, before correcting the code...
ALL =
Columns 1 through 14
0 0 5 1 2 3 4 5 0 0 7 1 2 3
Columns 15 through 25
4 5 6 7 0 0 2 1 2 0 0
ans =
0
ans =
'in while loop'
DATA1_ALL =
0 0 5 1 2 3 4 5
ans =
'pausing now, press any key...'
ans =
'in while loop'
DATA1_ALL =
0 0 7 1 2 3 4 5 6 7
ans =
'pausing now, press any key...'
ans =
'in while loop'
DATA1_ALL =
0 0 2 1 2
ans =
'pausing now, press any key...'
ans =
'in while loop'
Index exceeds matrix dimensions.
Error in break_out_of_while_loop (line 37)
DATA1_2 = fread(fileID,DATA1_1(3));
question is... how to stop the 'while' loop more gracefully, and execute remaining code after while loop... !!!
  4 Comments
Robert Brown
Robert Brown on 20 Apr 2021
Thanks for the comment !!!
@Image Analyst has answered the question using exactly this approach of the "try - catch" construct !!!
I didn't really know about the "try - catch" construct, so this was very educational to me.
I do appreciate you trying to help, and offering this advice !!!

Sign in to comment.

Accepted Answer

Image Analyst
Image Analyst on 19 Apr 2021
Edited: Image Analyst on 19 Apr 2021
Not sure you created the demo/sample file correctly. Here, see if this more robust way of reading solves your problem:
% Demo by Image Analyst.
clc; % Clear the command window.
fprintf('Beginning to run %s.m ...\n', mfilename);
close all; % Close all figures (except those of imtool.)
clearvars;
workspace; % Make sure the workspace panel is showing.
format short g;
format compact;
fontSize = 14;
% create sample data
A = uint8([0 0 2 1 2]);
B = uint8([0 0 5 1 2 3 4 5]);
C = uint8([0 0 7 1 2 3 4 5 6 7]);
D = uint8([0 0]); % D is incomplete data.... not following protocol described below...
% Create a file with sample data, assume you don't know what order...
filename = 'test.bin'
% if ~isfile(filename)
% Create file if it does not exist yet.
fileID = fopen('test.bin', 'wb');
fwrite(fileID, B, 'uint8');
fwrite(fileID, C,'uint8');
fwrite(fileID, A,'uint8');
fwrite(fileID, D,'uint8');
fclose(fileID);
% end
% Read a file of sample data
fileID = fopen(filename, 'rb');
fprintf('--------------------------------\nContents of %s:\n', filename);
ALL = fread(fileID)
fclose(fileID);
fprintf('--------------------------------\n');
% But, suppose the file has a huge amount (over a gigabyte) of data...
% don't want to read all that data in at the beginning,
% want to read in a block of data at a time and process it,
% then move to the next block...
% Imagine that a block of data is very very large...
% for this example A = 5 bytes, B = 8 bytes, and C = 10 bytes !!! much simplification
% suppose the first 2 bytes in the block are 0 0
% and suppose the 3rd byte in the block contains the number of remaining bytes in the
% block you are trying to read...
fileID = fopen('test.bin', 'rb');
loopCounter = 1;
maxIterations = 100; % Failsafe
while loopCounter < maxIterations
fprintf('In while loop, on iteration #%d...\n', loopCounter);
try
DATA1_1 = fread(fileID, 3, 'uint8')
catch ME
break;
end
if length(DATA1_1) < 3
% Not 3 bytes in what was read.
break;
end
fprintf(' Going to read %d bytes...\n', DATA1_1(3));
try
DATA1_2 = fread(fileID, DATA1_1(3), 'uint8');
catch ME
% There were not that many bytes left in the file so it throws an error.
% Break out of loop in that case.
break;
end
fprintf(' Here are the %d bytes for iteration #%d...\n', DATA1_1(3), loopCounter);
fprintf(' %d ', DATA1_2);
fprintf('\n');
DATA1_ALL = cat(1,DATA1_1,DATA1_2).'
fprintf('pausing now, press any key...\n');
promptMessage = sprintf('That is after iteration %d.\nDo you want to Continue processing with loop #%d,\nor Quit processing?', loopCounter, loopCounter+1);
titleBarCaption = 'Continue?';
buttonText = questdlg(promptMessage, titleBarCaption, 'Continue', 'Quit', 'Continue');
if contains(buttonText, 'Quit', 'IgnoreCase', true)
break; % or break or continue.
end
loopCounter = loopCounter + 1;
end
fprintf('Done with loop after %d iterations.\n', loopCounter);
fprintf('Done running %s.m\n', mfilename);
  4 Comments
Image Analyst
Image Analyst on 20 Apr 2021
Robert, well if you really also want to be an overachiever and award me extra "Reputation points" you can do so by clicking on the Vote icon on my two answers. Accepting (which you can do only once) gives two points. Votes (which you can do multiple) give two additional points per vote. Thanks in advance.
Also check out my File Exchange, which has a small fraction of my 300+ demos:

Sign in to comment.

More Answers (1)

Image Analyst
Image Analyst on 19 Apr 2021
Try memmapfile(). It's meant for dealing with enormous files. Sorry - I have not used it myself so write back here or call tech support if you need help with that function.
  1 Comment
Robert Brown
Robert Brown on 20 Apr 2021
This looks like very good advice as well !!! I don't really know how to use the memmapfile(), but I think it has high payoff for the types of data and array sizes that I am subjected to processing. I'm going to study up on this and learn how it is done. Thanks again !!!

Sign in to comment.

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!