How can I fopen/fwrite into memory, or convert my fread double array

15 views (last 30 days)
Hi, I am using a parfor loop to read images of jpeg compressed sequences (NorPix .seq).
Each worker fopen's the sequence and then fseek's to a previously specified 'readStart' and fread's the 'imageBufferSize'.
fid = fopen(fileName,'r','b');
fseek(fid,readStart,'bof');
JpegSEQ = fread(fid,imageBufferSize,'uint8','ieee-le');
Afterwards my working solution is to fwrite this 'JpegSEQ' variabe to a temporary jpg file.
tempName = [parallel_ID '_tmp.jpg'];
tempFile = fopen(tempName,'w');
fwrite(tempFile,JpegSEQ);
fclose(tempFile);
I = imread(tempName);
The JpegSEQ/imageBufferSize is variable due to compression (see below). However, fwrite will always generate the correct image dimensions of 420x2048 (WxH). Unfortunately, writing 8 temporary jpg to my HDD simultaneously will result in errors reading frames. If I do
pause(0.01);
after reading the image I can avoid all reading/writing errors, but with less performance.
Therefore, I would need a way to 'fwrite into memory' since this should be faster. Otherwise, I could just convert my JpegSEQ double array to a image of correct size, but reshape would not work due to the different JpegSEQ/imageBufferSize sizes.
readStart imageBufferSize
1028 115458
116494 116032
232534 115383
347925 115535
463468 116119
579595 115892
695495 115766
811269 115810
Does someone know a solution?
  3 Comments
Paul Siefert
Paul Siefert on 6 Aug 2018
I'm already using imread. You mean imwrite instead of fwrite? I will try that, but I thought the problem is the write/read speed of my HDD. Or do you think it is fwrite(tempFile,JpegSEQ) or fclose(tempFile) that is too slow to successfully imread the files?

Sign in to comment.

Accepted Answer

OCDER
OCDER on 6 Aug 2018
Edited: OCDER on 6 Aug 2018
Do you have SSD or HDD on your computer? SSD is much faster and perhaps this is the easy solution. Otherwise, you'd have to make your own uint8 matrix to jpg file converter, as so far, matlab only provides file-to-jpg converters such as rjpg8c.mexw64 located in the private folder of imagesci toolbox folder.
Instead of making temporary files in parallel, use serial processing to convert all your .Seq file into a folder containing many .jpg files once. Then use imread to load them all into a single matrix in memory, so you can do parallel processing.
1) Extract all .jpg files from a .seq file using your fread/fwrite method
2) Load all .jpg files into a single HxWx3xFrame matrix using imread
3) Use parallel processing to do what you have to do using parfor, etc.
Parallel processing is not great for reading/writing tasks to hard drive.
  7 Comments
Paul Siefert
Paul Siefert on 9 Aug 2018
Workaround of creating several temp files as shown in my 7 Aug 2018 at 14:38 comment works. I furthermore create a new temp file if fid is < 0. Thanks for your advice.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!