cell2mat loops & running out of memory
6 views (last 30 days)
Show older comments
Aloha,
I need to process 72 hours of sampling at a rate of 25.6 kHz. The MatLab files I am given have the form of a structure, with string fields containing instrument information, units, etc. The numerical data itself is contained in 262144x2 double fields named StreamSegment000000_000255, StreamSegment000256_000511, and so forth, each of which containing 10.24 seconds of data. The file I am currently working on has roughly 2 hours of data, divided into 703 10.24-second fields of size 262144x2 double and a 2.84-second field of size 72704x2 double (of course, files will have different sizes). The first thing I need to compute is 30-second RMS for all 72 hours. Here is my approach:
S=load('REC0049.mat');
cell0=struct2cell(S);
C=cell0(3:end,:); % crops out the first 2 arrays of junk information
Now cell2mat(C) will of course lead me "??? Error using ==> cat; Out of memory", so I tried to break down to several matrices.
for i=1:number
eval(sprintf('M%d = cell2mat(C((1+3*(i-1)):(3*i),:));', i));
end
If number=235, obtaining 235 matrices each containing 3 of these 10.24-second fields would allow me to easily compute 30.72-second rms (close enough) for these 2 hours. Of course, this works just fine for small "numbers", but I quickly encounter "*??? Error using ==> cat Out of memory" again.
I could try nested for-loops and progressively clear out the memory, but I am sure there are solutions that are a lot more elegant and efficient.
Thank you!
0 Comments
Accepted Answer
Matt J
on 4 Jan 2013
Edited: Matt J
on 4 Jan 2013
So if I understood you, the fields named StreamSegment000000_000255, StreamSegment000256_000511 are the fields of S when you do
S=load('REC0049.mat');
In other words, you generated REC0049.mat by using eval to auto-name 703 different variables and saved them to a .mat file? I hope you know that that's a bad idea, as described here.
Instead, you could use MATFILE to write the segments to and from the elements of a cell array C in REC0049.mat.
C=cell(1,number);
save REC0049 C;
matObj = matfile('REC0049.mat','Writable',true);
for i=1:number*3
matObj.C(1,i)={data_chunk_to_write};
end
Then later
rms=zeros(1,number);
for i=1:number
rms(i) = computeRMS( cell2mat(matObj.C(1, (1+3*(i-1)):(3*i),:) ) )
end
More Answers (1)
Malcolm Lidierth
on 5 Jan 2013
Depending on the MAT-file version, there may be something that could help in:
0 Comments
See Also
Categories
Find more on Data Type Conversion in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!