pre-allocating disk space for matfile objects containing cell arrays of large sparse matrices

I have a custom function that consumes O(n^2) RAM in intermediate steps and generates very sparse outputs. The current strategy is to slice inputs into RAM-fitting blocks and assemble the output from each block: https://www.mathworks.com/matlabcentral/answers/1742830-slicing-matrices-to-fit-in-memory

Now I am looking at temporarily store these outputs on disk pointed by a matfile object to make room for processing of the next slice. According to this it is preferrable to pre-allocate disk for matfile to achieve better performance: https://www.mathworks.com/help/matlab/import_export/troubleshooting-file-size-increases-unexpectedly-when-growing-an-array.html

However I am not sure how to do it for my case. Should I just use this?

TempMAT.A(i,j)=mat2cell(spalloc(m,n,ceil(SparsityEstimate*m*n)),m);
%A{i,j} is a m-by-n sparse matrix with a estimated sparasity of SparsityEstimate

Answer 1

[Prateekshya]

Hi Yi-xiao,

As per my understanding, you want to improve the performance for large matrices. The piece of code that you have given looks alright. Since you are dealing with sparse matrices, there is a way to maintain sparse matrices in MATLAB. For more information, you can also have a look at this link: https://in.mathworks.com/help/matlab/ref/sparse.html

I hope it helps!