MATLAB (Linux) crashes during parallel read in of files
6 views (last 30 days)
Show older comments
Hello,
I've had problems with MATLAB crashing (it simply crashes and exits) when I try to read in a large number of files within a parfor loop. I have noticed that it seems to happen when my RAM (monitored using htop) gets full and then MATLAB suddenly crashes.
I have upgraded my RAM to 16GB and it seemed to help in some cases, however the problem still occurs.
Is the only solution to upgrade my RAM, or can something else be done? Actually one other thing I tried was to use less CPU cores which seems to help.
I used to use MATLAB on Windows 11 and it didn't crash due to this.
Thanks!
2 Comments
Edric Ellis
on 10 Oct 2022
One thing you could try is to use parpool("threads") - not everything works there, but if it does, it often uses less memory.
Without some more details, it's hard to know exactly what to suggest - it's not clear if the problem is during the data processing inside the parfor loop, or during the transfer to and from the workers. Both are possible. You can use ticBytes and tocBytes to see how much data transfer there is.
Otherwise, you might need to see if you can reduce the amount of memory needed to process each file. Maybe you can read each file in pieces rather than all at once. Maybe something like datastore could help.
Answers (0)
See Also
Categories
Find more on Parallel for-Loops (parfor) in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!