Why do I receive an error "java.lang​.OutOfMemo​ryError: unable to create new native thread" while running my parallel jobs on a cluster?

I receive the following errors when I try to run my jobs on my cluster:

ERROR: Error using parallel.Job/submit (line 304) 
Java exception occurred: 
java.lang.OutOfMemoryError: unable to create new native thread 

Why do I receive this error and how can I resolve this?

Answer 1

[MathWorks Support Team]

On many Linux distributions, by default the process ulimit for a shell is set to a value like 1024. User limits provide control over the resources available to the shell and to processes started by it.

To check this try executing "ulimit -a" command on the terminal and see the "max user processes", i.e. "nproc " field. It is generally set to a number like 1024 by default, so we maY need to increase the value of this field because under the hood Linux count the number of threads which impacts large multi-threaded programs like MATLAB when run in parallel.

To change the value of "max user processes", you can use the following command in Linux shell:

ulimit -u 63536

 

When you check the "ulimit -a" again you should now see the new max user processes. This change may only apply to the active terminal, to make the changes appear session-wide permanently, you can adjust the nproc value in /etc/security/limits.conf, for example, you can add the following line if you wanted to remove the limit from a user named "user":

user        hard    nproc      unlimited

 

More info about how to adjust the ulimit process limit can be found from third-party resources online. This file may require root or sudo access to edit.

For more information on recommended system limits for Parallel Computing please see: https://www.mathworks.com/help/parallel-computing/recommended-system-limits-for-macintosh-and-linux.html