MATLAB crashes when running in a parallel bash mode

Hello,

bear with me for a few paragraphs while I explain the background of the problem as this is a meta-programming issue.

I have a bash script that I use to run MATLAB that looks like this:

xargs -P25 -d'\n' -I{} -t matlab -nodisplay -nodesktop -r "try;runinput={};MD_script_DoE_mass_time_RLV_Ncvx_SOCP1_14;end;exit" < MD_list_DoE_mass_time_RLV_Ncvx_SOCP1_14.txt

MD_script_DoE_mass_time_RLV_Ncvx_SOCP1_14 is a script file that contains a process, which uses the input case from MD_list_DoE_mass_time_RLV_Ncvx_SOCP1_14.txt. The bash script is reading line by line this text file, which really just contains numbers in descending order from 1 to 25. The script file then reads this as a variable called runinput and starts the process (I have a table in the matlab script file that gives the inputs to all the other variables of the problem based on a value from 1 to 25).

When I run the bash script, out of all the 25 processes, only 4 make it through. I get a whole bunch of errors for the others that indicate the following:

--------------------------------------------------------------------------------
              Assertion detected at Tue Feb 23 09:52:13 2021 +0100
--------------------------------------------------------------------------------

Configuration:
  Crash Decoding           : Disabled - No sandbox or build area path
  Crash Mode               : continue (default)
  Default Encoding         : UTF-8
  Deployed                 : false
  GNU C Library            : 2.26 stable
  Graphics Driver          : Unknown software 
  Graphics card 1          : 0x102b ( 0x102b ) 0x534 Version 0.0.0.0 (0-0-0)
  Java Version             : Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
  MATLAB Architecture      : glnxa64
  MATLAB Entitlement ID    : 846377
  MATLAB Root              : /usr/local/MATLAB/R2020a
  MATLAB Version           : 9.8.0.1538580 (R2020a) Update 6
  OpenGL                   : software
  Operating System         : "openSUSE Leap 15.0"
  Process ID               : 25593
  Processor ID             : x86 Family 6 Model 63 Stepping 2, GenuineIntel
  Session Key              : 20b9f250-ca98-4381-93f5-fb19d35701c2
  Static TLS mitigation    : Disabled: Unnecessary
  Window System            : No active display

Fault Count: 1


Additional Log Files:
  Java Crash Report        : /cala/alessandro/hs_error_pid25593.log
  Java Log                 : /cala/alessandro/java.log.25593


Assertion in void {anonymous}::mwJavaAbort() at javainit.cpp line 1395:
Fatal Java Exception. See this file for details: /cala/alessandro/hs_error_pid25593.log

Register State (captured):
  RAX = 0000000000000000  RBX = 00007f8c25c400a8
  RCX = 000000000000000a  RDX = 00007f8c25c1f698
  RSP = 00007f8aac223c50  RBP = 00007f8aac224010
  RSI = 00007f8c25be5ff6  RDI = 00007f8aac223c60

   R8 = 00007f8aac223f24   R9 = 00007f8aac223f24
  R10 = 00007f8c25be5ff6  R11 = 00007f8aac2243f0
  R12 = 00007f8c25c30028  R13 = 00007f8aac224890
  R14 = 00007f8c11604583  R15 = 00007f8c1160f4c0

  RIP = 00007f8c25b36a4e  EFL = 00007f8aac22416d

   CS = 3d38   FS = ac22   GS = 7f8a

Stack Trace (captured):
[  0] 0x00007f8c25b2d753    /usr/local/MATLAB/R2020a/bin/glnxa64/libmwfl.so+00853843
[  1] 0x00007f8c25b2d94c    /usr/local/MATLAB/R2020a/bin/glnxa64/libmwfl.so+00854348 _ZN10foundation4core4diag15stacktrace_base7captureEm+00000028
[  2] 0x00007f8c25b32b89    /usr/local/MATLAB/R2020a/bin/glnxa64/libmwfl.so+00875401
[  3] 0x00007f8c25b32cf0    /usr/local/MATLAB/R2020a/bin/glnxa64/libmwfl.so+00875760
[  4] 0x00007f8c115b07ee /usr/local/MATLAB/R2020a/bin/glnxa64/matlab_startup_plugins/jmi/../../../../bin/glnxa64/libmwjmi.so+01415150
[  5] 0x00007f8c08bb2700 /usr/local/MATLAB/R2020a/sys/java/jre/glnxa64/jre/lib/amd64/server/libjvm.so+09496320
[  6] 0x00007f8c08d77683 /usr/local/MATLAB/R2020a/sys/java/jre/glnxa64/jre/lib/amd64/server/libjvm.so+11351683
[  7] 0x00007f8c087844c7 /usr/local/MATLAB/R2020a/sys/java/jre/glnxa64/jre/lib/amd64/server/libjvm.so+05113031
[  8] 0x00007f8c08bb3e86 /usr/local/MATLAB/R2020a/sys/java/jre/glnxa64/jre/lib/amd64/server/libjvm.so+09502342
[  9] 0x00007f8c08bab74f /usr/local/MATLAB/R2020a/sys/java/jre/glnxa64/jre/lib/amd64/server/libjvm.so+09467727
[ 10] 0x00007f8c08bb53c8 /usr/local/MATLAB/R2020a/sys/java/jre/glnxa64/jre/lib/amd64/server/libjvm.so+09507784
[ 11] 0x00007f8c08d1fc0e /usr/local/MATLAB/R2020a/sys/java/jre/glnxa64/jre/lib/amd64/server/libjvm.so+10992654
[ 12] 0x00007f8c08bb18c2 /usr/local/MATLAB/R2020a/sys/java/jre/glnxa64/jre/lib/amd64/server/libjvm.so+09492674
[ 13] 0x00007f8c2795d569                             /lib64/libpthread.so.0+00030057
[ 14] 0x00007f8c270fca2f                                   /lib64/libc.so.6+01018415 clone+00000063

I also get an error on the ssh machine which I have attached since its too long to show here.

What is causing this issue? I have done some sample runs on my laptop with a few different runInput cases and it runs fine. It's a little bit tedious to debug the issue seeing as there are a lot of files involved and I don't run these processes with a GUI.

Answer 1

[Madheswaran]

Hi

Your MATLAB crashes are caused by memory allocation failures. As you can notice from the error message you have uploaded, there is "insufficient memory for the Java Runtime Environment to continue" and "Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory."

Since you are trying to launch 25 MATLAB instances simultaneously with xargs -P25, your system might not have enough resources to handle all 25 instances at once.

Try these solutions:

• Reduce the number of parallel processes by changing -P25 to something smaller like -P8 or -P4:

    xargs -P8 -d'\n' -I{} -t matlab -nodisplay -nodesktop -r "try;runinput={};MD_script_DoE_mass_time_RLV_Ncvx_SOCP1_14;end;exit" < MD_list_DoE_mass_time_RLV_Ncvx_SOCP1_14.txt

• Add more RAM to your system, increase swap space, or use a machine with more resources.
• Use the -nojvm flag if your code doesn't require Java functionality, refer to the following documentation: https://mathworks.com/help/matlab/ref/matlablinux.html

Hope this helps!