Parallel pool shut down during the iteration

75 views (last 30 days)
J Wang
J Wang on 17 Jul 2018
Commented: Daniel Poiesz on 4 Aug 2021
Hi everyone, I encountered the below errors during the iteration.
Please see the error info below.
Iteration 20 of 20 for XXXX completed.
IdleTimeout has been reached.
Parallel pool using the 'XXX' profile is shutting down.
Starting parallel pool (parpool) using the 'AAAA' profile ...
connected to XX workers.
Error using XXX...
Also in my home directory, I found some error files seem generated by JAVA.
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 262144 bytes for committing reserved memory.
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (os_linux.cpp:2640), pid=21819, tid=0x00007f9533511700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_144-b01) (build 1.8.0_144-b01)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.144-b01 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
--------------- T H R E A D ---------------
Current thread (0x00007f95498a1000): JavaThread "WorkerSessionBuilder.initiate-lab0" daemon [_thread_in_vm, id=29582, stack(0x00007f9533411000,0x00007f9533512000)]
Stack: [0x00007f9533411000,0x00007f9533512000], sp=0x00007f953350dd80, free space=1011k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0xacb18a] VMError::report_and_die()+0x2ba
V [libjvm.so+0x4ff4db] report_vm_out_of_memory(char const*, int, unsigned long, VMErrorType, char const*)+0x8b
After setting up the JAVA by
export _JAVA_OPTIONS="-Xmx110g"
Those
hs_error_pidXXX.log files
disappeared, but it still crashes during the iteration.
Would you please have a look and give me some advice? I am very new to Matlab. Any help would be highly appreciated. Thanks in advance.

Answers (1)

Kojiro Saito
Kojiro Saito on 17 Jul 2018
Could you check hard/soft limits are sufficient both for maximum number of process (nproc) and maximum number of file open(nofile) in your Linux machine.
ulimit -a
will show current hard/soft limits. You might need to increase these values for large parallel execution, for example, from 1024 to 65536.
  2 Comments
Navid Amini
Navid Amini on 8 Nov 2019
Dear Kojiro, your solution is perfect for me. Thank you very much.
Daniel Poiesz
Daniel Poiesz on 4 Aug 2021
What would you do for windows? I have parfor code that works upto 28 workers but idles out for 29-32. There are 32 available. The idel time is set for 30 minutes.

Sign in to comment.

Categories

Find more on Startup and Shutdown in Help Center and File Exchange

Products


Release

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!