OpenJDK OOM issue -

[Andrew Haley]

We meet one issue that the VM failed to initialize. The error log is as below. We checked both memory usage and thread number. They do not hit the limit. So could you please help to confirm why "java.lang.OutOfMemoryError: unable to create new native thread" error occurs? Many thanks.

Hard to say. If I had something I could reproduce and debug, I could
tell you the answer in a few minutes. Otherwise it's going to be
hard.

strace might help.

[David Holmes]

Hi Tim,

Hi OpenJDK dev group

This is better discussed on hotspot-dev, so redirecting there.

We meet one issue that the VM failed to initialize. The error log is as below. We checked both memory usage and thread number. They do not hit the limit. So could you please help to confirm why "java.lang.OutOfMemoryError: unable to create new native thread" error occurs? Many thanks.

Unfortunately there is no way to tell. As you indicate there appears to
be enough memory, and there appear to be enough threads/processes
available for creation.

Does this happen regularly or was this a one-of failure? You really need
to see the exact state of the machine at the time this happened.

David

"
on Sep 18 11:05:04 EEST 2017 2 or first INFO log missing: Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread
Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread
1. Memory Usage
MemFree: 898332 kB
From below core file generated during OMM, it can be seen about 900M physical memory available during that time.
2 Thread number
sh-4.1$ ps -eLf|wc -l
5326
sh-4.1$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 43497
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 43497
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Br,
Tim

[Poonam Parhar]

Hello Tim,

With CompressedOops enabled (which is enabled by default with 64-bit JVM), the Java heap may get placed in the lower virtual address space leaving very little space for the native heap allocations, and that can cause these kinds of failures even when there is lot of native memory available. Please read details here:

https://blogs.oracle.com/poonam/running-on-a-64bit-platform-and-still-running-out-of-memory

You can check the output collected with -XX:+PrintGCDetails that would show where your Java Heap is being based at.

Thanks,
Poonam

-----Original Message-----
From: David Holmes
Sent: Tuesday, September 19, 2017 3:46 AM
To: Yu, Tim (NSB - CN/Chengdu)
developers; Shen, David (NSB - CN/Chengdu)
Subject: Re: OpenJDK OOM issue -
Hi Tim,

Hi OpenJDK dev group

This is better discussed on hotspot-dev, so redirecting there.

We meet one issue that the VM failed to initialize. The error log is

as below. We checked both memory usage and thread number. They do not
hit the limit. So could you please help to confirm why
"java.lang.OutOfMemoryError: unable to create new native thread" error
occurs? Many thanks.
Unfortunately there is no way to tell. As you indicate there appears to
be enough memory, and there appear to be enough threads/processes
available for creation.
Does this happen regularly or was this a one-of failure? You really
need to see the exact state of the machine at the time this happened.
David

"
on Sep 18 11:05:04 EEST 2017 2 or first INFO log missing: Error
occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread Error
occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread
1. Memory Usage
MemFree: 898332 kB
From below core file generated during OMM, it can be seen about 900M

physical memory available during that time.

2 Thread number
sh-4.1$ ps -eLf|wc -l
5326
sh-4.1$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 43497
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 43497
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Br,
Tim

[Andrew Haley]

Hi OpenJDK dev group
We meet one issue that the VM failed to initialize. The error log is as below. We checked both memory usage and thread number. They do not hit the limit. So could you please help to confirm why "java.lang.OutOfMemoryError: unable to create new native thread" error occurs? Many thanks.

What OS is this?