Eclipse OpenJ9 can now provide Linux cgroup information as part of a Java dump file, commonly known as a
javacore. This can be very handy to debug clearly by having the information about the environment in which JVM is running and if JVM is aware of the limits imposed by the environment etc. In this article we’ll show you which cgroup information is captured in a javacore for the following scenarios:The cgroup section in a `javacore` can be found before the subcomponent dump and after the CPU Information section. The first line of the cgroup section (2CICONTINFO) tells you whether the JVM was running in a container environment. The second line (2CICGRPINFO) tells you whether the JVM was started with container support enabled. Subsequent lines provide the detailed cgroup information.
Container support is enabled by setting the `-XX:+UseContainerSupport` option on the command line when you run an application (Note: This is turned on by default in the latest releases of OpenJ9). This option turns on some additional intelligence about containerized environments in the OpenJ9 JVM. The JVM allocates a larger fraction of the available memory to the Java heap and is able to detect any limits that are imposed on the container by the host environment.
I. An application running on a traditional Linux system
In the first scenario, we produced a javacore file on a standard Linux environment by specifying the following `-Xdump` option for the `java -version`
java -Xdump:java:events=vmstop -version
Here is the section of the Javacore that contains the cgroup information:
|
2CICONTINFO Running in container : FALSE
2CICGRPINFO JVM support for cgroups enabled : FALSE
1CICGRPINFO Cgroup Information
NULL ————————————————————————
2CICGRPINFO subsystem : cpu
2CICGRPINFO cgroup name : /
3CICGRPINFO CPU Period : 100000 microseconds
3CICGRPINFO CPU Quota : Not Set
3CICGRPINFO CPU Shares : 1024
3CICGRPINFO Period intervals elapsed count : 0
3CICGRPINFO Throttled count : 0
3CICGRPINFO Total throttle time : 0 nanoseconds
2CICGRPINFO subsystem : cpuset
2CICGRPINFO cgroup name : /
3CICGRPINFO CPU exclusive : 1
3CICGRPINFO Mem exclusive : 1
3CICGRPINFO CPUs : 0-3
3CICGRPINFO Mems : 0
2CICGRPINFO subsystem : memory
2CICGRPINFO cgroup name : /
3CICGRPINFO Memory Limit : Not Set
3CICGRPINFO Memory + Swap Limit : Not Set
3CICGRPINFO Memory Usage : 4697415680 bytes
3CICGRPINFO Memory + Swap Usage : 4697415680 bytes
3CICGRPINFO Memory Max Usage : 0 bytes
3CICGRPINFO Memory limit exceeded count : 0
3CICGRPINFO Memory + Swap limit exceeded count : 0
3CICGRPINFO OOM Killer Disabled : 0
3CICGRPINFO Under OOM : 0
NULL
|
As expected for this scenario, the first line indicates that the JVM was not running in a container and the second line indicates that the JVM was started without enabling container support. The cgroup information shown describes the various cgroup subsystems and there respective attributes and values.
II. Javacore “Cgroup Information Section” on JVM running inside a container with no container limits set and container support not enabled.
In this scenario we create a container from a docker image named `openj9`, and pass to it the same command used in scenario 1.
docker run -it openj9
java -XX:-UseContainerSupport -Xdump:java:events=vmstop -version
Output :
|
2CICONTINFO Running in container : TRUE
2CICGRPINFO JVM support for cgroups enabled : FALSE
1CICGRPINFO Cgroup Information
NULL ————————————————————————
2CICGRPINFO subsystem : cpu
2CICGRPINFO cgroup name : /
3CICGRPINFO CPU Period : 100000 microseconds
3CICGRPINFO CPU Quota : Not Set
3CICGRPINFO CPU Shares : 1024
3CICGRPINFO Period intervals elapsed count : 0
3CICGRPINFO Throttled count : 0
3CICGRPINFO Total throttle time : 0 nanoseconds
2CICGRPINFO subsystem : cpuset
2CICGRPINFO cgroup name : /
3CICGRPINFO CPU exclusive : 0
3CICGRPINFO Mem exclusive : 0
3CICGRPINFO CPUs : 0-3
3CICGRPINFO Mems : 0
2CICGRPINFO subsystem : memory
2CICGRPINFO cgroup name : /
3CICGRPINFO Memory Limit : Not Set
3CICGRPINFO Memory + Swap Limit : Not Set
3CICGRPINFO Memory Usage : 26279936 bytes
3CICGRPINFO Memory + Swap Usage : 26279936 bytes
3CICGRPINFO Memory Max Usage : 26513408 bytes
3CICGRPINFO Memory + Swap Max Usage : 26513408 bytes
3CICGRPINFO Memory limit exceeded count : 0
3CICGRPINFO Memory + Swap limit exceeded count : 0
3CICGRPINFO OOM Killer Disabled : 0
3CICGRPINFO Under OOM : 0
NULL
|
In this cgroup section of the javacore file you can see that the dump was produced from a JVM that ran in a container. However, because there was no container limit set we get the default cgroup information. Also, we did not enable container support.
III. Javacore “Cgroup Information Section” on JVM running inside a container with no container limits set and container support enabled.
In this scenario we create a container from a docker image named `openj9`, and add `-XX:+UseContainerSupport` to the command in scenario 2.
docker run -it openj9
java -XX:+UseContainerSupport -Xdump:java:events=vmstop -version
Output :
|
2CICONTINFO Running in container : TRUE
2CICGRPINFO JVM support for cgroups enabled : TRUE
1CICGRPINFO Cgroup Information
NULL ————————————————————————
2CICGRPINFO subsystem : cpu
2CICGRPINFO cgroup name : /
3CICGRPINFO CPU Period : 100000 microseconds
3CICGRPINFO CPU Quota : Not Set
3CICGRPINFO CPU Shares : 1024
3CICGRPINFO Period intervals elapsed count : 0
3CICGRPINFO Throttled count : 0
3CICGRPINFO Total throttle time : 0 nanoseconds
2CICGRPINFO subsystem : cpuset
2CICGRPINFO cgroup name : /
3CICGRPINFO CPU exclusive : 0
3CICGRPINFO Mem exclusive : 0
3CICGRPINFO CPUs : 0-3
3CICGRPINFO Mems : 0
2CICGRPINFO subsystem : memory
2CICGRPINFO cgroup name : /
3CICGRPINFO Memory Limit : Not Set
3CICGRPINFO Memory + Swap Limit : Not Set
3CICGRPINFO Memory Usage : 26796032 bytes
3CICGRPINFO Memory + Swap Usage : 26796032 bytes
3CICGRPINFO Memory Max Usage : 26800128 bytes
3CICGRPINFO Memory + Swap Max Usage : 26800128 bytes
3CICGRPINFO Memory limit exceeded count : 0
3CICGRPINFO Memory + Swap limit exceeded count : 0
3CICGRPINFO OOM Killer Disabled : 0
3CICGRPINFO Under OOM : 0
NULL
|
In this cgroup section of the javacore file you can see that the dump was produced from a JVM that ran in a container. However, because there was no container limit set we get the default cgroup information. Also, we have enable container support.
IV. Javacore “Cgroup Information Section” on JVM running inside a container with container limits set and container support enabled
In this scenario we create a container from a docker image named `openj9`, and add container limits (-m, –cpu-quota, –cpu-period) to the command in scenario 3.
docker run -m2g –cpu-quota=”100000″ –cpu-period=”200000″ -it openj9
java -XX:+UseContainerSupport -Xdump:java:events=vmstop -version
Output :
|
2CICONTINFO Running in container : TRUE
2CICGRPINFO JVM support for cgroups enabled : TRUE
1CICGRPINFO Cgroup Information
NULL ————————————————————————
2CICGRPINFO subsystem : cpu
2CICGRPINFO cgroup name : /
3CICGRPINFO CPU Period : 200000 microseconds
3CICGRPINFO CPU Quota : 100000 microseconds
3CICGRPINFO CPU Shares : 1024
3CICGRPINFO Period intervals elapsed count : 16
3CICGRPINFO Throttled count : 2
3CICGRPINFO Total throttle time : 164156409 nanoseconds
2CICGRPINFO subsystem : cpuset
2CICGRPINFO cgroup name : /
3CICGRPINFO CPU exclusive : 0
3CICGRPINFO Mem exclusive : 0
3CICGRPINFO CPUs : 0-3
3CICGRPINFO Mems : 0
2CICGRPINFO subsystem : memory
2CICGRPINFO cgroup name : /
3CICGRPINFO Memory Limit : 2147483648 bytes
3CICGRPINFO Memory + Swap Limit : 4294967296 bytes
3CICGRPINFO Memory Usage : 32083968 bytes
3CICGRPINFO Memory + Swap Usage : 32083968 bytes
3CICGRPINFO Memory Max Usage : 32083968 bytes
3CICGRPINFO Memory + Swap Max Usage : 32083968 bytes
3CICGRPINFO Memory limit exceeded count : 0
3CICGRPINFO Memory + Swap limit exceeded count : 0
3CICGRPINFO OOM Killer Disabled : 0
3CICGRPINFO Under OOM : 0
NULL
|
In this cgroup section of the javacore file you can see that the dump was produced from a JVM that ran in a container. As container limits are set the cgroup information is displayed accordingly to the limits set.
Summary:
This diagnostic information helps developers to debug clearly by knowing if the JVM is running inside the container and if JVM is using container support to consider the limits set to the container.
For more on other diagnostic techniques for containerized Java, check out this post on how to gather diagnostic data from your containerized Java application on the fly. To learn more about OpenJ9 enhancements that optimize for running in containerized environments, please read this Eclipse OpenJ9 in Containers article .
As always, if you have questions or comments, please connect with us in our OpenJ9 slack workspace.