Introducing the Eclipse OpenJ9 JITServer Helm Chart

Posted on by chrisc666

What is JITServer Technology?

JITServer is an Eclipse OpenJ9 JVM technology that allows the JVM to delegate its JIT compilation duties to a separate process (the JITServer) that could be running on a remote machine. The JIT offloading approach offers numerous advantages, from improved start-up and ramp-up time, to reduced peak memory usage, reduced likelihood of out-of-memory scenarios and increased resiliency in the face of spurious crashes due to bugs in the JIT compiler. Moreover, the resource provisioning task is much simpler because the user can focus solely on the needs of her Java application rather than being forced to take into account the unexpected spikes in CPU or memory caused by compilation.

Starting with OpenJ9 release 0.18.1, JITServer is available as a technology preview on x86-64 Linux, while OpenJ9 release 0.23 has extended platform support to include 64-bit Linux on IBM Power systems and 64-bit Linux on IBM Z systems.

JITServer in Cloud Environments

Cloud computing is transforming the way of consuming computer services in a number of industries. Research shows that 90% of companies are transforming or have transformed their services to public, private or hybrid cloud environment. Furthermore, experts predict that 60% of commercial workloads are running on cloud environment [1] [2].

The JITServer technology is a natural fit for a cloud environment. The advantages of JITServer are more likely to become apparent in resource constrained environments (typically found in the cloud) where the application and JIT compiler are forced to compete for CPU and memory. Furthermore, JITServer effectively reduces the peak memory usage of a client JVM, allowing smaller containers to be used, increasing application density and thus, reducing operational costs. JITServer can be easily containerized and deployed to a cloud native environment (e.g. Kubernetes, OpenShift, etc.), which makes it even easier to run Java applications in densely packed cloud environments.

Eclipse OpenJ9 JITServer Helm Chart

You are at the right place if you would like to try out JITServer in a cloud environment. The OpenJ9 JITServer helm chart provides an easy and straightforward option for deploying and enabling JITServer technology with three simple steps. In the rest of this section, I will walk through the requirements and show you how to deploy the JITServer helm chart.

Requirements

  • Same JDK release version on both JITServer and client JVM
  • Basic knowledge of cluster administration and operation
  • Cluster
    • Kubernetes 3.11+, or
    • RedHat OpenShift 4.1+
  • Cluster RBAC
    • Kubernetes PodSecurityPolicies (PSP): spec.runAsUser: RunAsAny, or
    • RedHat OpenShift SecurityContextConstraints(SCC): anyuid
  • Helm CLI version 3.0+

Steps to deploying JITServer Technology

  1. Locating JITServer chart from helm repo

    1. Add OpenJ9 helm repo to your helm CLI

      $ helm repo add openj9 https://raw.githubusercontent.com/eclipse/openj9- 
utils/master/helm-chart/
"openj9" has been added to your repositories

    2. Verify helm repo is added and OpenJ9 JITServer chart is available

      $ helm repo ls
NAME       	URL
openj9     	https://raw.githubusercontent.com/eclipse/openj9-utils/master/
helm-chart/

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "openj9" chart repository
Update Complete. ⎈ Happy Helming!⎈

$ helm search repo openj9-jitserver
NAME                           CHART VERSION   APP VERSION   DESCRIPTION
openj9/openj9-jitserver-chart  0.24.0          0.24.0        Eclipse OpenJ9 
JITServer Helm Chart.

  2. Deploying JITServer chart on your cluster

    1. Install helm chart

      $ helm install jitserver-release openj9/openj9-jitserver-chart
NAME: jitserver-release
LAST DEPLOYED: Fri Jan 29 20:11:19 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
Welcome to OpenJ9 JITServer Helm Chart. The JITServer technology has been 
deployed successfully and ready for connection.

    2. Test chart deployment

      $ helm test jitserver-release
Pod jitserver-release-openj9-jitserver-chart-test-connection pending
Pod jitserver-release-openj9-jitserver-chart-test-connection succeeded
NAME: jitserver-release
LAST DEPLOYED: Fri Jan 29 20:11:19 2021
NAMESPACE: helm-chart-test
STATUS: deployed
REVISION: 1
TEST SUITE:     jitserver-release-openj9-jitserver-chart-test-connection
Last Started:   Fri Jan 29 20:41:14 2021
Last Completed: Fri Jan 29 20:41:26 2021
Phase:          Succeeded
NOTES:
Welcome to OpenJ9 JITServer Helm Chart, the application has been 
deployed successfully.

      Congratulations! The JITServer technology has been successfully deployed to your cluster.

  3. Enabling JITServer technology for your java application

    1. Find JITServer service endpoint (service name and port)

      $ export SERVICE_IP=$(kubectl get service --namespace default jitserver-release- 
openj9-jitserver-chart -o jsonpath="{..metadata.name}")
$ export SERVICE_PORT=$(kubectl get service --namespace default jitserver-release- 
openj9-jitserver-chart -o jsonpath="{.spec.ports[0].port}")
$ echo JITServer endpoint is $SERVICE_IP:$SERVICE_PORT
JITServer endpoint is jitserver-release-openj9-jitserver-chart:38400

    2. Update the environment variable JAVA_OPTIONS for your java application container to the following value (updating method may vary depending on application deployment):

      JAVA_OPTIONS = "-XX:+UseJITServer -XX:JITServerAddress=<SERVICE_IP> - 
XX:JITServerPort=<SERVICE_PORT>"

Configuring OpenJ9 JITServer Technology

Although the default configuration provided by the helm chart is sufficient to deploy a JITServer instance that is ready to serve a client JVM, sometimes you may wish to configure the JITServer and client JVM further. This section lists all available JITServer options that can be passed through a JVM environment variable, such as JAVA_OPTIONS.

  • Configuring JITServer address and port number

    • -XX:JITServerAddress=<String> Specifies the service name or IP address of JITServer. This value is set to localhost by default. This option takes effect only on the client JVM
    • -XX:JITServerPort=<Integer> Specifies the port number that JITServer listens on and client JVM connects to. This value is set to 38400 by default.

  • Configuring JIT compilation requests timeout value

    • -XX:JITServerTimeout=<Integer> Specifies a timeout value (in milliseconds) for socket operations. This value is set to 30000 ms for JITServer and 2000 ms for client JVM by default. The latter value may need to be increased accordingly if network latency is large.

  • Configuring network communication encryption between JITServer and client JVM using OpenSSL

    • Generate OpenSSL key and certificate using version 1.0.x or 1.1.x.

      openssl genrsa -out key.pem 2048

      openssl req -new -x509 -sha256 -key key.pem -out cert.pem -days 365

    • -XX:JITServerSSLKey=key.pem -XX:JITServerSSLCert=cert.pem Specify the private key and the certificate files at JITServer.

    • -XX:JITServerSSLRootCerts=cert.pem Specifies the certificate file at client JVM.

In order to include additional configurations during JITServer deployment, please update JAVA_OPTIONS environment variable in the helm chart. This can be done by adding “—set” options. For example, configuring JITServer to listen on port 9000 and to use a timeout value of 40000 ms can be achieved with options as shown below.

$ helm install --set env[0].name="JAVA_OPTIONS" --set env[0].value=
"-XX:JITServerPort=9000 -XX:JITServerTimeout=40000 " openj9-release openj9/openj9-
jitserver-chart

OpenJ9 JITServer Technology Deployment Considerations

Since cloud environments are usually complex and contain many moving parts, JITServer deployment topology may vary between different clusters and infrastructures. This section offers rules of thumb for optimizing JITServer technology functionality and performance on most clusters.

Compatible JDK Version

Please note that JITServer technology enforces version compatibility between JITServer and client JVM. In case of a version mismatch, JITServer will refuse connections from the client, which in turn may lead to performance degradations or even out-of-memory scenarios at the client, if deployments were configured based on the assumption that JITServer will be used.

The easiest and the recommended way to ensure compatibility is to base your Java application image and your JITServer image on the same Eclipse OpenJ9 container image that is available on Adopt OpenJDK Docker Hub. This guarantees the same JDK release version on both JITServer and client JVM.

An alternative method is manually matching the JDK release version. Inside your Java application image, run “java -version” to check the JDK release version. In the example below, the OpenJ9 “Java 11” JDK is running on “0.24.0” release version, “Linux” amd64 architecture. Find the corresponding image from Adopt OpenJDK Docker Hub through keywords search of Java version, release version and architecture.

$ java -version
openjdk version "11.0.10" 2021-01-19
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.10+9)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.24.0, JRE 11 Linux amd64-64-Bit Compressed
References 20210120_910 (JIT enabled, AOT enabled)
OpenJ9   - 345e1b09e
OMR      - 741e94ea8
JCL      - 0a86953833 based on jdk-11.0.10+9)

Lastly, override helm chart image repository and tag to Java application or Adopt OpenJDK image using the “--set” option during helm chart installation. As shown below.

helm install –set image.repository=”IMAGE_REPO” --set image.tag="IMAGE_TAG” jitserver-
release openj9-jitserver-chart

Deployment Topology

One or more client JVM(s) can connect to one JITServer instance as long as they are running the same JDK release version and enough resources are allocated to the JITServer instance. However, a single client JVM is not allowed to connect to multiple JITServer instances. To prevent this in a Kubernetes environment, the service for JITServer needs to have service.spec.sessionAffinity set to ClientIP.

Normally, Kubernetes clusters assign pods to nodes automatically based on cluster workload. However, developers with additional information of the workload can manually assign pods to particular nodes to further optimize performance with the help of node affinity. This technique could be used sometimes to place the client JVM and JITServer pods on the same node to minimize the network latency between the two.

Life Cycle Management

In some cases, you may want to upgrade the client JVM to a newer java version. The JITServer instance must be compatible with the client JVMs it serves. The safest way to do this is to deploy another JITServer helm chart with a newer version before upgrading the client JVM, then remove old JITServer instances.

If at all possible, avoid removing the JITServer instances before client JVMs (applications) terminate. If the memory limit for the container running the client JVM has been set based on the assumption that JITServer handles all JIT compilations, the application may experience a native out-of-memory scenario and be terminated. Once all applications are retired, you can safely remove the JITServer instance as shown below.

$ helm delete jitserver-release
release " jitserver-release " uninstalled

Container Resource Allocation

The following are recommended minimum values for CPU and memory resources at JITServer, based on the assumption of a single application connecting to the JITServer instance.

  • CPU Requested: 1 CPU
  • Memory Requested: 512 MB
  • Storage: none

If multiple applications connect concurrently to one JITServer instance, multiply the above values accordingly. Please note that, if multiple applications connect to the JITServer in a staggered fashion (i.e. with some delay between applications), then you could lower the above requirements.

Demo: Enabling JITServer Technology on a Sample Java Application

In this section, a step-by-step demo is provided that illustrates the process of enabling JITServer technology on Open Liberty Application Server as a sample java application. The application image is available at Open Liberty docker hub. Notice that the application image used in this demo is open-liberty:21.0.0.2-full-java8-openj9, which uses adoptopenjdk:8u282-b08-jre-openj9-0.24.0 as base image and is deployed as JITServer.

  1. Deploy a JITServer instance before deploying Java application

    First, deploy the OpenJ9 image as a JITServer instance using the helm chart. Override the image repository and tag with "--set" options.

    $ helm install jitserver-release --set image.repository=adoptopenjdk --set 
image.tag=8u282-b08-jre-openj9-0.24.0 openj9/openj9-jitserver-chart
NAME: jitserver-release
LAST DEPLOYED: Wed Feb 10 21:11:39 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
...

  2. Enable JITServer technology during application deployment

    Once the JITServer instance is deployed, deploy the java application image with JVM options that enables JITServer technology. The Open Liberty application is deployed using Open Liberty helm chart.

    The environment variable jvmArgs passes JITServer options -XX:+UseJITServer -XX:JITServerAddress=jitserver-release-openj9-jitserver-chart into client JVM.

    $ helm install my-web-app --set image.repository=open-liberty --set image.tag=21.0.0.2- 
full-java8-openj9 --set env.jvmArgs="-XX:+UseJITServer -XX:JITServerAddress=jitserver-
release-openj9-jitserver-chart" ibm-charts/ibm-open-liberty
NAME: my-web-app
LAST DEPLOYED: Wed Feb 10 21:15:43 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
...

  3. Verify that JITServer technology is enabled by observing verbose log or container resource consumption.

    Generated verbose logs on JITServer instance is an indicator of JITServer enablement. This can be achieved by appending -Xjit:verbose={JITServer} to the JVM options and verifying JITServer verbose logs appear in /tmp/output.log.

    $ kubectl exec jitserver-release-openj9-jitserver-chart-8695b7584f-k56lb cat 
/tmp/output.log
JITServer is currently a technology preview. Its use is not yet supported.
...
JITServer is ready to accept incoming requests
#JITServer: Server received request for stream 00007FBAFB13C2F0
...
#JITServer: compThreadID=0 has successfully compiled 
java/lang/Double.longBitsToDouble(J)D

Conclusion

The OpenJ9 JITServer technology is a great feature to try out if you are running java applications on cloud or in any resource-constrained environment. The JITServer helm chart automates the deployment process on Kubernetes or OpenShift clusters. Default configurations are packed into the helm chart which deploys JITServer within three simple steps, while it preserves the option for adding user-defined configurations for further optimization. Enabling JITServer technology on OpenJ9 JVMs allows your container to run with smaller size as well as improved throughput performance.

Want to know more about JITServer technology? Please check out the OpenJ9 JITServer helm chart repo and more interesting blogs on JITServer.

Leave a Reply