State of the art Java Virtual Machines (JVMs) employ Just-in-Time (JIT) compilers to improve the throughput of Java applications. However, JIT compilers do not come for free: they consume resources, in terms of CPU and memory, and therefore they can interfere with the smooth running of Java applications. Wouldn’t it be nice to keep all the advantages of dynamic compilation and eliminate its disadvantages? This is exactly what JITServer technology proposes to do. In Eclipse Openj9 JVM we have decoupled the JIT compiler from the rest of the JVM and made it run in its own independent process, either locally or on a remote machine. This process, referred to as JITServer, can easily be containerized and run in the cloud as a service, where it can be managed intelligently by an off-the-shelf orchestrator like Kubernetes or Docker Swarm.
Haven’t you just moved the JIT compilation overhead around?
On the surface JITServer moves the compilation overhead from one place to another, but such compilation consolidation strategy can reap the following benefits:
- Spikes in memory consumption due to compilation activity at the client JVM are eliminated reducing the likelihood of a spurious out-of-memory occurrence.
- The JIT compiler no longer steals CPU cycles from the Java application, thus eliminating performance hiccups, improving the quality-of-service (QoS) and possibly leading to a faster startup/rampup experience.
- Application resource provisioning is greatly simplified because the user can ignore the unpredictable effects of JIT compilation and focus on the CPU/memory needs of the Java application alone.
- Container sizing can be based solely on the resource usage of the Java application resulting in smaller containers, increased application density and reduced overall cost.
- Robustness of applications is increased because compile time crashes due to bugs in the JIT no longer bring down the JVM.
- Greater control over the resources devoted to compilation can be achieved because the number of JITServer container instances (and their size) can be scaled up and down independently of the Java applications. It’s worth mentioning that a single JITServer can fulfill the compilation needs for tens (if not hundreds) of JVMs. Moreover, for a typical usage pattern where requests from N client JVMs are staggered over time, the memory consumption at the server does not have to increase N times.
What’s the catch?
As you guessed, it’s not all rosy and we want to be forthcoming about disadvantages. The biggest challenge for this technology is network latency. In a perfect world the client JVM issues a compilation request and the JITServer replies with a compiled method body. In reality, during the compilation process, the JITServer sends many queries to the client asking it about the runtime environment: classes, methods, fields, profiling information, class hierarchy, etc. The JITServer employs aggressive caching to reduce the back-and-forth network communication, but because the requested information can change at runtime, caching is only partially effective.
Therefore, on one hand compilations are taking longer due to network latency, but on the other hand the JVM is no longer spending CPU cycles or memory on JIT compilation. These are antagonistic effects and the end result depends on the environment: If your application has plenty of CPU and memory resources at its disposal relative to its compilation needs, then a traditional JVM with embedded JIT compiler is still the better choice. Conversely, if your JVM needs to compile many methods in a CPU and memory constrained environment, then you should definitely give JITServer technology a try.
I am sold! Tell me how to use it
At the time of this writing JITServer technology is offered as a preview and is supported for Java8/11 on Linux on x86-64 (support for Linux on Z and Power systems is coming soon!). While we’ve tested functionality on the following Linux distributions: Ubuntu 16.04, Ubuntu 18.04, RHEL 7.6 and CentOS 7.6, we expect other modern Linux distributions to work equally well.
For those of you who want to build a JDK with JITServer technology we’ve included detailed instructions in Appendix A of this blog. It should be noted that, starting with release 0.18.0, the OpenJ9 builds available from AdoptOpenJDK (https://adoptopenjdk.net/) have JITServer technology embedded in them. Thus, all you need to do is use the right command line options to switch between three different personas:
- To launch OpenJ9 as a regular JVM nothing special needs to be done; just use the Java command as usual.
- To launch OpenJ9 in JVM client mode we need to specify “-XX:+UseJITServer” on the Java command line. Note that the client operates in vanilla JVM fashion if the server is not available.
- To launch OpenJ9 in server mode use the “jitserver” command. Under the covers this starts a JVM that listens for incoming compilation requests.
The following command line options can be used to further configure the JITServer and the client JVM processes:
- -XX:JITServerPort=<Integer> Specifies the port the server listens to for compilation requests. Default is 38400
- -XX:JITServerAddress=<String> Specifies the name or IP of the server. Default is localhost. This option makes sense only for the client JVM
- -XX:JITServerTimeout=<Integer> Specifies a timeout value (in milliseconds) for socket operations. Default is 30000 ms for JITServer and 2000 ms for the client JVM. The latter value may need to be increased if network latency is large
I am concerned about security. Do you support traffic encryption?
Absolutely! If so desired, the network communication between the client JVM and JITServer can be encrypted using OpenSSL 1.0.x or 1.1.x. To enable encryption, specify the private key and the certificate at the server:
-XX:JITServerSSLKey=key.pem -XX:JITServerSSLCert=cert.pemand use the certificate at the client:
-XX:JITServerSSLRootCerts=cert.pem To generate the keys and some self-signed certificates you could use something like:
$ openssl genrsa -out key.pem 2048
$ openssl req -new -x509 -sha256 -key key.pem -out cert.pem -days 365When generating the certificate, the Common Name (CN), should be the fully qualified domain name of the host that you intend to use the certificate with (if you plan on running the server on the same machine as the client, CN should be localhost).
Please note that encryption adds a significant amount of CPU overhead which in turn increases compilation latency, therefore consider whether you can do without. It may be helpful to know that the information exchanged between the client and server is limited to Java classes (bytecodes, class/method/field names, class hierarchy) and profiling information and does not include any user data that is stored in the Java heap (static final fields are an exception). If you enable encryption at the server, the JITServer only accepts encrypted requests, thus, to deal with un-encrypted traffic a separate instance of JITServer needs to be launched.
Conclusion
The JITServer technology preview embedded in the OpenJ9 JVM can offer relief from the negative effects of JIT compilation. As long as the network latency is reasonable, this technique could prove very useful for Java applications with many classes/methods that are running in resource constrained environments. Hence, this could be an effective way of reducing container sizes, improve application density and reduce overall cost. A performance evaluation of this technology is coming shortly in a follow-on blog.
Appendix A: Building an SDK with JITServer technology.
From the build point of view, JITServer has the same prerequisites as OpenJ9 (see https://www.eclipse.org/openj9/oj9_build.html) plus the protobuf package version 3.7.1. The latter can be easily compiled and installed on your system with the following commands (please note that gcc 7.3 or higher needs to be present):
$ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-cpp-3.7.1.tar.gz
$ tar -xvzf protobuf-cpp-3.7.1.tar.gz
$ cd protobuf-3.7.1
$ ./configure --disable-shared --with-pic
$ make
$ make install
$ ldconfigThe build process follows the same steps as for building a vanilla OpenJ9 JVM except that “–enable-jitserver” option needs to be provided to the “configure” command:
$ git clone https://github.com/ibmruntimes/openj9-openjdk-jdk8.git
$ cd openj9-openjdk-jdk8
$ bash get_source.sh
$ bash configure --with-freemarker-jar=/root/freemarker.jar --enable-jitserver
$ make clean
$ make all