OpenJ9 has been using the JPP to produce a custom set of libraries from a single JCL codebase since its inception. The customized libraries are used to build OpenJ9 JDKs. Initially, only the OpenJ9 JCL codebase is structured to support various JPP configurations.
Recently the JPP is extended to preprocess some of the JCL code in the extensions for OpenJDK. For the relation between OpenJ9 and the extensions, refer to What is Eclipse OpenJ9?
This blog first gives a short overview of JPP usage, then shows how it is extended to the extensions.
JPP expressions
There are a set of JPP expressions that appear as part of the JCL source code. Each expression evaluates to either true or false. If you are familiar with the use of #ifdef and #if in C compiler pre-processors then this will be recognizable to you.
The JPP expressions are written directly in the source code as comments.
Decide if a file is included
/*[INCLUDE-IF (JAVA_SPEC_VERSION >= 8) & CRIU_SUPPORT]*/
Only include the following code if the condition is met
/*[IF JAVA_SPEC_VERSION >= 11]*/
Only include the following code if the condition is NOT met
/*[ELSE] JAVA_SPEC_VERSION >= 11 */
End a directive
/*[ENDIF] JAVA_SPEC_VERSION >= 11 */
There are other JPP expression usages that are not covered here.
JPP configuration file
jpp_configuration.xml is used to keep configuration information. JPP first reads this file and loads configurations such as output path, flags, parameters, etc. before it builds a customized library.
A sample configuration for Java 17
<configuration
label="JAVA17"
outputpath="JAVA17/src"
flags="OPENJDK_METHODHANDLES"
dependencies="JAVA11"
jdkcompliance="1.8">
<classpathentry kind="src" path="src/java.base/share/classes"/>
<classpathentry kind="src" path="src/java.management/share/classes"/>
<classpathentry kind="src" path="src/jdk.attach/share/classes"/>
<classpathentry kind="src" path="src/jdk.jcmd/share/classes"/>
<classpathentry kind="src" path="src/jdk.management/share/classes"/>
<classpathentry kind="src" path="src/openj9.criu/share/classes"/>
<classpathentry kind="src" path="src/openj9.cuda/share/classes"/>
<classpathentry kind="src" path="src/openj9.dataaccess/share/classes"/>
<classpathentry kind="src" path="src/openj9.dtfj/share/classes"/>
<classpathentry kind="src" path="src/openj9.dtfjview/share/classes"/>
<classpathentry kind="src" path="src/openj9.gpu/share/classes"/>
<classpathentry kind="src" path="src/openj9.jvm/share/classes"/>
<classpathentry kind="src" path="src/openj9.sharedclasses/share/classes"/>
<classpathentry kind="src" path="src/openj9.traceformat/share/classes"/>
<classpathentry kind="src" path="src/openj9.zosconditionhandling/share/classes"/>
<classpathentry kind="lib" path="/binaries/common/ibm/ibmjzos.jar"/>
<classpathentry kind="lib" path="/binaries/vm/third/rt-compressed.sunJava17.jar"/>
<source path="src"/>
<parameter name="macro:define" value="JAVA_SPEC_VERSION=17"/>
<parameter name="msg:outputdir" value="java.base/share/classes/com/ibm/oti/util"/>
<parameter name="jxerules:outputdir" value="java/lang"/>
</configuration>
The set of flags defined by flags=
and those inherited from dependencies=
are used in JPP expressions. JAVA_SPEC_VERSION
is a special variable, always available, that equates to the version being built. Otherwise, only the flags found in the jpp_configuration.xml (and the -tag:define
command line option) are valid for use in expressions. Anything else results in an error when pre-processing.
JPP is built from source
JPP is built from the source in each OpenJ9 build.
A sample log output
Building OpenJ9 Java Preprocessor
Building /home/jenkins/workspace/Build_JDK17_x86-64_linux_OpenJDK17/build/linux-x86_64-server-release/support/j9tools/jpp.jar
JPP headless
OpenJ9 builds run JPP headless to generate the JCL code for its JDK level. As explained above, the source code is augmented by JPP expressions that define various subsets for the supported java versions and components.
Command line to preprocess OpenJ9 JCL source files
java -cp jpp.jar -Dfile.encoding=US-ASCII com.ibm.jpp.commandline.CommandlineBuilder -verdict -config JAVA17 -baseDir openj9-openjdk-jdk17/openj9/ -srcRoot jcl/ -xml jpp_configuration.xml -dest build/linux-x86_64-server-release/support/j9jcl -tag:define CRIU_SUPPORT;PLATFORM-xa64
The -config JAVA17
corresponds to <configuration label="JAVA17"
in jpp_configuration.xml.
The flags defined by -tag:define
are added in addition to the flags="OPENJDK_METHODHANDLES"
that is part of the JPP configuration, and also those inherited from dependencies="JAVA11"
.
Command line to preprocess DDR source files
java -cp jpp.jar -Dfile.encoding=US-ASCII com.ibm.jpp.commandline.CommandlineBuilder -verdict -config GENERIC -baseDir openj9-openjdk-jdk17/openj9/debugtools/DDR_VM/ -srcRoot src/ -xml jpp_configuration.xml -dest build/linux-x86_64-server-release/support/j9jcl/openj9.dtfj/share/classes -tag:define CRIU_SUPPORT;PLATFORM-xa64
JPP Plugin
The JPP Plugin is an Eclipse front-end to the OpenJ9 JCL Preprocessor.

The JPP plugin generates a pConfig
Eclipse Java project by using the outputpath
of the config i.e. pConfig JAVA17
for the JAVA17
configuration, which contains the preprocessed OpenJ9 Java code for Java 17
.
Any changes in OpenJ9 JCL are preprocessed and the pConfig project is updated as source files are saved.
JPP is extended to the Extensions for OpenJDK for OpenJ9
All source files within the closed folder are preprocessed by JPP.
java -cp jpp.jar -Dfile.encoding=US-ASCII com.ibm.jpp.commandline.CommandlineBuilder -verdict -config JAVA17 -baseDir openj9-openjdk-jdk17/ -srcRoot closed/ -xml jpp_configuration.xml -dest build/linux-x86_64-server-release/support/j9jcl -tag:define CRIU_SUPPORT;PLATFORM-xa64 -includeIfUnsure -noWarnIncludeIf
All source files copied into an overlay folder are preprocessed by JPP.
java -cp jpp.jar -Dfile.encoding=US-ASCII com.ibm.jpp.commandline.CommandlineBuilder -verdict -config JAVA17 -baseDir openj9-openjdk-jdk17/build/linux-x86_64-server-release/support/ -srcRoot overlay/ -xml jpp_configuration.xml -dest build/linux-x86_64-server-release/support/j9jcl -tag:define CRIU_SUPPORT;PLATFORM-xa64 -includeIfUnsure -noWarnIncludeIf
Because JPP doesn’t support the whole OpenJDK codebase, the selected Java files to be preprocessed by JPP are required to be copied using the following script. If you want to preprocess a new file, modify this script code snippet.
$(eval $(call SetupCopyFiles,COPY_OVERLAY_FILES, \
SRC := $(TOPDIR), \
DEST := $(SUPPORT_OUTPUTDIR)/overlay, \
FILES := \
src/java.base/share/classes/java/lang/ClassValue.java \
src/java.base/share/classes/java/security/Security.java \
src/java.base/share/classes/java/util/Timer.java \
src/java.base/share/classes/java/util/TimerTask.java \
src/java.base/share/classes/java/util/concurrent/ConcurrentHashMap.java \
src/java.base/unix/classes/java/lang/ProcessEnvironment.java \
))
There is a special case to handle StandardCharsets.java
since it is generated from StandardCharsets.java.template separately by GensrcCharsetMapping.gmk
.
$(CP) $(CHARSET_GENSRC_JAVA_DIR_BASE)/StandardCharsets.java $(SUPPORT_OUTPUTDIR)/overlay-gensrc/src/java.base/sun/nio/cs/
$(call RunJPP, JAVA$(VERSION_FEATURE), $(SUPPORT_OUTPUTDIR)/overlay-gensrc, /overlay-result)
$(CP) $(SUPPORT_OUTPUTDIR)/overlay-result/java.base/sun/nio/cs/StandardCharsets.java $(CHARSET_GENSRC_JAVA_DIR_BASE)/
java -cp jpp.jar -Dfile.encoding=US-ASCII com.ibm.jpp.commandline.CommandlineBuilder -verdict -config JAVA17 -baseDir openj9-openjdk-jdk17/build/linux-x86_64-server-release/support/ -srcRoot overlay-gensrc/ -xml jpp_configuration.xml -dest build/linux-x86_64-server-release/support/overlay-result -tag:define CRIU_SUPPORT;PLATFORM-xa64 -includeIfUnsure -noWarnIncludeIf
Summary
OpenJ9 JPP is extended to preprocess some selected folders and files within the OpenJ9 extensions of OpenJDK. Note the JPP Eclipse plugin doesn’t preprocess those files yet.
If you need a hand adding java files to be preprocessed by JPP, feel free open an OpenJ9 issue or reach out to me directly.
References:
[1] CRIU JPP for unix/classes/java/lang/ProcessEnvironment.java
[2] Apply JPP to JCL patch files within $(TOPDIR)/closed
[3] Add IncludeIfUnsure for $(SUPPORT_OUTPUTDIR)/overlay