Setting up an OpenJ9 Development Environment with Visual Studio Code + Docker + CMake

Setting up a proficient development environment on a large project such as OpenJ9 with so many pieces in play can be a daunting task. Often times, setting up such a development environment and being able to navigate the codebase with familiar tools and with the full power of an IDE is worth the effort in terms of productivity. This blog post will guide you through setting up a development environment with a one-click build trigger, full Intellisense support, and full debugging support using Visual Studio Code, Docker, DevContainers extensions, and CMake.

Here are some examples of what we’ll be able to achieve:

Prerequisites

This guide requires you to have the following tools installed:

• Visual Studio Code
• Docker
• Git
• Bash

Building the Docker image

In this walkthrough, we will be building a Java 8 development environment. The steps are nearly identical for other versions of Java. Building the Docker image with the build environment is detailed in the Building OpenJDK Version 8 with OpenJ9 document on the official OpenJ9 repository. We’ll speed through the build steps and running the image within an interactive shell:

$ wget https://raw.githubusercontent.com/eclipse/openj9/master/buildenv/docker/mkdocker.sh
$ bash mkdocker.sh --tag=openj9 --dist=ubuntu --version=16.04 --build

...

$ docker images
 REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
 openj9              latest              5cf625b78c20        1 minute ago        2.38GB
$ docker run -it openj9
$ docker ps
CONTAINER ID        IMAGE                    COMMAND             CREATED             STATUS              PORTS                NAMES
 7f74d8bca90f        openj9                   "/bin/bash"         9 minutes ago       Up 9 minutes        22/tcp               elastic_mcnulty

Preparing the source code in the Docker image

Once the image is built, we follow the next step in the build documentation from the previous section and load the various repositories needed to build OpenJDK with OpenJ9. However, we will not be invoking the build system through the command line.

$ git clone https://github.com/ibmruntimes/openj9-openjdk-jdk8.git
$ cd openj9-openjdk-jdk8
$ bash get_source.sh

The next step requires us to configure the build system. We will be building OpenJ9 with the new CMake build system. Note that OpenJ9 will eventually migrate to the CMake build system as the default (eclipse/openj9/#90). The documentation in the repository explains how we have to modify the configure argument:

$ bash configure --with-cmake --with-boot-jdk=/home/jenkins/bootjdks/jdk8 --with-native-debug-symbols=internal

We added the --with-native-debug argument to prevent the OpenJDK build system from stripping off the debug symbols from the OpenJ9 libraries when the build system generates the JDK images.

Now we are prepared to build OpenJDK with OpenJ9. However, rather than building the project through the command line, we now switch over to VS Code to carry on the configuration. We will eventually invoke the build process through VS Code, so it will be able to parse all the CMake metadata and configure the IDE on our behalf.

Configuring Visual Studio Code

While the container is running, we will attach VS Code using the official Remote – Containers extension, which you can find on the Marketplace or by following the link provided. This extension will allow VS Code to connect to a running container where we will tell the editor how to configure the environment for building OpenJDK with OpenJ9.

Once the extension is installed, we can attach VS Code to our running container by opening up the command palette (Cmd/CTRL + Shift + P) and typing “Remote Attach”:

Select the running container SHA that we built from our image (7f74d8bca90f in my case). This will now open up a new VS Code window and on the bottom left you will note the editor is now attached to the running Docker container:

What the Remote Containers extension did was attach to our running Docker container and install a VS Code Node.js server that communicates with the editor running on your workstation to provide various metadata. Your local editor is just the visualization of the VS Code instance running on the server. You can see the server process running within the Docker container:

root@7f74d8bca90f:~# ps -ef | grep server.sh
 root       109    94  0 00:01 ?        00:00:00 sh /root/.vscode-server/bin/fcac248b077b55bae4ba5bab613fd6e9156c2f0c/server.sh --disable-user-env-probe --use-host-proxy --disable-telemetry --port 0 --extensions-download-dir /root/.vscode-server/extensionsCache

Because the container VS Code server is the “real” editor and our local client VS Code instance is just a GUI visualizing what is happening on the container, it means that if we want to install any additional extensions, we will have to do so on the container itself. Thankfully VS Code makes this trivial.

To get full Intellisense support, debugging support, and CMake support we require a few additional extensions. Rather than looking up each one, we will just install the C/C++ Extensions Pack, which includes everything we want. Note that we will be installing this extension within the container:

Once installed, open up the Explorer tab and open up the /openj9-openjdk-jdk8 directory where we cloned the repository in the previous steps:

Once the folder is opened, you will note the Explorer view will have the entire directory contents visible. Next, we need to tell the VS Code CMake extension how to build the OpenJ9 project. The CMake extension will parse the CMake metadata and feed it to the VS Code C++ extension, which will allow for Intellisense to work.

Configuring VS Code and many of its extensions is done through various JSON files. In our case, we need a total of three JSON configuration files:

.vscode/settings.json

This configuration file will alter the VS Code settings within the container to configure CMake. We will effectively replicate the locations and configuration that the OpenJDK build system will use to configure OpenJ9 via CMake. Create the .vscode/settings.json file in the root of the repository and add the following settings:

{
    "cmake.buildDirectory": "${workspaceFolder}/build/linux-x86_64-normal-server-release/vm",
    "cmake.sourceDirectory": "${workspaceFolder}/openj9",
    "cmake.configureSettings": {
        "BOOT_JDK": "/home/jenkins/bootjdks/jdk8",
        "JAVA_SPEC_VERSION": "8",
        "OPENJ9_BUILD": "ON",
        "OMR_SEPARATE_DEBUG_INFO": "OFF",
        "J9VM_OMR_DIR": "${workspaceFolder}/omr"
    },
    "cmake.cacheInit": "runtime/cmake/caches/linux_x86-64_cmprssptrs.cmake",
    "search.useIgnoreFiles": false,
    "search.exclude": {
        "**/build": true
    },
    "cmake.buildTask": true
}

Note that the above settings.json file is catered towards the Java 8 build. If you’re trying this with a different Java version, for example Java 11, you will have to change the "BOOT_JDK" and "JAVA_SPEC_VERSION" settings.

.vscode/tasks.json

This configuration file sets up two commands that will invoke the build system. The settings.json file configured the VS Code CMake extension to invoke a custom task on build. We will define that task here:

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "Build OpenJ9",
            "command": "build",
            "type": "cmake",
        },
        {
            "label": "Build OpenJDK",
            "command": "make all \"EXTRA_CMAKE_ARGS=-DOMR_SEPARATE_DEBUG_INFO=OFF\"",
            "type": "shell",
            "group": {
                "kind": "build",
                "isDefault": true
            },
        }
    ]
}

The "cmake" task is the build task that simply invokes the CMake build system. This task will build OpenJ9 but it will not integrate the built libraries into a JDK image. We require the OpenJDK build system to do that, which is why we define another task called "Build OpenJDK" that will invoke the OpenJDK build system and package the JDK image for us. The latter is our default build task that will invoke when we click the Build button or via the build hotkey (Shift + Cmd/CTRL + B).

.vscode/launch.json

This configuration file tells VS Code how to launch the process, in our case for purposes of debugging. It is mostly auto-generated but we will just change the launch command to invoke java -version. You can of course change this to whatever you wish.

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "(gdb) Launch",
            "type": "cppdbg",
            "request": "launch",
            "program": "/openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/bin/java",
            "args": [
                "-version"
            ],
            "stopAtEntry": false,
            "cwd": "${workspaceFolder}",
            "environment": [],
            "externalConsole": false,
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ]
        }
    ]
}

After all the files have been created, your Explorer window should look like this:

Building with VS Code

Open up the command palette again and launch the Tasks: Run Build Task command (Shift + Cmd/CTRL + B):

VS Code opens the Terminal dialog and the OpenJDK OpenJ9 build should start:

Once the build completes, you can launch our run configuration defined in launch.json to invoke java -version by pressing F5 or via the Run menu option. You should see something like this:

Making a small change

To showcase the power of our new setup, let’s make a small change in the OpenJ9 Java bytecode interpreter loop to print the name of every method, which we execute through the invokestatic bytecode. First we locate the inline function that handles this bytecode in ByteCodeInterpreter.hpp:6821 (open the file via Cmd/CTRL + P hotkey and type in the name)

We’ll add a few lines here which I’ve bolded:

    VMINLINE VM_BytecodeAction
    invokestatic(REGISTER_ARGS_LIST)
    {
        U_16 index = *(U_16*)(_pc + 1);
        profileCallingMethod(REGISTER_ARGS);
        J9RAMStaticMethodRef *ramMethodRef = ((J9RAMStaticMethodRef*)J9_CP_FROM_METHOD(_literals)) + index;
        _sendMethod = ramMethodRef->method;

        PORT_ACCESS_FROM_VMC(_currentThread);
        char methodName[1024] = { 0 };
        getMethodName(PORTLIB, _sendMethod, _pc, methodName);
        j9tty_printf(PORTLIB, "invokestatic on %s\n", methodName);

        return GOTO_RUN_METHOD;
    }

In addition we’ll need to make the getMethodName function visible outside an #ifdef. To do this, we simply remove the #if defined(TRACE_TRANSITIONS) and #endif lines around the function definition in ByteCodeInterpreter.inc:52

Now rebuild the project (Shift + Cmd/CTRL + B) and the change will now be integrated. When we run our launch target (F5) you should see all methods dispatched via the invokestatic bytecode printed to the screen. Cool!

Live debugging

Our setup also allows for full debugging support. Unfortunately we don’t get this right out of the box because the default OpenJ9 build is built with optimizations enabled, and many of the variables we would like to examine during a debugging session may be unavailable due to C/C++ compiler optimizations. The OpenJ9 project does not currently have a target to enable proper debug builds (for example via CMAKE_BUILD_TYPE) but it will in the future. Fortunately for us, we have to make only one simple change in the gnu.cmake:23 file to disable optimization. Simply change -O3 to -O0 and rebuild the project.

After the project has been rebuilt with optimizations disabled we can unleash the full power of visual debugging via the VS Code interface. For this showcase, let’s debug the JIT compiler. The JIT compiler has many optimization passes that it executes during compilation of a Java method. Simplification optimization is perhaps the easiest one to understand. Simplifier transformations are mostly mechanical transformations, think reduction of an addition of two constants into the actual sum at compile time. In this case we’re going to place a breakpoint in OMRSimplifierHandlers.cpp:3216, which carries out a multiplication reduction.

Note the breakpoint can be seen on the bottom left of the Run view (Shift + Cmd/CTRL + D). One other change we’ll make is to add an additional option to our run command in launch.json. We will add the -Xjit:count=0 option, which will force the JVM to carry out synchronous compilations on first invocation of every method. This will ensure that we will hit our breakpoint. To do this, we simply modify the args parameter within the launch.json file we created earlier:

            "args": [
                "-Xjit:count=0",
                "-version"
            ],

Now we run the application (F5). And voilà! We’ve hit our breakpoint.

On the left side, we can see the backtrace for each thread and the locals information. In this case we hit the breakpoint on a JIT Compilation thread.

This particular location where we’ve placed a breakpoint is doing a multiplication reduction. The string right above out breakpoint gives a brief description of this transformation. It is transforming a multiplication into a shift with an addition or subtraction.

But what method are we compiling? Are we able to see this multiplication right in the Java code? To find out, we can use the Watch panel within the Run view (Shift Cmd/CTRL + D) to input an expression. The Watch view is shown on the middle left of the screen in the screenshot above. Try inputting the following expression to find the method we are currently compiling:

comp->signature()

In this case, the value of the expression is:

0x7f48f93f6ec0 "sun/util/locale/BaseLocale$Key.(Ljava/lang/String;Ljava/lang/String;)V"

Great, but can we find out more information? For example, what was the left hand side and right hand side of the multiplication, and what was the line number corresponding to this multiplication in the source code? We can of course! To find the line number we can add another expression:

comp->getLineNumber(node)

This expression yields 193. Progress, yay! What were the operands though? For that, we can place our cursor right over the node variable and take a peek inside. Navigating the data structures within node can tell us. For example, the node has two children (since multiplication has two operands), and taking a look at the second child we can see it is a JIT opcode of type TR::iconst (which represents 32-bit integer constants) and the value of the constant is 31:

Now we can go to the OpenJDK extensions repository and locate this expression in the Java source code to understand what is happening. We know the full method signature, the line number, and the value in question, so let’s correlate that.

We lookup the BaseLocale.java file on the openj9-openjdk-jdk8 repository and we find BaseLocale.java:193. As we can see, there is a multiplication by 31 on this line, just as we discovered:

h = 31 * h + LocaleUtils.toLower(region.charAt(i));

In this case, the JIT compiler is choosing to decompose this multiplication into the following expression:

h = (h << 5) - h + LocaleUtils.toLower(region.charAt(i));

which is semantically equivalent, but a more performant expression on the processor we are running on. Typically multiplications are more expensive operations to the processor than shifts and additions, which is why such a transformation is beneficial.

Tips and tricks

This guide explains the bare bones setup one can create. However, a crafty developer can of course tune everything to their own liking. Here are some tips and tricks I’ve discovered that are too verbose for this blog post but which you may employ in your own setup:

• The Remote – SSH extension works in almost the same way, allowing you to connect to a remote SSH server (could be a cloud hosted VM for example) instead of a Docker container
• You can mount Docker volumes on your image so your source code does not have to reside within the Docker container itself
• You can use a .devcontainer/devcontainer.json to teach VS Code how to automatically launch a fresh container with a full setup with a one button click
• You can use the "Build OpenJ9" task we’ve created earlier to only launch the OpenJ9 CMake build, which is super fast if you’re making frequent changes. This way you don’t have to wait for a lengthy (2 mins in my case) OpenJDK build system to generate the images. Once the libraries are built within the build/*/vm/ directory, you can just rsync them over into the build/*/images/ directory and have a very fast compile, run, debug cycle.
• You can setup c_cpp_properties.json to parse non-x86 code generators for Intellisense support (for example for AArch64, Power, Z, etc.)

Conclusion

In this blog post we explored a minimal, yet very powerful setup for developers to get started on debugging/developing features on OpenJ9. We’ve setup VS Code to be able to one-click build and run the OpenJDK with OpenJ9 project and we were able to make changes and debug Java programs with ease. If you have any further questions or would like to share your tips on improving the developer experience, feel free to come to our Slack instance and share your expertise. Until then, happy hacking!