Continuing my team’s work from the 2018 Extreme Blue (EB) term, I spent the first two weeks of my co-op wrapping up our JITaaS project so that it could be merged. During EB, my team was responsible for implementing a prototype for JITaaS (Just in Time Compiler as a Service). Runtimes are comprised of various components, such as garbage collection, porting, threading, and others. One of those components is the JIT, also known as dynamic compiler, which compiles the program at runtime, optimizing it according to application needs. With the current JIT architecture, every runtime contains their own JIT, which takes up a significant amount of space. Every time a program is run, the program or function needs to be JIT’d and that takes time. JITaaS removes the need for a JIT from each client, by putting it on the server, which reduces runtime footprint for each client. Instead of running and compiling everything on the client side, the client sends the Intermediate Language (IL) to be compiled by the JIT on the server side. Then, the server answers with machine code so that the client can continue to run the program. So, the first two weeks I spent cleaning the code and making sure the first part of this project was ready to merge. There are other parts to the technology which will be posted as soon as patents material is processed.
As for the rest of the co-op term, I was responsible for the improvement of arraylets processing. What are arraylets? When an array or any object is created in a Java program, the Garbage Collector (GC) is responsible for storing it in the heap. The GC has different policies, which is just another way of saying that the GC has different ways of managing and storing the object associated with the running program (more information on GC policies [1]). Some policies, like balanced [2], are region-based, meaning the heap associated with the running program and managed by the GC is divided into regions. Objects allocated by the program are stored into these regions, and the GC will allocate, free, and compact the heap as it sees fit. Therefore, when a big array is created, big enough that it won’t fit in a single region, the array will be stored as an arraylet [3]. The array will be stored in multiple regions which will eventually become a discontinuous or hybrid array. To represent such array, the GC creates an arraylet spine which has class and size information as well as pointers to the data associated with the array.
The following is a very high-level representation of an arraylet stored in a heap (not drawn to scale). Numbers represent the actual array data, and asterisks represent data being modified.
For simplicity, #1, #2, and #3 are pointers to the regions where the remainder of the array data is located. So, if this data were to be represented in a contiguous form, it would look something like:
There is a problem, however, with this array representation. JNI Critical array operations require contiguous representations of the array, which is not the case for arraylets. Therefore, whenever JNI Critical needs to use large arrays such as this one, it creates a temporary array, copies index by index all elements from the array where it is located in a leaf to the temporary array, performs the required operations and when it’s done, it copies everything back to the arraylet leaves. This process is quite expensive, and we introduced arraylet double mapping to remedy this issue. (A more detailed explanation of arraylets, arraylet leaves, and double mapping is available on the following posts: https://eclipse-omr.org/double-map-arraylets/ and https://blog.openj9.org/2019/04/26/double-map-arraylets.)
The illustration above depicts in a high level the objective of double mapping. Whenever a large array is created, the GC also creates a contiguous block of memory which serves as a mirror to the original array (which is stored as arraylet leaves). Each region containing the array data will be double mapped to a region of this contiguous block of memory. This is allowed because we can use mmap [4] (Linux) and MapViewOfFileEx [5] (Windows) to map an existing mapped object to another region of memory. The advantages of this array representation is two-fold. Any method requesting a contiguous representation of the array won’t need to copy element by element, because it can just use the “mirror” image of the arraylets (contiguous double mapped region of memory), meaning, every modification made to the contiguous region of memory will resonate to the arraylet leaves as well. Additionally, this representation not only reduces the size of arraylet spines, but it also reduces lookup time.
We were able to finish the implementation part for Linux systems. Experiments showed that when double mapping is used, JNI Critical performs array operations up to 30 times faster than before. We are currently working on the Windows platform so that we can leverage double mapping advantages there as well. Finally, after Windows we will aim for the AIX platform.
References:
[1] OpenJ9 GC -Xgcpolicy URL: https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/openj9/xgcpolicy/index.html
[2] Balanced Garbage Collection Policy URL: https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/com.ibm.java.vm.80.doc/docs/mm_gc_balanced.html
[3] Memory Management in Eclipse OpenJ9, Kim Briggs, January 11, 2019 https://developer.ibm.com/articles/garbage-collection-tradeoffs-and-tuning-with-openj9/
[4] mmap documentation page, http://man7.org/linux/man-pages/man2/mmap.2.html
[5] MapViewOfFile documentation page, https://docs.microsoft.com/en-us/windows/desktop/api/memoryapi/nf-memoryapi-mapviewoffileex