My name is Yingjie (Jenny) Chen and I am a computer science student from the University of Waterloo. This blog post summarizes my internship experience in the release engineering team of Eclipse OpenJ9.
Modifications to Jenkins Pipeline scripts
I made several enhancements to the pipelines, and I will introduce four specific examples.
Reduce Intermittent Network Failures
The first issue is the network communication. There are several failure scenarios such as uploading and downloading artifacts, cloning Git repositories, and setting the status of pull requests on GitHub. Since restarting an entire build is time-consuming, we can wrap the failed sections with a retry block and wait for some time between each retry. However, the current retry clause in Jenkins pipelines retries the code without a delay (Please see the issue: https://issues.jenkins-ci.org/browse/JENKINS-54735). A general function named retry_and_delay was defined to provide an optional delay.
Improve Pull Request Trigger Mechanism
Another update is the way to trigger builds on GitHub pull request.
Previously, users could trigger a build using only a single line of comment as below.Jenkins compile plinux jdk11
Now the trigger phrase to launch a build can be embedded in multiple lines of a comment. Users can have additional comments before and after the build trigger. However, the trigger must be on a separate line. For instance, the following example is now valid.
Let's see if this can compile.
Jenkins compile plinux jdk11
If it works we can merge it.
This change makes build triggers more user-friendly and forgiving.
Archive JavaDoc
Afterward, there is an issue with JavaDoc. JavaDoc is generated by the build pipelines, but it was not available to download. Since fixes for JavaDoc for OpenJ9 releases were tested via the pipelines, it would be helpful to check the output of JavaDoc. Therefore, a boolean parameter named ARCHIVE_JAVADOC was added to the pipelines allowing users to choose whether to have the JavaDoc archived at the end of a build. Its default value is set to be false because most of the users do not need it. This change facilitates the publication of JavaDoc at Eclipse OpenJ9.
Archive Diagnostic Files for z/OS Platform
For the last example, I added the functionality of archiving diagnostic files such as Javacores and core dumps to OpenJ9’s build pipeline jobs for z/OS operating systems and AdoptOpenJDK’s TestKitGen for all platforms. This helps to debug and diagnose issues with the Java virtual machines under test.
For z/OS operating systems, the core dumps are generated in MVS file system, so they need to be moved to the Unix file system to be included in the archive. We use the following command to move and concatenate the core dumps.cat '//${JVM.coredump.001}' >> ${core.dmp}
Since a crash can generate multiple diagnostic artifacts, it is necessary to capture all of them in a single compressed archive. For the build pipeline jobs, we use a simple find and tar command to create the archive.
As there are two steps involved in test pipeline jobs, Java virtual machines have the potential to crash in the first step of Java source compilation or the second step of test execution. Previously, core dumps were not captured for all the platforms in the first step, nor were they included in the archive for z/OS operating system in the second step. To improve these cases, a script named moveDmp.pl was introduced to move core dumps to a designated directory.
Research on how to use the JCasC plugin
I researched how to use the JCasC plugin to configure Jenkins servers. Since some configurations of machines, pipeline jobs and plugins are used commonly across several Jenkins servers, it is repetitive to configure each server manually. Automating configuration reduces the time to set up a Jenkins server and prevents human error.
According to Jenkins (n.d.), “Jenkins Configuration as Code provides the ability to define this whole configuration as a simple, human-friendly, plain text YAML syntax”. This means users can set up Jenkins and its plugins using a simple text file that can be version controlled in a Git repository. For example, many of the common plugins are supported such as GitHub OAuth, Artifactory, Slack, and Docker and there is documentation describing how to add support for more plugins. This YAML configuration file can be stored in a repository on GitHub and a link can be provided to the JCasC plugin. Having it in GitHub allows users to take advantage of revision history, comments, issues and pull requests.
I also investigated using a job to use the JCasC plugin safely. A job was configured to apply the configuration file loaded by the JCasC plugin. When an error occurs, the file will be removed to prevent the Jenkins server from crashing.
Conclusion
During this internship, I gained lots of professional skills and knowledge. I am very lucky and pleased to be part of the OpenJ9 team. I appreciate the help I received from my mentor Adam Brousseau and all my colleagues.
Reference
Jenkins (n.d.). Jenkins Configuration as Code. Retrieved December 18, 2019 from https://jenkins.io/projects/jcasc/