crush depth

Lurking Between Releases

A while back, I talked about moving to the bndtools suite of Maven plugins in preference to the Felix bundle plugin. The reason for this is better cross-IDE compatibility: Eclipse only really works with bndtools, whilst Intellij IDEA only really works with the Felix maven-bundle-plugin. This means that the plugin you choose to use in your Maven project ends up dictating which IDE developers must use in order to work with it. That's not great! However, I discovered that Intellij IDEA could be configured to use the OSGi manifest file that the bndtools Maven plugin creates, and therefore that means that developers can use either IDE as long as I use the bndtools plugins in my own projects.

Unfortunately, upon switching to the bndtools plugin in the primogenitor, I soon ran into MNG-6059 and MNG-5951. Briefly, Maven has some baked-in behaviour that is intended to produce metadata that follows the conventions people used when everyone used SVN to manage source code. However, those conventions result in Maven producing metadata that's essentially incorrect with regards to modern conventions. Specifically, Maven will automatically append module names to URIs specified in your project, because it assumes that you've laid out files in a particular way on remote servers. This automatic name appending could not previously be turned off. However, Maven 3.6.0 provided support for some attributes that you could place into your pom.xml files that would disable this automatic appending. Additionally, 3.6.0 comes with numerous fixes for working with Java 11 bytecode. As I recently decided that Java 11 was the new minimum version that I was going to support, 3.6.0 is really a requirement moving forward.

Unfortunately, this has now lead to further problems! One problem is that it's not possible to push a pom.xml file containing the new attributes to Maven Central because the version of Nexus running on Central can't parse the new POM files yet. The second problem is that the attributes are not inherited. I currently have over 850 pom.xml files spread over close to a hundred projects. There's just no way I'm going to edit each and every pom.xml file and add those attributes by hand. I want to specify them once in the primogenitor and have them apply everywhere. I've raised both points on the Maven mailing list, and my cries have been heard. All of this will hopefully be fixed in 3.6.1.

After all is said and done, this means that I can't start moving projects to Java 11 just yet or even do a full 3.0.0 release of the primogenitor because I need Maven 3.6.1 to do so. I wasn't available to test 3.6.0 much before it was released, but I can do plenty of testing for 3.6.1. It looks like I'll temporarily be in limbo for a couple of months!

Choosing Coffee

Now that jlink exists, it's become fashionable to bundle a JVM along with your Java bytecode. In applications that use the JPMS, the jlink tool can calculate the transitive closure of required modules and produce a tiny JVM that contains your application modules and only those platform modules required to run the application.

Because I'm using OSGi, I don't care about this particular aspect of jlink. What I am interested in is that, because of jlink, it has become a lot easier to bundle a JVM with an application. Projects such as AdoptOpenJDK produce production-ready builds of JVMs for a large range of platforms and architectures, including Hotspot and OpenJ9 VMs.

Now there is a downside to this: Java applications should, by their very nature, be completely platform-independent. If I give someone a jar file of Java 11 bytecode, it should run correctly on any JVM on any platform supporting Java 11+. If I instead start producing one distribution per platform and architecture including a bundled JVM, then by definition my application distributions are no longer platform-independent. Worse, even though jlink still bundles your application's bytecode along with the produced JVM, that bytecode is (by default) stored in an undocumented binary format along with the platform modules as one indivisible lump. I don't believe it's possible for the user to say "well, I don't want to use the bundled JVM, I'll just point my system JVM at the classes and use that".

What I would much rather do is still produce one distribution that contains a bundled JVM for each of the common platforms and architectures (let's say Linux x86_64, Windows x86_64, and MacOS x86_64) but also provide a basic shell script that can be used to start the application from a system JVM if the user so wishes (with the support caveat that if it doesn't work, they're on their own). The bytecode should still be stored exactly as it was in the original jar files (and in my case, would actually have to be stored like this because the jar files are OSGi bundles).

The ideal situation would be if I could have a Maven plugin that, given a range of platforms and architectures, automatically fetched one runtime suitable for each platform/architecture and then packaged up the whole application including all of the runtimes into one platform-independent distribution archive.

So, I've decided to write one!

The CoffeePick project provides an API and an interactive shell used to search for and download Java runtimes. The API works via a pluggable service provider model where new repositories of runtimes can be added at build-time or, if running on OSGi, at run-time.

Currently, arguably the two most popular places to get JVMs are:

The CoffeePick project currently contains providers for both of those repositories, and the AdoptOpenJDK module automatically finds new builds as the AdoptOpenJDK project publishes them.

There now follows small demo of the interactive shell.

First, we start up the shell:

$ java -jar com.io7m.coffeepick.shell/target/com.io7m.coffeepick.shell-0.0.1-main.jar
User directory: /home/rm/local/coffeepick
Loading repositories…
INFO com.io7m.coffeepick.client.vanilla.CoffeePickCatalog: setting up repository from provider net.java.jdk (urn:net.java.jdk)
INFO com.io7m.coffeepick.client.vanilla.CoffeePickCatalog: setting up repository from provider net.adoptopenjdk.raw (urn:net.adoptopenjdk.raw)

The shell uses jline3 internally and so has a full command-line history and limited tab-completion for command names.

Let's say we want to grab a Java 11 compatible runtime that will execute on Linux on the x86_64 architecture. The CoffeePick API refers to the current set of runtimes that we can download as the catalog. The catalog command, without any arguments, will list every available runtime:

[coffeepick]$ catalog
ID                                                                     | Arch     | Platform | Version  | VM       | Conf | Size       | Repository                   | Tags
8eee0aede947b804f9a5f49c8a38b52aace8a30a9ebd9383b7d06042fb5a237c       | aarch64  | linux    | 8        | hotspot  | jdk  | 75.38   MB | urn:net.adoptopenjdk.raw     | production
b0b3923a48243fa98b71c7fd8ff96def453e07b33156e90cb6851b7b7cacd4b1       | aarch64  | linux    | 8        | hotspot  | jre  | 39.53   MB | urn:net.adoptopenjdk.raw     | production
4f245d8bb2e6b84876edcc3573d9f8609e7596cb99102419b2174ed146aca642       | ppc64    | aix      | 8        | openj9   | jdk  | 87.50   MB | urn:net.adoptopenjdk.raw     | production
af2532a428fbeb6e6896279a874a3d4b574b74a6ebeae16e4771d7ad517f653a       | ppc64le  | linux    | 8        | openj9   | jdk  | 89.53   MB | urn:net.adoptopenjdk.raw     | production
e7fd779a6448259641565ca31f258d11052c9997e762bc63bbaa2b5776de98f1       | ppc64le  | linux    | 8        | openj9   | jre  | 48.62   MB | urn:net.adoptopenjdk.raw     | production
b092f4e6553a7c50872b0963f4de2b3a2bce78ce11d1bf1905a2ad5a6576924b       | s390x    | linux    | 8        | openj9   | jdk  | 90.37   MB | urn:net.adoptopenjdk.raw     | production
f0bc82fa49a2a02e881d403170cd8b273fc3d1585a4e65ff639666f056ce7362       | x64      | linux    | 8        | openj9   | jdk  | 90.12   MB | urn:net.adoptopenjdk.raw     | large-heap production
fd7035c60ad3060fccf7ab67c15b722302a1863ebddd3141dc3dd98bd0b25a68       | x64      | linux    | 8        | openj9   | jdk  | 90.24   MB | urn:net.adoptopenjdk.raw     | production
d1ee522053bb4135536b1d3173be8224bf54563d166e882b7ee2ccf609c2517f       | aarch64  | linux    | 9        | hotspot  | jdk  | 190.65  MB | urn:net.adoptopenjdk.raw     | production
36e4955c0fec6ff6494c5274e6950fdd45e73b11ebe97efdcb9768e07981f626       | aarch64  | linux    | 9        | hotspot  | jre  | 37.12   MB | urn:net.adoptopenjdk.raw     | production
d304cbfc9674ff1e1b0caf916b59c32ee9c477e5be6cfa071131b866cd54ec4c       | ppc64    | aix      | 9        | hotspot  | jdk  | 198.43  MB | urn:net.adoptopenjdk.raw     | production
359c65179f839c5ba0959710803c556c3ce2046dcb0762f2165595706b9b8272       | ppc64le  | linux    | 9        | hotspot  | jre  | 36.84   MB | urn:net.adoptopenjdk.raw     | production
bd4736feea702c859102bb78145fe3c7dbf6774ce1d66d00a9fc582d6bbeb564       | ppc64le  | linux    | 9        | hotspot  | jdk  | 190.36  MB | urn:net.adoptopenjdk.raw     | production
6382fe043c0ef1cf5b38cf3f70cc300bbdd66f16fa6e9b1ffbfe6a9cdda9011d       | ppc64le  | linux    | 9        | hotspot  | jdk  | 190.49  MB | urn:net.adoptopenjdk.raw     | production
84bbfca72cbabf04a2fd0e35d0faba03f91977573600edc23714b03aa1ec6fd9       | ppc64le  | linux    | 9        | hotspot  | jre  | 36.86   MB | urn:net.adoptopenjdk.raw     | production
0510af0db7531eb45bb8723810a10e146a80223cf5b72da077bc41e0e66724e3       | s390x    | linux    | 9        | hotspot  | jre  | 38.52   MB | urn:net.adoptopenjdk.raw     | production
7bfdc389c25820e08a85eb3a32c4193189f13945847619b14966f7fd45a5c345       | s390x    | linux    | 9        | hotspot  | jdk  | 193.60  MB | urn:net.adoptopenjdk.raw     | production
aa4fc8bda11741facaf3b8537258a4497c6e0046b9f4931e31f713d183b951f1       | x64      | linux    | 9        | hotspot  | jdk  | 207.46  MB | urn:net.adoptopenjdk.raw     | production
b58848cb7229a1dc2c135dbb85cce56cee33ab29849aa432e2ba26bf8f517dca       | x64      | linux    | 9        | hotspot  | jre  | 41.55   MB | urn:net.adoptopenjdk.raw     | production
b6a14f8dda60528bbb890877b62b7cf6e81bbbe19256bfd38a3ba4390c4eca5e       | x64      | linux    | 9        | hotspot  | jre  | 41.53   MB | urn:net.adoptopenjdk.raw     | production
ebe1eaad73584f70ab4e587ab9367fb15af820745979e68350fe80fadf615df8       | x64      | linux    | 9        | hotspot  | jdk  | 207.33  MB | urn:net.adoptopenjdk.raw     | production
bbe348baab3cca097b77f7fff6c8465858e642ad42c5851bde33a1553ca12653       | x64      | macos    | 9        | hotspot  | jdk  | 189.34  MB | urn:net.adoptopenjdk.raw     | production
f92931d258486d77dd986fa0fd2574075569e2f76624822d50bfae01ed72e7e7       | x64      | macos    | 9        | hotspot  | jre  | 35.70   MB | urn:net.adoptopenjdk.raw     | production
0373b5be04f98c0077c5b87c2020ecac2fedd36f68e75bbae38d433753506323       | x64      | windows  | 9        | hotspot  | jre  | 36.21   MB | urn:net.adoptopenjdk.raw     | production
b2de88155b2eaad123c2615f7eb3c0442d77545046bf85cfbcfe44fc97931dbd       | x64      | windows  | 9        | hotspot  | jdk  | 192.42  MB | urn:net.adoptopenjdk.raw     | production
3bb2faaba9f1a4f6e212ff34c5f2cfb4d5f083c1623cd2ad24108e0bb542e394       | x64      | windows  | 9        | hotspot  | jre  | 36.25   MB | urn:net.adoptopenjdk.raw     | production
709baaf217d24503dff3194fd66aa25ae5976b3f52ed3a564cdd270b6f5eef40       | x64      | windows  | 9        | hotspot  | jdk  | 192.27  MB | urn:net.adoptopenjdk.raw     | production
b66121b9a0c2e7176373e670a499b9d55344bcb326f67140ad6d0dc24d13d3e2       | aarch64  | linux    | 11       | hotspot  | jdk  | 193.39  MB | urn:net.adoptopenjdk.raw     | production
0f4edea90e57765562740721d052946408273e9744093266f65c02a7fb26800c       | ppc64    | aix      | 11       | openj9   | jre  | 46.93   MB | urn:net.adoptopenjdk.raw     | production
eaae79f3a89a6d71f34e9b41cbdfdb14f70988284d9e76427ed705514c614cc3       | ppc64    | aix      | 11       | openj9   | jdk  | 196.64  MB | urn:net.adoptopenjdk.raw     | production
39f97d76a0121f9a2b6e26bc201194921797fcc6c2181871086f09f8bce5862f       | ppc64    | aix      | 11       | openj9   | jdk  | 196.64  MB | urn:net.adoptopenjdk.raw     | production
001fd98ca5b5215eac616ed26341052ac1b426d7a55bbca7d28d48137fa9f295       | ppc64    | aix      | 11       | openj9   | jdk  | 196.65  MB | urn:net.adoptopenjdk.raw     | production
43f727ce6bca1ed826181f1a924c9f64a43728e3a222284fd2fbfbe9e4ef199c       | ppc64    | aix      | 11       | openj9   | jdk  | 196.65  MB | urn:net.adoptopenjdk.raw     | production
35e36d7a20a773e291b728de5d4977485a0d44a54f1433cdf6e6288c331557ea       | ppc64le  | linux    | 11       | openj9   | jdk  | 196.51  MB | urn:net.adoptopenjdk.raw     | production
4577350c35f4193430b3ae0094e54869b54aadbc76a50c1518069937a63d54d8       | ppc64le  | linux    | 11       | openj9   | jdk  | 196.52  MB | urn:net.adoptopenjdk.raw     | production
747c4613b36f6e86ee4a10848b1a7f009d1bde729acf31bb6350c3cc08a7475d       | ppc64le  | linux    | 11       | openj9   | jre  | 41.38   MB | urn:net.adoptopenjdk.raw     | production
fcc9a2b183a9238e1927d6088d7c99031f517d62026e8b33d02dfd883b1ca95e       | ppc64le  | linux    | 11       | openj9   | jdk  | 196.55  MB | urn:net.adoptopenjdk.raw     | production
c18364a778b1b990e8e62d094377af48b000f9f6a64ec21baff6a032af06386d       | ppc64le  | linux    | 11       | hotspot  | jdk  | 179.52  MB | urn:net.adoptopenjdk.raw     | production
fb3f4ddaf3e2a9490e34b385e00d76cf6142806d47bd619ef7c193c12acfbb01       | s390x    | linux    | 11       | openj9   | jdk  | 197.27  MB | urn:net.adoptopenjdk.raw     | production
249ad469735c42fa80fcf4123bc76f7e398ff3349ce6fb4bfa65a850fb70f6f8       | s390x    | linux    | 11       | openj9   | jdk  | 197.27  MB | urn:net.adoptopenjdk.raw     | production
119bd9d9faf1bb596b9ca74740f12cc23c20fe9331d312f1c2b59eb15a76625f       | x32      | windows  | 11       | hotspot  | jdk  | 174.24  MB | urn:net.adoptopenjdk.raw     | production
0b6050cc670eefd9465370ab19ae70401476430fca329e65f0dd636ca9cce9bd       | x64      | linux    | 11       | openj9   | jdk  | 195.41  MB | urn:net.adoptopenjdk.raw     | large-heap production
ef9bf07cba79082285a9d426ea4eb3e8df57561ce2afe07cc5f299a8fa203279       | x64      | linux    | 11       | openj9   | jdk  | 195.55  MB | urn:net.adoptopenjdk.raw     | production
10b2400add0070b64db91026f12b798708e59772a81492fd2b638638f71ed14f       | x64      | linux    | 11       | hotspot  | jdk  | 186.61  MB | urn:net.java.jdk             | early-access valhalla
cac1fe096ea161613d5afe8a04ce3294dc1b535f5f4ea1a4235ce425b2610678       | x64      | linux    | 11       | openj9   | jdk  | 195.36  MB | urn:net.adoptopenjdk.raw     | large-heap production
765947ab9457a29d2aa9d11460a4849611343c1e0ea3b33b9c08409cd4672251       | x64      | linux    | 11       | openj9   | jdk  | 195.50  MB | urn:net.adoptopenjdk.raw     | production
e0446e80eeb90ab97fd59f91ae094bdd658c354943d4e33e0ddf691714f9da4d       | x64      | linux    | 11       | openj9   | jdk  | 195.55  MB | urn:net.adoptopenjdk.raw     | production
518e9330f2aeb020bca31a07034a08c3a568823e703fc503e03206acdad9dae5       | x64      | linux    | 11       | openj9   | jre  | 40.91   MB | urn:net.adoptopenjdk.raw     | large-heap production
9306fa569d2d5abf982df0b4ffa5d30c652cf16139d7b2268db30d2886f3adbf       | x64      | linux    | 11       | openj9   | jdk  | 195.42  MB | urn:net.adoptopenjdk.raw     | large-heap production
7e9904e1bfa197e6e72570207cb2bd63dd7cc4bf5f97bc5454c1fc5a559a8bf1       | x64      | linux    | 11       | hotspot  | jdk  | 195.71  MB | urn:net.adoptopenjdk.raw     | production
a016413fd8415429b42e543fed7a1bee5010b1dbaf71d29a26e1c699f334c6ff       | x64      | linux    | 11       | openj9   | jre  | 40.96   MB | urn:net.adoptopenjdk.raw     | production
11cb48e3592f46f2c2dab095e3b9a4fec60b89199827f2b737a14b54dc5fd68d       | x64      | linux    | 11       | openj9   | jdk  | 195.36  MB | urn:net.adoptopenjdk.raw     | large-heap production
c5e9b588b4ac5b0bd5b4edd69d59265d1199bb98af7ca3270e119b264ffb6e3f       | x64      | macos    | 11       | openj9   | jdk  | 180.99  MB | urn:net.adoptopenjdk.raw     | production
0901dc5946fdf967f92f7b719ddfffdcdde5bd3fef86a83d7a3f2f39ddbef1f8       | x64      | macos    | 11       | openj9   | jre  | 38.63   MB | urn:net.adoptopenjdk.raw     | production
92494f79cc296625778a50d04816aabedca9cd9666b7bab8f1c3b96f06c82f97       | x64      | macos    | 11       | hotspot  | jdk  | 180.75  MB | urn:net.java.jdk             | early-access valhalla
c9a816d6a3f8aac9dc5b3b41c5a9e4e5460af433a06e003ae25d5a06dea8375f       | x64      | macos    | 11       | openj9   | jdk  | 180.95  MB | urn:net.adoptopenjdk.raw     | production
67785f46b264f7a91358cea41e4f7e0880685175a2a18f1f3890828a577fe067       | x64      | macos    | 11       | openj9   | jdk  | 180.95  MB | urn:net.adoptopenjdk.raw     | production
e219e7e2d586ed09ae65f4ec390fca5d5f0c37a61b47677648610194daf1aaa7       | x64      | macos    | 11       | hotspot  | jdk  | 189.91  MB | urn:net.adoptopenjdk.raw     | production
f34ced35547e53dd400c1ee99a63d8bf68e45370bc7b098e32f0b7107a60e00b       | x64      | macos    | 11       | openj9   | jdk  | 181.00  MB | urn:net.adoptopenjdk.raw     | production
b8960753a66190fa81982682357a2449b4183f3e23c20a5e3b4cf01e2989846b       | x64      | macos    | 11       | openj9   | jdk  | 181.00  MB | urn:net.adoptopenjdk.raw     | production
ef7c411c3b3e06f93874b15b598c36020b192a1a3096c5d9d5eae7799f8bcd49       | x64      | windows  | 11       | openj9   | jdk  | 196.96  MB | urn:net.adoptopenjdk.raw     | production
d104c7337ff3cd03fb37d048860192a3de5cbab4b158a681bc20cc521b6da954       | x64      | windows  | 11       | openj9   | jdk  | 196.90  MB | urn:net.adoptopenjdk.raw     | production
b1eb84f5bfce03058f266d7303ad02fcf4d6f2b05c989989ae7b0bd5a663a5a7       | x64      | windows  | 11       | hotspot  | jdk  | 192.97  MB | urn:net.adoptopenjdk.raw     | production
106c242e946a74ba71ff22b5a980c375b7cd13fd4550fb3698333e06a1f33243       | x64      | windows  | 11       | hotspot  | jdk  | 180.82  MB | urn:net.java.jdk             | early-access valhalla
d33551f15f8d4bc5f97471bca669872665e5db15cc8a5ac88f068e23f56bb129       | x64      | windows  | 11       | openj9   | jre  | 40.06   MB | urn:net.adoptopenjdk.raw     | production
7a6bb980b9c91c478421f865087ad2d69086a0583aeeb9e69204785e8e97dcfd       | x64      | linux    | 11.0.1   | hotspot  | jdk  | 187.60  MB | urn:net.java.jdk             | production
fa07eee08fa0f3de541ee1770de0cdca2ae3876f3bd78c329f27e85c287cd070       | x64      | macos    | 11.0.1   | hotspot  | jdk  | 182.09  MB | urn:net.java.jdk             | production
289dd06e06c2cbd5e191f2d227c9338e88b6963fd0c75bceb9be48f0394ede21       | x64      | windows  | 11.0.1   | hotspot  | jdk  | 187.42  MB | urn:net.java.jdk             | production
33a5f5b063dde3a66a98cf691bd00b21c012950ae1af253e19dcc68d74a75f65       | x64      | linux    | 12       | hotspot  | jdk  | 304.88  MB | urn:net.java.jdk             | early-access panama
95a033710ed71962c33b226afdbb222442bf0098ea3705448f0d14f0b3044bab       | x64      | macos    | 12       | hotspot  | jdk  | 233.21  MB | urn:net.java.jdk             | early-access panama

As can be seen, there are currently rather a lot of runtimes. We can start narrowing down the runtimes that we want by passing in search parameters. Searching is provided by the API directly and the API for this is believed to be stable, however the syntax for the shell is currently subject to change and may even be outdated by the time you read this. Let's limit ourselves to Linux runtimes:

[coffeepick]$ catalog platform:linux
ID                                                                     | Arch     | Platform | Version  | VM       | Conf | Size       | Repository                   | Tags
8eee0aede947b804f9a5f49c8a38b52aace8a30a9ebd9383b7d06042fb5a237c       | aarch64  | linux    | 8        | hotspot  | jdk  | 75.38   MB | urn:net.adoptopenjdk.raw     | production
b0b3923a48243fa98b71c7fd8ff96def453e07b33156e90cb6851b7b7cacd4b1       | aarch64  | linux    | 8        | hotspot  | jre  | 39.53   MB | urn:net.adoptopenjdk.raw     | production
af2532a428fbeb6e6896279a874a3d4b574b74a6ebeae16e4771d7ad517f653a       | ppc64le  | linux    | 8        | openj9   | jdk  | 89.53   MB | urn:net.adoptopenjdk.raw     | production
e7fd779a6448259641565ca31f258d11052c9997e762bc63bbaa2b5776de98f1       | ppc64le  | linux    | 8        | openj9   | jre  | 48.62   MB | urn:net.adoptopenjdk.raw     | production
b092f4e6553a7c50872b0963f4de2b3a2bce78ce11d1bf1905a2ad5a6576924b       | s390x    | linux    | 8        | openj9   | jdk  | 90.37   MB | urn:net.adoptopenjdk.raw     | production
fd7035c60ad3060fccf7ab67c15b722302a1863ebddd3141dc3dd98bd0b25a68       | x64      | linux    | 8        | openj9   | jdk  | 90.24   MB | urn:net.adoptopenjdk.raw     | production
f0bc82fa49a2a02e881d403170cd8b273fc3d1585a4e65ff639666f056ce7362       | x64      | linux    | 8        | openj9   | jdk  | 90.12   MB | urn:net.adoptopenjdk.raw     | large-heap production
d1ee522053bb4135536b1d3173be8224bf54563d166e882b7ee2ccf609c2517f       | aarch64  | linux    | 9        | hotspot  | jdk  | 190.65  MB | urn:net.adoptopenjdk.raw     | production
36e4955c0fec6ff6494c5274e6950fdd45e73b11ebe97efdcb9768e07981f626       | aarch64  | linux    | 9        | hotspot  | jre  | 37.12   MB | urn:net.adoptopenjdk.raw     | production
359c65179f839c5ba0959710803c556c3ce2046dcb0762f2165595706b9b8272       | ppc64le  | linux    | 9        | hotspot  | jre  | 36.84   MB | urn:net.adoptopenjdk.raw     | production
bd4736feea702c859102bb78145fe3c7dbf6774ce1d66d00a9fc582d6bbeb564       | ppc64le  | linux    | 9        | hotspot  | jdk  | 190.36  MB | urn:net.adoptopenjdk.raw     | production
6382fe043c0ef1cf5b38cf3f70cc300bbdd66f16fa6e9b1ffbfe6a9cdda9011d       | ppc64le  | linux    | 9        | hotspot  | jdk  | 190.49  MB | urn:net.adoptopenjdk.raw     | production
84bbfca72cbabf04a2fd0e35d0faba03f91977573600edc23714b03aa1ec6fd9       | ppc64le  | linux    | 9        | hotspot  | jre  | 36.86   MB | urn:net.adoptopenjdk.raw     | production
0510af0db7531eb45bb8723810a10e146a80223cf5b72da077bc41e0e66724e3       | s390x    | linux    | 9        | hotspot  | jre  | 38.52   MB | urn:net.adoptopenjdk.raw     | production
7bfdc389c25820e08a85eb3a32c4193189f13945847619b14966f7fd45a5c345       | s390x    | linux    | 9        | hotspot  | jdk  | 193.60  MB | urn:net.adoptopenjdk.raw     | production
aa4fc8bda11741facaf3b8537258a4497c6e0046b9f4931e31f713d183b951f1       | x64      | linux    | 9        | hotspot  | jdk  | 207.46  MB | urn:net.adoptopenjdk.raw     | production
ebe1eaad73584f70ab4e587ab9367fb15af820745979e68350fe80fadf615df8       | x64      | linux    | 9        | hotspot  | jdk  | 207.33  MB | urn:net.adoptopenjdk.raw     | production
b58848cb7229a1dc2c135dbb85cce56cee33ab29849aa432e2ba26bf8f517dca       | x64      | linux    | 9        | hotspot  | jre  | 41.55   MB | urn:net.adoptopenjdk.raw     | production
b6a14f8dda60528bbb890877b62b7cf6e81bbbe19256bfd38a3ba4390c4eca5e       | x64      | linux    | 9        | hotspot  | jre  | 41.53   MB | urn:net.adoptopenjdk.raw     | production
b66121b9a0c2e7176373e670a499b9d55344bcb326f67140ad6d0dc24d13d3e2       | aarch64  | linux    | 11       | hotspot  | jdk  | 193.39  MB | urn:net.adoptopenjdk.raw     | production
35e36d7a20a773e291b728de5d4977485a0d44a54f1433cdf6e6288c331557ea       | ppc64le  | linux    | 11       | openj9   | jdk  | 196.51  MB | urn:net.adoptopenjdk.raw     | production
4577350c35f4193430b3ae0094e54869b54aadbc76a50c1518069937a63d54d8       | ppc64le  | linux    | 11       | openj9   | jdk  | 196.52  MB | urn:net.adoptopenjdk.raw     | production
747c4613b36f6e86ee4a10848b1a7f009d1bde729acf31bb6350c3cc08a7475d       | ppc64le  | linux    | 11       | openj9   | jre  | 41.38   MB | urn:net.adoptopenjdk.raw     | production
fcc9a2b183a9238e1927d6088d7c99031f517d62026e8b33d02dfd883b1ca95e       | ppc64le  | linux    | 11       | openj9   | jdk  | 196.55  MB | urn:net.adoptopenjdk.raw     | production
c18364a778b1b990e8e62d094377af48b000f9f6a64ec21baff6a032af06386d       | ppc64le  | linux    | 11       | hotspot  | jdk  | 179.52  MB | urn:net.adoptopenjdk.raw     | production
fb3f4ddaf3e2a9490e34b385e00d76cf6142806d47bd619ef7c193c12acfbb01       | s390x    | linux    | 11       | openj9   | jdk  | 197.27  MB | urn:net.adoptopenjdk.raw     | production
249ad469735c42fa80fcf4123bc76f7e398ff3349ce6fb4bfa65a850fb70f6f8       | s390x    | linux    | 11       | openj9   | jdk  | 197.27  MB | urn:net.adoptopenjdk.raw     | production
e0446e80eeb90ab97fd59f91ae094bdd658c354943d4e33e0ddf691714f9da4d       | x64      | linux    | 11       | openj9   | jdk  | 195.55  MB | urn:net.adoptopenjdk.raw     | production
0b6050cc670eefd9465370ab19ae70401476430fca329e65f0dd636ca9cce9bd       | x64      | linux    | 11       | openj9   | jdk  | 195.41  MB | urn:net.adoptopenjdk.raw     | large-heap production
518e9330f2aeb020bca31a07034a08c3a568823e703fc503e03206acdad9dae5       | x64      | linux    | 11       | openj9   | jre  | 40.91   MB | urn:net.adoptopenjdk.raw     | large-heap production
9306fa569d2d5abf982df0b4ffa5d30c652cf16139d7b2268db30d2886f3adbf       | x64      | linux    | 11       | openj9   | jdk  | 195.42  MB | urn:net.adoptopenjdk.raw     | large-heap production
7e9904e1bfa197e6e72570207cb2bd63dd7cc4bf5f97bc5454c1fc5a559a8bf1       | x64      | linux    | 11       | hotspot  | jdk  | 195.71  MB | urn:net.adoptopenjdk.raw     | production
a016413fd8415429b42e543fed7a1bee5010b1dbaf71d29a26e1c699f334c6ff       | x64      | linux    | 11       | openj9   | jre  | 40.96   MB | urn:net.adoptopenjdk.raw     | production
ef9bf07cba79082285a9d426ea4eb3e8df57561ce2afe07cc5f299a8fa203279       | x64      | linux    | 11       | openj9   | jdk  | 195.55  MB | urn:net.adoptopenjdk.raw     | production
10b2400add0070b64db91026f12b798708e59772a81492fd2b638638f71ed14f       | x64      | linux    | 11       | hotspot  | jdk  | 186.61  MB | urn:net.java.jdk             | early-access valhalla
11cb48e3592f46f2c2dab095e3b9a4fec60b89199827f2b737a14b54dc5fd68d       | x64      | linux    | 11       | openj9   | jdk  | 195.36  MB | urn:net.adoptopenjdk.raw     | large-heap production
cac1fe096ea161613d5afe8a04ce3294dc1b535f5f4ea1a4235ce425b2610678       | x64      | linux    | 11       | openj9   | jdk  | 195.36  MB | urn:net.adoptopenjdk.raw     | large-heap production
765947ab9457a29d2aa9d11460a4849611343c1e0ea3b33b9c08409cd4672251       | x64      | linux    | 11       | openj9   | jdk  | 195.50  MB | urn:net.adoptopenjdk.raw     | production
7a6bb980b9c91c478421f865087ad2d69086a0583aeeb9e69204785e8e97dcfd       | x64      | linux    | 11.0.1   | hotspot  | jdk  | 187.60  MB | urn:net.java.jdk             | production
33a5f5b063dde3a66a98cf691bd00b21c012950ae1af253e19dcc68d74a75f65       | x64      | linux    | 12       | hotspot  | jdk  | 304.88  MB | urn:net.java.jdk             | early-access panama

That narrows things down a little, but there are still far too many matching runtimes. Let's specify that we want Linux, x86_64 (the CoffeePick API refers to 32-bit x86 as x32 and 64-bit x86 as x64 for uniformity), and let's specify a version range that says that we want any version of Java greater than or equal to 11 but less than 12:

[coffeepick]$ catalog platform:linux arch:x64 version:[11,12)
ID                                                                     | Arch     | Platform | Version  | VM       | Conf | Size       | Repository                   | Tags
e0446e80eeb90ab97fd59f91ae094bdd658c354943d4e33e0ddf691714f9da4d       | x64      | linux    | 11       | openj9   | jdk  | 195.55  MB | urn:net.adoptopenjdk.raw     | production
7e9904e1bfa197e6e72570207cb2bd63dd7cc4bf5f97bc5454c1fc5a559a8bf1       | x64      | linux    | 11       | hotspot  | jdk  | 195.71  MB | urn:net.adoptopenjdk.raw     | production
a016413fd8415429b42e543fed7a1bee5010b1dbaf71d29a26e1c699f334c6ff       | x64      | linux    | 11       | openj9   | jre  | 40.96   MB | urn:net.adoptopenjdk.raw     | production
ef9bf07cba79082285a9d426ea4eb3e8df57561ce2afe07cc5f299a8fa203279       | x64      | linux    | 11       | openj9   | jdk  | 195.55  MB | urn:net.adoptopenjdk.raw     | production
10b2400add0070b64db91026f12b798708e59772a81492fd2b638638f71ed14f       | x64      | linux    | 11       | hotspot  | jdk  | 186.61  MB | urn:net.java.jdk             | early-access valhalla
0b6050cc670eefd9465370ab19ae70401476430fca329e65f0dd636ca9cce9bd       | x64      | linux    | 11       | openj9   | jdk  | 195.41  MB | urn:net.adoptopenjdk.raw     | large-heap production
11cb48e3592f46f2c2dab095e3b9a4fec60b89199827f2b737a14b54dc5fd68d       | x64      | linux    | 11       | openj9   | jdk  | 195.36  MB | urn:net.adoptopenjdk.raw     | large-heap production
cac1fe096ea161613d5afe8a04ce3294dc1b535f5f4ea1a4235ce425b2610678       | x64      | linux    | 11       | openj9   | jdk  | 195.36  MB | urn:net.adoptopenjdk.raw     | large-heap production
518e9330f2aeb020bca31a07034a08c3a568823e703fc503e03206acdad9dae5       | x64      | linux    | 11       | openj9   | jre  | 40.91   MB | urn:net.adoptopenjdk.raw     | large-heap production
9306fa569d2d5abf982df0b4ffa5d30c652cf16139d7b2268db30d2886f3adbf       | x64      | linux    | 11       | openj9   | jdk  | 195.42  MB | urn:net.adoptopenjdk.raw     | large-heap production
765947ab9457a29d2aa9d11460a4849611343c1e0ea3b33b9c08409cd4672251       | x64      | linux    | 11       | openj9   | jdk  | 195.50  MB | urn:net.adoptopenjdk.raw     | production
7a6bb980b9c91c478421f865087ad2d69086a0583aeeb9e69204785e8e97dcfd       | x64      | linux    | 11.0.1   | hotspot  | jdk  | 187.60  MB | urn:net.java.jdk             | production

We are making progress. Let's assume, for the sake of this example, that we know our application isn't compatible with OpenJ9. We can require that the VM implementation be Hotspot:

[coffeepick]$ catalog platform:linux arch:x64 version:[11,12) vm:hotspot
ID                                                                     | Arch     | Platform | Version  | VM       | Conf | Size       | Repository                   | Tags
7e9904e1bfa197e6e72570207cb2bd63dd7cc4bf5f97bc5454c1fc5a559a8bf1       | x64      | linux    | 11       | hotspot  | jdk  | 195.71  MB | urn:net.adoptopenjdk.raw     | production
10b2400add0070b64db91026f12b798708e59772a81492fd2b638638f71ed14f       | x64      | linux    | 11       | hotspot  | jdk  | 186.61  MB | urn:net.java.jdk             | early-access valhalla
7a6bb980b9c91c478421f865087ad2d69086a0583aeeb9e69204785e8e97dcfd       | x64      | linux    | 11.0.1   | hotspot  | jdk  | 187.60  MB | urn:net.java.jdk             | production

Let's also assume that we're only interested in builds that are production-ready. The CoffeePick API allows for repositories to provide arbitrary tags to builds in order to further distinguish them beyond the standard metadata defined by the API. The convention is to tag production-ready builds with the production tag, so let's list builds that have that tag:

[coffeepick]$ catalog platform:linux arch:x64 version:[11,12) vm:hotspot require-tag:production
ID                                                                     | Arch     | Platform | Version  | VM       | Conf | Size       | Repository                   | Tags
7e9904e1bfa197e6e72570207cb2bd63dd7cc4bf5f97bc5454c1fc5a559a8bf1       | x64      | linux    | 11       | hotspot  | jdk  | 195.71  MB | urn:net.adoptopenjdk.raw     | production
7a6bb980b9c91c478421f865087ad2d69086a0583aeeb9e69204785e8e97dcfd       | x64      | linux    | 11.0.1   | hotspot  | jdk  | 187.60  MB | urn:net.java.jdk             | production

We've narrowed it down to two builds. We can see from the Repository column that one of those builds comes from the jdk.java.net repository and the other comes from AdoptOpenJDK. Let's arbitrarily decide that we want the one from jdk.java.net and download it:

[coffeepick]$ download 7a6bb980b9c91c478421f865087ad2d69086a0583aeeb9e69204785e8e97dcfd
[    2.10 MB /   187.60 MB] |+                                                           | 2.10MB/s
[    4.20 MB /   187.60 MB] |++                                                          | 2.10MB/s
[    7.72 MB /   187.60 MB] |+++                                                         | 3.52MB/s
[   10.49 MB /   187.60 MB] |++++                                                        | 2.77MB/s
[   12.60 MB /   187.60 MB] |+++++                                                       | 2.11MB/s

(Time passes...)

[  178.69 MB /   187.60 MB] |++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  | 2.35MB/s
[  180.67 MB /   187.60 MB] |++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  | 1.98MB/s
[  182.67 MB /   187.60 MB] |+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | 2.00MB/s
[  184.81 MB /   187.60 MB] |++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++| 2.14MB/s
[  187.15 MB /   187.60 MB] |++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++| 2.34MB/s
Download finished
Runtime archive: /home/rm/local/coffeepick/inventory/7a6bb980b9c91c478421f865087ad2d69086a0583aeeb9e69204785e8e97dcfd/archive

For safety reasons, the CoffeePick API automatically verifies that the downloaded archive matches the published checksum. This should provide a minimum level of confidence that you actually downloaded the right runtime, and that the runtime made it onto the disk without malicious or accidental modification. It's possible to manually verify an already-downloaded runtime at any time in the future with the verify command:

[coffeepick]$ verify 7a6bb980b9c91c478421f865087ad2d69086a0583aeeb9e69204785e8e97dcfd
Verified:
  Algorithm: SHA-256
  Received:  7a6bb980b9c91c478421f865087ad2d69086a0583aeeb9e69204785e8e97dcfd

Now that the runtime has been downloaded, it will appear in the inventory. In the CoffeePick API, the inventory is the set of runtimes that you currently have downloaded. The inventory command lists the downloaded runtimes and supports all of the same search parameters as the catalog command:

[coffeepick]$ inventory
ID                                                                     | Arch     | Platform | Version  | VM       | Conf | Size       | Repository                   | Tags
7e9904e1bfa197e6e72570207cb2bd63dd7cc4bf5f97bc5454c1fc5a559a8bf1       | x64      | linux    | 11       | hotspot  | jdk  | 195.71  MB | urn:net.adoptopenjdk.raw     | production
a016413fd8415429b42e543fed7a1bee5010b1dbaf71d29a26e1c699f334c6ff       | x64      | linux    | 11       | openj9   | jre  | 40.96   MB | urn:net.adoptopenjdk.raw     | production
d33551f15f8d4bc5f97471bca669872665e5db15cc8a5ac88f068e23f56bb129       | x64      | windows  | 11       | openj9   | jre  | 40.06   MB | urn:net.adoptopenjdk.raw     | production
7a6bb980b9c91c478421f865087ad2d69086a0583aeeb9e69204785e8e97dcfd       | x64      | linux    | 11.0.1   | hotspot  | jdk  | 187.60  MB | urn:net.java.jdk             | production

[coffeepick]$ inventory vm:openj9
ID                                                                     | Arch     | Platform | Version  | VM       | Conf | Size       | Repository                   | Tags
a016413fd8415429b42e543fed7a1bee5010b1dbaf71d29a26e1c699f334c6ff       | x64      | linux    | 11       | openj9   | jre  | 40.96   MB | urn:net.adoptopenjdk.raw     | production
d33551f15f8d4bc5f97471bca669872665e5db15cc8a5ac88f068e23f56bb129       | x64      | windows  | 11       | openj9   | jre  | 40.06   MB | urn:net.adoptopenjdk.raw     | production

Of course, a runtime isn't much use sitting there in the on-disk storage area. The unpack command unpacks a downloaded runtime into a user-specified directory:

[coffeepick]$ unpack --output /tmp/jdk11-linux-x64 --strip-leading-directory 7e9904e1bfa197e6e72570207cb2bd63dd7cc4bf5f97bc5454c1fc5a559a8bf1
[coffeepick]$

Now, in an ordinary shell:

$ ls -alF /tmp/jdk11-linux-x64/
total 4
drwxr-xr-x  10 rm   rm    220 2018-11-18 14:23 ./
drwxrwxrwt 198 root root 4420 2018-11-18 14:23 ../
drwxr-xr-x   2 rm   rm    680 2018-11-18 14:23 bin/
drwxr-xr-x   4 rm   rm    140 2018-11-18 14:23 conf/
drwxr-xr-x   4 rm   rm    100 2018-11-18 14:23 demo/
drwxr-xr-x   3 rm   rm    180 2018-11-18 14:23 include/
drwxr-xr-x   2 rm   rm   1440 2018-11-18 14:23 jmods/
drwxr-xr-x  72 rm   rm   1440 2018-11-18 14:23 legal/
drwxr-xr-x   6 rm   rm   1040 2018-11-18 14:23 lib/
drwxr-xr-x   4 rm   rm    100 2018-11-18 14:23 man/
-rw-r--r--   1 rm   rm   1202 2018-11-18 14:23 release

$ cat /tmp/jdk11-linux-x64/release 
IMPLEMENTOR="AdoptOpenJDK"
IMPLEMENTOR_VERSION="AdoptOpenJDK"
JAVA_VERSION="11.0.1"
JAVA_VERSION_DATE="2018-10-16"
MODULES="java.base java.compiler java.datatransfer java.xml java.prefs java.desktop java.instrument java.logging java.management java.security.sasl java.naming java.rmi java.management.rmi java.net.http java.scripting java.security.jgss java.transaction.xa java.sql java.sql.rowset java.xml.crypto java.se java.smartcardio jdk.accessibility jdk.internal.vm.ci jdk.management jdk.unsupported jdk.internal.vm.compiler jdk.aot jdk.internal.jvmstat jdk.attach jdk.charsets jdk.compiler jdk.crypto.ec jdk.crypto.cryptoki jdk.dynalink jdk.internal.ed jdk.editpad jdk.hotspot.agent jdk.httpserver jdk.internal.le jdk.internal.opt jdk.internal.vm.compiler.management jdk.jartool jdk.javadoc jdk.jcmd jdk.management.agent jdk.jconsole jdk.jdeps jdk.jdwp.agent jdk.jdi jdk.jfr jdk.jlink jdk.jshell jdk.jsobject jdk.jstatd jdk.localedata jdk.management.jfr jdk.naming.dns jdk.naming.rmi jdk.net jdk.pack jdk.rmic jdk.scripting.nashorn jdk.scripting.nashorn.shell jdk.sctp jdk.security.auth jdk.security.jgss jdk.unsupported.desktop jdk.xml.dom jdk.zipfs"
OS_ARCH="x86_64"
OS_NAME="Linux"
SOURCE=""

Finally, it's possible tell repository providers to self-update. This is useful in the case of a repository such as AdoptOpenJDK where new runtime builds are published frequently.

$ repository-update urn:net.adoptopenjdk.raw
Update of urn:net.adoptopenjdk.raw started…
[ 0.00   % ] |                                                            |
[ 1.79   % ] |++                                                          |
[ 3.57   % ] |+++                                                         |
[ 5.36   % ] |++++                                                        |
[ 7.14   % ] |+++++                                                       |

(Time passes...)

[ 91.07  % ] |+++++++++++++++++++++++++++++++++++++++++++++++++++++++     |
[ 92.86  % ] |++++++++++++++++++++++++++++++++++++++++++++++++++++++++    |
[ 94.64  % ] |+++++++++++++++++++++++++++++++++++++++++++++++++++++++++   |
[ 96.43  % ] |++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  |
[ 98.21  % ] |+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
Update of urn:net.adoptopenjdk.raw finished.

Subsequent executions of the catalog command will then show new releases (if any are available) in addition to those that were already available.

Future directions for the project:

  1. A Maven plugin

    I want to be able to do all of the above in an automated manner during builds. The API already allows for electively downloading or reusing cached runtimes, and runtimes can be specified directly by hash value for reproducibility in builds.

  2. Add shipilev.net builds

    Aleksey Shipilëv publishes various early-access builds so that people can try out new JDK features. I'd like to get his builds included into a repository provider (ideally with some way to find new builds without having to crawl over his entire site to discover them).

  3. A standard schema for repository metadata

    I already have a very strict XSD schema that describes a format for repository metadata. It would just be a matter of persuading others to use it, and allowing the API to consume XML metadata.

You Cannot Put That There

Maven 3.6.0 was released today. It adds the ability to add a child.inherit.append.path attribute to various places within a POM file to turn off Maven's automatic appending of child module names to various paths. This has been a long-standing complaint with Maven as it had essentially baked-in SVN path semantics to the model in a way that couldn't be turned off or worked around sanely.

Unfortunately, the Maven Central rules can't accept POM files that contain the new attributes:

[ERROR] Rule failure while trying to close staging repository with ID "comio7m-1402".
[ERROR] 
[ERROR] Nexus Staging Rules Failure Report
[ERROR] ==================================
[ERROR] 
[ERROR] Repository "comio7m-1402" failures
[ERROR]   Rule "sources-staging" failures
[ERROR]     * Illegal POM: /com/io7m/primogenitor/com.io7m.primogenitor/3.0.0-beta0018/com.io7m.primogenitor-3.0.0-beta0018.pom Unknown attribute 'child.inherit.append.path' for tag 'project' (position: START_TAG seen ...ache.org/xsd/maven-4.0.0.xsd\n  child.inherit.append.path=false>... @6:37) 
[ERROR]   Rule "javadoc-staging" failures
[ERROR]     * Illegal POM: /com/io7m/primogenitor/com.io7m.primogenitor/3.0.0-beta0018/com.io7m.primogenitor-3.0.0-beta0018.pom Unknown attribute 'child.inherit.append.path' for tag 'project' (position: START_TAG seen ...ache.org/xsd/maven-4.0.0.xsd\n  child.inherit.append.path=false>... @6:37) 
[ERROR]   Rule "pom-staging" failures
[ERROR]     * Invalid POM: /com/io7m/primogenitor/com.io7m.primogenitor/3.0.0-beta0018/com.io7m.primogenitor-3.0.0-beta0018.pom: Parsing Error: Unknown attribute 'child.inherit.append.path' for tag 'project' (position: START_TAG seen ...ache.org/xsd/maven-4.0.0.xsd\n  child.inherit.append.path=false>... @6:37) 
[ERROR] 
[ERROR] 
[ERROR] Cleaning up local stage directory after a Rule failure during close of staging repositories: [comio7m-1402]
[ERROR]  * Deleting context 2a74942ea74dc7.properties
[ERROR] Cleaning up remote stage repositories after a Rule failure during close of staging repositories: [comio7m-1402]
[ERROR]  * Dropping failed staging repository with ID "comio7m-1402" (Rule failure during close of staging repositories: [comio7m-1402]).

I've filed a bug.

Java 11 and Maven 3.6.0

Java 11 is out, and is the current LTS release (at least if you're using the Oracle JDK).

I'll be moving all of my code to start requiring JDK 11 at a minimum as of now.

Additionally, Maven 3.6.0 is currently in the process of being released, and it contains a fix for an issue that's important to me, so I suspect I'm also going to start requiring Maven 3.6.0 everywhere too.

No More OpenGL

I'm dropping OpenGL in favour of Vulkan.

No more broken drivers (probably). No more dealing with source code level management of shader code at runtime (no more sombrero). No more broken GLSL shader compilers, for that matter. No more weak typing (no more jcanephora). No more logarithmic depth buffer - Vulkan has reversed floating-point depth buffers accessible without extensions.

Planning to rewrite r2 using Vulkan, yielding r3. A few design issues that were imposed by the OpenGL 3.3 core requirement can be corrected.

Still need to give the Vulkan API a friendly face though, to make it feel like a Java API instead of a C++ API.