crush depth

Edit: This is wrong.

Wasted a day getting aggravated at the options for producing per-platform distributions for Java applications.

I have lots of applications. None of them are platform-specific in any way; the code is very much "write once, run anywhere". Additionally, they are fully modularized (and, indeed, probably only work correctly if everything is on the module path). They do, however, have some dependencies that are only automatic modules (but that nevertheless work correctly when placed on the module path).

Historically, Java applications were distributed either as a single executable jar file containing the entire bytecode of the application, or as a set of jar files in a directory such that the program is executed by running something equivalent to:

$ java -cp lib/* com.io7m.example.main

This has an obvious issue: You don't know which version of Java you're going to get when you run java. If you're not running the above on a command-line, but via some kind of frontend script (or double-clicking a desktop shortcut), and the Java version you have isn't compatible, you're likely just going to silently fail and do nothing. Users will burn you at the stake. Aside from this glaring problem, things are otherwise perfect:

• Your build system, theoretically, is completely platform-independent. If you've been careful about reproducible builds, your build will produce the exact same binary artifacts on any system no matter where or when it is run.
• The artifacts produced by your build are completely platform-independent. You can distribute a single archive file that works on all platforms.

Unfortunately, since the death of runtime delivery platforms such as Java Web Start, the only remaining way to deal with the "we don't know what runtime we might have" problem is to make your application distributions platform-specific and distribute a Java runtime along with your application.

Thankfully, there are APIs such as Foojay that allow for automatically downloading Java runtimes for platforms. These APIs can be used with, for example, the JDKs Maven Plugin to efficiently fetch Java runtimes and package them up with your application.

You can, therefore, have a Linux-specific distribution that contains your application's jar files in a lib subdirectory, and some kind of shell script that runs your included Java runtime with all of the jars in lib placed on the module or class path as necessary. You can obviously have a similar Windows-specific distribution that has the same arrangement but with a .bat file that serves the same purpose.

This maintains the advantage of the "historical" distribution method in that your build system remains completely platform independent. It does, however, gain the disadvantage that your build system no longer produces platform-independent artifacts. Despite Java being a very consciously platform-independent language, we're back to having to have platform-specific application distributions. I've decided I can live with this.

Worse, though, people want things like .exe files that can be double clicked on Windows. Ideally, they want those .exe files to have nice icons and that show meaningful values when looking at the properties:

A .bat file won't give you that on Windows. Additionally, Windows has things like the Process Explorer that will show all of your applications as being java.exe with a generic Java icon. Great.

On Linux, the whole issue is somewhat less of a problem because at least one of the following will probably be true:

• Your Linux desktop doesn't have icons. Noone cares.
• Someone is going to repackage your application in a distro-specific manner and users are going to get the correct Java runtime anyway, so they probably won't have any use for your platform-specific application distribution.
• You can use something like flatpak and set icons and metadata out-of-band.

So let's assume that I'm willing to do the following:

• My main application project will continue to produce a completely platform-independent distribution; I'll put the jar files that make up the application into a zip file. There is a hard requirement on not using any platform-specific tools, for the build to produce byte-for-byte identical outputs on any platform, and the build must be possible to complete on a single machine. The code can be executed on multiple different platforms during the build for automated testing purposes, but one-and-exactly-one machine is responsible for producing build artifacts that will be deployed to a repository somewhere. Users can, if they want, use this distribution directly on their own systems by running it with their installed java commands. It's their responsibility to use the right version, and deal with the consequences of getting it wrong.

• I'll maintain separate projects that take those platform-independent artifacts and repackage them as necessary using platform-specific tools. It must be possible for the platform-specific build for platform P to conclude in a single build, on a single machine running platform P. These platform-specific distributions will treat users as being functionally illiterate and must mindlessly work correctly via a single double-clickable entry point of some kind.

Why do I insist on having each build run to completion and produce something useful on a single machine? Because coordinating distributed systems is hard, and trying to guarantee any kind of atomicity with regards to releasing and deploying code over them is a fool's errand. At least if the platform-specific builds happen in single shots on independent systems, we can make the independent releases and deployment of code on individual platforms somewhat atomic. I may not release Linux versions on the same day that I release Windows versions. Fine.

Additionally, I want the platform-specific distributions to feel like they actually belong on the platform they're running on. I want my application to look like MyApplication.exe in the Windows Process Explorer; I don't want to see java.exe and a Duke icon.

So what are the options?

Well, the OpenJDK people have been going on about jlink for ages now. Unfortunately, jlink falls over at the very first hurdle: It can't work with automatic modules. This means that all of my dependencies have to be modularized, and I know that at least some of them never will be. There are tools like moditect that claim to be able to somewhat automatically modularize, but the issue is that this takes the resulting bytecode artifacts further and further from the code that was actually tested during the platform-independent build; any rewriting of code is a potential for bugs that occur where the test suite won't have the opportunity to find them.

This is unacceptable. Ultimately, using jlink means that either your application runs in class path mode (which mine will not), or your application and all of its transitive dependencies have to be fully modularized and all of the modules have to be statically compiled into the resulting runtime image as jmod files. I've tried, but this isn't workable.

Moving on, the OpenJDK project now has the jpackage tool that claims to be capable of producing platform-specific application packages.

Unfortunately, it fails in a number of ways. It suffers from the exact same failure modes as jlink with regards to requiring complete modularization. Additionally, on Windows, it requires the installation of elderly versions of wix that are awkward to get working correctly in CI systems due to requiring PATH mangling and other magic. These issues ultimately made jpackage a dead end. Annoyingly, it looked like jpackage would have gotten me there, as it does produce executables that have the correct names in Process Explorer, and does apparently allow for setting icon resources and the like.

Other systems exist, such as winrun4j and launch4j. Unfortunately, these are more or less unmaintained. Additionally, they don't know anything about Java modules as they pre-date the JPMS by many years. They ultimately demand your application run in class path mode. So, those are out too.

I toyed around with creating a kind of launcher.exe that simply executed a non-jlinked Java runtime included with the platform distribution with the right command line arguments to put everything onto the module path. This became a comedy of errors. I first attempted to write the launcher program in Java, and AOT compile it with GraalVM. This required the installation of Visual Studio, and after being unable to get it to actually find the Visual C++ compiler, I gave up. It became clear, anyway, that this wasn't going to be any use as the resulting executables don't allow for custom icons without using hacky closed-source third-part executable editor tools. There's a ticket about this that was closed without comment. Nice.

I then decided to try writing a launcher in C++ by simply downloading a portable development kit during the build and compiling a C++ program (with the correct resource script to include a nice icon and executable metadata). This didn't exactly fail, but ultimately didn't achieve what I wanted: The launcher can execute a Java runtime, but then we're back to the original problem of the program appearing as java.exe with a generic icon in process listings (because that's exactly what it is; it's just the java executable from the included runtime).

Ultimately, I gave up.

What can be done about this?

I'd really like some options for jpackage that work like this:

$ jpackage \
  --type app-image \
  --name MyApplication \
  --icon icon.png \
  --use-jdk-directly some-platform-specific-jdk \
  --use-jars-on-module-path lib

This would:

• Copy the Java modules given in some-platform-specific-jdk without trying to minimize the modules included by using jlink to try to work out which are needed. Just give me all of them, I don't care.
• Create an executable file called MyApplication with a nice icon given in icon.png
• Take all of the jar files in lib, and include them in a directory in the resulting app image directly without touching them. The executable should be configured internally to give the same results as if I had run java with -p lib.

I feel like the documentation almost suggests that this is already possible, but I just couldn't get it to work. The tool always tried to analyze the modules of my application and then loudly complain about automatic modules and fail, rather than just shutting up and producing an executable that placed the jar files on an included module path instead.

This would, presumably, work exactly as well on Windows as on Linux. I don't care about Mac support. It would also be great if it didn't use obsolete tools to produce executables.