crush depth

In a recent post to the bndtools-users mailing list, I expressed a bit of frustration with my experience of trying out Bndtools for what I think must be about the fourth time. Jürgen Albert had mentioned on the osgi-dev list that he had a working environment where, every time he clicked "Save" in his IDE, bundles would automatically be built and deployed to a running OSGi container. This gave him effectively live updates to running code during development. I'd still not been able to achieve this in my Maven and IDEA environment, so I took another look at Bndtools to see if it could be done. Any OSGi user will be familiar with being able to deploy new parts of a running system without having to restart it, but actually being able to instantly deploy code every time you change as little as a single line of code is probably not something that every OSGi user has access to.

On the bndtools-users list, Raymond Auge mentioned to me that he'd recently added functionality to an as-yet-unreleased version of the bnd-run-maven-plugin that might be able to solve the problem. It worked beautifully! Using the following setup, it's possible to get live code updates without having to stray outside of a traditional Maven setup; your build gets to remain a conventional Maven build and, if you don't use the feature, you probably don't have to know the plugin exists at all.

The setup described here assumes a multi-module Maven project. I describe it in these terms because all of my projects are multi-module, and it should be trivial to infer, for anyone familiar with Maven, the simpler single-module setup given the more complex multi-module setup.

A complete, working example project is available on GitHub. The example project contains:

• com.io7m.bndtest.api: A simple API specification for hello world programs.
• com.io7m.bndtest.vanilla: An implementation of the com.io7m.bndtest.api spec.
• com.io7m.bndtest.main: A consumer of the com.io7m.bndtest.api spec; it looks up an implementation and uses it.
• com.io7m.bndtest.run: Administrative information for setting up an OSGi runtime.

At the time of writing, the version of the bnd-run-maven-plugin you need is currently only available as a development snapshot. Add the following pluginRepository to the root POM of your Maven project:

  <pluginRepositories>
    <pluginRepository>
      <id>bnd-snapshots</id>
      <url>https://bndtools.jfrog.io/bndtools/libs-snapshot/</url>
      <layout>default</layout>
      <releases>
        <enabled>false</enabled>
      </releases>
    </pluginRepository>
  </pluginRepositories>

This will no longer be necessary once version 4.3.0 is officially released.

Add an execution of this plugin in the root POM of your project:

  <build>
    <plugins>
       ...
      <plugin>
        <groupId>biz.aQute.bnd</groupId>
        <artifactId>bnd-run-maven-plugin</artifactId>
        <version>4.3.0-SNAPSHOT</version>
      </plugin>
    </plugins>
  </build>

Add a module to your project (in the example code, this is com.io7m.bndtest.run) that will be used to declare the information and dependencies required to start a basic OSGi runtime. In this case, I use Felix and install a minimal set of services such as a Gogo shell and Declarative Services. These aren't necessary for the plugin to function, but the example code on GitHub uses Declarative Services, and the Gogo shell makes it easier to poke around in the OSGi runtime to see what's going on.

The POM file for this module should contain dependencies for all of the bundles you wish to install into the OSGi runtime, and all the OSGi framework and services. In my case this looks like:

  <dependencies>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>com.io7m.bndtest.api</artifactId>
      <version>${project.version}</version>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>com.io7m.bndtest.vanilla</artifactId>
      <version>${project.version}</version>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>com.io7m.bndtest.main</artifactId>
      <version>${project.version}</version>
    </dependency>

    <dependency>
      <groupId>org.apache.felix</groupId>
      <artifactId>org.apache.felix.framework</artifactId>
    </dependency>
    <dependency>
      <groupId>org.apache.felix</groupId>
      <artifactId>org.apache.felix.gogo.shell</artifactId>
    </dependency>
    <dependency>
      <groupId>org.apache.felix</groupId>
      <artifactId>org.apache.felix.gogo.command</artifactId>
    </dependency>
    <dependency>
      <groupId>org.apache.felix</groupId>
      <artifactId>org.apache.felix.shell.remote</artifactId>
    </dependency>
    <dependency>
      <groupId>org.apache.felix</groupId>
      <artifactId>org.apache.felix.shell</artifactId>
    </dependency>
    <dependency>
      <groupId>org.apache.felix</groupId>
      <artifactId>org.apache.felix.scr</artifactId>
    </dependency>
    <dependency>
      <groupId>org.apache.felix</groupId>
      <artifactId>org.apache.felix.log</artifactId>
    </dependency>
    <dependency>
      <groupId>org.osgi</groupId>
      <artifactId>osgi.promise</artifactId>
    </dependency>
  </dependencies>

A single execution of the bnd-run-maven-plugin is needed in this module, along with a simple bndrun file. Note that the name of the plugin exection must be unique across the entire set of modules in the Maven reactor. You will be referring to this name later on the command line, so pick a sensible name (we use main here). Note that the name of the bndrun file can be completely arbitrary; the name of the file does not have to be in any way related to the name of the plugin execution.

  <build>
    <plugins>
      <plugin>
        <groupId>biz.aQute.bnd</groupId>
        <artifactId>bnd-run-maven-plugin</artifactId>
        <executions>
          <execution>
            <id>main</id>
            <configuration>
              <bndrun>main.bndrun</bndrun>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

The main.bndrun file is unsurprising to anyone with a passing familiarity with bnd. We specify that we want Felix to be the OSGi framework, and we list the set of bundles that should be installed and executed. We don't bother to specify version numbers because this information is already present in the Maven POM.

$ cat main.bndrun

-runfw: \
  org.apache.felix.framework

-runee: JavaSE-11

-runrequires: \
  osgi.identity;filter:='(osgi.identity=com.io7m.bndtest.main)'

-runbundles: \
  com.io7m.bndtest.api,           \
  com.io7m.bndtest.main,          \
  com.io7m.bndtest.vanilla,       \
  org.apache.felix.gogo.command,  \
  org.apache.felix.gogo.runtime,  \
  org.apache.felix.gogo.shell,    \
  org.apache.felix.log,           \
  org.apache.felix.scr,           \
  org.apache.felix.shell,         \
  org.apache.felix.shell.remote,  \
  osgi.promise,                   \

Now that all of this is configured, it's necessary to do a full Maven build and execute the plugin. Note that, due to a Maven limitation, it is necessary to call the bnd-run-maven-plugin in the same operation as the package lifecycle. In other words, execute this:

$ mvn package bnd-run:run@main

The @main suffix to the run command refers to the named main execution we declared earlier. The above command will run a Maven build and then start an OSGi instance and keep it in the foreground:

$ mvn package bnd-run:run@main
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] com.io7m.bndtest                                                   [pom]
[INFO] com.io7m.bndtest.api                                               [jar]
[INFO] com.io7m.bndtest.vanilla                                           [jar]
[INFO] com.io7m.bndtest.main                                              [jar]
[INFO] com.io7m.bndtest.run                                               [jar]
...

[INFO] ---------------< com.io7m.bndtest:com.io7m.bndtest.run >----------------
[INFO] Building com.io7m.bndtest.run 0.0.1-SNAPSHOT                       [5/5]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
...
[INFO] --- bnd-run-maven-plugin:4.3.0-SNAPSHOT:run (main) @ com.io7m.bndtest.run ---
____________________________
Welcome to Apache Felix Gogo

g! lb
START LEVEL 1
   ID|State      |Level|Name
    0|Active     |    0|System Bundle (6.0.3)|6.0.3
    1|Active     |    1|com.io7m.bndtest.api 0.0.1-SNAPSHOT - Bnd experiment (API) (0.0.1.201905160844)|0.0.1.201905160844
    2|Active     |    1|com.io7m.bndtest.main 0.0.1-SNAPSHOT - Bnd experiment (Main consumer) (0.0.1.201905160845)|0.0.1.201905160845
    3|Active     |    1|com.io7m.bndtest.vanilla 0.0.1-SNAPSHOT - Bnd experiment (Vanilla implementation) (0.0.1.201905160845)|0.0.1.201905160845
    4|Active     |    1|Apache Felix Gogo Command (1.1.0)|1.1.0
    5|Active     |    1|Apache Felix Gogo Runtime (1.1.2)|1.1.2
    6|Active     |    1|Apache Felix Gogo Shell (1.1.2)|1.1.2
    7|Active     |    1|Apache Felix Log Service (1.2.0)|1.2.0
    8|Active     |    1|Apache Felix Declarative Services (2.1.16)|2.1.16
    9|Active     |    1|Apache Felix Shell Service (1.4.3)|1.4.3
   10|Active     |    1|Apache Felix Remote Shell (1.2.0)|1.2.0
   11|Active     |    1|org.osgi:osgi.promise (7.0.1.201810101357)|7.0.1.201810101357

At this point, there is a running OSGi container in the foreground. The bnd-run-maven-plugin is monitoring the jar files that were created as part of the Maven build and, if those jar files are changed, the plugin will notify the framework that it's necessary to reload the code.

Now, given that we want to make rebuilding and updating running code as quick as possible, we almost certainly don't want to have to do a full Maven build every time we change a line of code. Luckily, we can configure IDEA to do the absolute minimum amount of work necessary to achieve this.

First, open the Run Configurations window:

Add a Maven build configuration:

The precise Maven command-line used will depend on your project setup. In the example code, I have an execution of the bnd-maven-plugin named generate-osgi-manifest that does the work necessary to create an OSGi manifest. I also have an execution of the maven-jar-plugin named default-jar that adds the generated manifest to the created jar file. I also declare a property io7m.quickBuild which turns off any plugin executions that aren't strictly necessary during development. All of these things are configurations declared in the primogenitor parent POM that I use for all projects, but none of them should be in any way unfamiliar to anyone that has put together a basic bnd-maven-plugin-based project. The gist is: Tell Maven to run the bnd-maven-plugin and generate an OSGi manifest for each module, and then tell it to create a jar for each module.

The other important item is to add a Before Launch option (at the bottom of the window) that runs a normal IDE build before Maven is executed. The reason for this is speed: We want the IDE to very quickly rebuild class files (this is essentially instant on my system), and then we want to execute the absolute bare minimum in terms of Maven goals after the IDE build.

One small nit here: The root module of your project will almost certainly have a packaging of type pom instead of type jar. The bnd-maven-plugin will (correctly) skip the creation of a manifest file for this module, and this will then typically cause the maven-jar-plugin to complain that there is no manifest file to be inserted into the jar that it's creating. A quick and dirty workaround is to do this in the root of the project:

mkdir -p target/classes/META-INF
touch target/classes/META-INF/MANIFEST.MF

In the example project, I include a small run.sh script that actually does this as part of starting up the OSGi container:

#!/bin/sh

mkdir -p target/classes/META-INF
touch target/classes/META-INF/MANIFEST.MF
exec mvn package bnd-run:run@main

I strongly recommend not doing this as part of the actual Maven build itself; this is something that's only needed by developers who are making use of this live code updating setup during development, and isn't something that the Maven build needs to know about. By executing specific plugins, we're not calling Maven in the way that it would normally be called and therefore it's up to us to ensure that things are set up correctly so that the plugins can work.

With both of these things configured, we should now be able to click Run in IDEA, and the code will be built in a matter of seconds:

Assuming that our OSGi container is still running somewhere, the rebuilt jar files will be detected and the code will be reloaded automatically:

[INFO] Detected change to /home/rm/doc/dev/2019/05/bndtest/com.io7m.bndtest.api/target/com.io7m.bndtest.api-0.0.1-SNAPSHOT.jar. Reloading.
Speak: onDeactivate
Speak: onActivate
Speak: Hello again, again!

Once again, OSGi demonstrates how unique it currently is: I don't know of any other system that provides a type-safe, version-safe, dynamic update system such as this. I know of other systems that do it dangerously, or in a type-unsafe way, or without the benefits of strong API versioning, but none that do things as safely and correctly as OSGi.