“The classpath is dead“ … that’s what Mark Reinhold (Principal Engineer @ Sun) said at a general session during JavaOne 2009. It’s the type of statement that should resonate with any of us who have been victims of classpath/jar hell. Mark announced this in the context of reviewing the new language features due to be released in JDK 7 (ETA 2010) or more specifically the new Java module system codenamed Jigsaw.

Module Systems and Jigsaw

Jigsaw is a low level implementation of a module system, which also includes some language and VM changes that are specified in JSR 294 (note that Jigsaw is being done outside of any JSR and is not a reference implementation of 294). Another implementation of a module system most Java developers are likely more familiar with is OSGi. But what is a module system and why do we need one?

Depencies on multiple versions (of module B in this example)

A module system enables modular programming, it allows developers to separate code into clearly defined modules that can specify what other modules they depend on and their versions, can be internally modified without breaking other modules that depend on it and can hide its own internals from external modules.

A module system can solve the classpath/jar hell problem by leveraging the previously mentioned features in order to allow an application to use the right versions of the set of libraries it requires. In the diagram shown here, we have an APP module that depends on the A, B and C modules. But module C also depends on B, however this dependency is to a different version of B than the one APP needs. A useful module system would then allow us to have 2 versions of the B module, with the compatible version and implementation visible to the appropriate modules (in this case APP and C).

Also, by clearly expressing module dependencies, we no longer have to look at random lib folders or hidden documentation when using a 3rd party library, just so we can guess what the corresponding jar files are needed to run it. With the system implemented by Jigsaw, we can simply introspect the module and let it tell us what those dependencies are. It takes the guesswork and the dark magic completely out of the equation.

But back to Mark’s presentation at the general session. The main reasons to introduce Jigsaw into JDK 7 are the following:

• Eliminate classpath complexity (classpath/jar hell problem already discussed).
• Increase performance. The VM download size can be made smaller, and startup time should improve because for small programs (say HelloWorld) unnecessary classes don’t have to be loaded at all. Mark showed how he could run HelloWorld by only using 2 modules listed by the new jmod command; jdk.base@7-ea and jdk.boot@7-ea.
• Enable native integration for installing modules.

Mark demonstrated that last point by showing a demo of him installing various JDK packages via the Synaptic Package Manager.

Installing Java modules via Synaptic Package Manager

Alex Buckley’s presentation

I attended Alex Buckley‘s (Computational Theologist @ Sun) session on “Modularity in the Java Programming Language: JSR 294 and beyond”, which had a lot more details on the specific language changes to be implemented in order to support this new feature.

The most noticeable aspect of this is the introduction of module-info.java. This file goes at the top of your module filesystem, and is the class that specifies at compile and runtime the definition of the module. Here is an example, coming from Alex’s slides;

module org.planetjdk.aggregator {
          system jigsaw;
          requires module jdom;
          requires module tagsoup;
          requires module rome;
          requires module rome-fetcher;
          requires module joda-time;
          requires module java-xml;
          requires module java-base;
          class org.planetjdk.aggregator.Main;
 }

Our module is “org.planetjdk.aggregator” and uses the jigsaw module system. Depenencies are denoted by using requires and providing the module name, Alex calls this part of the module metadata the “logical classpath” of the module. You can also define the main class for the module which in this case is “org.planetjdk.aggregator.Main“.

You’ll also notice that the module name is quite long “org.planetjdk.aggregator”, while the list of required modules uses shorter names. This can be done by supplying a module alias. In the previous example, we could have added the line;

provides module aggregator;

This would then allow other modules to say “requires module aggregator” and successfully link to this module by this shorter form of the name.

Also, you can specify your module version and version dependencies in module-info.java. This updated example illustrates this mechanism;

module org.planetjdk.aggregator @ 1.0 {
          system jigsaw;
          requires module jdom @ 1.*;
          requires module tagsoup @ 1.2.*;
          requires module rome @ =1.0;
          requires module rome-fetcher @ =1.0;
          requires module joda-time @ [1.6,2.0);
          requires module java-xml @ 7.*;
          requires module java-base @ 7.*;
          class org.planetjdk.aggregator.Main;
}

Note also that you can only have one module-info.java per module, in only one location, so if you have a big source tree and you want to create multiple modules from it you will have to either separate your source into different trees or use some creative scripting to accomplish this at build time. I believe Mark mentioned they had to use some scripted solution for the jdk base as I believe it's one tree.

The rest of Alex's presentation focused on how to locate modules at compile and runtime by using the "modulepath", the use of the new module keyword and how it affects accessibility, and he gives a glimpse as to how they plan to support other module systems (for example, the version notation can be different when using other module systems).

This is all better explained in Alex's slides, so I encourage everyone to take a look at his presentation (link provided above).

The Modular Java Platform: Q&A

There were rumors in the meeting that the clown is a secret member of the Java Posse

Wednesday night, I attended a BOF like meeting hosted by Mark and Alex setup in order for the community to openly ask and voice any concerns over the work presented by them on modular Java. The setup was sort of a round table configuration (without the round table) in a very informal setting (btw it was hard for participants in the back to listen well, hopefully microphones are used next time). The first question was kind of funny, somebody asked Alex what a "Computational Theologist" does (the title makes it sound like the equivalent of a Thomas Aquinas for the nerd world).

After that, it seemed some people were very concerned about how to deploy modules, patch jars and other deployment issues. My impression was that a lot of people had a problem with the native installation part of the proposal. Mark answered that Java developers have gotten used to being in our own little jar world, separate from the OS integration issues of module and library management and that it was time for Java to integrate more with these systems. He has a point, do we really have to deploy log4j jars with all our products over and over instead of simply having a log4j installed in the system that all modules can refer to?

However, installation with native package systems is a big change. And it is also unclear what that is going to look like in Windows. It seemed like the Windows part of the equation has not been finalized, would Java 7 include some type of Java specific module registry or would it leverage something like .NET's Global Assembly Cache?

Somebody bought up the point that this new system was imposing more work on open source projects. That is, today you want to release your library as a jar, you know it's going to work everywhere. With Jigsaw, projects may have to deliver a large set of native module packages depending on all the platforms they want to deploy to. Many suggested to Mark and Alex that a cross platform package format was needed, but Mark acknowledged that Sun is not really working on that and they're just worrying mainly about specifying the module file structure and leaving the rest to the native package systems.

I noticed a few members from the OSGi community in attendance, I recognized Peter Kriens (member of the OSGi Alliance), Richard S. Hall (From Apache Felix, works for Sun in the Glassfish team) and BJ Hargrave (CTO of the OSGi Alliance, committer on Eclipse Equinox and IBM employee). Peter and Richard are both members of the expert group for JSR 294, so I was a bit surprised they weren't at all in "harmony" with Mark and Alex and the work they've done with Jigsaw. See, last time I saw discussions on this topic, JSR 277 was still active (it is on hold for now) and I was under the impression that Sun and OSGi had reached some type of "gentleman's agreement". That doesn't seem to be the case at all.

OSGi is a very mature technology and has more features than Jigsaw. However OSGi doesn't address the issue of native package installation, and you could argue that it would not be suitable to solve the issue of modularizing the JDK itself. I think on that last point, many people don't mind Jigsaw even on the OSGi side, the arguments come when you say Jigsaw could be used as an application level module system. Additionally OSGi still has some key features that are missing in Jigsaw; it allows more dynamic loading of modules (useful for plugins) and is backwards compatible (Jigsaw will only be supported from Java 7 and up).

Having said that, one thing I like more about Jigsaw is how it defines dependencies is the module-info.java file. It is a much better syntax than the error prone use of manifest files used in OSGi, where an extra space can break the whole thing for you. Adding keywords for module accessibility is also a very nice thing, and it seems the goal is to also let a modular system like OSGi take advantage of these features. Now how can these both systems inter-operate between each other in a single VM? I have no idea how that works ... yet.

In the end, the Q&A showed that there is a lot of controversy and confusion over this new feature. There was also a noticeable lack of questions during the meeting, with a few of the same people asking all the questions. I got the impression that the need and functionality of a module system in Java is not generally fully understood, so to me that means a lot more work needs to be done to explain just what is going on and why there is so much effort going into this area.

Some key questions I'd like to see clarified:

• A more technical and detailed explanation of why Jigsaw was chosen over OSGi.
• Will the JDK provide tools for wrapping modules in native packages?
• Is a cross platform package system completely out of the question?
• How would multiple modules systems work together in the same VM/application instance?
• What happens when an applet or javafx application requires core JDK modules that are not present in the current installation?
• Native library module integration issues (I sort of asked this in the Q&A). Where are native packages placed, and what happens when multiple modules include the same native libraries (say 2 modules have the same DLL embedded in them)?

This doesn't affect me, why should I care?

These changes will affect everybody, at the very least under the hood your core Java packages will be loaded in a more modularized fashion. For people who would like to run Java in other devices, you will notice that you have better ways to split up your Java distribution to only the parts that you need.

However, if you are interested in scaling your code the right way and modularizing it, you should start becoming familiar with what is going on in this space. For your applications, you'll still have the choice of using existing modular systems like OSGi or using the new facilities provided by Java 7. In the future, we should expect more details to come out as to how you can combine both (Jigsaw and OSGi). Others might prefer not to depend on an external Java project like OSGi and leverage the core support that will be provided for modules.

On the other hand, some people criticize that his is a repeat of the java.util.logging fiasco and say they can mostly ignore these changes. The jury is still out, but this is a very fundamental change in the language and platform and a very different scenario than choosing what logging package to include in the core JDK.

Oh and you might soon see new annoying interview or exam code questions trying to trip you up with gems like this one (from Alex's slides and this blog entry):

package module;
// BTW you should NEVER write code like this!!!
module class module {
        module module module = new
            module();
        module module() {}
}

Anyways, sorry for the longish blog post but hopefully this sheds some light into some of the discussions about this topic that took place at JavaOne. If you are interested in getting more in depth details about this topic, please take a look at some of the following links;

Java SE: Project Jigsaw: Modularizing JDK 7
Project Jigsaw (Mark's blog)
The Modular Java Platform (Mark's blog)
The massive, monolithic JDK (Mark's blog)
Packaging Java code (Mark's blog)
Project Jigsaw: Language changes for Modules
Project Jigsaw (Peter Kriens, his version of the "saga of Java modularity)
Project Jigsaw #2 (Peter Kriens, some criticisms of the proposed system)
Project Jigsaw #3 (Peter Kriens, Jigsaw != JSR 294)
Jigsaw - the death knell of JSR277?
Jigsaw vs OSGi?
A wrinkle with 'module' (Alex's blog)
Modularity in Java (Alex's presentation at DEVOXX 2008; covers module friendship, "virtual modules", multi-module packages, type observability and other topics not covered in the JavaOne slides I linked to in this article)

Update. Some additional links that have come to my attention after publishing this blog post;

Jigsaw Feedback Roundup (Dalibor Topic)
Java Posse #259 - Jigsaw and JSR 294 Interview (Java Posse podcast interview with Mark Reinhold and Alex Buckley)