6/28/12

Red Hat acquires FuseSource: Shaping the Future

On 27 June 2012, Red Hat announced the acquisition of FuseSource. This move will definitely shape the future of OSS integration.

Of course, this is not "yet another blog post" about spreading the news, it's more like a "personal thoughts about the acquisition".

The past

When I started studying computer science back in the 90s, I needed a brand new PC to start working on. Before I even got that new PC, I made sure that I have my hands on the operating system of my choice:

My first Linux 

The last thing I could imagine back then, is that the company with the cool looking hat would acquire the company I am working at. And how could I? I would at least need a time machine in order to do so.

A time machine from Back to the Future

The present

Ironically enough, the time machine in the picture above (from the famous trilogy back to the future) was set, to travel to 27 June 2012 (the day the acquisition was announced). Which is just awesome, because it allows me to use it in this post and add a funny tone.

"Back to the Future" control panel
So here we are today. The acquisition is announced, FuseSource is joining Red Hat's Middleware division unit.

The future

So what happens next? Well, when you bring together teams of indisputable talent you open up so many possibilities, that is making the future really hard to predict.

Yoda: "Impossible to see, the future is"

One thing is for sure. The future will definitely be really exciting and this move will definitely shape the future of OSS integration.

Thoughts

I am really happy the FuseSource has found a new home. I am more excited the FuseSource new home is an open source company. In fact, the FuseSource development and business model is the one introduced by Red Hat, so this is a sign that both companies have a common understanding on how to do open source.

FuseSource provided me with the best job I've ever had, mostly due to the fact that I had the luck to work with highly skilled people. The possibility of working in a wider environment built on top of the same principals (high skill & extra coolness) can only be thrilling.




6/13/12

OSGification: The good, the bad & the purist

Prologue
If the title doesn't make it obvious this post is all about OSGification. It's an attempt to present and categorize practices in the following categories:
  • Good practices
  • Bad practices
  • Pure practices
Some consider consider pure solutions to be the best. I fully agree that they are the best, when building a project from scratch. When migrating existing projects to OSGi you sometimes need to make sacrifices and follow a not so pure path. Especially when we are talking about libraries or frameworks that can also live outside OSGi, too.

Purpose
The main reason I am writing this is that I often encounter people that consider "pure practices" panacea and will not consider adopting a less pure solution, even if this means not adopting a solution at all. So this is an attempt to present the pros and cons of each approach, so that you can make your own conclusions.

This post assumes that you have basic understanding of OSGi.

How do I make a project OSGi compliant?
In this section I will give a really brief overview of the OSGification process. Keep in mind that its "really brief".

To make your project OSGi compliant you need to take the following steps
  1. Provide a proper MANIFEST with proper package imports/exports etc.
  2. Resolve class loading issues.
  3. Make sure that all runtime dependencies are OSGi compliant.  
In order to provide a correct MANIFEST for your bundle, you need to identify which are the runtime requirements of your bundle. Gather all the packages that your bundle references at runtime and add them to the Import-Package section of your bundle. For each of the packages one or more versions of the package might satisfy the needs of your bundle so you can use version ranges. For example:

Import-Package: org.slf4j;version=[1.4,2)

Tools that can aid you in this process are bnd, the maven-bundle-plugin etc. Those tools will do a great job in the process of aiding you in identifying those packages for you. But is this enough? Well, not always. Some times a package can be used without being referenced directly in the source (by using the classloaders etc). This is something that you have to deal on your own (will get to that later)

Once you are through in creating the imports, you also need to specify the exports of your bundle. This is easy. All you need to do is to specify all the packages of your bundle that you want to be accessed/imported by other bundles. Also specifying a version for those packages is important, because this will allow you to make use of the version oriented OSGi features (e.g. multiple versions, version ranges etc)

This will be enough for installing your bundle inside an OSGi container, but it will not always guarantee the runtime behavior of your bundle. As there might be class loading issues you need to resolve.

Common Class Loading issues

Class.forName() [1] This generally needs to be avoided inside OSGi. In most cases it will fail resulting in a ClassNotFoundException. You can read more about it in Neil Bartlett blog about OSGi readiness. You can work around this problem by specifying the class loaders that can load the class (replace it with classLoader.loadClass())

The problem is to know which is the class loader to use. If your bundle somehow imports (lot of ways to achieve this) the package of the target class, then the you can use the bundle class loader. If not things get a bit more complicated. 

Allowing your classes to be loaded from other bundles [2]
The same problem that your bundle will have in order to load classes, will have other bundles loading your classes. Unfortunately there is no global solution for this and its cases can be treated differently. It depends on the bundle, the framework and the sacrifices you are willing to make.

Singletons & statics in OSGi  [3] A lot of people do not actually realize that a singleton pattern guarantees a single instance per ClassLoader. In OSGi this means that each bundle will get its own instance of the singleton. There is nothing wrong with the pattern, it has to do with the way that java handles static variables (single instance per ClassLoader). So when using statics, make sure the don't directly cross bundle boundaries.

Java Service Loader [4] Java allows you to load object that are defined under META-INF/services/yet.another.interface. This works great when using flat class loaders but will not work that great when your class loaders are isolated. 

Bad approaches
This section discusses about approaches that are generally considered as bad. I'd say don't be afraid to use one of them just because they are generally considered bad, as long as you know the side effects and you are ready to live with them. Maybe the term last resort would be more appropriate to describe them.


Creating Uber Jars


Sometimes people in order to avoid the overhead of bundling all their runtime dependencies and also avoid class loading issues between their bundles and their dependencies they are bundling everything together under a single bundle. This results in having everything loaded from a single class loader which will eliminate challenges [1], [2] & [3] which were mentioned above.

This comes at the cost that you will not be able to add, remove or update a part of your application, since everything is part of the same artifact. I would avoid that approach, but its still better than nothing.

Fragments & Dynamic Imports


A fragment can attach itself on one existing bundle. Once the bundle gets attached both bundles can share classes and resources. This sounds cool, but there are two things that you might want to consider. The first one is that a bundle can be attached to one and only one bundle (can have a single host) and that may be limiting for your needs. The second problem is that in order to attach a bundle to a "host" bundle, you will need to refresh the host bundle. That will cause a chain reaction refreshing all bundles that depend from the host. The refreshing action will restart the activator of the bundle that is being refreshed and that is not always nice.

 Dynamic Imports is an approach that is used for dealing with class loading issues [1] and it actually allows you to specify imports with wildchards. This usally serves the need of loading classes from packages that are not known in advance. However, this can have lot of side effects, such as unwanted wirings between bundles that can affect the process of adding removing or updating bundles.

I would use fragments if I had no other means of solving my problem, but I would avoid dynamic imports at all.

Good approaches
This section discusses about approaches that are generally applied, but they are not really pure. This means that even though they do work without serious side effects there are not the best thing to do. However, in many cases they are a realistic approach that will get you to the "OSGi ready" ready state.


Thread Context ClassLoader


I tried to explain above why the Class.forName is something that will probably not work inside OSGi. But there are a lot of libraries out there that heavily rely on it. On top of that there is a good chance that you are using it too inside your application. A potential solution is to "fallback" to the thread context class loader (a.k.a TCCL).  This approach is based on the fact that a library may not be able to possibly know how to load class using its name, but the caller of the library might do. So the caller may set the thread context class loader and the library may use that to load classes.

Imagine a library that deserializes data into Java Objects. That library will try to load the class most probably using Class.forName and will fail. If you modified the library to "fallback" to the TCCL if it failed to load the class, you could have the code that uses that library to set the value of TCCL just before calling the library and then restore it to its original value after the invocation. Of course this assumes that the class loader you are going to use as TCCL is able to load the class. In many cases that is valid for a bundle that uses a library and this is why it usually works.

Some real world examples:


The fact that this approach assumes that you will be able to set the TCCL right before the invocation and also that the caller bundle will be able to load the target class does not guarantee that the approach will work in all cases. In most cases it does, yet there still might be cases were it will fail.

This is also the reason why many consider this approach "a bad practice". In practice I see more and more libraries and frameworks using this approach and it seems that it works with no side effects. I think that the key here is to know when to use it and when to go by a more pure approach.


Pure approaches


Object Factories and Resource Loaders


With this approach you avoid direct loading of the class or the resource and instead you delegate to a Factory or Loader. For application that is intended to run both in and out of OSGi you can have a default implementation that will assume flat class loaders, but inside OSGi you can have an implementation that makes use of OSGi services in order to load classes, create objects or load resources.

Passing the Class Loader


Structuring your API in such a way that whenever it comes to loading classes, to allow the passing the class loader. Although this has the least possible side effects, its not always feasible.

Use a BundleListener as a complement to the ServiceLoader

As I already mentioned above the Java ServiceLoader will not work that well inside OSGi. An approach that you can follow to work around this problem, is to use a bundle listener that will listen for bundle events and for each bundle that gets installed, it will look for META-INF/services/yet.another.interface. It can then use the bundle class loader to load and instantiate the implementation of the Service. Finally it can either register it to the OSGi service registry or to a local registry from which the bundle can look the service up.

Please note, that I am not sure if this is commonly used practice (haven't seen it documented), but it worked for me quite well in the past, without side effects so I decided to add it in this section. Feel free to drop a comment if you think the opposite.

Also note, that there is also Apache Aries - SPI Fly which intends to provide a global solution to the bridging the java service loader with the OSGi service registry. Finally, I've read some things about the OSGi 5 and some work related to the java service loader, but I don't know more than that yet.

Final Thoughts


I'll repeat myself by saying that the initial motivation for this blog post was the fact that every now and then I encounter people that are "pure or nothing". I think that its better to go by a not sure pure approach, that do nothing. Especially for existing projects, the road to OSGification can be cross in many steps and not single one.

The last year I've been spending time on Jclouds working on the OSGi support. The initial approach was to just get it running in OSGi. In every single release of jclouds improvements are applied and with the help and feedback of the community, some questionable approaches have been replaced with more solid ones. I feel that this attitude should be an example for projects that consider providing OSGi support. "No need to go for all or nothing. Step by step is also an option!"