Archive for November, 2008

Running OSGI/Eclipse plug-ins from within a normal Java application – Part 2 of 2

Monday, November 24th, 2008

In Part One I explained how to start up a whole Eclipse/OSGi plug-in system from a normal Java application. This worked fine, except there was no way to exchange data between the two.

I’ve finally figured out a way. (It was more difficult than I expected!) So here is Part Two, where I describe how.

First of all, let’s talk about what is not possible.

  • It’s not possible to allow external code (the ‘testExternal’ project from the last article) to access classes within any existing plug-in in the embedded Eclipse. At least, not without using reflection. (See this bug).
  • It’s not possible to allow the plug-ins to load code from JARs and libraries outside of Eclipse. (Exporting the Eclipse product fails).

So what are we left with?

Fortunately, there’s a way of creating a Third Kind of thing. This is called a ‘framework extension bundle’ (I think). This is a type of Eclipse plug-in which is an extension to the OSGi framework. The various classloaders running in the Eclipse world always check with any such extensions to see if a class can be loaded through them. So, the classes in such plug-ins are globally available to all other Eclipse plug-ins. Better still, Eclipse doesn’t attempt to load the classes in this plug-in… it assumes it will have been loaded as part of the process of loading Eclipse itself. That means you can load it using the classpath mechanism, which means it’s accessible to your code outside Eclipse as well as your code inside it.

Specifically, here’s what you need to do.

  1. As before, organise your code into internal code which will run inside Eclipse, and external code which will run outside. But this time, you need a third thing: crossover code which will provide the data structures that need to be accessible from the internal and external code.
  2. Create a new Eclipse Plug-in Project for this crossover code.
  3. Create classes, etc. inside that plug-in.
  4. Now the clever bit: in the MANIFEST.MF for that plug-in, add the following line:
    Fragment-Host: system.bundle; extension:=framework
    That means that the classes in this plug-in are regarded as part of the system – just like java.lang.String and similar. They will be made available to all plug-ins within the whole Eclipse system. Furthermore, they will be loaded using the standard Java class loader and the normal class path, which means they’re also accessible outside Eclipse.
  5. In your ‘internal’ plug-in (which as you’ll remember was an Eclipse IApplication) you can now access these data structures, fill in details, and return them. Note that you do not need to specify your ‘crossover’ plug-in as a dependency of your internal plug-in: it’s always available.
  6. Specifically your IApplication might look like this:
    import com.me.crossover.ResultsHolder;

    public class MyInternal implements IApplication {

      public Object start(IApplicationContxt context) throws Exception {
        IThingy foo = ; // retrieve from other plug-ins etc.
        ResultsHolder rh = new ResultsHolder();
        rh.name = foo.name;
        return rh;
      }

      public void stop() {

      }
    }

  7. In your ‘external’ project, add the ‘crossover’ project to the build path.
  8. In your ‘external’ code, take the value returned from org.eclipse.core.runtime.adaptor.EclipseStarter.run and cast it to the ResultsHolder. You can now access its fields, call its methods etc.

Now, when you run your external system, ensure you add the ‘crossover’ plug-in to the class path. Bob’s your uncle.

Obviously it’s a right pain to have to ‘trans-ship’ information from the data structures used within Eclipse into other classes which you can access externally. But right now, that seems unavoidable.

Running OSGI/Eclipse plug-ins from within a normal Java application – Part 1 of 2

Thursday, November 13th, 2008

Eclipse is a great plug-in system for Java. But superficially, it appears a bit like some allege the GPL to be – a virus! It appears that, if any of your code is an Eclipse plug-in, the whole Java system has to be.

Not true, it turns out! You can embed a whole Eclipse system, with all sorts of plug-ins, in an existing Java interpreter process. But frankly, it’s a fiddle. This note explains how. A subsequent posting will explain how you can exchange information between the Eclipse and non-Eclipse parts of the process.

But first of all, why? If you’ve got some code in Eclipse plug-ins that you wish to run from an existing Java system, your options are either to spawn a new process to run it, or to follow this recipe and run the plug-ins inside your existing Java interpreter. Why would you want to do the latter?

  • No overhead of starting a new process
  • No overhead of recompiling the class file ‘hotspots’ to native code each time
  • No overhead of running the ‘static’ cod to construct static data members
  • Easier exchange of information – freely pass Java classes back and forth

Obviously, there’s not usually any point in spawning a whole Eclipse GUI from some external Java code. But there are many libraries and whole systems written as Eclipse plug-ins, making use of the powerful OSGi plug-in system at its core. Using those is often something that an external Java program would want to do.

Enough of the ‘why’. What about the how?

  1. Work out exactly how you need to interact with your Eclipse system. You’re going to be running some code inside Eclipse, which will need to perform all the interactions with your external code. Your ‘internal’ code has full access to al the Eclipse plug-in APIs. Ideally, the information exchange between the ‘internal’ and ‘external’ code should be minimal, since it’s tricky.
  2. Create a new Eclipse plug-in project. This will house your ‘internal’ code. Let’s assume we’re going to call it ‘testInternal’.
  3. Edit the MANIFEST.MF for testInternal needs to depend upon the Eclipse plug-in(s) whose APIs you wish to use, and also on org.eclipse.core.runtime.
  4. Under Extensions, add an extension to the extension point org.eclipse.core.runtime.applications. You’ll need specify a ‘run’ element pointing at a class which implements IApplication. Go ahead and create that class. Meanwhile, the relevant part of your plugin.xml should look something like:
       <extension
             id="testInternal"
             point="org.eclipse.core.runtime.applications">
          <application
                cardinality="singleton-global"
                thread="main"
                visible="true">
             <run
                   class="testInternal.TestInternal">
             </run>
          </application>
       </extension>
    
  5. In your testInternal class that you just created, fill in the code you want to run into your start method. It’s a good idea, for now, to put some statements that will show output indicating whether this code has run. For example:
      public Object start(IApplicationContext context) throws Exception {
        System.out.println("Got into start");
        String[] args = (String[]) context.getArguments().get(
            "application.args");
        // Interact with Eclipse plug-ins as much as you like
        System.out.println("Exited start");
        return EXIT_OK;
      }
    
  6. Note that in the above code, we’re not really attempting any information exchange with the world outside of Eclipse plug-ins. If you happen to want to exchange Strings, you’re fine: you can read the arguments using the code shown. You can return any type of object at all, but as we’ll see later there are problems related to class loaders, so in practice you will need to return one of the core Eclipse objects such as a String, a Long etc.
  7. In this project, create a new Product Configuration. (File – New – Other – Plugin Development – Product Configuration).
  8. This Product Configuration should include all the plug-ins that you need. That includes your testInternal plug-in, but also all those on which it depends. (The “Select Required” button is helpful here). Make sure you specify your newly created ‘application’ extension as the one to launch.
  9. Export this Product Configuration. File – Export – Eclipse Product – Directory.
  10. You’ve now got a standalone Eclipse installation which will run your ‘application’ code and call into the plug-ins it needs to call into. It’s worth testing this. Simply run the Eclipse executable that’s been created, and you should see the results of the print statements in your code.
  11. Now we need to attack the ‘external’ code and get it to run the ‘internal’ code from within the same java process. But first, an interlude: here’s how the normal Eclipse launch process works. It is relevant, trust me!
    • The eclipse.exe launcher is complex, but its net effects when launching Eclipse amounts to approximately: java -jar org.eclipse.osgi.XYZ.jar -configuration configuration/ .... (Here, XYZ is just a version number).
    • The java process looks for, and finds, a ‘main’ function (as specified in the manifest inside that jar) within org.eclipse.core.runtime.adaptor.EclipseStarter.
    • The ‘main’ function initialises OSGi and Eclipse, loads plug-ins according to the configuration file at configuration/config.ini and then passes control to the start function of the relevant IApplication.
  12. We need to do the same in our code. Fortunately, org.eclipse.core.runtime.adaptor.EclipseStarter has some static methods we can call, to mimic the action of its main. Here’s the code you need.
      private static final String INSTALL_DIR = "...";
    
      public static void launchTestInternalFromOutside() {
        System.setProperty("eclipse.application", "testInternal.testInternal");
        System.setProperty("osgi.configuration.area", INSTALL_DIR+"/configuration");
        System.setProperty("osgi.install.area","file://"+INSTALL_DIR);
        System.setProperty("osgi.framework","file://"+INSTALL_DIR+
          "/plugins/org.eclipse.osgi_3.4.2.R34x_v20080826-1230.jar");
        Object o;
        try {
          System.out.println("About to start Eclipse");
          o = org.eclipse.core.runtime.adaptor.EclipseStarter.run(new String[]{"myParam"},null);
          if (o != null) {
            System.out.println(o.toString());
          } else
            System.out.println("o was null");
        } catch (Exception e) {
          e.printStackTrace();
        }
      }
    
    
  13. Obviously, you also need to add the org.eclipse.osgi.XYZ.jar to the classpath of this code.
  14. INSTALL_DIR is the directory of the Eclipse that you exported.
  15. The various properties are the parameters that Eclipse and OSGi needs to get started. (I couldn’t find these documented anywhere; a bit of debugging revealed what they needed to be, but you may have less luck – fiddle with them).
  16. And that’s it. Run this code, and you should see the whole Eclipse framework started, plug-ins loaded, and your ‘testInternal’ code run all from the same Java process.

So what are the caveats?

  • Data exchange between your ‘internal’ and ‘external’ code is tricky. The reason is that Eclipse plug-ins use their own class loaders (as explained in this excellent article). So, if your internal code returns a com.frangible.Sprocket, and your external code tries to cast that to a com.frangible.Sprocket, you’ll get a ClassCastException. Huh? That’s because there are two separately loaded versions of the Sprocket class, loaded by different class loaders. As far as Java is concerned, they’re separate classes. I haven’t exactly worked out the best way round this, but Part Two of this article will reveal my findings when I do. If I do.
  • That org.eclipse.core.runtime.adaptor.EclipseStarter.run method is slow. In all fairness, it has to load a lot of code. There are other static methods in the same class which should, hopefully, allow you to ‘start’ Eclipse once but then ‘run’ it many times. Presumably, this means all the heavyweight plug-in loading will occur once, and subsequent times you’ll be able simple to run your testInternal code without all that overhead. Worth looking into.
  • Something strange is going on upon exit. Control is always returned to my launchTestIntrnalFromOutside method, but the process doesn’t necessarily actually exit. Perhaps Eclipse has some sort of shutdown hook. I have yet to look into this.

So, this is a work in progress but it’s a start. Good luck!