Source Bundles

This page contains the problem description behind source bundles in Eclipse. As of version 3.4, Eclipse plans to support individual JAR'd source bundles as described by Individual Source Bundles.

A recap of the situation

Today we tend to have a source feature for every binary feature. The source features include a source plugin as well as some number of ISV doc and potentially example plugins.

The source plugin in turns contains an "src" directory and in that directory is a directory for every plugin in the corresponding binary feature. Each source folder contains one or more source zips corresponding to the JARs in the binary versions of those plugins. The plugin also contains a plugin.xml that contributes an extension to the org.eclispe.pde.core.source extension point. An example is given below.

 <extension point="org.eclipse.pde.core.source">
   <location path="src" /> 
 </extension>

Today the Eclipse project ships 4 source features for a total of 6 source plugins (there are some platform-specific source plugins). Each of the plugins is installed as a directory rather than being a JAR.

The problem

The 3.3 plan includes an item for shipping finer grained components. This is meant to make acquiring Help and Update function (for example) easier as there would be a well-defined feature that captures the set of bundles needed for that function.

If we follow the current pattern, each of these new features will spawn a new source feature. These features and plugins are autogenerated so this is not an overwhelming development load but it does increase the feature count significantly. Note also that there will be a large number of features containing only one (or a small number) of plugins. Function such as Search, Jobs, etc can quite reasonably be used independently. The point of the 3.3 plan item is to make these easily accessible.

The Orbit project will exacerbate this problem by introducing a large number of standalone bundles for which we will want to make source available. Creating binary and source features for each of these would not be feasible. Note also that in many cases the Orbit team will have source archives from the originating project and would like to avoid repackaging them. Instead the should be able to bundle them as they would any other JAR.

There is an additional problem discovering where the source for a given plugin lives. Take this little quiz: Where is the source for org.eclipse.help? And for org.eclipse.help.base? The challenge is in mapping back from binary bundle to containing feature and then to source feature. The answers to the quiz are not given here. The best way to understand the problem is to go through the pain of this process yourself, so this is left as an exercise for the reader.

The solution

The proposal here is to package the source for each bundle in a corresponding source bundle. The 3.3 plan also includes some work towards finer-grained (well righter-grained) provisioning. This will enable delivery of the new finer grained components and allow these new source bundles to be provisioned independently (or at least at the right granularity). For example, developers would not need to download all of the source for the Platform just to see the source for one part.

Note that this eliminates the need for reverse mappings from binary to source. The source bundle id is either a mechanical mapping of the binary bundle id or explicitly noted in the metadata of the binary bundle.

One of the first reactions is "that will double the number of bundles in the SDK and performance will take a nose dive". True and false. If we ship all the source bundles in the SDK then indeed the bundle count will double. However, there is actually no need whatsoever for the source bundles to be "installed". That is, the IDE itself does not make use of the bundles. The source developers want is the source of the bundles in the __target__. By default the running IDE configuration is the target and the source plug-ins are there. We just don't need to install them in the runtime.

As pointed out before, as we advance in the provisioning story, it will become increasingly irrelevant to ship all the source with the SDK. Developers will be able to get the source quickly and easily as they would any other bundle.

Outstanding issues

• The shape of the source contribution needs to be ironed out. The extension mechanism should be retained for backwards compatibility but the new mechanism likely should avoid injecting plugin.xmls if possible. Perhaps some markup in the source bundle's manifest.mf
• The update.configurator and other mechanisms may need to be modified to avoid installing source bundles into the Equinox framework.
• PDE Build needs to generate the new bundles and the source feature generator would have to be updated to do the new thing. This may be subtle if we want to be able to generate source features in the old form.

See also Individual Source Bundles