PDEBuild/Individual Source Bundles

Starting in Eclipse 3.4, source can now be shipped as individual bundles instead of the old way of one bundle containing the source for multiple plug-ins. This allows us to be more flexible around how source is delivered, and has the happy side effect of reducing the path length problems sometimes encountered when using Feature Qualifier Suffixes.

This page outlines the steps necessary to generate individual source bundles when building your project. See the help page for a review of the old way of generating source.

individualSourceBundles

In your build configuration's build.properties file, set the property

individualSourceBundles=true

This tells pde.build that you want to generate the new-style source bundles. If you are lucky, this is all that is required. However, generally, things are never that easy.

generate.feature

In a feature's build.properties file, the generate.feature property tells pde.build to generate a source feature.

generate.feature@<source feature id> = <feature id> [, feature@<feature id>] [, plugin@<plugin id>[;version=<pluginVersion>][;unpack="false"]]

When generating individual source bundles, this property remains as before, the difference will be noticed in the resulting source feature. Before, the source feature would have included 1 source plugin + a source fragment for each platform being built. In the new way, the source feature will include a source bundle for each plug-in/fragment listed in the original feature.

Plug-ins that were included in the source feature via the plugin@ syntax will not get corresponding source bundles. This is useful for adding doc plug-ins to the source feature.

Some plug-ins that were included in the originating feature (ie doc plugins) may not have source and should be excluded from the generated source feature. This can be accomplished with the addition of exclude@<plugin id> arguments to the generate.feature property.

Example:

generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",exclude@org.eclipse.jdt.doc.user

generate.plugin

The generate.plugin generates a source bundle based on a given feature. When generating individual source bundles, this changes to be based on a given plug-in:

generate.plugin@<source plug-in id>=<plug-in id>

The generate.plugin property was used by features to include source without having a source feature (even though behind the scenes an entire source feature was generated). When generating individual source bundle, features will need to include a *.source bundle for each plug-in along with a corresponding generate.plugin property for each one.

Or, the generate.plugin property can be changed to a generate.feature property and the source feature included. By default a generated source feature does not have a bin.includes and will be excluded from the build.

Example: The sdk.examples feature used to look like this in the feature.xml

   <feature id="org.eclipse.sdk.examples"  ... > 
        ....
       <plugin id="org.eclipse.sdk.examples.source"  version="0.0.0"/>
       <plugin id="org.eclipse.sdk.examples.source.win32.win32.x86" version="0.0.0"/>
   </feature>

and in the feature's build.properties

   generate.plugin@org.eclipse.sdk.examples.source=org.eclipse.sdk.examples

This changes to the following in the feature.xml:

   <feature id="org.eclipse.sdk.examples" ...>
      ...
      <includes id="org.eclipse.sdk.examples.source" version="0.0.0"/>
      ...
   </feature>

and this in the feature's build.properties

   generate.feature@org.eclipse.sdk.examples.source=org.eclipse.sdk.examples

Or in the feature.xml:

   <feature id="org.eclipse.sdk.examples" ...>
      ...
      <plugin id="org.eclipse.compare.examples.source"  version="0.0.0"/>
      <plugin id="org.eclipse.debug.examples.core.source" version="0.0.0"/>
      <plugin id="org.eclipse.swt.examples.source" version="0.0.0"/>
   </feature>

and in the feature's build.properties

   generate.plugin@org.eclipse.compare.examples.source=org.eclipse.compare.examples
   generate.plugin@org.eclipse.debug.examples.core.source=org.eclipse.debug.examples.core
   generate.plugin@org.eclipse.swt.examples.source=org.eclipse.swt.examples

generateSourceBundle

Specific plug-ins may not require source bundles because they don't actually contain source. This may occur with platform specific fragments that only contain a native library. In this case, the bundles may be excluded by the feature as outlined above in generate.feature.

Or, bundles can explicitly specify in their own build.properties file that no source bundle should be generated for them:

generateSourceBundle=false

src.zip in CVS (Custom Content)

Some bundles (for example org.junit4, or org.eclipse.osgi.util) have a src.zip in CVS that is included in the source bundle via the src.includes property. When building individual source bundles, this src.zip should be unzipped into the root of the source bundle. This can be accomplished using customBuildCallbacks in the plug-in.

As an extension to "src.zip in CVS" above this, custom content can always be included in the generated source bundle using the src.includes property and the post.gather.source custom callback.

Source Roots

In order for PDE/UI to properly access it, custom source should be place in proper location. Class file libraries are mapped to source locations in a predictable manner. The root of the plug-in maps to the root of the source bundle. This is the main case for most jarred plug-ins. If the plug-in contains additional libraries "foo/lib.jar", the source is expected to be in corresponding "foo/libsrc/" folders in the source bundles. Example:

org.eclipse.pde.core.jar		org.eclipse.pde.core.source.jar
  - org/**/*.class			  - org/**/*.java
  - ant_tasks/pde-ant.jar		  - ant_tasks/pde-antsrc/org/**/*.java

Eclipse-SourceBundle: org.eclipse.pde.core;version="3.4.0.N20071128-0010";roots:="ant_tasks/pde-antsrc,."

org.eclipse.pde.build/			org.eclipse.pde.build.source.jar
  - pdebuild.jar			  - pdebuildsrc/org/**/*.java
  - lib/pdebuild-ant.jar		  - lib/pdebuild-antsrc/org/**/*.java

Eclipse-SourceBundle: org.eclipse.pde.build;version="3.4.0.N20071128-0010";roots:="pdebuildsrc,lib/pdebuild-antsrc"

The roots directive on the Eclipse-SourceBundle header should list the root folders contained in the source bundle. When adding additional source to a source bundle, set the property src.additionalRoots in the plug-in's build.properties file.
Example:

org.junit4 :
   build.properties
      src.includes = about.html,junitsrc.zip
      src.additionalRoots=junitsrc
      customBuildCallbacks=customBuildCallbacks.xml

   customBuildCallbacks.xml
    	<target name="post.gather.sources" >
		<mkdir dir="${target.folder}/junitsrc"/>
		<unzip src="${target.folder}/junitsrc.zip" dest="${target.folder}/junitsrc" overwrite="false"/>
		<delete file="${destination.temp.folder}/junitsrc.zip" />		
	</target>

Branding for the Source Feature

Previously, the generated source plug-in served as a branding plug-in for the generated source feature. Branding files (about.properties, eclipse32.gif, etc) were provided using the sourceTemplatePlugin directory in the original feature.

When generating individual source bundles, the branding plugin for the source feature will be the source bundle corresponding to the original branding plugin. The files from the sourceTemplatePlugin directory will be copied into this branding source plugin.

sourceTemplatePlugin vs sourceTemplateBundle

Previously, the contents of the sourceTemplatePlugin directory of the original feature was copied into the source plug-in. When generating individual source bundles, this remains true only for the branding source bundle (see above).

For all the other source bundles, the contents of a sourceTemplateBundle directory will be copied over.

Custom build.xml

When using a custom build script, source bundle generation can be turned off in the build.properties file using the generateSourceBundle=false property. Or, if a source bundle is desired, the custom build.xml must contain a gather.individual.sources target.

For example, org.eclipse.swt.gtk.linux.x86 contains the following :

	<target name="copy.src" depends="init">
		<copy todir="${destination.temp.folder}/" failonerror="true" overwrite="true">
			<fileset dir="${plugindir}/Eclipse SWT/cairo/" includes="**/*.java" excludes=""/>
			.....
			<fileset dir="${plugindir}/Eclipse SWT Theme/gtk/" includes="**/*.java" excludes=""/>
			<fileset dir="${plugindir}/Eclipse SWT/common/" includes="**/version.txt" excludes=""/>
		</copy>
	</target>
	<target name="gather.individual.sources" depends="init">
		<antcall target="copy.src"/>
		<copy todir="${destination.temp.folder}">
			<fileset dir="${basedir}" includes="about.html,about_files/"/>
		</copy>
	</target>

Related Bugs

bug 202462 : Simplify the way source is contributed
bug 208821 : Change SDK to use individual source bundles
bug 148782 : Let's re-design how source is associated with code
bug 206596 : Source bundles should be in new format