Ant/How to add new Ant versions

• User Guide
• Developers
• Test Script
• Java 8

This wiki discusses the steps to add a new version of Apache Ant to the Eclipse platform. The intended audience is committers, but everyone is free to contribute versions.

How it works

Eclipse consumes Apache Ant from Orbit as a 'normal' Eclipse bundle rather than try to integrate Apache Ant as-is. Below is a quick overview of the process, with the following sections providing more details.

A quick overview of the process:

• Ant is taken from Apache site and converted to the Orbit bundle format.
• A CQ is opened to get legal approval to use the new version
• Once approved, the new version is added to Orbit
• Smoke testing and test builds are done
• Assuming no problems - the platform is updated

Creating the Orbit bundle

The first step is to create the new Orbit bundle that can be used in your workspace (or installed in your host) to test with. This section assumes you are familiar with the terminology of Eclipse bundles, using the PDE tools, testing / running JUnit tests and have read the relevant how-tos from the Orbit wiki about adding / configuring bundles.

The bundle

The best way to see the expected shape of the Ant bundle is simply to look at an older (or current) version used by Eclipse. You can grab an older version from the CVS repository, but you will have to rename it in your workspace (or its name will collide with the new version you are trying to create). At the time of this writing the version used in Eclipse is 1.9.2, so you will want to check out the 1_9_2 branch of Ant from orbit.

Once in your workspace, the bundle will look like the following:

The important parts to note in the bundle are:

• about_files - contain all of the license files directly from Apache
• bin - contains all of the script files from the binary distribution of Apache Ant
• etc - contains all of the styles sheets for Ant tasks
• lib - contains all of the jar files from the binary distribution of Apache Ant
• source-bundle - contains the Orbit-formatted source bundle, whose content come from the source distribution of Apache Ant (more on this later)

The remainder of the files / folders are standard Eclipse bundle files which don't require special mention. Any of them that requires editing will be mentioned in the following sections.

Grab the new Apache Ant version

The first step to creating our new orbit bundle is grabbing the binary and source distributions of Apache Ant.

They can be found on their respective download pages:

• binary distributions
• source distributions

Once you download the two versions, extract them some where convenient. Also take note the Apache download page mentions you might need special steps to extract the distributions due to really long file names.

Create the workspace bundle

With the latest version of Apache Ant acquired and an old version of the Ant bundle to compare to, lets create the new version in our workspace. A note here: it is a good idea to have no other bundles open in you workspace while you are working on Apache Ant. The reason for this is that Apache Ant is a low-level bundle and bringing it into your workspace / editing it seems to cause confusion in your target platform. So best to just keep all your other bundles closed while you create the new version.

Lets go:

1. Create the org.apache.ant bundle
2. Fill in the MANIFEST.MF information as required. 'Id' is org.apache.ant, 'version' is 1.9.3_qualifier (for example), 'name' is Apache Ant and 'provider' is Eclipse Orbit. Make sure to also externalize these values for translation (you can use the PDE wizard for this).
3. Add a bundle dependency on org.eclipse.osgi in the MANIFEST.MF (do not include version constraints on it)
4. Add a BREE of J2SE-1.5 in the MANIFEST.MF and click the update classpath settings link.
5. Copy the /about.html file from the old version to the new version. Open it and change the date to the current date (update the copyright as needed), Update the version of Ant mentioned and update the link to the what's new for the current release.
6. Copy all of the files from the .settings folder of the old version into the .settings folder of the new version
7. Edit the build path properties and set the default output folder to be org.apache.ant/bin-generated
8. Create /about_files, /bin, /etc, /lib, /source-bundle and /source-bundle/libsrc folders
9. Copy the /MANIFEST.MF/eclipse.inf file from the old version to the new version
10. Copy the build.properties from the old version to the new version (update the copyright as needed)

Thats it! At this point we have the complete bundle shape, all that remains is to add the updated Ant libraries / files.

Update the binary parts

First we will start with updating the binary parts of the bundle. This involves copying files from the binary distribution of Ant. To simplify the steps I will assume the root of the binary distribution of Apache Ant to be /binary. So for example when I talk about copying a file from the bin folder I will refer to it like: /binary/bin.

Lets go:

1. Copy all of the files from the /binary/bin folder to the /bin folder of the project. These are a bunch of *.sh, *.pl, *.py, *.bat and *.cmd files.
2. Copy all of the *.xsl files from /binary/etc to the /etc folder of the project. DO NOT copy the jar file.
3. Copy the /binary/etc/checkstyle folder (and all of its contents) to the /etc folder of the project.
4. Copy all of the jar files from /binary/lib to the /lib folder of the project. DO NOT copy any of the pom files or anything else, only the jar files.
5. Open the MANIFEST.MF editor and go to the Runtime page. Now using the Add... button in the Classpath section, add all of the jar files from the /lib folder to the classpath. Once that is done, use the Add... button in the Exported Packages section to export all of the org.apache.tools.* packages.
6. Now open the .classpath file to confirm it contains all of the library entries like: <classpathentry exported="true" kind="lib" path="lib/ant.jar"/>
7. Open the project Java build path properties and go to the Libraries tab. Confirm all of the libraries are there. For each one of the libraries we want to make sure the source path is correct - to do so expand each library and edit the source attachment value. Each value is from the workspace and points to /org.apache.ant/source-bundle/libsrc. After you are done setting the source paths, go to the Order and Export page and confirm all of the Ant libraries are checked (if not, do so).
8. Reopen (or go to) the MANIFEST.MF editor and on the overview page hit the update classpath settings link. Now open the .classpath file and confirm you have all of the library entries like the following: <classpathentry exported="true" kind="lib" path="lib/ant.jar" sourcepath="source-bundle/libsrc"/>

If everything checks out with the manifest and classpath, then we are done. If not go back through the steps to confirm they are all done.

Update the source parts

Next we update the source parts of the bundle. This involves copying files from the source distribution of Ant. To simplify the steps I will assume the root of the source distribution of Apache Ant to be /source. So for example when I talk about copying a file from the src folder I will refer to it like: /source/src.

Lets go:

1. Save the license text from [1] to /about_files/LICENSE-2.0.txt in the project
2. Copy /source/LICENSE and /source/NOTICE to the /about_files folder in the project
3. Copy /about_files/DOM-LICENSE.html and /about_files/SAX-LICENSE.html from the old project to the /about_files folder of the new project
4. Copy the entire /about_files folder (and all its contents) from the new project into the /source-bundle folder of the new project - giving us a /source-bundle/about_files folder (and contents)
5. Copy the /etc folder (and all its contents) from the new project to the /source-bundle folder - giving us a /source-bundle/etc folder (and contents)
6. Copy the about.html file from the new project to the /source-bundle folder
7. Copy the /source-bundle/build.properties and /source-bundle/plugin.properties files from the old project into the /source-bundle folder of the new project (update the copyrights as needed)
8. Copy the /source-bundle/META-INF folder (and all its contents) from the old project to the /source-bundle folder of the new project. Open the copied /source-bundle/META-INF/MANIFEST.MF file and update the bundle version number to match the new project version
9. Copy the entire directory (and all its contents) from /source/src/main into the /source-bundle/libsrc folder of the new project - giving us a /source-bundle/libsrc/org rooted directory in the new project

Thats it! We now have the source bundle prepared.

At this point the Apache bundle is fully functional, so any projects that were closed earlier can be reopened. If there is some weirdness happening, like an error showing up on the Plug-in Dependencies container, simply restart Eclipse and do a full clean - like I mentioned having Apache Ant in your workspace can cause the PDE to behave strangely.

In the next section we will talk about testing the bundle and some expected changes to the Platform Ant code.

Updating Platform Ant

Depending on the scope of changes from the previous release, there could be nothing but test updates to make. In this section, I will only discuss running the test suites and the required updates to bundle versions and tests. I also am assuming you have the platform Ant source in your workspace.

Any changes you make to platform Ant should be attached to the bugzilla that was opened for updating to the new version of Apache Ant.

A complete test pass of the smoke-testing script must be completed. See the test script page for more details.

Updating versions

If the new version of Apache Ant changes API or requires significant changes that make it no longer compatible with the previous version, you will have to go through all of the platform Ant bundles and makes sure they have a minimum requirement on the new version. If, however, the changes are minimal (a point release) you should not have to make any changes to the platform Ant required versions.

Updating the tests

One of the first things you would have done with your new Apache Ant bundle is to run the test suites to make sure there are no regressions.

In doing so, you would find two tests that will always fail when a new version of Apache Ant is used:

• org.eclipse.ant.tests.core.tests.OptionTests
• org.eclipse.ant.tests.ui.debug.PropertyTests

The reason being that these two tests both test the version that is returned from launching Ant, so you have to update the constants in both classes to the latest version string.

Approval for use

Once you are happy with the new version and everything is working as expected, we have to open a CQ to get approval to add the new version to Orbit and to use it in the platform.

To get approval:

1. Go to https://dev.eclipse.org/ipzilla/ and find the most recent approved update for Apache Ant. We want the CQ approved for use in eclipse.platform. As of this writing that is: Apache Ant 1.9.2
2. Actually creating a new CQ will require a committer, so if you are not one, find one to do it for you and ask really nicely. Remind the committer (or yourself) to mention the previously approved version in the description of the new CQ (the reason we looked it up)
3. Once the CQ is open you have to attach the source WITHOUT any embedded archives. So for our snazzy new bundle we created in the Creating the Orbit bundle section, we have to remove the /lib folder completely before zipping it up and attaching to the CQ
4. The CQ will go through a few stages, but just remember if it is set back to awaiting_committer you have to address the issues raised and DO NOT forget to return it to the IP team (use the button).

Eventually the CQ will be approved and the checkintocvs keyword will be added. Leading to the next section, actually getting it into Orbit.

Put it in Orbit

With the use of the new version approved in the last section, we need to commit the new bundle to Orbit. Like creating the CQ, this step requires commit rights, specifically to Orbit. If you do not have commit rights to Orbit you can find a committer to help on the Obit committers page. Ideally you would ask the committer listed as being responsible for the particular project, failing that asking another one won't hurt. As of this writing the the committer responsible for org.apache.ant is Michael Rennie.

Once the bundle has been added to Orbit, you can delete it from your workspace and grab the version committed to Orbit!

Testing and Test builds

Now that we have the latest and greatest new version in Orbit, we need a test build with the latest version to make sure it works with the rest of the platform (smoke testing).

Before bugging David Williams for run a test build for the platform, go through the relevant steps on the Orbit builds wiki to produce a testable build yourself. Once you are confident, you can request David run a build of the platform with the new version of Apache Ant.

With a freshly minted platform build in your hands, at a minimum confirm:

• The names of the new Apache Ant bundles (core and source) show up correctly in the About dialog
• Debug something to ensure the Apache Ant source can be found
• Open a type from Apache Ant to make sure the source can be found
• Close Apache Ant in your workspace and re-run the unit tests to make sure they all still pass.
• A complete test pass of the smoke-testing script must be completed. See the test script page for more details.