RMF/Contributor Guide/Build Process

We're still in the incubator phase - therefore, the release process is still being developed. Here are our guidelines so far:

The Most Important Stuff

• Our release manager is Lukas Ladenberger. Please coordinate with Lukas.
• We use the Git Flow Process as our release process
• Important: Never commit to master! (unless you are the release manager). Development takes place on the develop branch.
• The only branches on the server are master and develop. All other branches that are mentioned in git-flow are local branches

Glossary

(17-Jan-2012) There has been a lot of discussion on rmf-dev. Some of the confusion stems from sloppy terminology. Therefore, here a little glossary:

Release 

A Release is a super-official artifact that is typically published after reaching a number of milestones and building a few release candidates. The first release is planned for April 1st, 2014 (not a joke :-)).

Milestone 

A milstone is a well-defined point towards a release. For now, we create milestones informally if we need an intermediate well-defined build.

Release Candidate 

After the last milestone before a release, one or more release candidates are built. No major features may go into a RC, only bug fixes. As of this writing, release candidates are not used.

Nightly Built 

A nightly built is typically generated by an continuous integration system, based on the latest code in the repository (develop branch in Git) and may be instable. It is published, so that daring users can try it out. As we currently don't have a CI system, we don't have nightly builts.

Snapshot 

Deprecated! Similar to a nightly built, but built on demand from the latest sources and published (master branch in Git).

Integration 

We use the term "Integration" to mark the end of a development iteration. We use a schedule-driven (rather than feature-driven) development process. Integrations are created in the Milestone-Field in Bugzilla. At the beginning of a development iteration, all bugs are reviewed by the team and assigned to an integration.

Milestone Process

Milestones are identified by a tag in git and by being located in the milestone folder, for binaries. Milestones are marked as follows, assuming Milestone 1 of RMF 0.10.0:

• git Tag name: milestone-0.10.0-M1
• Subfolder name for binaries (somewhere underneath a milestone folder): 0.10.0-M1

Release Process

The RMF build process is inspired by the Git Flow Process. Most of the text used in this chapter are citations from [1]. In general, this section describes a step by step instruction how to create a new release of RMF.

(1) Prepare release

• Review and revise the Roadmap. Review open issues (bugs and feature requests) and check if all bugs have been fixed and all feature requests have been implemented scheduled for this release. Not fixed bugs and not implemented feature request are assigned to the upcoming release.
• Check if the rmf.develop.mars build ([2]) builds without errors. Please fix the build, in case of an corrupt build.

(2) Merge develop branch into master branch

Use the following git commands to merge the develop branch into the master branch:

$ git checkout master
$ git merge --no-ff develop
$ git push

Now check that the Git revision ID on Hudson is still the one you have now in the master branch.

(3) Creating a tag

The next step consists of creating a tag for the release. In Git:

$ git tag -a release-0.4.0

No we have to push our tag. In Git:

$ git push origin release-0.4.0

(4) Publish Artifacts

In the next step you have to publish the different artifacts, which can be found in Jenkins in the last successful build section:

• RMF Feature update site

Put the different files in the correct folders on the eclipse server. The RMF download folder structure is well documented: RMF/Contributor_Guidline/Build_Process#Download_Folder_Structure.

Edit the two files compositeArtifacts.xml and compositeContent.xml in the updates folder and add a new line in the children section pointing to the new release version.

Here is an example, showing the compositeContent.xml file, where we added the new release version 0.4.0:

<?xml version='1.0' encoding='UTF-8'?>
<?compositeArtifactRepository version='1.0.0'?>
<repository name='"RMF Feature Update Site"'
    type='org.eclipse.equinox.internal.p2.artifact.repository.CompositeArtifactRepository' version='1.0.0'>
  <properties size='1'>
    <property name='p2.timestamp' value='1243822502440'/>
  </properties>
  <children size='3'>
	<child location='releases/0.3.1'/>
	<child location='releases/0.3.2'/>
        <child location='releases/0.4.0'/>
  </children>
</repository>

PLEASE NOTE: You have to increase the size attribute of the children tag!

(5) Bump version numbers

PLEASE NOTE: Switch back do the deveop branch!

In the last you have to bump the version numbers to the next version (in our case version 0.5.0). This is still a manual task which is cumbersome and error prone, since, there are many places to bump the version number. We have to improve this. There is already a bug entry for this task: https://bugs.eclipse.org/bugs/show_bug.cgi?id=388421

At the following places you have to bump the version number:

• In all plugin.xml and feature.xml files in the corresponding org.eclipse.rmf.* plugin or features respectively. PLEASE NOTE: you have to keep the .qualifier postfix of all version numbers.
• In all pom.xml files in the corresponding org.eclipse.rmf.* plugin or features respectively. PLEASE NOTE: you have to keep the -SNAPSHOT postfix of all version numbers.
• In the rmf-pror-standalone.product file in the org.eclipse.rmf.releng.product.
• In the category.xml file in the org.eclipse.rmf.releng.repository.
• Take care of the dependencies: In the MANIFEST.MF files in the corresponding org.eclipse.rmf.* plugin or features respectively.

Finally, commit and push the bumped version numbers:

$ git commit -a -m "Bumped version numbers to 0.5.0"
$ git push

We are done! :)

Conventions

Version Numbers

We have version numbers for:

• Plugins
• Features
• Product
• Project

We have the following rules:

• While in the incubation phase, all major version numbers remain 0.
• While in the incubation phase, we will always generate qualifiers.
• Plugins: The minor and service numbers are incremented according to the Eclipse version rules.
• Features: The minor and service numbers are incremented according to the Eclipse version rules.

We have not decided yet what to do about the version number of product and project. One suggestion is:

• Create an "overarching" feature that aggregates all other features (e.g. an SDK Feature). Once such a feature is created, its version number is incremented according to the Eclipse version rules. Use the same version number for the product.

(mj) I don't quite like this, as it feels arbitrary and rather meaningless. OTOH, it would work and is kind of predictable.

Once we leave incubator phase, we will join the Eclipse release train and therefore conform the the relase train requirements. The major version number will graduate beyond zero, once we leave incubation. For now, the plugin and feature name must follow a (Incubation) mark.

GIT

• We will tag snapshots as snapshot-0.1.0-yyyymmddhhmm (yyyymmddhhmm <=> qualifier).
• Only the release manager creates tags.

Publishing Artifacts

Only snapshots artifacts will be published, both as stand-alone builds and update site builds. Nightly builds are available via Jenkins and won't be published!

Snapshot-Release Checklist: Before publishing a snapshot, update the snapshot release checklist (it is located in the dump folder in git). (HTML version)

We still have to figure out how the version number is constructed, the git tag is appended. Examples for the ProR standalone version:

• eclipse-rmf-pror-0.1.0-snapshot-yyyymmddhhmm-incubation-linux.gtk.x86.zip
• eclipse-rmf-pror-0.1.0-nightly-yyyymmddhhmm-incubation-linux.gtk.x86.zip

yyyymmddhhmm <=> qualifier

Product and Update Site Structure

Development Process

For now, we use a schedule-driven, iterative development process. An iteration is 8 weeks long and consists of the following steps:

• Review and revise the Roadmap
• Review open issues (bugs and feature requests) and decide which issues should be resolved in the upcoming iteration. Those issues will be assigned to the corresponding integration (e.g. i12.03 for the March 2012 integration).
• Developers get busy implementing.
• At the end of the iteration, a snapshot is published, both in the form of a product built and update site (Bonus: once we have a CI system, this is probably just a link to the corresponding nightly built).
• Rinse, repeat.

Download Folder Structure

Currently, we have the following download folder structure:

• rmf (root folder)
  • updates (Contains the different feature updates sites)
    • snapshot (Contains the update sites for the snapshot builds) (Since version 0.10.0 deprecated!)
      • <version number> (e.g. 0.1.0)
    • releases (Contains the update sites for the release builds)
      • <version number> (e.g. 0.1.0)
    • latest (Contains the update site for the latest build from Jenkins)
      • This folder contains a reference to [3].
    • milestones (Contains the milestone builds)
      • <version number> (e.g. 0.1.0-M1 for milestone build version 0.1.0 M1)
  • product-updates (Contains the different product updates sites) (Since version 0.10.0 deprecated!)
  • downloads (Contains all other artifacts like RCP builds) (Since version 0.10.0 deprecated!)

Since, the RMF download page is automatically build based on this structure, it is important to keep it!

More information: [4].

Release Reviews

All official Releases must have a successful Release Review before being made available for download. The purposes of a Release Review are: to summarize the accomplishments of the release, to verify that the IP Policy has been followed and all approvals have been received, to highlight any remaining quality and/or architectural issues, and to verify that the project is continuing to operate according to the Principles and Purposes of Eclipse [5]. More information about a Release Review can be found at: Development_Resources/HOWTO/Release_Reviews.

IP Log

The Development_Resources/IP_Log is an important artifact within the release review. The log tracks

• Licenses;
• Past and present committers;
• Third-party libraries; and
• Contributions from outside the project (i.e. non-committers)

The Eclipse Ecosystem provides a tool for automatically generating the IP Log from git, bugzilla, ipzilla, etc. The automatic IP Log of the RMF project is available at: [6]

To fully leverage the value of the Automated IP Log Tool, you need to:

• Keep your project metadata up-to-date
• Mark IP Contributions [7] in Bugzilla by setting the iplog tag to
  • + contains IP that was used in the code
  • - IP is not used in the code
• Follow the guidelines for handling Git contributions
• Create contribution questionnaires (CQs) where appropriate

Notes

• The IP Contribution Review Tool [8] can help finding IP issues in Bugzilla.
• To look up if a contributor has a valid Contributor License Agreement: [9]
• To check if all required CQs are available please check the content of the dowload directory: [10]

Release Engineering

Note: This is an initial PROPOSAL for the RMF build system. It is still under discussion.

Requirements

• The build system shall run on the local workstation as well as on the Eclipse build server [11]
• The build system shall create the following artifacts:
  • update site
  • standalone executables for windows, linux, mac
• The build system shall support automated
  • test execution
  • static code analysis
  • javadoc generation
• The build system shall be based on a technology that is recommended by the Eclipse foundation. This will reduce maintenance costs and it is more likely to get support and find documentation.

Build Infrastructure

In order to be able to maintain Eclipse Software for a long period of time, the Eclipse Foundation has started the Long Term Support Program LTS. Although it is acceptable to have different build systems within the Eclipse Community Build Technologies it is recommended to use the Eclipse Common Build Infrastructure CBI. From a technology point of view, the CBI is a combination of Hudson, Maven, Tycho, and build signing.

For the sake of evaluation an initial build system that is based on Tycho is now available in the development branch for the RMF repository. In order to make use of the Tycho build scripts in your local workspace please install the Maven support for Eclipse m2eclipse. In the workspace you will find the following projects that are related to building the projects

• org.eclipse.rmf.releng contains the parent pom file which is the starting point for the build. When starting the pom file, you will need to select a target platform. This is done by selecting the profile platform-juno, platform-kepler, kepler-luna or platform-mars
• org.eclipse.rmf.releng.target contains the target definition
• org.eclipse.rmf.releng.repository contains the configuration for generating a p2 repository (update-site) (the update site is created into /target/repository)
• org.eclipse.rmf.releng.product contains the configuration for generating the RMF standalone products (the products are created into /target/products)

All other features, plugins and tests are extended by a pom file. Project specific configuration should be done here. For adding new test cases to the build system, please create a test suite class that follows the naming pattern 'All*Tests.java'