MoDisco/Releng/How to Use

The MoDisco project is built using Buckminster, with this releng project. (for old Athena build, see this page)

For more explanation about how the build process works, see also How it Works?.

What is built?

org.eclipse.gmt.modisco.all is the feature that is built (defined in buckminster.cspec). This feature must include all other features that must be built.

The releng project contains a Buckminster rmap, which specifies how to get the plugins and features that are to be built. For each plugin or feature, the map defines its location on a version control system (CVS, SVN), and the version that must be used.

How to start a build?

Automatically

• Nightly builds are run every 6 hours everyday, at 02:10, 08:10, 14:10 and 20:10 EST, if the MoDisco SVN changed since the last build.

• Integration builds are started manually.

Manually

• Only committers in the MoDisco project, and members of the Callisto-dev group can launch MoDisco build jobs from Hudson.
• Nightly build : go to https://hudson.eclipse.org/hudson/job/modisco-nightly/
• Integration build : go to https://hudson.eclipse.org/hudson/job/modisco-integration/

Then:

• In Hudson, click on Build Now, change the build parameters as needed (see #Build parameters), and click on Build.
• You can then click on the job name in the Build History section in the left column, and then on Console Output, to follow build progress in real time.

How to publish a build?

Automatically

Successful Nightly and Integration builds are automatically published to download.eclipse.org. For example, a nightly build for version 0.8.0, created on October 6 2009 at 11:54 EST would be published to:

http://download.eclipse.org/modeling/mdt/modisco/downloads/drops/0.8.0/N200910061154

These builds can then be seen and downloaded from http://www.eclipse.org/MoDisco/downloads/, where additional information is available (test results, build log).

With a script

Run /opt/public/modeling/mdt/modisco/manualPromote.sh and fill in the parameters.

Manually

Stable, Maintenance and Release builds are not automatically published. They should be first tested internally before publishing. For example, to publish the 0.8.0M2 milestone build:

• First, fetch the build archive to test (from Hudson, or using wget or scp for example).
• Then, test the build locally
• Then, publish it:

ssh <commiterid>@build.eclipse.org
wget https://hudson.eclipse.org/hudson/job/modisco-integration/113/artifact/S201009291117.zip
unzip S201009291117.zip -d /home/data/httpd/download.eclipse.org/modeling/mdt/modisco/downloads/drops/0.9.0/

Add a new update site with the new build to the composite:

• unzip the content in a new folder with the release number:

unzip MODISCO-Update-0.8.0M2.zip -d /home/data/httpd/download.eclipse.org/modeling/mdt/modisco/updates/milestones/0.10/M3

• update compositeContent.xml and compositeArtifacts.xml to add a reference to your newly added update site
  • set the value of p2.timestamp to the result of "date +%s000"
• You can enable download stats on the repository by running addDownloadStats.sh on the update site
• Check that the new build appears on http://www.eclipse.org/MoDisco/downloads/.
• Builds can be hidden from this page before a release by modifying downloads-scripts.php in www/MoDisco on :pserver:anonymous@dev.eclipse.org:/cvsroot/org.eclipse
• Update the archive site with the new build:
  • copy the drop folder (eg. "S200910061155") into /home/data/httpd/archive.eclipse.org/modeling/mdt/modisco/downloads/drops/x.y.z
  • update /home/data/httpd/archive.eclipse.org/modeling/mdt/modisco/downloads/index.html with a link to the newly added update zip
• Tag the build
• Wipe out the Hudson workspace to free up room on the server.

Simultaneous Release

If the build must be part of the simultaneous release, you must also:

• Use the B3 Aggregator (or a text editor if the modification is trivial) to update the build model for the aggregator (after checking it out from CVS).
• Open juno.b3aggr in the B3 Aggregator editor, right-click on the first root node and select Validate Aggregation. Check in the error log that there are no errors.

Build parameters

Hudson builds expect these parameters:

• BUILDTYPE : the kind of build, represented by a code letter (see this page):
  • N: Nightly
  • I: Integration
  • M: Maintenance (NOT milestone)
  • S: Stable (for Milestones and Release Candidate builds)
  • R: Release
• BUILD_ALIAS : the name of the release (0.8.0, 0.9.0M1, etc.); blank for nightly and integration builds
• SIGN_UPDATE_SITE : whether to sign the update site (takes about an hour on the Eclipse build server); mandatory for releases
• VERSION : the version being built. Dictates in which folder the build will be placed under the download "drops" folder.
• PROJRELENGROOT : The path on the SVN (trunk, branch or tag) from which to take the version of the releng project that must be used for this build.

Building locally

TODO

Tagging

To tag a release, use this script: tag.sh.
Releases should have a tag like R0_7_1, and milestones should have a tag like S0_9_0_M2.

Tag names
Since the tag name is used as a bundle qualifier by PDE Build, it should not contain periods. For example, use R0_7_1 instead of v0.7.1

Checking

Check bundles

Check that each bundle contains an about.html file:

for f in $( ls *.jar ); do unzip -t $f | grep -q about.html || echo $f; done

or for plug-ins in your workspace:

find $workspaceRoot -name 'build.properties' | while read i; do grep -q about.html "$i" || echo "$i"; done

Check licenses

find $workspaceRoot -type f -name 'license.html' | while read i; do md5 $i; done

Check Manifest.MF

find $workspaceRoot -name 'MANIFEST.MF' | while read i; do grep -q Bundle-Vendor "$i" || echo "$i"; done
find $workspaceRoot -name 'feature.xml' | while read i; do grep -q provider-name "$i" || echo "$i"; done

Check simrel reports

for url in \
   "http://build.eclipse.org/juno/simrel/reports/layoutCheck.txt" \
   "http://build.eclipse.org/juno/simrel/reports/verifydiroutput/unsigned.txt" \
   "http://build.eclipse.org/juno/simrel/reports/versionPatternCheck.txt" \
   "http://build.eclipse.org/juno/simrel/reports/breedata.txt" \
   "http://build.eclipse.org/juno/simrel/reports/pack200data.txt" \
   "http://build.eclipse.org/juno/simrel/reports/nonUniqueVersions.txt"
do
  echo "checking $url"
  curl -s -S "$url" | egrep 'modisco|facet'
done

url="http://build.eclipse.org/juno/simrel/reports/licenseConsistency.html"
echo "checking $url"
curl -s -S $url > licenseConsistency
n=$(cat licenseConsistency | grep -n "Features with matching" | sed 's/:.*//')
cat licenseConsistency | head -n $n | egrep 'modisco|facet'

url="http://build.eclipse.org/juno/simrel/reports/featureNames.html"
echo "checking $url"
curl -s -S $url > featureNames
n=$(cat featureNames | grep -n "Probably correct names" | sed 's/:.*//')
cat featureNames | head -n $n | egrep 'modisco|facet'

url="http://build.eclipse.org/juno/simrel/reports/bundleNames.html"
echo "checking $url"
curl -s -S $url > bundleNames
n=$(cat bundleNames | grep -n "Probably correct bundle name" | sed 's/:.*//')
cat bundleNames | head -n $n | egrep 'modisco|facet'

url="http://build.eclipse.org/juno/simrel/reports/providerNames.html"
echo "checking $url"
curl -s -S $url > providerNames
n=$(cat providerNames | grep -n "Probably using correct provider name" | sed 's/:.*//')
cat providerNames | head -n $n | egrep 'modisco|facet'

url="http://build.eclipse.org/juno/simrel/reports/copyrights.html"
echo "checking $url"
curl -s -S $url > copyrights
n=$(cat copyrights | grep -n "Features with copyrights that are probably ok" | sed 's/:.*//')
cat copyrights | head -n $n | egrep 'modisco|facet'

url="http://build.eclipse.org/juno/simrel/reports/esdata.txt"
echo "checking $url"
curl -s -S $url > esdata
n=$(cat esdata | grep -n "Bundles without an Eclipse-SourceReference" | sed 's/:.*//')
cat esdata | tail -n +$n | egrep 'modisco|facet'