Corona/Releng/Build

Eclipse Home

Wiki Home

Development

---

Build at build.eclipse.org

The Corona build process is installed on build.eclipse.org in directory /shared/technology/corona/build. But, by default there is Java 1.4 available, while Corona requires 1.5. To override default java, append to ~/.bashrc :

# java 1.5
export JAVA_ROOT="/opt/ibm/java2-ppc-50"
export JAVA_BINDIR="$JAVA_ROOT/bin"
export JAVA_HOME="$JAVA_ROOT"
export JDK_HOME="$JAVA_ROOT"
export SDK_HOME="$JAVA_ROOT"
export JRE_HOME="$JAVA_ROOT/jre"
export PATH=$JAVA_BINDIR:$PATH

Now log out and log in again. You should be able to use build as long as you are in coronaadmin group.

Installation and Configuration of Automated Build Process for Corona

The document describes how to configure and run automated builds of Corona project. The description is intended for Linux platform.

Build prerequisities

To set up the build environment it is required to have:

• Java5 visible in PATH
• Set JAVA_HOME to point at the JDK installation
• cvs, zip and unzip visible in PATH

Steps Required for Setting Headless Build Platform

Automatic Set Up by Script

Script needs update - use Manual setup for now.

A script that sets up build environment is available under CVS at /cvsroot/technology/org.eclipse.corona/build/setupBuild.sh. The script downloads all required packages and configures build environment. Tasks performed by setupBuild.sh correspond those described in Manually Setting Up Build Platform.

To set up build platform:

• get setupBuild.sh from CVS
• change setupBuild.sh file properties to executable:

  chmod u+x setupBuild.sh

• run setupBuild.sh with parameters:
  • -h build_host - folder that will be set up as Eclipse instance used to build Corona"
  • [ -c cvs_connection ] - connection type (typically "ext" or "pserver")

  ext connection type excludes anonymous user, default "anonymous"

  • [ -u cvs_user_name ] - user name used to connect to CVS server, "anonymous" is accepted for pserver

  connection only, default "pserver"

For a detailed discussion on firewall and security restrictions see Troubleshooting Network Configuration

Manual Setup

Manually Setting Up Build Platform

For headless build the infrastructure provided by the PDE and the RelEng (Release Engineering) plugins is needed.

• Import from CVS /cvsroot/technology/org.eclipse.corona/build/

  Let the directory, in which the build is run, be "build"

• chmod +x build/*.sh
• Import from CVS standalone package org.eclipse.releng.basebuilder (the latest version) org.eclipse.releng.basebuilder should be installed in "build/../org.eclipse.releng.basebuilder".
• Download and install:
  • Eclipse Platform
  • ECF 1.0 - note ECF has permissions only for owner; you can correct it by 'chmod -R +rX eclipse'
  • Equinox Bundles: org.eclipse.equinox.log, org.eclipse.equinox.event, org.eclipse.equinox.http
  • EMF XSD SDO
• Edit build/build.properties and modify:
  • baseLocation - the directory in which Eclipse platform is installed
  • base - the directory baseLocation/..
• If you want to automatically publish build results:
  • Enable ftp client in org.apache.ant plugin of org.eclipse.releng.basebuilder:
    • copy jar file commons-net-1.4.1.jar to plugins/org.apache.ant.XXX/lib
    • modify MANIFEST.MF of org.apache.ant: add commons-net-1.4.1.jar to Bundle-ClassPath
  • Configure property files of the build as described in section Adjusting Build Configuration To Your Machine earlier on this page.

Manually Setting Up Configuration for Automated Tests

To enable automated tests during build:

• Supply the following zip/tar build/../download3.3 - those are versions of projects to test with. You must rename files manually.
  • ecf-sdk-latest.zip
  • eclipse-equinox-latest.zip
  • eclipse-test-framework-latest.zip
  • eclipse-sdk-latest-linux-gtk.tar.gz
  • emf-sdo-xsd-sdk-latest.zip

Adjusting Build Configuration To Your Machine

Despite automated configuration of most issues, it is necessary to edit property files to adjust the build to machine specific settings. The following configuration changes should be made:

1. Edit build.properties
   • myFetchTag - leave HEAD if you want to build current development version or change to CVS tag/branch you want to build.
   • eclipseCvsAccess - if you want to change access to CVS repository in maps, set what should be placed instead of ":pserver:anonymous@dev.eclipse.org:". If build.eclipse.org the property is empty, which allows direct local access to CVS, without networking. Not set will use default.
2. Edit and change build/build.properties file (only if you want to automatically publish build):
   • ftpServer - IP of ftp server to copy build results
   • ftpUser, ftpPassword, remoteDirectory - connection parameters on the remote server
3. Edit and change updater.corona.properties file for RSS generation
   • branch - the number of branch that is build
   • buildAlias - branch number and name

Configuring Download Side

Results of each build should be propagated to an http server under DocumentRoot/corona/drops/buildId subdirectory. The folder DocumentRoot/corona must contain the files: build_typese.html, dlconfig.txt and index.php. The files are available in plugin org.eclipse.releng.basebuilder under subfolder builderTemplate\publishingFiles\dropSiteRootFiles\

Troubleshooting Network Configuration

Network access problems.

The setupBuild script uses two types of transport to fetch required resources: cvs and http. Depending on security settings enabled by your system administrator, one or both of there connections can be restricted. Typical solutions are:

• for www access: define http_proxy system variable
• for cvs access: check which access method is available. cvs can connect to a server using either ext (i.e. ssh protocol) or pserver. pserver access can be used if and only if the user is anonymous. ssh access requires a valid user and a password.

Execution of Build Scripts

Now the build is configured on build.eclipse.org. To run the build:

• log in to the machine
• cd to /shared/technology/corona/build folder
• if you want to build on a specified tag, update the myFetchTag property to the desired tag / branch in the build.properties; for trunk, set HEAD
• execute the main build script, the script is:
  • cleanBuild.sh X - X denotes the type of build, that is X = M, I, N, S, runs the build, tests, generates html data and published build results

for example, to execute a maintenace build, call ./cleanBuild.sh M

• buildFeed.sh - generates and publishes RSS Feed

Operations Done During Build Phases

---

Execution of buildCorona script comprises:

• cleaning previous build output
• fetching build scripts and properties required by the build process such as:

1. run.bat/run.sh - scripts starting an Eclipse instance as AntRunner
2. buildCorona.xml - build script file controling the build sequence
3. customTargets.xml - build script file enabling customization of the build process, defines the map files to be fetched and the features to be built
4. build.properties - contains the properties needed for fetching and building the sources

• executing run.bat/run.sh script

---

Build Phases

Build Plug-ins (buildAll task)

buildAll task is responsible for building Corona's plug-ins and creating Corona's features.
The most important tasks from this build steps will be described below.

• getMapFiles task (customTarget.xml).

This task is responsible for retrieving corona's map files into local build directory. Map files describe the location of corona's plug-in's and features. Location of map files is specified in build.properties file in following section:

 #skipMaps=true
 mapsRepo=:pserver:anonymous@206.191.52.50:/cvsroot/technology
 mapsRoot=org.eclipse.corona/maps
 mapsCheckoutTag=HEAD

• allElementsDelegator task(allElements.xml).

  This task consists following subTasks:

  • allElements.org.eclipse.corona.client

    Responsible for retrieving and building org.eclipse.corona.client feature.

  • allElements.org.eclipse.corona.server

    Responsible for retrieving and building org.eclipse.corona.server feature.

  • allElements.org.eclipse.corona.doc-feature

    Responsible for retrieving and creating Corona's documentation feature.

  • allElements.org.eclipse.corona.test.client

    Responsible for retrieving and creating Corona's client test feature.

  • allElements.org.eclipse.corona.test.server

    Responsible for retrieving and creating Corona's server test feature.

  If you'd like to add new feature please have in mind following things

  • Create/update proper map file located in org.eclipse.corona/maps

    feature@org.eclipse.corona.<feature name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify feature location>

    plugin@org.eclipse.corona.<plugin name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify plugin location>

    plugin@org.eclipse.corona.<plugin name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify plugin location>

  • Add new section within allElements.xml file:
    • in target allElementsDelegator

      <antcall target="allElements.org.eclipse.corona.<feature name>"/>

    • create new target allElements.org.eclipse.corona.<feature name>

      <target name="allElements.org.eclipse.corona.<feature name>">

        <ant antfile="${genericTargets}" target="${target}" >

          <property name="type" value="feature" />

          <property name="id" value="org.eclipse.corona.<feature name>" />

        </ant>

      </target>

    • create new target assemble.org.eclipse.corona.<feature name>

        <target name="assemble.org.eclipse.corona.client">

          <ant antfile="${assembleScriptName}" dir="${buildDirectory}"/>

        </target>

Publish Plug-ins (publish task)

Important tasks/files for publishing corona to the web site:

index.html generation

• <build directory>/testManifest.xml

  The file contain two types of top level elements:

  • <zipType>

    Example

      <zipType name="zipTypeName">

        <platform id="ID" name="applicable platforms" fileName='<a href="file.zip">file.zip</a>'></platform>

      </zipType>

    • zipType is a category of binary distributable element.

      Zip type name should be an unique identifier. It will be used in index.html.template file as a placeholder where platform information will be put.

    • platform is a description of a platform-specific or non-platform specific binary distributable.

      id - unique identifier for the platform relevant to the zip file referenced in fileName attribute. Used to map compile logs and test logs listed in the manifest to a specific platform of a zipType.

      name - text that should appear on download page to describe where the binary can be installed, ie. "ALL" or "Windows 98/ME/2000/XP".

      fileName - html source inserted on download index page to produce link to the distributable.

  • <logFile>

    Example

      <logFile name="pluginname/myjar.bin.log">

        <effectedFile id="ID"></effectedFile>

      </logFile>

    • logFile - maps compile logs and JUnit test results to a given download.

      effectedFile - references the value of the id attribute for a platform element. Used to mark a <platform> with a status depending on the log content.

• <build directory>/templateFiles/index.html.template

  The template for creating index.html page, this file should contain placeHolders for rows of links to platform-specific and/or platform-independant runtimes for a given download type. Place holders are defined in testManifext.xml file as a zipType name attribute and should be also defined in buildAll.xml file, in publish task as a dropTokenList property.

  <property name="dropTokenList" value="%sdk%,%runtime%,%examples%,%tests%" />

• <build directory>/templateFiles/index.php.template
• <build directory>/templateFiles/testResults.php.template

Build Corona Documentation Plugin

Corona's build process is able to build corona's documentation feature, following prerequisites are required:

• Linux machine.
• Installed ANT build tool.
• JAVA_HOME environment variable pointing to Java home directory e.g. 'export JAVA_HOME=/etc/jdk1.5.0_09'.
• ANT_HOME environment variable pointing to ANT home directory e.g. 'export ANT_HOME=/etc/apache-ant-1.6.5'.
• javaDoc property set in build.properties file, as follows

 # Default for java doc generation.
 javaDoc=true

See in HINTS section how to generate JavaDoc without a documentation plugin

Stages of build process

• preBuild - fetching and concatenating map files (in corona.map dynamic setting of cvs tag name is made on the basis of fetchTag property value defined inside build.properties)
• fetch - retrieving project resources on the basis of the map files definition
• generate - generating build scripts for fetched resources
• process - compiling sources, coping results to build output directory or creating jar files containing the build output
• assemble - creating zip archive containg all build results

Useful links concerning eclipse build process

• Build and Test Automation for plug-ins and features
• Automating Builds with PDE BUILD
• RelEng FAQ

Hints

JavaDoc generation

Corona's JavaDoc always will be generated and integrated within documentation plugin. There is a possibility to generate javadoc without documentation plugin. Following prerequisites has to be set.

• LINUX
  • Installed ANT.
  • JAVA_HOME environment variable pointing to Java home directory.
  • ANT_HOME envitonment variable pointing to ANT home directory.
  • Documentation plugin should be on the same level as other corona's plugins

To start generation run script antJavadoc.sh located in org.eclipse.corona.doc\build.

• WINDOWS
  • Installed cygwin.
  • JAVA_HOME environment variable pointing to Java home directory.
  • ANT_HOME envitonment variable pointing to ANT home directory.
  • Documentation plugin should be on the same level as other corona's plugins

To start generation run under cygwin script antJavadoc.sh located in org.eclipse.corona.doc\build.