STP/Build 101

Introduction

This page is meant to serve as a general overview to how the STP build system works, what each of the directories you see are for, and how to go about making the most commonly required updates.

Getting started

Firstly, you'll need to checkout the build system. Typically, you should just able to check out the latest version of the build:

cvs -d :pserver:anonymous@dev.eclipse.org/cvsroot/stp co build

The only thing that you can change which will influence the build at this point is the stuff in the cruise directory. The releng / releng.stpbuilder give you an indication of what will happen on the build, but changing them at this point will not result in any changes to the build that will be executed. Why? Well the first thing that happens when you run the build from the cruise directory is that it will proceed to checkout a specific version of releng / releng.stpbuilder from CVS, and then begin the build from that directory. So if you're changing any files in those directories, commit first, then build.

OK, so what do these directories actually do, and why would you need to change them. Lets now start to look at this in a little more detail. After checkout of build, you get a relatively simple looking directory structure as follows:

 build
        cruise
        releng
        releng.stpbuilder

So, what are these directories for then? Let's start with the cruise directory....

The cruise directory

The cruise directory is where you'll go to do a build. It's default action is build / test everything, created deployment packages, update sites, etc, and rsync everything over to the main eclipse development machines so it's available for download. Don't just build after checkout though, it will most likely fail.

Start by looking at the build.properties file. You'll need to change the following in it:

build.home= the location that you've checkout the build directory into

mapVersionTag= The version of the build system that you'd like to checkout. Current 3.3 builds are tagged with STP_33_M? where ? represents the build number.
mapCvsRoot=where cvsroot is, e.g. :pserver:anonymous@dev.eclipse.org/cvsroot/stp

Now define the type of build you wish to do:

buildType=I (I for integration, S for stream stable)

And change the following to suit the platform that you're building STP on:

baseos=linux
basews=gtk
basearch=x86

So, before running ant, you'll need to set up some classpath stuff. export J2SE15=${JAVA_5_HOME}/jre/lib/rt.jar

Yip, you need Java 1.5 to build STP.

Now run ant, sit back and wait :-)

The releng directory

The releng directory contains descriptions for all the main components that are part of the STP. When you add new features, plugins, etc, you'll be working with the contents of this directory. In releng you'll see a maps directory. In here you'll see map files for:

* All the features that are required to be built (e.g.  stp-bpmn.map )
* All the plugins corresponding to these features  (e.g. stp-bpmn-plugins.map)
* A dependency.properties file that is used to describe what are required to build the various plugins.

The releng.stpbuilder directory

The releng.stpbuilder contains all the build infrastructure pieces to build the stp. Here you will find:

components - custom PDE based build targets here for each component that's part of the stp. In the dependency.properties file you may need to update it if you are placing additional dependencies for the component. Here, all dependencies are extraced. If you have any custom build rules that need to be executed, here the customTargets.xml is what you should be looking towards.

scripts - Scripts that generate the update site, download pages, and execute the copy over to dev.eclipse.org.

distribution Contain some template files, etc. that will form the content of the download page.

NB: The build.properties file in the root generally requires updating as we change eclipse milestons. The launcher that is used to eclipse needs to be specified correctly in here. Symptons that mean you haven't updated this file correctly are that the build fails in the test phase.

Directories that get created during the build

builders: this is the builder that is used to launch all eclipse processes to generate build.xml's for compilation. 
build-node: a checkout of the build directory, the version of which is specified in your original build.properties file in the cruise directory.
build-stp-I: here you'll find deep, deep down the built plugins (in workdir) and the eclipse platform used to build them (eclipse).

Common Tasks

To migrate through the variuos europa builds, you'll need to update the depdency.properties file in releng/maps. This is reasonably well documented. Remember to update the luancher information both there, and in releng.stpbuilder as well. The biggest pain here is correctly identifying all the download sites.

Update Sites

The Stp update site url is http://download.eclipse.org/stp/updates/site.xml To modify the update site

• ssh to build.eclipse.org with your committer account.
• cd ~/downloads/stp/updates
• modify the site.xml file

Notice:There is one associateSites.xml file under the same dir. which is point to Apache CXF and Apache Tuscany update sites. So when user want to install stp from update site, it will also install org.apache.cxf and org.apache.tuscany plugins automatically

STP download Site

Steps to update the stp download site.

• login to build.eclipse.org
• build and upload the new drop according to steps above
• copy the new drop dir from ~/downloads/stp/committers/drops to ~/downloads/stp/downloads/drops
• wait for about 20 mins for the website to get synced.

Cruise Control

CruiseControl is a framework for a continuous build process. It includes, but is not limited to, plugins for email notification, Ant, and various source control tools. A web interface is provided to view the details of the current and previous builds.

The following are the steps to establish a Cruise Control for stp.The directory that you've checkout the build directory into will be referred to as BUILD_HOME.(Please refer to "Getting started" above.)

• 1. Download CruiseControl 2.6.2 source distribution

http://downloads.sourceforge.net/cruisecontrol/cruisecontrol-src-2.6.2.zip?modtime=1177629239&big_mirror=0

Unzip the downloaded file to your applications directory. The top-level directory created when you unzipped this file will be referred to as INSTALL_DIR.

• 2. Build CruiseControl

To build the CruiseControl you will need to execute the build scripts in %INSTALL_DIR%/main

cd %INSTALL_DIR%/main 
./build.sh

• 3. Build reporting web application

To build the reporting web application you will need to execute the build scripts in %INSTALL_DIR%/reporting/jsp

cd %INSTALL_DIR%/reporting/jsp 
./build.sh -Duser.log.dir=%BUILD_HOME%/build/cruise/logs -Duser.build.status.file=buildstatus.txt -Dcruise.build.artifacts.dir=%BUILD_HOME%/artifacts

The configured cruisecontrol.war file will be created in the directory: %INSTALL_DIR%/reporting/jsp/dist.

If you want to get the build artifacts link working in the web, you should add an <artifactspublisher> element to the <publishers> element of your config.xml located in %BUILD_HOME%/build/cruise. For example:

<artifactspublisher
  dir="${build.home}/build/build-stp-I/workdir"
  dest="${build.home}/artifacts/${project.name}"/>

• 4. Download Tomcat

http://tomcat.apache.org/

Unzip the downloaded file to your applications directory. The top-level directory created when you unzipped this file will be referred to as TOMCAT_HOME.

• 5. Deploy reporting web application on Tomcat

Copy the cruisecontrol.war file created in stpe 3 from the %INSTALL_DIR%/reporting/jsp/dist directory to the %TOMCAT_HOME%/webapps directory.

cp %INSTALL_DIR%/reporting/jsp/dist/cruisecontrol.war %TOMCAT_HOME%/webapps

• 6. Set properties

Most of stp specific properties for Cruise Control are defined in the file "%BUILD_HOME%/build/cruise/cruise.properties". See below:

*build.home: the location you've checkout the build directory into. Should be the same as %BUILD_HOME%.
*cvs.cvsroot: the location of the CVS repository.
*cvs.module: the name of the CVS module used for check modifications. 
*time.interval: the interval period in seconds for cruise to restart build process.
*time.quietperiod: the number of seconds required to have passed since the last modification before a build can commence.
*build.result.url: the url for cruise build result. (e.g. http://localhost:8080)
*build.buildmaster.email: the email address that will be used to send build notification email.
*mail.host: the smtp server that will send the notification email.
*subject.prefix: the prefix for the subject of notification email.

• 7. Start tomcat

cd %TOMCAT_HOME%/bin
./startup.sh

Port 8080 will be used as the default http port for tomcat.

• 9. Start cruise control.

cd %BUILD_HOME%/build/cruise
./cc.sh

Port 8000 will be used as the default http port for JMX. If you want to change this port, you should make sure that the jmx port defined in %BUILD_HOME%/build/cruise/cc.sh should be the same as the one defined in %TOMCAT_HOME%/webapps/cruisecontrol/WEB-INF/web.xml. Otherwise, you can not access JMX control panel through the tomcat web server.

• 10. For more detailed informations about CruiseControl, please refer to the docs in the location:

%INSTALL_DIR%/docs