Build and Release Process

Daily Builds & Tests

Overall, seems a bit too complicated. Perhaps we can push the edge of the tools available to us without creating new ones. That would seem like it would cost less. Additional comments below. As of now, do not approved.

Build Contents

• Each new build differs from its predecessor by a set of discrete checkins, each checkin represented by a bugzilla.

Q: Do we want to mark the CVS checked in files with the related bugzilla number by including the bugzilla number in the CVS comment or something similar?

• Each checkin may involve one or many files, and is composed of a specific set of files at specific versions (listed in the bugzilla?)
• The checkins can have overlapping changes with other checkins, but they are discrete in that they are applied serially such that they can be undone by just reverting to a previous version (by backing out checkins) of all the files involved in the checkin.
• The set of checkins in any given build will be listed on the web page (or wiki) as a description of the build.
• By adding a comment containing the build label to the relevant bugzillas a cross reference of bugzilla to build and build to bugzillas can be maintained.

Q: I assume that the daily build is based on the CVS-HEAD. Do we plan to provide "fix"-builds for earlier releases? For example provide a 0.3.2 build during the 0.4 development?

Q: Does CVS support this sort of fine grain control between bugzilla and the source repository? I'm not sure it does. In which case, this level of granular tracking will require some sort of tool or website that allows or forces developers to attach every update to bugzilla and then attach every bugzilla to the resulting CVS label. That's hard.

Ownership

• Anyone who ever checks anything in (that is all committers) are generally responsible for clean builds and tests.
• Clean daily builds and tests keep the system stable, prevent regressions, and avoid down time.
• Anytime you check something in you are specifically responsible for monitoring the builds and tests until they pass.
• That means that for at least 24 hours after your checkin you are available, responsive and prepared to help with any issues that crop up.

Stewardship

• On a rotating basis some human who can access the build and test machines, and is an Aperi committer, will be responsible for seeing to it that the builds and tests run to completion.
• In the event of problems with the build & test system itself they are expected to resolve them as soon as possible and get the build and test restarted
• In the event of problems with the build caused by developer checkins they are expected to follow up with developers to ensure fixes are made quickly, and then get the build and test restarted.
• Any build failure will be reported as a bugzilla to the aperi.build component for tracking of the actual issue as well as trend analysis, etc.

Q: will a bugzilla be opened for each problem?
A: I think the right answer is 'yes' to keep things nice and public and provide a DB of issues so we can analyze for stats and trends and such. We already have a component for aperi.build I think.

Nightly Builds

• Every 24 hours at GMT-8 10:00 PM (PDT) the state of the HEAD branch of the CVS repository will be labeled with a time stamp
• The time stamp will be encoded with the release number, date, time, and serial build number: Aperi-RMm-YYMMDD-HHMMSS
• A module list will be maintained (in CVS) that list all the modules that are part of the current build, labeling and extracting will be restricted to those modules. This could be a PDE build style map file or .pfd file. This map file will be named Aperi-RM.m.{map|pfd} and will be labeled with the time stamp at build time as well.

Q: Perhaps we should start with end user requirements. These are design suggestions. Why is labeling restricted? Right now, the org.eclipse.aperi module is labelled in its entirety, including some plugins that are neither extracted nor built. We have a .psf file listing all of the included components. I suppose this can be included in the build with a label on it. Is there a use for it?

• The source will be extracted based on that time stamp and the system will be built.
• Build results (install files, and any other artifacts) will be archived, but will not be uploaded to the Eclipse download site (Q: Does this mean they will be archived in cvs? Good question - archived where?)
• The build artifacts will have the build time stamp embedded in the name
• The GUI ’about’ and server version #’s will have the build time stamp embedded
• Every runtime log file should have the build time stamp embedded in it as well

Q: Again, Is there a use for it? Is this worth the development effort?

Nightly Tests

• If/when the system is successfully built the install bundles will be automatically copied to prepared test systems
• The prepared systems will automatically run a test install and minimal acceptance test
• One each for Derby and DB2 on each platform Windows and Linux
• Each test Install/run should be less than 60 minutes to keep turnaround time reasonable

Q: Does this suggest a junit test framework. This would be ideal and appropriate. However, this would endure significant development costs. Such an item, in my mind is worth it and very useful for adopters. However, it costs.

Build and/or Test Results

• Results of the nightly build will be emailed to a new email distribution list ’aperi-builds@eclipse.org’
• The email will indicate the time stamp, and the SUCCESS or FAIL of the build
• The tail of the build log or the portion of the build log that contains build failures will be embedded in the body of the email.
• Results of the nightly tests will be emailed separately, one for each test system, to ’aperi-builds@eclipse.org’
• The email will indicate the time stamp, and the SUCCESS or FAIL of test
• Upon failure a zip file of the relevant logs will be attached to the email.
• In addition all results, and logs will be posted on a public web/wiki for your browsing convenience.
• Checkins, other than to fix broken builds and tests, are frozen until the builds and tests pass
• Check the results of the nightly runs before checking in

(Comment: I think the new mailing list is a good idea)

Weekly Promotion to ’Good Build’ & Upload to Eclipse

• Once a week the steward will pick a build towards the end of the week (Thursday evening/Friday morning typically) that has built clean and passed the minimum acceptance tests to promote to ’good build’.
• The good build will relabeled in CVS with the ’Aperi-RM.n-GOOD’ label. There is only one such label. It just gets moved for each promotion.
• Thus, a casual user can easily find the source for the latest good build.
• The build artifacts of the promoted build will be uploaded to the Aperi download site and the ’Latest Good Build’ link updated.
• Thus, a casual user can easily find the install bundle for the latest good build

Milestone Releases (MS)

• Spaced out more or less evenly across the release schedule will be milestone releases.
• Milestone releases contain complete DCUT (code complete, unit tested) increments of functionality (they are a subset of the full functionality of the target release).
• Milestone releases are number RVM.m.MS1 through RVM.n.MSn (R0.4.MS2 for example).
• MSn is the last milestone release and as such contains all the functions (albeit largely untested) targeted for the full release.
• A milestone release is created simply by relabeling and renaming a GOOD build, updating the downloads page and sending out an email saying such and such MS is available. The contents of the MS are determined by the project schedule.
• Thus, a casual user can look at the download page and find latest milestone release

Release Candidates (RC)

• In the end game of the project we will reach a point where all test have been run, bugs fixed, and the builds are stable
• At that time we will promote a GOOD build to a release candidate build
• Release Candidates are number RVM.m.RC1 through RVM.n.RCn (R0.4.RC7 for example).
• The procedure for creating a release candidate is pretty much the same as for milestone releases: relabeling and renaming a GOOD build, updating the downloads page and sending out an email saying such and such release candidate is available.

Q: Would the name GA candidate be less confusing?
A: I think Release Candidate (RC) is common Eclipse nomenclature.

(Comment: I personally just like Release candidate..i don't really like GA for open source since everything is available all the time. )

Q: What about the process for documenting what is in each milestone and the release candidate? Where will post that?
A: I think that should go here on this page as well (will add)

Q: I'd like to see some tie of the build process to the IP Log approvals etc..so we ensure we are including only what is approved in the IP Log in the builds. Any ideas on how to do this?
A: Good point, but I don't know how to do it other than manually. It is the case that Tom is opening bugzillas for each approval, so that bugzilla would be listed in the build contents.

GA Builds (GA)

• We haven’t yet decided what the criteria is to promote a release candidate to GA, but when we do the procedure will go like this:
• The procedure for creating a GA is pretty much the same as for milestone releases: relabeling and renaming an RC, updating the downloads page and sending out an email saying such and such GA is available.

Process approved at the Aperi Development Status Meeting 06/25/07; Tom Guinane