Lightning Retrospective Guide

This page provides both a sample of a good retrospective and a natural place to add your retrospective for the Europa Build Workshop. You won't need to make slides to present at the workshop. We'll provide access to these pages and a big font.

(your project name here)

Describe yourselves, your project and your relationship to other developers on the team. Consider describing ...

• years of experience in development, doing release engineering, and specifically building Eclipse projects.
• how you work, on a good day, or on a not so good day. how does your work change as you ramp-down toward release.
• what automation you use and how often you change it. how late in a project you would consider automation changes and how you would know that they work.

Describe your Callisto experience. Consider describing ...

• special adjustments you made because of your participation in the simultanious release.
• tactical communication regarding inbound and outbound bits.
• strategic communication with respect to the Callisto planning team.

Describe your hopes and ambitions for Europa. Consider describing ...

• facilities you wish you had and how they might work and who you would trust to build them.
• precedures that would make your life less stressful or more efficient. also when and how you would escape those precedures in an emergency.
• communication mechanisms that would support both routine and exceptional circumstances that are sure to arise as more projects join the release train.

Eclipse Foundation / IT

I'd like to offer a retrospective from the Foundation's point-of-view, as we were heavily involved in (and impacted by) the actual release. These are some observations we wrote down:

• Having all the mirrors ready for the release saved us tens of thousands of dollars in bandwidth and server resources.

• The last minute builds introduced big delays. How can we avoid this?

• Communication: we need key contacts for each project (perhaps two) with Instant Messenger and cell phone access.

• We need a way to test the download experience from a user's point-of-view before going live. Wayne was able to wing this with Callisto, but a better, broader process is needed.

• Birt: the Birt build team went MIA for a while, which introduced delays.

• How to withdraw a project: what happens if, for some reason, a project cannot commit to the release at the very last minute?

• Backup for final build/release: we had a hard time reaching David Williams during the final Daze, as he was stuck on a plane. No one else knew what magic he needed to perform to promote everything to a state of readiness.

• Releasing on a friday afternoon before a long weekend (in Canada/USA) is boring (IMHO)
  • Agree that friday afternoon is not a good time, it is Saturday in Shanghai, we have to work on weekends for release.--Sue Lee

Eclipse Top-Level Project (Platform, JDT, PDE & Equinox)

Platform-releng-europa-blurb

EMF, UML2, EMFT, GEF

Build Retrospective: EMF, UML2, EMFT, GEF

BIRT

I have been BIRT build & release engineer for a year. Another member of BIRT release team is Xiaoying Gu. She joined the team half a year ago. We both have no build experience and Eclipse project experience before.

BIRT haven't use PDE build to build BIRT. We have our own build framework, written in Ant. We write build script for each BIRT plugin, and it will be invoked by top-level build script. Plugin build script is maintained by plugin owner. Top-level build scripts are maintained by build team. The build system usually works well. But there are three shortages in our build system:

• A gap between development environment and build environment. Developers works in Eclipse workbench, but we build under Ant. There is the gap naturally. It causes many troubles, so we are working hard to get rid of the gap by “Eclipse Way”.
• Low efficiency. It will take 3 hours to make one build before, now 2 hours, but we still want to make it faster.
• Hard to build BIRT. Top-level build scripts are not checked into CVS because the build framework is not designed for open source to some extent.

We have close relationship with other developers on the team. Following are our responsibilities:

• Top priority task: provide daily build to QAs and developers. Without it they can't begin their work.
• Help developers when they need to add a new plugin to BIRT daily build, help them to solve plugin build script issue. Run test builds to make sure the changes will not break next build.
• Maintain feature structure
• Coordinate cross team build tasks
• Monitor rolling build, once there is a compilation error, notify developer to fix it, thus avoid to break next day's daily build
• Run regression test after daily build is ready
• Optimize our build framework

Callisto experience

• Use manifest in BIRT plugin
• Convert folder plugin to single-jar plugin
• Create Eclipse features based on functionality, hence improve distribution and installation of BIRT
• Plugin versioning: use 4 digit build number; specify ranges when requiring plugins; upgrade version number only when the plugin has a change

There were many adjustments because of participation in the simultanious release. We really learned a lot of things from Callisto. Here I must apologize for MIA in Callisto final release. I made a big mistake on release time. I confused by the time difference and took for June 30th (Shanghai time) as the release date. And at the same time we met uploading issue that day. The real problem is I didn't seek appropriate help from appropriate person, and didn't notify Callisto release team about it. It is the worst day since I became a build engineer. You can image when you open your eyes, there are so many emergency emails and calls from HQ. Oh, it's too bad. So I promise it will never happen again.

Europa Ambitions

• Switch to PDE build to build BIRT. This means we will rewrite our build system. And the whole build system will be available in CVS. It is a big challenge for us. Hope to learn the best practices for PDE build.

Facilities I wish we had

• Efficient and stable upload tool
• Web page for new eclipse projects on suggested best practices for build
  • Ward, this is an interesting discussion topic for the committer community (if not for Europa), as lots of projects ask webmaster about it. --Denis.roy.eclipse.org 22:10, 9 September 2006 (EDT)

TPTP

TPTP uses PDE build with extensions to support the requirements of the project.

The build for this project is unique is several ways:

• It's a large project with 68 committers and 129 plugins.
• We also have components written in C/C++ and can execute outside the eclipse environment. The native code is supported on over 10 platforms.
• The build is done in collaboration between IBM and Intel.
• TPTP is dependent on many other eclipse projects. Projects that we directly or indirectly depend on include EMF, WTP, BIRT, GEF, JEM and AspectJ.
• TPTP also has non-eclipse 3rd party dependencies: Xerces, commons logging, log4j, Batik, Muse, mx4j

Challenges of the build team include:

• Builds break when we most badly need a good build, because developers tends to check in code near the end of a development iteration. When too many changes happen at the same time, problems occur.
• Some new features require changes to the build scripts, but the requirements are identified late in the cycle, not giving the build team enough time to react.

Challenges specific to the Callisto release:

• Implementing the new version numbering guideline
• Sharing common 3rd party plugins with other eclipse projects, and ensuring these shared plugins are exactly the same in different projects
• The eclispe updater doesn't behave the way I expect. Sometimes it is difficult to tell if it is an updater bug or it is the problem with my site setup.

Ambitions for Europa:

• Use the Orbit project to solve the common 3rd party plugins sharing problem

Buckminster

The founders of Buckminster are all senior developers with +10 years of release engineering expericence. We've learned that patch frameworks and ad-hoc solutions are very common in this domain and often a great source of frustration. We want to provide a framework that will encourage you to "do it right" from the beginning.

We would like to present Buckminster and two use cases.

Our hope is that we will be able to discuss further requirements that would make Buckminster a truly useful tool for the Europa release, and if possible, entice others to help us fill in the missing gaps before we make our first real release.

Linux Distributions

Ben and Andrew have been working on Eclipse-related projects for several years. They both work in Red Hat's Toronto office on Fedora Eclipse and Red Hat Developer Suite. The Linux Distributions Project began this year as a joint project of multiple Linux distributions. We were consistently facing and solving the same problems so we decided to work together with greater visibility at eclipse.org. The project is unique among those represented in that it does not produce plugins at this time. In terms of automation, we have developed some scripts to help us more easily build plugins that do not use releng.

We were not "part" of Callisto but we have experience building a number of Callisto projects.

What is packaging and why do we do it?

Packaging is building, bundling, and integrating open source software projects into the larger distribution -- in our case, Fedora. There are many different distributions (ie. Debian, RHEL, Suse, Fedora, Gentoo, Ubuntu) and each is represented as part of the Linux Distros project

Why do we do it?

Because Eclipse rocks and we want to bring that awesomeness to users of the distributions.

Why not just use upstream downloads or update sites?

To integrate with distribution mechanisms and system software (ie. php, apache <-> phpeclipse; SDK <-> update sites; SWT <-> Firefox; etc.) Part of our packaging efforts include building from source. There are various reasons that we do this and for the sake of brevity it's probably better if we talk about this in person. Please feel free to ask.

Our hope is that we can learn more about the projects represented here and their respective build procedures. We would like to eventually get most -- if not all -- of the participating projects shipping in the various distributions. We have had a few issues with building some projects in the past and would like to propose additions to the common building procedures to enable the distributions to more easily package and ship these projects. Specifically, we would like to influence areas related to building from source. Reproducibility of test results is also of paramount importance to our project.