Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Europa Build Workshop Breakout Results
Contents
Build Best Practices Breakout
(computer crashed in the middle of typing in the result)
- Proper versioning
- We discovered there are three projects that have created tooling for checking version numbers conform to this specification Version_Numbering (BIRT,WTP, TPTP and Platform). Before anyone creates anymore tools :-) - please contact one of the teams to use one of the existing tools. Links to be added. Consolidation or pick best one to reduce overlap.
- Making builds reproducible
- Tag files from each release with the same human-readable tag
- Track API changes
- Hubert Leung uses Jdiff http://jdiff.org
- WTP also provides a tool (Jeff Liu).
- avoid circular dependencies
- documentation up-to-date; current link available on website; remove old contents
- backup redundant systems
- separate publishing script from “build” (compilation)
- able to turn functions on the build on or off e.g. turn off publishing
- regularly scheduled builds
- have an update site; optimize update sites
- pack 200 Pack200. How to implement in builds - see
- feature design – group plug-ins based on functional grouping, optimize feature design for PDE build.
- Hubert wrote a tool that generates a report that shows the feature structure and the dependancies that may be useful to other teams. Also displays the feature that each plugin resides in and additional information. Also, it displays whether a tool is shipped as a jar and has a plugin.xml or MANIFEST.MF.
- He also has an additional tool that conducts dependancy analysis.
Go to Europa Build Workshop Home page
Buckminster/Maven
Day 1 Discussions
- Common Model for Project Artifacts/Components
- Development of a Backbone
- Graphical Model Input
- Buckminster and Buckminster model for Eclipse-based Model of a Project
- Maven in Eclipse and Maven model
- Maven
Day 1 Actions
- Closer integration of Maven and Buckminster (Thomas/John)
- Adoption of an existing Callisto project to help revise that project's build by the Europa release (group)
- Document the migration/conversion process to Buckminster/Maven (group)
- Agreed that BIRT will be the test case project (Sue/Thomas/John)
Day 2 Discussions
- BIRT Build Review
- Denis offered build.eclipse.org build server for BIRT to run its builds (providing shell access to Eclipse committers)
- (awaiting notes from Thomas)
- Adoption of ECF Project (to have both a large and small build project converted for Europa)
Day 2 Actions
- Subscription to the Users and Dev lists at Buckminster
- Maven/Buckminster
- BIRT
- ECF
- CVS location to be sent to Buckminster team (Pete)
- Set up ECF locally w/ Buckminster (Henrik)
Future activities/planning will be coordinated by Natalie.
RSS
- One artifact of an EMF build (Thu 00:00 EDT) being promoted (Thu 01:00 EDT) is that an RSS file is updated and scp'd to download.eclipse.org
- The UML project consumes the RSS feed to trigger a new build (Thu 03:00 EDT)
- Platform will be publishing RSS soon (very soon)
- UML assumes a linear ordering (reads just one feed, not many feeds, since it depends on EMF which in turn depends on Eclipse)
- RSS feed supports a variety of status codes: build done, tests done, etc. though a definitive list has yet to be stated.
- This is a TODO for Nick to define the spec, though I'm of course open to suggestions to make the spec useful to all
- RSS is on a download site so that mirrors will be updated with files and then the RSS - as such, you can watch a given mirror's copy of the feed in order to know when the mirror's got new upstream project bits on which you depend
- RSS is committed to CVS for history/auditing
- Dennis and Nick recommend this should be adopted for cross-project notifications (and optionally response) by Europa projects to try it, use it and improve it
- Nick is willing to be the point person for initial communication, documentation, assistance
- Which builds to publish for? nightly? integration? good? failed?
- Kim brings up "how to define what 'good' is - what if it was a test fixture failure (as determined by a human) instead of a test failure - then the human says 'this is a good build' and yet the RSS says 'bad'".
- Bjorn argues for more information rather than less - let the downstream consumers decide what to do with the extra information (e.g., ignore nightly builds, ignore bad builds, ...)
- As such, we would suggest that if you publish a build, it should includes an RSS entry (be it of type N or I/M or S or R). Build status in the feed (PASSED, FAILED, IN PROGRESS, PENDING, UNKNOWN, etc.) can default to a given state and be updated as needed by Ant tasks
- Additional information including implementation details and RSS feed schema
- Working implementations for EMF and UML2
Bjorn, Ward, Nick (and others) would like to see everyone adopting the process of publishing feeds within the next month, and listening to feeds (and determining how to respond) the month thereafter. Ideally, this means we could try some RSS build cascades (or at least RSS information cascades) by early November. There was a general consensus that this was a good idea, so I'm hoping everyone can commit to this schedule.
--nickb
Nick's Tools
Nick shows off new Search CVS tool for browsing CVS and Bugzilla. (This tool is also linked from our generated release notes.) If you'd like your project indexed in this database, add your name and /cvsroot/path/to/project(s) here. I'll update this when you're indexed so you can play with it.
- org.eclipse.corona
- org.eclipse.emf, org.eclipse.emf.releng.build, org.eclipse.emf.ecore.sdo, org.eclipse.xsd
- org.eclipse.emft
- org.eclipse.gmf
- org.eclipse.uml2, org.eclipse.uml2.releng
- www
- emf
- emft
- uml2
Buckminster Demo
The Buckminster team gave a demo.
Common Build Infrastructure
Goals:
- Shorter learning curve / quicker startup
- Consistency
- Best practices
- Outreach Activity
Scope:
- What is provided / what form
- Benefits to Foundation / Projects
- Committers and projects need not worry about so much setup/infrastructure
Wishlist
Bjorn advises to ask for the moon, so...
- vservers / vmware instances to start builds from clean states
- add build infrastructure to vserver image file so more consistency
- common management interface / new committer tools on dev.eclipse?
- project dashboard interface, eg., http://packages.qa.debian.org/f/firefox.html
- existing simpler example (EMFT projects):
- PDE?: decouple java compilation from native building from packaging for projects with platform dependent code (eg., TPTP)
- staff at eclipse.org to maintain infrastructure
- staff at eclipse.org to manage release train
Eclipse.org Current Build Infrasctructure Offerings
- Option 1: unmanaged vserver (*.eclipse.org)
- users get root access, running Xen
- no management provided by Eclipse
- two physical servers to share load
- currently limited by IP address pool
- hardware limited by budget
- requires extra maintenance / security / duplication of effort
- Option 2: managed "massive massive quad-CPU beast": build.eclipse.org
- Backups provided, no root access
Best Practices
- Let projects choose whether to publish builds from build server to mirrored, so as to minimize bandwidth impact at Eclipse
- Protect access to web UI to kick builds using LDAP so only project committers can run their own builds
Actions
- Set up a vserver for Buckminster/BIRT builds
- Assist in migrating ECF to web UI for kicking builds & running tests on their vserver using EMFT as example
- Explore Buckminster as wrapper for running EMF/EMFT builds
- Explore creating a web UI for running Buckminster builds (PHP, XMLRPC?)