Helios/Final Daze

Our ramp-up to our annual Simultaneous Release is similar to previous years, but there are some important differences, so read closely. Also, there are more projects releasing this year than every before, so it is all the more important to follow the schedules, and keep everyone informed of any issues.

The following is a "blow-by-blow" schedule of the final days leading up to the Helios simultaneous release. This is one of the responsibilities of our heterogeneous projects working together towards having a simultaneous release.

Failure to follow these procedures and schedules can have dire consequences, which we all have seen before at one time or another -- if not done in the right way, in the right order, at the right time, mirrors get busy replicating one project, but other projects have difficulty accessing eclipse.org to produce their final builds much less upload them, so a "chain reaction" begins and everyone is much delayed simply due to network and infrastructure bottlenecks -- or, I should say, from someone not following these procedures and schedules! :)

Plus, this schedule and procedure should ensure the mirror system get's fully populated before demand for downloads gets in full swing -- meaning much fewer delays, 403's, and failed downloads for end-users -- which all Eclipse users will appreciate.

New for Helios

• Download Stats for p2. This can be enabled in the p2 repo of a project as well as in the central p2 repo (.../releases/helios). If you want stats, make sure your artifact repo(s) is(are) set up for them and artifacts appropriately tagged. See Equinox_p2_download_stats and bug 310132 for more details.

From Now to 6/17

Clean up "old" builds and releases

Delete or archive "old" builds or releases and even milestone builds that are mirrored. This is important to reduce the disk space used, not just on eclipse.org servers, but more important, the space used on the mirror servers. Not only is this a "nice" thing to always do, it is essential for the yearly Simultaneous Release, as it can help avoid those mirror servers from filling up once they get the final release and they themselves having problems mirroring Helios, once it is released. And, of course, if they have problems, we all have problems! Everyone should have space at, if not already using, archive.eclipse.org, and if you are not sure how to archive files, please ask (it is easy).

We ask this clean up to be finished at least one week ahead of the release so a) all mirrors can be sure to easily get "synced up" with the new minimal amounts, and b) allows time for some inspections and audits and projects that appear to be non-compliant will be asked to explain their use of Eclipse.org resources.

Final Build 6/16

The Milestones and Release Candidate dates are defined in Helios#Milestones_and_Release_Candidates

The "final build" means that all content must be frozen; features, plugins, versions, etc. This content might still be on a "temporary" site (e.g. .../milestones) and later (such on 6/22) moved to a "permanent" site (e.g. .../updates) but during that move, there should be no change to plugins, features, versions, or repo meta data (artifacts.xml/jar files and content.xml/jar files).

Finish Helios Info Center 6/15 to 6/19

Provide info center jars in bug 315157.

From Now to 6/21

Prepare releases offline

That is, prepare your final releases zipped files, update jars, etc., but do not put zipped up builds, update jars, etc., in their final (mirrored) release areas of eclipse.org until instructed to do so. This may require some projects to "turn off" some final step of their build process, depending on how they do it, and how and when its copied to eclipse.org. If you have a special circumstance that does not fit in the following procedure and schedule, please send a note to discuss with our Eclipse Webmaster (Denis Roy) at webmaster@eclipse.org and work out a plan to accommodate your special situation.

Note: this may mean some projects need to "adjust" their final build procedures, so they do not rely on some other projects official, final, mirrored release, but this should not be required as any "pre-req'd" projects should just be making minor changes to doc, or highly local and internal bug fixes (nothing that would be changing constants or API's or even non API signatures!). If you do need some official, final release of a pre-req project, please work with them to find out where to get it from (other than a mirrored location).

All times below are in Eastern Daylight Time (which is UTC-4 in June, during Daylight Saving Time).

Note: since we do not have "sign off" pages any longer, if anyone is the least bit late (or even close to being late) to meeting deadlines, please keep everyone informed via cross-project mailing list.

Helios site and EPP preparation 6/17 to 6/21

• staging update site will be created to verify all are accurate, and ready to be copied to releases area
  • David to send email when all aggregate builds done, and that 'staging' is ready for EPP final builds.
  • David to promote staging repo to its secret place (where p2 can not see it yet) so the bulk of the artifacts can mirror ahead of release day.
• EPP will start building the final packages as soon as the Helios staging update site is ready (releases/staging and eclipse/updates/3.6milestones)
  • EPP package maintainers will receive a mail with a link to the final EPP packages
  • EPP package maintainers need to test the final packages and must send their 'go' to the epp-dev mailing list

Monday 6/21

NOTE from Helios SR1 release day: Mirror-overload issue should be better addressed in future

There is no longer any need to wait until "day before" for projects to put their artifacts in their final location, as long as leave "invisible" until the official release. Everyone waiting until the day before means the mirrors are very busy pulling content the day before, and have trouble getting caught up and in a steady state before the actual release. So, one action is to make sure everyone is properly educated about his. It might take some thing more formal ... some projects promote 5 days before, some 4 days before, some 3 days before, etc. Also, may need better directions on how to promote but leave invisible ... perhaps many projects wait and promote at last minute because they don't know how to do that?

from 9:00 EDT to 17:00 EDT (at the latest)

• projects have prepared zips downloads and all update site files

• Teams are free to push final builds in their final location at any time, hiding them from public web pages
• if anyone needs to update their web-pages ahead of time, with other projects final URLs, please ask on cross-project mailing list if you do not know what it will be.
• Projects must ensure update sites and *.build files stay coordinated, so sites could be re-created if required.

• Final EPP Package builds, if required. (/releases/helios and /eclipse/updates/ will be the input for the packages);

depending on the server load the build takes between 4 and 8 hours

kmoir>I don't want the final Helops build to refer to bundles in our milestones repo. It should point to our releases repo. This means that our bundles will be available on the Monday 21 instead of Wednesday.

• last opportunity for anyone to declare "emergency, stop ship" (and it had really better be an emergency! A simple build machine crash does not qualify ... better have backups! ... and, maybe, a lab fire might qualify if it is wide spread :)

Tuesday 6/22

9:00am EDT

• EPP package maintainers declared their 'go' on the epp-dev mailing list
• Teams are free to push final builds in at any time, hiding them from public web pages

This Includes:

• Projects to upload/copy/rsync zip files. [Tip: Do not use -t in your rsync commands, but let the file time take on the current time the copy was done ... the reason is that the mirroring system uses timestamps to infer what's been mirrored and what has not, and a file that appears "old" may end up being assumed mirrored, when it has not been mirrored yet ... which will result in users getting lots of 404s from mirror site URLs].
• Projects to update to project's 'released' update sites.

• Helios "staging" copied to "released" Reminder: until this day, until this push, /releases/helios will have "old" content, from the last RC build.
  • update (or verify) /releases/helios has correct mirroring URL
  • update (or verify) that composite repository URL points to final /eclipse/updates URL (http://download.eclipse.org/eclipse/updates/3.6/)
  • NEW: test download stats on /releases/helios

• copy EPP packages to their final location /technology/epp/downloads/releases/helios/ <== this is super important, as the Main Downloads Page and the Packaging Site makes the EPP packages the default!

[Nathan] Disable the Cron job from updating the Packaging Site Till 9AM Wednesday

By 10:00am EDT

• Denis prepares torrent files
• Denis sends email to eclipse-mirrors asking for manual sync
• Nathan publishes 'Helios is here for Friends only' page for the main downloads page; links for Friends mirrors and Bit Torrents only
• Denis Emails Friends of Eclipse with details on how to get the downloads from the friends mirror.

16:00

• wait overnight, see if mirrors get the files
• projects should prepare final web pages, announcements, press releases, etc.

Wednesday 6/23

08:30 EDT

• Denis to update alias for http://help.eclipse.org to go to http://help.eclipse.org/helios
• Denis to do a mirror-site sanity check and consult with P2 team. How many? Are they all done?

Assuming all is well,

09:00 EDT

• Denis to announce on cross-project that Projects can now make their download pages visible.

• David to "flip switch" on composite repo, making release content visible to p2, and removing any previous release candidate pointers.

• Markus to "flip switch" on EPP repo /technology/epp/packages/helios - should be synchronized with the Helios p2 repo switch

• Nathan

1. Put web pages live, make announcements.
2. Update Homepage / Promotion / Friends of Eclipse Images to post release.
3. Re-enable Packaging Site Cron Job and Promote Final Release to Front Page.

13:41 EDT

• Earth Shattering Kaboom - http://earthquake.usgs.gov/earthquakes/dyfi/events/us/2010xwa7/us/index.html