Modeling Project Releng/Releasing/Update Site Generation

For an overview of how update sites are generated -- including site XML, jars, digest, and p2 Metadata, see this blog: Update, Updated.

Overview

• Every new build produces a "category.xml" - it's not a valid XML document, just a fragment for the category contributed to the overall site*.xml

view-source:http://modeling.eclipse.org/modeling/emf/updates/milestones/emf-emf-2.5-S_r0/category.xml

• Each new build (say, for EMF (Core) 2.5 on milestones site) will push out the oldest of the 3 stable builds and keep only the new one + one older one. For I/M builds, we keep the three most recent ones; for R, we keep one. Note that what's published to download.eclipse.org and what's held on build servers may differ (more on the build server, less published).

http://emft.eclipse.org/modeling/emf/updates/milestones/

http://modeling.eclipse.org/modeling/emf/updates/milestones/

• Every new build also collects a list of jars that are still required for the 1-3 builds kept on the site per project. This is done by storing a copy of each build's features/ and plugins/ folder ON THE BUILD SERVER. The list of jars to be kept is held on download.eclipse.org, and each build server produces its own list.

http://download.eclipse.org/modeling/emf/updates/milestones/jarlist.clean.modeling.eclipse.org.txt

http://download.eclipse.org/modeling/emf/updates/milestones/jarlist.clean.emft.eclipse.org.txt

http://download.eclipse.org/modeling/emf/updates/milestones/jarlist.clean.emf.torolab.ibm.com.txt

• This is done so that we don't end up with an ever-larger update site with irrelevant / unreferenced / obsolete jars. We need multiple files because not all builds are produced on the same server, and thus we'd either need to store all the update site jars + category.xml for each build on download.eclipse.org, or cheat by use of the listfile. I went the listfile route to save disc usage on download.eclipse.

• Every time a build is published, all jars NOT on the aggregate jarlist.clean*.txt files for a given site are deleted. All category.xml files for that site are merged to create the new site.xml files.

• The system, in theory, does its own maintenance and cleanup every time a new build is added to the pile. Also in theory, we should never end up with site.xmls that reference jars that are missing because the full list of required jars comes from Master zip (signed + packed jars) -> unpacked site on build server -> directory listing of included jars -> jarlist.clean*.txt files.

---

See also Update Site Generation - Troubleshooting.