Platform-releng/How to do miscellaneous releng tasks

There are times when some quirky thing out of the ordinary has to be done. This page captures some of them. They are likely easily to get out of date or details change from case to case, but ... thought they would at least serve as "hints" in case others have to ever do these things (and they serve as reminders to me :). They are likely too be quirky and volatile to be part of a FAQ, but might evolve to be a "procedures document". Thought I'd try to capture them every time I do one that I find confusing or hard to keep straight, or when someone asks about, so it is not a complete or exhaustive document, but again, hints. Note that most of these procedures require shell access to build.eclipse.org.

Mailing list and bugzilla

It almost goes without saying, but for someone "starting fresh" in Platform Release Engineering, the bugzilla "user" to subscribe to "listen to" is platform-releng-inbox@eclipse.org. The mailing list to subscribe to is platform-releng-dev@eclipse.org. Ideally should also monitor the "cross project" bugzilla component and mailing list: cross-project.inbox@eclipse.org and cross-project-issues-dev@eclipse.org, respectively.

How to restart Jenkins tests

Tests can reliably be re-ran, even a "long time" after initial run (assuming the build is till on "downloads"), because we save all the relevant data on "downloads", and the test's "input parameters", as a whole, specify exactly what to run, and what to use to "publish" the results. Note: currently, we can only run "I-builds" with the two part time stamp. We can not run, for example, the tests from "S-4.5M4-201412151800" (But, normally, S-4.5M4-201412151800 corresponds exactly to I20141215-1800 so that would be the build to use to re-run tests from a milestone.

https://ci.eclipse.org/releng/view/Automated%20tests/

Hopefully the test "names" are self explanatory, such as 'ep45I-unit-win32' is for the unit tests for Windows 32 bit machine, for the Eclipse 4.5 I-builds. All these test jobs are pretty much identical, but ran as different jobs for two reasons: 1) it improves the automatic history "book keeping". So for example if the number of tests failures increase or decrease, then you are comparing "apples to apples". 2) There are times that "machine restrictions" apply, for example, on Windows, we allow only one build to run, at a time, whereas on the Mac, we allow more than one to run.

Preparation

To "re-run" a test, you need three pieces of information, the buildId, such as M20150204-1700, the 3-digit build stream, such as 4.4.2, and the "hash tag" of the aggregator for that build, such as 115d147f542bfcfeeba452946993c2f2578e85a8.

If the build ran once (i.e. is in "history") these values are in the "parameters" field of the existing test attempt. If even lost in history, the values can be obtained from the download directory.

To Re-run

You need to login to the "Releng Jenkins" (and, for that one, it is your committer ID, and password, not your email, as it is on JIPP instances). Click the "Build now" link, and you will be presented with a form to fill-in the 3 values from above. Click on "ok" (Labeled 'Build'), and check back to see if it's running! (You should at least see it "queued up" if it can not run right away, due to the test machine being busy).

How to see tests results on Jenkins

Our jobs on Hudson are collected in the Automated tests view. The view shows the status of the last job ran (or, current job running). To see history, you need to click on one job. To see which test job corresponds to which build, you need to "drill down" and look at "Parameters" of each job.

How to change to web pages for builds or test results

This section is a brief outline of the files and steps involved in the web pages for "drops" (builds) and test results.

A key piece of the work is done by a custom ant task, which is found in TestResultsGenerator.java in the eclipse.platform.releng.buildtools repository. That jar file (bundle) along with several others and a feature is built on Hudson and put in a p2 repository on the build machine, under /shared/eclipse/buildtools. (Much of this is done and triggered manually -- it is not part of the regular build, and not completely automated, mostly because it is rarely done).

For the main drop page, the file index.template.php is "ran through" that ant task, which fills in specific artifacts to download, partially based on the testManifest.xml file.

For the Test Results page, a file named testResults.php "controls" what is displayed, but it does not act as a template for the Ant task. Instead, the ant task creates files which the testResults.php file "includes" (if they exist).

The "drop page" (index.php) and the "compiler logs" (compilerSummary.html) are typically generated once, after a build is done. The Test Results summaries are re-computed several times in response to receiving a "done" signal from Hudson when each platform is done with its testing. Each "regeneration" assumes that all "test results" files are still available, even of already generated the summaries for that set previously.

The included files are mostly pure HTML with a minimum of "style" specified. Instead, the HTML elements inherit their style from a "static" file named resultsSection.css which is activated by being in the "resultsSecion" div.

That resultsSection.css is "included" by DL.thin.header.php.html. That "thin header" is a special version of the "Solstice Theme" which provides a minimal amount of "extra" things, and is also good because it can display on a non-Eclipse.org downloads machine (such as the Eclipse build machine, or even a "local build" machine) whereas the full Solstice theme requires access to Eclipse.org databases.

The DL.thin.header.php.html is a highly customized version of one that can be obtained from Eclipse.org. See the Solstice documention for more information. But in short, the thin header template can be obtained with

wget -O DL.thin.header.php.NEW.html  https://eclipse.org/eclipse.org-common/themes/solstice/html_template/index.php?theme=solstice&layout=thin-header

From time to time, that file should be obtained and compared with our customized version, to see if anything has changed. It is recommended to use WTP to format it and also format or edit CSS files like the resultsSection.css file. In addition to providing consistent formatting, for easier comparisons, it has a built-in color editor that is handy on finding good values of our custom colors to use.

There are other, similar, files used for our download pages (i.e. "included") such as eclipseDownloadPage.css and eclipseDownloadPage.js but the hope is this bare outline would help someone get started when changes or fixes are needed in the future.

Other releng tasks

See Platform-releng/Platform_Build_Automated#Routine_release_engineering_tasks_for_builds for other, more routine releng tasks.