SimRel/Simultaneous Release Requirements/Appendix

Simultaneous Release Requirements Appendix

Updated November 7, 2013

Authored and maintained by the Eclipse Planning Council

Contact: David Williams

This document is an appendix to the Simultaneous Release Requirements document, and is intended to provide supplementary information for the primary document. This may be in the form of greater detail or explanations for a stated requirement. It is generally intended to be used as a reference from the primary document, and not read on its own.

Normal Release Requirements

Formal (standard format) plans, early (M4)

Target Environments

It is a similar case for specifying what minimum (or, maximum) Java level to use (such as Java 5, Java 6, Java 7). We do not mandate any particular level, but the question often gets asked, so we simply clarify here that projects should well document what version of Java they generally write to and test with. (Note: individual bundles have their own specific BREE's, but that is a very specific "micro" level specification, and does not reflect the overall level of Java targeted, or tested.)

It would be anticipated that this year, for Kepler, many Eclipse users would still be using Java 6, but projects should test extensively on Java 7. In general, it is recommended to write your code to the lowest level possible, but test on the highest level possible.

Of course, the "target environments" you focus on, in your plans, depends on your project. For example, some runtime projects might focus on what servers they work with and test, some projects might focus on what browsers they target, test, and support, or similar.

To satisfy this requirement, please document your supported platforms in your standard project plan by having a section labeled, exactly, as <target_environments>. See template in standard plan reference and for an example, other than the Platform's, see the subject section in Web Tools plan.

Be sure to raise any "target platform level" questions or issues to the cross-project list, especially if a your project is "moving up" a level and might affect projects or adopters that depend on it.

Compatibility with Previous Releases

Note: this primarily applies to projects that have "released before", but even those releasing for the first time should be aware of this requirement as there could be things done "now" for future compatibility and migration ... these need to be "thought through" even for your first release, to make sure your second release maintains appropriate compatibility.

The intent, here, is for projects to document their intent and whether or not bugs about compatibility would be considered valid. For example, a project might document "we support only a one-time migration of projects, and if it is a shared project, all developers must move to new version at the same time". While that level of compatibility is less than ideal, at least then adopters know what to expect and can make their plans based on that information.

IP Documentation and Logs (RC1)

Projects are encouraged to create drafts of the Projects IP Logs even earlier. The development process requires the IP Log to always be accurate, but experience shows there often are not, until examined for a release, and there are typically many issues to resolve, or fixed, before the final submission to the IP staff. For example, sometimes a CQ might have the wrong flag, and cause it to not show up in the Auto IP Log, or perhaps a common, already approved 3rd party jar was was used, but no CQ for that specific project entered one for the new version. The purpose of having these early drafts is so that projects get familiar with what is required, and do not allow work to build up until the ver, since that could cause a "bottleneck" at a critical time and jeopardize having the ability to resolve all issues in time to be released. Also, some adopters will want to look at early drafts to see what 3rd party requirements are associated with the code they are planning to adopt.

A good guideline is to have a draft of your IP Log by M5, and plan for it to be complete for M7.

Note that being in the Simultaneous Release will give your IP CQ requests some higher priority in getting evaluated, in order to make the date. The higher priority treatment is only for the 5 months or so before the release (after the deadline for CQs, typically M5). The reason being, of course, is that the rest of the year the IP staff must also get work done for maintenance releases and projects not on the release train. During that part of the year (roughly July to February every year) all CQs are prioritized in a uniform way. But, there is a deadline for CQs for the yearly release, usually in February, end of M5, to have all required CQs submitted, on file (but not necessarily approved by then).

Release Review and compliance to requirements documentation (RC3)

Normally, in release review documentation, Project Leads must document and state that the project's release complies with all the normal release requirements, such as IP requirements. In addition, for a Simultaneous Release, Project Leads must document their verification that the project complies with all extra requirements of this Simultaneous Release document, as they apply to their project, and document any exceptions, there in the release review documentation. This is intended to be a few short sentences or paragraphs ... not a detailed checklist. [This is the replacement or "low cost alternative" to tracking on Foundation Portal, which was deemed unmaintainable. Eventually some of the URLs etc. requested in the old tracking system will be in the new "release portal" but not yet (see bug 363524.]

Required for good adoption

The items in this category are, in a sense, optional. That is, what, exactly, is done by a project is optional, but it is required for projects to document what they do. These are often "best practices" that many projects have found essential at driving good adoption, plus the items sometimes speak to the quality of the project (quality as an Eclipse "good citizen", as opposed to their code quality or architecture). But, their importance is not be as universally relevant to all projects and their adopters, hence it is only required that each project document what they do for these items, but exactly what they do is up to the best judgment of the project and their community.

Engage Community

The Project should actively engage their community to get feedback on milestone builds, and document how they do that. One way to do this is to have a New & Noteworthy for each milestone. New and Noteworthy documents should be something readable and usable not just a static list of all the bugs. Corollary: individual new & noteworthy should be linked in to a collective New & Noteworthy.

Usability

Should follow the User Interface Guidelines. The UI Checklist is a good place to start. Also, when possible, it is good to have a UI walkthrough.

Performance

Project should have measurable performance criteria that are regularly tested against. Projects should devote at least one milestone to performance and scalability improvements. This is important to the Simultaneous Release since a project or plugin by itself might seem to perform ok, but taken together as part of hundreds of bundles, it might cause in unusable performance.

Builds

Projects must have a mature, stable build process: documented, scripted, repeatable, and executable by others on their own system. Ideally there would be just a few commands to run from a Linux shell to accomplish the build, but there might need to be a page of "set up" instructions of what's required (e.g. if Git needs to be installed, certain version of Java, etc.) Of course, there may be some parts of it that do not work completely (such as, being signed by the Eclipse code certificate would not work when ran by others, on non-Eclipse.build systems) but the build should not fail if it does not sign, and otherwise should produce essentially the same functional output as the "officially signed build". Note, this build should include both producing the code and producing the runnable unit tests.

Unit Tests

Projects should have unit tests with enough documented instructions that others (adopters or extenders) can run in their own environments and configurations, independent of doing a build. Ideally this could be done with just a few commands, but as with builds, might require a description of machine set-up.For one, example, see WTP Stand alone unit tests.

Ramp Down Planned and Defined

Projects must have a written ramp down policy by M6, at the latest, and provide link. The plan should describe when the project plans to be feature complete, have API frozen, and similar. See Platform 3.5 Endgame plan as a guideline. See also Juno Final Daze.)

Accessibility

Projects should design and test for accessibility compliance, following established guidelines and Eclipse fundamental techniques to achieve accessibility. Projects must document their accessibility work and compliance. Ideally this would be by using a publicly available checklists, such as

• http://www.itic.org/public-policy/accessibility/
• http://www.section508.gov/
• http://www.w3.org/TR/WCAG/

See the WTP Accessibility Page for how one project documented their checklists.

Note, projects can document their work or compliance as a negative, such as "We do not do any accessibility work or testing and do not know the degree of our compliance. While we will consider accessibility bugs as valid, it is unlikely we would give them a high priority". But its important to document if that is the case, so adopters know.

If possible, and appropriate, accessibility testing tools can be leveraged such as NVDA.

The main accessibility article at Eclipse Corner is also very helpful. (thanks goes to Todd Creasey).

APIs

Use only APIs

Projects should leverage only published APIs of dependencies. All deviations must be documented in bugzillas. These bugzillas may be of the type that a dependent project should provide a required API, or of the type that a consuming project must move to some API that already exists. Note that technically there is no obligation for consumed projects to provide API that is requested ... that depends on many things ... but the main goal of requiring these bugzilla entries is to provide some documentation and measure of the amount of risk associated with non-API use.

Document APIs provided and publish API policy

APIs should be provided, so others can properly extend functionality, and presumably most of that detail is defined in JavaDoc. But, there should be a high level statement of how consuming projects can determine what is API; how 'APIs' are distinguished from non-API and 'provisional' API, if any. It is recommended that non-API be marked with x-internal in the bundle's manifest. Also, API policy should include what the commitment is to API, how long maintained after deprecated, etc. As one example, see WTP API Policy.

Retention Policy

Projects should define and document their retention policy. This should include both zip distributions and repositories. For examples, see WTP Retention Policy and Eclipse Project Update Sites.

Make it easy to get released source from repository

Projects should make it easy for potential contributers (and adopters) to get the source used in the release, from the source control repository (that is, source that exactly matches what was used in a build or release). This might be done by some form of documentation, or tag, or by producing a "team project set" during your build and making that available from your downloads page.

Another good way to accomplish this, while maybe not suitable for all projects, is to make your bundle's source repository "self documenting" by using the Eclipse-SourceReference directive in your manifest.mf file (its very easy to have this added if your build makes use of PDE build or its properties). This requirement serves two purposes.

First, we want to be sure adopters can get your source used in a release, in case they need to create some fix, even at some distant point in the future. But, also, this serves the community purpose of making it easier for contributors to provide patches for bugs.

Excel in National Language support

User Interface

Use the best NL Java Libraries (ICU4J)

Projects should use ICU4J, where they have "user strings", in order to excel in NL support. (The latest recommended ICU4J bundles will be in Orbit).

Use the most efficient message bundles

Projects should use Eclipse message bundles unless there are technical reasons not to, since (with or without translations) these are known to be more memory efficient that some other forms of handling UI-releated strings. This is important to Eclipse and adoption is some adopters end up using thousands of bundles ... so, that's a lot of strings!

Support translations

All strings must be externalized, and projects must participate in Babel, meaning it is registered and available for string translation, etc. Projects must freeze the UI sufficiently early to allow the Babel project time to translate strings so there can be simultaneous release of translated versions. This typically means the UI should be frozen by M6 (a "freeze" means all major changes and additions are done by M6, and changes after that are done in a controlled, well documented, well reviewed fashion, so Babel translators can more easily keep up with late changes). The project should use the Babel Pseudo Translation Test to verify their translatability. See bug 217339.

Enable development for all languages

Projects should design and test for enabling development for all languages including bidi, unicode characters, etc. This is different than "translating" the User Interface and would apply even to "runtime only" components. For one example, while using an English version of Eclipse Web Tools Platform, someone should be able to create a Chinese language web application and have it (and debug logs) displayed correctly.

New and Noteworthy

The EMO will assemble an aggregated New and Noteworthy document, targeting the end-user audience. Content is assembled in AsciiDoc format in a Gerrit repository that is created by the EMO at least one month in advance of the official release date. When the repository is avaialable, the EMO creates a record to track the assembly of the document in the Community/Cross-Project Bugzilla component and announces the availability of the repository on the cross-project-issues-dev mailing list.

Project teams should observe the existing content and provide content in a similar format via Gerrit pushes.

Note that this is an end-user document, and so we try to avoid sectioning content by project, and instead try to group content by subject area or functionality. The source files are divided along similar subject area boundaries. Reviewers are welcome.

Note also that we will update the one new and noteworthy document with each point release.

Additional Information

Planning Council Exception Process

Exceptions for any rule or schedule can be made if there are good enough reasons to. This same exception process will be followed for things like "requests to respin" a build after a deadline. The process to get any exception must be open and well documented and follow these steps:

First, the Project's PMC must approve the request for an exception and it is the PMC (not the lone Project or committer) that makes the request to the Planning Council. The Planning Council member that represents the PMC is the one to bring the issue forward to the Planning Council.

Second, the exception requires at least 3 positive votes from active Planning Council members and no negative votes. When time is a factor (e.g. requests for rebuilds) the deadline for voicing a negative vote is basically by the time 3 votes are documented. But when time is not a factor, such as when requesting an exception to one of the criteria, then a period of one week will pass before being final, to allow times for concerns or negative votes to be voiced even after 3 positive votes. If there are not enough positive votes within one week, then the request for exception will be considered 'failed'. Note that "3" was chosen under the assumption that the Planning Council member representing the PMC would vote for it (since that PMC must approve it initially) so that means 2 others must also vote for it, for 3 total.

Depending on the timing, the issue and votes will be documented in either the Planning Council Meeting minutes, or on the Planning Council mailing list.

Testing of Simultaneous Release Repository

Some of the items marked "(tested)" means that there is some degree of automatic checking and reporting done on the repository. The reports, for example, see the Luna Reports do not test "project by project" but as a whole, give some idea of where there are violations or problems. Ideally, the common repo tests should be a final sanity check, and projects can include their own automatic testing during their own build processes. See reporting FAQs for more information. Projects themselves (and/or adopters) are responsible for checking these reports ... that is, the Planning Council does not "police" compliance, the projects and PMCs do.