Eclipse Quality

Eclipse Quality means extensible frameworks and exemplary tools developed in an open, inclusive, and predictable process involving the entire community. From the "consumption perspective," Eclipse Quality means good for users (exemplary tools - cool/compelling to use, indicative of what is possible) and ready for plug-in developers (deliver usable building blocks - with APIs). From the "creation perspective," Eclipse Quality means working with a transparent and open process, open and welcoming to participation from technical leaders, regardless of affiliation.

Eclipse Quality is not a binary statement. Rather, quality is a spectrum and Eclipse projects are expected to, release-by-release, improve their quality. Eclipse Quality is an evolution over the life of the project and the evaluation of a project's quality must be appropriate to the project's maturity level. For example, a project in the Incubation Phase will have more Provisional APIs and fewer Platform APIs than a project in the Mature Phase. Similarly, a 1.0 Release will have fewer API clients than a 2.0 Release; and a 3.0 Release will have a better release-to-release migration plan than a 2.0 Release; etc.

Note the Eclipse Quality is about both extensible frameworks and exemplary tools - great tools are important for attracting the users, who then attract the ecosystem, that then provide members, who then contribute resources, who then create additional valuable frameworks and tools. Neither frameworks without users nor tools without frameworks are interesting points along the software development spectrum.

GEclipse Quality goals:

• Platform quality frameworks. (note that "Platform quality" here refers to high quality APIs [1] (such as a module API in NetBeans [2] or a Java platform API [3] or the Windows API [4]) and not to the Eclipse Platform project).
• Platform quality tools.¹
• All exemplary tools are built on platform APIs.
• Performance and scalability to enterprise class use.
• Automated tests and a quality plan that can be used by any Eclipse developer.
  We want any Eclipse user to be able to run all the tests and verify that they have a working and correct installation of the frameworks and tools.
• A release-to-release migration plan potentially including automated tools for conversions. Release-to-release migration includes not only the APIs, but also the artifacts generated by the tools (e.g., configuration files, persisted tables, etc).
• API stability.
• Predictable behavior and a predictable rate of change.
• Demonstrated community involvement.
• RIP due diligence as per the Eclipse IP Policy and other Eclipse processes [5],[6], [7], [8]. (This is the only required Quality element, the rest are goals.)

APIs, Provisional or Not

In an earlier draft document, Eclipse Quality APIs [9], a platform API was defined to have:

• A specification. A description of the cover story and the necessary details. Specifications should be clear about what is defined and what is not defined. Furthermore, specifications should indicate what areas are likely to change in the future (e.g., this two-valued parameter will probably have N-values in the future). A specification is a difficult document to write, and it is more than just a paragraph or two about the component.
• A test suite.
• An implementation.
• One or more clients. Usually, just having “a client” is not sufficient – an Eclipse quality API client is ideally developed by a separate team. A client (API user) that is written by the same team that also creates the API implementation usually suffers from information leakage and thus is not a good test of the API. Even better is two or more clients, each developed by a separate team, all of whom communicate only through the API documents.
• A support promise. An implicit or explicit promise about how stable the API will be from release to release.

Note that the API (Application Programming Interface) is the entire public programming surface of the framework, thus it includes not only Java classes, packages, and interfaces, but also extension points, file formats, generated meta-data, and even objects that are passed through to internal packages.

The question arises of what to do about code that is intended to become API in a future release, but is not up to these standards in the current release. We can consider the following cases:

	Specification	Test Suite	Implementation	Clients	Support Promise	Package
RPlatform API	yes	yes	yes	yes	yes	public
Provisional	yes	yes	yes	yes	not quite	public
Incomplete	incomplete	incomplete	yes	yes	yes	public
Experimental	yes	yes	yes	yes	?	Gprovisional
Non-API	-	-	yes	-	none	internal

Provisional, Incomplete, and Experiment APIs are collectively referred to as Transient APIs. The interesting cases are Provisional and Incomplete. Under this definition, Provisional API is real API in all senses (good specification, good Javadoc, good tests, etc) but the project does not feel that it has had sufficient community feedback to completely freeze the APIs yet. The TPTP project uses Provisional APIs for all new API introductions: the new API is released as provisional in release X and then hardened into platform APIs in release X+1.

Incomplete APIs are those where the project wrote the code before defining the APIs and thus the documentation and/or test suite is incomplete, but the project intends to support the API. Incomplete APIs may appear in milestone releases (although the lack of documentation will make it difficult for the plug-in developer community to adopt and verify those APIs), but Incomplete APIs must not appear in releases.

---

¹ Admittedly, there is not a good definition for "platform quality tools" yet. A future version of this document will try to clarify this.

This page is moderated by the EMO