EclipseQualityRequirements

Pierre Gaufillet (Airbus) and me (Boris Baldassari, SQuORING Technologies) have previously worked on a draft of a quality model that would fit our context of software. We tried to identify the main quality concerns for Eclipse and Polarsys, and came up with a sort-of wish list. As for now, Eclipse quality concerns are scattered across the wiki and projects, and there is no single place to refer to for quality requirements or definitions.

The first words that catch the eye when scrolling through the Eclipse documents are quality products available in a process-controlled manner, with a transparent and open process. This section investigates the definition of quality for the Eclipse community.

Location of informations

The main places to get information on these subjects are listed thereafter:

• Eclipse Quality: http://wiki.eclipse.org/Eclipse_Quality
• Development Conventions and Guidelines: http://wiki.eclipse.org/Development_Conventions_and_Guidelines
• Eclipse development process: http://www.eclipse.org/projects/dev_process/development_process.php
• Coding Conventions: http://wiki.eclipse.org/Coding_Conventions
• Naming Conventions: http://wiki.eclipse.org/Naming_Conventions
• Development Resources: http://wiki.eclipse.org/Development_Resources

Identified Requirements for Product

Considering the vast amount and diversity of the projects under the Eclipse umbrella, there must be no single definition of quality to fit them all. However, Eclipse has some recommended practices and concerns about product quality. Projects are then expected to extend this foundation.

Major concerns identified for Eclipse products quality are linked to the development context of the foundation (open source, very large code base and thousands of contributor worldwide), and its architecture (modular stacks of components). It must be highlighted that product quality is not clearly defined on the public wiki, neither for its definition nor for how it may be assessed. Furthermore, almost all product-related rules (with a few exceptions, like for packages naming) are optional guidelines.

Crawling through the wiki the main concerns that can be identified are (in an ISO-9126 state of mind):

• Maintainability: The capability of the software product to be modified .
  • Analysability: The capability of the software product to be diagnosed for deficiencies or causes of failures in the software, or for the parts to be modified to be identified .
  • Changeability: The capability of the software product to enable a specified modification to be implemented.
  • Reusability: The degree to which an asset can be used in more than one system, or in building other assets .
• Reliability: The capability of the software product to maintain a specified level of performance in cases of software faults or of infringement of its specified interface .
  • Maturity: The capability of the software product to avoid failure as a result of faults in the software .

Maintainability concerns are quite common for open-source projects, as identified by many studies (see Spinellis et al.). Reliability in the Eclipse context gains some importance due to the bundling of many different components in a single stack of software: the failure of a single component may affect the whole stack. In the context of Polarsys and critical embedded systems, reliability is of course even more important.

Identified Requirements for Process

Process rules are intended to garantee some common requirements for Eclipse projects like predictability and IP clearance. They are thoroughly described and commented on the public wiki in [15], with key principles of the Eclipse foundation, project structure and organisation, projects’ phases. Process-oriented rules represent most of the required constraints clearly identified for the projects.

The Eclipse development process defines maturity phases for projects with associated quality constraints. The more advanced a project is, the more tight are the quality constraints: e.g. milestones, reviews, community support. We address in this document only the incubating and mature phases because they are the most important phases to setup, and establish quality in subprojects. Pre-proposal, proposal, and top-level phases may be addressed later with a different scope.

For the Incubation phase we could identify the following requirements:

• IP Due Diligence
• Platform quality APIs .
• Extensible Frameworks and exemplary tools
• Regular Milestones
• Interim releases

For the Mature phase we could establish the following added requirements:

• Quality
• Predictability
• Communities
• Release Reviews
• Good citizen open source behaviour

Identified Requirements for Community

Eclipse communities

Communities lie at the heart of Eclipse ecosystem, and thus are often cited and required in the evaluation of a project. The Eclipse development process defines three stakeholders as target communities for all projects entering the foundation:

• Contributors and Committers A thriving, diverse and active community of developers is the key component of any Eclipse Project. Ideally, this community should be an open, transparent, inclusive, and diverse community of Committers and (non-Committer) Contributors. Attracting new Contributors and Committers to an open source project is time consuming and requires active recruiting, not just passive ”openness”. The Project Leadership must make reasonable efforts to encourage and nurture promising new Contributors.
• Users An active and engaged user community is proof-positive that the Project’s exemplary tools are useful and needed. Furthermore, a large user community is one of the key factors in creating a viable ecosystem around an Eclipse project, thus encouraging additional open source and commercial organizations to participate. Like all good things, a user community takes time and effort to bring to fruition, but once established is typically self-sustaining.
• Adopters An active and engaged adopter/plug-in developer community is the only way to prove that an Eclipse project is providing extensible frameworks and extensible tools accessible via documented APIs. Reuse of the frameworks within the companies that are contributing to the project is necessary, but not sufficient to demonstrate an adopter community. Again, creating, encouraging, and nurturing an adopter community outside of the Project’s developers takes time, energy, and creativity by the Project Leadership, but is essential to the Project’s long-term open source success.

In our context of quality assessment and for KISS reasons, we consider only two communities: developers and users (the latter actually including adopters).

• Developers, including committers, and contributors.
• Users, including Eclipse's users and adopters.

Concerns

• Activity: amount of work done recently on the project.
• Diversity: number of different actors who worked recently on the project.
• Support: how fast and complete are answers to inquiries. This has been decomposed in our quality model in two sub characteristics: responsiveness (how fast...) and support (how complete...).