Skip to main content

Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Kepler Working Proposal

Context

Introduction

Community-oriented software development is widely seen as one of the most efficient, effective ways to promote both quality and reuse in the development process. Today's open source software communities provide a means to logically break down and organize otherwise monolithic collections of libraries into functionally-focused modules that are self-describing, discrete, and reusable. This community-oriented development model enables a collaborative approach to providing the common subsystems that many applications need. Harnessing the efforts of a diverse development community allows a shared project to accumulate skill, experience, and testing from all of the applications using it. By building an ecosystem of such software communities, each focused on delivering a single, well-articulated feature set, developers are free to seek the best implementation possible for each subsystem, then reuse it. Focusing each project on a single, well-defined problem promotes clearer architecture through a narrowed project scope. Such projects also tend to render a more reliable solution, since they draw on a larger set of use cases to produce a more diverse set of project tests. Additionally, as bugs surface and are fixed within these projects, the improvements benefit the entire community. Finally, the shared infrastructure code provided by these projects leads to greater consistency across grouped or related applications , and makes it easier for developers to integrate these projects with one another. Organizations can reap great rewards in terms of improved application quality and enhanced developer effectiveness by fostering rich, community-oriented development environments.

Unfortunately, today's development processes often impose a high price in the form of complex infrastructure and provisioning requirements, making the efficiency benefits of a community-oriented development model nearly impossible to realize. Infrastructure-related complexity is manifested in the many, fragmentary systems that currently inhibit information flow throughout the development life cycle. Further complicating matters is the considerable overhead imposed on engineering through the the micro-management of tasks and information specified by most traditional development methodologies. Every software project has the same essential requirements, such as supporting its users (architects, developers, project managers, etc.), tracking software issues and their respective resolution activities, managing requirements, and generating system designs. Whenever a new project is formed, the complexity or inefficiency involved in the existing development processes is propagated, further increasing the volume of noise that pollutes the engineering environment, inhibiting effective collaboration. In the end, the information overload suffered by community participants makes the cost of implementing a highly modular software ecosystem prohibitive. With today's project-infrastructure requirements, the ideal concept of community-oriented software development simply doesn't scale.

Objective

The purpose of Kepler is to address the complexities involved with provisioning, managing, and using a shared infrastructure in order to support a community-oriented development model. Our strategy will consist of the following:

  1. Derive an abstract set of tasks required to support software communities, based on tools commonly used today.
  2. Develop a framework to orchestrate the execution of related tasks and task groups, with the goal of enabling discovery, access, and control of any third-party, community-infrastructure system from within Kepler.
  3. Provide access to each set of framework features through a rich, Eclipse-based environment, thereby enabling users to participate in community activities without leaving their development environment.

Principles

Kepler will provide extensible frameworks and exemplary tools for participation in and management of a large, diverse software community environment. In pursuit of this mission, Kepler will be guided by the following principles:

Implementation Neutrality

We intend to create project-metadata management frameworks that provide equal access to all tools. Our goal is to integrate and simplify the development process in existing environments, and the best way to achieve this is to design a framework which allows existing products to integrate easily.

Open, Standards-based Orientation

This project will strive to use open standards for all information storage and communications APIs, in order to facilitate tighter integration with third-party tools. Where existing standards exist, we will attempt to use them. Where they do not exist, or are deemed unworkable, data formats used will be published and will reflect industry best practices for such designs.

Eclipse Community Involvement

We believe the success of Kepler is contingent upon successful integration with the rest of the Eclipse community. In order to achieve the level of usability required to serve the user community properly, it is critical to engage the broader Eclipse development community and reuse whatever components, technologies, and techniques that make sense.

User Community Involvement

The purpose of Kepler is to make community-oriented software development simpler. Moreover, we believe it is essential that Kepler integrate well with existing Community Infrastructure Tools tool-sets. In order to support this, it is critical to maintain tight bonds with our user community, to be characterized by constant feedback and discussion between project developers and project users.

Usability Focus

Perfect integration of third-party tools is of very limited value if the integrated environment is itself unusable. Therefore, Kepler will maintain a strong focus on the experience of the target audience when designing both user-interface components and the integration framework itself.

Transparency

In the tradition of the Eclipse community, key discussions and documentation will be easily accessible to the foundation membership at large.

Incremental Delivery

Due to the many potential aspects of community-oriented development which this project may improve, we believe that it is critical to provide concrete, usable features as they become available, without waiting for a complete feature set. In this way, incremental roll-out of successively larger feature sets will give the developer and user communities an earlier opportunity to shape Kepler, and make it the best it can be. MUST, SHOULD, MAY Specification Semantics

In order to support the most advanced features of the most advanced tools applicable to each framework, each feature will strive to mark each requirement as one of MUST, SHOULD, or MAY to indicate the features that are required for minimal participation via the framework, and which additional features are supported by the framework for tools that can provide them. These feature annotations are provided in an attempt to avoid the least-common denominator effect, where a framework is limited by the feature-set of the most rudimentary tool to be integrated. See RFC 2119

Scope

Kepler intends to start with small, high-impact steps to make concrete progress toward its goals as soon as possible. As new opportunities to support software communities are identified, the implementation of the overall Kepler project will gradually expand within each of its main feature categories to provide a richer toolset to its users. However, regardless of the scope of implementation, the project will remain true to one goal: minimizing the complexity involved with managing and participating in an ecosystem of distinct, but interdependent, software communities. The following are an initial list of feature categories to address some areas of community-oriented development in which we believe Kepler can make significant early progress:

Community Project Model

In order to provide frameworks for working with community-oriented project infrastructure, and in order unite the frameworks of Kepler behind the common goal of lowering the barriers to a community-oriented development model, it is essential that Kepler frameworks operate on a common model representing community-oriented projects. To this end, Kepler will introduce a community-project model, to define and maintain the specification for projects which are candidates for community-infrastructure support. While this model may implement a concrete API reading and writing to some "native" format, it will concern itself primarily with providing a means of building a project model in memory, and adapting other systems to provide project information that can be consumed by other participants in the Kepler framework.

The community project model will implement the following requirements:

  • MUST define appropriate extension points to enable third-party tool configuration to be added to the model.
  • MUST provide mechanisms for extensibility, versioning, and backward compatibility in the project model.
  • MUST provide dependency information
    • SHOULD support dependency searching
    • MAY support optional (user-configured) substitution of workspace projects for stated binary dependencies
  • MUST provide project-identification information for integration of the project into the larger software ecosystem
  • MUST track the origins of project-model metadata
  • SHOULD provide mechanisms for modifying project-model metadata files
  • MAY provide a mechanism for writing changed metadata back to the appropriate file using existing metadata-model APIs
  • MAY provide a compound navigator or editor (embedding or linking to existing metadata editors) which allows users single-point access to all project-model data elements
  • SHOULD provide integration metadata for build servers, where applicable
    • SHOULD include any applicable notification configurations, to enable build servers to provide feedback to the community
  • SHOULD provide all applicable provisioning information for a project, including:
    • mailing list addresses (subscribe, unsubscribe, archive; one set per mailing list)
    • SCM location (developer access, anonymous access, and browsing locations, where applicable)
    • issue tracker information (system identifier and location)
    • web-site locations
    • forum location
    • committer information (including authorization or role where applicable)

Sources of inspiration for the community project model will include Apache Maven's POM (http://maven.apache.org/ref/current/maven-model/maven.html), Buckminster's Component Specification model (CSPEC), DOAP (http://usefulinc.com/doap) and Eclipse's own plugin.xml, feature.xml, and .project/.classpath/etc. project binding files, among others.

Kepler Preferences

  • SHOULD support configuration of dependency-search sources (globally, or per-project)
  • MUST support tool-specific preferences, if they are provided by the third-party integration plugin
    • (tool configuration refers to things settings required to run the build tool, such as ANT_HOME)
    • SHOULD manage tool configuration per-project, or globally (per-workspace?)

Build Management

One key feature commonly found in successful community-oriented software initiatives is the ability to build the project from source code with a minimum of required specialized knowledge or configuration effort. Aside from supporting users who choose to build from source, this feature is critical for all developers in the community, to ensure that when new code is committed, it does not introduce major problems. The tighter the integration between the project's build system and the developer's environment, the more likely developers are to run project builds, and the less likely that an integration build will fail as a result of code changes. To this end, one of the features of Kepler will be concerned with integrating with build tools, through Eclipse technologies like Buckminster, to extract any interesting community information.

The following are some initial build-management feature requirements for the Kepler framework:

  • MUST extract information from different build systems
  • MUST integrate with Buckminster for build tool support
  • MUST integrate with Problems and Errors views
  • MUST provide bidirectional synchronization of community information changes to and from the build metadata
  • MUST detect supported build files with community metadata, and add appropriate tool-configuration sections to the project model
  • MUST support editing of community metadata in build metadata using graphical tools

Build Server

Another common feature of successful community-oriented software projects is the implementation of continuous integration (CI) and other server-side build infrastructure. Performing ongoing, consistent build-and-test cycles for a project - rather than just building the software when it's time to release a new version - reduces the risks associated with a project release, and helps to ensure that the community always has up-to-date information about the health and progress of the project. Additionally, since CI builds happen frequently, developers get much quicker feedback when something breaks. With the timely feedback provided by a CI environment, developers can resolve problems as they surface in the build, while they're still relatively easy to diagnose and fix. Finally, server-side build infrastructure enables several advanced development practices, such as proactive detection of errors due to dependency upgrades and binary promotion of project build results. Another feature set of Kepler will be concerned with interacting with server-side build environments.

The following are some initial build-server feature requirements for the Kepler framework:

  • MUST Store information of the build servers type
  • MUST Store information of the location of the build servers
  • SHOULD Add a graphical view of the build servers

Project Community Provisioning and Maintenance

The ability to provision and manage infrastructure-related configuration for multiple projects in a consistent manner is one of the most intensive overhead activities for a community-oriented software ecosystem. When the ecosystem allows the promotion of community members from user to developer, such as in the open-source realm, maintaining these community-support systems becomes all the more difficult. Therefore, to lessen the burden of adopting a community-oriented development model on large scales, Kepler will introduce several project provisioning features.

The following are some initial project-provisioning feature requirements for the Kepler framework:

General Features

  • MUST support existing community-infrastructure tools
  • MUST provision new services for existing projects as needed

Project Provisioning

Initializes the following, using existing infrastructure (shared by multiple projects):

  • MUST: continuous integration (single-instance, clustered, and farmed scenarios)
  • MUST: mailing lists and associated rules
  • MUST: forum
  • MUST: integrate with Mylyn for issue tracking
  • SHOULD: wiki
  • MUST: integrate with Eclipse Team Support for version control (CVS, Subversion, etc.)
  • SHOULD: static web sites
  • MUST: Eclipse IDE configuration for the project

Project Maintenance

  • MUST support addition and removal of developers (committers)
  • SHOULD support modification of developer authorization/roles

NOTE: Our initial goal for this feature of Kepler will be to support the community infrastructure of the Eclipse Foundation itself. In order to achieve this, Kepler development team will need to work closely with the Eclipse IT team. Since the IT team is responsible for maintaining and provisioning infrastructure for the Eclipse Foundation communities, they represent a rich source of knowledge about the workflows and current tools used to manage these systems. Additionally, any implementation of the project provisioning feature which aims to serve the Eclipse Foundation will have to have access and buy-in from the IT team.

Multi-Community Visibility

One of the great costs associated with subdividing a monolithic code base into modules is the complexity involved with participating in the resulting menagerie of projects. Instead of interacting with a single system for each life-cycle task, developers must manage information in dozens. For example, where a monolithic project might house all of its open issues in a single, application-wide tracking interface, issues logged against an ecosystem of community-oriented projects are housed in multiple separate issue-tracking repositories; one per project. Needless to say, modularization under these circumstances severely complicates the development and project management tasks of any application based on such an ecosystem. Similarly, community participation may take place largely on project-specific mailing lists, making it more difficult to maintain a single conversation thread that addresses multiple projects. Even the process of searching for documentation can become quite complex, as each project may host separate forums, FAQs, and project-documentation web sites.

In order to overcome the scalability problem and realize the benefits of a community-oriented development process, Kepler will reduce the burden incurred through a multi-project infrastructure by introducing a multi-community visibility feature. This project will deliver a framework and exemplary tooling that provide integrated, read-write views of information flow among and through multiple software communities.

The following are some initial multi-community feature requirements for the Kepler framework:

  • MUST provide issue-tracker reporting (integration with Mylyn)
  • MUST aggregate and organize mailing list traffic (including easy cross-posting)
  • MUST support forum searches (including aggregation of results)
  • SHOULD support web-site documentation browsing (and search?)
  • SHOULD support aggregation of structured documents, like FAQs
  • MUST support aggregation of CI results
  • MUST support aggregated management of and reporting on project binaries

Interaction with existing projects

  • Buckminster: interaction with build systems (Maven, Ant, PDE,...)
    • Shared libraries for interaction with Maven used by Kepler and Buckminster are going through the IP verification process
  • Mylyn: interaction with issue tracking systems
  • Eclipse Team Support: interaction with source control management systems

Organization

This project proposes to work closely with individuals and organizations, both inside and outside the Eclipse Foundation, in order to arrive at an architecture that is relevant to as wide a community as is practically possible. To begin, the Buckminster project at Eclipse has expressed interest in exploring opportunities to integrate with and help shape the features detailed above. Proposed Project Lead and Initial Committers

  • Carlos Sanchez, DevZuz - Project Lead
  • Philip Dodds, DevZuz
  • Brett Porter, DevZuz
  • Mike Lim, Exist
  • Glen Gonzales, Exist
  • Maria Odea Ching, Exist
  • Jonas Lim, Exist
  • Cata Pineda, Exist
  • Erle Mantos, Exist
  • Genesis Deiparine, Exist
  • Thomas Hallgren, Cloudsmith
  • Henrik Lindberg, Cloudsmith

Mentors

  • Thomas Hallgren, Cloudsmith
  • Oisin Hurley, IONA

Interested Parties

The following organizations and individuals have expressed interest in this project:

Initial contributions

DevZuz is contributing the initial implementation mainly consisting on the community model and EMF representations and Maven metadata adaptor.

Retrieved from "https://wiki.eclipse.org/index.php?title=Kepler_Working_Proposal&oldid=288030"

This page was last modified 17:46, 1 February 2012 by Eclipsepedia anonymous user Unnamed Poltroon.

Back to the top