COSMOS dev process

Introduction

This document describes the development process for the COSMOS project. The intent is for this document to complement the Eclipse Development Process.

Development & Testing

3rd party code

3rd party code is defined as code, tests, documentation etc. that you did not write, even if it comes from an EPL Eclipse Project.

All 3rd party code must be noted in our IP log, and we must keep the IP log current in order to avoid a last-minute rush right before we try to release.

• If 3rd party code becomes obsolete, open a bug against cosmos.web to have the item marked obsolete in the IP log.
• If 3rd party code needs to move from one location, e.g. plug-in, to another, open a bug against cosmos.web to have the two about.html files for those plug-ins updated
• If you need to change the version of 3rd party code, or add a dependency on new 3rd party code, check with the Archiecture Lead first to get an architectural review. Once the Architecture Lead has given permission, open a bugzilla and notify Ruth. Be aware that if Eclipse Legal does not give full approval -- parallel IP approval is not enough -- of the item that any work that depends on that 3rd party code cannot be part of the release.
• If the 3rd party code comes from a fellow contributor, the contributor must attach a patch containing the code to bugzilla, and after the committer has reviewed and approved that patch, the committer must mark it with the iplog+ flag. Note that these contributions must be approved by the Technology PMC or the Project Lead before they are committed to the database. By marking the bugzilla patch with iplog+, the contribution is automatically added to our IP log.
• You must follow Eclipse Legal's instructions. For your convenience, this is their poster, but you are required to follow all instructions, not just this summary. The famous IP process poster

Enhancement Requests

• At any time during the release cycle, enhancement requests can be submitted by creating a Bugzilla entry.

Defect Reporting, Fixing and Verification

Please reference the Eclipse guidelines for the lifecycle of a bugzilla entry available at Bugzilla Use. These are part of the overall Eclipse Development Process. Another good source of information on enhancements/bug is the Bug Reporting FAQ.

Reporting:

• At any time during the release cycle, defects can be reported by opening a Bugzilla entry against the appropriate component.

Fixing and Verification:

1. When a developer decides to work on the defect he/she assigns the defect to him/herself, completes the fix and checks it in to CVS
   1. The developer must include the Bugzilla entry number and summary in the CVS comment during the CVS checkin. This step is important because these comments will be automatically extracted by the build process to produce a list of bugs that have been fixed in the next build.
2. When the fix has been checked in, the developer marks the defect as FIXED
   1. The testing and verification information (e.g. specific test cases) MUST be included in the bugzilla entry as a comment when the developer marks it "Fixed"
   2. When the test cases (JUnit/Manual) are checked in, they MUST specify the bugzilla entry number that they test
   3. The test case description MUST include the bugzilla entry that it addresses
3. After the next build, the user who opened the defect tests to ensure the fix is correct, and marks the defect as VERIFIED
   1. If the user is not available/capable to verify the code, then it is the responsibility of the component owner to validate the code and move the bugzilla entry to "Verify"
4. Prior to the start of the next iteration, the team lead reviews and closes all verified defects

Iterations

• An iteration is a 4-6 week period consisting of requirements, design, code and test activities, the output of which is a stable, integrated and tested release of the system.
• The number and schedule of iterations will be determined by the project team at the beginning of the release cycle and documented in the Release Plan.

Required steps per iteration:

1. Identify and accept requirements

• The Community will identify the set of use cases that should be targeted for this iteration based on the milestone goals, and create associated enhancement requests. (Note: Use cases are documented on the COSMOS Use Cases wiki page.)
• Developers will be assigned to create high-level designs, dependency analysis and sizing estimates for each enhancement request.
• The Community should review open enhancement requests (and associated designs and sizing estimates), assign a priority based on the release themes and use cases, and assign them to a development resource. Each enhancement request should be associated with a specific use case, as defined in the previous bullet.
• The priority of enhancements should be marked according to the following definitions:
  • P1 - This is a must-have enhancement OR this enhancement blocks another enhancement that has a priority of P1 or P2.
  • P2 - This is a high priority enhancement, key to satisfying the goals of the iteration.
  • P3 - This is a medium priority enhancement that can be moved to a future iteration with minimal impact to the goals of the current iteration.
  • P4 & P5 - This is a low priority enhancement that will be addressed as time and resources permit.
• A review of the requirements list for all subprojects will be completed during the COSMOS Community call prior to the start of the iteration for final acceptance.

2. Complete Designs

• Assigned developer(s) should create designs and post them on the COSMOS wiki for Community review.

3. Complete Coding/Implementation

A. Assigned developer(s) should check code into CVS to be integrated into the build.

• Developers should reference the Eclipse Development Conventions and Guidelines for consistent development. Deviations or clarifications are noted as follows:
  • COSMOS API Contract describes package naming conventions used in COSMOS
• Developers are encouraged to synchronize their development environments with CVS at least weekly.
• Developers are encouraged to check in small increments of code (250 lines or less) on a frequent basis. Each check-in does not have to represent a completed function; however, the code must compile successfully.
• Per the Eclipse Guidelines, a contribution form must be completed for any contributions over 250 lines of code.
• All files should have appropriate copyrights. Developers can periodically use the Eclipse copyright tool to ensure this happens.
• If a developer modifies a file that contains a copyright from a company other than his/her own employer, the developer should make sure that "and others" is added after the company name, and that the developer's company contribution is added to "Contributors".

Example:

A developer from IBM modifies a file created by and previously only modified by CA Inc.

/*******************************************************************************
 * Copyright (c) 2008 CA Inc. and others.
 * All rights reserved. This program and the accompanying materials
 * are made available under the terms of the Eclipse Public License v1.0
 * which accompanies this distribution, and is available at
 * http://www.eclipse.org/legal/epl-v10.html
 *
 * Contributors:
 *     CA Inc. - initial API and implementation
 *     IBM Corporation - features xx
 *******************************************************************************/

• The developer must include the Bugzilla entry number and summary in the CVS comment during the CVS checkin. This step is important because these comments will be automatically extracted by the build process to produce a list of bugs that have been fixed in the next build.

B. When the code has been checked in, the developer marks the enhancement as FIXED

• The testing and verification information (e.g. specific test cases) MUST be included in the bugzilla entry as a comment when the developer marks the enhancement "Fixed". To help maintain a stable code base, test information is required for check-ins of full enhancements and for partial enhancements that others are dependent on.
• When the test cases (JUnit/Manual) are checked in, they MUST have the bugzilla entry number specificed that they test
• The test case description MUST include the bugzilla entry that it addresses

C. After the next build, the user who opened the defect or enhancement tests to ensure the fix is correct in the build, and marks the bugzilla entry as VERIFIED

• If the user is not available/capable to verify the code, then it is the responsibility of the component owner to validate the code and move the bugzilla entry to "Verify"

D. Prior to the start of the next iteration, the team lead reviews and closes all verified enhancements and defects

4. Complete Test Pass

• During the iteration test phase, the code base is frozen.
• Only fixes for problems found during testing should be checked in during the test pass. All code must be reviewed and approved before checking in. Developers should post request for approval in the bugzilla entry and post a note to the cosmos-mgmt mailing list detailing the changes that need to be checked in, the reason this change is critical for the current iteration and the risk associated with introducing this change in the build, using the stop ship template. The Release Engineering team will be notified of any approved fixes.
• Each subproject should run the appropriate JUnit or manual tests. JUnits should be run individually so that results can be reported accurately.
• Once a subproject team determines that the test has completed successfully, the team lead should post a message to the dev mailing list indicating that the JUnit or manual tests for the subproject have been completed.
• Once all subprojects have confirmed successful completion of the test, the release engineering team will declare a candidate build and the QA team will be notified to begin QA test activities.
• The QA team will open bugs for any problems found during their testing activities and post daily progress reports.
• If a fix is approved for check-in, the Release Engineering team will declare a new candidate build which includes the fix.
• Once the QA team completes their testing activities, they will post a final test pass report and the COSMOS PMC will begin evaluation of the iteration exit criteria.

Iteration Exit Criteria

• All P1 and P2 enhancements complete
• No P1 or P2 defects and 100% test attempt. Any exceptions require Community discussion and PMC sign-off.
• Test results posted; All exceptions reviewed before deferral/discharge

TPTP Test Framework Overview

TPTP Test Framework Overview

Daily Builds

Release Engineering Team

• The Release Engineering Team will be responsible for seeing to it that the builds and tests run to completion.
• In the event of problems with the build & test system itself they are expected to resolve them as soon as possible and get the build and test restarted.
• In the event of problems with the build caused by developer checkins they are expected to follow up with developers to ensure fixes are made quickly, and then get the build and test restarted.
• Any build failure will be reported as a bugzilla to the build component for tracking of the actual issue as well as trend analysis, etc.

Ownership

• All committers are responsible for clean builds and tests. Clean daily builds and tests keep the system stable, prevent regressions, and avoid down time. Anytime you check something in, you are responsible for monitoring the builds and tests until they pass. This means that for at least 24 hours after your checkin, you are available, responsive and prepared to help with any issues that crop up.

Daily Builds

• Every 24 hours at GMT-5 10:00 PM (ET) the state of the head branch of the CVS repository will be labeled with a time stamp
• The time stamp will be encoded with the release number, date, time, and serial build number: COSMOS-RMm-YYMMDDHHMMSS
• A module list will be maintained (in CVS) that lists all the modules that are part of the current build, labeling and extracting will be restricted to those modules. This could be a PDE build style map file or .pfd file. This map file will be named COSMOS-RM.m.{map|pfd} and will be labeled with the time stamp at build time as well.

Daily Tests??

• Where are build and/or test results posted?

Weekly Integration Builds

Weekly Integration Builds

• Weekly integration builds will be run every Tuesday at 12:00 p.m. ET
• Code cutoff for integration builds is as follows:
  • All patches should be in bugzilla by Monday at 12:00 p.m. ET
  • All code should be checked into CVS by 11:45 a.m. ET on Tuesday
  • CVS will be locked until the build has completed successfully (typically for 1 hour). No code should be checked in from 11:45 a.m. on Tuesday until notification of build completion has been sent out.
• Each subproject team should run JUnits on the candidate build and post results by 10:00 a.m. on Wednesday. (Note: The Data Visualization team will perform a smoke test on the UI since the tests are manual. Also, the use of TPTP for generating reports is not required for weekly integration builds.)
• RE team will post notification to cosmos-dev when the integration build is ready - target availability is 4:00 p.m. on Wednesdays

Milestone Release Builds (MS)

• Milestone releases, which will be spaced out across the release schedule, contain complete DCUT (code complete, unit tested) increments of functionality (they are a subset of the full functionality of the target release).
• A set of use cases will be identified and documented on the COSMOS Use Cases wiki page for each milestone release.
• Milestone releases are numbered RVM.m.M1 through RVM.n.Mn (R1.0.M2 for example).
• A milestone release is created by relabeling and renaming a good build, updating the downloads page and sending out an email saying the milestone release is available. The contents of the milestone are determined by the project schedule.
• Users can look at the download page and find latest milestone release.
• Notification of new milestones should be posted to the web site and COSMOS newsgroup and blog with a high-level list of significant features and fixes

Milestone Release Exit Criteria

• All P1 and P2 enhancements complete
• No P1 or P2 defects and 100% tests attempted. Any exceptions require Community discussion and PMC sign-off.
• Test results posted; All exceptions reviewed before deferral/discharge
• No usage of internal API outside of your own component, even within COSMOS
• No plug-ins or features are missing About.html or other legal information
• Copyright statements are present and correct in all files
• All strings have been externalized as the team developed
• All logging is consistent with COSMOS logging techniques
• All bugzillas that are complete have been closed
• All contributions from non-committers, if used, have been marked iplog+
• Clean API: all API documented, package names following the API naming conventions, etc.

Release Candidate Builds (RC)

• Near the end of the project when all tests have been run, bugs fixed, and the builds are stable, the release engineering team will promote a release candidate build.
• Release Candidates are number RVM.m.RC1 through RVM.n.RCn (R0.4.RC7 for example).
• A release candidate is created by relabeling and renaming a GOOD build, updating the downloads page and sending out an email saying the release candidate is available.

GA Builds (GA)

• A GA build is created by relabeling and renaming a GOOD release candidate build, updating the downloads page and sending out an email saying the GA release is available.
• A release candidate build can be declared GA when the following criteria have been satisfied:
  • No high severity defects and 100% test attempt/pass
  • Test results posted; All exceptions reviewed before deferral/discharge
• Notification of new GA releases should be posted to the web site and COSMOS newsgroup with a high-level list of significant features and fixes

References

• Eclipse Development Process
• Bug Reporting FAQ
• How to Internationalize your Eclipse Plug-in