ICE Development Process and Guidelines

The ICE development process has evolved over the years to better fit the needs of the developers and stakeholders. The process was created using two flavors of the Rational Unified Process framework (Model-Driven Systems Development and Agile Model Driven Development) with techniques from Extreme Programming and simple common sense added to keep the team flexible, lean, and responsive.

The team tries to follow the guidelines laid down in the Agile Manifesto and the Twelve Principles of Agile in so far as to make technical sense for the project. As it turns out, they almost always do, with the one exception of planning. (Working for the U.S. DOE means that we sometimes have to develop a plan and stick to it, although we hate deadlines fiercely.) We also prefer to work by the rule that "done means DONE!" such that when we say something is done it means that all the code is implemented, all the tests have passed and Jay has probably already given a demonstration of it. ;-)

This page is under construction, just like everything else on the project.

ICE is developed iteratively with each month-long iteration starting and ending on the 10th of the calendar month. Each iteration focuses on several key activities and goals. "Review, retrospect, and plan" (RRP) meetings are held as needed, but never more than once per iteration, to track progress and define the direction of the next iteration.

The ICE team meets daily at 14:00 Eastern Time in our office suite (ICE HQ) to plan activities for the day and review progress from the previous day.

The Most Important Things: Deployment and Testing

The ICE Development Team very firmly believes that working software is the primary measure of progress. Working software is to software engineering as an experiment is to science. Just as Richard Feynman said of scientific experimentation, "The principle of science, the definition, almost, is the following: The test of all knowledge is experiment. Experiment is the sole judge of scientific ‘truth’." The team believes successful deployment and happy customers are the sole judge of software development success.

The ICE Development Team practices continuous integration and test-driven development to stay on track and continuously deliver a working product. The team also requires that each developer commit a working build and that each developer test their pieces of ICE, including launching the binary, before new code or updated code is committed.

Development Phases

The development of new or existing pieces of ICE happens in phases, with activities from each phase happening within each development iteration. Activities from the different phases are not performed in a certain order in a given iteration and, for a single task, multiple activities from different phases may be performed.

1. Business modeling - High-level discussions with stakeholders about a new feature or modification that would be valuable.
2. Requirements gathering - Interviews, discussions, literature reviews and code prototyping to determine the requirements necessary to add new capabilities to, or extend the current capabilities of ICE.
3. Analysis and design - In-depth design and analysis activities for a particular capability, possibly including but not limited to: modeling, in-depth team design meetings, definition of tests and task development, and prioritization.
4. Code authoring - Production code is developed based on the requirements, the analysis of those requirements, and the new designs. Code authoring includes authoring tests.
5. Further testing and deployment - ICE is tested continuously and new binary builds are released.

Done is Done

One key principle by which the team operates is the concept of "done" meaning completely done. If a developer tells another developer that they have finished something, we expect that to mean that there is no outstanding work on that thing and that the developer is both free to move to something else and other developers can start consuming their work. A more lax definition of completion leads to a situation where plans can not be made and deadlines will always be missed. For that matter, it is much easier to plan when someone admits up front that they are not yet done because resources can be allocated as required to complete the outstanding work and no additional work will be added because the manager thought the developer was finished.

Issue Tracking

Issues for ICE are tracked at the ICE GitHub Issues page. Bugs are assigned as required with appropriate labels and milestones. Issues that were requested by users and people from outside the team should be labeled with "user request."

Milestones are tied to releases or sponsor deliverables and should only be added by ORNL staff members.

Committers may add labels as needed. Labels should clear, easy to understand, and describe the package or component that will be modified ("data structures" or "client.widgets"). The more labels assigned to an issue, the better.

Change Tracking and Version Control

ICE is managed in a Git repository. See Getting ICE for more information.

The GitHub project page also displays recent updates on the front page, as well detailed statistics about commits and traffic using the tabs located on the right-hand side. Information about the success or failure of the continuous build cycle is emailed to the ice-dev <at> eclipse.org list and archived for later review.

Task Planning and Project Management

Jay Jay Billings, Alex McCaskey and Greg Watson are the principal investigators (PI), lead architects and leader developers, and general handymen of the ICE team. They jointly act as product owners and communicate the needs of sponsors and customers to the ICE team. (Note that they are not "product owners" in the SCRUM sense because they actively participates in development.)

Deliverables and milestones for the NEAMS program are managed in the PICS:NE system and approved by the NEAMS program managers. Milestones for non-NEAMS work are managed by Jay and/or Alex, and the respective contacts on those projects.

The high-level milestones and deliverables for the different projects are used to scope the work and development activities of the ICE team. Tasks are developed and assigned at Jay's discretion, but based on the input of the ICE development team. Most of the tasks are created and assigned after a planning meeting, but some tasks are created over the course of the iteration.

Bugs and regressions are reported to the appropriate section on the ICE Bugzilla tracker. Tickets are assigned to the appropriate ICE team member or members depending on the task. Feature requests are reported to the same system and handled in the same way.

Code Development Guidelines

Developing code is only part of the whole development process in ICE and it accounts for less than half of the development activities. There are really only a few rules to remember for writing code in ICE:

1. You will not break the build.
2. You WILL NOT break the build.
3. If someone says there is not enough documentation, you have to write more.
4. Every class must be tested.
5. Only make "whole" commits to the repository, never partial commits.
6. No cursing in source code, no slang in source code.
7. Code development will go on as long as it has to.
8. If it's your first time contributing code to ICE, you have to write the code.

Each of these rules have a purpose:

• A working build is critical to successfully delivering a working product often and continuously. This is the one cardinal sin on the ICE project. Each developer must commit a working build and each developer must test their pieces of ICE, including launching the binary, before new code or updated code is committed.
• Documentation is critical to the success of any source project and we all have to do our part to make sure that ICE is fully documented.
• All classes in ICE are tested in some way, even though some classes are tested more thoroughly than others.
• All of the code for a new capability or extension should be committed to the repository at the same time to prevent bugs and to keep our build from breaking.
• The source code for ICE is developed for an audience of professionals and it should reflect that. Curse words and slang in the source code is embarrassing in that context. Slang and funny themes are allowed in tests, though, because they are not distributed with ICE and they make writing the huge tests more enjoyable.
• We avoid deadlines. Writing good code requires time and testing it requires more time.
• Every new member of the ICE development team goes through a training exercise to teach them the skills necessary to contribute code to ICE. The first piece of code that they contribute to ICE must be written solely by them to prove that they are ready to contribute to a very active project.

That all being said, these are not exact hard and fast rules. They're more like guidelines actually, and examples likely exist where all of them have been violated. Following them as much as possible, though, keeps us all working smoothly.

What Happens if the Build Breaks?

A broken build is considered the worst possible thing that can happen to our team aside from losing funding to continue doing our work, because it is one of the only few other things that can stop us from developing our product. We are a little atypical among DOE teams because of our focus on continuous integration and testing, so we get asked this a lot and the conversation goes something like this:

Asker

What happens if the ICE build breaks?

Jay

The project is headed for a disaster of biblical proportions.

Asker

What do you mean, "biblical"?

Alex

What he means is Old Testament, my friend, real wrath-of-God type stuff.

Jay

Exactly.

Alex

Fire and brimstone coming down from the skies! Rivers and seas boiling!

Jordan

Forty years of darkness! Earthquakes, volcanoes...

Taylor

The dead rising from the grave!

Jay

Human sacrifice, dogs and cats living together... mass hysteria!

Asker

All right, all right! I get the point!

Most of the time the build breaks, one or more developers will show up in the office of the offending committer or send emails to complain about the situation. Normally one or more of these developers will also join the developer who broke the build in a team programming session to fix the problem if it is not a simple fix. If the error cannot be easily fixed, the changes are reverted in the master until the build passes again and, if needed, put into a branch. The decision to make a branch is based on the size of the commit that broke the build and how much work it will take to fix it.

Reverting is a rare occurrence on the project because most of the team members can fix even very large problems in thirty minutes or less. It has only happened a few times in the history of the project. If the master undergoes a revert, it is the responsibility of the person who broke the build to fix the problem and verify it with other developers before committing to the trunk again.

Code Text Style

K&R is the default style in Eclipse and the preferred style of development on ICE. Your code can be auto-formatted into K&R by hitting Ctrl+Shift+F in the Eclipse IDE.

Eclipse Mars saw the default text width updated to 120 characters wide. We still require 80 characters wide because we have a lot of machines with smaller form factors. You can update the maximum line width in your settings by following the instructions here.

Required Code Conventions

If a class overrides Object.hashCode() or Object.equals(), then it must override both. Special care should be taken that the argument to Object.equals() is also an instance of Object and not an instance of the concrete class. Good examples of these overrides are available on the internet or in the data structures plugin.

Comparing strings should be done with care such that possibly null strings are not accessed. For example,

String value = null;
value.equals("name"); // Fails with a NullPointerException.
"name".equals(value); // Correct, does not fail. Completely valid comparison.

Naming Conventions

There are a few naming conventions in ICE. One deals with naming subclasses or interfaces. They are normally named in a way that reflects the class they are subclassing or the interface being implemented. For example, the class DataComponent reflects that it is a realization of the Component interface and the class EntryComposite reflects that it is a subclass of Entry.

This is just a convention and it is okay to name a class something else, as long as the name is descriptive.

The other set of guidelines is a much more fluid, structural approach to general naming conventions for the attributes and methods of a class. A class (or a class that implements an interface) contains a set of actions to invoke (methods) and properties. Usually, these actions modify the properties set on these classes. In essence, many of the attributes and methods in ICE are set accordingly to this metaphoric grammatical relationship: an attribute's name should be a noun, and a method a verb. In addition, these entities should also start lower case and only utilize upper case letters to symbolize spaces for phrases (i.e. camel case). Developers should avoid using underscores or any other special characters unless absolutely necessary.

Logging

System.out and System.err should never be used for logging information. Detailed instructions on the proper method for logging in ICE are documented on our Logging page.

Restricted Language Features

Some features of Java and C++ have restricted use in ICE because they encourage bad programming or are obscure.

instanceof

The use of Java's instanceof operator is strictly forbidden in ICE, unless it is used in a routine related to I/O, or a class related to the Eclipse UI. The instanceof operator is relatively efficient and makes coding easy, but that comes a cost of violating the abstraction established by ICE's class hierarchy, and the proliferation of switch statements across the code. That said, the instanceof operator is required when working with the HDF5 libraries and is the de facto standard for type discovery in Eclipse widgets. It is also used in ICE in the database routines to quickly determine whether or not a component should be skipped because it is, for example, some heavyweight thing like an IReactorComponent. If a developer feels that instanceof is required in a class, they have to bring it to Jay for approval (which was the case in all three of the above cases).

Things Jay Hates

Switches - Switches are bad because they contain very specific code and breed like rabbits if that code is needed elsewhere. So, small switches for very specific low-level purposes are OK, but large switches that define behavior needed in multiple places are forbidden because those switches will just be copied and copied and copied... leading to really gnarly and inefficient code. Some good ways to remove switches are polymorphism, the visitor pattern, or direct access to an array containing function pointers or interfaces.

Singletons - Singletons are very, very bad. In general, Singleton OSGi Services are OK and expected, but classes that behave as singletons without registering as discoverable or declarative services are forbidden because they almost always violate encapsulation and make thorough testing with fakes impossible.

Require-Bundle - Using Require-Bundle instead of Import-Package creates strong, explicit dependencies between bundles and makes it impossible to switch out services or bundles without breaking the build. Import-Package should always be used, even it it requires a few more imports than one might expect.

Rules We Break

The following are a list of rules we break and the reasons why. Most of these "rules" come from SonarQube, where we have a custom profile with them disabled.

• Returning ArrayList<T> instead of List<T> - ArrayList<T> is the List<T> of choice in ICE and we explicitly return the concrete type because we want developers to know that they can retrieve and interact with the array if they need it. This is a very common development need for scientific applications that require the best performance.
• Logging should be private static final - This is inconsistent with the guidance provided on the SLF4J website, which says that they have extensively tested their loggers and there is no reason to prefer one declaration over the other.

UML Modeling

This is a legacy section primarily maintained for information and historical value.

The Unified Modeling Language (UML) is a fantastic way to share and document ideas about software.

Using UML provides a common, graphical language in which ideas and techniques can be discussed, refined and settled upon before authoring code. This helps manage the complexity of the system by maintaining scope and traceability of the requirements. It also minimizes bloating of the source code because only the things that are modeled are implemented. In a way, this combats a problem that is the exact opposite of analysis paralysis: it keeps developers focused and on task when they are developing code and prevents authoring unnecessary functionality. Models in UML can be very detailed and may even be extended and customized to represent individual domains. A UML model also helps with test-driven development.

ICE was built from a very large UML model that was developed by the team for exactly these reasons. The UML model was translated directly to source code using model-to-source transformations over a period of three years. The team traditionally required that every part of ICE be modeled in UML, however, this was abandoned completely once ICE became an Eclipse project because all of the package names had to change. The team still requires that all of its full time developers know UML and that all new systems are modeled in UML, but that is mostly done on whiteboards and in notebooks these days.

It is important to note that it is not necessary to model every piece of every bundle. Sometimes it does not make sense to model a small class that is only used in one place, an attribute that has a very specific type, or one that is extremely private. For that matter, it also doesn't make sense to model many private operations and attributes because they expose implementation details in the model. (Protected and public operations and attributes should always be modeled because they are used by others.) The choice of what private things are modeled and how far the model goes is left up to the developer with the approval of the Lead Architect.

In some cases, entire packages may not be explicitly modeled in the UML. The item.nuclear package in the Item bundle is a good example, because the SHARP classes there do not require any special pieces, are located there mostly to be consistent with other nuclear energy packages, and are identical in design to most external ICE plugins. Again, decisions on whether or not packages like this should be modeled are left up to the Lead Architect.

Plugins that simply subclass pieces of these bundles or implement service interfaces do not require UML modeling. The Eclipse UI bundles in ICE have been extensively modeled, but UML modeling is no longer required for those bundles because of the difficulty of reverse engineering the Eclipse Rich Client Platform into a UML model.