Architecture Council/Meetings/API Deprecation 20080119

Meeting Title:	AC Call on API Deprecation
Date & Time:	Monday January 19, 2009 at 1600 UTC / 0800 SFO / 1100 Ottawa / 1700 Berlin HTML | iCal
Dial-in:	(+1) 613.287.8000 (Ottawa and international) or 866.362.7064 (toll-free North America) passcode 464440#
Backup Dial-in:	International +44 (0)1452 567588 / Freephone +1 (866) 6161738 / Passcode: 0587322148 #

Attendees

• Doodle Poll for scheduling

EclipseSource	RT, PDE	Chris Anisczyk, Jeff McAffer (part time)
IBM	Eclipse Project	John Arthorne, Boris Bokowski, Philippe Mulet, Mike Wilson, Darin Wright
OC Systems	TPTP	Oliver Cole
STAR	WTP	Dave Carver
Wind River	DSDP	Martin Oberhuber, Michael Scharf
Obeo	Modeling, strategic	Frederic Brún

Missing: BIRT, DTP, STP, Tools, Technology

Notes

• See Architecture Council/Minutes January 15 2009#New Topics for previous discussions
• Part of the larger story on Eclipse Adoption.

Summary

Summary may not always be accurate. See the #Raw Notes below for details.

• Eclipse APIs become ever more complicated and duplicated, causing bloat and making it harder to understand.
  • Jeff: Putting the bar too high for new API hinders innovation - having an "API exit strategy" helps creating new stuff more easily.
  • Boris: Removing stuff is a value/cost decision. We need to have numbers how many clients are affected to estimate the value/cost.
  • zx: We need to have communication channels to announce new/changed API and to collect feedback.
  • Michael: The problem is legacy binary bundles which likely do not give feedback but are affected. How can these get notified and provide feedback? We may need tooling for binary bundles. Not in PDE API Tooling yet, WTP might help.
• Martin: Why haven't we been deprecating so far, how can we sell deprecation to the Community?
  • Eclipse Platform promise was that stuff stays around forever
  • Clear statements about what deprecation means - how long will stuff stay around? might be forever in some cases, based on feedback
  • Clear Feedback channel and time how long feedback can be given
  • Tools to detect breakage, and to ease migration
  • Soft deprecation (for cases where the new story doesn't cover all cases) vs. hard deprecation
    • Deprecation is just a tag (with pointer to alternative API) at first
    • Jeff, John: There are techniques for backward compatibility such as separate bundles for old stuff, or fragments
• What could deprecation policies look like?
  • Deprecated stuff may get removed after 2 release trains (subject to feedback)
  • May get removed on the next major version bump (what this means may vary from project to project)
    • Bumping the Major always causes a ripple effect -- see the recent ICU 4.0 problem due to non-Eclipse-Versioning-Scheme
  • Are train projects allowed to use deprecated API, yes or no?
    • Not allowing puts higher burden on train projects since it forces them to act. This makes deprecation harder to sell.
    • Whether they use it only matters if the consumer isn't evolved at the same time as the producer. Does that matter to end users?

Questions

• Should the deprecation rules be the same for all projects, or different from project to project?
  • We want consistency across Eclipse, but some projects are younger / more agile than others.
• By what channels can the Community be informed about deprecating API?
• How long does deprecated API need to stay around? Does it depend from case to case?
• What's the role of the Release Train?

Next steps

• Martin created bug 261544 for follow-up discussions on bugzilla
• Boris to provide input on Thought Experiments (what if ... )

Raw Notes

• For consistency, we should have a consistent policy across Eclipse
• Been extremely scared about deprecating (or even removing) things in the past ... want us not to be scared too much, a Policy will help
• Boris: Not removing stuff was not due to being scared, but because of the sheer number of adopters... need to balance cost of removing API against the value of API. Take the version numbering guidelines for example: removing stuff requires bumping the major version, which will cause a major ripple effect!
  • One easy answer is: deprecate API as it makes sense, and when bumping the major anyways, get rid of all deprecation. Bump the major when we think there's enough value.
• Chris: Another approach is to remove deprecated stuff after one release
• Oliver: Automated tooling to announce deprecated stuff, allow clients to lobby
• John: There are approaches to get the best of both worlds. In Equinox, put new API into a separate bundle (which allows to remove the entire old bundle, but still have it available as a separate bundle for those who need it). Works well when there are no singletons, but won't work when deprecating just few methods. Another method is using fragments.
• Boris: How much API would we want to get rid of? And how much value we'd get from that?
• Darin Wright: No Tooling for this in PDE API Tooling yet.
• Dave Carver: WTP tooling might help. Need scans to get numbers about how much deprecation we talk.

• Martin: In Galileo, we cannot get rid of things but we can deprecate stuff. In the past, mere deprecation has led to an outcry ("Preferences"). Deprecating NOW could answer Boris' question ("how much"). How can we address community concerns around deprecation?
• Dave: Sun / Internet research: Been deprecating after 1 release / after 2 releases.
• Michael: Even for the Java SDK, some deprecated things are still in there...

• Darin: It's harder than just talking about deprecation... we have lots of IStuff2, IStuff3, ... also, in terms of functions we have lots of support for non-OSGi plugins
• Martin: So we know what we'd want to deprecate... What's the reason why we don't go ahead and deprecate right away?
• Philippe: We agree (as a group) that deprecation doesn't give a license to kill. We need to agree to a policy.
• Martin: It's a chicken-and-egg problem: without deprecating, we won't find out what people really need.
• Philippe: Deprecating stuff is: telling people what we don't like to see what they care about.
• McQ: When we want to take stuff into a new direction, we have to have the option of deprecating. Would like to say a promise of "two release trains" for deprecated stuff.
• Boris: Platforms are in a different boat... Windows, for instance, doesn't get rid of stuff within 2 years.
• Michael: "Live" plugins are not the problem. Old legacy plugins are the problem.
• Jeff: Not talking about "internal" stuff. Our current mindset is "don't make it API unless you're 100% sure". This is a damper to innovation! New approach exemplified in bug 253777 asynchronous IFuture. The interest in adding new API is tied to "what happens to old API when it is wrong", i.e. the API Policy.
  • We need to make it easier to add API when we think it's useful and reasonable (even if not 100% perfect).

• Dave: Unclear policies in WTP

• Martin: Why don't we just get deprecated in and wait for response?
  • Jeff: deprecation needs to contain a story what to do instead.
  • Michael: Binary plugins won't get notified of deprecation!
  • Philippe: To-date, our rule in the Platform has been that "nothing is ever going to go away". That's why we are reluctant deprecating stuff.
  • Martin: why don't we seed the process by marking stuff as deprecated... not necessarily saying that it's going to go away.
  • Michael: Even if we don't necessarily retire deprecated stuff, deprecation is helpful for users.
  • John: Two different cases: Deprecating just to tell users there's something better, or deprecating in the intent to remove stuff.
  • McQ: In the first case, there's no reason not to do it. In the second case, there is a threat to all adopters when they have to change.
  • Martin: Do we need a separate "soft deprecation" tag for the 1st case?
  • Boris: Only reason for "soft deprecation" is (in Platform UI) when we have a new story that covers most of the stuff, but not all corner cases (i.e. the replacement story is not complete). What advice do we give (as AC) for cases like this?

• Boris: Want a document like "this is the simple rule how long deprecated API is going to stay around".
  • Suggestion: Remove when the version number is bumped.
• McQ: What does that mean for random projects? May I just get rid of stuff when I bump my major?
  • Boris: ICU4J doesn't follow the Eclipse version number scheme... stuff stops working with major bump
• Martin: Create an API Deprecation Document on the Wiki
  • McQ: How are we going to enforce control over the rules
  • Martin: Should it be the same for all projects? Some projects may want to be more agile than others, and that may be OK with their communities.
  • Boris: Write down some thought experiments, e.g. "What would happen if (e.g. EMF) would change its API in non-compatible ways?
  • Darin: "Eclipse Pure API" tag for train projects?

• Martin: We agree that we want deprecation, how can we best sell it to the Community in a positive way?
  • Soft deprecation
  • Feedback channels / scanners
  • Promise to keep deprecated around
  • Highlighting the Values of the new stuff (such as simplified, smaller API)

• McQ: We should expect that release trains work.
  • Talking to people involved in actively developed projects is easy. What to do about legacy ones?
  • We ought to make the Train work as a group, and that needs putting restrictions on projects