WTP/Conventions of bug priority and severity

Purpose of this Document

This document is to expand on the meanings of Priority and Severity in bugzilla, as those fields are used in WTP, by convention.

These fields are important to "get right" for many reasons. For one, it helps make sure important bugs are not "lost" in the chaos of a release. Also, for people who open bugs, it is important to be accurate so their reputation of "providing good bug reports" is untarnished. Let's be honest ... someone who repeatedly opens bugs as critical which are not critical (by our conventions) begins to be treated as The Boy Who Cried Wolf.

How to set Severity and Priority

As stated in the Bug Reporting FAQ

Severity is assigned by a user and describes the level of impact the bug is having on them. Priority is assigned by the developer and describes the importance a developer places on fixing the bug. P1 is the highest priority, P5 is the lowest.

Bugzilla's own documentation gives a general definitions of priority and severity which is pretty much what we follow in WTP. But below is additional information that will hopefully provide guidance for more accurate and consistent use.

Severity

Severity is set by the originator and should accurately reflect the impact the bug has on the originator. Commiters will usually not change a bugs severity setting, honoring the originators trouble to set it in the first place, but we will if it is obviously incorrect and there is no response to questions or requests for more information about why the severity was marked as it was. It is in the originators long-term best interests to set this field correctly so they maintain a good reputation for good bug reports. You might get extra attention once by marking something critical when it is not, but then from then on get even less attention if it was felt the severity was greatly exaggerated.

Severity	Standard Bugzilla definition	WTP convention
Blocker	Blocks development and/or testing work.	Same meaning in WTP. Notice it would be rare for an end-user to have a 'blocking' bug. This isn't the common English definition of "blocks me from doing what I want" but instead blocks extenders development or adopter testing. For example, for an end-user, if the XML editor did not open at all, that might be 'blocking' but even if validation or content assist did not work at all, it would not be 'blocking'. But, if an adopter was trying to do final test of some validation issue, and it did not work at all, then that would be blocking. Also note: if there is any workaround found for a blocking issue, then it is no longer blocking and severity field should be corrected.
Critical	Crashes, loss of data, severe memory leak.	critical refers to the impact of the bug ... Crashes, loss of data, severe memory leak ... not its importance to the originator (use bugzilla votes and brief prose comments to express your perceived importance).
Major	Major loss of function.	Major is the severity to use when something seriously gets in the way of getting your work done. For example, can not run on server, no content assist at all.
Normal	Regular issue, some loss of functionality under specific circumstances.	Same meaning in WTP. This is default severity. Use it if you are not sure. This is also typically the severity to use if there is an error in one of our APIs (e.g. incorrect parameter) or an error in what is documented for the API (e.g. returns null even though documented not to return null).
Minor	Minor loss of function, or other problem where easy workaround is present.	Problems which are worth reporting (and fixing) but do not really interfere with normal work or use.
Trivial	Cosmetic problem like misspelled words or misaligned text.	Not often used. Most cosmetic problems in WTP are considered at least minor bugs. A good use of trivial, though, is to report things like spelling mistakes in JavaDoc.
Enhancement	Request for enhancement.	Use this to mark requests for completely new things. Sometimes there's a gray area between what is "completely new" versus what is "expected, but found to be missing". The later could be considered a 'normal' bug, so if some funtion really was expected and wasn't there then open as a 'normal' bug and committers will say if it really should be an enhancement request. A request to provide an API would normally be an enhancement request.

Priority

Priority is set by Project Lead or the bug's assignee. It should reflect the order that things are worked on. So, P1 bugs should be addressed first, etc. For that same reason, the priority of bugs may change over time. For example, during every milestone's planning phase, it is often useful to go through all the bugs in a component, marking some as P1 for that milestone, some as P2, etc. Once those are gone, then plan another set to work on next. Similarly, occasionally after work starts on a bug, it may be decided it is not really a P1 or P2 after all (for example, it is discovered it requires much more work than at first thought).

Priority	Standard Bugzilla definition	WTP convention
1	most important	Must fix for next release or milestone. Literally. We will slip dates for P1's, so use sparingly.
2		Very important for next release or milestone. Nearly all P2's should be released, so do not over use.
3		Of normal importance. Default starting point. While triaging a bug it is good to consider its priority and adjust higher or lower if appropriate.
4		Important, but less so. Would expect to be fixed someday.
5	least important	A valid bug, but no plans to investigate further or fix. All 'P5' bugs also imply "helpwanted", but "helpwanted" can also be used on any other priority.

Related Documentation

WTP Bugs: Workflow and Conventions

Hotbug Policy

Bug Reporting Tips References