Issue Labels

Issue labels

Issues can be given a number of different labels, in the following categories. All issues should be given one type label. Other labels are optional.

type

This label defines what type of issue it is; all issues should be given one of these type labels. We give an issue just a single type label, based on the best fit for that issue. (One reason for just assigning a single type label is that it facilitates periodically reviewing issues one type at a time: if an issue had multiple types, we would waste time reviewing it multiple times.)

• Issues requiring code (or documentation) changes

  • type: bug: Something is working incorrectly. This may cause incorrect science results - in which case it should also be given the tag: bug - impacts science label - or may have some other impact, such as the model crashing. If this causes significant problems in important model configurations, the issue should also be given the tag: bug - critical label.

  • type: code cleanup: Code changes whose primary purpose is to improve the internal code structure (i.e., refactoring). These may result in behavior changes (e.g., answer changes due to reordering calculations), but these behavior changes are incidental rather than the main purpose of the code changes. This can also include additions or edits needed in code comments.

  • type: documentation: Additions or edits needed in user-facing documentation. This includes the user's guide, tech note, wiki, README files, and xml files. In most cases, issues with this label have no impact on the running of the code and should not require testing. However, this is not always the case: for example, changes in xml files require at least some testing.

  • type: enhancement: Code additions/modifications that bring in new capabilities or improve the behavior of existing capabilities. The changes can be of a scientific or purely software (e.g., user interface) nature; if this involves major new science, the issue should also be given the tag: enh - major new science label. Note that, if the issue mainly involves changing internal code structure rather than external behavior, it should be given the type: code cleanup label rather than this one.

  • type: tests: Additions or changes to tests (either unit tests or system tests). (Other issue types, such as enhancements, often involve adding tests as well. Issues with the type: tests label are just about adding/changing tests.)

• Issues not requiring code (or documentation) changes (at least, not initially: e.g., a "discussion" or "needs investigation" issue could lead to the need for changes later) (note: these labels have a prefix of - simply to force them to all appear together in GitHub's alphabetic sorting)

  • type: -discussion: This issue's resolution involves discussion within the github issue page.

  • type: -investigation: A task that involves some investigation. For example, this could involve performance evaluations. As opposed to type: discussion, there will often be significant work needed to close this issue. Once this is resolved, other issues (enhancements or bugs) may be opened based on the findings.

  • type: -support: A user or developer has run into trouble or needs support with getting something to work.

priority

This label describes where this issue should fall on the priority list - i.e., whether it should be addressed soon, or whether it can wait for a while. Issues not given a priority label implicitly have a medium priority.

• priority: high: This issue should be addressed soon

• priority: low: This issue can be deferred for a while

blocked

This label describes various reasons why an issue can't be dealt with immediately. We can periodically review issues with these labels to see if the condition blocking a given issue no longer applies.

• blocked: answer changing: This issue would be relatively easy to resolve, but we are waiting to resolve it until we can accept answer changes on master. (Note: many answer-changing issues will NOT have this label; this is mainly used for issues for which the only thing stopping us from fixing it right away is that it changes answers.)

• blocked: dependency: This issue depends on some other issue, so we won't start working on it until that issue is resolved. Both issues should reference the other issue somewhere in their comments:

  • The blocked issue should use the phrase Blocked by #XXX in a comment

  • The blocking issue should be given the label, tag: blocker, and should use the phrase Blocks #XXX in a comment

closed

These labels can be applied to issues that are closed without being fixed.

• closed: duplicate: This issue is a duplicate of another issue. The duplicate issue number should be noted in an issue comment.

• closed: invalid: This is not really an issue. For example, someone may have thought there was a bug, but in fact the code is operating as intended or there was user error.

• closed: moved: This is an issue with some other component, so the issue has been moved to a different repository.

• closed: wontfix: We have decided not to fix this bug, not to implement this enhancement, etc.

tag

These are miscellaneous labels that can be applied to any issue to help people find certain classes of issues later.

• tag: blocker: This tag should be used for issues that are blocking some other issue. The other issue should have the label, blocked: dependency. This issue should use the phrase Blocks #XXX in a comment. The purpose of this label is to help us remember to remove the blocked: dependency label from the other issue once this one is resolved; in addition, by calling out that this issue is a blocker, we may give it higher priority.

• tag: bug - critical: This tag can be applied to bugs, and means that this bug causes significant problems in important model configurations.

• tag: bug - impacts science: This tag can be applied to bugs, and means that this bug causes significantly incorrect results in the model's science, at least in some configurations. Bugs that cause the model to crash are not given this label: this is reserved for problems where the model runs but gives incorrect results (which is often worse than crashing).

• tag: enh - major new science: This tag can be applied to enhancements, and means that this enhancement brings in major new science capabilities.

• tag: good first issue: This would be a good issue for someone who is just starting to contribute to CTSM.

• tag: next: We have identified this as an issue that should get some attention soon (e.g., in the next week or two). This differs from priority: even low priority issues may be tagged next, e.g., if we want to look back and see if this is still an issue. Often an issue will be given this a label during an issue triage to note that someone should look at this and either resolve it or decide what to do about it.

  • This label should be removed once the near-term action has been taken, so that only issues that still need some near-term action keep this label. (e.g., if you look into an issue and decide it should be low priority to actually resolve it, then the tag: next label should be removed.)

  • Not all issues that are being actively worked on need to have this label: this label is meant as a convenient way to remember what we have decided to work on next, but if you don't need this label in order to remember the issue, then there's no need to apply it.

• tag: simple bfb: This issue should be relatively easy to fix and should not change answers for any configuration (i.e., it is bit-for-bit). Therefore, it is a good candidate for being combined with other bit-for-bit issues in running the test suite.

• tag: support tools only: Code changes that only affect auxiliary support tools, without affecting the main CTSM code or the standard input file support chain. This would include tools in the "contrib" directory as well as other auxiliary tools that don't have standard tests and are not used as part of normal operation of either CTSM or the creation of surface datasets. For example, this includes refactor tools, tools to create lists of input files for XML, etc.

Some good starting points for issue searches

Issues that should be reviewed regularly

Before doing this search

Before doing the below search, we should first review any outstanding next issues to see what near-term priorities still need to be done (or if there are any next issues that should have that label removed):

is:issue is:open label:"tag: next"

Search for issues that should be reviewed regularly

A starting point for this search is:

is:issue is:open -label:"priority: low" -label:"blocked: dependency" no:project

On top of that, we will often filter on specific types, etc.

Notes on what this excludes

This excludes:

• Low priority issues: these should be periodically reviewed, too, but at a lower frequency

• Issues that are blocked by other issues: these should periodically be reviewed to check whether the blocking issue has been resolved, but they don't need to be included in our standard issue triage

• Issues in projects: these are excluded under the assumption that projects have their own prioritization / triaging process

  • Note about adding issues to projects: Adding an issue to a project removes it from our normal triage process. So, if there is an issue that you want to include in a project but also want to include in the normal triage process (typically because it is useful to resolve this issue for general reasons as well as for that one project), then introduce a note in the project that references the issue, but don't actually add the issue to the project. For example, if an issue is partly for the sake of improving performance but also partly for some other purpose, then do not add that issue to the performance project; instead, add a note in the performance project that references this issue.

Strategies for dealing with issues during an issue triage

During an issue triage, we aim to either increase or decrease the visibility/priority of most issues; the "no action" choice should only be done for a minority of issues. Possible actions are:

• To increase an issue's visibility/priority, do one or more of the following:

  • Add the priority: high label

  • Add the tag: next label, indicating that someone should look at this more in the next week or two

  • Assign it to someone (which generally indicates that that person should work on the issue relatively soon)

  • Add it to the Upcoming tags project

  • Add the tag: simple bfb label, so that it will be considered for the next "simple bfb" tag

  • Add the tag: good first issue label, so that it will be considered for new contributors to CTSM

• To decrease an issue's visibility/priority, do one or more of the following:

  • Close it, adding the appropriate closed label

  • Add the priority: low label

  • Add the blocked: dependency label

• Other actions:

  • Add the issue to a project; then, in the future, the issue will be considered in the planning process for that project rather than our standard triaging process; however, see the note above about adding issues to projects

  • Do nothing (this should only be done for a minority of issues)

No type label

All issues should have a type label. Those that don't may be missed in issue reviews.

is:issue is:open -label:"type: -discussion" -label:"type: -investigation" -label:"type: -support" -label:"type: bug" -label:"type: code cleanup" -label:"type: documentation" -label:"type: enhancement" -label:"type: tests"