Skip to content

Triage

Jump to bottom
Ray Rischpater, KF6GPE edited this page Jul 7, 2020 · 152 revisions

Guidelines for triage

Triaging issues

The process of triaging bugs is to first go through these bug lists and make sure they have all been processed as described below:

  1. recently filed bugs with none of the classification labels.

  1. older bugs with none of the classification labels.

  2. bugs with no priority.

Provide an initial priority for the issue. In general, the priorities you should consider are:

  • P4 for bona fide bugs in current functionality (including regressions)
  • P5 for feature requests.
  • P6 for feature requests we're unlikely to work on in the immediate future, according to our Roadmap.

If it looks like it's critical --- as in something that stops further builds or is a major regression in core functionality affecting all users, consider the P0 label. If you assign the P0 label to a bug, you must reach out to a prospective owner to pass the baton for investigation. Try the hackers channel on the Discord if you don't know someone who can help, or reach out to @kf6gpe, @pcsosinski, or @hixie. The P0 bug is reserved for things that "stop the presses", as they also block further rolls to customers.

Like P0, we reserve P1 and P2 labels for things that should surface in our weekly triage meeting. In general, P1 issues are bugs blocking top-tier customers from shipping immediately or breaks in core functionality affecting all users, and P2 issues are bugs that block top-tier customers from shipping soon, or breaks in core functionality affecting most users.

P3 issues are issues that have been triaged up from P4 by team triage.

Finally, one should look at the least recently updated bugs of any kind and see if they are still relevant, applying new labels, updating the subject line if necessary, providing an update if necessary, etc.

General

First, look at the bug report, and try to understand what the described problem is. Edit the original comment to remove boilerplate that the bug reporter didn't remove. Edit the original comment to add backticks (```) around blocks of stack traces, code, the output of shell scripts like flutter doctor, etc. Ensure that the title is a meaningful summary of the issue. These changes make the bug much easier to manage.

If their report is unclear, doesn't give sufficient steps to reproduce, or is otherwise lacking in sufficient detail for us to act on it, add a polite comment asking for additional information, add the waiting for customer response label, then skip the remaining steps.

If the bug is still unclear -- we have previously asked for more detail, and the bug reporter has had a chance to provide additional feedback, but has not been able to do so in a way that makes the bug actionable -- either apologize for us not being able to fix it and then close the bug, or add the waiting for customer response label, depending on your confidence that the reporter will be able to eventually provide sufficient detail. Then, skip the remaining steps. It is fine to be aggressive in closing bugs where the issue is not clear, because we have plenty of other issues where the bug is clear and there's really no value to us in having low-quality bugs open in our bug database.

If the issue describes something that you know for a fact has been fixed since the bug report was filed, add a cheerful comment saying so, close the issue, and skip the remaining steps.

If the bug is clear enough for us to act on it, continue with the following steps. To reach this step, the bug should be actionable, with clear steps to reproduce the problem. We have enough bugs filed that we will not run out of work any time soon; therefore, it is reasonable to be quite aggressive in establishing if a bug is sufficiently clear.

Duplicates

If you recognize that this bug is a duplicate of an existing bug, add a reference to that bug in a comment, then close the bug. Skip the remaining steps. As you triage more and more bugs you will become more and more familiar with the existing bugs and so you will get better and better at marking duplicates in this way.

Requests for help (documentation issues)

If the bug report is a question, then it probably belongs in Stack Overflow or on our #help channel or some other forum for getting help. However, if it appears that the reporter did try to read our documentation to find the answer, and failed, or, if you look in our documentation and find it is inadequate here, then please consider it a documentation bug (and update the summary accordingly).

If you are confident our official documentation (on flutter.dev or api.flutter.dev) fully answers their question, then provide a link to the relevant page and close the issue, being very polite and asking them to reopen if the documentation is not sufficiently clear for them.

Labels

General rule: The more labels an issue has, the better! See also: List of labels

If any of the labels prefixed with "a:" apply to the issue, add those labels. If the bug is specific to a particular platform, consider adding one of the "platform-*" labels. If it's specific to running on a desktop platform, include "a: desktop" as well. If it's an issue involving integrating Flutter as a library into an existing mobile application, add a: existing-apps.

Add any of the applicable "severe: *" labels; typically only one will apply but sometimes severe: regression will apply in conjunction with one of the others.

If it's a very serious bug that should block releases, consider adding the P0 label as well. This label is reserved for build breaks, regressions in metrics, issues causing data loss for end users, etc. that would effectively block us from shipping tip of tree to users.

If it seems to affect a lot of people but doesn't quite rise to the level of a release blocker, add the customer: crowd label (which means it affects many people) and then either the P1 label (if it's blocking people) or the P2 label (if it's not blocking people but is nonetheless serious). This will flag the bug for consideration during weekly triage (see the second section of this document). For more details on the "crowd" designation, see Issue Hygiene.

If it's an issue that is repeatedly frustrating developers when using non-experimental Flutter features (e.g. performing mobile app builds, dealing with text input, using supported IDEs), and is widespread, add a: annoyance. If unsure, consult with DevRel (filiph@).

If it's something that looks trivial to fix, add the easy fix label.

Finally, label the issue based on what area of the project the bug relates to:

  • If it's a documentation bug, add one of the labels with the "d:" prefix.
  • If it's a bug with the underlying engine, add engine and optionally one of the bugs with the "e:" prefix. Consider one of the bugs with the "dependency" prefix if the bug is related to one of those areas.
  • If it's a bug with the Flutter framework, add framework and optionally one of the bugs with the "f:" prefix.
    • If it's specific to Material widgets or the Gallery, include the f: material label.
    • If it's specific to iOS widgets, include the f: cupertino label.
  • If it's a bug with plugins, add plugin and optionally one of the labels with the "p:" prefix.
  • If it's a bug with packages, add package and optionally one of the labels with the "p:" prefix.
  • If it's a bug with the "flutter" tool, add tool and optionally one of the labels with the "t:" prefix.
  • If it's specific to writing desktop Windows, macOS, Linux, or Web apps with Flutter, add the a: desktop label.
  • If it's related to a specific platform, add one of the "platform" labels. In particular, issues involving the Web backend should have the platform-web label.
  • If it's related to our testing infrastructure (LUCI, Cocoon, devicelab, Cirrus, etc), add the team: infra label.
  • If it's related to the developer tools -- that is, if it has or is going to receive t: devtools, or Android Studio, move it to the flutter/devtools repository for further triage, unless it looks like the first step is a change to the core parts of Flutter (engine or framework).

Bugs relating to the website should be moved to the flutter/website repo.

Bugs relating to the IDEs should be moved to the flutter/flutter-intellij repo.

Issues requiring domain expertise

Sometimes it's not clear what the appropriate labels are for an issue, or you suspect it may be a duplicate but you lack enough domain knowledge to know for sure. If you know someone who has the expertise to triage the issue, you can assign the issue to them and add the label assigned for triage, so the person knows that they are not expected to fix it, but to triage it.

Additional comments

If you have something to say regarding the bug, for example if you happen to notice what the problem is, or if you have some insight based on having seen many other bugs over time, feel free to add a comment to that effect. Your experience is valuable and may help both the reporter and the rest of the Flutter team.

Triaging PRs

The process for triaging PRs is to look at this list:

  1. the PRs with none of the classification labels.

When triaging PRs, it's important that incoming PRs:

  • Contain tests for the functionality they're providing. If it's a bug fix, there should be tests to ensure that we don't regress the bug fix. If it's new functionality, the functionality should have tests as well.

  • Code and tests should match our Style guide. Look carefully for spelling mistakes and violations of our coding style; invite the contributor to fix these.

For PRs, each PR should match one of the following categories:

  • It can be in a specific repository (e.g. the website or engine repository) that has a narrow scope. These do not need labels. (Only the "flutter" repository is considered to not have a narrow scope currently.)

  • It can have one of the following labels: framework, f: material design, f: cupertino, tool, platform-web, a: desktop, team: infra, a: existing apps.

  • It can have the will need additional triage label. Use this if it's not clear what label it should have.

Critical triage

News

We also want to regularly check https://developer.apple.com/news/ for updates that might affect us.

Issues

When doing the critical issue triage (as opposed to the new issue triage as above), we check each bug in the following lists:

  • P0: all bugs should be assigned and progress should be being made on all of them.
  • P1: all bugs should have an appropriate milestone (typically the current milestone, or for issues that we cannot do immediately, "Unassigned customer work"); bugs we are working on should be assigned; if there hasn't been an update in the last two weeks, an update should be posted.
  • P2: all bugs should have an appropriate milestone.
  • Bugs and PRs flagged for additional triage: figure out what should be done with the bug, then remove the will need additional triage label.
  • First triage of the month: check if the current milestone needs to be closed.
  • Overdue milestone: all bugs should have their milestone and assignee updated. If the issue isn't being worked on, it should be unassigned and moved to a long-term milestone; do not merely keep pushing every issue to the next milestone.

In addition, a bot takes care of these, but it's good to check on them:

  • Bugs pending feedback: close bugs with outstanding requests after 4 weeks, remove the label for bugs with answers.

Occasionally, when time allows, the following are worth taking a look at also:

  • Flakes: Check a few of the oldest flakes: are they still a problem (or is the test enabled and working now?). Is anyone looking at them?
  • Upcoming milestones: all bugs should be assigned to engineers who have acknowledged that this is what they'll be working on.
  • Most requested features: none of the top 20ish bugs are new, and they've all been considered before and have appropriate milestones.
  • Most requested bugs: nothing surprising in the list; consider adding customer: crowd P2 labels.
  • Oldest bugs; see if they can be closed or updated.

PRs

We should (but currently do not) regularly check that the following PRs are not being neglected, and have appropriate labels:

Area-focused triage

We intend for each area of the product to go through the following triage regularly:

  • Look at open PRs and review them.
  • Look at open bugs and determine what needs to be worked on.

It is recommended to do these in separate dedicated meetings. For teams with multiple areas of focus (e.g. design languages), it's recommended that each area of focus have its own meeting.

Within each meeting, the links below provide the order in which issues should be looked at, to best align with the team's unique needs.

Teams can use a variety of tools to prioritize bugs.

  • Teams should use the priority labels P0...P6.
  • Another tool is the Projects page on GitHub. Feel free to create new projects as desired, and use it as you see fit.
  • In some circumstances, we can create new milestones that parallel the dated milestones. Contact @Hixie before going down this route.

Engine

PR triage

Issue triage

If you come across a bug that is unrelated to the engine, remove the engine label and leave a comment explaining why. That will send it back to triage.

Design languages

Material Design

If you come across a bug that is unrelated to material design, remove the f: material design label and leave a comment explaining why. That will send it to the framework triage process.

Framework (excluding Material Design and Cupertino)

If you come across a bug that is unrelated to the framework, remove the framework label and leave a comment explaining why. That will send it back to triage.

Tool

If you come across a bug that is unrelated to the tool, remove the tool label and leave a comment explaining why. That will send it back to triage.

Web

If you come across a bug that is unrelated to the Web backend, remove the platform-web label and leave a comment explaining why. That will send it back to triage.

Mobile Platforms

Plugins, Packages, and Ecosystem

Plugins

Packages

If you come across a bug that is unrelated to our plugins or packages, remove the plugin or package label and leave a comment explaining why. That will send it back to triage.

Desktop

If you come across a bug that is unrelated to desktop app development, remove the a: desktop label and leave a comment explaining why. That will send it back to triage.

Infrastructure

If you come across a bug that is unrelated to the testing infrastructure, remove the team: infra label and leave a comment explaining why. That will send it back to triage.

Adding a new team

To add a new team:

  1. Create a new section above, copying-and-pasting one of the existing sections, and update it accordingly.
  2. Update all the parts of this wiki page that are labeled (in the source) with <!--ALL-LABELS-->.

Flutter • Write test, find bug • Embrace the yak shave • Our commitments to racial equity • https://flutter.dev/

Flutter Wiki

Process

Framework repo

Engine repo

Android

Plugins and packages repos

Infrastructure

Release Information

Experimental features

Clone this wiki locally