Skip to content

Commit

Permalink
Troubleshoot backport failures
Browse files Browse the repository at this point in the history
Lately we see a lot of backport workflows fail. The pattern always
seems to be that the first backport (to branch/v17) succeeds, but
subsequent backports (to v15 and v16) almost always fail.

The failure looks to be a GitHub-specific failure, not a generic git
failure:

    remote: fatal error in commit_refs
      To https://github.com/gravitational/teleport
      ! [remote rejected]       bot/backport-51989-branch/v15 > bot/backport-51989-branch/v15 (failure)
    error: failed to push some refs to https://github.com/gravitational/teleport

To aid in troubleshooting we:
1. Temporarily reverse the order of branches. This will tell us if it's
   always the first branch that succeeds, or if it's always branch/v17
   that succeeds.
2. Add the --verbose flag to the git push command in case we get any
   more debug info.
  • Loading branch information
zmb3 committed Feb 10, 2025
1 parent e2c9ce7 commit 4532c9c
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion bot/internal/bot/backport.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,10 @@ func (b *Bot) Backport(ctx context.Context) error {
g = b.c.Git
}

// Temporary experiment to change the order of backport attempts.
// TODO(zmb3): remove this
slices.Reverse(branches)

// Loop over all requested backport branches and create backport branch and
// GitHub Pull Request.
for _, base := range branches {
Expand Down Expand Up @@ -249,7 +253,7 @@ func (b *Bot) createBackportBranch(ctx context.Context, organization string, rep
}

// Push the backport branch to Github.
if err := git("push", "origin", newHead); err != nil {
if err := git("push", "--verbose", "origin", newHead); err != nil {
return trace.Wrap(err)
}

Expand Down

0 comments on commit 4532c9c

Please sign in to comment.