Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ccl/crosscluster/physical: TestStreamingRegionalConstraint failed #129614

Closed
cockroach-teamcity opened this issue Aug 25, 2024 · 5 comments
Closed
Assignees
Labels
branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 branch-release-24.1 Used to mark GA and release blockers, technical advisories, and bugs for 24.1 branch-release-24.2.1-rc C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. P-3 Issues/test failures with no fix SLA T-disaster-recovery

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Aug 25, 2024

ccl/crosscluster/physical.TestStreamingRegionalConstraint failed on release-24.2.1-rc @ 23a8ffc28d2f31eef0cd13eca69693dc249d1196:

=== RUN   TestStreamingRegionalConstraint
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestStreamingRegionalConstraint1898422763
    test_log_scope.go:81: use -show-logs to present logs inline
    replication_stream_e2e_test.go:1386: -- test log scope end --
test logs left over in: outputs.zip/logTestStreamingRegionalConstraint1898422763
--- FAIL: TestStreamingRegionalConstraint (182.16s)
    --- FAIL: TestStreamingRegionalConstraint/fromSys=true (102.59s)

Parameters:

  • attempt=1
  • deadlock=true
  • run=2
  • shard=9
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/disaster-recovery

This test on roachdash | Improve this report!

Jira issue: CRDB-41615

@cockroach-teamcity cockroach-teamcity added branch-release-24.2.1-rc C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-disaster-recovery labels Aug 25, 2024
msbutler added a commit to msbutler/cockroach that referenced this issue Aug 26, 2024
msbutler added a commit to msbutler/cockroach that referenced this issue Aug 26, 2024
@exalate-issue-sync exalate-issue-sync bot added the P-1 Issues/test failures with a fix SLA of 1 month label Aug 26, 2024
@msbutler msbutler added P-2 Issues/test failures with a fix SLA of 3 months and removed release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. P-1 Issues/test failures with a fix SLA of 1 month labels Sep 26, 2024
@exalate-issue-sync exalate-issue-sync bot added P-3 Issues/test failures with no fix SLA and removed P-2 Issues/test failures with a fix SLA of 3 months labels Dec 10, 2024
@tbg
Copy link
Member

tbg commented Dec 19, 2024

I just had this fail here: here on #137750. I have no idea what the error is. I downloaded the artifacts and the logs all look like a clean shutdown.

@dt
Copy link
Member

dt commented Dec 19, 2024

In the full test shard log we see testcluster.go:485: unable to retrieve conf reader: span configs not available, but the output in that area looks a little funny -- maybe something somewhere printed 108 to stderr at the wrong moment -- so engflow isn't correctly including that line with the specific test's logs and you only see it in the whole shard log.

That line does however correspond to a fatal during server start:

		if err := tc.WaitForFullReplication(); err != nil {
			tc.Stopper().Stop(ctx)
			t.Fatal(err)
		}
	}```

@dt
Copy link
Member

dt commented Dec 19, 2024

I don't think the failure @tbg linked to has anything to do with this test. I just saw the same error in a failed test in kvcoord's TestMuxRangeFeedDoesNotStallOnError over here: https://mesolite.cluster.engflow.com/invocations/default/69c843f7-e5a7-4ff6-9a70-65b9b58bf0ef

@arulajmani
Copy link
Collaborator

arulajmani commented Dec 20, 2024

The span config related failures should be papered over by 71cdabc.

msbutler added a commit to msbutler/cockroach that referenced this issue Dec 23, 2024
Copy link

blathers-crl bot commented Dec 23, 2024

Based on the specified backports for linked PR #137925, I applied the following new label(s) to this issue: branch-release-23.2, branch-release-24.1. Please adjust the labels as needed to match the branches actually affected by this issue, including adding any known older branches.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@blathers-crl blathers-crl bot added branch-release-24.1 Used to mark GA and release blockers, technical advisories, and bugs for 24.1 branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 labels Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 branch-release-24.1 Used to mark GA and release blockers, technical advisories, and bugs for 24.1 branch-release-24.2.1-rc C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. P-3 Issues/test failures with no fix SLA T-disaster-recovery
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

5 participants