Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Zulip tests and re-enable #441

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

gnprice
Copy link
Member

@gnprice gnprice commented Jan 15, 2025

This has the update to unblock autocomplete changes:
zulip/zulip-flutter#1190
flutter/flutter#143249

and with that, tests pass again on Flutter main.

Also install libsqlite3-dev on Linux, a package which is no longer
included in GitHub's new version of the "ubuntu-latest" images. The
lack of it causes nondeterministic failures when the runner happens
to use the new image:
#441 (comment)

The actual shared library file libsqlite3.so.0.8.6 -- in the package
libsqlite3-0 -- is still installed by default anyway. This package
adds the symlink at "libsqlite3.so", which is the filename some
libraries apparently look for.

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read the Flutter Style Guide recently, and have followed its advice.
  • I signed the CLA.
  • I listed at least one issue that this PR fixes in the description above.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is test-exempt.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

This has the update to unblock autocomplete changes:
  zulip/zulip-flutter#1190
  flutter/flutter#143249

and with that, tests pass again on Flutter main.
@gnprice
Copy link
Member Author

gnprice commented Jan 15, 2025

Tests pass in Zulip's own CI on Flutter main, after zulip/zulip-flutter#1190, so I expect they'll pass here too. We'll have confirmation on that soon.

@justinmc @Piinks I saw a mention at #437 (comment) that there was more than one upstream change these tests were blocking. I guess any others haven't yet landed, given that tests pass — what's the other PR (or PRs) involved? (I did just re-scan through Zulip's PR queue and my email, but didn't spot anything.) Happy to take updates as needed.

@Piinks
Copy link
Contributor

Piinks commented Jan 15, 2025

There were a couple PRs that were seeing test flakes actually, @gnprice do you have a sense for how flaky some tests may be in this test suite?

@gnprice
Copy link
Member Author

gnprice commented Jan 15, 2025

We've had a flaky test or two in the past, but I'm not aware of any right now. If you can track down the affected test runs or PRs, I'd be glad to investigate.

@gnprice
Copy link
Member Author

gnprice commented Jan 15, 2025

(I've also now kicked off running our test suite a bunch of times in a loop, on my own machine — so if there is a flake currently present, hopefully that will find it.)

@Piinks
Copy link
Contributor

Piinks commented Jan 15, 2025

Other than this comment #437 (comment)
I also heard about it on https://github.com/flutter/flutter/pull/161091/checks which ran into

| ✅ /tmp/flutter_customer_testing.zulip.FCQOYP/tests/test/notifications/display_test.dart: NotificationOpenPayload parse: fails when host is not "notification"
|
| ::error::2120 tests passed, 5 failed, 8 skipped.
ERROR: One or more tests from zulip failed.
Tests finished in 76.33 seconds per iteration.

@gnprice
Copy link
Member Author

gnprice commented Jan 15, 2025

Thanks — looking into those.

@gnprice
Copy link
Member Author

gnprice commented Jan 15, 2025

I've investigated and I believe there are no flakes in the tests. There is an infra issue, which caused one and probably both of the failures mentioned above. I'll deal with that.

The failure at #437 (comment) comes from libsqlite3.so being missing on the system. It's currently nondeterministic whether that's present, because this repo's CI uses GitHub's ubuntu-latest image, and they're in the middle of a rollout to change what that means:

The failure on https://github.com/flutter/flutter/pull/161091/checks quoted above is hard to pin down because the checks have been re-run since then, and there doesn't seem to be a way to get more details from GitHub (though I spent some time digging for them). Do you have a more complete log than that snippet?

Lacking more info on that one, though: "5 failed" is exactly the number of tests that would fail for lack of libsqlite3.so, so with no sign of another issue it's probably that one.

Then to actively search out any flakes:

This package is no longer included in GitHub's new version of the
"ubuntu-latest" images.  That causes nondeterministic failures when
the runner happens to use the new version:
  flutter#441 (comment)

The actual shared library file libsqlite3.so.0.8.6 -- in the package
libsqlite3-0 -- is still installed by default anyway.  This package
adds the symlink at "libsqlite3.so", which is the filename some
libraries apparently look for.
@gnprice
Copy link
Member Author

gnprice commented Jan 15, 2025

Fixed that infra issue by installing the package again. Also updated the PR description so it provides an accurate commit message for a squash-merge commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants