Add E2E tests for bevy build
#222
Merged
+5,013
−2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
TimJentzsch
added
A-CI
TimJentzsch
added
the
S-Needs-Review
3 tasks
BD103
reviewed
Jan 24, 2025
DaAlbrecht
reviewed
Jan 26, 2025
- Switch to`assert_cmd` - rename the `dev` tests to `debug` - rename `clean_target_binaries` to `clean_target_artifacts` and require the path to be already set to the target instead of joining inside. - Simplify `executable`
DaAlbrecht
approved these changes
Jan 26, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Objective
Closes #221.
Improve our ability to catch errors in the Bevy CLI by using end-to-end tests, which install the CLI and execute it on a test repository.
This should help us prevent errors like #218 in the future.
Solution
Add a new test suite for
bevy buildin the integrationtestsfolder.It defines some utilities for the tests and then installs the CLI and executes
bevy buildcommands on a test repository.The test repository is currently a simple workspace with a singular Bevy application.
The Bevy app is just the minimal default setup.
In the future, we can expand this or add more packages to test more behavior.
The tests assert that the command exits successfully and that the expected binaries are created.
It tests dev and release profiles as well as the
webequivalents.The binaries are cleaned up before the tests to ensure that the checks work as intended, but other build artifacts are retained across runs to speed up the tests.
In the future, we might have to adjust our caching strategy for the integration tests, as the test repo uses a different target directory.
To avoid the tests conflicting with each other, I use
serial_testto prevent them running in parallel.We also now need to install the Bevy dependencies on Linux in CI, I adopted the strategy used by the Bevy repo.
Note
Keep in mind that running the tests will now overwrite your installed Bevy CLI version.
Future work
bevy build web --bundlebevy runandbevy run web(they don't terminate, so are harder to test)