chore(ci): run lint in a separate build before running the tests #2876
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for testcontainers-go ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
mdelapenya
force-pushed
the
run-lint-first
branch
from
2fc8641 to
e30d9a0
Compare
This reverts commit e30d9a0.
It will create a nicer UI render
v1v
reviewed
Nov 13, 2024
Co-authored-by: Victor Martinez <[email protected]>
* main: fix: update GH actions for uploading/downloading artifacts (testcontainers#2888) feat(ci): Enable master moby with rootless (testcontainers#2880)
* main: fix: do not run sonar upload when ryuk is disabled (testcontainers#2889)
* main: fix: only upload to sonar on ubuntu-latest (testcontainers#2891) fix: build artifact name properly (testcontainers#2890)
mdelapenya
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
It extracts the golangci-lint step to a separate job, so that it can be called separately before the tests.
To detect the changes, we have created a shell script,
scripts/changed-modules.sh. This script receives one single env var as input parameter,ALL_CHANGED_FILES, which on CI will be provided by a Github action, https://github.com/tj-actions/changed-files, that puts in there a list of the modified files, comparing the current PR changeset with the parent branch (main). We can tune this up to always check the latestmainbranch (open to discussion here).The script will compare all existing modules (looking up all the
go.modfiles of interest, no test files) with the modified files, building an array of modified modules. At the moment there is modified file in the core, in other words the entire root directory without modulegen/modules/examples, all the modules will be included in the build. There are some exclusions for the code: files in the.devcontainer, .vscode, docs, scriptswill not cause all modules will be included. So it could be the case that no modules could end up in the build if only files in those exclusions are changed. In that case, we are going to conditionally skip the lint, test and upload-to-sonar jobs.Finally, for running the tests, we are going to keep the matrix for running them in multiple Go versions and OSs, with the following rules:
ubuntu-latest.ubuntu-latest,macos-latestandwindows-latest.BTW I was able to iterate faster on this at the moment I discovered the power of https://github.com/nektos/act. Fantastic tool!
Why is it important?
There are two major goals:
As a drawback, in my opinion, is that two different builds with two different changesets won't run the same set of tests, so they are not deterministic. I think this is a minor detail that would be satisfied with the improved experience for module contributors at CI time. For core maintainers, we will probably run all the modules per PR, but that could be expected to verify we are not breaking the downstream modules.
How to test this PR
The shell script is the key part, and it's explained in the comments for the script. It requires running the script with the required environment variables for the different use cases:
A Go file from the core module is modified:
ALL_CHANGED_FILES="examples/nginx/go.mod examples/foo/a.txt a/b/c/d/a.go" ./scripts/changed-modules.sh
The output should be: all modules.
A file from a module in the modules dir is modified:
ALL_CHANGED_FILES="modules/nginx/go.mod" ./scripts/changed-modules.sh
The output should be: just the modules/nginx module.
A file from a module in the examples dir is modified:
ALL_CHANGED_FILES="examples/nginx/go.mod" ./scripts/changed-modules.sh
The output should be: just the examples/nginx module.
A Go file from the modulegen dir is modified:
ALL_CHANGED_FILES="modulegen/a.go" ./scripts/changed-modules.sh
The output should be: just the modulegen module.
A non-Go file from the core dir is modified:
ALL_CHANGED_FILES="README.md" ./scripts/changed-modules.sh
The output should be: all modules.
A file from two modules in the modules dir are modified:
ALL_CHANGED_FILES="modules/nginx/go.mod modules/localstack/go.mod" ./scripts/changed-modules.sh
The output should be: the modules/nginx and modules/localstack modules.
Files from the excluded dirs are modified:
ALL_CHANGED_FILES="docs/a.md .vscode/a.json .devcontainer/a.json scripts/a.sh" ./scripts/changed-modules.sh
The output should be: no modules.
Follow-ups
IMPORTANT!: We need to update the branch settings so that the new checks are used to allow the merge.