Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to run on multiple directories #271

Closed
sergevonage opened this issue Jun 21, 2021 · 10 comments
Closed

How to run on multiple directories #271

sergevonage opened this issue Jun 21, 2021 · 10 comments
Labels
question Further information is requested

Comments

@sergevonage
Copy link

I have a monorepo and want run linter on dir1 and dir2. How can I do it?

working-directory: dir1 dir2

doesn't work:

working-directory (dir1  dir2) was not a path
@ChrisSargent
Copy link

I also have this question.

We have a mixed codebase like:

packages
- Go
- -- cmd
- ---- module1
- ---- module2
- Typescript

Currently, we run linting locally with a command like this from the 'Go' directory: find . -type f -name go.mod -maxdepth 3 -execdir golangci-lint run --fast \;

I have tried a few times to set it up in a better way but I always end up hitting the 'there are no go files error'.

Is there a way to install this action (so we get all the right installations and caches) but use a custom command to run?

@jonny-rimek

This comment was marked as off-topic.

@KEINOS
Copy link

KEINOS commented Dec 7, 2022

Finally, I found out how.

tl; dr

You need to place them in the args directive.

ts; dr

golangci-lint run --config ./.github/golangci.yml ./... ./_example/dir1 ./_alt/dir2

The above case will be:

name: golangci-lint

on:
  workflow_dispatch:
  pull_request:
    branches:
      - main

jobs:
  golangci:
    name: lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: golangci-lint
        uses: golangci/golangci-lint-action@v3
        with:
          version: latest
          args: --config ./.github/golangci.yml ./... ./_example/dir1 ./_alt/dir2

@serbrech
Copy link

@KEINOS I thought that the ./... would already run it on ./_example/dir and on ./_alt/dir2 since it runs it recursively on all folder below .

@KEINOS
Copy link

KEINOS commented Dec 12, 2022

@serbrech

I thought so too. But since the directory begins with "_", it omits from "./..." unless specifying them. It took me time to find it out.

@ChrisSargent
Copy link

FYI - just ran in to this issue again. Easiest workaround we found was to pass --help as an arg, which prevents the linters from actually running in this action. Then we can use our own custom commands for running it later in the job.

@ldez ldez added the question Further information is requested label Jun 24, 2023
@faddat
Copy link

faddat commented Jan 17, 2024

this is on topic as it gets (daily user)

@faddat
Copy link

faddat commented Jan 17, 2024

@serbrech -- but it does not :D

brandur added a commit to riverqueue/river that referenced this issue Feb 25, 2024
Like with running Go tests, the lints for each separate Go module must
be running separately. This is currently being accomplished in CI by
giving each Go module its own lint job using golangci-lint's GitHub
Action.

Unfortunately, this is really not working out well. Although the lints
themselves are all very fast to run (just a couple seconds), the post
run step where it saves it cache is quite slow, about two minutes, and
that's happening for each of the lint steps, making the entire lint job
take almost _ten minutes_ to run.

Here, try to hack around the problem by having the main lint job fetch
golangci-lint, but then only run `--help`, and then do the real linting
as a separate step (one that doesn't use the GitHub Action) that calls
into our `make lint` target to run lints for each Go module.

A downside is that it may not annotate lint problems on the GitHub PR,
which is something that theoretically happened before, although it never
seemed to work for me. This might not be that bad though because it
could as a side effect improve the log output, which is terrible for the
GitHub Action step because it doesn't include files or line numbers.

[1] golangci/golangci-lint-action#271
brandur added a commit to riverqueue/river that referenced this issue Feb 25, 2024
Like with running Go tests, the lints for each separate Go module must
be running separately. This is currently being accomplished in CI by
giving each Go module its own lint job using golangci-lint's GitHub
Action.

Unfortunately, this is really not working out well. Although the lints
themselves are all very fast to run (just a couple seconds), the post
run step where it saves it cache is quite slow, about two minutes, and
that's happening for each of the lint steps, making the entire lint job
take almost _ten minutes_ to run.

Here, try to hack around the problem by having the main lint job fetch
golangci-lint, but then only run `--help`, and then do the real linting
as a separate step (one that doesn't use the GitHub Action) that calls
into our `make lint` target to run lints for each Go module.

A downside is that it may not annotate lint problems on the GitHub PR,
which is something that theoretically happened before, although it never
seemed to work for me. This might not be that bad though because it
could as a side effect improve the log output, which is terrible for the
GitHub Action step because it doesn't include files or line numbers.

[1] golangci/golangci-lint-action#271
brandur added a commit to riverqueue/river that referenced this issue Feb 25, 2024
Like with running Go tests, the lints for each separate Go module must
be running separately. This is currently being accomplished in CI by
giving each Go module its own lint job using golangci-lint's GitHub
Action.

Unfortunately, this is really not working out well. Although the lints
themselves are all very fast to run (just a couple seconds), the post
run step where it saves it cache is quite slow, about two minutes, and
that's happening for each of the lint steps, making the entire lint job
take almost _ten minutes_ to run.

Here, try to hack around the problem by having the main lint job fetch
golangci-lint, but then only run `--help`, and then do the real linting
as a separate step (one that doesn't use the GitHub Action) that calls
into our `make lint` target to run lints for each Go module.

A downside is that it may not annotate lint problems on the GitHub PR,
which is something that theoretically happened before, although it never
seemed to work for me. This might not be that bad though because it
could as a side effect improve the log output, which is terrible for the
GitHub Action step because it doesn't include files or line numbers.

Also, I notice that the lints/tests in `Makefile` had been commented out
for `./cmd/river` for some reason (maybe something I did during the
driver refactor and forgot to fix), so I uncommented them.

[1] golangci/golangci-lint-action#271
brandur added a commit to riverqueue/river that referenced this issue Feb 25, 2024
Like with running Go tests, the lints for each separate Go module must
be running separately. This is currently being accomplished in CI by
giving each Go module its own lint job using golangci-lint's GitHub
Action.

Unfortunately, this is really not working out well. Although the lints
themselves are all very fast to run (just a couple seconds), the post
run step where it saves it cache is quite slow, about two minutes, and
that's happening for each of the lint steps, making the entire lint job
take almost _ten minutes_ to run.

Here, try to hack around the problem by having the main lint job fetch
golangci-lint, but then only run `--help`, and then do the real linting
as a separate step (one that doesn't use the GitHub Action) that calls
into our `make lint` target to run lints for each Go module.

A downside is that it may not annotate lint problems on the GitHub PR,
which is something that theoretically happened before, although it never
seemed to work for me. This might not be that bad though because it
could as a side effect improve the log output, which is terrible for the
GitHub Action step because it doesn't include files or line numbers.

Also, I notice that the lints/tests in `Makefile` had been commented out
for `./cmd/river` for some reason (maybe something I did during the
driver refactor and forgot to fix), so I uncommented them.

[1] golangci/golangci-lint-action#271
@brandur
Copy link

brandur commented Feb 25, 2024

We needed to be able to do this on multiple Go modules that we have in the same repository. The trick of passing args: <dir1> <dir> <dir3> didn't work because golangci-lint won't do more than one module in a pass.

Chris' answer above to configure golangci-lint's GitHub Action with args: --help to trick it into downloading the binary, not actually run, then adding a separate step to do the linting, seems to have done the trick. Full code:

      - name: Lint
        uses: golangci/golangci-lint-action@v4
        with:
          # golangci-lint needs to be run separately for every Go module, and
          # its GitHub Action doesn't provide any way to do that. Have it fetch
          # the golangci-lint binary, trick it into not running by sending only
          # `--help`, then run the full set of lints below. DO NOT run separate
          # modules as separate golangci-lint-action steps. Its post run caching
          # can be extremely slow, and that's amplified in a very painful way if
          # it needs to be run multiple times.
          args: --help
          version: ${{ env.GOLANGCI_LINT_VERSION }}

      - name: Run lint
        run: make lint

With our make target being:

.PHONY: lint
lint:
	cd . && golangci-lint run --fix
	cd cmd/river && golangci-lint run --fix
	cd riverdriver && golangci-lint run --fix
	cd riverdriver/riverdatabasesql && golangci-lint run --fix
	cd riverdriver/riverpgxv5 && golangci-lint run --fix
	cd rivertype && golangci-lint run --fix

A side effect is that it no longer annotates the GitHub PR with lint failures like the GitHub Action step would (--out-format=github-actions), but if brings filenames and line numbers back, personally I don't think that's a downside.

brandur added a commit to riverqueue/river that referenced this issue Feb 25, 2024
Like with running Go tests, the lints for each separate Go module must
be running separately. This is currently being accomplished in CI by
giving each Go module its own lint job using golangci-lint's GitHub
Action.

Unfortunately, this is really not working out well. Although the lints
themselves are all very fast to run (just a couple seconds), the post
run step where it saves it cache is quite slow, about two minutes, and
that's happening for each of the lint steps, making the entire lint job
take almost _ten minutes_ to run.

I think what might've been happening is that each lint job was
overwriting the last job's cache, which is why each post run step seemed
to be doing so much work. I didn't validate this hypothesis, but it
seems like a strong possibility.

Here, try to hack around the problem by having the main lint job fetch
golangci-lint, but then only run `--help`, and then do the real linting
as a separate step (one that doesn't use the GitHub Action) that calls
into our `make lint` target to run lints for each Go module.

A downside is that it may not annotate lint problems on the GitHub PR,
which is something that theoretically happened before, although it never
seemed to work for me. This might not be that bad though because it
could as a side effect improve the log output, which is terrible for the
GitHub Action step because it doesn't include files or line numbers.

Also, I notice that the lints/tests in `Makefile` had been commented out
for `./cmd/river` for some reason (maybe something I did during the
driver refactor and forgot to fix), so I uncommented them.

[1] golangci/golangci-lint-action#271
brandur added a commit to riverqueue/river that referenced this issue Feb 25, 2024
Like with running Go tests, the lints for each separate Go module must
be running separately. This is currently being accomplished in CI by
giving each Go module its own lint job using golangci-lint's GitHub
Action.

Unfortunately, this is really not working out well. Although the lints
themselves are all very fast to run (just a couple seconds), the post
run step where it saves it cache is quite slow, about two minutes, and
that's happening for each of the lint steps, making the entire lint job
take almost _ten minutes_ to run.

I think what might've been happening is that each lint job was
overwriting the last job's cache, which is why each post run step seemed
to be doing so much work. I didn't validate this hypothesis, but it
seems like a strong possibility.

Here, try to hack around the problem by having the main lint job fetch
golangci-lint, but then only run `--help`, and then do the real linting
as a separate step (one that doesn't use the GitHub Action) that calls
into our `make lint` target to run lints for each Go module.

A downside is that it may not annotate lint problems on the GitHub PR,
which is something that theoretically happened before, although it never
seemed to work for me. This might not be that bad though because it
could as a side effect improve the log output, which is terrible for the
GitHub Action step because it doesn't include files or line numbers.

Also, I notice that the lints/tests in `Makefile` had been commented out
for `./cmd/river` for some reason (maybe something I did during the
driver refactor and forgot to fix), so I uncommented them.

[1] golangci/golangci-lint-action#271
brandur added a commit to riverqueue/river that referenced this issue Feb 29, 2024
Like with running Go tests, the lints for each separate Go module must
be running separately. This is currently being accomplished in CI by
giving each Go module its own lint job using golangci-lint's GitHub
Action.

Unfortunately, this is really not working out well. Although the lints
themselves are all very fast to run (just a couple seconds), the post
run step where it saves it cache is quite slow, about two minutes, and
that's happening for each of the lint steps, making the entire lint job
take almost _ten minutes_ to run.

I think what might've been happening is that each lint job was
overwriting the last job's cache, which is why each post run step seemed
to be doing so much work. I didn't validate this hypothesis, but it
seems like a strong possibility.

Here, try to hack around the problem by having the main lint job fetch
golangci-lint, but then only run `--help`, and then do the real linting
as a separate step (one that doesn't use the GitHub Action) that calls
into our `make lint` target to run lints for each Go module.

A downside is that it may not annotate lint problems on the GitHub PR,
which is something that theoretically happened before, although it never
seemed to work for me. This might not be that bad though because it
could as a side effect improve the log output, which is terrible for the
GitHub Action step because it doesn't include files or line numbers.

Also, I notice that the lints/tests in `Makefile` had been commented out
for `./cmd/river` for some reason (maybe something I did during the
driver refactor and forgot to fix), so I uncommented them.

[1] golangci/golangci-lint-action#271
@ldez
Copy link
Member

ldez commented May 1, 2024

duplicate of #74

@ldez ldez closed this as completed May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

8 participants