Lychee not detecting markdown files

[vipulgupta2048]

I am running Lychee as a GitHub action on https://github.com/balena-io/docs/, Lchee's GitHub action got updated v1.2.0 which lead to the pattern **/*.md failing.

For some reason, this in the file https://github.com/balena-io/docs/blob/master/.github/workflows/link-checker.yml

with:
           args: >
             --config ./lychee.toml
             --github-token ${{ secrets.GITHUB_TOKEN }}
             **/*.md

translates to just finding 2 markdown files in the repository when the action is running. Refer: https://github.com/balena-io/docs/runs/4626369590?check_suite_focus=true#step:5:19

lychee --format markdown --output /tmp/lychee/out.md --config ./lychee.toml --github-token *** pages/404.md shared/icon.md

I am trying to debug if there is something on our end that changed or is it Lychee's action update that caused this. I ran Lychee locally to confirm the pattern lychee --no-progress --config lychee.toml **/*.md and that worked just fine. Do let me know if you need any more information to help debug this further.

[vipulgupta2048]

This issue might have been better off in https://github.com/lycheeverse/lychee-action/issues instead. I thought I was opening it there, can admins transfer this to the other repository.

[mre]

lychee **/*.md uses the glob expansion from your shell while lychee "**/*.md" uses lychee's internal glob expansion (with the glob crate). So if you use a different shell locally than gets used in Github Actions, the result might be different. Try quoting the glob for consistent behavior.
Unfortunately there's not much we can do other than making sure that all examples show quoted glob expressions. We'll probably also switch the default action to checking all markdown and html files by default thanks to the new lychee ., which walks over the directory tree recursively and in parallel so it is much faster than glob. This is available in the latest lychee master, but not in the action yet.

So in summary, can you try quoting the glob until we publish a new version of the action?

[vipulgupta2048]

You got it, I created a PR following your suggestion: balena-io/docs#2168
Here's the error I see.

[vipulgupta2048]

Ran and approved your PR run: https://github.com/balena-io/docs/pull/2169/files

[mre]

I'm afraid that will probably not work. :/
There's a syntax error in the if condition, but more importantly I couldn't make it work yesterday in #69. You can check at the commits to see what I tried.

• unquoted -> your current issue
• '**/*.md' -> recursive wildcards must form a single path component
• './**/*.md' -> this is what the error above tries to tell us I guess, but it now finds no files

I'm a bit out of ideas for the moment as to what this could be.
From what I heard, globset behaves slightly differently, so perhaps we should try that next.

[mre]

See also: rust-lang/glob#83

We wanted to move to globset at some point anyway, because then we can support multiple glob patterns and exclusions: lycheeverse/lychee#284

[mre]

@vipulgupta2048, #73 should fix it. It was a combination of issues. To make sure this doesn't happen anymore in the future, I've added some integration tests.
I'll go ahead and merge it so you can give it a shot by using lycheeverse/lychee-action@master.

[mre]

Merged to master now. Please let me know if you encounter any additional issues. If not, I'll release a new version from master soon.

[vipulgupta2048]

Hey @mre Thanks for your effort on this.
master version works now, check out the run here: https://github.com/balena-io/docs/runs/5093030921?check_suite_focus=true
Except for a few thread panic errors that I see in the logs, I think the glob paths work after putting quotes around them.

[mre]

Yeah the thread panic is because of seanmonstar/reqwest#1399. Not much we can do about it other than waiting or maintaining a fork of reqwest with a patch until this gets merged. 😕 I'm actually considering to create a temporary fork just for this.

[mre]

Apart from that, thanks for the feedback. 😄

[vipulgupta2048]

Right on right on, looking forward to the new release for the action.

[mre]

Alright, new release is out. 😄 https://github.com/lycheeverse/lychee-action/releases/tag/v1.2.1