Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrectly handling leading wildcard #34

Open
b4hand opened this issue Feb 12, 2018 · 2 comments
Open

Incorrectly handling leading wildcard #34

b4hand opened this issue Feb 12, 2018 · 2 comments

Comments

@b4hand
Copy link
Contributor

b4hand commented Feb 12, 2018

For the given robots.txt file:

User-Agent: *
Disallow: */test

The path /test should not be allowed.

@b4hand
Copy link
Contributor Author

b4hand commented Feb 12, 2018

I've done some investigation on this. It's not happening at the Directive level but at the Agent level where we normalize the paths as URLs. The string */test is being transformed into /*/test before being handed to the Directive object and obviously /*/test doesn't match /test.

@panthony
Copy link

@b4hand Hello 👋

I'm currently hitting this issue and this is kind of blocker for me, do you know why it has not been fixed after a year?

I could give it a shot but I did not do CPP since school and I was wondering if you hit something harder to fix than anticipated or just let it go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants