We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is there an easy way in the DSL to write write rules for all user agents instead. So instead of...
googlebot do ... end
...or...
user_agent[:googlebot, :bingbot] do ... end
...is there an equivalent of the hard-coded phrase Use_agent: * that would allow any user agent at the top of robots.txt ?
Use_agent: *
robots.txt
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Is there an easy way in the DSL to write write rules for all user agents instead. So instead of...
...or...
...is there an equivalent of the hard-coded phrase
Use_agent: *
that would allow any user agent at the top ofrobots.txt
?The text was updated successfully, but these errors were encountered: