Update crawler.mdx

added FAQ for password protected sites
algolia · Jan 21, 2025 · 7f670b6 · 7f670b6
1 parent 7857cf8
commit 7f670b6
Showing 1 changed file with 4 additions and 0 deletions.
diff --git a/packages/website/docs/crawler.mdx b/packages/website/docs/crawler.mdx
@@ -106,6 +106,10 @@ No. Well, you can but once you do things will not work correctly. We automatical
 
 The fastest way will be to connect with us on our [Discord](https://alg.li/discord). Alternatively, email us at the address below and we will get to it as soon as we can.
 
+## Can I use the Crawler on password protected sites?
+
+The Crawler as used with DocSearch applications cannot be used for password protected sites that require a login. If you need this functionality, you need to utilize a regular Algolia plan https://www.algolia.com/pricing and add a crawler to it. Note that while it is free to add a pay-as-you-go crawler, the free tier does have limitations.
+
 ## Links related to the migration
 
 - [Docusaurus blog post](https://docusaurus.io/blog/2021/11/21/algolia-docsearch-migration)