From 452203feb470c6a712e5ca6a632e68cceedc7af1 Mon Sep 17 00:00:00 2001
From: younglim <younglim@users.noreply.github.com>
Date: Wed, 20 Dec 2023 17:28:12 +0800
Subject: [PATCH] Update README.md with followRobots option

---
 README.md | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/README.md b/README.md
index 2fa4f131..124e55f9 100644
--- a/README.md
+++ b/README.md
@@ -286,6 +286,9 @@ Options:
                   telemetry purposes.
                   Defaults to "{}"
                                                         [string] [default: "{}"]
+  -r, --followRobots                 Option for crawler to adhere to robots.txt
+                                     rules if it exists
+                                 [string] [choices: "yes", "no"] [default: "no"]
 Examples:
   To scan sitemap of website:', 'node cli.js -c [ 1 | Sitemap ] -d <device> -u
    <url_link> -w <viewportWidth>