Add "-k", "-j" flags, display urls in reports and updated form submis…

…sions (#136) * Processing of results into basic form & then encoding it to send to Google Form * Updated Google Form Submission field (htmlSnippets) * Mandatory "-k" flag in node cli to submit name and email to Google Form * Relevant change to flow of questions in node index to submit name and email to Google Form * "-j" flag in node cli to enter preferred label for custom flow scans * Include question in node index to enter label for custom flow scans * Changes in report to display list of urls scanned and custom flow label if indicated by users
GovTechSG · Jul 14, 2023 · 030cf87 · 030cf87
1 parent 7a50a64
commit 030cf87
Show file tree

Hide file tree

Showing 23 changed files with 1,024 additions and 279 deletions.
diff --git a/README.md b/README.md
@@ -52,7 +52,7 @@ Purple HATS can perform the following to scan the target URL.
 - Results will be compiled in JSON format, followed by generating a HTML report.
 - To start using Purple HATS, run `node index`. Questions will be prompted to assist you in providing the right inputs.
 
-> NOTE: For your initial scan, there may be some loading time required before use.
+> NOTE: For your initial scan, there may be some loading time required before use. Purple-HATS will also ask for your name and email address and collect your app usage data to personalise your experience. Your information fully complies with [GovTech’s Privacy Policy](https://www.tech.gov.sg/privacy/).
 
 ### Scan Selection
 
@@ -61,13 +61,16 @@ You can interact via your arrow keys.
 ```shell
 % node index
 ┌────────────────────────────────────────────────────────────┐
-│ Welcome to HATS Accessibility Testing Tool!                │
-│ We recommend using Chrome browser for the best experience. │
+│  Purple HATS (ver 0.9.1)                                   │
+│  We recommend using Chrome browser for the best experience.│
+│                                                            │
+│ Welcome back User!!                                        │
+│ (Refer to readme.txt on how to change your profile)        │
 └────────────────────────────────────────────────────────────┘
-? What would you like to scan today?
-❯ sitemap
-  website
-  custom flow 
+? What would you like to scan? (Use arrow keys)
+❯ Sitemap
+  Website
+  Custom
 ```
 
 ### Headless Mode
@@ -77,10 +80,13 @@ Headless mode would allow you to run the scan in the background. If you would li
 ```shell
  % node index
 ┌────────────────────────────────────────────────────────────┐
-│ Welcome to HATS Accessibility Testing Tool!                │
+│ Purple HATS (ver 0.9.1)                                    │
 │ We recommend using Chrome browser for the best experience. │
+│                                                            │
+│ Welcome back User!                                         │
+│ (Refer to readme.txt on how to change your profile)        │
 └────────────────────────────────────────────────────────────┘
-? What would you like to scan today? Sitemap
+? What would you like to scan? Sitemap
 ? Do you want purple-hats to run in the background? (Y/n) Y
 ```
 
@@ -89,12 +95,23 @@ Headless mode would allow you to run the scan in the background. If you would li
 ```shell
 % node index
 ┌────────────────────────────────────────────────────────────┐
-│ Welcome to HATS Accessibility Testing Tool!                │
+│ Purple HATS (ver 0.9.1)                                    │
 │ We recommend using Chrome browser for the best experience. │
+│                                                            │
+│ Welcome back User!                                         │
+│ (Refer to readme.txt on how to change your profile)        │
 └────────────────────────────────────────────────────────────┘
-? What would you like to scan today? Sitemap
+? What would you like to scan? Sitemap
 ? Do you want purple-hats to run in the background? Yes
-? Please enter URL to sitemap:  https://www.sitemaps.org/sitemap.xml
+? Which screen size would you like to scan? (Use arrow keys) Desktop
+? Please enter URL or file path to sitemap, or drag and drop a sitemap file here:  https://www.sitemaps.org/sitemap.xml
+
+
+ Scanning website...
+
+
+ Fetching URLs. This might take some time...
+
 
 Scanning website...
 
@@ -114,12 +131,17 @@ If the sitemap URL provided is invalid, an error message will be prompted for yo
 ```shell
 % node index
 ┌────────────────────────────────────────────────────────────┐
-│ Welcome to HATS Accessibility Testing Tool!                │
+│ Purple HATS (ver 0.9.1)                                    │
 │ We recommend using Chrome browser for the best experience. │
+│                                                            │
+│ Welcome back User!                                         │
+│ (Refer to readme.txt on how to change your profile)        │
 └────────────────────────────────────────────────────────────┘
-? What would you like to scan today? Website
+? What would you like to scan? Website
 ? Do you want purple-hats to run in the background? Yes
-? Please enter URL of website:  https://www.sitemaps.org
+? Which screen size would you like to scan? (Use arrow keys) Desktop
+? Please enter URL of website:  https://www.domain.org
+
 ```
 
 If the website URL provided is invalid, an error message will be prompted for you to provide a valid input.
@@ -130,17 +152,21 @@ If the website URL provided is invalid, an error message will be prompted for yo
 
 ### Customised Mobile Device Scan
 
-```
+``` shell
+% node index
 ┌────────────────────────────────────────────────────────────┐
-│ Welcome to HATS Accessibility Testing Tool!                │
+│ Purple HATS (ver 0.9.1)                                    │
 │ We recommend using Chrome browser for the best experience. │
+│                                                            │
+│ Welcome back User!                                         │
+│ (Refer to readme.txt on how to change your profile)        │
 └────────────────────────────────────────────────────────────┘
-? What would you like to scan today? website
+? What would you like to scan? Website
 ? Do you want purple-hats to run in the background? No
-? Which screen size would you like to scan? (Use arrow keys) 
-  Desktop 
-❯ Mobile 
-  Custom 
+? Which screen size would you like to scan? (Use arrow keys) (Use arrow keys)
+❯ Desktop 
+  Mobile
+  Custom
 ```
 
 Choose `Mobile` for a default mobile screen size scan and `Custom` to choose a device or specify viewport width options.
@@ -150,35 +176,46 @@ Choose `Mobile` for a default mobile screen size scan and `Custom` to choose a d
 Custom flow allows you to record a series of actions in the browser and re-play them and Purple hats will trigger the accessibility scan at each step.  This is useful to scan websites that require user and form input.  The recorded script will be stored as `generatedScript*.js`.
 
 1. Start by choosing the `Custom flow` in the menu selection.
+
 ```shell
 % node index
 ┌────────────────────────────────────────────────────────────┐
-│ Welcome to HATS Accessibility Testing Tool!                │
+│ Purple HATS (ver 0.9.1)                                    │
 │ We recommend using Chrome browser for the best experience. │
+│                                                            │
+│ Welcome back User!                                         │
+│ (Refer to readme.txt on how to change your profile)        │
 └────────────────────────────────────────────────────────────┘
-? What would you like to scan today? (Use arrow keys)
-  sitemap 
-  website 
-❯ custom flow 
+? What would you like to scan?
+  Sitemap
+  Website
+❯ Custom
   ```
-2. Specify the URL of the starting page you wish to scan
-3. A Chrome and Playwright Inspector window will appear.  Navigate through the pages you would like to conduct an accessibility scan.
-4. Close the Chrome window.  Purple HATS will then proceed to re-run your recorded actions and scan each page for accessibility.
+
+1. Specify the URL of the starting page you wish to scan
+2. A Chrome and Playwright Inspector window will appear.  Navigate through the pages you would like to conduct an accessibility scan.
+3. Close the Chrome window.  Purple HATS will then proceed to re-run your recorded actions and scan each page for accessibility.
 
 Other options:
+
 - You can specify sites to exclude from accessibility scan (e.g. login page) by adding a pattern of the domain to `exclusions.txt`. An example of `exclusions.txt`:
-```
+
+``` txt
 \.*singpass.gov.sg\.*
 ```
+
 - You can re-run your accessibility scan by running `node generatedScript-PHScan_...js` file that is generated.
 
 **Caution**: During the custom flow, sensitive information such as username and passwords might be stored in `generatedScript*.js` as part of the recording.
 
 #### Known Issues
+
 If the custom flow fails to start, you might be runnning multiple versions of Playwright. Re-install Playwright:
+
 1. On Windows, delete the folder `%USERPROFILE%\AppData\Local\ms-playwright` where `%USERPROFILE%` is typically located at `C:\Users\<username>`.
 2. On MacOS, delete the folder `~/Library/Caches/ms-playwright` where `~` refers to `/Users/<username>`.
 3. Within PowerShell (Windows) or Terminal (MacOS) app, run the following two commands to re-install Playwright:
+
 ```Shell
 npx [email protected] install
 ```
@@ -209,7 +246,14 @@ Options:
   -s, --strategy       Strategy to choose which links to crawl in a website scan
                        . Defaults to "same-domain".
                                        [choices: "same-domain", "same-hostname"]
-
+  -k, --nameEmail      To personalise your experience, we will be
+                       collecting your name, email address and usage data. 
+                       Your information fully complies with GovTech’s Privacy Policy. 
+                       Please provide your name and email address 
+                       in this format "John Doe:[email protected]".                   
+                                        [string] [required]
+  -j, --customFlowLabel  Give Custom Flow Scan a label for easier reference in t
+                         he report                                      [string]
 Examples:
   To scan sitemap of website:', 'node cli.js -c [ 1 | Sitemap ] -d <device> -u
    <url_link> -w <viewportWidth>
@@ -220,6 +264,7 @@ Examples:
 ```
 
 ### Device Options
+
 <details>
   <summary>Click here for list of device options supported</summary>
 
@@ -338,6 +383,7 @@ Examples:
 - "Pixel 5 landscape"
 - "Moto G4"
 - "Moto G4 landscape"
+
 </details>
 
 If the device name contains ```(``` and ```)```, wrap the device name in single quotes when entered into the CLI.
@@ -415,6 +461,10 @@ If you find a scan takes too long to complete due to large website, or there are
 
 To do this, open `constants\constants.js` with a text editor. Change the value for `maxRequestsPerCrawl` to a smaller number like 10, e.g. `export let maxRequestsPerCrawl = 10;` and save the file. Start a new purple-hats scan.
 
+## How do I change the name and email address used for node index scans?
+
+You can edit the userData.txt which is located at %AppData%/Purple HATS on Windows and ~/.Library/Application Support/Purple HATS on Mac.
+
 ## Additional Information on Data
 
 Purple HATS uses third-party open-source tools that may be downloaded over the Internet during the installation process of Purple HATS. Users should be aware of the libraries used by examining `package.json`.

diff --git a/cli.js b/cli.js
@@ -16,6 +16,8 @@ import {
   cloneEdgeProfiles,
   deleteClonedChromeProfiles,
   deleteClonedEdgeProfiles,
+  validEmail,
+  validName,
 } from './constants/common.js';
 import { cliOptions, messageOptions } from './constants/cliFunctions.js';
 import constants, {
@@ -121,6 +123,32 @@ Usage: node cli.js -c <crawler> -d <device> -w <viewport> -u <url> OPTIONS`,
 
     return option;
   })
+  .coerce('k', nameEmail => {
+    if (nameEmail.indexOf(':') === -1) {
+      printMessage(
+        [`Invalid format. Please provide your name and email address separated by ":"`],
+        messageOptions,
+      );
+      process.exit(1);
+    }
+    const [name, email] = nameEmail.split(':');
+    if (name === '' || name === undefined || name === null) {
+      printMessage([`Please provide your name.`], messageOptions);
+      process.exit(1);
+    }
+    if (!validName(name)) {
+      printMessage([`Invalid name. Please provide a valid name.`], messageOptions);
+      process.exit(1);
+    }
+    if (!validEmail(email)) {
+      printMessage(
+        [`Invalid emaill address. Please provide a valid email adress.`],
+        messageOptions,
+      );
+      process.exit(1);
+    }
+    return nameEmail;
+  })
   .check(argvs => {
     if (argvs.scanner === 'custom' && argvs.maxpages) {
       throw new Error('-p or --maxpages is only available in website and sitemap scans.');
@@ -140,7 +168,7 @@ const scanInit = async argvs => {
   argvs.scanner = constants.scannerTypes[argvs.scanner];
   argvs.headless = argvs.headless === 'yes';
   argvs.browserToRun = constants.browserTypes[argvs.browserToRun];
-
+  
   let useChrome = false;
   let useEdge = false;
   let chromeDataDir = null;
@@ -229,6 +257,7 @@ const scanInit = async argvs => {
   if (argvs.scanner === constants.scannerTypes.website && !argvs.strategy) {
     argvs.strategy = 'same-domain';
   }
+
   const statuses = constants.urlCheckStatuses;
 
   // File clean up after url check
@@ -324,7 +353,7 @@ const scanInit = async argvs => {
 
   if (argvs.scanner === constants.scannerTypes.custom) {
     try {
-      await playwrightAxeGenerator(argvs.url, data);
+      await playwrightAxeGenerator(data);
     } catch (error) {
       silentLogger.error(error);
       printMessage([

diff --git a/combine.js b/combine.js
@@ -6,6 +6,7 @@ import crawlDomain from './crawlers/crawlDomain.js';
 import { generateArtifacts } from './mergeAxeResults.js';
 import { getHost, createAndUpdateResultsFolders, createDetailsAndLogs } from './utils.js';
 import constants, { basicAuthRegex } from './constants/constants.js';
+import { submitFormViaPlaywright } from './constants/common.js';
 
 const combineRun = async (details, deviceToScan) => {
   const envDetails = { ...details };
@@ -14,6 +15,7 @@ const combineRun = async (details, deviceToScan) => {
   const {
     type,
     url,
+    nameEmail,
     randomToken,
     deviceChosen,
     customDevice,
@@ -86,7 +88,17 @@ const combineRun = async (details, deviceToScan) => {
 
   if (scanDetails.urlsCrawled.scanned.length > 0) {
     await createAndUpdateResultsFolders(randomToken);
-    await generateArtifacts(randomToken, url, type, deviceToScan);
+    const basicFormHTMLSnippet = await generateArtifacts(randomToken, url, type, deviceToScan,  urlsCrawled.scanned);
+    const [name, email] = nameEmail.split(':');
+    await submitFormViaPlaywright(
+      browser,
+      userDataDirectory,
+      url,
+      type,
+      email,
+      name,
+      JSON.stringify(basicFormHTMLSnippet),
+    );
   } else {
     printMessage([`No pages were scanned.`], constants.alertMessageOptions);
   }

diff --git a/constants/cliFunctions.js b/constants/cliFunctions.js
@@ -74,6 +74,19 @@ export const cliOptions = {
     requiresArg: true,
     demandOption: false,
   },
+  j: {
+    alias: 'customFlowLabel',
+    describe: 'Give Custom Flow Scan a label for easier reference in the report',
+    type: 'string',
+    requiresArg: true, 
+    demandOption: false
+  },
+  k: {
+    alias: 'nameEmail',
+    describe: `To personalise your experience, we will be collecting your name, email address and app usage data. Your information fully complies with GovTech’s Privacy Policy. Please provice your name and email address in this format "John Doe:[email protected]".`,
+    type: 'string',
+    demandOption: true,
+  },
 };
 
 export const configureReportSetting = isEnabled => {