Skip to content

Latest commit

 

History

History
336 lines (248 loc) · 13.2 KB

INTEGRATION.md

File metadata and controls

336 lines (248 loc) · 13.2 KB
Raw

Integrating Purple HATS with end-to-end testing frameworks

Purple HATS provides functionalities that makes it possible to be integrated with end-to-end testing frameworks such as Cypress and Playwright.

Prerequisites

In order to use this functionality, the testing framework must support:

  • Execution of scripts in a NodeJS environment.
  • Injection of JavaScript into the document that is being tested.
  • Execution of JavaScript in the context of the document and retrieval of results back into the NodeJS environment after execution.

How to include Purple HATS in your project

  1. Add Purple HATS to your project by running the following command:

    npm install --save-dev @govtechsg/purple-hats

  2. In the file of choice, import Purple HATS using:

    import purpleHatsInit from '@govtechsg/purple-hats'

    Note that Purple HATS should be imported in a script that runs in a NodeJS environment.

  3. Create an instance of Purple HATS with:

    const ph = await purpleHatsInit(entryUrl)

    entryUrl should be a valid URL referring to the domain of the website to be scanned with Purple HATS.

API Reference

async purpleHatsInit(entryUrl, testLabel, name, email, needsReview, includeScreenshots, viewportSettings, thresholds, scanAboutMetadata)

Returns an instance of Purple HATS

Parameters
  • entryUrl
    • Initial URL to start the purple hats scan
  • testLabel
    • Label for test in report
  • name
    • For PH data collection purposes
  • email
    • For PH data collection purposes
  • needsReview (optional)
    • Show potential false positive issues in the report. Defaults to false.
  • includeScreenshots (optional)
    • Include screenshots of affected elements in the report. Defaults to false.
  • viewportSettings (optional)
    • Viewport settings used in cypress tests needed to optimize screenshot function. Defaults to cypress’ default viewport settings. Example: { width: 1000, height: 600 }
  • thresholds (optional)
    • Object containing the max number of mustFix or goodToFix issue occurrences before an error is thrown for test failure. Does not fail tests by default. Example: { mustFix: 1, goodToFix: 3 }
  • scanAboutMetadata (optional)
    • Include additional information in the Scan About section of the report by passing in a JSON object.

Purple HATS Instance

Properties

scanDetails

Object containing details about the scan

  • E.g. { startTime: 1700104789495, crawlType: 'Customized', requestUrl: 'https://govtechsg.github.io', urlsCrawled: { } }

randomToken

Unique identifier for the scan instance

Methods

getScripts()

Get the axe-core script to be injected into the browser

  • runA11yScan(elementsToScan) Runs axe scan on the current page.

    Parameter(s):

    • elementsToScan: Specifies which element should and which should not be tested

    Returns:

    • Object consisting of the current page url, current page title and axe scan result. { pageUrl, pageTitle, axeScanResults }

async pushScanResults(res, metadata, elementsToClick)

Process scan results to be included in the report.

Parameter(s):

  • res: Object consisting of the current page url, current page title and axe scan result. {pageUrl, pageTitle, axeScanResults}
  • metadata (optional): Additional information to be displayed for each page scanned in the report
  • elementsToClick (optional): Elements clicked during the test to reveal hidden elements. Required to be able identify hidden elements if they were scanned for screenshot purposes. Ensure selectors resolve to a single element.

Returns:

  • Object containing the number of mustFix and goodToFix issue occurrences for this scan run e.g. { mustFix: 1, goodToFix: 5 }

testThresholdsAndReset()

Checks the accumulated issue occurrences count against the specified threshold and resets the issues count

  • Throws an error if the number of accumulated mustFix or goodToFix issue occurrences exceeds either of the specified thresholds

async terminate()

Stops the Purple HATS instance and generates the scan report and other scan result artifacts

How to use

Example usages for Cypress and Playwright can be found in this section.

With reference to an instance of Purple HATS as ph:

  1. Fetch the necessary scripts needed to be injected to document to be scanned by executing ph.getScripts(). The scripts will be returned as a string.
  2. Inject the scripts into the document to be scanned. The easiest way that this can be done is by using eval() in the document's environment.
    • Note that this step needs to be done for every page visited.
  3. Run a scan by executing runA11yScan() in the document's environment.
    • By default, the scan will be run for the entire page.
    • It is possible to run the scan for specific sections or elements in the page. One way to do this is to pass an array of CSS selectors of the elements to be scanned into runA11yScan. For example, runA11yScan(['#my-component', 'button']). Other acceptable forms of argument can be found here.
  4. Pass the scan results back into the NodeJS environment where ph is in.
  5. Push the results using await ph.pushScanResults(scanResults).
  6. Repeat steps 2-5 as many times as desired.
  7. Terminate Purple HATS by using await ph.terminate(). A folder containing the details and report of your scan will be created, under the directory results which can be found in your project's root directory.

Example usages

Cypress

Click here to see an example usage in an E2E Cypress test

We will be creating the following files in a demo Cypress project:

├── cypress
│   ├── e2e
│   │   └── spec.cy.js
│   └── support
│       └── e2e.js
├── cypress.config.js
└── package.json

Create a package.json by running npm init . Accept the default options or customise it as needed.

Change the type of npm package to module by running npm pkg set type="module";

Install the following node dependencies by running npm install cypress @govtechsg/purple-hats --save-dev

Navigate to node_modules/@govtechsg/purple-hats and run npm install within the folder to install remaining Purple HATS dependencies:

cd node_modules/@govtechsg/purple-hats
npm install
cd ../../..

Create cypress.config.js with the following contents, and change your Name, E-mail address, and boolean value for whether rule items requiring manual review in the report should be displayed below:

import { defineConfig } from "cypress";
import purpleHatsInit from "@govtechsg/purple-hats";

// viewport used in tests to optimise screenshots
const viewportSettings = { width: 1920, height: 1040 };
// specifies the number of occurrences before error is thrown for test failure
const thresholds = { mustFix: 4, goodToFix: 5 };
// additional information to include in the "Scan About" section of the report
const scanAboutMetadata = { browser: 'Chrome (Desktop)' };

const ph = await purpleHatsInit(
    "https://govtechsg.github.io", // initial url to start scan
    "Demo Cypress Scan", // label for test
    "Your Name",
    "[email protected]",
    false, // whether to show false positive issues in the report
    true, // include screenshots of affected elements in the report
    viewportSettings,
    thresholds,
    scanAboutMetadata,
);

export default defineConfig({
    taskTimeout: 120000, // need to extend as screenshot function requires some time
    viewportHeight: viewportSettings.height,
    viewportWidth: viewportSettings.width,
    e2e: {
        setupNodeEvents(on, _config) {
            on("task", {
                getPhScripts() {
                    return ph.getScripts();
                },
                async pushPhScanResults({res, metadata, elementsToClick}) {
                    return await ph.pushScanResults(res, metadata, elementsToClick);
                },
                returnResultsDir() {
                    return `results/${ph.randomToken}_${ph.scanDetails.urlsCrawled.scanned.length}pages/reports/report.html`;
                },
                finishPhTestCase() {
                    ph.testThresholdsAndReset();
                    return null;
                },
                async terminatePh() {
                    return await ph.terminate();
                },
            });
        },
    },
});

Create a sub-folder and file cypress/support/e2e.js with the following contents::

Cypress.Commands.add("injectPhScripts", () => {
    cy.task("getPhScripts").then((s) => {
        cy.window().then((win) => {
            win.eval(s);
        });
    });
});

Cypress.Commands.add("runPhScan", (items={}) => {
    cy.window().then(async (win) => {
        const { elementsToScan, elementsToClick, metadata } = items; 
        const res = await win.runA11yScan(elementsToScan);
        cy.task("pushPhScanResults", {res, metadata, elementsToClick}).then((count) => { return count });
    });
});

Cypress.Commands.add("finishPhTestCase", () => {
    cy.task("finishPhTestCase");
});

Cypress.Commands.add("terminatePh", () => {
    cy.task("terminatePh");
});

Create cypress/e2e/spec.cy.js with the following contents:

describe("template spec", () => {
    it("should run purple HATS", () => {
        cy.visit(
            "https://govtechsg.github.io/purple-banner-embeds/purple-integrated-scan-example.htm"
        );
        cy.injectPhScripts();
        cy.runPhScan();
         cy.get("button[onclick=\"toggleSecondSection()\"]").click();
        // Run a scan on <input> and <button> elements
        cy.runPhScan({
            elementsToScan: ["input", "button"], 
            elementsToClick: ["button[onclick=\"toggleSecondSection()\"]"],
            metadata: "Clicked button"
        });

        cy.finishPhTestCase(); // test the number of issue occurrences against specified thresholds

        cy.terminatePh();
    });
});

Run your test with npx cypress run .

You will see Purple HATS results generated in results folder.

Playwright

Click here to see an example usage in Playwright

Create a package.json by running npm init . Accept the default options or customise it as needed.

Install the following node dependencies by running npm install playwright @govtechsg/purple-hats --save-dev

Navigate to node_modules/@govtechsg/purple-hats and run npm install within the folder to install remaining Purple HATS dependencies.

On your project's root folder, create a Playwright test file ph-playwright-demo.js:

import { chromium } from "playwright";
import purpleHatsInit from "@govtechsg/purple-hats";

// viewport used in tests to optimise screenshots
const viewportSettings = { width: 1920, height: 1040 };
// specifies the number of occurrences before error is thrown for test failure
const thresholds = { mustFix: 4, goodToFix: 5 };
// additional information to include in the "Scan About" section of the report
const scanAboutMetadata = { browser: 'Chrome (Desktop)' };

const ph = await purpleHatsInit(
    "https://govtechsg.github.io", // initial url to start scan
    "Demo Playwright Scan", // label for test
    "Your Name",
    "[email protected]",
    false, // whether to show false positive issues in the report
    true, // include screenshots of affected elements in the report
    viewportSettings,
    thresholds,
    scanAboutMetadata,
);

(async () => {
    const browser = await chromium.launch({
        headless: false,
    });
    const context = await browser.newContext();
    const page = await context.newPage();

    const runPhScan = async (elementsToScan) => {
        const scanRes = await page.evaluate(
            async elementsToScan => await runA11yScan(elementsToScan),
            elementsToScan,
        );
        await ph.pushScanResults(scanRes);
    };

    await page.goto('https://govtechsg.github.io/purple-banner-embeds/purple-integrated-scan-example.htm');
    await page.evaluate(ph.getScripts());
    await runPhScan();

    await page.getByRole('button', { name: 'Click Me' }).click();
    // Run a scan on <input> and <button> elements
    await runPhScan(['input', 'button'])

    ph.testThresholdsAndReset(); // test the number of issue occurrences against specified thresholds

    // ---------------------
    await context.close();
    await browser.close();
    await ph.terminate();
})();

Run your test with node ph-playwright-demo.js .

You will see Purple HATS results generated in results folder.