Skip to content

Commit

Permalink
chore: allow storing aria snapshots in files (#33919)
Browse files Browse the repository at this point in the history
  • Loading branch information
pavelfeldman authored Dec 10, 2024
1 parent 200e868 commit a25bda6
Show file tree
Hide file tree
Showing 12 changed files with 370 additions and 26 deletions.
2 changes: 1 addition & 1 deletion docs/src/api/class-locator.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ Additional locator to match.
- returns: <[string]>

Captures the aria snapshot of the given element.
Read more about [aria snapshots](../aria-snapshots.md) and [`method: LocatorAssertions.toMatchAriaSnapshot`] for the corresponding assertion.
Read more about [aria snapshots](../aria-snapshots.md) and [`method: LocatorAssertions.toMatchAriaSnapshot#2`] for the corresponding assertion.

**Usage**

Expand Down
61 changes: 56 additions & 5 deletions docs/src/api/class-locatorassertions.md
Original file line number Diff line number Diff line change
Expand Up @@ -446,7 +446,7 @@ Expected options currently selected.
* since: v1.49
* langs: python

The opposite of [`method: LocatorAssertions.toMatchAriaSnapshot`].
The opposite of [`method: LocatorAssertions.toMatchAriaSnapshot#2`].

### param: LocatorAssertions.NotToMatchAriaSnapshot.expected
* since: v1.49
Expand Down Expand Up @@ -2121,7 +2121,58 @@ Expected options currently selected.
* since: v1.23


## async method: LocatorAssertions.toMatchAriaSnapshot
## async method: LocatorAssertions.toMatchAriaSnapshot#1
* since: v1.50
* langs:
- alias-java: matchesAriaSnapshot

Asserts that the target element matches the given [accessibility snapshot](../aria-snapshots.md).

**Usage**

```js
await expect(page.locator('body')).toMatchAriaSnapshot();
await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'snapshot' });
await expect(page.locator('body')).toMatchAriaSnapshot({ path: '/path/to/snapshot.yml' });
```

```python async
await expect(page.locator('body')).to_match_aria_snapshot(path='/path/to/snapshot.yml')
```

```python sync
expect(page.locator('body')).to_match_aria_snapshot(path='/path/to/snapshot.yml')
```

```csharp
await Expect(page.Locator("body")).ToMatchAriaSnapshotAsync(new { Path = "/path/to/snapshot.yml" });
```

```java
assertThat(page.locator("body")).matchesAriaSnapshot(new LocatorAssertions.MatchesAriaSnapshotOptions().setPath("/path/to/snapshot.yml"));
```

### option: LocatorAssertions.toMatchAriaSnapshot#1.name
* since: v1.50
* langs: js
- `name` <[string]>

Name of the snapshot to store in the snapshot folder corresponding to this test. Generates ordinal name if not specified.

### option: LocatorAssertions.toMatchAriaSnapshot#1.path
* since: v1.50
- `path` <[string]>

Path to the YAML snapshot file.

### option: LocatorAssertions.toMatchAriaSnapshot#1.timeout = %%-js-assertions-timeout-%%
* since: v1.50

### option: LocatorAssertions.toMatchAriaSnapshot#1.timeout = %%-csharp-java-python-assertions-timeout-%%
* since: v1.50


## async method: LocatorAssertions.toMatchAriaSnapshot#2
* since: v1.49
* langs:
- alias-java: matchesAriaSnapshot
Expand Down Expand Up @@ -2170,12 +2221,12 @@ assertThat(page.locator("body")).matchesAriaSnapshot("""
""");
```

### param: LocatorAssertions.toMatchAriaSnapshot.expected
### param: LocatorAssertions.toMatchAriaSnapshot#2.expected
* since: v1.49
- `expected` <string>

### option: LocatorAssertions.toMatchAriaSnapshot.timeout = %%-js-assertions-timeout-%%
### option: LocatorAssertions.toMatchAriaSnapshot#2.timeout = %%-js-assertions-timeout-%%
* since: v1.49

### option: LocatorAssertions.toMatchAriaSnapshot.timeout = %%-csharp-java-python-assertions-timeout-%%
### option: LocatorAssertions.toMatchAriaSnapshot#2.timeout = %%-csharp-java-python-assertions-timeout-%%
* since: v1.49
2 changes: 1 addition & 1 deletion docs/src/aria-snapshots.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ structure of a page, use the [Chrome DevTools Accessibility Pane](https://develo

## Snapshot matching

The [`method: LocatorAssertions.toMatchAriaSnapshot`] assertion method in Playwright compares the accessible
The [`method: LocatorAssertions.toMatchAriaSnapshot#2`] assertion method in Playwright compares the accessible
structure of the locator scope with a predefined aria snapshot template, helping validate the page's state against
testing requirements.

Expand Down
2 changes: 1 addition & 1 deletion docs/src/release-notes-csharp.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ toc_max_heading_level: 2

### Aria snapshots

New assertion [`method: LocatorAssertions.toMatchAriaSnapshot`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
New assertion [`method: LocatorAssertions.toMatchAriaSnapshot#2`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.

```csharp
await page.GotoAsync("https://playwright.dev");
Expand Down
2 changes: 1 addition & 1 deletion docs/src/release-notes-java.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ toc_max_heading_level: 2

### Aria snapshots

New assertion [`method: LocatorAssertions.toMatchAriaSnapshot`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
New assertion [`method: LocatorAssertions.toMatchAriaSnapshot#2`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.

```java
page.navigate("https://playwright.dev");
Expand Down
2 changes: 1 addition & 1 deletion docs/src/release-notes-js.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import LiteYouTube from '@site/src/components/LiteYouTube';

### Aria snapshots

New assertion [`method: LocatorAssertions.toMatchAriaSnapshot`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
New assertion [`method: LocatorAssertions.toMatchAriaSnapshot#2`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.

```js
await page.goto('https://playwright.dev');
Expand Down
2 changes: 1 addition & 1 deletion docs/src/release-notes-python.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ toc_max_heading_level: 2

### Aria snapshots

New assertion [`method: LocatorAssertions.toMatchAriaSnapshot`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
New assertion [`method: LocatorAssertions.toMatchAriaSnapshot#2`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.

```python
page.goto("https://playwright.dev")
Expand Down
2 changes: 1 addition & 1 deletion packages/playwright-core/types/types.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12428,7 +12428,7 @@ export interface Locator {

/**
* Captures the aria snapshot of the given element. Read more about [aria snapshots](https://playwright.dev/docs/aria-snapshots) and
* [expect(locator).toMatchAriaSnapshot(expected[, options])](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-match-aria-snapshot)
* [expect(locator).toMatchAriaSnapshot(expected[, options])](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-match-aria-snapshot-2)
* for the corresponding assertion.
*
* **Usage**
Expand Down
69 changes: 55 additions & 14 deletions packages/playwright/src/matchers/toMatchAriaSnapshot.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,25 @@
import type { LocatorEx } from './matchers';
import type { ExpectMatcherState } from '../../types/test';
import { kNoElementsFoundError, matcherHint, type MatcherResult } from './matcherHint';
import { colors } from 'playwright-core/lib/utilsBundle';
import { EXPECTED_COLOR } from '../common/expectBundle';
import { callLogText } from '../util';
import { callLogText, sanitizeFilePathBeforeExtension, trimLongString } from '../util';
import { printReceivedStringContainExpectedSubstring } from './expect';
import { currentTestInfo } from '../common/globals';
import type { MatcherReceived } from '@injected/ariaSnapshot';
import { escapeTemplateString } from 'playwright-core/lib/utils';
import { escapeTemplateString, isString, sanitizeForFilePath } from 'playwright-core/lib/utils';
import fs from 'fs';
import path from 'path';

type ToMatchAriaSnapshotExpected = {
name?: string;
path?: string;
} | string;

export async function toMatchAriaSnapshot(
this: ExpectMatcherState,
receiver: LocatorEx,
expected: string,
options: { timeout?: number, matchSubstring?: boolean } = {},
expectedParam: ToMatchAriaSnapshotExpected,
options: { timeout?: number } = {},
): Promise<MatcherResult<string | RegExp, string>> {
const matcherName = 'toMatchAriaSnapshot';

Expand All @@ -39,7 +45,7 @@ export async function toMatchAriaSnapshot(
throw new Error(`toMatchAriaSnapshot() must be called during the test`);

if (testInfo._projectInternal.ignoreSnapshots)
return { pass: !this.isNot, message: () => '', name: 'toMatchAriaSnapshot', expected };
return { pass: !this.isNot, message: () => '', name: 'toMatchAriaSnapshot', expected: '' };

const updateSnapshots = testInfo.config.updateSnapshots;

Expand All @@ -48,12 +54,25 @@ export async function toMatchAriaSnapshot(
promise: this.promise,
};

if (typeof expected !== 'string') {
throw new Error([
matcherHint(this, receiver, matcherName, receiver, expected, matcherOptions),
`${colors.bold('Matcher error')}: ${EXPECTED_COLOR('expected',)} value must be a string`,
this.utils.printWithType('Expected', expected, this.utils.printExpected)
].join('\n\n'));
let expected: string;
let expectedPath: string | undefined;
if (isString(expectedParam)) {
expected = expectedParam;
} else {
if (expectedParam?.path) {
expectedPath = expectedParam.path;
} else if (expectedParam?.name) {
expectedPath = testInfo.snapshotPath(sanitizeFilePathBeforeExtension(expectedParam.name));
} else {
let snapshotNames = (testInfo as any)[snapshotNamesSymbol] as SnapshotNames;
if (!snapshotNames) {
snapshotNames = { anonymousSnapshotIndex: 0 };
(testInfo as any)[snapshotNamesSymbol] = snapshotNames;
}
const fullTitleWithoutSpec = [...testInfo.titlePath.slice(1), ++snapshotNames.anonymousSnapshotIndex].join(' ');
expectedPath = testInfo.snapshotPath(sanitizeForFilePath(trimLongString(fullTitleWithoutSpec)) + '.yml');
}
expected = await fs.promises.readFile(expectedPath, 'utf8').catch(() => '');
}

const generateMissingBaseline = updateSnapshots === 'missing' && !expected;
Expand Down Expand Up @@ -102,8 +121,24 @@ export async function toMatchAriaSnapshot(
if ((updateSnapshots === 'all') ||
(updateSnapshots === 'changed' && pass === this.isNot) ||
generateMissingBaseline) {
const suggestedRebaseline = `toMatchAriaSnapshot(\`\n${escapeTemplateString(indent(typedReceived.regex, '{indent} '))}\n{indent}\`)`;
return { pass: this.isNot, message: () => '', name: 'toMatchAriaSnapshot', suggestedRebaseline };
if (expectedPath) {
await fs.promises.mkdir(path.dirname(expectedPath), { recursive: true });
await fs.promises.writeFile(expectedPath, typedReceived.regex, 'utf8');
const relativePath = path.relative(process.cwd(), expectedPath);
if (updateSnapshots === 'missing') {
const message = `A snapshot doesn't exist at ${relativePath}, writing actual.`;
testInfo._hasNonRetriableError = true;
testInfo._failWithError(new Error(message));
} else {
const message = `A snapshot is generated at ${relativePath}.`;
/* eslint-disable no-console */
console.log(message);
}
return { pass: true, message: () => '', name: 'toMatchAriaSnapshot' };
} else {
const suggestedRebaseline = `toMatchAriaSnapshot(\`\n${escapeTemplateString(indent(typedReceived.regex, '{indent} '))}\n{indent}\`)`;
return { pass: false, message: () => '', name: 'toMatchAriaSnapshot', suggestedRebaseline };
}
}
}

Expand Down Expand Up @@ -134,3 +169,9 @@ function unshift(snapshot: string): string {
function indent(snapshot: string, indent: string): string {
return snapshot.split('\n').map(line => indent + line).join('\n');
}

const snapshotNamesSymbol = Symbol('snapshotNames');

type SnapshotNames = {
anonymousSnapshotIndex: number;
};
31 changes: 31 additions & 0 deletions packages/playwright/types/test.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8430,6 +8430,37 @@ interface LocatorAssertions {
timeout?: number;
}): Promise<void>;

/**
* Asserts that the target element matches the given [accessibility snapshot](https://playwright.dev/docs/aria-snapshots).
*
* **Usage**
*
* ```js
* await expect(page.locator('body')).toMatchAriaSnapshot();
* await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'snapshot' });
* await expect(page.locator('body')).toMatchAriaSnapshot({ path: '/path/to/snapshot.yml' });
* ```
*
* @param options
*/
toMatchAriaSnapshot(options?: {
/**
* Name of the snapshot to store in the snapshot folder corresponding to this test. Generates ordinal name if not
* specified.
*/
name?: string;

/**
* Path to the YAML snapshot file.
*/
path?: string;

/**
* Time to retry the assertion for in milliseconds. Defaults to `timeout` in `TestConfig.expect`.
*/
timeout?: number;
}): Promise<void>;

/**
* Asserts that the target element matches the given [accessibility snapshot](https://playwright.dev/docs/aria-snapshots).
*
Expand Down
Loading

0 comments on commit a25bda6

Please sign in to comment.