Support top-layer <dialog> recording & replay #1503

Juice10 · 2024-06-10T09:04:21Z

Adds support for recording & replaying Top Layer <dialog> elements. These elements float over any other elements no matter the z-index.

What this PR does not do:

trigger showModal() on rrweb-snapshot's rebuild
- when rebuild is called we have no guarantee the element is added to the dom, and .showModal() has to be attached to be called, so the responsibility to do this is hoisted to rrweb & rrdom.
add support for popover elements.
add support for requestFullScreen elements.

register order of top layer dialog elements, this is impossible to do without monkey patching HTMLDialogElement.showModal, and although that is possible I didn't have time to implement it. However most of the time this PR should sufficient. Unfortunately there is no way to query the dom for the order of elements in the top layer, as that would have made this doable without monkey patch, and recordable in rrweb-snapshot.

exact case where this breaks:

  const dialog1 = document.createElement('dialog') as HTMLDialogElement;
  dialog1.className = 'dialog1';
  document.body.appendChild(dialog1);
  const dialog2 = document.createElement('dialog') as HTMLDialogElement;
  dialog1.className = 'dialog2';
  document.body.appendChild(dialog2);
  dialog2.showModal(); // <== Note that dialog TWO is being triggered first, if dialog 1 was triggered first, this would have replayed as expected.
  dialog1.showModal();

changeset-bot · 2024-06-10T09:04:25Z

🦋 Changeset detected

Latest commit: 9f2a27c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages

Name	Type
rrweb-snapshot	Major
rrdom	Major
rrweb	Major
@rrweb/types	Major
@rrweb/rrweb-plugin-canvas-webrtc-record	Major
@rrweb/rrweb-plugin-canvas-webrtc-replay	Major
@rrweb/rrweb-plugin-console-record	Major
@rrweb/rrweb-plugin-console-replay	Major
@rrweb/rrweb-plugin-sequential-id-record	Major
@rrweb/rrweb-plugin-sequential-id-replay	Major
rrdom-nodejs	Major
rrweb-player	Major
@rrweb/all	Major
@rrweb/replay	Major
@rrweb/record	Major
@rrweb/packer	Major
@rrweb/web-extension	Major
rrvideo	Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

packages/rrweb-snapshot/src/snapshot.ts

…rweb into juice10/record-dialog

daibhin

Looks good!! Thanks for taking this on :)

packages/rrweb/test/replay/dialog.test.ts

packages/rrweb-snapshot/src/types.ts

…10/record-dialog

rrdom downcases all attribute names which made `rr_openMode` tricky to deal with

YunFeng0817

Thanks for implementing this. Overall, it looks good

packages/rrweb/src/replay/index.ts

packages/rrweb/src/replay/dialog/index.ts

packages/rrweb/src/replay/index.ts

#1503 (comment)

rrdom already handles dialog element creation it's self

…rweb into juice10/record-dialog

packages/rrdom/src/document.ts

@YunFeng0817

* chore: its important to run `yarn build:all` before running `yarn dev` * feat: trigger showModal from rrdom and rrweb * feat: Add support for replaying modal and non modal dialog elements * chore: Update dev script to remove CLEAR_DIST_DIR flag * Get modal recording and replay working * DRY up dialog test and dedupe snapshot images * feat: Refactor dialog test to use updated attribute name * feat: Update dialog test to include rr_open attribute * chore: Add npm dependency [email protected] * Add more test cases for dialog * Clean up naming * Refactor dialog open code * Revert changed code that doesn't do anything * Add documentation for unimplemented type * chore: Remove unnecessary comments in dialog.test.ts * rename rr_open to rr_openMode * Replace todo with a skipped test * Add better logging for CI * Rename rr_openMode to rr_open_mode rrdom downcases all attribute names which made `rr_openMode` tricky to deal with * Remove unused images * Move after iframe append based on @YunFeng0817's comment rrweb-io#1503 (comment) * Remove redundant dialog handling from rrdom. rrdom already handles dialog element creation it's self * Rename variables for dialog handling in rrweb replay module * Update packages/rrdom/src/document.ts --------- Co-authored-by: Eoghan Murray <[email protected]>

@Paulhejia

* Skip mask check on leaf elements (rrweb-io#1512) * Minor fixup for rrweb-io#1349; the 'we can avoid the check on leaf elements' optimisation wasn't being applied as `n.childNodes` was always truthy even when there were no childNodes. Changing it to `n.childNodes.length` directly there (see rrweb-io#1402) actually caused a bug as during a mutation, we serialize the text node directly, and need to jump to the parentElement to do the check. This is why I've reimplemented this optimisation inside `needMaskingText` where we are already had an `isElement` test Thanks to @Paulhejia (https://github.com/Paulhejia/rrweb/) for spotting that `Boolean(n.childNodes)` is aways true. * Assuming all jest should have been removed in rrweb-io#1033 (rrweb-io#1511) * all references to jest should have been removed in rrweb-io#1033 * clarify that `cross-env` is used to ensure that environmental variables get applied on Windows (previous usage of cross-env was removed in rrweb-io#1033) * Fix async assertions in test files (rrweb-io#1510) * fix: await assertSnapshot in test files for async assertions * Fix maskInputFn is ignored during the creation of the full snapshot (rrweb-io#1386) Fix that the optional `maskInputFn` was being accidentally ignored during the creation of the full snapshot * Improve development tooling (rrweb-io#1516) - Running `yarn build` in a `packages/*/` directory will trigger build of all dependencies too, and cache them if possible. - Fix for `yarn dev` breaking for `rrweb` package whenever changing files in `rrweb` package - Update typescript, turbo, vite and vite-plugin-dts - Require `workspaces-to-typescript-project-references` from `prepublish` * Version Packages (alpha) (rrweb-io#1513) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Keep all packages in sync * feat: add new css parser - postcss (rrweb-io#1458) * feat: add new css parser * make selectors change * selectors and tests * media changes * remove old css references * better variable name * use postcss and port tests * fix media test * inline plugins * fix failing multiline selector * correct test result * move tests to correct file * cleanup all tests * remove unused css-tree * update bundle * cleanup dependencies * revert config files to master * remove d.ts files * update snapshot * reset rebuilt test * apply fuzzy css matching * remove extra test * Fix imports * Newer versions of nswapi break rrdom-nodejs tests. Example: FAIL test/document-nodejs.test.ts > RRDocument for nodejs environment > RRDocument API > querySelectorAll TypeError: e[api] is not a function ❯ byTag ../../node_modules/nwsapi/src/nwsapi.js:390:37 ❯ Array.<anonymous> ../../node_modules/nwsapi/src/nwsapi.js:327:113 ❯ collect ../../node_modules/nwsapi/src/nwsapi.js:1578:32 ❯ Object._querySelectorAll [as select] ../../node_modules/nwsapi/src/nwsapi.js:1533:36 ❯ RRDocument.querySelectorAll src/document-nodejs.ts:96:24 * Migrate from jest to vitest * Order of selectors has changed with postcss * Remove unused eslint --------- Co-authored-by: Justin Halsall <[email protected]> * fix: console assert only logs when arg 0 is falsy (rrweb-io#1530) * fix: console assert only logs when arg 0 is falsy * [Feature] Include takeFullSnapshot function in rrweb (rrweb-io#1527) * export takeFullSnapshot function in rrweb * chore: reduce flakey test due to '[vite] connected' message (rrweb-io#1525) * fix: duplicate textContent for style element cause incremental style mutation invalid (rrweb-io#1417) fix style element corner case - historically we have recorded duplicated css content in certain cases (demonstrated by the attached replayer test). This fix ensures that the replayer doesn't doubly add the content, which can cause problems when further mutations occur --------- Review and further tests contributed by: Eoghan Murray <[email protected]> * Added support for deprecated addRule & removeRule methods (rrweb-io#1515) * Added support for deprecated addRule & removeRule methods * Respect addRule default value * fix: nested stylesheets should have absolute URLs (rrweb-io#1533) * Replace relative URLs with absolute URLs when stringifying stylesheets * Add test to show desired behavior for imported stylesheets from seperate directory * Rename `absoluteToStylesheet` to `absolutifyURLs` and call it once after stringifying imported stylesheet * Don't create the intermediary array of the spread operator * Formalize that `stringifyRule` should expect a sheet href * Ensure a <style> element can also import and gets it's url absolutized * Handle case where non imported stylesheet has relative urls that need to be absolutified * Clarify in test files where jpegs are expected to appear in absolutified urls * Move absolutifyURLs call for import rules out of trycatch * Add a benchmarking test for stringifyStylesheet * Avoid the duplication on how to fall back --------- Co-authored-by: Eoghan Murray <[email protected]> Co-authored-by: eoghanmurray <[email protected]> * Support top-layer <dialog> recording & replay (rrweb-io#1503) * chore: its important to run `yarn build:all` before running `yarn dev` * feat: trigger showModal from rrdom and rrweb * feat: Add support for replaying modal and non modal dialog elements * chore: Update dev script to remove CLEAR_DIST_DIR flag * Get modal recording and replay working * DRY up dialog test and dedupe snapshot images * feat: Refactor dialog test to use updated attribute name * feat: Update dialog test to include rr_open attribute * chore: Add npm dependency [email protected] * Add more test cases for dialog * Clean up naming * Refactor dialog open code * Revert changed code that doesn't do anything * Add documentation for unimplemented type * chore: Remove unnecessary comments in dialog.test.ts * rename rr_open to rr_openMode * Replace todo with a skipped test * Add better logging for CI * Rename rr_openMode to rr_open_mode rrdom downcases all attribute names which made `rr_openMode` tricky to deal with * Remove unused images * Move after iframe append based on @YunFeng0817's comment rrweb-io#1503 (comment) * Remove redundant dialog handling from rrdom. rrdom already handles dialog element creation it's self * Rename variables for dialog handling in rrweb replay module * Update packages/rrdom/src/document.ts --------- Co-authored-by: Eoghan Murray <[email protected]> * Added session downloader for chrome extension (rrweb-io#1522) * Added session downloader for chrome extension - The session list now has a button to download sessions as .json files for use with rrweb-player - Improved styling for the delete and download buttons * Reverse monkey patch built in methods to support LWC (rrweb-io#1509) * Get around monkey patched Nodes * inlineImages: Setting of `image.crossOrigin` is not always necessary (rrweb-io#1468) Setting of the `crossorigin` attribute is not necessary for same-origin images, and causes an immediate image reload (albeit from cache) necessitating the use of a load event listener which subsequently mutates the snapshot. This change allows us to avoid the mutation of the snapshot for the same-origin case. * Modify inlineImages test to remove delay and show that we can inline images without mutation * Add an explicit test for when the `image.crossOrigin = 'anonymous';` method is necessary. Uses a combination of about:blank and our test server to simulate a cross-origin context * Other test changes: there were some spurious rrweb mutations being generated by the addition of the crossorigin attribute that are now elimnated from the rrweb/__snapshots__/integration.test.ts.snap after this PR - this is good * Move `childNodes` to @rrweb/utils * Use non-monkey patched versions of the `childNodes`, `parentNode` `parentElement` `textContent` accessors * Add getRootNode and contains, and add comprehensive todo list * chore: Update turbo.json tasks for better build process * Update caniuse-lite * chore: Update eslint-plugin-compat to version 5.0.0 * chore: Bump @rrweb/utils version to 2.0.0-alpha.15 * delete unused yarn.lock files * Set correct @rrweb/utils version in package.json * Migrate over some accessors to reverse-monkey-patched version * Add missing functions * Fix illegal invocation error * Revert closer to what it was. This feels incorrect to me (Justin Halsall), but some of the tests break without it so I'm restoring this to be closer to its original here: https://github.com/rrweb-io/rrweb/blame/cfd686d488a9b88dba6b6f8880b5e4375dd8062c/packages/rrweb-snapshot/src/snapshot.ts#L1011 * Reverse monkey patch all methods LWC hijacks * Make tests more stable * Safely handle rrdom nodes in hasShadowRoot * Remove duplicated test * Use variable `serverURL` in test * Use monorepo default browserlist * Fix typing issue for new typescript * Remove unused package * Remove unused code * Add prefix to reverse-monkey-patched methods to make them more explicit * Add default exports to @rrweb/utils --------- Co-authored-by: Eoghan Murray <[email protected]> * Version Packages (alpha) (rrweb-io#1526) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Single style capture (rrweb-io#1437) Support a contrived/rare case where a <style> element has multiple text node children (this is usually only possible to recreate via javascript append) ... this PR fixes cases where there are subsequent text mutations to these nodes; previously these would have been lost * In this scenario, a new CSS comment may now be inserted into the captured `_cssText` for a <style> element to show where it should be broken up into text elements upon replay: `/* rr_split */` * The new 'can record and replay style mutations' test is the principal way to the problematic scenarios, and is a detailed 'catch-all' test with many checks to cover most of the ways things can fail * There are new tests for splitting/rebuilding the css using the rr_split marker * The prior 'dynamic stylesheet' route is now the main route for serializing a stylesheet; dynamic stylesheet were missed out in rrweb-io#1533 but that case is now covered with this PR This PR was originally extracted from rrweb-io#1475 so the initial motivation was to change the approach on stringifying <style> elements to do so in a single place. This is also the motivating factor for always serializing <style> elements via the `_cssText` attribute rather than in it's childNodes; in rrweb-io#1475 we will be delaying populating `_cssText` for performance and instead recorrding them as assets. Thanks for the detailed review to Justin Halsall <[email protected]> & Yun Feng <https://github.com/YunFeng0817> * Simplify the hover replacement function (rrweb-io#1535) Simplify the hover replacement function, which has been borrowed from postcss-pseudo-classes Note: 'parses nested commas in selectors correctly' was failing after this PR, however I don't think that the previous behaviour was desirable, so have added a new test to formalize this expectation * fix some typos in optimize-storage.md (rrweb-io#1565) * fix some typos in optimize-storage.md * Update docs/recipes/optimize-storage.md * Create metal-mugs-mate.md --------- Co-authored-by: Justin Halsall <[email protected]> * fix(rrdom): Ignore invalid DOM attributes when diffing (rrweb-io#1561) * fix(rrdom): Ignore invalid DOM attributes when diffing (rrweb-io#213) We encountered an issue where replays with invalid attributes (e.g. `@click`) would break rendering the replay after seeking. The exception bubbles up to [here](https://github.com/rrweb-io/rrweb/blob/62093d4385a09eb0980c2ac02d97eea5ce2882be/packages/rrweb/src/replay/index.ts#L270-L279), which means the replay will continue to play, but the replay mirror will be incomplete. Closes https://github.com/getsentry/team-replay/issues/458 * add changeset * fix(snapshot): dimensions for blocked element not being applied (rrweb-io#1331) fix for replay of a blocked element when using 'fast forward' (rrdom) - Dimensions were not being properly applied when you seek to a position in the replay. Need to use `setProperty` rather than trying to set the width/height directly * ref: isParentRemoved to cache subtree (rrweb-io#1543) * ref: isParentRemoved to cache subtree * ref: cache at insertion too * ref: remove wrapper function --------- Co-authored-by: Justin Halsall <[email protected]> * changeset to 2.0.13 * fix snapshot build --------- Co-authored-by: Eoghan Murray <[email protected]> Co-authored-by: Justin Halsall <[email protected]> Co-authored-by: Alexey Babik <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: David Newell <[email protected]> Co-authored-by: Paul D'Ambra <[email protected]> Co-authored-by: Christopher Arredondo <[email protected]> Co-authored-by: Yun Feng <[email protected]> Co-authored-by: minja malešević <[email protected]> Co-authored-by: Jeff Nguyen <[email protected]> Co-authored-by: eoghanmurray <[email protected]> Co-authored-by: Arun Kunigiri <[email protected]> Co-authored-by: Riadh Mouamnia <[email protected]> Co-authored-by: Billy Vong <[email protected]> Co-authored-by: Jonas <[email protected]> Co-authored-by: Shay Malchi <[email protected]>

Juice10 marked this pull request as draft June 10, 2024 09:04

eoghanmurray reviewed Jun 10, 2024

View reviewed changes

packages/rrweb-snapshot/src/snapshot.ts Outdated Show resolved Hide resolved

WIP dialog recording

78849f3

Juice10 force-pushed the juice10/record-dialog branch from 59de782 to 78849f3 Compare June 10, 2024 11:16

Juice10 added 5 commits June 10, 2024 13:37

chore: its important to run yarn build:all before running yarn dev

c1d850f

feat: trigger showModal from rrdom and rrweb

b657233

feat: Add support for replaying modal and non modal dialog elements

337b4c4

chore: Update dev script to remove CLEAR_DIST_DIR flag

29d4827

Get modal recording and replay working

dcc867c

Juice10 force-pushed the juice10/record-dialog branch from e884282 to dcc867c Compare June 12, 2024 11:43

Juice10 added 15 commits June 12, 2024 14:58

DRY up dialog test and dedupe snapshot images

1144c9d

Fix eslint error

7507f4d

feat: Refactor dialog test to use updated attribute name

b8679a0

feat: Update dialog test to include rr_open attribute

6ec4727

chore: Add npm dependency [email protected]

f7db71c

Fix dialog test

b300674

Add more test cases for dialog

178ec1f

Add changesets

4229391

Clean up naming

b0065ce

Apply formatting changes

25678f4

Refactor dialog open code

b71c8b7

Merge branch 'juice10/record-dialog' of https://github.com/rrweb-io/r…

05d4cf0

…rweb into juice10/record-dialog

Revert changed code that doesn't do anything

e731636

Add documentation for unimplemented type

184e4fa

chore: Remove unnecessary comments in dialog.test.ts

6a58edc

Juice10 requested a review from eoghanmurray June 12, 2024 15:01

Juice10 changed the title ~~[draft] dialog recording~~ Support top-layer <dialog> recording & replay Jun 12, 2024

Juice10 marked this pull request as ready for review June 12, 2024 15:19

YunFeng0817 self-requested a review June 12, 2024 16:55

daibhin approved these changes Jun 13, 2024

View reviewed changes

packages/rrweb/test/replay/dialog.test.ts Outdated Show resolved Hide resolved

Juice10 commented Jun 13, 2024

View reviewed changes

packages/rrweb-snapshot/src/types.ts Outdated Show resolved Hide resolved

Juice10 added 6 commits June 26, 2024 10:44

Merge branch 'master' of https://github.com/rrweb-io/rrweb into juice…

8c51129

…10/record-dialog

rename rr_open to rr_openMode

0df7994

Replace todo with a skipped test

1ce8f70

Add better logging for CI

8aeb47f

Rename rr_openMode to rr_open_mode

985ac9f

rrdom downcases all attribute names which made `rr_openMode` tricky to deal with

Remove unused images

d693da9

Juice10 force-pushed the juice10/record-dialog branch from a83c6e1 to d693da9 Compare June 26, 2024 12:48

YunFeng0817 reviewed Jul 5, 2024

View reviewed changes

packages/rrweb/src/replay/index.ts Outdated Show resolved Hide resolved

packages/rrweb/src/replay/dialog/index.ts Outdated Show resolved Hide resolved

packages/rrweb/src/replay/index.ts Outdated Show resolved Hide resolved

Juice10 and others added 6 commits July 9, 2024 13:39

Move after iframe append based on @YunFeng0817's comment

760908f

#1503 (comment)

Remove redundant dialog handling from rrdom.

2dfeb01

rrdom already handles dialog element creation it's self

Merge branch 'master' into juice10/record-dialog

63c0def

Rename variables for dialog handling in rrweb replay module

21c4826

Merge branch 'juice10/record-dialog' of https://github.com/rrweb-io/r…

424e4c8

…rweb into juice10/record-dialog

Merge branch 'master' into juice10/record-dialog

30c7780

eoghanmurray reviewed Jul 31, 2024

View reviewed changes

packages/rrdom/src/document.ts Outdated Show resolved Hide resolved

Update packages/rrdom/src/document.ts

9f2a27c

YunFeng0817 approved these changes Aug 2, 2024

View reviewed changes

Juice10 merged commit 335639a into master Aug 2, 2024
11 checks passed

Juice10 deleted the juice10/record-dialog branch August 2, 2024 07:53

github-actions bot mentioned this pull request Aug 2, 2024

Version Packages (alpha) #1526

Merged

pauldambra mentioned this pull request Nov 27, 2024

Significant amount of blank screen recordings on Safari since 1.186.3 PostHog/posthog-js#1561

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support top-layer <dialog> recording & replay #1503

Support top-layer <dialog> recording & replay #1503

Juice10 commented Jun 10, 2024 •

edited

Loading

changeset-bot bot commented Jun 10, 2024 •

edited

Loading

daibhin left a comment

YunFeng0817 left a comment

Support top-layer <dialog> recording & replay #1503

Support top-layer <dialog> recording & replay #1503

Conversation

Juice10 commented Jun 10, 2024 • edited Loading

changeset-bot bot commented Jun 10, 2024 • edited Loading

🦋 Changeset detected

daibhin left a comment

Choose a reason for hiding this comment

YunFeng0817 left a comment

Choose a reason for hiding this comment

Juice10 commented Jun 10, 2024 •

edited

Loading

changeset-bot bot commented Jun 10, 2024 •

edited

Loading