Add documentation for Screen Capture extensions, element capture and region capture #36939

chrisdavidmills · 2024-11-23T16:52:11Z

Description

Motivation

Chrome supports extensions to the Screen Capture API that allow developers to limit the region of the capture. Namely, these are:

The Element Capture API, which is supported from Chrome 132 (see https://chromestatus.com/feature/5198989277790208).
The Region Capture API, which is supported from Chrome 104 (see https://chromestatus.com/feature/5712447794053120).

This PR adds documentation for these extensions under a new tree "Screen Capture extensions", which I felt was preferable to adding a separate docs tree for multiple APIs that do very similar things.

Additional details

Related issues and pull requests

Fixes mdn/mdn#580

BCD addition: Chrome 132 supports Element Capture API browser-compat-data#25232.
dom-examples addition: Add DOM examples for Screen Capture API dom-examples#285. The linked demos won't work until this PR is merged.

github-actions · 2024-11-23T16:54:02Z

Preview URLs (11 pages)

Flaws (4)

Note! 7 documents with no flaws that don't need to be listed. 🎉

URL: /en-US/docs/Web/API/BrowserCaptureMediaStreamTrack/restrictTo
Title: BrowserCaptureMediaStreamTrack: restrictTo() method
Flaw count: 1

bad_bcd_queries:
- No BCD data for query: api.BrowserCaptureMediaStreamTrack.restrictTo

URL: /en-US/docs/Web/API/RestrictionTarget
Title: RestrictionTarget
Flaw count: 1

bad_bcd_queries:
- No BCD data for query: api.RestrictionTarget

URL: /en-US/docs/Web/API/RestrictionTarget/fromElement_static
Title: RestrictionTarget: fromElement() static method
Flaw count: 1

bad_bcd_queries:
- No BCD data for query: api.RestrictionTarget.fromElement_static

URL: /en-US/docs/Web/API/Screen_Capture_extensions
Title: Screen Capture extensions
Flaw count: 1

bad_bcd_queries:
- No BCD data for query: api.RestrictionTarget

External URLs (8)

URL: /en-US/docs/Web/API/Screen_Capture_extensions
Title: Screen Capture extensions

https://developer.chrome.com/docs/web-platform/element-capture (1 time) (Note! This may be a new URL 👀)
https://developer.chrome.com/docs/web-platform/region-capture (1 time) (Note! This may be a new URL 👀)
https://element-capture-demo.glitch.me/ (1 time) (Note! This may be a new URL 👀)
https://mdn.github.io/dom-examples/screen-capture-api (1 time) (Note! This may be a new URL 👀)

URL: /en-US/docs/Web/API/Screen_Capture_extensions/Element_Region_Capture
Title: Using the Element Capture and Region Capture APIs

https://element-capture-demo.glitch.me/ (1 time) (Note! This may be a new URL 👀)
https://mdn.github.io/dom-examples/screen-capture-api/basic-screen-capture (1 time) (Note! This may be a new URL 👀)
https://mdn.github.io/dom-examples/screen-capture-api/element-capture (2 times) (Note! This may be a new URL 👀)
https://mdn.github.io/dom-examples/screen-capture-api/region-capture (2 times) (Note! This may be a new URL 👀)

(comment last updated: 2024-11-29 10:06:42)

skyclouds2001 · 2024-11-24T17:55:40Z

mdn/mdn#580 could be close with this PR

chrisdavidmills · 2024-11-25T09:37:19Z

mdn/mdn#580 could be close with this PR

Cool, thanks for the heads-up @skyclouds2001. I'll update the description to mention it.

…ls/content into region-and-element-capture

files/en-us/web/api/browsercapturemediastreamtrack/cropto/index.md

files/en-us/web/api/browsercapturemediastreamtrack/clone/index.md

files/en-us/web/api/croptarget/fromelement_static/index.md

files/en-us/web/api/croptarget/index.md

beaufortfrancois · 2024-11-27T14:58:53Z

files/en-us/web/api/browsercapturemediastreamtrack/clone/index.md

+
+// Capture video stream from user's webcam and isolate video track
+const stream =
+  await navigator.mediaDevices.getDisplayMedia(displayMediaOptions);


I believe you can remove displayMediaOptions everywhere as {video: true} is the default where it works.

The getDisplayMedia() call does work with no options, yes. My concern with this is that to get Region or Element capture working, you need to do a same-tab capture, otherwise, it doesn't work, and to achieve this, you need the preferCurrentTab: true option. Therefore, to be on the safe side, I've decided to include the following minimal options object at the start of each one of these code snippets:

// Options for getDisplayMedia() const displayMediaOptions = { preferCurrentTab: true, };

Gotcha. Note that preferCurrentTab is a hint, and that user can still pick another tab/screen/display.

When I run my demo on Chrome, it only gives me the option of selecting the current tab when I start the capture. This is what I get:

In Chrome yes, but spec says "SHOULD". See https://wicg.github.io/prefer-current-tab/#ref-for-dom-mediastreamconstraints-prefercurrenttab-1

ok, fair enough. I have updated the text in the guide to make it clearer that this is a hint:

This hint suggests that the browser should offer the user's current tab as the most prominent capture source in the dialog that asks them what to share. Chrome, for example, only gives them this option when preferCurrentTab: true is set.

files/en-us/web/api/browsercapturemediastreamtrack/cropto/index.md

files/en-us/web/api/browsercapturemediastreamtrack/restrictto/index.md

files/en-us/web/api/screen_capture_extensions/element_region_capture/index.md

beaufortfrancois · 2024-11-27T15:20:38Z

files/en-us/web/api/screen_capture_extensions/element_region_capture/index.md

+
+This could be a major problem if that content is private, for example message notifications or speaker notes. If you can't guarantee that your app can keep private content from being shared, then you may lose trust in your users.
+
+As a result, it is generally recommended that you use the newer Element Capture API over the older Region Capture API, unless you need to support older browser versions that do not support Element Capture.


I'm not sure that's the right conclusion we aimed regarding these two APIs.
My understanding is that both APIs have legitimate use cases, and that Element Capture does not supersede Region Capture.

FYI @eladalon1983

I'd appreciate some guidance here on what to say here @beaufortfrancois @eladalon1983 — I am not clear on what the distinct use cases are for each.

If it's okay with you, let's wait until mid-January when @eladalon1983 returns, as there's nothing urgent that needs to be addressed immediately.

Sure, that's fine by me. Do you want to get this published before that, and then update with better information once @eladalon1983 returns, or hold off on publication?

Let's hold off if that's okay.

OK, that's fine.

beaufortfrancois · 2024-11-29T08:56:22Z

files/en-us/web/api/screen_capture_extensions/element_region_capture/index.md

+  video: {
+    displaySurface: "window",
+  },
+  audio: false,


audio: false is the default option

ok, makes sense to remove it in the guide. I have done so.

Adding landing page for Screen Capture extrensions

6edd354

chrisdavidmills requested review from a team as code owners November 23, 2024 16:52

chrisdavidmills requested review from sideshowbarker and bsmth and removed request for a team November 23, 2024 16:52

chrisdavidmills marked this pull request as draft November 23, 2024 16:52

github-actions bot added Content:WebAPI Web API docs size/m [PR only] 51-500 LoC changed labels Nov 23, 2024

Add usage guide

3e80b04

chrisdavidmills mentioned this pull request Nov 25, 2024

Add DOM examples for Screen Capture API mdn/dom-examples#285

Merged

Add reference pages

bc36f1c

github-actions bot added size/l [PR only] 501-1000 LoC changed and removed size/m [PR only] 51-500 LoC changed labels Nov 26, 2024

chrisdavidmills marked this pull request as ready for review November 26, 2024 14:58

chrisdavidmills added 3 commits November 26, 2024 14:58

Merge branch 'main' into region-and-element-capture

96bc1f5

Fix a couple of flaws

27e5207

Merge branch 'region-and-element-capture' of github.com:chrisdavidmil…

536f60a

…ls/content into region-and-element-capture

chrisdavidmills mentioned this pull request Nov 26, 2024

Chrome 132 supports Element Capture API mdn/browser-compat-data#25232

Merged

beaufortfrancois reviewed Nov 27, 2024

View reviewed changes

Fixes for beaufortfrancois review comments

5c20d74

beaufortfrancois reviewed Nov 29, 2024

View reviewed changes

chrisdavidmills added 2 commits November 29, 2024 11:00

remove audio: false,

2e95813

Make it clearer that preferCurrentTab is a hint

cb3315e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add documentation for Screen Capture extensions, element capture and region capture #36939

Add documentation for Screen Capture extensions, element capture and region capture #36939

chrisdavidmills commented Nov 23, 2024 •

edited

Loading

github-actions bot commented Nov 23, 2024 •

edited

Loading

skyclouds2001 commented Nov 24, 2024

chrisdavidmills commented Nov 25, 2024

beaufortfrancois Nov 27, 2024

chrisdavidmills Nov 29, 2024

beaufortfrancois Nov 29, 2024

chrisdavidmills Nov 29, 2024

beaufortfrancois Nov 29, 2024

chrisdavidmills Nov 29, 2024

beaufortfrancois Nov 27, 2024 •

edited

Loading

chrisdavidmills Nov 27, 2024

beaufortfrancois Nov 28, 2024

chrisdavidmills Nov 29, 2024

beaufortfrancois Nov 29, 2024

chrisdavidmills Nov 29, 2024

beaufortfrancois Nov 29, 2024

chrisdavidmills Nov 29, 2024


		This could be a major problem if that content is private, for example message notifications or speaker notes. If you can't guarantee that your app can keep private content from being shared, then you may lose trust in your users.

		As a result, it is generally recommended that you use the newer Element Capture API over the older Region Capture API, unless you need to support older browser versions that do not support Element Capture.

Add documentation for Screen Capture extensions, element capture and region capture #36939

Are you sure you want to change the base?

Add documentation for Screen Capture extensions, element capture and region capture #36939

Conversation

chrisdavidmills commented Nov 23, 2024 • edited Loading

Description

Motivation

Additional details

Related issues and pull requests

github-actions bot commented Nov 23, 2024 • edited Loading

skyclouds2001 commented Nov 24, 2024

chrisdavidmills commented Nov 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

beaufortfrancois Nov 27, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chrisdavidmills commented Nov 23, 2024 •

edited

Loading

github-actions bot commented Nov 23, 2024 •

edited

Loading

beaufortfrancois Nov 27, 2024 •

edited

Loading