Add a section for the background blur field in VideoFrameMetadata #159

markafoltz · 2024-11-12T20:03:59Z

See https://github.com/markafoltz/camera-effects for more details.

Preview | Diff

See https://github.com/markafoltz/camera-effects for more details.

markafoltz

Overall looks good, a couple of feedbacks on the IDL.

index.html

markafoltz · 2024-11-12T20:06:13Z

index.html

@@ -1876,6 +1876,76 @@ <h3>Examples</h3>
      </pre>
    </section>
  </section>
+  <section>
+    <h2>Camera Effect Status</h2>
+    <p>The {{VideoFrameMetadata}} interface exposes the effect state as a property,


Suggested change

The {{VideoFrameMetadata}} interface exposes the effect state as a property,

The {{VideoFrameMetadata}} interface exposes the blur state as a property,

markafoltz · 2024-11-12T20:07:27Z

index.html

+        <dl class="dictionary-members" data-link-for="VideoFrameMetadata" data-dfn-for="VideoFrameMetadata">
+          <dt><dfn><code>background_blur</code></dfn> of type <code>{{BackgroundBlur}}</code></dt>
+          <dd>
+            <p>The state of the background blur effect for the current frame.</p>


Should explain what it means for the property to be present or missing.

markafoltz · 2024-11-12T20:10:11Z

index.html

+      <h3>{{BackgroundBlur}}</h3>
+      <pre class="idl">
+dictionary BackgroundBlur {
+  boolean enabled;


I think this should be a required property. If background_blur is present, enabled should have a value.

aboba

Other than the MSTG/VTG issue, looks ok.

aboba · 2024-11-22T22:39:10Z

index.html

+const videoTrack = stream.getVideoTracks()[0];
+const blurIndicator = document.getElementById("blurIndicator");
+const trackProcessor = new MediaStreamTrackProcessor({ track: videoTrack });
+const trackGenerator = new MediaStreamTrackGenerator({ kind: "video" });


You need to use the standardized API (VideoTrackGenerator)

markafoltz · 2024-11-26T15:26:10Z

index.html

+      <section class="notoc">
+        <h4>Members</h4>
+        <dl class="dictionary-members" data-link-for="VideoFrameMetadata" data-dfn-for="VideoFrameMetadata">
+          <dt><dfn><code>background_blur</code></dfn> of type <code>{{BackgroundBlur}}</code></dt>


Properties should be camelCase.

eladalon1983 · 2024-11-29T18:31:04Z

index.html

+    <section>
+      <h3>Example</h3>
+      <pre class="example">
+// main.js:


Is this contrasted with code in another .js file or elsewhere?

It's just leftover from a copy/paste. Removed.

* background_blur -> backgroundBlur * MediaStreamTrackGenerator -> VideoTrackGenerator * Removed main.js in code snippet

brychanrobot · 2025-01-31T23:51:00Z

Ping on this. I think it's ready for merge as I've addressed all of the comments.

brychanrobot · 2025-02-04T19:12:08Z

CC: @guidou, @alvestrand, @jan-ivar, @youennf for review in the next editors meeting.

jan-ivar · 2025-02-05T17:04:44Z

index.html

+          <dt><dfn><code>backgroundBlur</code></dfn> of type <code>{{BackgroundBlur}}</code></dt>
+          <dd>
+            <p>The state of the background blur effect for the current frame. If
+              the property isn't present, it indicates that the user agent doesn't


Suggested change

the property isn't present, it indicates that the user agent doesn't

the property isn't [=map/exist|present=], it indicates that the user agent doesn't

...but see overlapping suggestion below before committing.

jan-ivar · 2025-02-05T17:11:58Z

index.html

+            <p>The state of the background blur effect for the current frame. If
+              the property isn't present, it indicates that the user agent doesn't
+              support reporting blur state.


Is it supposed to be present on VideoFrames from non-camera sources? The text seems to assume every VideoFrame has a background blur effect state, when AFAIK only cameras have this? Are we to intuit {backgroundBlur: false} in those cases? If so, we might want to clarify that.

What's the precedent here for metadata specific to only certain sources?

I've opened w3c/webcodecs#879, but unless there's a precedent (which I couldn't find), it seems better to me for it to only be present in VideoFrames from cameras. That way false represents a user setting,.

This way, downstream code can differentiate false that from absence (unknown) and make a more informed decision about transmission.

This would have the effect of revealing that the VideoFrame came from a camera, but I can't think of a situation where that would be problematic.

How about:

Suggested change

The state of the background blur effect for the current frame. If

the property isn't present, it indicates that the user agent doesn't

support reporting blur state.

The state of the background blur effect for the current frame, if

[=map/exist|present=]. Absence might indicate that the frame is not

from a camera, or the user agent might not support reporting blur state.

?

I've softened it a bit with "might" since the VideoFrame has a constructor, which means presence and values might have more to do with what metadata was passed in by JS.

jan-ivar · 2025-02-05T18:42:22Z

index.html

+      which allows apps to know the state for every frame. This is important for
+      scenarios where the app must ensure user privacy by never sending an


Let's avoid normative "must". I also don't think we want to overstate the security properties of camera background blurring. OS implementations might differ significantly, and they don't necessarily pass everyone's definition of a security measure.

Suggested change

which allows apps to know the state for every frame. This is important for

scenarios where the app must ensure user privacy by never sending an

which allows apps to know the state for every frame. This can be helpful for

scenarios where the app wishes to help protect user privacy by never sending an

jan-ivar · 2025-02-05T18:45:53Z

index.html

+    <p>The {{VideoFrameMetadata}} interface exposes the blur state as a property,
+      which allows apps to know the state for every frame. This is important for
+      scenarios where the app must ensure user privacy by never sending an
+      un-blurred frame off of the user's device.</p>


Mediacapture uses "device" to mean cameras and mics. Maybe:

Suggested change

un-blurred frame off of the user's device.

un-blurred frame off of the user's system.

Add a section for the background blur field in VideoFrameMetadata

a1077cb

See https://github.com/markafoltz/camera-effects for more details.

markafoltz commented Nov 12, 2024

View reviewed changes

Address PR comments

2e42d6a

guidou requested review from alvestrand, aboba, jan-ivar, youennf and guidou November 22, 2024 21:51

aboba requested changes Nov 22, 2024

View reviewed changes

markafoltz commented Nov 26, 2024

View reviewed changes

eladalon1983 reviewed Nov 29, 2024

View reviewed changes

Address PR comments

5c7c78f

* background_blur -> backgroundBlur * MediaStreamTrackGenerator -> VideoTrackGenerator * Removed main.js in code snippet

jan-ivar requested changes Feb 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a section for the background blur field in VideoFrameMetadata #159

Add a section for the background blur field in VideoFrameMetadata #159

markafoltz commented Nov 12, 2024 •

edited by pr-preview bot

Loading

markafoltz left a comment

markafoltz Nov 12, 2024

brychanrobot Nov 15, 2024

markafoltz Nov 12, 2024

brychanrobot Nov 15, 2024

markafoltz Nov 12, 2024

brychanrobot Nov 15, 2024

aboba left a comment

aboba Nov 22, 2024

brychanrobot Jan 14, 2025

markafoltz Nov 26, 2024

brychanrobot Jan 14, 2025

eladalon1983 Nov 29, 2024

brychanrobot Jan 14, 2025

brychanrobot commented Jan 31, 2025

brychanrobot commented Feb 4, 2025

jan-ivar Feb 5, 2025 •

edited

Loading

jan-ivar Feb 5, 2025

jan-ivar Feb 5, 2025 •

edited

Loading

jan-ivar Feb 5, 2025

	<p>The {{VideoFrameMetadata}} interface exposes the effect state as a property,
	<p>The {{VideoFrameMetadata}} interface exposes the blur state as a property,

	the property isn't present, it indicates that the user agent doesn't
	the property isn't [=map/exist\|present=], it indicates that the user agent doesn't

		which allows apps to know the state for every frame. This is important for
		scenarios where the app must ensure user privacy by never sending an

	un-blurred frame off of the user's device.</p>
	un-blurred frame off of the user's system.</p>

Add a section for the background blur field in VideoFrameMetadata #159

Are you sure you want to change the base?

Add a section for the background blur field in VideoFrameMetadata #159

Conversation

markafoltz commented Nov 12, 2024 • edited by pr-preview bot Loading

markafoltz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aboba left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brychanrobot commented Jan 31, 2025

brychanrobot commented Feb 4, 2025

jan-ivar Feb 5, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jan-ivar Feb 5, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markafoltz commented Nov 12, 2024 •

edited by pr-preview bot

Loading

jan-ivar Feb 5, 2025 •

edited

Loading

jan-ivar Feb 5, 2025 •

edited

Loading