Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DENG-6360 complete backfill with shredder mitigation for desktop aua v4 #6539

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

lucia-vargas-a
Copy link
Contributor

@lucia-vargas-a lucia-vargas-a commented Nov 20, 2024

Description

This PR completes the backfill for firefox_desktop_derived.active_users_aggregates_v4 and updates the views to use the new version. Details in the related ticket.

Reviewer, please note that approving this PR indicates that KPI data is validated and correct to move to production.

Related Tickets & Documents

DENG-6360

Reviewer, please follow this checklist

┆Issue is synchronized with this Jira Task

@dataops-ci-bot

This comment has been minimized.

@lucia-vargas-a lucia-vargas-a changed the title Deng 6360 complete backfill with shredder mitigation for desktop aua v4 DENG-6360 complete backfill with shredder mitigation for desktop aua v4 Nov 21, 2024
@lucia-vargas-a lucia-vargas-a requested review from bochocki and a team November 21, 2024 10:35
@dataops-ci-bot
Copy link

Integration report for "Update desktop column names in view active_users_aggregates. Use table_name in the metadata template."

sql.diff

Click to expand!
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_analytics_aggregations.py /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_analytics_aggregations.py	2024-11-21 10:35:29.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py	2024-11-21 10:49:09.000000000 +0000
@@ -627,9 +627,9 @@
         retries=0,
     )
 
-    checks__warn_firefox_desktop_derived__active_users_aggregates__v3 = bigquery_dq_check(
-        task_id="checks__warn_firefox_desktop_derived__active_users_aggregates__v3",
-        source_table='active_users_aggregates_v3${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
+    checks__warn_firefox_desktop_derived__active_users_aggregates__v4 = bigquery_dq_check(
+        task_id="checks__warn_firefox_desktop_derived__active_users_aggregates__v4",
+        source_table='active_users_aggregates_v4${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_desktop_derived",
         project_id="moz-fx-data-shared-prod",
         is_dq_check_fail=False,
@@ -745,9 +745,9 @@
         parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
-    firefox_desktop_active_users_aggregates_v3 = bigquery_etl_query(
-        task_id="firefox_desktop_active_users_aggregates_v3",
-        destination_table='active_users_aggregates_v3${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
+    firefox_desktop_active_users_aggregates_v4 = bigquery_etl_query(
+        task_id="firefox_desktop_active_users_aggregates_v4",
+        destination_table='active_users_aggregates_v4${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_desktop_derived",
         project_id="moz-fx-data-shared-prod",
         owner="[email protected]",
@@ -762,17 +762,17 @@
     )
 
     with TaskGroup(
-        "firefox_desktop_active_users_aggregates_v3_external",
-    ) as firefox_desktop_active_users_aggregates_v3_external:
+        "firefox_desktop_active_users_aggregates_v4_external",
+    ) as firefox_desktop_active_users_aggregates_v4_external:
         ExternalTaskMarker(
-            task_id="bqetl_dynamic_dau__wait_for_firefox_desktop_active_users_aggregates_v3",
+            task_id="bqetl_dynamic_dau__wait_for_firefox_desktop_active_users_aggregates_v4",
             external_dag_id="bqetl_dynamic_dau",
-            external_task_id="wait_for_firefox_desktop_active_users_aggregates_v3",
+            external_task_id="wait_for_firefox_desktop_active_users_aggregates_v4",
             execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=51300)).isoformat() }}",
         )
 
-        firefox_desktop_active_users_aggregates_v3_external.set_upstream(
-            firefox_desktop_active_users_aggregates_v3
+        firefox_desktop_active_users_aggregates_v4_external.set_upstream(
+            firefox_desktop_active_users_aggregates_v4
         )
 
     firefox_ios_active_users_aggregates_v3 = bigquery_etl_query(
@@ -959,12 +959,12 @@
         fenix_active_users_aggregates_v3
     )
 
-    checks__warn_firefox_desktop_derived__active_users_aggregates__v3.set_upstream(
+    checks__warn_firefox_desktop_derived__active_users_aggregates__v4.set_upstream(
         wait_for_checks__fail_telemetry_derived__clients_last_seen__v2
     )
 
-    checks__warn_firefox_desktop_derived__active_users_aggregates__v3.set_upstream(
-        firefox_desktop_active_users_aggregates_v3
+    checks__warn_firefox_desktop_derived__active_users_aggregates__v4.set_upstream(
+        firefox_desktop_active_users_aggregates_v4
     )
 
     checks__warn_firefox_ios_derived__active_users_aggregates__v3.set_upstream(
@@ -1015,7 +1015,7 @@
         wait_for_fenix_derived__metrics_clients_last_seen__v1
     )
 
-    firefox_desktop_active_users_aggregates_v3.set_upstream(
+    firefox_desktop_active_users_aggregates_v4.set_upstream(
         wait_for_checks__fail_telemetry_derived__clients_last_seen__v2
     )
 
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_dynamic_dau.py /tmp/workspace/generated-sql/dags/bqetl_dynamic_dau.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_dynamic_dau.py	2024-11-21 10:35:29.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_dynamic_dau.py	2024-11-21 10:49:11.000000000 +0000
@@ -126,10 +126,10 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_firefox_desktop_active_users_aggregates_v3 = ExternalTaskSensor(
-        task_id="wait_for_firefox_desktop_active_users_aggregates_v3",
+    wait_for_firefox_desktop_active_users_aggregates_v4 = ExternalTaskSensor(
+        task_id="wait_for_firefox_desktop_active_users_aggregates_v4",
         external_dag_id="bqetl_analytics_aggregations",
-        external_task_id="firefox_desktop_active_users_aggregates_v3",
+        external_task_id="firefox_desktop_active_users_aggregates_v4",
         execution_delta=datetime.timedelta(seconds=35100),
         check_existence=True,
         mode="reschedule",
@@ -194,5 +194,5 @@
     )
 
     telemetry_derived__segmented_dau_28_day_rolling__v1.set_upstream(
-        wait_for_firefox_desktop_active_users_aggregates_v3
+        wait_for_firefox_desktop_active_users_aggregates_v4
     )
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived: active_users_aggregates_v4
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived: active_users_aggregates_v3
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v4: backfill.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived: active_users_aggregates_v4
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived: active_users_aggregates_v4
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived: active_users_aggregates_v4
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived: active_users_aggregates_v4
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived: active_users_aggregates_v4
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/active_users_aggregates: schema.yaml
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:33:26.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-20"
+    DATE(submission_timestamp) >= "2024-11-21"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:33:26.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-20"
+    DATE(submission_timestamp) >= "2024-11-21"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:33:26.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-20"
+    DATE(submission_timestamp) >= "2024-11-21"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:33:26.000000000 +0000
@@ -87,7 +87,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-20"
+    DATE(submission_timestamp) >= "2024-11-21"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:30:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:33:26.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-20"
+    DATE(submission_timestamp) >= "2024-11-21"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-11-21 10:29:41.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-11-21 10:38:34.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-11-21 10:29:41.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-11-21 10:38:46.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:33:27.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-20"
+    DATE(submission_timestamp) >= "2024-11-21"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2024-11-21 10:29:41.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2024-11-21 10:38:44.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2024-11-21 10:29:41.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2024-11-21 10:38:52.000000000 +0000
@@ -48,6 +48,10 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
+  description: null
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/use_counters/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/use_counters/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/use_counters/schema.yaml	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/use_counters/schema.yaml	2024-11-21 10:40:59.000000000 +0000
@@ -1,11 +1,11 @@
 fields:
 - name: normalized_app_id
-  type: STRING
   mode: NULLABLE
+  type: STRING
   description: App ID of the channel data was received from
 - name: normalized_channel
-  type: STRING
   mode: NULLABLE
+  type: STRING
   description: Normalized channel name
 - name: additional_properties
   type: STRING
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/checks.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/checks.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/checks.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/checks.sql	1970-01-01 00:00:00.000000000 +0000
@@ -1,152 +0,0 @@
-
-#warn
-WITH daily_users_sum AS (
-  SELECT
-    SUM(daily_users),
-  FROM
-    `{{ project_id }}.{{ dataset_id }}.{{ table_name }}`
-  WHERE
-    submission_date = @submission_date
-),
-distinct_client_count_nightly_base AS (
-  SELECT
-    client_info.client_id,
-    "nightly" AS channel,
-  FROM
-    `moz-fx-data-shared-prod.org_mozilla_fenix_live.baseline_v1`
-  WHERE
-    DATE(submission_timestamp) = @submission_date
-    AND mozfun.norm.fenix_app_info("org_mozilla_fenix", client_info.app_build).channel = "nightly"
-    -- NOTE: The next two tables `org_mozilla_fenix_nightly_live.baseline_v1` and `org_mozilla_fennec_aurora_live.baseline_v1`
-    -- are not used as application IDs in Glean, but are also not yet marked as deprecated because they still count for KPIs:
-    -- Related PR https://github.com/mozilla/probe-scraper/pull/640.
-  UNION ALL
-  SELECT
-    client_info.client_id,
-    "nightly" AS channel,
-  FROM
-    `moz-fx-data-shared-prod.org_mozilla_fenix_nightly_live.baseline_v1`
-  WHERE
-    DATE(submission_timestamp) = @submission_date
-    AND mozfun.norm.fenix_app_info(
-      "org_mozilla_fenix_nightly",
-      client_info.app_build
-    ).channel = "nightly"
-  UNION ALL
-  SELECT
-    client_info.client_id,
-    "nightly" AS channel,
-  FROM
-    `moz-fx-data-shared-prod.org_mozilla_fennec_aurora_live.baseline_v1`
-  WHERE
-    DATE(submission_timestamp) = @submission_date
-    AND mozfun.norm.fenix_app_info(
-      "org_mozilla_fennec_aurora",
-      client_info.app_build
-    ).channel = "nightly"
-),
-distinct_client_count_base AS (
-    -- release channel
-  SELECT
-    client_info.client_id,
-    "release" AS channel,
-  FROM
-    `moz-fx-data-shared-prod.org_mozilla_firefox_live.baseline_v1`
-  WHERE
-    DATE(submission_timestamp) = @submission_date
-    AND mozfun.norm.fenix_app_info("org_mozilla_firefox", client_info.app_build).channel = "release"
-    -- beta channel
-  UNION ALL
-  SELECT
-    client_info.client_id,
-    "beta" AS channel,
-  FROM
-    `moz-fx-data-shared-prod.org_mozilla_firefox_beta_live.baseline_v1`
-  WHERE
-    DATE(submission_timestamp) = @submission_date
-    AND mozfun.norm.fenix_app_info(
-      "org_mozilla_firefox_beta",
-      client_info.app_build
-    ).channel = "beta"
-    -- NOTE: nightly table also contains some entries considered to be "beta" channel by our ETL
-    -- this is why the below entries are included here.
-  UNION ALL
-  SELECT
-    client_info.client_id,
-    "beta" AS channel,
-  FROM
-    `moz-fx-data-shared-prod.org_mozilla_fenix_live.baseline_v1`
-  WHERE
-    DATE(submission_timestamp) = @submission_date
-    AND mozfun.norm.fenix_app_info("org_mozilla_fenix", client_info.app_build).channel = "beta"
-    -- nightly channel
-  UNION ALL
-  SELECT
-    client_id,
-    channel
-  FROM
-    distinct_client_count_nightly_base
-  LEFT JOIN
-    `moz-fx-data-shared-prod.fenix.baseline_clients_last_seen` AS baseline_clients_last_seen
-    USING (client_id)
-  WHERE
-    baseline_clients_last_seen.submission_date = @submission_date
-    AND baseline_clients_last_seen.days_since_seen = 0
-),
-distinct_client_counts_per_channel AS (
-  SELECT
-    channel,
-    COUNT(DISTINCT client_id) AS distinct_client_count,
-  FROM
-    distinct_client_count_base
-  GROUP BY
-    channel
-),
-distinct_client_count AS (
-  SELECT
-    SUM(distinct_client_count),
-  FROM
-    distinct_client_counts_per_channel
-)
-SELECT
-  IF(
-    ABS((SELECT * FROM daily_users_sum) - (SELECT * FROM distinct_client_count)) > 10,
-    ERROR(
-      CONCAT(
-        "Daily_users mismatch between the firefox_ios live (`org_mozilla_firefox_live`, `org_mozilla_fenix_live.baseline_v1`,`org_mozilla_firefox_beta_live.baseline_v1`,`org_mozilla_fenix_nightly_live.baseline_v1`, `org_mozilla_fennec_aurora_live.baseline_v1`) and active_users_aggregates (`{{ dataset_id }}.{{ table_name }}`) tables is greater than 10.",
-        " Live table count: ",
-        (SELECT * FROM distinct_client_count),
-        " | active_users_aggregates (DAU): ",
-        (SELECT * FROM daily_users_sum),
-        " | Delta detected: ",
-        ABS((SELECT * FROM daily_users_sum) - (SELECT * FROM distinct_client_count))
-      )
-    ),
-    NULL
-  );
-
-#fail
-WITH dau_current AS (
-  SELECT
-    SUM(dau) AS dau
-  FROM
-    `{{ project_id }}.{{ dataset_id }}.{{ table_name }}`
-  WHERE
-    submission_date = @submission_date
-),
-dau_previous AS (
-  SELECT
-    SUM(dau) AS dau
-  FROM
-    `{{ project_id }}.{{ dataset_id }}.{{ table_name }}`
-  WHERE
-    submission_date = DATE_SUB(@submission_date, INTERVAL 1 DAY)
-)
-SELECT
-  IF(
-    ABS((SELECT SUM(dau) FROM dau_current) / (SELECT SUM(dau) FROM dau_previous)) > 1.5,
-    ERROR(
-      "Current date's DAU is 50% higher than in previous date. See source table (`{{ project_id }}.{{ dataset_id }}.{{ table_name }}`)!"
-    ),
-    NULL
-  );
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/metadata.yaml	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,60 +0,0 @@
-friendly_name: Fenix Active Users Aggregates
-description: |-
-  This table contains dau, wau, mau, daily users,
-  weekly users and monthly users for Fenix,
-  aggregated by submission_date, attribution, channel,
-  country, city, device model, distribution_id, os details
-  and activity segment.
-
-  - dau is counting the users who reported a ping on the date and
-  are qualified as active users.
-  - daily_users counts all the users who reported a ping on the date.
-  Only dau is exposed in the view telemetry.active_users_aggregates.
-
-  The table is labeled as "change_controlled", which implies
-  that changes require the approval of at least one owner.
-
-  The label "shredder mitigation" indicates that this table is set up for
-  managed backfill with shredder mitigation, as described in
-  https://mozilla.github.io/bigquery-etl/cookbooks/creating_a_derived_dataset/#initiating-the-backfill.
-
-  Proposal:
-  https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
-owners:
-- [email protected]
-- mozilla/kpi_table_reviewers
-labels:
-  incremental: true
-  change_controlled: true
-  shredder_mitigation: true
-  owner1: lvargas
-bigquery:
-  time_partitioning:
-    type: day
-    field: submission_date
-    require_partition_filter: true
-    expiration_days: null
-  range_partitioning: null
-  clustering:
-    fields:
-    - country
-    - app_name
-    - attribution_medium
-    - channel
-workgroup_access:
-- role: roles/bigquery.dataViewer
-  members:
-  - workgroup:mozilla-confidential
-references:
-  checks.sql:
-  - ..
-  - moz-fx-data-shared-prod.fenix.baseline_clients_last_seen
-  - moz-fx-data-shared-prod.org_mozilla_fenix_live.baseline_v1
-  - moz-fx-data-shared-prod.org_mozilla_fenix_nightly_live.baseline_v1
-  - moz-fx-data-shared-prod.org_mozilla_fennec_aurora_live.baseline_v1
-  - moz-fx-data-shared-prod.org_mozilla_firefox_beta_live.baseline_v1
-  - moz-fx-data-shared-prod.org_mozilla_firefox_live.baseline_v1
-  query.sql:
-  - fenix.firefox_android_clients
-  - moz-fx-data-shared-prod.fenix.active_users
-  - moz-fx-data-shared-prod.fenix.metrics_clients_last_seen
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/query.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/query.sql	1970-01-01 00:00:00.000000000 +0000
@@ -1,202 +0,0 @@
---- Query generated via sql_generators.active_users.
-WITH attribution_data AS (
-  SELECT
-    client_id,
-    adjust_network,
-    install_source
-  FROM
-    `moz-fx-data-shared-prod.fenix.firefox_android_clients`
-),
-baseline AS (
-  SELECT
-    submission_date,
-    normalized_channel,
-    client_id,
-    days_active_bits,
-    days_created_profile_bits,
-    normalized_os,
-    normalized_os_version,
-    locale,
-    city,
-    country,
-    app_display_version,
-    device_model,
-    first_seen_date,
-    submission_date = first_seen_date AS is_new_profile,
-    distribution_id,
-    isp,
-    app_name,
-    activity_segment AS segment,
-    is_daily_user,
-    is_weekly_user,
-    is_monthly_user,
-    is_dau,
-    is_wau,
-    is_mau
-  FROM
-    `moz-fx-data-shared-prod.fenix.active_users`
-  WHERE
-    submission_date = @submission_date
-),
-metrics AS (
-    -- Metrics ping may arrive in the same or next day as the baseline ping.
-  SELECT
-    client_id,
-    ARRAY_AGG(normalized_channel IGNORE NULLS ORDER BY submission_date ASC)[
-      SAFE_OFFSET(0)
-    ] AS normalized_channel,
-    ARRAY_AGG(uri_count IGNORE NULLS ORDER BY submission_date ASC)[SAFE_OFFSET(0)] AS uri_count,
-    ARRAY_AGG(is_default_browser IGNORE NULLS ORDER BY submission_date ASC)[
-      SAFE_OFFSET(0)
-    ] AS is_default_browser
-  FROM
-    `moz-fx-data-shared-prod.fenix.metrics_clients_last_seen`
-  WHERE
-    DATE(submission_date)
-    BETWEEN @submission_date
-    AND DATE_ADD(@submission_date, INTERVAL 1 DAY)
-  GROUP BY
-    client_id
-),
-unioned AS (
-  SELECT
-    baseline.client_id,
-    baseline.segment,
-    baseline.app_name,
-    baseline.app_display_version AS app_version,
-    baseline.normalized_channel,
-    IFNULL(baseline.country, '??') country,
-    baseline.city,
-    baseline.days_created_profile_bits,
-    baseline.device_model,
-    baseline.isp,
-    baseline.is_new_profile,
-    baseline.locale,
-    baseline.first_seen_date,
-    baseline.normalized_os,
-    baseline.normalized_os_version,
-    COALESCE(
-      SAFE_CAST(NULLIF(SPLIT(baseline.normalized_os_version, ".")[SAFE_OFFSET(0)], "") AS INTEGER),
-      0
-    ) AS os_version_major,
-    COALESCE(
-      SAFE_CAST(NULLIF(SPLIT(baseline.normalized_os_version, ".")[SAFE_OFFSET(1)], "") AS INTEGER),
-      0
-    ) AS os_version_minor,
-    COALESCE(
-      SAFE_CAST(NULLIF(SPLIT(baseline.normalized_os_version, ".")[SAFE_OFFSET(2)], "") AS INTEGER),
-      0
-    ) AS os_version_patch,
-    baseline.submission_date,
-    metrics.uri_count,
-    metrics.is_default_browser,
-    baseline.distribution_id,
-    CAST(NULL AS string) AS attribution_content,
-    CAST(NULL AS string) AS attribution_source,
-    CAST(NULL AS string) AS attribution_medium,
-    CAST(NULL AS string) AS attribution_campaign,
-    CAST(NULL AS string) AS attribution_experiment,
-    CAST(NULL AS string) AS attribution_variation,
-    CAST(NULL AS FLOAT64) AS active_hours_sum,
-    is_daily_user,
-    is_weekly_user,
-    is_monthly_user,
-    is_dau,
-    is_wau,
-    is_mau
-  FROM
-    baseline
-  LEFT JOIN
-    metrics
-    ON baseline.client_id = metrics.client_id
-    AND baseline.normalized_channel IS NOT DISTINCT FROM metrics.normalized_channel
-),
-unioned_with_attribution AS (
-  SELECT
-    unioned.*,
-    attribution_data.install_source,
-    attribution_data.adjust_network
-  FROM
-    unioned
-  LEFT JOIN
-    attribution_data
-    USING (client_id)
-),
-todays_metrics AS (
-  SELECT
-    segment,
-    app_version,
-    attribution_medium,
-    attribution_source,
-    attribution_medium IS NOT NULL
-    OR attribution_source IS NOT NULL AS attributed,
-    city,
-    country,
-    distribution_id,
-    EXTRACT(YEAR FROM first_seen_date) AS first_seen_year,
-    is_default_browser,
-    COALESCE(REGEXP_EXTRACT(locale, r'^(.+?)-'), locale, NULL) AS locale,
-    app_name AS app_name,
-    normalized_channel AS channel,
-    normalized_os AS os,
-    normalized_os_version AS os_version,
-    os_version_major,
-    os_version_minor,
-    submission_date,
-    client_id,
-    uri_count,
-    active_hours_sum,
-    adjust_network,
-    install_source,
-    is_daily_user,
-    is_weekly_user,
-    is_monthly_user,
-    is_dau,
-    is_wau,
-    is_mau
-  FROM
-    unioned_with_attribution
-)
-SELECT
-  todays_metrics.* EXCEPT (
-    client_id,
-    is_daily_user,
-    is_weekly_user,
-    is_monthly_user,
-    is_dau,
-    is_wau,
-    is_mau,
-    uri_count,
-    active_hours_sum
-  ),
-  COUNTIF(is_daily_user) AS daily_users,
-  COUNTIF(is_weekly_user) AS weekly_users,
-  COUNTIF(is_monthly_user) AS monthly_users,
-  COUNTIF(is_dau) AS dau,
-  COUNTIF(is_wau) AS wau,
-  COUNTIF(is_mau) AS mau,
-  SUM(uri_count) AS uri_count,
-  SUM(active_hours_sum) AS active_hours,
-FROM
-  todays_metrics
-GROUP BY
-  segment,
-  app_version,
-  attribution_medium,
-  attribution_source,
-  attributed,
-  city,
-  country,
-  distribution_id,
-  first_seen_year,
-  is_default_browser,
-  locale,
-  app_name,
-  channel,
-  os,
-  os_version,
-  os_version_major,
-  os_version_minor,
-  submission_date,
-  adjust_network,
-  install_source
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/schema.yaml	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v4/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,113 +0,0 @@
-fields:
-- name: segment
-  type: STRING
-  mode: NULLABLE
-  description: Classification of client_ids based on usage and active state.
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-  description: Browser name.
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-  description: Browser version installed on the client.
-- name: channel
-  type: STRING
-  mode: NULLABLE
-  description: Browser installation channel installed on the client.
-- name: country
-  type: STRING
-  mode: NULLABLE
-  description: Country reported by the client.
-- name: city
-  type: STRING
-  mode: NULLABLE
-  description: City reported by the client.
-- name: locale
-  type: STRING
-  mode: NULLABLE
-  description: Locale reported by the client, which is a combination of language and regional settings.
-- name: first_seen_year
-  type: INTEGER
-  mode: NULLABLE
-  description: Year extracted from the first_seen_date, that corresponds to the date when the first ping was received.
-- name: os
-  type: STRING
-  mode: NULLABLE
-  description: Operating system reported by the client.
-- name: os_version
-  type: STRING
-  mode: NULLABLE
-  description: OS version reported by the client.
-- name: os_version_major
-  type: INTEGER
-  mode: NULLABLE
-  description: Major or first part of the OS version reported by the client.
-- name: os_version_minor
-  type: INTEGER
-  mode: NULLABLE
-  description: Minor or second part of the OS version reported by the client.
-- name: submission_date
-  type: DATE
-  mode: NULLABLE
-  description: Date when ping is received on the server side.
-- name: is_default_browser
-  type: BOOLEAN
-  mode: NULLABLE
-  description: Whether the browser is set as the default browser on the client side.
-- name: distribution_id
-  type: STRING
-  mode: NULLABLE
-  description: A string containing the distribution identifier. This was used to identify installs from Mozilla Online, but now also identifies partnership deal distributions.
-- name: attribution_source
-  type: STRING
-  mode: NULLABLE
-  description: The utm_term this install is attributed to. Reported by the install referrer service, not Adjust.
-- name: attribution_medium
-  type: STRING
-  mode: NULLABLE
-  description: The utm_medium this install is attributed to. Reported by the install referrer service, not Adjust.
-- name: attributed
-  type: BOOLEAN
-  mode: NULLABLE
-  description: True if the attribution source and medium are present.
-- name: adjust_network
-  type: STRING
-  mode: NULLABLE
-  description: The source of a client installation.
-- name: install_source
-  type: STRING
-  mode: NULLABLE
-  description: The id of the browser distribution made available in installation sources.
-- name: daily_users
-  type: INTEGER
-  mode: NULLABLE
-  description: Count of users who report a ping in a day.
-- name: weekly_users
-  type: INTEGER
-  mode: NULLABLE
-  description: Count of users who have reported a ping over the last 7 days.
-- name: monthly_users
-  type: INTEGER
-  mode: NULLABLE
-  description: Count of users who have reported a ping over the last 28 days.
-- name: dau
-  type: INTEGER
-  mode: NULLABLE
-  description: Count of users who reported a ping on the submission_date that qualify as active.
-- name: wau
-  type: INTEGER
-  mode: NULLABLE
-  description: Count of users who have reported a ping over the last 7 days and qualify as active.
-- name: mau
-  type: INTEGER
-  mode: NULLABLE
-  description: Count of users who have reported a ping over the last 28 days and qualify as active.
-- name: uri_count
-  type: INTEGER
-  mode: NULLABLE
-  description: Count of uri.
-- name: active_hours
-  type: FLOAT64
-  mode: NULLABLE
-  description: Count of active hours.
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:33:27.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-20"
+    DATE(submission_timestamp) >= "2024-11-21"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/metadata.yaml	2024-11-21 10:30:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/metadata.yaml	2024-11-21 10:44:34.000000000 +0000
@@ -10,4 +10,4 @@
   - workgroup:mozilla-confidential
 references:
   view.sql:
-  - moz-fx-data-shared-prod.firefox_desktop_derived.active_users_aggregates_v3
+  - moz-fx-data-shared-prod.firefox_desktop_derived.active_users_aggregates_v4
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/schema.yaml	2024-11-21 10:30:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/schema.yaml	2024-11-21 10:38:47.000000000 +0000
@@ -1,8 +1,4 @@
 fields:
-- name: segment
-  type: STRING
-  mode: NULLABLE
-  description: Classification of client_ids based on usage and active state.
 - name: channel
   type: STRING
   mode: NULLABLE
@@ -20,16 +16,11 @@
   mode: NULLABLE
   description: Locale reported by the client, which is a combination of language and
     regional settings.
-- name: first_seen_year
-  type: INTEGER
-  mode: NULLABLE
-  description: Year extracted from the first_seen_date, that corresponds to the date
-    when the first ping was received.
 - name: os
   type: STRING
   mode: NULLABLE
-  description: Operating system reported by the client.
-- name: os_version
+  description: Operating System reported by the client.
+- name: os_version_build
   type: STRING
   mode: NULLABLE
   description: OS version reported by the client.
@@ -63,10 +54,15 @@
   mode: NULLABLE
   description: The utm_medium this install is attributed to. Reported by the install
     referrer service, not Adjust.
-- name: attributed
-  type: BOOLEAN
+- name: segment_dau
+  type: STRING
   mode: NULLABLE
-  description: True if the attribution source and medium are present.
+  description: Classification of client_ids based on usage and active state.
+- name: first_seen_year_new
+  type: INTEGER
+  mode: NULLABLE
+  description: Year extracted from the first_seen_date, that corresponds to the date
+    when the first ping was received.
 - name: daily_users
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/view.sql	2024-11-21 10:30:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/active_users_aggregates/view.sql	2024-11-21 10:38:18.000000000 +0000
@@ -3,7 +3,7 @@
   `moz-fx-data-shared-prod.firefox_desktop.active_users_aggregates`
 AS
 SELECT
-  * EXCEPT (app_version, app_name, uri_count, active_hours),
+  * EXCEPT (app_version, app_name),
   app_name,
   app_version,
   `mozfun.norm.browser_version_info`(app_version).major_version AS app_version_major,
@@ -12,4 +12,4 @@
   `mozfun.norm.browser_version_info`(app_version).is_major_release AS app_version_is_major_release,
   `mozfun.norm.os`(os) AS os_grouped
 FROM
-  `moz-fx-data-shared-prod.firefox_desktop_derived.active_users_aggregates_v3`
+  `moz-fx-data-shared-prod.firefox_desktop_derived.active_users_aggregates_v4`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml	2024-11-21 10:29:41.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml	2024-11-21 10:38:37.000000000 +0000
@@ -1,64 +1,66 @@
 fields:
-- description: Unique ID for the client installation.
-  mode: NULLABLE
-  name: client_id
+- name: client_id
   type: STRING
-- description: Sample ID - A number ranging from 0 - 99 based on client ID; used to pull a small sample of data related to a subset of clients over time
   mode: NULLABLE
-  name: sample_id
-  type: INT64
-- description: Submission Date
+  description: Unique ID for the client installation.
+- name: sample_id
+  type: INTEGER
   mode: NULLABLE
-  name: submission_date
+  description: Sample ID - A number ranging from 0 - 99 based on client ID; used to
+    pull a small sample of data related to a subset of clients over time
+- name: submission_date
   type: DATE
-- description: First Seen Date - The date this client was first seen
   mode: NULLABLE
-  name: first_seen_date
+  description: Submission Date
+- name: first_seen_date
   type: DATE
-- description: Days Since First Seen - The number of days since the client was first seen
   mode: NULLABLE
-  name: days_since_first_seen
-  type: INT64
-- description: Days Since Active
+  description: First Seen Date - The date this client was first seen
+- name: days_since_first_seen
+  type: INTEGER
   mode: NULLABLE
-  name: days_since_active
-  type: INT64
-- description: First Reported Country - The country this client ID was first reported from
+  description: Days Since First Seen - The number of days since the client was first
+    seen
+- name: days_since_active
+  type: INTEGER
   mode: NULLABLE
-  name: first_reported_country
+  description: Days Since Active
+- name: first_reported_country
   type: STRING
-- description: Attribution
   mode: NULLABLE
-  name: attribution
+  description: First Reported Country - The country this client ID was first reported
+    from
+- name: attribution
   type: RECORD
+  mode: NULLABLE
   fields:
-  - mode: NULLABLE
-    name: source
+  - name: source
     type: STRING
+    mode: NULLABLE
     description: Attribution Source
-  - mode: NULLABLE
-    name: medium
+  - name: medium
     type: STRING
+    mode: NULLABLE
     description: Attribution Medium
-  - mode: NULLABLE
-    name: campaign
+  - name: campaign
     type: STRING
+    mode: NULLABLE
     description: Attribution Campaign
-  - mode: NULLABLE
-    name: content
+  - name: content
     type: STRING
+    mode: NULLABLE
     description: Attribution Content
-  - mode: NULLABLE
-    name: experiment
+  - name: experiment
     type: STRING
+    mode: NULLABLE
     description: Attribution Experiment
-  - mode: NULLABLE
-    name: variation
+  - name: variation
     type: STRING
+    mode: NULLABLE
     description: Attribution Variation
-  - mode: NULLABLE
-    name: dltoken
+  - name: dltoken
     type: STRING
+    mode: NULLABLE
     description: Attribution Download Token
   - name: dlsource
     type: STRING
@@ -68,40 +70,43 @@
     type: STRING
     mode: NULLABLE
     description: Attribution UA
-- description: Active
-  mode: NULLABLE
-  name: active
-  type: INT64
-- description: Ad Clicks - The number of ad clicks from this client on the submission date
+  description: Attribution
+- name: active
+  type: INTEGER
   mode: NULLABLE
-  name: ad_clicks
-  type: INT64
-- description: Total Historic Ad Clicks - The number of ad clicks from this client on or before the submission date
+  description: Active
+- name: ad_clicks
+  type: INTEGER
   mode: NULLABLE
-  name: total_historic_ad_clicks
-  type: INT64
-- description: Days Seen Bytes
+  description: Ad Clicks - The number of ad clicks from this client on the submission
+    date
+- name: total_historic_ad_clicks
+  type: INTEGER
   mode: NULLABLE
-  name: days_seen_bytes
+  description: Total Historic Ad Clicks - The number of ad clicks from this client
+    on or before the submission date
+- name: days_seen_bytes
   type: BYTES
-- description: Pattern
   mode: NULLABLE
-  name: pattern
+  description: Days Seen Bytes
+- name: pattern
   type: INTEGER
-- description: Death Time
   mode: NULLABLE
-  name: death_time
+  description: Pattern
+- name: death_time
   type: INTEGER
-- description: Max Days
   mode: NULLABLE
-  name: max_days
+  description: Death Time
+- name: max_days
   type: INTEGER
-- description: Markov States
   mode: NULLABLE
-  name: markov_states
+  description: Max Days
+- name: markov_states
   type: RECORD
-  fields:
-  - description: Desktop States V1
     mode: NULLABLE
-    name: desktop_states_v1
+  fields:
+  - name: desktop_states_v1
     type: STRING
+    mode: NULLABLE
+    description: Desktop States V1
+  description: Markov States
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml	2024-11-21 10:29:41.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml	2024-11-21 10:38:48.000000000 +0000
@@ -1,48 +1,47 @@
 fields:
-- description: Submission Timestamp
-  mode: NULLABLE
-  name: submission_timestamp
+- name: submission_timestamp
   type: TIMESTAMP
-- description: Normalized Country Code, Examples - US, AR, BR, etc.
   mode: NULLABLE
-  name: normalized_country_code
+  description: Submission Timestamp
+- name: normalized_country_code
   type: STRING
-- description: Normalized Channel, Examples - release, nightly, aurora, esr, beta
   mode: NULLABLE
-  name: normalized_channel
+  description: Normalized Country Code, Examples - US, AR, BR, etc.
+- name: normalized_channel
   type: STRING
-- description: Document ID
   mode: NULLABLE
-  name: document_id
+  description: Normalized Channel, Examples - release, nightly, aurora, esr, beta
+- name: document_id
   type: STRING
-- description: Pocket Enabled
   mode: NULLABLE
-  name: pocket_enabled
+  description: Document ID
+- name: pocket_enabled
   type: BOOLEAN
-- description: Pocket Sponsored Stories Enabled
   mode: NULLABLE
-  name: pocket_sponsored_stories_enabled
+  description: Pocket Enabled
+- name: pocket_sponsored_stories_enabled
   type: BOOLEAN
-- description: Newtab Locale
   mode: NULLABLE
-  name: newtab_locale
+  description: Pocket Sponsored Stories Enabled
+- name: newtab_locale
   type: STRING
-- description: App Build
   mode: NULLABLE
-  name: app_build
+  description: Newtab Locale
+- name: app_build
   type: STRING
-- description: App Display Version
   mode: NULLABLE
-  name: app_display_version
+  description: App Build
+- name: app_display_version
   type: STRING
-- description: Client ID
   mode: NULLABLE
-  name: client_id
+  description: App Display Version
+- name: client_id
   type: STRING
+  mode: NULLABLE
+  description: Client ID
 - name: events
   type: RECORD
   mode: REPEATED
-  description: Events
   fields:
   - name: category
     type: STRING
@@ -51,7 +50,6 @@
   - name: extra
     type: RECORD
     mode: REPEATED
-    description: Extras
     fields:
     - name: key
       type: STRING
@@ -61,6 +59,7 @@
       type: STRING
       mode: NULLABLE
       description: Value
+    description: Extras
   - name: name
     type: STRING
     mode: NULLABLE
@@ -69,3 +68,4 @@
     type: INTEGER
     mode: NULLABLE
     description: Event Timestamp
+  description: Events
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:33:27.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-20"
+    DATE(submission_timestamp) >= "2024-11-21"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:30:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:33:27.000000000 +0000
@@ -77,7 +77,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-20"
+    DATE(submission_timestamp) >= "2024-11-21"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/event_monitoring_live_v1/materialized_view.sql	2024-11-21 10:33:27.000000000 +0000
@@ -67,7 +67,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2024-11-20"
+    DATE(submission_timestamp) >= "2024-11-21"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/checks.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/checks.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/checks.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/checks.sql	1970-01-01 00:00:00.000000000 +0000
@@ -1,180 +0,0 @@
-
-#warn
-WITH dau_sum AS (
-  SELECT
-    SUM(dau),
-  FROM
-    `{{ project_id }}.{{ dataset_id }}.{{ table_name }}`
-  WHERE
-    submission_date = @submission_date
-),
-live_table_dau_count_base AS (
-  SELECT
-    client_id,
-    SUM(
-      payload.processes.parent.scalars.browser_engagement_total_uri_count_normal_and_private_mode
-    ) AS scalar_parent_browser_engagement_total_uri_count_normal_and_private_mode_sum,
-    SUM(
-      payload.processes.parent.scalars.browser_engagement_total_uri_count
-    ) AS scalar_parent_browser_engagement_total_uri_count_sum,
-    SUM(
-      COALESCE(
-        payload.processes.parent.scalars.browser_engagement_active_ticks,
-        payload.simple_measurements.active_ticks
-      )
-    ) AS active_ticks
-  FROM
-    `moz-fx-data-shared-prod.telemetry_live.main_v5`
-  WHERE
-    DATE(submission_timestamp) = @submission_date
-    AND normalized_app_name = 'Firefox'
-    AND document_id IS NOT NULL
-  GROUP BY
-    client_id
-),
-overactive AS (
-  SELECT
-    client_id
-  FROM
-    live_table_dau_count_base
-  GROUP BY
-    client_id
-  HAVING
-    COUNT(*) > 150000
-),
-client_summary AS (
-  SELECT
-    client_id,
-    SUM(
-      COALESCE(
-        scalar_parent_browser_engagement_total_uri_count_normal_and_private_mode_sum,
-        scalar_parent_browser_engagement_total_uri_count_sum
-      )
-    ) AS total_uri_count,
-    SUM(active_ticks / (3600 / 5)) AS active_hours_sum,
-  FROM
-    live_table_dau_count_base
-  LEFT JOIN
-    overactive
-    USING (client_id)
-  WHERE
-    overactive.client_id IS NULL
-  GROUP BY
-    client_id
-),
-last_seen AS (
-  SELECT
-    client_id,
-    days_since_seen,
-  FROM
-    `moz-fx-data-shared-prod.telemetry.clients_last_seen`
-  WHERE
-    submission_date = @submission_date
-),
-live_table_dau_count AS (
-  SELECT
-    COUNTIF(active_hours_sum > 0 AND total_uri_count > 0 AND days_since_seen = 0)
-  FROM
-    client_summary
-  LEFT JOIN
-    last_seen
-    USING (client_id)
-)
-SELECT
-  IF(
-    ABS((SELECT * FROM dau_sum) - (SELECT * FROM live_table_dau_count)) > 10,
-    ERROR(
-      CONCAT(
-        "DAU mismatch between the live (`telemetry_live.main_v5`) and active_users_aggregates (`{{ dataset_id }}.{{ table_name }}`) tables is greater than 10.",
-        " Live table count: ",
-        (SELECT * FROM live_table_dau_count),
-        " | active_users_aggregates (DAU): ",
-        (SELECT * FROM dau_sum),
-        " | Delta detected: ",
-        ABS((SELECT * FROM dau_sum) - (SELECT * FROM live_table_dau_count))
-      )
-    ),
-    NULL
-  );
-
-#warn
-WITH daily_users_sum AS (
-  SELECT
-    SUM(daily_users),
-  FROM
-    `{{ project_id }}.{{ dataset_id }}.{{ table_name }}`
-  WHERE
-    submission_date = @submission_date
-),
-distinct_client_count_base AS (
-  SELECT
-    client_id
-  FROM
-    `moz-fx-data-shared-prod.telemetry_live.main_v5`
-  WHERE
-    DATE(submission_timestamp) = @submission_date
-    AND normalized_app_name = 'Firefox'
-    AND document_id IS NOT NULL
-),
-overactive AS (
-  SELECT
-    client_id
-  FROM
-    distinct_client_count_base
-  GROUP BY
-    client_id
-  HAVING
-    COUNT(*) > 150000
-),
-distinct_client_count AS (
-  SELECT
-    COUNT(DISTINCT client_id),
-  FROM
-    distinct_client_count_base
-  FULL OUTER JOIN
-    overactive
-    USING (client_id)
-  WHERE
-    overactive.client_id IS NULL
-)
-SELECT
-  IF(
-    ABS((SELECT * FROM daily_users_sum) - (SELECT * FROM distinct_client_count)) > 10,
-    ERROR(
-      CONCAT(
-        "Daily_users mismatch between the live (`telemetry_live.main_v5`) and active_users_aggregates (`{{ dataset_id }}.{{ table_name }}`) tables is greater than 10.",
-        " Live table count: ",
-        (SELECT * FROM distinct_client_count),
-        " | active_users_aggregates (daily_users): ",
-        (SELECT * FROM daily_users_sum),
-        " | Delta detected: ",
-        ABS((SELECT * FROM daily_users_sum) - (SELECT * FROM distinct_client_count))
-      )
-    ),
-    NULL
-  );
-
-WITH dau_current AS (
-  SELECT
-    SUM(dau) AS dau
-  FROM
-    `{{ project_id }}.{{ dataset_id }}.{{ table_name }}`
-  WHERE
-    submission_date = @submission_date
-),
-dau_previous AS (
-  SELECT
-    SUM(dau) AS dau
-  FROM
-    `{{ project_id }}.{{ dataset_id }}.{{ table_name }}`
-  WHERE
-    submission_date = DATE_SUB(@submission_date, INTERVAL 1 DAY)
-)
-SELECT
-  IF(
-    ABS((SELECT SUM(dau) FROM dau_current) / (SELECT SUM(dau) FROM dau_previous)) > 1.5,
-    ERROR(
-      "Current date's DAU is 50% higher than in previous date. See source table (`{{ project_id }}.{{ dataset_id }}.{{ table_name }}`)!"
-    ),
-    NULL
-  );
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/metadata.yaml	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,59 +0,0 @@
-friendly_name: Firefox Desktop Active Users Aggregates
-description: |-
-  This table contains dau, wau, mau, daily users,
-  weekly users and monthly users for Firefox Desktop,
-  aggregated by submission_date, attribution, channel,
-  country, city, device model, distribution_id, os details
-  and activity segment.
-
-  - dau is counting the users who reported a ping on the date and
-  are qualified as active users.
-  - daily_users counts all the users who reported a ping on the date.
-  Only dau is exposed in the view telemetry.active_users_aggregates.
-
-  The table is labeled as "change_controlled", which implies
-  that changes require the approval of at least one owner.
-
-  The label "shredder mitigation" indicates that this table is set up for
-  managed backfill with shredder mitigation, as described in
-  https://mozilla.github.io/bigquery-etl/cookbooks/creating_a_derived_dataset/#initiating-the-backfill.
-
-  Proposal:
-  https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
-owners:
-- [email protected]
-- mozilla/kpi_table_reviewers
-labels:
-  incremental: true
-  change_controlled: true
-  shredder_mitigation: true
-  dag: bqetl_analytics_aggregations
-  owner1: lvargas
-scheduling:
-  dag_name: bqetl_analytics_aggregations
-  task_name: firefox_desktop_active_users_aggregates_v3
-  date_partition_offset: -1
-bigquery:
-  time_partitioning:
-    type: day
-    field: submission_date
-    require_partition_filter: true
-    expiration_days: null
-  range_partitioning: null
-  clustering:
-    fields:
-    - country
-    - app_name
-    - attribution_medium
-    - channel
-workgroup_access:
-- role: roles/bigquery.dataViewer
-  members:
-  - workgroup:mozilla-confidential
-references:
-  checks.sql:
-  - ..
-  - moz-fx-data-shared-prod.telemetry.clients_last_seen
-  - moz-fx-data-shared-prod.telemetry_live.main_v5
-  query.sql:
-  - moz-fx-data-shared-prod.telemetry.desktop_active_users
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/query.sql	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/query.sql	1970-01-01 00:00:00.000000000 +0000
@@ -1,89 +0,0 @@
---- Query generated via sql_generators.active_users.
-WITH todays_metrics AS (
-  SELECT
-    client_id,
-    activity_segment AS segment,
-    app_name,
-    app_version AS app_version,
-    normalized_channel AS channel,
-    IFNULL(country, '??') country,
-    IFNULL(city, '??') city,
-    COALESCE(REGEXP_EXTRACT(locale, r'^(.+?)-'), locale, NULL) AS locale,
-    EXTRACT(YEAR FROM first_seen_date) AS first_seen_year,
-    os,
-    COALESCE(
-      `mozfun.norm.windows_version_info`(os, normalized_os_version, windows_build_number),
-      normalized_os_version
-    ) AS os_version,
-    COALESCE(
-      CAST(NULLIF(SPLIT(normalized_os_version, ".")[SAFE_OFFSET(0)], "") AS INTEGER),
-      0
-    ) AS os_version_major,
-    COALESCE(
-      CAST(NULLIF(SPLIT(normalized_os_version, ".")[SAFE_OFFSET(1)], "") AS INTEGER),
-      0
-    ) AS os_version_minor,
-    submission_date,
-    COALESCE(
-      scalar_parent_browser_engagement_total_uri_count_normal_and_private_mode_sum,
-      scalar_parent_browser_engagement_total_uri_count_sum
-    ) AS uri_count,
-    is_default_browser,
-    distribution_id,
-    attribution_source,
-    attribution_medium,
-    attribution_medium IS NOT NULL
-    OR attribution_source IS NOT NULL AS attributed,
-    is_daily_user,
-    is_weekly_user,
-    is_monthly_user,
-    is_dau,
-    is_wau,
-    is_mau,
-    active_hours_sum
-  FROM
-    `moz-fx-data-shared-prod.telemetry.desktop_active_users`
-  WHERE
-    submission_date = @submission_date
-)
-SELECT
-  todays_metrics.* EXCEPT (
-    client_id,
-    is_daily_user,
-    is_weekly_user,
-    is_monthly_user,
-    is_dau,
-    is_wau,
-    is_mau,
-    uri_count,
-    active_hours_sum
-  ),
-  COUNTIF(is_daily_user) AS daily_users,
-  COUNTIF(is_weekly_user) AS weekly_users,
-  COUNTIF(is_monthly_user) AS monthly_users,
-  COUNTIF(is_dau) AS dau,
-  COUNTIF(is_wau) AS wau,
-  COUNTIF(is_mau) AS mau,
-  SUM(uri_count) AS uri_count,
-  SUM(active_hours_sum) AS active_hours,
-FROM
-  todays_metrics
-GROUP BY
-  segment,
-  app_name,
-  app_version,
-  channel,
-  country,
-  city,
-  locale,
-  first_seen_year,
-  os,
-  os_version,
-  os_version_major,
-  os_version_minor,
-  submission_date,
-  is_default_browser,
-  distribution_id,
-  attribution_source,
-  attribution_medium,
-  attributed
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/schema.yaml	2024-11-21 10:30:21.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v3/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,105 +0,0 @@
-fields:
-- name: segment
-  type: STRING
-  mode: NULLABLE
-  description: Classification of client_ids based on usage and active state.
-- name: app_name
-  type: STRING
-  mode: NULLABLE
-  description: Browser name.
-- name: app_version
-  type: STRING
-  mode: NULLABLE
-  description: Browser version installed on the client.
-- name: channel
-  type: STRING
-  mode: NULLABLE
-  description: Browser installation channel installed on the client.
-- name: country
-  type: STRING
-  mode: NULLABLE
-  description: Country reported by the client.
-- name: city
-  type: STRING
-  mode: NULLABLE
-  description: City reported by the client.
-- name: locale
-  type: STRING
-  mode: NULLABLE
-  description: Locale reported by the client, which is a combination of language and regional settings.
-- name: first_seen_year
-  type: INTEGER
-  mode: NULLABLE
-  description: Year extracted from the first_seen_date, that corresponds to the date when the first ping was received.
-- name: os
-  type: STRING
-  mode: NULLABLE
-  description: Operating system reported by the client.
-- name: os_version
-  type: STRING
-  mode: NULLABLE
-  description: OS version reported by the client.
-- name: os_version_major
-  type: INTEGER
-  mode: NULLABLE
-  description: Major or first part of the OS version reported by the client.
-- name: os_version_minor
-  type: INTEGER
-  mode: NULLABLE
-  description: Minor or second part of the OS version reported by the client.
-- name: submission_date
-  type: DATE
-  mode: NULLABLE
-  description: Date when ping is received on the server side.
-- name: is_default_browser
-  type: BOOLEAN
-  mode: NULLABL

⚠️ Only part of the diff is displayed.

Link to full diff

Comment on lines +229 to +256
elif browser.name == "firefox_desktop":
write_sql(
output_dir=output_dir,
full_table_id=f"{target_project}.{browser.name}.{BASE_NAME}",
basename="view.sql",
sql=reformat(
view_template.render(
project_id=target_project,
app_name=browser.name,
table_name=TABLE_NAME,
)
),
skip_existing=False,
)
else:
write_sql(
output_dir=output_dir,
full_table_id=f"{target_project}.{browser.name}.{BASE_NAME}",
basename="view.sql",
sql=reformat(
view_template.render(
project_id=target_project,
app_name=browser.name,
table_name=TABLE_NAME,
)
),
skip_existing=False,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two conditions seem to be identical... is that intentional? If so, could you remove the elif and just use the else?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants