DENG-6880 - Daily aggregations for NewTab #6863

gkatre · 2025-01-24T06:54:39Z

Description

The daily aggregations to address the retention requirements of the Desktop New Tab Dashboard

The following daily aggregation tables are created/updates:

newtab_clients_daily_aggregates_v1: The missing aggregations are added (the aggregation granularity remains the same as earlier)
newtab_daily_interactions_aggregates_v1: New table to aggregate interactions based on the newtab_interactions_v1 table
newtab_conditional_daily_aggregates_v1: Has the computations for conditional computations of client and visit aggregations

Related Tickets & Documents

DENG-6880

Reviewer, please follow this checklist

┆Issue is synchronized with this Jira Task

alekhyamoz · 2025-01-24T15:50:59Z

@gkatre can we combine conditional_daily_aggregate with newtab_daily_interactions table?
I worry that the conditional table might not surface easily.

...moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/schema.yaml

…al_daily_aggregates_v1/schema.yaml Co-authored-by: Ben Wu <[email protected]>

gkatre · 2025-01-24T19:21:20Z

@gkatre can we combine conditional_daily_aggregate with newtab_daily_interactions table? I worry that the conditional table might not surface easily.

@alekhyamoz the reason i did not combine newtab_daily_interactions_aggregates with newtab_conditional_daily_aggregates is because they have different granularities.

newtab_daily_interactions_aggregates is a table that can be rolled up. But, the newtab_conditional_daily_aggregates table computes the distinct client and visits on those granularities.

The below listed fields used in the granularities are expected to be mostly mutually exclusive so as to not double count the distinct clients and visits.

Example: A client who has pocket_enabled (TRUE) may not also show up as FALSE, therefore would not end up with double counting clients

country_code
pocket_enabled
pocket_sponsored_stories_enabled
topsites_enabled

cc: @bani : Would you also be able to validate these conditional computations in the newtab_conditional_daily_aggregates_v1 table? Thanks!

alekhyamoz · 2025-01-24T20:27:57Z

@gkatre can we combine conditional_daily_aggregate with newtab_daily_interactions table? I worry that the conditional table might not surface easily.

@alekhyamoz the reason i did not combine newtab_daily_interactions_aggregates with newtab_conditional_daily_aggregates is because they have different granularities.

newtab_daily_interactions_aggregates is a table that can be rolled up. But, the newtab_conditional_daily_aggregates table computes the distinct client and visits on those granularities.

The below listed fields used in the granularities are expected to be mostly mutually exclusive so as to not double count the distinct clients and visits.

Example: A client who has pocket_enabled (TRUE) may not also show up as FALSE, therefore would not end up with double counting clients
* country_code

* pocket_enabled

* pocket_sponsored_stories_enabled

* topsites_enabled
cc: @bani : Would you also be able to validate these conditional computations in the newtab_conditional_daily_aggregates_v1 table? Thanks!

@gkatre in that case can we add the columns from conditional table to the newtab_daily_interactions view?
That way the columns will surface to anyone who is try to look for them?

gkatre · 2025-01-24T20:31:36Z

@gkatre in that case can we add the columns from conditional table to the newtab_daily_interactions view? That way the columns will surface to anyone who is try to look for them?

We can go the other way. We can rollup and add the columns from newtab_daily_interactions_aggregates_v1 to the newtab_conditional_daily_aggregates.

newtab_daily_interactions_aggregates_v1 is more granular and can be rolled up

alekhyamoz

lgtm. Thank you for working on this G. I will also let @bani review this.

bani · 2025-01-24T20:43:06Z

sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/query.sql

+  ) AS pocket_signed_in_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND pocket_is_signed_in, newtab_visit_id, NULL)
+  ) AS pocket_signed_in_visit_count,


Do we need one for counting just homepage category enabled/disabled independent of source?
Maybe I missed it but I couldn't find.

For this widget: https://mozilla.cloud.looker.com/explore/firefox_desktop/newtab_interactions?qid=qOIDEWK6Oo10MxPDZcg7rn&origin_space=464&toggle=vis

Also for this widget with newtab_category: https://mozilla.cloud.looker.com/explore/firefox_desktop/newtab_interactions?qid=L3HcgEzCJWLQBVDaYeVVbg&origin_space=464&toggle=vis

I think due to the count distinct we can't just SUM home vs newtab in the end.

I have it here:

homepage category enabled

homepage category disabled

Would this work? Or perhaps I could add newtab_homepage_category and/or newtab_newtab_category to the granularity (GROUP BY) if you feel its ok (that is: we will not end up double counting clients and visits)

My point was that all aggregations of that field have either newtab_open_source = 'about:home' or newtab_open_source = 'about:newtab' but the query on the explore doesn't include anything about newtab_open_source and I'm not sure how that affects the numbers.

Ok. Just to be safe, instead of adding newtab_homepage_category and newtab_newtab_category as a dimension in the GROUP BY, I will add a set of 2 distinct count metrics:

one set for newtab_homepage_category enabled

one set for newtab_newtab_category enabled

@bani added the aggregates for newtab_homepage_category enabled and newtab_newtab_category enabled

gkatre · 2025-01-24T20:47:00Z

@alekhyamoz if there are any views to be updated, we can always do those anytime later. We dont need to hold up on the deployment/backfills for the retention project.

As long as we are ok with the way these 3 tables (newtab_clients_daily_aggregates_v1, newtab_daily_interactions_aggregates_v1, newtab_conditional_daily_aggregates_v1) are created, we could perhaps proceed?

bani · 2025-01-24T20:50:42Z

sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/query.sql

+  topsites_rows,
+  is_new_profile,
+  activity_segment,
+  COUNT(DISTINCT client_id) AS client_count,


This data will be an issue since it'll be quite common to have a client doing interactions across different dimensions in a day. But it doesn't seem feasible to create tables for each so we may need to just double check is stakeholders if there are any specific ones that really require no double counting.

I agree. If you feel by having it is confusing, I can remove these count(distinct --) fields from that table:

client_count

legacy_telemetry_client_count

newtab_interactions_visits

I think we need to keep it as that's the closes we have to what is currently in the dashboard. It'll just be overcounting in certain cases once we SUM.

bani · 2025-01-24T20:51:24Z

sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/query.sql

+  submission_date,
+  search_engine,
+  search_access_point,
+  pocket_story_position,


This is one dimension I'm particularly worried about inflating all distinct client counts.

I feel that table is too granular. Therefore the distinct counts may not go well in that table because its not possible to rollup those values. I can remove the 3 count distinct fields, if you feel thats the right thing to do.

We have the conditional aggregate table for such distinct counts where needed

Ok, good to know that we covered all use cases.

bani · 2025-01-24T21:02:14Z

Do we have something for mozdata.telemetry.newtab_visits?

gkatre · 2025-01-24T21:10:24Z

Do we have something for mozdata.telemetry.newtab_visits?

The newtab_clients_daily_v1 seems to be a rollup of the newtab_visits_v1. I haven't tried to re-engineer any of those tables

bani · 2025-01-24T21:27:27Z

It's also worth highlighting to stakeholders that they have a lot of "COUNT DISTINCT" across time, and this is not something we'll be able to replicate for different timeframes in the past.

bani · 2025-01-24T21:32:18Z

It seems the dashboard is using some columns from newtab_visits_v1 that don't exist in newtab_clients_daily_v1, such as had_non_impression_engagement in https://mozilla.cloud.looker.com/explore/firefox_desktop/newtab_visits?qid=PF3wIICSN13op7HN9ZeTKT&origin_space=464&toggle=vis

But it's unclear if there is an use case for historical data there due to the same issue of unclear timeframes for aggregation.

EDIT: this one actually look at the % over time, is this data point covered? https://mozilla.cloud.looker.com/explore/firefox_desktop/newtab_visits?qid=dh3tzf1nHzKbGVOATeQKrM&origin_space=464&toggle=vis

gkatre · 2025-01-24T21:44:56Z

It seems the dashboard is using some columns from newtab_visits_v1 that don't exist in newtab_clients_daily_v1, such as had_non_impression_engagement in https://mozilla.cloud.looker.com/explore/firefox_desktop/newtab_visits?qid=PF3wIICSN13op7HN9ZeTKT&origin_space=464&toggle=vis

But it's unclear if there is an use case for historical data there due to the same issue of unclear timeframes for aggregation.

EDIT: this one actually look at the % over time, is this data point covered? https://mozilla.cloud.looker.com/explore/firefox_desktop/newtab_visits?qid=dh3tzf1nHzKbGVOATeQKrM&origin_space=464&toggle=vis

@bani I agree completely! I also feel doing that conditional aggregation table is so much effort that I do not know if there is value there. Also, if we are going to build such computations, it should be easily usable and also something thats easy to maintain. Not sure if we are doing such computations elsewhere.

I am also not sure what the stakeholders expectations are on historic data and if requirements are going to be relatively static to justify the need to build materialized aggregation tables.

gkatre · 2025-01-24T22:10:59Z

@bani You are correct that had_non_impression_engagement does not exist in newtab_interactions_v1 as well.
The conditional aggregation table has been constructed based on the available fields from the newtab_interactions_v1 table. While most fields are available there, there may be a few that are missing.
I feel at some point we may need to plan to re-engineer all those tables and simplify the aggregations. I do not think we would be able to do this in the limited time we have for the retention project

gkatre · 2025-01-24T22:29:15Z

@alekhyamoz / @bani Is there anything you feel is a blocker to proceed with deploying and start backfilling these tables?

dataops-ci-bot · 2025-01-24T23:28:12Z

Integration report for "Merge branch 'main' into newtab/DENG-6880-newtab_update_aggregations"

`sql.diff`

Click to expand!

diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_newtab.py /tmp/workspace/generated-sql/dags/bqetl_newtab.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_newtab.py	2025-01-24 23:24:43.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_newtab.py	2025-01-24 23:27:28.000000000 +0000
@@ -106,6 +106,36 @@
         depends_on_past=False,
     )
 
+    telemetry_derived__newtab_conditional_daily_aggregates__v1 = bigquery_etl_query(
+        task_id="telemetry_derived__newtab_conditional_daily_aggregates__v1",
+        destination_table="newtab_conditional_daily_aggregates_v1",
+        dataset_id="telemetry_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="[email protected]",
+        email=[
+            "[email protected]",
+            "[email protected]",
+            "[email protected]",
+        ],
+        date_partition_parameter="submission_date",
+        depends_on_past=False,
+    )
+
+    telemetry_derived__newtab_daily_interactions_aggregates__v1 = bigquery_etl_query(
+        task_id="telemetry_derived__newtab_daily_interactions_aggregates__v1",
+        destination_table="newtab_daily_interactions_aggregates_v1",
+        dataset_id="telemetry_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="[email protected]",
+        email=[
+            "[email protected]",
+            "[email protected]",
+            "[email protected]",
+        ],
+        date_partition_parameter="submission_date",
+        depends_on_past=False,
+    )
+
     telemetry_derived__newtab_interactions__v1 = bigquery_etl_query(
         task_id="telemetry_derived__newtab_interactions__v1",
         destination_table="newtab_interactions_v1",
@@ -144,6 +174,14 @@
         telemetry_derived__newtab_clients_daily__v1
     )
 
+    telemetry_derived__newtab_conditional_daily_aggregates__v1.set_upstream(
+        telemetry_derived__newtab_interactions__v1
+    )
+
+    telemetry_derived__newtab_daily_interactions_aggregates__v1.set_upstream(
+        telemetry_derived__newtab_interactions__v1
+    )
+
     telemetry_derived__newtab_interactions__v1.set_upstream(
         wait_for_checks__fail_telemetry_derived__unified_metrics__v1
     )
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry: newtab_conditional_daily_aggregates
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry: newtab_daily_interactions_aggregates
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived: newtab_conditional_daily_aggregates_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived: newtab_daily_interactions_aggregates_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_conditional_daily_aggregates/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_conditional_daily_aggregates/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_conditional_daily_aggregates/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_conditional_daily_aggregates/metadata.yaml	2025-01-24 23:22:52.000000000 +0000
@@ -0,0 +1,16 @@
+friendly_name: Newtab Conditional Daily Aggregates
+description: |-
+  Conditional computations for daily aggregation for newtab.
+owners:
+- [email protected]
+labels:
+  owner1: gkatre
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:dataops-managed/taar
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.telemetry_derived.newtab_conditional_daily_aggregates_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_conditional_daily_aggregates/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_conditional_daily_aggregates/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_conditional_daily_aggregates/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_conditional_daily_aggregates/view.sql	2025-01-24 23:19:25.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.telemetry.newtab_conditional_daily_aggregates`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.telemetry_derived.newtab_conditional_daily_aggregates_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_daily_interactions_aggregates/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_daily_interactions_aggregates/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_daily_interactions_aggregates/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_daily_interactions_aggregates/metadata.yaml	2025-01-24 23:22:52.000000000 +0000
@@ -0,0 +1,16 @@
+friendly_name: Newtab Interactions Daily Aggregates
+description: |-
+  A daily aggregation of newtab interactions, partitioned by day.
+owners:
+- [email protected]
+labels:
+  owner1: gkatre
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:dataops-managed/taar
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.telemetry_derived.newtab_daily_interactions_aggregates_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_daily_interactions_aggregates/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_daily_interactions_aggregates/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_daily_interactions_aggregates/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/newtab_daily_interactions_aggregates/view.sql	2025-01-24 23:19:25.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.telemetry.newtab_daily_interactions_aggregates`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.telemetry_derived.newtab_daily_interactions_aggregates_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_clients_daily_aggregates_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_clients_daily_aggregates_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_clients_daily_aggregates_v1/query.sql	2025-01-24 23:19:03.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_clients_daily_aggregates_v1/query.sql	2025-01-24 23:19:25.000000000 +0000
@@ -49,6 +49,20 @@
   SUM(pocket_saves) AS pocket_saves,
   SUM(sponsored_pocket_saves) AS sponsored_pocket_saves,
   SUM(organic_pocket_saves) AS organic_pocket_saves,
+  SUM(pocket_thumb_voting_events) AS pocket_thumb_voting_events,
+  SUM(topic_selection_opened) AS topic_selection_opened,
+  SUM(topic_selection_updates) AS topic_selection_updates,
+  SUM(weather_widget_clicks) AS weather_widget_clicks,
+  SUM(weather_widget_change_display_to_detailed) AS weather_widget_change_display_to_detailed,
+  SUM(weather_widget_change_display_to_simple) AS weather_widget_change_display_to_simple,
+  SUM(wallpaper_clicks) AS wallpaper_clicks,
+  SUM(
+    IF(topic_selection_selected_topics_first_time, 1, 0)
+  ) AS topic_selection_selected_topics_first_time_count,
+  SUM(sponsored_pocket_dismissals) AS sponsored_pocket_dismissals,
+  SUM(organic_pocket_dismissals) AS organic_pocket_dismissals,
+  SUM(topic_selection_dismissals) AS topic_selection_dismissals,
+  SUM(visits_with_default_ui) AS visits_with_default_ui,
 FROM
   `moz-fx-data-shared-prod.telemetry_derived.newtab_clients_daily_v1`
 WHERE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_clients_daily_aggregates_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_clients_daily_aggregates_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_clients_daily_aggregates_v1/schema.yaml	2025-01-24 23:19:03.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_clients_daily_aggregates_v1/schema.yaml	2025-01-24 23:19:25.000000000 +0000
@@ -149,3 +149,39 @@
 - name: organic_pocket_saves
   type: INTEGER
   mode: NULLABLE
+- name: pocket_thumb_voting_events
+  type: INTEGER
+  mode: NULLABLE
+- name: topic_selection_opened
+  type: INTEGER
+  mode: NULLABLE
+- name: topic_selection_updates
+  type: INTEGER
+  mode: NULLABLE
+- name: weather_widget_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: weather_widget_change_display_to_detailed
+  type: INTEGER
+  mode: NULLABLE
+- name: weather_widget_change_display_to_simple
+  type: INTEGER
+  mode: NULLABLE
+- name: wallpaper_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: topic_selection_selected_topics_first_time_count
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_pocket_dismissals
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_pocket_dismissals
+  type: INTEGER
+  mode: NULLABLE
+- name: topic_selection_dismissals
+  type: INTEGER
+  mode: NULLABLE
+- name: visits_with_default_ui
+  type: INTEGER
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/metadata.yaml	2025-01-24 23:22:36.000000000 +0000
@@ -0,0 +1,28 @@
+friendly_name: Newtab Conditional Daily Aggregates
+description: |-
+  Conditional computations for daily aggregation for newtab.
+owners:
+- [email protected]
+labels:
+  application: firefox
+  incremental: true
+  schedule: daily
+  dag: bqetl_newtab
+  owner1: gkatre
+scheduling:
+  dag_name: bqetl_newtab
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: true
+    expiration_days: null
+  range_partitioning: null
+  clustering:
+    fields:
+    - country_code
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references: {}
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/query.sql	2025-01-24 23:19:25.000000000 +0000
@@ -0,0 +1,183 @@
+SELECT
+  submission_date,
+  country_code,
+  pocket_enabled,
+  pocket_sponsored_stories_enabled,
+  topsites_enabled,
+  COUNT(DISTINCT client_id) AS total_client_count,
+  COUNT(DISTINCT newtab_visit_id) AS total_visit_count,
+  -- conditional
+  COUNT(DISTINCT IF(channel = 'release', client_id, NULL)) AS client_count,
+  COUNT(DISTINCT IF(channel = 'release', newtab_visit_id, NULL)) AS visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND newtab_homepage_category = 'enabled', client_id, NULL)
+  ) AS homepage_category_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND newtab_homepage_category = 'enabled', newtab_visit_id, NULL)
+  ) AS homepage_category_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND newtab_newtab_category = 'enabled', client_id, NULL)
+  ) AS newtab_category_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND newtab_newtab_category = 'enabled', newtab_visit_id, NULL)
+  ) AS newtab_category_visit_count,
+  COUNT(
+    DISTINCT IF(
+      channel = 'release'
+      AND newtab_open_source = 'about:home'
+      AND newtab_homepage_category = 'enabled',
+      client_id,
+      NULL
+    )
+  ) AS home_default_client_count,
+  COUNT(
+    DISTINCT IF(
+      channel = 'release'
+      AND newtab_open_source = 'about:home'
+      AND newtab_homepage_category = 'enabled',
+      newtab_visit_id,
+      NULL
+    )
+  ) AS home_default_visit_count,
+  COUNT(
+    DISTINCT IF(
+      channel = 'release'
+      AND newtab_open_source = 'about:newtab'
+      AND newtab_newtab_category = 'enabled',
+      client_id,
+      NULL
+    )
+  ) AS newtab_default_client_count,
+  COUNT(
+    DISTINCT IF(
+      channel = 'release'
+      AND newtab_open_source = 'about:newtab'
+      AND newtab_newtab_category = 'enabled',
+      newtab_visit_id,
+      NULL
+    )
+  ) AS newtab_default_visit_count,
+  COUNT(
+    DISTINCT IF(
+      channel = 'release'
+      AND newtab_open_source = 'about:home'
+      AND newtab_homepage_category != 'enabled',
+      client_id,
+      NULL
+    )
+  ) AS home_nondefault_client_count,
+  COUNT(
+    DISTINCT IF(
+      channel = 'release'
+      AND newtab_open_source = 'about:home'
+      AND newtab_homepage_category != 'enabled',
+      newtab_visit_id,
+      NULL
+    )
+  ) AS home_nondefault_visit_count,
+  COUNT(
+    DISTINCT IF(
+      channel = 'release'
+      AND newtab_open_source = 'about:newtab'
+      AND newtab_newtab_category != 'enabled',
+      client_id,
+      NULL
+    )
+  ) AS newtab_nondefault_client_count,
+  COUNT(
+    DISTINCT IF(
+      channel = 'release'
+      AND newtab_open_source = 'about:newtab'
+      AND newtab_newtab_category != 'enabled',
+      newtab_visit_id,
+      NULL
+    )
+  ) AS newtab_nondefault_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND newtab_open_source = 'about:welcome', client_id, NULL)
+  ) AS welcome_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND newtab_open_source = 'about:welcome', newtab_visit_id, NULL)
+  ) AS welcome_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND searches > 0, client_id, NULL)
+  ) AS searched_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND searches > 0, newtab_visit_id, NULL)
+  ) AS searched_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND sponsored_topsite_impressions > 0, client_id, NULL)
+  ) AS sponsored_topsite_impressed_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND sponsored_topsite_impressions > 0, newtab_visit_id, NULL)
+  ) AS sponsored_topsite_impressed_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND sponsored_topsite_clicks > 0, client_id, NULL)
+  ) AS sponsored_topsite_clicked_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND sponsored_topsite_clicks > 0, newtab_visit_id, NULL)
+  ) AS sponsored_topsite_clicked_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND topsite_impressions > 0, client_id, NULL)
+  ) AS topsite_impressed_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND topsite_impressions > 0, newtab_visit_id, NULL)
+  ) AS topsite_impressed_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND topsite_clicks > 0, client_id, NULL)
+  ) AS topsite_clicked_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND topsite_clicks > 0, newtab_visit_id, NULL)
+  ) AS topsite_clicked_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND sponsored_pocket_impressions > 0, client_id, NULL)
+  ) AS sponsored_pocket_impressed_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND sponsored_pocket_impressions > 0, newtab_visit_id, NULL)
+  ) AS sponsored_pocket_impressed_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND sponsored_pocket_clicks > 0, client_id, NULL)
+  ) AS sponsored_pocket_clicked_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND sponsored_pocket_clicks > 0, newtab_visit_id, NULL)
+  ) AS sponsored_pocket_clicked_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND organic_pocket_impressions > 0, client_id, NULL)
+  ) AS organic_pocket_impressed_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND organic_pocket_impressions > 0, newtab_visit_id, NULL)
+  ) AS organic_pocket_impressed_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND organic_pocket_clicks > 0, client_id, NULL)
+  ) AS organic_pocket_clicked_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND organic_pocket_clicks > 0, newtab_visit_id, NULL)
+  ) AS organic_pocket_clicked_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND pocket_impressions > 0, client_id, NULL)
+  ) AS pocket_impressed_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND pocket_impressions > 0, newtab_visit_id, NULL)
+  ) AS pocket_impressed_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND pocket_clicks > 0, client_id, NULL)
+  ) AS pocket_clicked_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND pocket_clicks > 0, newtab_visit_id, NULL)
+  ) AS pocket_clicked_visit_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND pocket_is_signed_in, client_id, NULL)
+  ) AS pocket_signed_in_client_count,
+  COUNT(
+    DISTINCT IF(channel = 'release' AND pocket_is_signed_in, newtab_visit_id, NULL)
+  ) AS pocket_signed_in_visit_count,
+FROM
+  `moz-fx-data-shared-prod.telemetry_derived.newtab_interactions_v1`
+WHERE
+  submission_date = @submission_date
+GROUP BY
+  submission_date,
+  country_code,
+  pocket_enabled,
+  pocket_sponsored_stories_enabled,
+  topsites_enabled
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/schema.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/schema.yaml	2025-01-24 23:19:25.000000000 +0000
@@ -0,0 +1,142 @@
+fields:
+- name: submission_date
+  type: DATE
+  mode: NULLABLE
+- name: country_code
+  type: STRING
+  mode: NULLABLE
+- name: pocket_enabled
+  type: BOOLEAN
+  mode: NULLABLE
+- name: pocket_sponsored_stories_enabled
+  type: BOOLEAN
+  mode: NULLABLE
+- name: topsites_enabled
+  type: BOOLEAN
+  mode: NULLABLE
+- name: total_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: total_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: homepage_category_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: homepage_category_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: newtab_category_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: newtab_category_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: home_default_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: home_default_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: newtab_default_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: newtab_default_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: home_nondefault_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: home_nondefault_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: newtab_nondefault_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: newtab_nondefault_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: welcome_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: welcome_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: searched_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: searched_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_topsite_impressed_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_topsite_impressed_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_topsite_clicked_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_topsite_clicked_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: topsite_impressed_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: topsite_impressed_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: topsite_clicked_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: topsite_clicked_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_pocket_impressed_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_pocket_impressed_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_pocket_clicked_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_pocket_clicked_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_pocket_impressed_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_pocket_impressed_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_pocket_clicked_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_pocket_clicked_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: pocket_impressed_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: pocket_impressed_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: pocket_clicked_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: pocket_clicked_visit_count
+  type: INTEGER
+  mode: NULLABLE
+- name: pocket_signed_in_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: pocket_signed_in_visit_count
+  type: INTEGER
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/metadata.yaml	2025-01-24 23:22:36.000000000 +0000
@@ -0,0 +1,29 @@
+friendly_name: Newtab Interactions Daily Aggregates
+description: |-
+  A daily aggregation of newtab interactions, partitioned by day.
+owners:
+- [email protected]
+labels:
+  application: firefox
+  incremental: true
+  schedule: daily
+  dag: bqetl_newtab
+  owner1: gkatre
+scheduling:
+  dag_name: bqetl_newtab
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: true
+    expiration_days: null
+  range_partitioning: null
+  clustering:
+    fields:
+    - channel
+    - country_code
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references: {}
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/query.sql	2025-01-24 23:19:25.000000000 +0000
@@ -0,0 +1,83 @@
+SELECT
+  submission_date,
+  search_engine,
+  search_access_point,
+  pocket_story_position,
+  default_private_search_engine,
+  default_search_engine,
+  os,
+  os_version,
+  country_code,
+  channel,
+  browser_version,
+  browser_name,
+  newtab_open_source,
+  pocket_is_signed_in,
+  pocket_enabled,
+  pocket_sponsored_stories_enabled,
+  topsites_enabled,
+  newtab_homepage_category,
+  newtab_newtab_category,
+  newtab_search_enabled,
+  topsites_rows,
+  is_new_profile,
+  activity_segment,
+  COUNT(DISTINCT client_id) AS client_count,
+  COUNT(DISTINCT legacy_telemetry_client_id) AS legacy_telemetry_client_count,
+  COUNT(DISTINCT newtab_visit_id) AS newtab_interactions_visits,
+        -- Topsite
+  SUM(topsite_clicks) AS topsite_clicks,
+  SUM(sponsored_topsite_clicks) AS sponsored_topsite_clicks,
+  SUM(organic_topsite_clicks) AS organic_topsite_clicks,
+  SUM(topsite_impressions) AS topsite_impressions,
+  SUM(sponsored_topsite_impressions) AS sponsored_topsite_impressions,
+  SUM(organic_topsite_impressions) AS organic_topsite_impressions,
+  SUM(topsite_dismissals) AS topsite_dismissals,
+  SUM(sponsored_topsite_dismissals) AS sponsored_topsite_dismissals,
+  SUM(organic_topsite_dismissals) AS organic_topsite_dismissals,
+        -- Search
+  SUM(searches) AS searches,
+  SUM(tagged_search_ad_clicks) AS tagged_search_ad_clicks,
+  SUM(tagged_search_ad_impressions) AS tagged_search_ad_impressions,
+  SUM(follow_on_search_ad_clicks) AS follow_on_search_ad_clicks,
+  SUM(follow_on_search_ad_impressions) AS follow_on_search_ad_impressions,
+  SUM(tagged_follow_on_search_ad_clicks) AS tagged_follow_on_search_ad_clicks,
+  SUM(tagged_follow_on_search_ad_impressions) AS tagged_follow_on_search_ad_impressions,
+        -- Pocket
+  SUM(pocket_impressions) AS pocket_impressions,
+  SUM(sponsored_pocket_impressions) AS sponsored_pocket_impressions,
+  SUM(organic_pocket_impressions) AS organic_pocket_impressions,
+  SUM(pocket_clicks) AS pocket_clicks,
+  SUM(sponsored_pocket_clicks) AS sponsored_pocket_clicks,
+  SUM(organic_pocket_clicks) AS organic_pocket_clicks,
+  SUM(pocket_saves) AS pocket_saves,
+  SUM(sponsored_pocket_saves) AS sponsored_pocket_saves,
+  SUM(organic_pocket_saves) AS organic_pocket_saves,
+FROM
+  `moz-fx-data-shared-prod.telemetry_derived.newtab_interactions_v1`
+WHERE
+  submission_date = @submission_date
+GROUP BY
+  submission_date,
+  search_engine,
+  search_access_point,
+  pocket_story_position,
+  default_private_search_engine,
+  default_search_engine,
+  os,
+  os_version,
+  country_code,
+  channel,
+  browser_version,
+  browser_name,
+  newtab_open_source,
+  pocket_is_signed_in,
+  pocket_enabled,
+  pocket_sponsored_stories_enabled,
+  topsites_enabled,
+  newtab_homepage_category,
+  newtab_newtab_category,
+  newtab_search_enabled,
+  topsites_rows,
+  is_new_profile,
+  activity_segment
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/schema.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/newtab_daily_interactions_aggregates_v1/schema.yaml	2025-01-24 23:19:25.000000000 +0000
@@ -0,0 +1,145 @@
+fields:
+- name: submission_date
+  type: DATE
+  mode: NULLABLE
+- name: country_code
+  type: STRING
+  mode: NULLABLE
+- name: channel
+  type: STRING
+  mode: NULLABLE
+- name: browser_version
+  type: STRING
+  mode: NULLABLE
+- name: browser_name
+  type: STRING
+  mode: NULLABLE
+- name: search_engine
+  type: STRING
+  mode: NULLABLE
+- name: search_access_point
+  type: STRING
+  mode: NULLABLE
+- name: default_search_engine
+  type: STRING
+  mode: NULLABLE
+- name: default_private_search_engine
+  type: STRING
+  mode: NULLABLE
+- name: pocket_story_position
+  type: INTEGER
+  mode: NULLABLE
+- name: newtab_open_source
+  type: STRING
+  mode: NULLABLE
+- name: pocket_is_signed_in
+  type: BOOLEAN
+  mode: NULLABLE
+- name: pocket_enabled
+  type: BOOLEAN
+  mode: NULLABLE
+- name: pocket_sponsored_stories_enabled
+  type: BOOLEAN
+  mode: NULLABLE
+- name: topsites_enabled
+  type: BOOLEAN
+  mode: NULLABLE
+- name: newtab_homepage_category
+  type: STRING
+  mode: NULLABLE
+- name: newtab_newtab_category
+  type: STRING
+  mode: NULLABLE
+- name: topsites_rows
+  type: INTEGER
+  mode: NULLABLE
+- name: is_new_profile
+  type: BOOLEAN
+  mode: NULLABLE
+- name: activity_segment
+  type: STRING
+  mode: NULLABLE
+- name: client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: legacy_telemetry_client_count
+  type: INTEGER
+  mode: NULLABLE
+- name: newtab_interactions_visits
+  type: INTEGER
+  mode: NULLABLE
+- name: topsite_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_topsite_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_topsite_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: topsite_impressions
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_topsite_impressions
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_topsite_impressions
+  type: INTEGER
+  mode: NULLABLE
+- name: topsite_dismissals
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_topsite_dismissals
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_topsite_dismissals
+  type: INTEGER
+  mode: NULLABLE
+- name: searches
+  type: INTEGER
+  mode: NULLABLE
+- name: tagged_search_ad_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: tagged_search_ad_impressions
+  type: INTEGER
+  mode: NULLABLE
+- name: follow_on_search_ad_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: follow_on_search_ad_impressions
+  type: INTEGER
+  mode: NULLABLE
+- name: tagged_follow_on_search_ad_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: tagged_follow_on_search_ad_impressions
+  type: INTEGER
+  mode: NULLABLE
+- name: pocket_impressions
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_pocket_impressions
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_pocket_impressions
+  type: INTEGER
+  mode: NULLABLE
+- name: pocket_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_pocket_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_pocket_clicks
+  type: INTEGER
+  mode: NULLABLE
+- name: pocket_saves
+  type: INTEGER
+  mode: NULLABLE
+- name: sponsored_pocket_saves
+  type: INTEGER
+  mode: NULLABLE
+- name: organic_pocket_saves
+  type: INTEGER
+  mode: NULLABLE

Link to full diff

alekhyamoz · 2025-01-24T23:29:57Z

It seems the dashboard is using some columns from newtab_visits_v1 that don't exist in newtab_clients_daily_v1, such as had_non_impression_engagement in https://mozilla.cloud.looker.com/explore/firefox_desktop/newtab_visits?qid=PF3wIICSN13op7HN9ZeTKT&origin_space=464&toggle=vis

But it's unclear if there is an use case for historical data there due to the same issue of unclear timeframes for aggregation.

EDIT: this one actually look at the % over time, is this data point covered? https://mozilla.cloud.looker.com/explore/firefox_desktop/newtab_visits?qid=dh3tzf1nHzKbGVOATeQKrM&origin_space=464&toggle=vis

Is this referring to the weekly rollover? If so, I’ve already mentioned to Carolyn that we won’t be able to include metrics that rely on weekly rolling calculations; however, we will still be able to collect daily metrics.

gkatre added 4 commits January 21, 2025 10:35

DENG-6880 updates to include new fields and aggregations

4330322

adding interactions aggregations

5593c06

removing unwanted aggs

5a0d694

DENG-6880 - Daily aggregations for NewTab

c95300d

gkatre requested review from bani and alekhyamoz January 24, 2025 06:54

gkatre and others added 2 commits January 24, 2025 10:10

updating schema.yaml

b58055e

Merge branch 'main' into newtab/DENG-6880-newtab_update_aggregations

a5ec6e2

This comment has been minimized.

Sign in to view

BenWu reviewed Jan 24, 2025

View reviewed changes

...moz-fx-data-shared-prod/telemetry_derived/newtab_conditional_daily_aggregates_v1/schema.yaml Outdated Show resolved Hide resolved

Update sql/moz-fx-data-shared-prod/telemetry_derived/newtab_condition…

793aa7b

…al_daily_aggregates_v1/schema.yaml Co-authored-by: Ben Wu <[email protected]>

This comment has been minimized.

Sign in to view

alekhyamoz approved these changes Jan 24, 2025

View reviewed changes

bani reviewed Jan 24, 2025

View reviewed changes

gkatre requested a review from bani January 24, 2025 22:43

Adding homepage_category and newtab_category computations

fa5746a

This comment has been minimized.

Sign in to view

gkatre and others added 2 commits January 24, 2025 15:17

formatting

4a1f958

Merge branch 'main' into newtab/DENG-6880-newtab_update_aggregations

7f9e282

bani approved these changes Jan 27, 2025

View reviewed changes

gkatre added this pull request to the merge queue Jan 27, 2025

Merged via the queue into main with commit d9bbcbf Jan 27, 2025
21 checks passed

gkatre deleted the newtab/DENG-6880-newtab_update_aggregations branch January 27, 2025 17:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DENG-6880 - Daily aggregations for NewTab #6863

DENG-6880 - Daily aggregations for NewTab #6863

gkatre commented Jan 24, 2025 •

edited by data-sync-user

Loading

alekhyamoz commented Jan 24, 2025

This comment has been minimized.

This comment has been minimized.

gkatre commented Jan 24, 2025 •

edited

Loading

alekhyamoz commented Jan 24, 2025

gkatre commented Jan 24, 2025

alekhyamoz left a comment

bani Jan 24, 2025

bani Jan 24, 2025

gkatre Jan 24, 2025 •

edited

Loading

bani Jan 24, 2025

gkatre Jan 24, 2025

gkatre Jan 24, 2025

gkatre commented Jan 24, 2025

bani Jan 24, 2025

gkatre Jan 24, 2025

bani Jan 24, 2025

bani Jan 24, 2025

gkatre Jan 24, 2025

bani Jan 24, 2025

bani commented Jan 24, 2025

gkatre commented Jan 24, 2025

bani commented Jan 24, 2025

bani commented Jan 24, 2025 •

edited

Loading

gkatre commented Jan 24, 2025

gkatre commented Jan 24, 2025 •

edited

Loading

gkatre commented Jan 24, 2025

This comment has been minimized.

dataops-ci-bot commented Jan 24, 2025

alekhyamoz commented Jan 24, 2025

DENG-6880 - Daily aggregations for NewTab #6863

DENG-6880 - Daily aggregations for NewTab #6863

Conversation

gkatre commented Jan 24, 2025 • edited by data-sync-user Loading

Description

Related Tickets & Documents

alekhyamoz commented Jan 24, 2025

This comment has been minimized.

This comment has been minimized.

gkatre commented Jan 24, 2025 • edited Loading

alekhyamoz commented Jan 24, 2025

gkatre commented Jan 24, 2025

alekhyamoz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gkatre Jan 24, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gkatre commented Jan 24, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bani commented Jan 24, 2025

gkatre commented Jan 24, 2025

bani commented Jan 24, 2025

bani commented Jan 24, 2025 • edited Loading

gkatre commented Jan 24, 2025

gkatre commented Jan 24, 2025 • edited Loading

gkatre commented Jan 24, 2025

This comment has been minimized.

dataops-ci-bot commented Jan 24, 2025

Integration report for "Merge branch 'main' into newtab/DENG-6880-newtab_update_aggregations"

sql.diff

alekhyamoz commented Jan 24, 2025

gkatre commented Jan 24, 2025 •

edited by data-sync-user

Loading

gkatre commented Jan 24, 2025 •

edited

Loading

gkatre Jan 24, 2025 •

edited

Loading

bani commented Jan 24, 2025 •

edited

Loading

gkatre commented Jan 24, 2025 •

edited

Loading

`sql.diff`