Skip to content

feat(SubPlat): Attribution Channel Group field in logical & service subscription ETLs (DENG-10391)#8756

Draft
phil-lee70 wants to merge 19 commits intomainfrom
deng-10391
Draft

feat(SubPlat): Attribution Channel Group field in logical & service subscription ETLs (DENG-10391)#8756
phil-lee70 wants to merge 19 commits intomainfrom
deng-10391

Conversation

@phil-lee70
Copy link
Contributor

@phil-lee70 phil-lee70 commented Jan 26, 2026

Description

This PR adds the channel Attribution Channel Group field as channel_group to logical and service subscriptions.

It creates a udf to map utm_source to channel_group, and then passes the field to downstream schema.yaml files:

logical_subscriptions_history_v1
service_subscriptions_history_v1

Related Tickets & Documents

Reviewer, please follow this checklist

@phil-lee70 phil-lee70 requested a review from sean-rose January 26, 2026 15:20
@phil-lee70 phil-lee70 marked this pull request as draft January 26, 2026 15:21
@dataops-ci-bot

This comment has been minimized.

@dataops-ci-bot

This comment has been minimized.

@dataops-ci-bot

This comment has been minimized.

BenWu and others added 16 commits February 4, 2026 12:16
Bumps [rich-click](https://github.com/ewels/rich-click) from 1.9.5 to 1.9.7.
- [Release notes](https://github.com/ewels/rich-click/releases)
- [Changelog](https://github.com/ewels/rich-click/blob/main/CHANGELOG.md)
- [Commits](ewels/rich-click@v1.9.5...v1.9.7)

---
updated-dependencies:
- dependency-name: rich-click
  dependency-version: 1.9.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…performance DB. (#8796)

* feat(firefoxdotcom): added site_engagement_events to enable a looker performance DB.

* feat(firefoxdotcom): added site_engagement_events to enable a looker performance DB, removed depends on logic.

* chore(site_engagement_events_v1): updates to depends on logic
…til.common` to avoid a circular import (bug 2013847). (#8799)
* Add a second version of the forecast table with updated schema. This enables forecasts with either glean or legacy telemetry.

* Fix bugs and typos
* feat: delete apple_ads_external namespace as no longer needed

* feat: delete bqetl_fivetran_apple_ads DAG definition as there are no more queries associated with it

* feat: delete reference from apple_ads_external in bqetl config and docs
…ting (DENG-9664) (#8749)

* Add payment_method field to SubPlat consolidated reporting

* add payment_method_id to google and apple subscriptions

* remove payment_method_id and add payment_method logic

* add payment tables to stripe_external dataset and to downstream yaml files

* change table name and correct yaml files

* correct yaml files

---------

Co-authored-by: Philip Lee <plee@Philips-MacBook-Pro.local>
* fix: add error handling to newtab duration calcs

* fix avg duration field type

* Update sql/moz-fx-data-shared-prod/firefox_desktop_derived/newtab_clients_daily_v2/query.sql

Co-authored-by: Lucia <30448600+lucia-vargas-a@users.noreply.github.com>

* Update sql/moz-fx-data-shared-prod/firefox_desktop_derived/newtab_visits_daily_v2/query.sql

Co-authored-by: Lucia <30448600+lucia-vargas-a@users.noreply.github.com>

* fix formatting of case

---------

Co-authored-by: Lucia <30448600+lucia-vargas-a@users.noreply.github.com>
* [DENG-859] Support query.py in `bqetl query backfill`

* remove unneeded

* Support non-click scripts

* Update test
@dataops-ci-bot
Copy link

Integration report for "Merge branch 'main' into deng-10391"

sql.diff

Click to expand!
Only in /tmp/workspace/generated-sql/sql/mozfun/norm: map_utm_to_channel_group
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/mozfun/norm/map_utm_to_channel_group/metadata.yaml /tmp/workspace/generated-sql/sql/mozfun/norm/map_utm_to_channel_group/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/mozfun/norm/map_utm_to_channel_group/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/mozfun/norm/map_utm_to_channel_group/metadata.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -0,0 +1,18 @@
+---
+friendly_name: Map UTM Source to Channel Group
+description: |
+  Maps UTM source parameters to predefined channel groups for attribution analysis.
+
+  This function categorizes UTM sources into channel groups:
+  - Marketing Owned: Mozilla-owned marketing channels (newsletters, blogs, social)
+  - Product Owned: In-product referrals (newtab, Firefox features, Pocket)
+  - Marketing Paid: Paid advertising channels (Google, Facebook, Display)
+  - Direct: Direct traffic from Mozilla properties
+  - Miscellaneous: Other or unmapped sources
+
+  The function uses pattern matching on utm_source values to assign appropriate
+  channel classifications for marketing attribution and analytics.
+
+  Data source:
+  - Marketing UTM Parameters: https://docs.google.com/spreadsheets/d/1YSQriUKwq9sz4WLBKRAfupb9OWSOJabm7Qo3RrIf7kI/edit?gid=1059980294#gid=1059980294
+  - UTM Source mapping: Building an MRR Dataset Script Code Block 30: https://drive.google.com/drive/folders/1xBG2zgmIMgrX6Rvrw3y1ZAJbn8ffA3Af
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/mozfun/norm/map_utm_to_channel_group/README.md /tmp/workspace/generated-sql/sql/mozfun/norm/map_utm_to_channel_group/README.md
--- /tmp/workspace/main-generated-sql/sql/mozfun/norm/map_utm_to_channel_group/README.md	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/mozfun/norm/map_utm_to_channel_group/README.md	2026-02-04 17:26:41.000000000 +0000
@@ -0,0 +1,15 @@
+Usage:
+
+```sql
+SELECT
+  mozfun.norm.map_utm_to_channel_group(utm_source) AS channel_group
+FROM
+  your_table
+```
+
+Returns one of the following channel group categories:
+- `Marketing Owned` - Mozilla-owned marketing properties
+- `Product Owned` - In-product features and Firefox properties
+- `Marketing Paid` - Paid advertising channels
+- `Direct` - Direct traffic
+- `Miscellaneous` - Other or unmapped sources
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/mozfun/norm/map_utm_to_channel_group/udf.sql /tmp/workspace/generated-sql/sql/mozfun/norm/map_utm_to_channel_group/udf.sql
--- /tmp/workspace/main-generated-sql/sql/mozfun/norm/map_utm_to_channel_group/udf.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/mozfun/norm/map_utm_to_channel_group/udf.sql	2026-02-04 17:26:41.000000000 +0000
@@ -0,0 +1,184 @@
+CREATE OR REPLACE FUNCTION norm.map_utm_to_channel_group(utm_source STRING)
+RETURNS STRING AS (
+  CASE
+    -- Marketing Owned
+    WHEN utm_source LIKE '%mozilla.org-whatsnew%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%mozilla.org-welcome%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%blog.mozilla.org%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%fxnews%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%fxavpn%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%fxatips%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%fxakip%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%vpnwaitlist%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%monitor%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%twitter.com%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%fxaonboardingemail%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%invite%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%pockethits%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%pkt-hits%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%news%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%sync-onboarding%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%stage.fxprivaterelay.nonprod.cloudops.mozgcp.net%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%instagram%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%wrapped_email%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%relay-onboarding%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%oim%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%firefox-desktop%'
+      THEN 'Marketing Owned'
+    -- Direct
+    WHEN utm_source = 'www.mozilla.org-vpn-product-page'
+      THEN 'Direct'
+    WHEN utm_source = 'google-play'
+      THEN 'Direct'
+    WHEN utm_source = 'product'
+      THEN 'Direct'
+    -- Product Owned
+    WHEN utm_source LIKE '%about-prefs%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%leanplum-push-notification%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%firefox-browser%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%newtab%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%monitor.firefox.com%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%fx-monitor%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%relay-firefox-com.translate.goog%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%accounts.firefox.com%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%privatebrowser%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%pocket%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%fx-vpn-windows%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%toolbar%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%spotlight-modal%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%thunderbird%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%sponsoredtile%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%activity-stream%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%mozilla.org-firefox-accounts%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%mozilla.org-firefox_home%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%send.firefox.com%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%addons.mozilla.org%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%premium.firefox.com%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%fpn.firefox.com%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%leanplum-push-qa%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%fx-ios-vpn%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%modal%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%fx-relay-addon%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%fx-relay%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%mozilla.org-firefox-browsers%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%new-tab-ad%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%firefox%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%pocket_saves%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%www.mozilla.org-vpn-or-proxy%'
+      THEN 'Product Owned'
+    WHEN utm_source LIKE '%fx-vpn-iOSs%'
+      THEN 'Product Owned'
+    -- Marketing Paid
+    WHEN utm_source LIKE '%facebook%'
+      OR utm_source LIKE '%instagram%'
+      THEN 'Marketing Paid'
+    WHEN utm_source LIKE '%google%'
+      THEN 'Marketing Paid'
+    WHEN utm_source = 'reddit'
+      THEN 'Marketing Paid'
+    WHEN utm_source = 'dv360'
+      THEN 'Marketing Paid'
+    WHEN utm_source LIKE '%saasworthy.com%'
+      THEN 'Marketing Paid'
+    WHEN utm_source LIKE '%youtube%'
+      THEN 'Marketing Paid'
+    -- Miscellaneous (catch-all for remaining patterns)
+    WHEN utm_source LIKE '%relay%'
+      THEN 'Miscellaneous'
+    WHEN utm_source LIKE '%desktop-signup-flow%'
+      THEN 'Miscellaneous'
+    WHEN utm_source = 'invalid'
+      THEN 'Miscellaneous'
+    WHEN utm_source LIKE '%multi.account.containers%'
+      THEN 'Miscellaneous'
+    WHEN utm_source IN (
+        'yahoo',
+        'bdmtools',
+        'private-relay',
+        'Blog',
+        'duckduckgo',
+        'bing',
+        'www.mozilla.org-vpn-info',
+        'FuckOff',
+        '(not set)',
+        'baidu',
+        'chrome',
+        'vpnsite',
+        'about-preferences',
+        'gk_test',
+        'yandex',
+        'manual_testing',
+        'demo_1_server',
+        'Drippler',
+        'vpn.',
+        'saashub',
+        'addon',
+        'teaching-the-peeps',
+        'test0706',
+        'pocket_mylist',
+        'devtools',
+        'fpn-default',
+        'fxa'
+      )
+      THEN 'Miscellaneous'
+    -- Default for any unmapped values (based on the data, this should be Marketing Owned for most mozilla domains)
+    WHEN utm_source LIKE '%mozilla.org%'
+      THEN 'Marketing Owned'
+    WHEN utm_source LIKE '%.mozilla.org%'
+      THEN 'Marketing Owned'
+    -- Final fallback
+    ELSE 'Miscellaneous'
+  END
+);
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_logical_subscriptions_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_logical_subscriptions_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_logical_subscriptions_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_logical_subscriptions_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -353,6 +353,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_logical_subscriptions_v1_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_logical_subscriptions_v1_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_logical_subscriptions_v1_live/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_logical_subscriptions_v1_live/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -353,6 +353,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_service_subscriptions_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_service_subscriptions_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_service_subscriptions_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_service_subscriptions_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -446,6 +446,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_service_subscriptions_v1_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_service_subscriptions_v1_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_service_subscriptions_v1_live/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/daily_active_service_subscriptions_v1_live/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -446,6 +446,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscription_events_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscription_events_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscription_events_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscription_events_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -363,6 +363,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
@@ -778,6 +783,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscription_events_v1_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscription_events_v1_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscription_events_v1_live/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscription_events_v1_live/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -363,6 +363,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
@@ -778,6 +783,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_history_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_history_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_history_v1/query.sql	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_history_v1/query.sql	2026-02-04 17:26:41.000000000 +0000
@@ -67,6 +67,24 @@
   FULL JOIN
     `moz-fx-data-shared-prod.subscription_platform_derived.stripe_logical_subscriptions_attribution_v2` AS attribution_v2
     USING (subscription_id)
+),
+subscription_attributions_with_channel AS (
+  SELECT
+    subscription_id,
+    first_touch_attribution,
+    CASE
+      WHEN last_touch_attribution IS NULL
+        THEN NULL
+      ELSE (
+          SELECT AS STRUCT
+            lta.*,
+            norm.map_utm_to_channel_group(lta.utm_source) AS channel_group
+          FROM
+            (SELECT last_touch_attribution AS lta)
+        )
+    END AS last_touch_attribution
+  FROM
+    subscription_attributions
 )
 SELECT
   history.id,
@@ -137,8 +155,8 @@
     history.subscription.auto_renew_disabled_at,
     history.subscription.has_refunds,
     history.subscription.has_fraudulent_charges,
-    subscription_attributions.first_touch_attribution,
-    subscription_attributions.last_touch_attribution,
+    subscription_attributions_with_channel.first_touch_attribution,
+    subscription_attributions_with_channel.last_touch_attribution,
     history.subscription.initial_discount_name,
     history.subscription.initial_discount_promotion_code,
     history.subscription.current_period_discount_name,
@@ -168,5 +186,5 @@
   countries
   ON history.subscription.country_code = countries.code
 LEFT JOIN
-  subscription_attributions
-  ON history.subscription.id = subscription_attributions.subscription_id
+  subscription_attributions_with_channel
+  ON history.subscription.id = subscription_attributions_with_channel.subscription_id
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_history_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_history_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_history_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_history_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -359,6 +359,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/logical_subscriptions_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -332,6 +332,11 @@
     mode: NULLABLE
     description: |-
       Last-touch attribution UTM term.
+  - name: channel_group
+    type: STRING
+    mode: NULLABLE
+    description: |-
+      Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
 - name: initial_discount_name
   type: STRING
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_logical_subscriptions_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_logical_subscriptions_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_logical_subscriptions_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_logical_subscriptions_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -358,6 +358,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_logical_subscriptions_v1_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_logical_subscriptions_v1_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_logical_subscriptions_v1_live/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_logical_subscriptions_v1_live/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -358,6 +358,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_service_subscriptions_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_service_subscriptions_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_service_subscriptions_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_service_subscriptions_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -451,6 +451,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_service_subscriptions_v1_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_service_subscriptions_v1_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_service_subscriptions_v1_live/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/monthly_active_service_subscriptions_v1_live/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -451,6 +451,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_daily_active_logical_subscriptions_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_daily_active_logical_subscriptions_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_daily_active_logical_subscriptions_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_daily_active_logical_subscriptions_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -353,6 +353,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_daily_active_service_subscriptions_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_daily_active_service_subscriptions_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_daily_active_service_subscriptions_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_daily_active_service_subscriptions_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -446,6 +446,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_logical_subscription_events_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_logical_subscription_events_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_logical_subscription_events_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_logical_subscription_events_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -363,6 +363,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
@@ -778,6 +783,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_monthly_active_logical_subscriptions_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_monthly_active_logical_subscriptions_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_monthly_active_logical_subscriptions_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_monthly_active_logical_subscriptions_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -358,6 +358,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: initial_discount_name
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_monthly_active_service_subscriptions_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_monthly_active_service_subscriptions_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_monthly_active_service_subscriptions_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_monthly_active_service_subscriptions_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -451,6 +451,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_service_subscription_events_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_service_subscription_events_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_service_subscription_events_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/recent_service_subscription_events_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -456,6 +456,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
@@ -909,6 +914,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscription_events_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscription_events_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscription_events_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscription_events_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -456,6 +456,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
@@ -909,6 +914,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscription_events_v1_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscription_events_v1_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscription_events_v1_live/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscription_events_v1_live/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -456,6 +456,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
@@ -909,6 +914,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_history_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_history_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_history_v1/query.sql	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_history_v1/query.sql	2026-02-04 17:26:41.000000000 +0000
@@ -91,6 +91,24 @@
     `moz-fx-data-shared-prod.subscription_platform_derived.stripe_service_subscriptions_attribution_v2` AS attribution_v2
     USING (subscription_id)
 ),
+subscription_attributions_with_channel AS (
+  SELECT
+    subscription_id,
+    first_touch_attribution,
+    CASE
+      WHEN last_touch_attribution IS NULL
+        THEN NULL
+      ELSE (
+          SELECT AS STRUCT
+            lta.*,
+            norm.map_utm_to_channel_group(lta.utm_source) AS channel_group
+          FROM
+            (SELECT last_touch_attribution AS lta)
+        )
+    END AS last_touch_attribution
+  FROM
+    subscription_attributions
+),
 subscriptions_history AS (
   SELECT
     CONCAT(
@@ -160,8 +178,8 @@
       history.subscription.ongoing_discount_ends_at,
       history.subscription.has_refunds,
       history.subscription.has_fraudulent_charges,
-      subscription_attributions.first_touch_attribution,
-      subscription_attributions.last_touch_attribution,
+      subscription_attributions_with_channel.first_touch_attribution,
+      subscription_attributions_with_channel.last_touch_attribution,
       history.subscription.ended_reason,
       CONCAT(
         IF(
@@ -184,8 +202,8 @@
     AND history.valid_from >= subscriptions_history_periods.started_at
     AND history.valid_from < subscriptions_history_periods.ended_at
   LEFT JOIN
-    subscription_attributions
-    ON subscriptions_history_periods.subscription_id = subscription_attributions.subscription_id
+    subscription_attributions_with_channel
+    ON subscriptions_history_periods.subscription_id = subscription_attributions_with_channel.subscription_id
 ),
 synthetic_subscription_ends_history AS (
   -- Synthesize subscription end history records if subscriptions get downgraded to no longer include a service.
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_history_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_history_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_history_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_history_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -447,6 +447,11 @@
       mode: NULLABLE
       description: |-
         Last-touch attribution UTM term.
+    - name: channel_group
+      type: STRING
+      mode: NULLABLE
+      description: |-
+        Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
   - name: ended_reason
     type: STRING
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_v1/schema.yaml	2026-02-04 17:28:36.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/subscription_platform_derived/service_subscriptions_v1/schema.yaml	2026-02-04 17:26:41.000000000 +0000
@@ -425,6 +425,11 @@
     mode: NULLABLE
     description: |-
       Last-touch attribution UTM term.
+  - name: channel_group
+    type: STRING
+    mode: NULLABLE
+    description: |-
+      Categorized marketing channel group (Direct, Marketing Owned, Marketing Paid, Product Owned, or Miscellaneous)
 - name: ended_reason
   type: STRING
   mode: NULLABLE

Link to full diff

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants