-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new category field #132
Changes from all commits
932edc0
99afc0d
5e315e6
2957030
7822cec
d57c075
4bda432
5a97f6e
ad968f8
cebe8a6
1ab6ea3
e4505bd
e0b3439
f184aad
64f3544
2974eb2
db77823
65feffa
35f0438
3a1ca0f
34dc093
6ff3090
567ab91
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,28 @@ | ||
# dbt_hubspot_source v0.17.0 | ||
|
||
## Breaking Change (`--full-refresh` required after upgrading) | ||
- Introduced a new `category` column to the below models. This association field differentiates records by either HUBSPOT_DEFINED (default label) or USER_DEFINED (custom label) and was introduced to the Hubspot connector in October 2024. See the [connector release notes](https://fivetran.com/docs/connectors/applications/hubspot/changelog#october2024) for more. | ||
- `stg_hubspot__deal_company` | ||
- `stg_hubspot__deal_contact` | ||
- `stg_hubspot__engagement_company` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @fivetran-reneeli I do not see the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Great question! So these models leverage a macro to bring through all the columns and therefore the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah makes sense! Thanks for that. |
||
- `stg_hubspot__engagement_contact` | ||
- `stg_hubspot__engagement_deal` | ||
- `stg_hubspot__ticket_company` | ||
- `stg_hubspot__ticket_contact` | ||
- `stg_hubspot__ticket_deal` | ||
- `stg_hubspot__ticket_engagement` | ||
|
||
- Added explicit casting of `pipeline_id` to string in `stg_hubspot__deal_pipeline_stage` to ensure successful downstream joins. | ||
|
||
## Bug Fixes | ||
- Updated the unique key to include the new `category` field for the following models in order for the uniqueness tests to pass: | ||
- `stg_hubspot__deal_contact` | ||
- `stg_hubspot__deal_company` | ||
|
||
## Under the Hood | ||
- Updated the respective seed files, `get_[source_table]_column` macros, and documentation of the mentioned models to include the `category` field. | ||
- Removed `property_hs_timestamp` and `property_hs_createdate` field casting for the `engagement_meeting` source table in the `integration_tests/dbt_project.yml` file since they are not present in the seed data. | ||
|
||
# dbt_hubspot_source v0.16.0 | ||
[PR #129](https://github.com/fivetran/dbt_hubspot_source/pull/129) includes the following updates: | ||
|
||
|
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,13 +1,13 @@ | ||
name: 'hubspot_source_integration_tests' | ||
version: '0.16.0' | ||
version: '0.17.0' | ||
profile: 'integration_tests' | ||
config-version: 2 | ||
|
||
models: | ||
+schema: "{{ 'hubspot_sqlw_tests_1' if target.name == 'databricks-sql' else 'hubspot' }}" | ||
+schema: "{{ 'hubspot_sqlw_tests_7' if target.name == 'databricks-sql' else 'hubspot' }}" | ||
|
||
vars: | ||
hubspot_schema: hubspot_source_integration_tests_28 | ||
hubspot_schema: hubspot_source_integration_tests_34 | ||
hubspot_source: | ||
# hubspot_service_enabled: true # enable when generating docs | ||
# hubspot_deal_enabled: true # enable when generating docs | ||
|
@@ -18,6 +18,7 @@ vars: | |
# hubspot_contact_merge_audit_enabled: true # enable when generating docs | ||
# hubspot_using_all_email_events: true # enable when generating docs | ||
# hubspot_merged_deal_enabled: true # enable when generating docs | ||
# hubspot_ticket_deal_enabled: true # enable when generating docs | ||
hubspot_company_property_history_identifier: "company_property_history_data" | ||
hubspot_company_identifier: "company_data" | ||
hubspot_contact_identifier: "contact_data" | ||
|
@@ -86,24 +87,26 @@ seeds: | |
+alias: company_data | ||
+enabled: "{{ true if target.type == 'postgres' else false }}" | ||
+column_types: | ||
id: bigint | ||
id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
company_data_snowflake: | ||
+alias: company_data | ||
+enabled: "{{ true if target.type == 'snowflake' else false }}" | ||
+column_types: | ||
id: bigint | ||
id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
company_data_databricks: | ||
+alias: company_data | ||
+enabled: "{{ true if target.type in ('databricks','databricks-sql') else false }}" | ||
+column_types: | ||
id: bigint | ||
id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
deal_data: | ||
+column_types: | ||
deal_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
owner_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
_fivetran_synced: timestamp | ||
property_closedate: timestamp | ||
property_createdate: timestamp | ||
deal_pipeline_id: "{{ 'varchar(100)' if target.type in ('redshift','postgres') else 'string'}}" | ||
deal_pipeline_stage_id: "{{ 'varchar(100)' if target.type in ('redshift','postgres') else 'string'}}" | ||
deal_contact_data: | ||
+column_types: | ||
contact_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
|
@@ -139,15 +142,14 @@ seeds: | |
+column_types: | ||
engagement_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
deal_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
created_at: timestamp | ||
engagement_data: | ||
+column_types: | ||
id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
engagement_meeting_data: | ||
+column_types: | ||
engagement_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
_fivetran_synced: timestamp | ||
property_hs_createdate: timestamp | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. was there a reason this casting was removed for these fields? it wasn't called out in the CHANGELOG. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah I had realized that these fields were not existent in the seed data and therefore not necessary. I'll call this out in the Under the Hood section too |
||
property_hs_timestamp: timestamp | ||
engagement_email_data: | ||
+column_types: | ||
engagement_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}" | ||
|
@@ -173,6 +175,7 @@ seeds: | |
deal_pipeline_stage_data: | ||
+column_types: | ||
stage_id: "{{ 'varchar(100)' if target.type in ('redshift','postgres') else 'string'}}" | ||
pipeline_id: "{{ 'varchar(100)' if target.type in ('redshift','postgres') else 'string'}}" | ||
email_event_sent_data_snowflake: | ||
+alias: email_event_sent_data | ||
+enabled: "{{ true if target.type == 'snowflake' else false }}" | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
company_id,deal_id,type_id | ||
12345,6789,101112 | ||
56789,1234,101112 | ||
company_id,deal_id,type_id,category | ||
12345,6789,101112,HUBSPOT_DEFINED | ||
56789,1234,101112,HUBSPOT_DEFINED |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,12 @@ | ||
contact_id,deal_id,_fivetran_synced,type_id | ||
1499466,86133127,2022-04-14 4:30:42,101112 | ||
1499466,86134193,2022-04-14 4:45:43,123456 | ||
1499368,86014324,2022-04-14 5:30:45,567891 | ||
1499468,86136486,2022-04-14 6:30:45,101112 | ||
1499469,86139887,2022-04-14 7:45:44,123456 | ||
1499469,86142279,2022-04-14 8:30:44,101112 | ||
1499596,86324169,2022-04-16 5:00:42,123456 | ||
1499595,86318106,2022-04-16 4:45:44,567891 | ||
1499597,86328114,2022-04-16 6:45:43,101112 | ||
1499597,86325643,2022-04-16 6:45:43,567891 | ||
1499598,86326974,2022-04-16 8:00:44,123456 | ||
contact_id,deal_id,_fivetran_synced,type_id,category | ||
1499466,86133127,2022-04-14 4:30:42,101112,HUBSPOT_DEFINED | ||
1499466,86134193,2022-04-14 4:45:43,123456,HUBSPOT_DEFINED | ||
1499368,86014324,2022-04-14 5:30:45,567891,HUBSPOT_DEFINED | ||
1499468,86136486,2022-04-14 6:30:45,101112,HUBSPOT_DEFINED | ||
1499469,86139887,2022-04-14 7:45:44,123456,HUBSPOT_DEFINED | ||
1499469,86142279,2022-04-14 8:30:44,101112,HUBSPOT_DEFINED | ||
1499596,86324169,2022-04-16 5:00:42,123456,HUBSPOT_DEFINED | ||
1499595,86318106,2022-04-16 4:45:44,567891,HUBSPOT_DEFINED | ||
1499597,86328114,2022-04-16 6:45:43,101112,HUBSPOT_DEFINED | ||
1499597,86325643,2022-04-16 6:45:43,567891,HUBSPOT_DEFINED | ||
1499598,86326974,2022-04-16 8:00:44,123456,HUBSPOT_DEFINED |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,2 @@ | ||
company_id,engagement_id,engagement_type,type_id,_fivetran_synced | ||
9991774791,31479928586,TASK,192,2023-06-08 23:22:50.829000 | ||
company_id,engagement_id,engagement_type,type_id,_fivetran_synced,category | ||
9991774791,31479928586,TASK,192,2023-06-08 23:22:50.829000,HUBSPOT_DEFINED |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,2 @@ | ||
contact_id,engagement_id,engagement_type,type_id,_fivetran_synced | ||
501,19732910159,CALL,194,2023-06-08 23:22:49.869000 | ||
contact_id,engagement_id,engagement_type,type_id,_fivetran_synced,category | ||
501,19732910159,CALL,194,2023-06-08 23:22:49.869000,HUBSPOT_DEFINED |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,2 @@ | ||
deal_id,engagement_id,engagement_type,type_id,_fivetran_synced | ||
10828857779,31479928586,TASK,216,2023-06-08 23:22:50.768000 | ||
deal_id,engagement_id,engagement_type,type_id,_fivetran_synced,category | ||
10828857779,31479928586,TASK,216,2023-06-08 23:22:50.768000,HUBSPOT_DEFINED |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,10 @@ | ||
_fivetran_synced,ticket_id,company_id | ||
2020-07-09 11:06:21.056,1,1 | ||
2020-07-09 11:06:21.056,2,1 | ||
2020-07-09 11:06:21.056,3,1 | ||
2020-07-09 11:06:21.056,4,1 | ||
2020-07-09 11:06:21.056,5,1 | ||
2020-07-09 11:06:21.056,6,1 | ||
2020-07-09 11:06:21.056,7,2 | ||
2020-07-09 11:06:21.056,8,2 | ||
2020-07-09 11:06:21.056,9,2 | ||
_fivetran_synced,ticket_id,company_id,category | ||
2020-07-09 11:06:21.056,1,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,2,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,3,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,4,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,5,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,6,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,7,2,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,8,2,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,9,2,HUBSPOT_DEFINED |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,10 @@ | ||
_fivetran_synced,ticket_id,contact_id | ||
2020-07-09 11:06:21.056,1,1 | ||
2020-07-09 11:06:21.056,2,2 | ||
2020-07-09 11:06:21.056,3,3 | ||
2020-07-09 11:06:21.056,4,1 | ||
2020-07-09 11:06:21.056,5,2 | ||
2020-07-09 11:06:21.056,6,3 | ||
2020-07-09 11:06:21.056,7,1 | ||
2020-07-09 11:06:21.056,8,2 | ||
2020-07-09 11:06:21.056,9,3 | ||
_fivetran_synced,ticket_id,contact_id,category | ||
2020-07-09 11:06:21.056,1,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,2,2,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,3,3,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,4,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,5,2,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,6,3,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,7,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,8,2,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,9,3,HUBSPOT_DEFINED |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,10 @@ | ||
_fivetran_synced,ticket_id,deal_id | ||
2020-07-09 11:06:21.056,1,1 | ||
2020-07-09 11:06:21.056,2,2 | ||
2020-07-09 11:06:21.056,3,3 | ||
2020-07-09 11:06:21.056,4,1 | ||
2020-07-09 11:06:21.056,5,2 | ||
2020-07-09 11:06:21.056,6,3 | ||
2020-07-09 11:06:21.056,7,1 | ||
2020-07-09 11:06:21.056,8,2 | ||
2020-07-09 11:06:21.056,9,3 | ||
_fivetran_synced,ticket_id,deal_id,category | ||
2020-07-09 11:06:21.056,1,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,2,2,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,3,3,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,4,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,5,2,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,6,3,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,7,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,8,2,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,9,3,HUBSPOT_DEFINED |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,10 @@ | ||
_fivetran_synced,ticket_id,engagement_id | ||
2020-07-09 11:06:21.056,1,1 | ||
2020-07-09 11:06:21.056,2,2 | ||
2020-07-09 11:06:21.056,3,3 | ||
2020-07-09 11:06:21.056,4,14 | ||
2020-07-09 11:06:21.056,5,5 | ||
2020-07-09 11:06:21.056,6,12 | ||
2020-07-09 11:06:21.056,7,1 | ||
2020-07-09 11:06:21.056,8,2 | ||
2020-07-09 11:06:21.056,9,3 | ||
_fivetran_synced,ticket_id,engagement_id,category | ||
2020-07-09 11:06:21.056,1,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,2,2,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,3,3,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,4,14,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,5,5,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,6,12,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,7,1,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,8,2,HUBSPOT_DEFINED | ||
2020-07-09 11:06:21.056,9,3,HUBSPOT_DEFINED |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you bump the README version up to [">=0.17.0", "<0.18.0"]?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thank you!