Skip to content

Commit

Permalink
Feat: Add overwrite_dedup destination sync mode
Browse files Browse the repository at this point in the history
  • Loading branch information
evantahler committed Jun 24, 2024
1 parent 8ff163c commit fd8b3a3
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 8 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -524,22 +524,22 @@ definitions:
items:
type: string
generation_id:
description:
description:
"Monotically increasing numeric id representing the current generation of a stream. This id can be shared across syncs.
If this is null, it means that the platform is not supporting the refresh and it is expected that no extra id will be added to the records and no data from previous generation will be cleanup.
"
type: integer
minimum_generation_id:
description:
description:
"The minimum generation id which is needed in a stream. If it is present, the destination will try to delete the data that are part of a generation lower than this property.
If the minimum generation is equals to 0, no data deletion is expected from the destiantion
If this is null, it means that the platform is not supporting the refresh and it is expected that no extra id will be added to the records and no data from previous generation will be cleanup.
"
type: integer
sync_id:
description:
description:
"Monotically increasing numeric id representing the current sync id. This is aimed to be unique per sync.
If this is null, it means that the platform is not supporting the refresh and it is expected that no extra id will be added to the records and no data from previous generation will be cleanup.
Expand All @@ -557,6 +557,7 @@ definitions:
- overwrite
#- upsert_dedup # TODO chris: SCD Type 1 can be implemented later
- append_dedup # SCD Type 1 & 2
- overwrite_dedup # thanks, resumable full refresh
ConnectorSpecification:
type: object
additionalProperties: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -525,24 +525,24 @@ definitions:
items:
type: string
generation_id:
description:
description:
"Monotically increasing numeric id representing the current generation of a stream. This id can be shared across syncs.
If this is null, it means that the platform is not supporting the refresh and it is expected that no extra id will be added to the records and no data from previous generation will be cleanup.
"
type: integer
minimum_generation_id:
description:
description:
"The minimum generation id which is needed in a stream. If it is present, the destination will try to delete the data that are part of a generation lower than this property.
If the minimum generation is equals to 0, no data deletion is expected from the destiantion
If this is null, it means that the platform is not supporting the refresh and it is expected that no extra id will be added to the records and no data from previous generation will be cleanup.
"
type: integer
sync_id:
description:
description:
"Monotically increasing numeric id representing the current sync id. This is aimed to be unique per sync.
If this is null, it means that the platform is not supporting the refresh and it is expected that no extra id will be added to the records and no data from previous generation will be cleanup.
"
type: integer
Expand All @@ -558,6 +558,7 @@ definitions:
- overwrite
#- upsert_dedup # TODO chris: SCD Type 1 can be implemented later
- append_dedup # SCD Type 1 & 2
- overwrite_dedup # thanks, resumable full refresh
OAuth2Specification:
type: object
additionalProperties: true
Expand Down

0 comments on commit fd8b3a3

Please sign in to comment.