Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(source-amazon-ads): set is_resumable only for Fullrefresh streams #48343

Merged
merged 8 commits into from
Nov 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ data:
connectorSubtype: api
connectorType: source
definitionId: c6b0a29e-1da9-4512-9002-7bfd0cba2246
dockerImageTag: 6.1.2
dockerImageTag: 6.1.3
dockerRepository: airbyte/source-amazon-ads
documentationUrl: https://docs.airbyte.com/integrations/sources/amazon-ads
githubIssueLabel: source-amazon-ads
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = [ "poetry-core>=1.0.0",]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
version = "6.1.2"
version = "6.1.3"
name = "source-amazon-ads"
description = "Source implementation for Amazon Ads."
authors = [ "Airbyte <[email protected]>",]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ class AttributionReport(AmazonAdsStream):
https://advertising.amazon.com/API/docs/en-us/amazon-attribution-prod-3p/#/
"""

is_resumable = False
primary_key = None
data_field = "reports"
page_size = 300
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,6 @@ class BasicAmazonAdsStream(Stream, ABC):
Base class for all Amazon Ads streams.
"""

is_resumable = False

def __init__(self, config: Mapping[str, Any], profiles: List[dict[str, Any]] = None):
self._profiles = profiles or []
self._client_id = config["client_id"]
Expand Down Expand Up @@ -183,6 +181,7 @@ class SubProfilesStream(AmazonAdsStream):
Stream for getting resources with pagination support and getting resources based on list of profiles set by source.
"""

is_resumable = False
page_size = 100

def __init__(self, *args, **kwargs):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ class Portfolios(AmazonAdsStream):
https://advertising.amazon.com/API/docs/en-us/reference/2/portfolios
"""

is_resumable = False
primary_key = "portfolioId"

def path(self, **kwargs) -> str:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ class Profiles(AmazonAdsStream):
https://advertising.amazon.com/API/docs/en-us/reference/2/profiles#/Profiles
"""

is_resumable = False
primary_key = "profileId"

def path(self, **kwargs) -> str:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -307,9 +307,6 @@ def state(self):
def state(self, value):
self._state = deepcopy(value)

def get_updated_state(self, current_stream_state: Dict[str, Any], latest_data: Mapping[str, Any]) -> Mapping[str, Any]:
return self._state

def _update_state(self, profile: dict[str, Any], report_date: str):
report_date = pendulum.from_format(report_date, self.REPORT_DATE_FORMAT).date()
look_back_date = pendulum.today(tz=profile["timezone"]).date().subtract(days=self._look_back_window - 1)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

import pendulum
import requests_mock
from airbyte_cdk.models import AirbyteStateBlob
from airbyte_cdk.models import Level as LogLevel
from airbyte_cdk.models import SyncMode
from airbyte_cdk.test.mock_http import HttpMocker, HttpRequestMatcher
Expand Down Expand Up @@ -211,6 +212,7 @@ def test_given_file_when_read_brands_v3_report_then_return_records(self, http_mo
)

output = read_stream("sponsored_brands_v3_report_stream", SyncMode.full_refresh, self._config)
assert output.most_recent_state.stream_state == AirbyteStateBlob({"1":{'reportDate': '2024-11-06'}})
assert len(output.records) == 1

@HttpMocker()
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/amazon-ads.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@ Information about expected report generation waiting time can be found [here](ht

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------|
| 6.1.3 | 2024-11-05 | [48343](https://github.com/airbytehq/airbyte/pull/48343) | Set is_resumable only for FullRefresh streams |
| 6.1.2 | 2024-11-04 | [48138](https://github.com/airbytehq/airbyte/pull/48138) | Add error message for TooManyRequests exception |
| 6.1.1 | 2024-11-04 | [48128](https://github.com/airbytehq/airbyte/pull/48128) | Fix date parse in report streams |
| 6.1.0 | 2024-11-01 | [47940](https://github.com/airbytehq/airbyte/pull/47940) | Bump CDK to ^5 |
Expand Down
Loading