Skip to content

Commit

Permalink
Declarative CDK: Fix None error on stream_slice (#35879)
Browse files Browse the repository at this point in the history
  • Loading branch information
maxi297 authored Mar 7, 2024
1 parent 88314dd commit 858e61d
Showing 1 changed file with 7 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,13 @@ def read_records(
"""
:param: stream_state We knowingly avoid using stream_state as we want cursors to manage their own state.
"""
if stream_slice is None:
# As the parameter is Optional, many would just call `read_records(sync_mode)` during testing without specifying the field
# As part of the declarative model without custom components, this should never happen as the CDK would wire up a
# SinglePartitionRouter that would create this StreamSlice properly
# As part of the declarative model with custom components, a user that would return a `None` slice would now have the default
# empty slice which seems to make sense.
stream_slice = StreamSlice(partition={}, cursor_slice={})
if not isinstance(stream_slice, StreamSlice):
raise ValueError(f"DeclarativeStream does not support stream_slices that are not StreamSlice. Got {stream_slice}")
yield from self.retriever.read_records(self.get_json_schema(), stream_slice)
Expand Down

0 comments on commit 858e61d

Please sign in to comment.