Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Treat streams with LOG_BASED replication as unsorted #301

Merged
merged 1 commit into from
Nov 30, 2023
Merged

fix: Treat streams with LOG_BASED replication as unsorted #301

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions tap_postgres/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

This includes PostgresStream and PostgresConnector.
"""

from __future__ import annotations

import datetime
Expand All @@ -20,6 +21,7 @@
from singer_sdk import typing as th
from singer_sdk.helpers._state import increment_state
from singer_sdk.helpers._typing import TypeConformanceLevel
from singer_sdk.streams.core import REPLICATION_INCREMENTAL
from sqlalchemy import nullsfirst
from sqlalchemy.engine import Engine
from sqlalchemy.engine.reflection import Inspector
Expand Down Expand Up @@ -481,3 +483,10 @@ def logical_replication_connection(self):
application_name="tap_postgres",
connection_factory=extras.LogicalReplicationConnection,
)

# TODO: Make this change upstream in the SDK?
# I'm not sure if in general SQL databases don't guarantee order of records log
# replication, but at least Postgres does not.
def is_sorted(self) -> bool:
"""Return True if the stream is sorted by the replication key."""
return self.replication_method == REPLICATION_INCREMENTAL