Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DynamoDB source connector refresh schema with latest fields #54730

Open
1 task done
jaimiestein opened this issue Feb 28, 2025 · 0 comments
Open
1 task done

DynamoDB source connector refresh schema with latest fields #54730

jaimiestein opened this issue Feb 28, 2025 · 0 comments

Comments

@jaimiestein
Copy link

Connector Name

source-dynamodb

Connector Version

0.3.9

What step the error happened?

Updating the connector

Relevant information

Since DynamoDB doesn't have a schema definition, it uses a scan to detect the fields. However it's not detecting new fields from the latest records in my data. I need a way to either detect new fields during a stream sync or have the refresh schema use query with ScanIndexForward=False instead

Relevant log output

Contribute

  • Yes, I want to contribute
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants