Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Combine TransactionsGraphql and Transactions streams into a single Transactions stream in the catalog #34546

Conversation

girarda
Copy link
Contributor

@girarda girarda commented Jan 26, 2024

What

  • Combine Transactions and TransactionsGraphql under a single Stream from the user's perspective
  • This should smooth out the migration a bit by letting the users only care about what matters. they don't need to decide whether to use the bulk or rest api - only whether they care about the user_id field

How

  • Modify select_transactions_stream to return the right implementation for the transactions stream instead of filtering them from the list of streams
  • Add a CAT with the new config field. A new test config was added to google secret manager

Recommended reading order

  1. airbyte-integrations/connectors/source-shopify/source_shopify/source.py
  2. airbyte-integrations/connectors/source-shopify/source_shopify/streams/streams.py
  3. airbyte-integrations/connectors/source-shopify/unit_tests/test_source.py
  4. airbyte-integrations/connectors/source-shopify/integration_tests/configured_catalog.json
  5. airbyte-integrations/connectors/source-shopify/acceptance-test-config.yml

Copy link

vercel bot commented Jan 26, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 29, 2024 7:04pm

@octavia-squidington-iii octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/shopify labels Jan 26, 2024
Copy link
Contributor

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

  • PR name follows PR naming conventions
  • Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
  • Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
  • You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
  • Secrets in the connector's spec are annotated with airbyte_secret
  • All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
  • Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
  • Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
  • If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

  1. Check for hidden checklists in your PR description

  2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

@girarda girarda changed the base branch from master to baz/source-shopify/fix-substream-state-filterring January 26, 2024 04:02
@girarda girarda marked this pull request as ready for review January 26, 2024 04:02
@octavia-squidington-iv octavia-squidington-iv requested review from a team January 26, 2024 04:04
@octavia-squidington-iii octavia-squidington-iii removed the area/documentation Improvements or additions to documentation label Jan 26, 2024
@girarda girarda requested a review from bazarnov January 27, 2024 19:14
@girarda girarda changed the title Alex/shopify bulk Combine TransactionsGraphql and Transactions streams into a single Transactions stream in the catalog Jan 27, 2024
Copy link
Contributor

@alafanechere alafanechere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving on behalf of connector ops the change to the acceptance-test-config.yml file.

@girarda
Copy link
Contributor Author

girarda commented Jan 29, 2024

closing because this was already added to #32345

@girarda girarda closed this Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants