Skip to content
This repository has been archived by the owner on Apr 11, 2024. It is now read-only.

Add support for Databricks Dataframe support #16

Open
9 tasks
sunank200 opened this issue Feb 8, 2023 · 0 comments
Open
9 tasks

Add support for Databricks Dataframe support #16

sunank200 opened this issue Feb 8, 2023 · 0 comments

Comments

@sunank200
Copy link
Collaborator

Please describe the feature you'd like to see

  • Add DataProvider for Databricks - read/write methods
  • Add non-native transfer implementation for GCS to Databricks
  • Add non-native transfer implementation for S3 to Databricks
  • Add non-native transfer example DAG for Sqlite to Databricks
  • Add non-native transfer example DAG for Snowflake to Databricks
  • Add example DAG
  • Add tests with 90% coverage

Acceptance Criteria

  • All checks and tests in the CI should pass
  • Unit tests (90% code coverage or more, once available)
  • Integration tests (if the feature relates to a new database or external service)
  • Example DAG
  • Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
  • Exception handling in case of errors
  • Logging (are we exposing useful information to the user? e.g. source and destination)
  • Improve the documentation (README, Sphinx, and any other relevant)
  • How to use Guide for the feature (example)
@sunank200 sunank200 transferred this issue from astronomer/astro-sdk Mar 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant