Feature/hgi 6721 #4
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Introduced a method to create temporary tables in Snowflake, allowing for better handling of dynamic queries. - Updated `get_table_columns` and `discover_catalog` functions to utilize the new temporary table creation method. - Enhanced type hinting for function parameters to improve code clarity. - Removed hardcoded SQL queries in favor of a dedicated method for fetching data types. - Added logic to drop temporary tables after use to prevent clutter in the database. - Adjusted logging and error handling for improved debugging. This refactor aims to streamline the interaction with Snowflake and enhance the overall maintainability of the codebase.
- Updated `discover_catalog` to accept a new parameter for controlling query result limits, enhancing flexibility. - Replaced the temporary table creation method with a more versatile `create_table` method that allows for optional query limits. - Adjusted the handling of temporary tables to ensure they are dropped after use, maintaining a clean database state. - Modified related functions (`do_discover`, `get_streams`, and `do_sync`) to accommodate changes in the catalog discovery process and temporary table management. These changes improve the efficiency and clarity of the Snowflake integration, ensuring better resource management and code maintainability.
butkeraites-hotglue
requested review from
hsyyid,
arilton,
keyn4 and
brenhogan
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.