Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Support Vector data types for macro snowflake__get_columns_in_relation #1098

Open
2 tasks done
chamilton61 opened this issue Jun 28, 2024 · 2 comments · May be fixed by #1169
Open
2 tasks done

[Bug] Support Vector data types for macro snowflake__get_columns_in_relation #1098

chamilton61 opened this issue Jun 28, 2024 · 2 comments · May be fixed by #1169
Labels
enhancement New feature or request triage

Comments

@chamilton61
Copy link

Is this a new bug in dbt-snowflake?

  • I believe this is a new bug in dbt-snowflake
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When trying to run an incremental load referencing a table that has a column that is cast as a VECTOR data type, I get the following error: Could not interpret data_type "VECTOR(FLOAT, 1536)": could not convert "FLOAT" to an integer

This appears to be due to the from_description Column function here expecting only integers within parenthesis.

Expected Behavior

Expected column data type to be properly parsed and reproduced in the incremental table.

Steps To Reproduce

do dbt run on a model that has an incremental load materialization and has a column that is the VECTOR data type

Relevant log output

15:40:11  Snowflake adapter: Error running SQL: macro get_columns_in_relation
15:40:11  Snowflake adapter: Rolling back transaction.
15:40:11  On <model name>: Close
15:40:11  Runtime Error in model <model name> (<model name>.sql)
  Could not interpret data_type "VECTOR(FLOAT, 1536)": could not convert "FLOAT" to an integer

Environment

- OS: Ubuntu 22.04.4 LTS
- Python:  3.11.7
- dbt-core: 1.8.3
- dbt-snowflake: 1.8.3

Additional Context

suspected cause of the issue:
https://github.com/dbt-labs/dbt-snowflake/blob/2576a08379e248d393b9a698e58fac0d32c23648/dbt/include/snowflake/macros/adapters.sql
https://github.com/dbt-labs/dbt-adapters/blob/main/dbt/adapters/base/column.py#L130

@chamilton61 chamilton61 added bug Something isn't working triage labels Jun 28, 2024
@achawkins achawkins linked a pull request Aug 20, 2024 that will close this issue
4 tasks
@amychen1776 amychen1776 changed the title [Bug] macro snowflake__get_columns_in_relation does not handle VECTOR data types correctly [Bug] Support Vector data types for macro snowflake__get_columns_in_relation Aug 28, 2024
@amychen1776 amychen1776 added enhancement New feature or request and removed bug Something isn't working labels Aug 28, 2024
@cgarretty
Copy link

+1 - we're also being impacted by this.

@brandonpeebles
Copy link

+1 — also been dealing with this error trying to reference source iceberg tables that contain Array or Object data types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request triage
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants