Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

alias column for snowflake external table #257

Merged
merged 6 commits into from
Apr 11, 2024

Conversation

cakkinep
Copy link
Contributor

@cakkinep cakkinep commented Feb 26, 2024

Description & motivation

resolves: #213
These changes will specifically solve the issue with snowflake tables where the underlying file format or table format has a column named VALUE ( error below) . In general this change will allow aliasing column name for any reason.

Encountered an error while running operation: Database Error
091311 (42601): SQL compilation error:
'VALUE' cannot be used as a column name for external tables.

More about the changes:

  • check if alias exists for a column, if it exists it will use alias for column name and reference original column name in the file/table format.

Checklist

  • I have verified that these changes work locally

  • I have updated the README.md (if applicable)

  • I have added an integration test for my fix/feature (if applicable)

@cakkinep cakkinep requested a review from jeremyyeo as a code owner February 26, 2024 17:00
@cakkinep
Copy link
Contributor Author

@jeremyyeo , the CI seems to be failing, it is not related to this PR

@wjhrdy
Copy link

wjhrdy commented Feb 27, 2024

This is useful. Thanks for the work on this.

@thomas-vl
Copy link
Contributor

@jeremyyeo , the CI seems to be failing, it is not related to this PR

CI fails because of a protobuf issue in DBT Core, I have created a PR with a temporary fix that will resolve this issue:
#258

@dataders dataders modified the milestone: 1.0.0 Apr 4, 2024
@@ -26,12 +26,14 @@
{%- endfor -%}{%- endif -%}
{%- if not infer_schema -%}
{%- for column in columns %}
{%- set column_alias = column.alias if column.alias else column.name %}
{%- set column_alias_quoted = adapter.quote(column_alias) if column.quote else column_alias %}
{%- set column_quoted = adapter.quote(column.name) if column.quote else column.name %}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this line needed or used anymore?

Suggested change
{%- set column_quoted = adapter.quote(column.name) if column.quote else column.name %}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is needed as the column_quoted is used in the expression part of the definition, we can't use column_alias_quoted in value: , hope that clarifies the need.

Copy link
Collaborator

@dataders dataders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cakkinep thanks for opening! Can you add a test case to integration_tests/models/plugins/snowflake/snowflake_external.yml that takes advantage of this aliasing option?

@cakkinep
Copy link
Contributor Author

cakkinep commented Apr 11, 2024

@dataders Added integration test to validate column alias.

Copy link
Collaborator

@dataders dataders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @cakkinep!

@dataders dataders merged commit a56a255 into dbt-labs:main Apr 11, 2024
3 checks passed
@cakkinep cakkinep mentioned this pull request Apr 11, 2024
3 tasks
dataders added a commit that referenced this pull request Apr 12, 2024
dataders added a commit that referenced this pull request Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ability to add column alias
4 participants