Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] unit test input SQL does not escape column names #11057

Closed
2 tasks done
kdeggelman opened this issue Nov 26, 2024 · 1 comment
Closed
2 tasks done

[Bug] unit test input SQL does not escape column names #11057

kdeggelman opened this issue Nov 26, 2024 · 1 comment
Labels
bug Something isn't working duplicate This issue or pull request already exists

Comments

@kdeggelman
Copy link

kdeggelman commented Nov 26, 2024

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

The CTE that contains the input data seems to follow this format select safe_cast(<value> as <data type>) as <column name>,

For example,

safe_cast('''compact''' as STRING) as group

Expected Behavior

The CTE should escape the column names. For example, (in BigQuery):

safe_cast('''compact''' as STRING) as `group`

Steps To Reproduce

Running with dbt=1.8.8
Registered adapter: bigquery=1.8.3
  1. Create a source dataset/table that uses reserved words as a column name:
create or replace table vehicles.cars as (
  select 1 as id, 'compact' as `group`
)

models/src_vehicles.yaml

sources:
  - name: vehicles
    tables:
      - name: cars
        columns:
          - name: id
          - name: group
  1. Create a model that utilizes the source:

models/cars_by_group.sql

with cars as (
    select * from {{ source('vehicles', 'cars') }}
),

count_by_group as (
    select
        group,

        count(*) as count,
    from cars
)

select * from count_by_group
  1. Create a unit test for the model:

models/_properties.yml

unit_tests:
  - name: test_count_of_cars_by_group
    model: cars_by_group
    given:
      - input: source('vehicles', 'cars')
        rows:
          - { id: 1, group: 'compact' }
          - { id: 2, group: 'compact' }
          - { id: 3, group: 'midsize' }
          - { id: 4, group: 'suv' }
          - { id: 5, group: 'suv' }
          - { id: 6, group: 'suv' }
    expect:
      rows:
        - { group: 'compact', count: 2 }
        - { group: 'midsize', count: 1 }
        - { group: 'suv', count: 3 }
  1. Run the test
    dbt test -s test_count_of_cars_by_group

  2. Inspect the SQL

select * from (
        with  __dbt__cte__cars as (

-- Fixture for cars
select safe_cast(1 as INT64) as id, safe_cast('''compact''' as STRING) as group
union all
select safe_cast(2 as INT64) as id, safe_cast('''compact''' as STRING) as group
union all
select safe_cast(3 as INT64) as id, safe_cast('''midsize''' as STRING) as group
union all
select safe_cast(4 as INT64) as id, safe_cast('''suv''' as STRING) as group
union all
select safe_cast(5 as INT64) as id, safe_cast('''suv''' as STRING) as group
union all
select safe_cast(6 as INT64) as id, safe_cast('''suv''' as STRING) as group
), cars as (
    select * from __dbt__cte__cars
),

Relevant log output

21:38:54    Runtime Error in unit_test test_count_of_cars_by_group (models/etl/splits_and_stats/splits_and_stats.yml)
  An error occurred during execution of unit test 'test_count_of_cars_by_group'. There may be an error in the unit test definition: check the data types.
   Database Error
    Syntax error: Unexpected keyword GROUP at [20:75]

Environment

- OS: MacOS 15
- Python: 3.11.9
- dbt: 1.8.8
- dbt-bigquery: 1.8.3

Which database adapter are you using with dbt?

bigquery

Additional Context

Might be similar to:

@kdeggelman kdeggelman added bug Something isn't working triage labels Nov 26, 2024
@kdeggelman
Copy link
Author

After doing some more digging, I found that this is a duplicate of dbt-labs/dbt-adapters#205

@dbeatty10 dbeatty10 added duplicate This issue or pull request already exists and removed triage labels Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

2 participants