Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ttl not added to local table in distributed_incremental materialization #321

Open
dl-github-reporter opened this issue Jul 18, 2024 · 0 comments · May be fixed by #430
Open

ttl not added to local table in distributed_incremental materialization #321

dl-github-reporter opened this issue Jul 18, 2024 · 0 comments · May be fixed by #430
Labels
bug Something isn't working

Comments

@dl-github-reporter
Copy link

Describe the bug

Tried to materialize table with distributed_incremental materialization.
TTL setting doesn't exist for local table.

Steps to reproduce

  1. Add model test_model with config. Run dbt build
    {{ config(
     materialized='distributed_incremental',
     engine='ReplicatedReplacingMergeTree(datetime)',
     incremental_strategy='append',
     order_by='datetime, object_id',
     partition_by='toYYYYMM(datetime)',
     ttl='toStartOfYear(datetime + toIntervalYear(5))',
     sharding_key='cityHash64(object_id)'
    ) }}
     
    SELECT * FROM {{ source('default', 'stg_test_table') }}
  2. Check local table with
SHOW CREATE default.test_model_local
  1. Engine will be
ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}', datetime)
PARTITION BY toYYYYMM(datetime)
ORDER BY (datetime, object_id)
SETTINGS index_granularity = 8192

Expected behaviour

Engine of local table should contain TTL setting

ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}', datetime)
PARTITION BY toYYYYMM(datetime)
ORDER BY (datetime, object_id)
TTL toStartOfYear(datetime + toIntervalYear(5))
SETTINGS index_granularity = 8192

Code examples, such as models or profile settings

version: 2

sources:
  - name: default
    tables:
      - name: stg_test_table

dbt and/or ClickHouse server logs

Configuration

Environment

  • dbt version: 1.7.15
  • dbt-clickhouse version: 1.7.7
  • clickhouse-driver version (if using native): 0.2.6
  • clickhouse-connect version (if using http): 0.7.12
  • Python version: 3.10
  • Operating system: python:3.10-slim docker image

ClickHouse server

  • ClickHouse Server version: 24.3.4.147
  • CREATE TABLE statements for tables involved:
CREATE TABLE default.stg_test_table ON CLUSTER '{cluster}'
(
  object_id String,
  datetime DateTime,
  field1 String,
  field2 String
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}')
ORDER BY object_id;
@dl-github-reporter dl-github-reporter added the bug Something isn't working label Jul 18, 2024
arastopchin1 pushed a commit to arastopchin1/dbt-clickhouse that referenced this issue Feb 26, 2025
arastopchin1 pushed a commit to arastopchin1/dbt-clickhouse that referenced this issue Feb 26, 2025
arastopchin1 added a commit to arastopchin1/dbt-clickhouse that referenced this issue Feb 26, 2025
@arastopchin1 arastopchin1 linked a pull request Feb 28, 2025 that will close this issue
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant