Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to apply partition/bloom filter to old data? Does rewrite_data_files/rewrite_manifests procedure work? #11878

Open
madeirak opened this issue Dec 27, 2024 · 3 comments
Labels
question Further information is requested

Comments

@madeirak
Copy link

Query engine

Spark

Question

rewrite_data_files/rewrite_manifests procedure DOC:
https://iceberg.apache.org/docs/latest/spark-procedures/#rewrite_manifests

How to apply data written before adding new configuration of partition/ bloom filter??

@madeirak madeirak added the question Further information is requested label Dec 27, 2024
@hashmapybx
Copy link

by the way, ALTER TABLE prod.db.sample SET TBLPROPERTIES . Do you meet any other problems?

@madeirak
Copy link
Author

madeirak commented Jan 6, 2025

by the way, ALTER TABLE prod.db.sample SET TBLPROPERTIES . Do you meet any other problems?

After adding the bloom filter related table properties, the data written will have the bloom filter related metadata created. So how can the data written before adding the table properties also have these bloom filter related metadata created?

@LoseYSelf
Copy link

may be you can insert overwrite with the old data. @madeirak

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants