Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-5880][CORE] Ignore fallback for ColumnarWriteFilesExec children #7113

Merged
merged 2 commits into from
Sep 5, 2024

Conversation

wForget
Copy link
Member

@wForget wForget commented Sep 4, 2024

What changes were proposed in this pull request?

As title

Fixes: #5880

How was this patch tested?

added unit test and manual test:

create table wz_test_20240904 (a int) STORED AS PARQUET;
insert into wz_test_20240904 values (1);

before:
img_v3_02ed_6783be03-2d39-449d-bdf5-a8efceb2b39g

after this:
img_v3_02ed_8daaf8be-e484-43a8-9f65-a9f22c0cb34g

@github-actions github-actions bot added the CORE works for Gluten Core label Sep 4, 2024
Copy link

github-actions bot commented Sep 4, 2024

#5880

@wForget wForget changed the title [GLUTEN-5880] Ignore fallback for VeloxColumnarWriteFilesExec children [GLUTEN-5880][CORE] Ignore fallback for VeloxColumnarWriteFilesExec children Sep 4, 2024
Copy link

github-actions bot commented Sep 4, 2024

Run Gluten Clickhouse CI

@wForget wForget changed the title [GLUTEN-5880][CORE] Ignore fallback for VeloxColumnarWriteFilesExec children [GLUTEN-5880][CORE] Ignore fallback for ColumnarWriteFilesExec children Sep 4, 2024
@wForget
Copy link
Member Author

wForget commented Sep 4, 2024

@Yohahaha @zhztheplayer could you please take a look?

checkWriteFilesAndGetChild(df2)
val fallbackSummary = GlutenImplicits
.collectQueryExecutionFallbackSummary(spark, df2.queryExecution)
assert(fallbackSummary.numFallbackNodes == 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would you port the test to Spark34 module?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, ported

Yohahaha
Yohahaha previously approved these changes Sep 4, 2024
Copy link
Contributor

@Yohahaha Yohahaha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! thank you

Copy link

github-actions bot commented Sep 4, 2024

Run Gluten Clickhouse CI

Copy link
Member

@zhztheplayer zhztheplayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@Yohahaha Yohahaha merged commit 8b5e3ce into apache:main Sep 5, 2024
43 checks passed
dcoliversun pushed a commit to dcoliversun/gluten that referenced this pull request Sep 11, 2024
sharkdtu pushed a commit to sharkdtu/gluten that referenced this pull request Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CORE works for Gluten Core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[VL] Noisy wrong fallback message after case-class refactor
3 participants