Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Make sure ColumnarSubqueryBroadcastExec compatible with fallen back vanilla exchange in AQE #6019

Closed

Conversation

zhztheplayer
Copy link
Member

@zhztheplayer zhztheplayer commented Jun 7, 2024

This is a RAS fix but also makes the code used by legacy planner more robust too.

For input plan:

ColumnarSubqueryBroadcast dynamicpruning#1601, 0, [d_date_sk#1519L], [id=#1441]
+- AdaptiveSparkPlan isFinalPlan=false
   +- BroadcastExchange HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=1413]
      +- Project [d_date_sk#1519L]
         +- Filter ((((isnotnull(d_year#1525L) AND isnotnull(d_moy#1527L)) AND (d_year#1525L = 1999)) AND (d_moy#1527L = 1)) AND isnotnull(d_date_sk#1519L))
            +- FileScan parquet spark_catalog.default.date_dim[d_date_sk#1519L,d_year#1525L,d_moy#1527L] Batched: true, DataFilters: [isnotnull(d_year#1525L), isnotnull(d_moy#1527L), (d_year#1525L = 1999), (d_moy#1527L = 1), isnot..., Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/tmp/tpcds-generated-1000.0/date_dim], PartitionFilters: [], PushedFilters: [IsNotNull(d_year), IsNotNull(d_moy), EqualTo(d_year,1999), EqualTo(d_moy,1), IsNotNull(d_date_sk)], ReadSchema: struct<d_date_sk:bigint,d_year:bigint,d_moy:bigint>

When the project is not offloaded for some reason, RAS planner may decide to keep BroadcastExchange as a row plan to avoid transitions. In that case, ColumnarSubqueryBroadcast will fail execution.

This patch fixes the issue.

@apache apache deleted a comment from github-actions bot Jun 7, 2024
@apache apache deleted a comment from github-actions bot Jun 7, 2024
Copy link

github-actions bot commented Jun 7, 2024

Run Gluten Clickhouse CI

@zhztheplayer zhztheplayer changed the title [VL] Make sure ColumnarSubqueryBroadcastExec's child outputs columnar data [VL] Make sure ColumnarSubqueryBroadcastExec compatible with fallen back vanilla exchange Jun 7, 2024
Copy link

github-actions bot commented Jun 7, 2024

Run Gluten Clickhouse CI

@zhztheplayer zhztheplayer changed the title [VL] Make sure ColumnarSubqueryBroadcastExec compatible with fallen back vanilla exchange [VL] Make sure ColumnarSubqueryBroadcastExec compatible with fallen back vanilla exchange in AQE Jun 7, 2024
@apache apache deleted a comment from github-actions bot Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant