Skip to content

Commit

Permalink
fix match
Browse files Browse the repository at this point in the history
  • Loading branch information
@zwangsheng
zwangsheng committed Feb 23, 2024
1 parent 6895da9 commit cb9100f
Show file tree
Hide file tree
Showing 13 changed files with 522 additions and 572 deletions.
91 changes: 43 additions & 48 deletions backends-velox/src/test/resources/tpch-approved-plan/gluten-bhj-vanilla-be/spark322/1.txt
View file

Large diffs are not rendered by default.

131 changes: 63 additions & 68 deletions backends-velox/src/test/resources/tpch-approved-plan/gluten-bhj-vanilla-be/spark322/12.txt
View file
Original file line number Diff line number Diff line change
@@ -1,47 +1,46 @@
== Physical Plan ==
AdaptiveSparkPlan (45)
AdaptiveSparkPlan (44)
+- == Final Plan ==
VeloxColumnarToRowExec (31)
+- ^ SortExecTransformer (29)
+- ^ InputIteratorTransformer (28)
+- ^ InputAdapter (27)
+- ^ ShuffleQueryStage (26)
+- ColumnarExchange (25)
+- ^ ProjectExecTransformer (23)
+- ^ RegularHashAggregateExecTransformer (22)
+- ^ InputIteratorTransformer (21)
+- ^ InputAdapter (20)
+- ^ ShuffleQueryStage (19)
+- ColumnarExchange (18)
+- ^ ProjectExecTransformer (16)
+- ^ FlushableHashAggregateExecTransformer (15)
+- ^ ProjectExecTransformer (14)
+- ^ GlutenBroadcastHashJoinExecTransformer Inner (13)
:- ^ InputIteratorTransformer (9)
: +- ^ InputAdapter (8)
: +- ^ RowToVeloxColumnar (7)
: +- ^ BroadcastQueryStage (6)
: +- BroadcastExchange (5)
: +- VeloxColumnarToRowExec (4)
: +- ^ FilterExecTransformer (2)
: +- ^ Scan parquet (1)
+- ^ ProjectExecTransformer (12)
+- ^ FilterExecTransformer (11)
+- ^ Scan parquet (10)
VeloxColumnarToRowExec (30)
+- ^ SortExecTransformer (28)
+- ^ InputIteratorTransformer (27)
+- ^ InputAdapter (26)
+- ^ ShuffleQueryStage (25)
+- ColumnarExchange (24)
+- ^ RegularHashAggregateExecTransformer (22)
+- ^ InputIteratorTransformer (21)
+- ^ InputAdapter (20)
+- ^ ShuffleQueryStage (19)
+- ColumnarExchange (18)
+- ^ ProjectExecTransformer (16)
+- ^ FlushableHashAggregateExecTransformer (15)
+- ^ ProjectExecTransformer (14)
+- ^ GlutenBroadcastHashJoinExecTransformer Inner (13)
:- ^ InputIteratorTransformer (9)
: +- ^ InputAdapter (8)
: +- ^ RowToVeloxColumnar (7)
: +- ^ BroadcastQueryStage (6)
: +- BroadcastExchange (5)
: +- VeloxColumnarToRowExec (4)
: +- ^ FilterExecTransformer (2)
: +- ^ Scan parquet (1)
+- ^ ProjectExecTransformer (12)
+- ^ FilterExecTransformer (11)
+- ^ Scan parquet (10)
+- == Initial Plan ==
Sort (44)
+- Exchange (43)
+- HashAggregate (42)
+- Exchange (41)
+- HashAggregate (40)
+- Project (39)
+- BroadcastHashJoin Inner BuildLeft (38)
:- BroadcastExchange (34)
: +- Filter (33)
: +- Scan parquet (32)
+- Project (37)
+- Filter (36)
+- Scan parquet (35)
Sort (43)
+- Exchange (42)
+- HashAggregate (41)
+- Exchange (40)
+- HashAggregate (39)
+- Project (38)
+- BroadcastHashJoin Inner BuildLeft (37)
:- BroadcastExchange (33)
: +- Filter (32)
: +- Scan parquet (31)
+- Project (36)
+- Filter (35)
+- Scan parquet (34)


(1) Scan parquet
Expand Down Expand Up @@ -137,106 +136,102 @@ Input [3]: [l_shipmode#X, sum#X, sum#X]
Keys [1]: [l_shipmode#X]
Functions [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END), sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)]
Aggregate Attributes [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X]
Results [3]: [l_shipmode#X, sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X]

(23) ProjectExecTransformer
Output [3]: [l_shipmode#X, sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS high_line_count#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS low_line_count#X]
Input [3]: [l_shipmode#X, sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X]
Results [3]: [l_shipmode#X, sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS high_line_count#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS low_line_count#X]

(24) WholeStageCodegenTransformer (X)
(23) WholeStageCodegenTransformer (X)
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: false

(25) ColumnarExchange
(24) ColumnarExchange
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: rangepartitioning(l_shipmode#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]

(26) ShuffleQueryStage
(25) ShuffleQueryStage
Output [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: 2

(27) InputAdapter
(26) InputAdapter
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]

(28) InputIteratorTransformer
(27) InputIteratorTransformer
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]

(29) SortExecTransformer
(28) SortExecTransformer
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: [l_shipmode#X ASC NULLS FIRST], true, 0

(30) WholeStageCodegenTransformer (X)
(29) WholeStageCodegenTransformer (X)
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: false

(31) VeloxColumnarToRowExec
(30) VeloxColumnarToRowExec
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]

(32) Scan parquet
(31) Scan parquet
Output [2]: [o_orderkey#X, o_orderpriority#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderkey)]
ReadSchema: struct<o_orderkey:bigint,o_orderpriority:string>

(33) Filter
(32) Filter
Input [2]: [o_orderkey#X, o_orderpriority#X]
Condition : isnotnull(o_orderkey#X)

(34) BroadcastExchange
(33) BroadcastExchange
Input [2]: [o_orderkey#X, o_orderpriority#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]

(35) Scan parquet
(34) Scan parquet
Output [5]: [l_orderkey#X, l_shipdate#X, l_commitdate#X, l_receiptdate#X, l_shipmode#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_commitdate), IsNotNull(l_receiptdate), IsNotNull(l_shipdate), In(l_shipmode, [MAIL,SHIP]), GreaterThanOrEqual(l_receiptdate,1994-01-01), LessThan(l_receiptdate,1995-01-01), IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_shipdate:date,l_commitdate:date,l_receiptdate:date,l_shipmode:string>

(36) Filter
(35) Filter
Input [5]: [l_orderkey#X, l_shipdate#X, l_commitdate#X, l_receiptdate#X, l_shipmode#X]
Condition : ((((((((isnotnull(l_commitdate#X) AND isnotnull(l_receiptdate#X)) AND isnotnull(l_shipdate#X)) AND l_shipmode#X IN (MAIL,SHIP)) AND (l_commitdate#X < l_receiptdate#X)) AND (l_shipdate#X < l_commitdate#X)) AND (l_receiptdate#X >= 1994-01-01)) AND (l_receiptdate#X < 1995-01-01)) AND isnotnull(l_orderkey#X))

(37) Project
(36) Project
Output [2]: [l_orderkey#X, l_shipmode#X]
Input [5]: [l_orderkey#X, l_shipdate#X, l_commitdate#X, l_receiptdate#X, l_shipmode#X]

(38) BroadcastHashJoin
(37) BroadcastHashJoin
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None

(39) Project
(38) Project
Output [2]: [o_orderpriority#X, l_shipmode#X]
Input [4]: [o_orderkey#X, o_orderpriority#X, l_orderkey#X, l_shipmode#X]

(40) HashAggregate
(39) HashAggregate
Input [2]: [o_orderpriority#X, l_shipmode#X]
Keys [1]: [l_shipmode#X]
Functions [2]: [partial_sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END), partial_sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)]
Aggregate Attributes [2]: [sum#X, sum#X]
Results [3]: [l_shipmode#X, sum#X, sum#X]

(41) Exchange
(40) Exchange
Input [3]: [l_shipmode#X, sum#X, sum#X]
Arguments: hashpartitioning(l_shipmode#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]

(42) HashAggregate
(41) HashAggregate
Input [3]: [l_shipmode#X, sum#X, sum#X]
Keys [1]: [l_shipmode#X]
Functions [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END), sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)]
Aggregate Attributes [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X]
Results [3]: [l_shipmode#X, sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS high_line_count#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS low_line_count#X]

(43) Exchange
(42) Exchange
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: rangepartitioning(l_shipmode#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]

(44) Sort
(43) Sort
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: [l_shipmode#X ASC NULLS FIRST], true, 0

(45) AdaptiveSparkPlan
(44) AdaptiveSparkPlan
Output [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: isFinalPlan=true
Loading

0 comments on commit cb9100f

Please sign in to comment.