Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CH]mismatched headers in broadcast join #6257

Closed
lgbo-ustc opened this issue Jun 27, 2024 · 1 comment · Fixed by #6258
Closed

[CH]mismatched headers in broadcast join #6257

lgbo-ustc opened this issue Jun 27, 2024 · 1 comment · Fixed by #6258
Labels
bug Something isn't working triage

Comments

@lgbo-ustc
Copy link
Contributor

Backend

CH (ClickHouse)

Bug description

[Expected behavior] and [actual behavior].

org.apache.gluten.exception.GlutenException: For broadcast join, we must not change the columns name in the right table.
left header:col_0,
right header: broadcast_right_uid#631 -> broadcast_right_uid#631
0. Poco::Exception::Exception(String const&, int) @ 0x0000000012baec39
1. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000062cab3b
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000619520c
3. DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x000000000639836b
4. local_engine::JoinRelParser::parseJoin(substrait::JoinRel const&, std::unique_ptr>, std::unique_ptr>) @ 0x000000000663236c
5. local_engine::JoinRelParser::parseOp(substrait::Rel const&, std::list>&) @ 0x00000000066305a3
6. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000668a0e7
7. local_engine::RelParser::parseOp(substrait::Rel const&, std::list>&) @ 0x0000000006658e78
8. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000668a0e7
9. local_engine::RelParser::parseOp(substrait::Rel const&, std::list>&) @ 0x0000000006658e78
10. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000668a0e7
11. local_engine::JoinRelParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000663053f
12. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000668a0e7
13. local_engine::RelParser::parseOp(substrait::Rel const&, std::list>&) @ 0x0000000006658e78
14. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000668a0e7
15. local_engine::RelParser::parseOp(substrait::Rel const&, std::list>&) @ 0x0000000006658e78
16. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000668a0e7
17. local_engine::JoinRelParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000663053f
18. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000668a0e7
19. local_engine::RelParser::parseOp(substrait::Rel const&, std::list>&) @ 0x0000000006658e78
20. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000668a0e7
21. local_engine::RelParser::parseOp(substrait::Rel const&, std::list>&) @ 0x0000000006658e78
22. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list>&) @ 0x000000000668a0e7
23. local_engine::SerializedPlanParser::parse(std::unique_ptr>) @ 0x0000000006688c7f
24. local_engine::SerializedPlanParser::parse(String const&) @ 0x000000000669a50e
25. Java_org_apache_gluten_vectorized_ExpressionEvaluatorJniWrapper_nativeCreateKernelWithIterator @ 0x000000000617a058

	at org.apache.gluten.vectorized.ExpressionEvaluatorJniWrapper.nativeCreateKernelWithIterator(Native Method)
	at org.apache.gluten.vectorized.CHNativeExpressionEvaluator.createKernelWithBatchIterator(CHNativeExpressionEvaluator.java:93)
	at org.apache.gluten.backendsapi.clickhouse.CHIteratorApi.genFinalStageIterator(CHIteratorApi.scala:288)
	at org.apache.gluten.execution.WholeStageZippedPartitionsRDD.$anonfun$compute$1(WholeStageZippedPartitionsRDD.scala:59)
	at org.apache.gluten.utils.Arm$.withResource(Arm.scala:25)
	at org.apache.gluten.metrics.GlutenTimeMetric$.millis(GlutenTimeMetric.scala:37)
	at org.apache.gluten.execution.WholeStageZippedPartitionsRDD.compute(WholeStageZippedPartitionsRDD.scala:46)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:365)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:329)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:365)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:329)
	at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

No response
@lgbo-ustc lgbo-ustc added bug Something isn't working triage labels Jun 27, 2024
@lgbo-ustc
Copy link
Contributor Author

names in both header are the same, the type may be defferent.

@lgbo-ustc lgbo-ustc mentioned this issue Jun 27, 2024
Merged
liuneng1994 pushed a commit that referenced this issue Jun 28, 2024
@lgbo-ustc
[GLUTEN-6257][CH] Mismatched headers in broadcast join #6258
2bb813c 
What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)

Fixes: #6257

How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

unit tests

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working triage
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[GLUTEN-6257][CH] Mismatched headers in broadcast join bigo-sg/gluten
1 participant
@lgbo-ustc