[CORE] Add GlutenImplicits to get FallbackSummary easily

[ulysses-you]

What changes were proposed in this pull request?

This pr adds a a helper class to get the Gluten fallback summary from a Spark Dataset. If AQE is enabled, but the query is not materialized, then this method will re-plan the query execution with disabled AQE. It is a workaround to get the final plan, and it may cause inconsistent results with a materialized query. However, we have no choice.

For example:

import org.apache.spark.sql.execution.GlutenImplicits._
val df = spark.sql("SELECT  FROM t")
df.fallbackSummary

How was this patch tested?

add test

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

• Other pull requests

[github-actions]

Run Gluten Clickhouse CI

[ulysses-you]

cc @zhouyuan @PHILO-HE thank you

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[PHILO-HE]

Thanks for your work!

[PHILO-HE]

This feature is not workable for CH backend? cc @zzcclp.

[ulysses-you]

It should work with CH backend. There are some different behavior, so I disable these tests for CH backend. e.g., velox backend would add one more project before shuffle, velox backend supports columnar cache, etc..

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[PHILO-HE]

Hi @ulysses-you, could you also update the doc? This may be a good place: https://github.com/oap-project/gluten/blob/main/docs/get-started/Velox.md

[ulysses-you]

@PHILO-HE yes, added the docs

[github-actions]

Run Gluten Clickhouse CI

[PHILO-HE]

Thanks!

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_3599_time.csv	log/native_master_11_05_2023_16e287450_time.csv	difference	percentage
q1	32.73	34.85	2.126	106.49%
q2	24.76	25.12	0.356	101.44%
q3	37.88	40.11	2.231	105.89%
q4	36.83	37.09	0.264	100.72%
q5	69.01	71.44	2.431	103.52%
q6	6.49	9.10	2.608	140.18%
q7	85.59	87.61	2.027	102.37%
q8	86.81	85.22	-1.592	98.17%
q9	123.02	121.12	-1.894	98.46%
q10	51.97	51.83	-0.138	99.74%
q11	20.23	19.68	-0.549	97.29%
q12	25.58	28.01	2.433	109.51%
q13	49.44	48.61	-0.833	98.32%
q14	17.57	19.14	1.571	108.94%
q15	30.50	33.03	2.522	108.27%
q16	16.71	16.10	-0.611	96.35%
q17	102.53	102.08	-0.447	99.56%
q18	147.61	149.04	1.430	100.97%
q19	14.80	18.25	3.449	123.31%
q20	30.65	30.91	0.269	100.88%
q21	222.07	222.95	0.878	100.40%
q22	13.76	13.47	-0.291	97.88%
total	1246.52	1264.76	18.242	101.46%