[GLUTEN-6151] Reset local property after finishing write operator

[JkSelf]

What changes were proposed in this pull request?

In parquet write, we use several local properties to pass the status in session. We need reset them after finishing write operator.

How was this patch tested?

Existing unit tests.

[github-actions]

#6151

[JkSelf]

@FelixYBW

[github-actions]

Run Gluten Clickhouse CI

[FelixYBW]

Are here the best place to reset? are they the per session config, right? what happens if two tasks from different queries are scheduled to the same spark session? is it possible?

@zhztheplayer

[FelixYBW]

tested 2 SQLs running in the same context. The second SQL can also correctly offload.

[github-actions]

Run Gluten Clickhouse CI

[FelixYBW]

COnfirmed bug is fixed. Thank you, Ke

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_6163_time.csv	log/native_master_06_21_2024_ef7b2c516_time.csv	difference	percentage
q1	35.81	35.14	-0.665	98.14%
q2	23.26	26.25	2.986	112.84%
q3	40.41	38.62	-1.787	95.58%
q4	32.31	34.48	2.172	106.72%
q5	72.99	70.26	-2.725	96.27%
q6	9.88	7.96	-1.920	80.57%
q7	82.24	85.08	2.842	103.46%
q8	84.35	83.21	-1.141	98.65%
q9	118.67	122.00	3.326	102.80%
q10	45.47	44.94	-0.534	98.83%
q11	20.58	22.82	2.245	110.91%
q12	26.27	24.55	-1.716	93.47%
q13	42.08	38.63	-3.444	91.82%
q14	19.16	18.18	-0.978	94.90%
q15	32.69	30.13	-2.567	92.15%
q16	14.19	16.41	2.223	115.67%
q17	105.51	104.52	-0.985	99.07%
q18	147.25	148.17	0.913	100.62%
q19	13.80	13.73	-0.074	99.46%
q20	30.08	26.41	-3.668	87.81%
q21	264.39	262.85	-1.533	99.42%
q22	12.21	12.55	0.347	102.84%
total	1273.59	1266.90	-6.684	99.48%

[GlutenPerfBot]

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query	log/native_master_06_22_2024_time.csv	log/native_master_06_21_2024_ef7b2c516b_time.csv	difference	percentage
q1	15.16	14.36	-0.799	94.73%
q2	18.57	19.39	0.815	104.39%
q3	3.78	4.36	0.576	115.23%
q4	63.88	63.56	-0.319	99.50%
q5	6.35	6.51	0.164	102.58%
q6	3.61	2.55	-1.067	70.47%
q7	4.19	5.63	1.440	134.39%
q8	5.33	3.38	-1.946	63.49%
q9	18.55	18.91	0.361	101.94%
q10	9.73	11.58	1.848	118.99%
q11	38.81	39.46	0.648	101.67%
q12	1.55	2.65	1.105	171.53%
q13	5.62	5.17	-0.446	92.06%
q14a	43.61	42.21	-1.405	96.78%
q14b	40.52	40.56	0.034	100.08%
q15	4.98	3.09	-1.887	62.07%
q16	38.91	39.52	0.603	101.55%
q17	5.58	5.72	0.142	102.55%
q18	7.73	6.32	-1.409	81.76%
q19	2.16	2.13	-0.036	98.34%
q20	1.43	1.42	-0.007	99.54%
q21	1.01	1.02	0.013	101.30%
q22	11.22	8.64	-2.584	76.97%
q23a	81.49	80.43	-1.055	98.71%
q23b	104.13	102.05	-2.084	98.00%
q24a	76.32	72.62	-3.699	95.15%
q24b	77.46	79.47	2.012	102.60%
q25	4.57	4.41	-0.156	96.58%
q26	2.77	2.87	0.102	103.69%
q27	3.29	3.10	-0.191	94.19%
q28	21.88	22.16	0.285	101.30%
q29	11.05	10.92	-0.132	98.80%
q30	4.35	7.52	3.173	173.00%
q31	6.72	6.43	-0.290	95.68%
q32	1.19	1.13	-0.058	95.07%
q33	7.44	4.85	-2.592	65.19%
q34	3.62	3.46	-0.165	95.45%
q35	12.72	6.63	-6.099	52.07%
q36	3.17	4.71	1.533	148.30%
q37	4.62	4.17	-0.452	90.22%
q38	11.71	11.67	-0.046	99.60%
q39a	3.42	3.21	-0.210	93.85%
q39b	3.06	2.85	-0.208	93.18%
q40	3.60	3.72	0.125	103.47%
q41	0.64	0.65	0.012	101.93%
q42	0.94	1.17	0.236	125.24%
q43	5.20	7.67	2.472	147.54%
q44	8.06	9.10	1.046	112.99%
q45	3.59	3.62	0.027	100.76%
q46	3.34	3.33	-0.010	99.70%
q47	13.98	14.06	0.076	100.54%
q48	4.51	4.48	-0.037	99.19%
q49	10.99	9.47	-1.523	86.15%
q50	25.97	23.98	-1.985	92.36%
q51	8.69	8.51	-0.184	97.89%
q52	1.02	0.97	-0.045	95.61%
q53	1.79	2.06	0.276	115.41%
q54	3.34	3.50	0.154	104.61%
q55	1.05	1.05	-0.003	99.76%
q56	4.48	4.47	-0.017	99.61%
q57	8.52	8.63	0.115	101.35%
q58	2.67	4.91	2.246	184.27%
q59	15.44	13.93	-1.517	90.18%
q60	4.89	4.76	-0.132	97.31%
q61	5.40	7.86	2.455	145.46%
q62	4.33	4.62	0.290	106.69%
q63	2.24	2.26	0.022	100.98%
q64	49.25	48.58	-0.677	98.63%
q65	13.73	17.10	3.368	124.52%
q66	3.30	3.34	0.039	101.18%
q67	349.13	353.18	4.052	101.16%
q68	3.67	3.82	0.157	104.27%
q69	6.57	6.27	-0.302	95.40%
q70	12.45	13.07	0.619	104.97%
q71	2.24	2.32	0.086	103.82%
q72	189.24	189.12	-0.114	99.94%
q73	3.01	2.50	-0.511	83.02%
q74	21.28	21.05	-0.227	98.93%
q75	22.94	23.05	0.113	100.49%
q76	8.59	9.55	0.957	111.14%
q77	2.17	2.19	0.017	100.78%
q78	42.08	44.09	2.007	104.77%
q79	3.71	3.49	-0.215	94.20%
q80	11.29	12.63	1.342	111.89%
q81	5.08	5.32	0.247	104.87%
q82	9.40	9.99	0.590	106.27%
q83	1.49	1.71	0.220	114.81%
q84	2.86	2.72	-0.143	95.00%
q85	6.89	6.85	-0.040	99.41%
q86	3.31	3.31	0.005	100.15%
q87	12.34	15.72	3.371	127.31%
q88	25.63	24.71	-0.917	96.42%
q89	3.23	8.74	5.506	270.19%
q90	8.93	4.39	-4.534	49.21%
q91	2.64	2.55	-0.099	96.27%
q92	1.30	1.40	0.097	107.45%
q93	28.74	28.73	-0.012	99.96%
q94	21.26	21.39	0.139	100.65%
q9	84.68	83.36	-1.313	98.45%
q5	3.71	3.74	0.032	100.85%
q96	12.29	12.27	-0.017	99.86%
q97	2.05	2.06	0.006	100.31%
q98	9.35	8.91	-0.435	95.35%
q99	9.35	8.91	-0.435	95.35%
total	1931.77	1934.79	3.022	100.16%

[GlutenPerfBot]

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query	log/native_master_06_23_2024_time.csv	log/native_master_06_22_2024_1e06169cde_time.csv	difference	percentage
q1	14.36	15.16	0.793	105.52%
q2	17.41	18.57	1.161	106.67%
q3	4.39	3.78	-0.604	86.24%
q4	63.67	63.88	0.216	100.34%
q5	7.83	6.35	-1.482	81.08%
q6	4.33	3.61	-0.713	83.52%
q7	6.31	4.19	-2.126	66.33%
q8	5.15	5.33	0.180	103.50%
q9	17.06	18.55	1.493	108.75%
q10	11.81	9.73	-2.075	82.43%
q11	36.31	38.81	2.504	106.90%
q12	2.74	1.55	-1.196	56.37%
q13	6.44	5.62	-0.822	87.24%
q14a	41.21	43.61	2.399	105.82%
q14b	42.82	40.52	-2.296	94.64%
q15	2.74	4.98	2.231	181.27%
q16	38.75	38.91	0.165	100.43%
q17	4.69	5.58	0.893	119.05%
q18	6.41	7.73	1.314	120.49%
q19	3.24	2.16	-1.073	66.84%
q20	1.37	1.43	0.062	104.50%
q21	4.09	1.01	-3.083	24.59%
q22	7.99	11.22	3.232	140.45%
q23a	84.91	81.49	-3.428	95.96%
q23b	102.60	104.13	1.532	101.49%
q24a	80.62	76.32	-4.298	94.67%
q24b	78.10	77.46	-0.635	99.19%
q25	4.30	4.57	0.269	106.26%
q26	2.64	2.77	0.125	104.75%
q27	3.09	3.29	0.207	106.71%
q28	23.43	21.88	-1.551	93.38%
q29	7.25	11.05	3.808	152.55%
q30	4.23	4.35	0.114	102.69%
q31	7.54	6.72	-0.820	89.13%
q32	1.18	1.19	0.010	100.87%
q33	4.77	7.44	2.677	156.16%
q34	3.70	3.62	-0.081	97.82%
q35	8.88	12.72	3.845	143.30%
q36	3.50	3.17	-0.323	90.76%
q37	4.04	4.62	0.587	114.54%
q38	12.05	11.71	-0.337	97.20%
q39a	4.14	3.42	-0.728	82.44%
q39b	2.98	3.06	0.075	102.53%
q40	4.71	3.60	-1.111	76.40%
q41	0.57	0.64	0.071	112.56%
q42	0.95	0.94	-0.010	98.96%
q43	5.56	5.20	-0.363	93.47%
q44	8.92	8.06	-0.859	90.36%
q45	3.56	3.59	0.032	100.89%
q46	3.22	3.34	0.126	103.93%
q47	14.40	13.98	-0.414	97.12%
q48	4.60	4.51	-0.087	98.11%
q49	13.37	10.99	-2.376	82.23%
q50	20.89	25.97	5.077	124.30%
q51	8.48	8.69	0.209	102.47%
q52	1.09	1.02	-0.077	92.99%
q53	1.93	1.79	-0.142	92.62%
q54	3.23	3.34	0.114	103.54%
q55	0.97	1.05	0.081	108.31%
q56	4.60	4.48	-0.120	97.39%
q57	8.59	8.52	-0.071	99.17%
q58	2.51	2.67	0.151	106.02%
q59	16.42	15.44	-0.980	94.03%
q60	4.76	4.89	0.130	102.73%
q61	5.36	5.40	0.044	100.82%
q62	4.59	4.33	-0.254	94.46%
q63	2.17	2.24	0.063	102.91%
q64	48.21	49.25	1.046	102.17%
q65	15.52	13.73	-1.788	88.48%
q66	3.47	3.30	-0.172	95.04%
q67	353.26	349.13	-4.126	98.83%
q68	3.97	3.67	-0.307	92.27%
q69	10.93	6.57	-4.350	60.18%
q70	8.82	12.45	3.629	141.14%
q71	2.25	2.24	-0.013	99.44%
q72	192.25	189.24	-3.018	98.43%
q73	2.83	3.01	0.182	106.44%
q74	21.44	21.28	-0.162	99.25%
q75	23.26	22.94	-0.326	98.60%
q76	11.74	8.59	-3.148	73.18%
q77	2.30	2.17	-0.127	94.46%
q78	39.08	42.08	2.999	107.67%
q79	3.45	3.71	0.255	107.39%
q80	14.92	11.29	-3.633	75.66%
q81	5.00	5.08	0.080	101.59%
q82	6.85	9.40	2.551	137.23%
q83	1.53	1.49	-0.042	97.28%
q84	2.76	2.86	0.097	103.50%
q85	6.53	6.89	0.356	105.45%
q86	3.28	3.31	0.025	100.77%
q87	13.58	12.34	-1.238	90.88%
q88	27.59	25.63	-1.967	92.87%
q89	3.32	3.23	-0.082	97.53%
q90	4.50	8.93	4.431	198.57%
q91	2.63	2.64	0.019	100.73%
q92	1.38	1.30	-0.081	94.11%
q93	28.76	28.74	-0.023	99.92%
q94	21.10	21.26	0.159	100.75%
q9	82.75	84.68	1.928	102.33%
q5	3.30	3.71	0.413	112.53%
q96	12.36	12.29	-0.069	99.45%
q97	2.10	2.05	-0.045	97.87%
q98	9.52	9.35	-0.171	98.21%
q99	9.52	9.35	-0.171	98.21%
total	1937.03	1931.77	-5.263	99.73%

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_master_06_23_2024_time.csv	log/native_master_06_21_2024_ef7b2c516_time.csv	difference	percentage
q1	36.88	35.14	-1.737	95.29%
q2	23.65	26.25	2.596	110.97%
q3	42.16	38.62	-3.542	91.60%
q4	33.05	34.48	1.425	104.31%
q5	69.90	70.26	0.361	100.52%
q6	7.95	7.96	0.015	100.19%
q7	80.73	85.08	4.354	105.39%
q8	82.15	83.21	1.053	101.28%
q9	119.99	122.00	2.009	101.67%
q10	46.07	44.94	-1.131	97.55%
q11	20.48	22.82	2.340	111.43%
q12	26.53	24.55	-1.984	92.52%
q13	38.12	38.63	0.513	101.35%
q14	18.74	18.18	-0.558	97.02%
q15	32.95	30.13	-2.820	91.44%
q16	14.22	16.41	2.192	115.41%
q17	104.72	104.52	-0.195	99.81%
q18	148.08	148.17	0.084	100.06%
q19	14.64	13.73	-0.915	93.75%
q20	30.40	26.41	-3.985	86.89%
q21	261.83	262.85	1.026	100.39%
q22	12.33	12.55	0.219	101.77%
total	1265.58	1266.90	1.321	100.10%

[zhztheplayer]

Are here the best place to reset? are they the per session config, right? what happens if two tasks from different queries are scheduled to the same spark session? is it possible?

@zhztheplayer

I recall that this part of code relies on thread local storages which should make the invocations thread safe. Having said that we can still look for better choices, e.g., passing through arguments, to improve portability of the code.