Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improvement](mtmv) Refactor the plan getting logic when create mtmv,and support to collect view from plan #40428

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

seawinde
Copy link
Contributor

@seawinde seawinde commented Sep 5, 2024

Proposed changes

Refactor some plan getting logic when create materialized view, and support to collect view from plan.
this is the precondition to support create materialized view with view.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@seawinde
Copy link
Contributor Author

seawinde commented Sep 5, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38289 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f1c0fc0f4a9ad6672e6fe32c150dddd7b5f9c489, data reload: false

------ Round 1 ----------------------------------
q1	17603	4328	4286	4286
q2	2021	195	199	195
q3	11954	1009	1072	1009
q4	10515	761	692	692
q5	7768	2818	2848	2818
q6	224	143	148	143
q7	960	607	600	600
q8	9330	2097	2086	2086
q9	7193	6556	6559	6556
q10	7012	2209	2166	2166
q11	456	243	247	243
q12	402	237	231	231
q13	18659	3120	3029	3029
q14	290	236	243	236
q15	539	481	486	481
q16	595	510	520	510
q17	992	752	687	687
q18	7351	6877	6763	6763
q19	1382	1032	1101	1032
q20	693	342	330	330
q21	4013	3227	3170	3170
q22	1125	1040	1026	1026
Total cold run time: 111077 ms
Total hot run time: 38289 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4379	4294	4302	4294
q2	394	269	279	269
q3	2891	2660	2643	2643
q4	1976	1641	1681	1641
q5	5689	5711	5833	5711
q6	229	139	134	134
q7	2221	1893	1819	1819
q8	3311	3459	3529	3459
q9	8922	8841	8730	8730
q10	3675	3440	3373	3373
q11	606	502	519	502
q12	894	668	664	664
q13	14978	3308	3366	3308
q14	320	308	289	289
q15	559	485	503	485
q16	641	577	613	577
q17	1872	1566	1527	1527
q18	8214	7795	7974	7795
q19	1729	1610	1677	1610
q20	2195	1869	1931	1869
q21	5723	5577	5611	5577
q22	1161	1058	1037	1037
Total cold run time: 72579 ms
Total hot run time: 57313 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193557 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f1c0fc0f4a9ad6672e6fe32c150dddd7b5f9c489, data reload: false

query1	1244	896	907	896
query2	6456	1979	1929	1929
query3	10748	4124	3903	3903
query4	59434	27288	23310	23310
query5	5599	515	500	500
query6	425	179	162	162
query7	5870	299	299	299
query8	325	217	220	217
query9	8943	2521	2524	2521
query10	485	280	279	279
query11	18253	14971	15533	14971
query12	160	103	110	103
query13	1583	403	413	403
query14	11374	7785	7585	7585
query15	240	177	173	173
query16	7175	521	492	492
query17	1125	571	606	571
query18	1813	305	303	303
query19	287	146	145	145
query20	131	113	109	109
query21	208	105	104	104
query22	4453	4475	4394	4394
query23	34374	33685	33259	33259
query24	5962	2898	2911	2898
query25	536	390	384	384
query26	677	157	150	150
query27	1793	286	280	280
query28	3826	2053	2049	2049
query29	652	412	404	404
query30	238	165	153	153
query31	926	726	797	726
query32	77	51	59	51
query33	433	297	286	286
query34	879	489	475	475
query35	840	708	724	708
query36	1070	947	964	947
query37	149	94	93	93
query38	4023	3968	3966	3966
query39	1427	1442	1395	1395
query40	204	114	112	112
query41	47	46	44	44
query42	111	94	95	94
query43	494	464	467	464
query44	1110	758	746	746
query45	201	168	168	168
query46	1082	723	760	723
query47	1887	1768	1824	1768
query48	381	304	326	304
query49	802	441	448	441
query50	817	431	423	423
query51	7084	6877	6736	6736
query52	99	83	85	83
query53	253	180	189	180
query54	563	455	464	455
query55	76	74	80	74
query56	268	273	255	255
query57	1165	1100	1078	1078
query58	223	237	238	237
query59	3029	2695	2904	2695
query60	295	264	267	264
query61	107	106	108	106
query62	747	666	642	642
query63	209	188	187	187
query64	2862	678	666	666
query65	3239	3168	3184	3168
query66	690	348	343	343
query67	15478	15132	15078	15078
query68	2959	583	594	583
query69	393	281	285	281
query70	1166	1074	1065	1065
query71	344	283	278	278
query72	5115	4171	4204	4171
query73	753	330	339	330
query74	9223	8873	8908	8873
query75	3397	2685	2739	2685
query76	1331	1048	1009	1009
query77	519	352	324	324
query78	11510	9424	9208	9208
query79	1054	540	531	531
query80	714	529	529	529
query81	483	250	240	240
query82	258	149	148	148
query83	182	161	155	155
query84	261	82	87	82
query85	762	358	344	344
query86	307	299	301	299
query87	4532	4382	4301	4301
query88	3533	2400	2401	2400
query89	379	282	291	282
query90	1969	194	194	194
query91	137	116	113	113
query92	62	51	51	51
query93	1047	537	540	537
query94	715	313	307	307
query95	354	266	268	266
query96	593	268	273	268
query97	3192	3082	3082	3082
query98	231	211	346	211
query99	1534	1317	1283	1283
Total cold run time: 306745 ms
Total hot run time: 193557 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.81 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f1c0fc0f4a9ad6672e6fe32c150dddd7b5f9c489, data reload: false

query1	0.04	0.04	0.04
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.08	0.07
query5	0.51	0.49	0.50
query6	1.13	0.74	0.73
query7	0.02	0.02	0.02
query8	0.06	0.04	0.04
query9	0.54	0.50	0.50
query10	0.56	0.57	0.54
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.65	0.58	0.58
query14	2.07	2.07	2.07
query15	0.87	0.84	0.84
query16	0.38	0.39	0.38
query17	1.04	1.03	1.00
query18	0.22	0.20	0.20
query19	1.79	1.70	1.81
query20	0.01	0.01	0.00
query21	15.39	0.68	0.68
query22	3.98	7.15	2.59
query23	18.29	1.37	1.28
query24	2.14	0.23	0.22
query25	0.17	0.08	0.08
query26	0.27	0.17	0.17
query27	0.07	0.07	0.08
query28	13.21	1.03	1.00
query29	12.61	3.36	3.36
query30	0.24	0.06	0.06
query31	2.87	0.41	0.39
query32	3.25	0.48	0.48
query33	2.99	2.95	3.03
query34	16.95	4.39	4.40
query35	4.58	4.49	4.49
query36	0.67	0.50	0.47
query37	0.18	0.16	0.15
query38	0.15	0.16	0.15
query39	0.05	0.04	0.04
query40	0.16	0.13	0.13
query41	0.10	0.06	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.61 s
Total hot run time: 32.81 s

@morrySnow morrySnow force-pushed the refactor_plan_used_when_create_mtmv branch from f1c0fc0 to b67dd37 Compare September 11, 2024 15:07
@morrySnow
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37642 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b67dd37b1c6c3fdb5d5048688b88ac3bdb017312, data reload: false

------ Round 1 ----------------------------------
q1	17655	4373	4303	4303
q2	2042	190	183	183
q3	11851	985	1305	985
q4	10508	718	684	684
q5	7748	2891	2824	2824
q6	227	141	139	139
q7	967	615	603	603
q8	9323	2067	2088	2067
q9	7242	6473	6569	6473
q10	6995	2238	2156	2156
q11	464	253	260	253
q12	403	229	225	225
q13	17760	3114	3080	3080
q14	281	232	247	232
q15	547	487	487	487
q16	513	435	444	435
q17	976	660	705	660
q18	7358	6820	6901	6820
q19	1392	944	1001	944
q20	707	340	330	330
q21	4357	2890	2757	2757
q22	1101	1002	1003	1002
Total cold run time: 110417 ms
Total hot run time: 37642 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4329	4334	4271	4271
q2	386	291	276	276
q3	2889	2663	2669	2663
q4	1955	1667	1727	1667
q5	5613	5714	5691	5691
q6	223	135	152	135
q7	2235	1803	1841	1803
q8	3292	3433	3435	3433
q9	8896	8881	8899	8881
q10	3586	3355	3362	3355
q11	609	533	525	525
q12	840	620	639	620
q13	12745	3391	3304	3304
q14	319	300	284	284
q15	534	496	493	493
q16	561	494	495	494
q17	1835	1528	1517	1517
q18	8252	7871	7943	7871
q19	1740	1527	1646	1527
q20	2161	1925	1872	1872
q21	5749	5606	5411	5411
q22	1113	1027	1041	1027
Total cold run time: 69862 ms
Total hot run time: 57120 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196451 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b67dd37b1c6c3fdb5d5048688b88ac3bdb017312, data reload: false

query1	1252	873	886	873
query2	6349	1908	1855	1855
query3	10592	4076	4005	4005
query4	59829	25924	23149	23149
query5	5010	511	531	511
query6	399	178	173	173
query7	5633	304	293	293
query8	341	222	223	222
query9	7636	2514	2521	2514
query10	417	297	266	266
query11	15478	15073	15090	15073
query12	146	115	112	112
query13	1458	401	358	358
query14	10595	7059	7444	7059
query15	222	179	180	179
query16	6800	438	499	438
query17	1117	578	561	561
query18	1536	299	311	299
query19	213	157	161	157
query20	124	115	120	115
query21	221	108	105	105
query22	4564	4421	4419	4419
query23	34220	34005	33521	33521
query24	5979	2852	2838	2838
query25	502	385	383	383
query26	609	153	153	153
query27	1614	270	278	270
query28	3666	2036	2019	2019
query29	640	408	412	408
query30	233	149	147	147
query31	947	721	785	721
query32	70	55	53	53
query33	433	286	285	285
query34	889	487	482	482
query35	824	714	719	714
query36	1061	922	916	916
query37	140	91	80	80
query38	4134	4010	3940	3940
query39	1456	1397	1388	1388
query40	191	111	116	111
query41	45	46	41	41
query42	116	96	97	96
query43	510	473	453	453
query44	1096	748	734	734
query45	199	164	164	164
query46	1080	734	718	718
query47	1897	1818	1850	1818
query48	368	291	296	291
query49	797	460	450	450
query50	860	430	414	414
query51	7035	6791	7115	6791
query52	102	88	89	88
query53	254	184	180	180
query54	550	463	457	457
query55	78	76	76	76
query56	281	253	261	253
query57	1203	1099	1071	1071
query58	233	228	229	228
query59	3079	2842	2706	2706
query60	289	267	263	263
query61	105	100	101	100
query62	745	656	653	653
query63	212	188	186	186
query64	1353	733	662	662
query65	3240	3150	3185	3150
query66	675	338	344	338
query67	15651	15302	15322	15302
query68	1641	539	542	539
query69	400	262	281	262
query70	1127	1056	1108	1056
query71	338	272	282	272
query72	4655	3891	3944	3891
query73	741	321	327	321
query74	9120	8950	8890	8890
query75	3336	2669	2715	2669
query76	1336	1077	1010	1010
query77	441	324	316	316
query78	9937	9464	9282	9282
query79	913	886	870	870
query80	879	812	793	793
query81	471	264	255	255
query82	265	258	264	258
query83	195	192	190	190
query84	262	105	107	105
query85	706	398	389	389
query86	324	315	303	303
query87	4323	4385	4304	4304
query88	4688	4026	4031	4026
query89	375	363	367	363
query90	2293	316	309	309
query91	124	122	124	122
query92	84	78	78	78
query93	911	902	902	902
query94	764	370	380	370
query95	452	414	413	413
query96	468	468	467	467
query97	3107	3101	3166	3101
query98	236	233	216	216
query99	1445	1283	1261	1261
Total cold run time: 295950 ms
Total hot run time: 196451 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b67dd37b1c6c3fdb5d5048688b88ac3bdb017312, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.66	0.08	0.10
query5	0.52	0.49	0.51
query6	1.14	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.05	0.05
query9	0.54	0.50	0.48
query10	0.56	0.54	0.55
query11	0.17	0.11	0.12
query12	0.15	0.13	0.12
query13	0.61	0.59	0.60
query14	1.39	1.42	1.43
query15	0.85	0.83	0.84
query16	0.37	0.39	0.38
query17	1.05	1.01	0.98
query18	0.18	0.18	0.17
query19	1.94	1.80	1.85
query20	0.02	0.01	0.01
query21	15.41	0.65	0.65
query22	4.05	7.48	2.18
query23	18.68	1.32	1.26
query24	2.10	0.22	0.22
query25	0.16	0.08	0.08
query26	0.27	0.18	0.18
query27	0.08	0.07	0.08
query28	13.24	1.02	1.01
query29	12.59	3.35	3.34
query30	0.24	0.06	0.05
query31	2.90	0.40	0.40
query32	3.22	0.48	0.48
query33	2.98	3.06	2.99
query34	17.26	4.34	4.47
query35	4.49	4.55	4.47
query36	0.66	0.48	0.48
query37	0.18	0.16	0.16
query38	0.15	0.16	0.14
query39	0.04	0.04	0.04
query40	0.16	0.13	0.12
query41	0.09	0.06	0.05
query42	0.07	0.06	0.05
query43	0.04	0.04	0.03
Total cold run time: 110.63 s
Total hot run time: 31.72 s

@seawinde
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39820 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6fa6b4f4a56261a9f4333339d1316955841df542, data reload: false

------ Round 1 ----------------------------------
q1	17606	4230	4178	4178
q2	2042	186	183	183
q3	10639	1284	1408	1284
q4	10342	1049	1037	1037
q5	7721	3154	3125	3125
q6	226	141	139	139
q7	1020	638	619	619
q8	9453	2024	2054	2024
q9	6799	6298	6290	6290
q10	7019	2503	2531	2503
q11	449	261	247	247
q12	411	224	226	224
q13	17771	3066	3050	3050
q14	288	249	255	249
q15	565	502	476	476
q16	558	423	426	423
q17	994	937	922	922
q18	7416	6864	6806	6806
q19	1392	1228	1224	1224
q20	596	322	324	322
q21	3911	3526	3510	3510
q22	1076	1016	985	985
Total cold run time: 108294 ms
Total hot run time: 39820 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4258	4225	4192	4192
q2	344	234	228	228
q3	2898	2887	2908	2887
q4	1947	1959	1999	1959
q5	5779	5749	5738	5738
q6	231	143	137	137
q7	2132	1800	1781	1781
q8	3371	3401	3418	3401
q9	8795	8837	8738	8738
q10	3480	3608	3645	3608
q11	594	490	487	487
q12	794	638	667	638
q13	9470	3285	3139	3139
q14	309	273	269	269
q15	559	509	497	497
q16	534	479	482	479
q17	1798	1775	1759	1759
q18	8459	7869	7963	7869
q19	1773	1753	1773	1753
q20	2142	1888	1879	1879
q21	5831	5707	5731	5707
q22	1153	1034	990	990
Total cold run time: 66651 ms
Total hot run time: 58135 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 199894 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6fa6b4f4a56261a9f4333339d1316955841df542, data reload: false

query1	1291	873	915	873
query2	6230	1772	1731	1731
query3	10667	3825	3952	3825
query4	56999	25182	23967	23967
query5	5061	541	527	527
query6	307	171	173	171
query7	5519	309	300	300
query8	283	220	213	213
query9	5894	2448	2446	2446
query10	428	308	283	283
query11	16047	15573	15504	15504
query12	160	103	104	103
query13	1376	405	377	377
query14	10294	7202	7435	7202
query15	204	181	179	179
query16	7087	481	445	445
query17	1148	608	594	594
query18	1908	297	305	297
query19	213	155	158	155
query20	132	112	106	106
query21	221	105	104	104
query22	4777	4536	4504	4504
query23	34402	34051	33695	33695
query24	6202	3122	3062	3062
query25	497	399	402	399
query26	615	160	157	157
query27	1556	283	287	283
query28	2794	2040	2012	2012
query29	671	416	422	416
query30	223	151	148	148
query31	956	755	798	755
query32	75	54	57	54
query33	455	303	306	303
query34	912	495	484	484
query35	852	736	742	736
query36	1032	907	935	907
query37	141	88	83	83
query38	4079	3890	3952	3890
query39	1454	1402	1405	1402
query40	210	115	117	115
query41	48	47	45	45
query42	124	99	100	99
query43	490	451	438	438
query44	1185	762	759	759
query45	203	178	168	168
query46	1093	792	826	792
query47	1912	1772	1786	1772
query48	377	288	293	288
query49	732	459	450	450
query50	994	442	444	442
query51	6873	6823	6906	6823
query52	102	90	91	90
query53	268	187	183	183
query54	577	466	465	465
query55	82	78	80	78
query56	285	263	270	263
query57	1220	1099	1104	1099
query58	242	238	243	238
query59	2553	2515	2503	2503
query60	299	282	279	279
query61	105	101	99	99
query62	820	690	662	662
query63	232	189	199	189
query64	1634	692	704	692
query65	3239	3199	3176	3176
query66	675	345	343	343
query67	15928	15539	15426	15426
query68	2024	578	586	578
query69	403	290	296	290
query70	1187	1105	1077	1077
query71	352	294	294	294
query72	4795	4220	4191	4191
query73	777	328	334	328
query74	9249	9010	8907	8907
query75	3362	2765	2721	2721
query76	1365	1345	1361	1345
query77	529	351	339	339
query78	9937	9733	9350	9350
query79	994	899	890	890
query80	1034	886	865	865
query81	485	269	278	269
query82	1385	274	278	274
query83	210	200	200	200
query84	270	119	112	112
query85	803	489	481	481
query86	364	326	307	307
query87	4439	4528	4407	4407
query88	4142	4080	4063	4063
query89	394	378	380	378
query90	2074	323	320	320
query91	139	137	135	135
query92	82	79	80	79
query93	1103	1048	1043	1043
query94	790	392	374	374
query95	513	443	437	437
query96	473	473	476	473
query97	3138	3194	3319	3194
query98	228	230	223	223
query99	1729	1323	1282	1282
Total cold run time: 294433 ms
Total hot run time: 199894 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.81 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6fa6b4f4a56261a9f4333339d1316955841df542, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.66	0.07	0.07
query5	0.51	0.49	0.51
query6	1.14	0.73	0.72
query7	0.02	0.02	0.02
query8	0.06	0.04	0.05
query9	0.56	0.53	0.51
query10	0.58	0.58	0.59
query11	0.16	0.12	0.12
query12	0.14	0.12	0.12
query13	0.63	0.61	0.61
query14	1.47	1.49	1.47
query15	0.90	0.88	0.88
query16	0.37	0.36	0.37
query17	1.03	1.03	1.04
query18	0.16	0.17	0.16
query19	1.89	1.80	1.80
query20	0.01	0.01	0.01
query21	15.41	0.67	0.67
query22	4.34	7.92	1.07
query23	18.22	1.39	1.37
query24	2.26	0.23	0.22
query25	0.18	0.08	0.08
query26	0.29	0.19	0.18
query27	0.08	0.08	0.08
query28	13.19	1.01	0.99
query29	12.52	3.36	3.30
query30	0.24	0.05	0.05
query31	2.92	0.42	0.42
query32	3.23	0.50	0.49
query33	3.02	3.05	3.07
query34	15.46	4.31	4.30
query35	4.37	4.32	4.33
query36	0.69	0.49	0.50
query37	0.19	0.17	0.16
query38	0.16	0.15	0.16
query39	0.05	0.04	0.04
query40	0.16	0.13	0.14
query41	0.10	0.05	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.05
Total cold run time: 108.83 s
Total hot run time: 30.81 s

@seawinde seawinde force-pushed the refactor_plan_used_when_create_mtmv branch from 4ccb8ee to 6fa6b4f Compare September 13, 2024 08:09
@seawinde
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 43356 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6fa6b4f4a56261a9f4333339d1316955841df542, data reload: false

------ Round 1 ----------------------------------
q1	17615	7522	7210	7210
q2	2034	186	181	181
q3	10551	1409	1416	1409
q4	10301	1028	1059	1028
q5	7713	3243	3167	3167
q6	242	149	150	149
q7	1035	645	635	635
q8	9461	2092	2015	2015
q9	6895	6350	6364	6350
q10	7074	2526	2515	2515
q11	446	258	258	258
q12	414	234	230	230
q13	17766	3037	3054	3037
q14	299	247	263	247
q15	590	554	528	528
q16	552	430	431	430
q17	1034	966	952	952
q18	7496	6895	6939	6895
q19	1383	1240	1241	1240
q20	609	329	327	327
q21	3912	3562	3559	3559
q22	1097	994	994	994
Total cold run time: 108519 ms
Total hot run time: 43356 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7172	7130	7160	7130
q2	340	243	239	239
q3	2996	2901	2919	2901
q4	1947	1964	1982	1964
q5	5466	5460	5491	5460
q6	238	145	142	142
q7	2070	1661	1656	1656
q8	3270	3348	3334	3334
q9	8477	8426	8455	8426
q10	3398	3457	3460	3457
q11	589	468	470	468
q12	756	569	587	569
q13	10399	3007	3023	3007
q14	292	263	274	263
q15	561	522	512	512
q16	496	449	444	444
q17	1756	1737	1738	1737
q18	7980	7690	7566	7566
q19	1719	1709	1705	1705
q20	2046	1795	1817	1795
q21	5725	5293	5382	5293
q22	1068	1040	1011	1011
Total cold run time: 68761 ms
Total hot run time: 59079 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195514 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6fa6b4f4a56261a9f4333339d1316955841df542, data reload: false

query1	832	387	389	387
query2	6077	1770	1758	1758
query3	6506	209	221	209
query4	28393	24323	24447	24323
query5	3742	572	536	536
query6	266	178	171	171
query7	4575	308	306	306
query8	285	235	229	229
query9	8469	2591	2589	2589
query10	452	292	295	292
query11	16213	15556	15562	15556
query12	161	111	105	105
query13	1639	400	377	377
query14	9560	6645	6714	6645
query15	224	177	178	177
query16	7208	465	451	451
query17	1016	590	595	590
query18	1978	302	300	300
query19	195	151	154	151
query20	124	114	114	114
query21	209	112	106	106
query22	4675	4596	4382	4382
query23	34610	33956	33975	33956
query24	10082	3107	3096	3096
query25	629	387	393	387
query26	1625	159	155	155
query27	2776	278	286	278
query28	5996	2113	2079	2079
query29	983	428	407	407
query30	247	159	156	156
query31	974	779	800	779
query32	86	55	57	55
query33	626	299	300	299
query34	905	496	493	493
query35	874	750	725	725
query36	1073	918	900	900
query37	172	81	98	81
query38	4050	3880	3951	3880
query39	1474	1420	1429	1420
query40	238	125	113	113
query41	50	46	45	45
query42	120	97	97	97
query43	503	443	457	443
query44	1165	794	778	778
query45	198	167	171	167
query46	1118	828	801	801
query47	1899	1777	1797	1777
query48	375	288	289	288
query49	854	440	453	440
query50	908	437	464	437
query51	7132	6852	6842	6842
query52	100	88	89	88
query53	262	184	186	184
query54	656	465	464	464
query55	79	79	77	77
query56	286	289	262	262
query57	1213	1105	1091	1091
query58	248	229	235	229
query59	2758	2503	2409	2409
query60	303	267	276	267
query61	103	102	102	102
query62	930	669	663	663
query63	217	196	189	189
query64	4430	677	725	677
query65	3258	3159	3166	3159
query66	1137	291	306	291
query67	16012	15592	15893	15592
query68	2886	856	870	856
query69	410	327	342	327
query70	1178	1179	1143	1143
query71	346	343	340	340
query72	4865	3386	3399	3386
query73	582	582	587	582
query74	9373	9068	9267	9068
query75	3113	2978	2982	2978
query76	1559	835	837	835
query77	425	406	399	399
query78	9556	9309	9349	9309
query79	886	885	864	864
query80	828	822	804	804
query81	449	261	267	261
query82	264	263	267	263
query83	190	186	194	186
query84	248	108	107	107
query85	707	421	398	398
query86	332	310	334	310
query87	4483	4353	4349	4349
query88	4133	4057	4058	4057
query89	375	364	371	364
query90	1501	321	314	314
query91	125	124	124	124
query92	81	76	77	76
query93	1040	1036	1020	1020
query94	585	389	395	389
query95	507	415	411	411
query96	467	464	467	464
query97	3104	3138	3168	3138
query98	228	228	233	228
query99	1502	1317	1292	1292
Total cold run time: 272861 ms
Total hot run time: 195514 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.79 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6fa6b4f4a56261a9f4333339d1316955841df542, data reload: false

query1	0.05	0.05	0.04
query2	0.07	0.03	0.04
query3	0.23	0.04	0.04
query4	1.68	0.07	0.07
query5	0.51	0.49	0.52
query6	1.13	0.73	0.73
query7	0.02	0.02	0.02
query8	0.06	0.05	0.04
query9	0.57	0.52	0.52
query10	0.56	0.57	0.55
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.62	0.61	0.60
query14	1.46	1.46	1.48
query15	0.90	0.88	0.88
query16	0.36	0.37	0.36
query17	1.01	1.01	1.02
query18	0.16	0.16	0.16
query19	1.93	1.77	1.78
query20	0.01	0.01	0.01
query21	15.40	0.70	0.69
query22	4.19	7.98	1.09
query23	17.99	1.28	1.30
query24	2.29	0.22	0.21
query25	0.19	0.10	0.08
query26	0.29	0.17	0.18
query27	0.07	0.07	0.08
query28	13.18	1.13	1.09
query29	12.53	3.37	3.33
query30	0.24	0.06	0.05
query31	2.86	0.42	0.42
query32	3.22	0.50	0.49
query33	3.00	3.04	3.09
query34	15.46	4.32	4.33
query35	4.36	4.34	4.36
query36	0.69	0.50	0.49
query37	0.19	0.16	0.16
query38	0.17	0.15	0.15
query39	0.04	0.04	0.05
query40	0.16	0.13	0.14
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 108.36 s
Total hot run time: 30.79 s

@seawinde seawinde force-pushed the refactor_plan_used_when_create_mtmv branch from 6fa6b4f to a1bf800 Compare November 27, 2024 03:03
@seawinde seawinde marked this pull request as ready for review November 27, 2024 07:45
@seawinde
Copy link
Contributor Author

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants