Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement]downgrade resource tag when there is not queryable replica #44255

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

wangbo
Copy link
Contributor

@wangbo wangbo commented Nov 19, 2024

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
Add a user property allow_resource_tag_downgrade , when allow_resource_tag_downgrade = true, it means when resource tag has no a queryable replica,then resource tag is disabled and try to find a queryable replica.
In some scenarios, query success rate takes precedence over resource isolation.

currently support stream load and select.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@wangbo
Copy link
Contributor Author

wangbo commented Nov 19, 2024

run buildall

@wangbo
Copy link
Contributor Author

wangbo commented Nov 19, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 44590 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ae741b9d76b7e2a5e3d35be44275c37ec0d3f62e, data reload: false

------ Round 1 ----------------------------------
q1	17604	7493	7250	7250
q2	2232	1166	1163	1163
q3	9959	1157	1182	1157
q4	10223	745	714	714
q5	7568	2693	2633	2633
q6	241	145	144	144
q7	967	629	592	592
q8	9386	2369	2364	2364
q9	6623	6400	6358	6358
q10	7001	2307	2297	2297
q11	471	262	259	259
q12	414	213	206	206
q13	17770	3040	3049	3040
q14	234	211	218	211
q15	589	531	525	525
q16	662	577	580	577
q17	964	535	486	486
q18	7252	6690	6718	6690
q19	1347	1029	1017	1017
q20	2905	2708	2683	2683
q21	4071	3215	2897	2897
q22	1379	1349	1327	1327
Total cold run time: 109862 ms
Total hot run time: 44590 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7290	7238	7408	7238
q2	337	232	241	232
q3	3095	3002	2981	2981
q4	2040	1809	1791	1791
q5	5624	5700	5741	5700
q6	226	139	137	137
q7	2248	1867	1791	1791
q8	3322	3502	3502	3502
q9	8875	8908	8854	8854
q10	3627	3599	3591	3591
q11	606	499	496	496
q12	815	634	638	634
q13	12622	3266	3259	3259
q14	305	280	274	274
q15	565	527	513	513
q16	699	650	663	650
q17	1848	1629	1610	1610
q18	8200	7609	7845	7609
q19	1711	1632	1557	1557
q20	2151	1887	1855	1855
q21	5581	5264	5449	5264
q22	621	543	538	538
Total cold run time: 72408 ms
Total hot run time: 60076 ms

@yiguolei yiguolei added usercase Important user case type label dev/3.0.x labels Nov 19, 2024
@wangbo wangbo changed the title [Improvement]skip resource tag when there is not queryable replica [Improvement]downgrade resource tag when there is not queryable replica Nov 20, 2024
@wangbo
Copy link
Contributor Author

wangbo commented Nov 20, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40148 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6f61ec8a1ecf9e98f705e97b1168e953f8b5f523, data reload: false

------ Round 1 ----------------------------------
q1	17598	7758	7475	7475
q2	2043	189	192	189
q3	10515	1137	1139	1137
q4	10240	753	664	664
q5	7600	2770	2700	2700
q6	241	150	154	150
q7	984	630	627	627
q8	9260	1844	1919	1844
q9	6586	6386	6376	6376
q10	6990	2295	2326	2295
q11	476	259	260	259
q12	426	223	221	221
q13	17784	3027	3056	3027
q14	250	210	209	209
q15	583	529	523	523
q16	655	576	575	575
q17	972	497	537	497
q18	7256	6766	6782	6766
q19	1331	1021	938	938
q20	486	182	184	182
q21	4122	3293	3173	3173
q22	380	321	330	321
Total cold run time: 106778 ms
Total hot run time: 40148 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7422	7377	7262	7262
q2	332	228	239	228
q3	2939	2838	2823	2823
q4	1971	1763	1688	1688
q5	5391	5462	5443	5443
q6	217	146	144	144
q7	2165	1756	1703	1703
q8	3281	3391	3429	3391
q9	8497	8503	8546	8503
q10	3530	3442	3467	3442
q11	610	502	506	502
q12	782	579	625	579
q13	11849	3100	3080	3080
q14	296	267	260	260
q15	556	517	518	517
q16	673	618	654	618
q17	1821	1599	1574	1574
q18	7983	7540	7554	7540
q19	1682	1539	1516	1516
q20	2061	1807	1893	1807
q21	5509	5270	5313	5270
q22	644	596	551	551
Total cold run time: 70211 ms
Total hot run time: 58441 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192334 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6f61ec8a1ecf9e98f705e97b1168e953f8b5f523, data reload: false

query1	963	387	385	385
query2	6531	2236	2168	2168
query3	6748	214	217	214
query4	34229	23560	24128	23560
query5	4351	456	474	456
query6	288	188	192	188
query7	4637	306	295	295
query8	294	237	233	233
query9	9566	2664	2650	2650
query10	477	244	241	241
query11	18154	15193	15395	15193
query12	152	106	111	106
query13	1651	411	397	397
query14	9182	7765	7678	7678
query15	267	180	187	180
query16	8210	499	503	499
query17	1671	583	560	560
query18	2155	305	300	300
query19	365	162	156	156
query20	157	118	109	109
query21	212	102	101	101
query22	4415	4286	4265	4265
query23	35169	34119	34945	34119
query24	11255	2437	2509	2437
query25	710	383	394	383
query26	1790	150	154	150
query27	2865	285	278	278
query28	7927	2428	2417	2417
query29	985	401	413	401
query30	305	159	154	154
query31	1027	793	791	791
query32	97	59	64	59
query33	776	286	277	277
query34	989	496	519	496
query35	849	722	725	722
query36	1119	944	970	944
query37	155	71	76	71
query38	4352	4240	4203	4203
query39	1459	1480	1422	1422
query40	281	110	101	101
query41	47	46	44	44
query42	104	98	100	98
query43	546	502	509	502
query44	1262	807	808	807
query45	189	166	165	165
query46	1148	728	684	684
query47	1957	1825	1880	1825
query48	411	312	312	312
query49	1222	392	376	376
query50	802	385	384	384
query51	7354	7184	7077	7077
query52	104	92	87	87
query53	259	184	176	176
query54	1154	401	408	401
query55	76	80	77	77
query56	251	231	238	231
query57	1318	1201	1146	1146
query58	244	222	240	222
query59	3321	3121	3083	3083
query60	274	249	250	249
query61	111	133	110	110
query62	862	686	668	668
query63	220	187	192	187
query64	5146	651	636	636
query65	3324	3260	3263	3260
query66	1221	322	314	314
query67	16215	15950	15619	15619
query68	4482	545	546	545
query69	419	252	271	252
query70	1220	1172	1175	1172
query71	320	249	253	249
query72	6422	4065	4010	4010
query73	771	363	362	362
query74	10260	9009	8981	8981
query75	3440	2660	2696	2660
query76	2828	1150	1074	1074
query77	489	283	277	277
query78	10491	9523	9503	9503
query79	1503	581	599	581
query80	1019	441	431	431
query81	539	242	230	230
query82	997	122	118	118
query83	241	156	150	150
query84	289	74	69	69
query85	1204	300	294	294
query86	366	294	306	294
query87	4773	4622	4681	4622
query88	3534	2205	2168	2168
query89	399	291	290	290
query90	1939	192	188	188
query91	148	106	105	105
query92	57	52	50	50
query93	1101	538	540	538
query94	786	304	292	292
query95	365	253	246	246
query96	617	274	283	274
query97	2880	2730	2706	2706
query98	228	204	198	198
query99	1550	1344	1306	1306
Total cold run time: 301604 ms
Total hot run time: 192334 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.08 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6f61ec8a1ecf9e98f705e97b1168e953f8b5f523, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.04	0.03
query3	0.23	0.08	0.08
query4	1.61	0.10	0.10
query5	0.43	0.39	0.40
query6	1.15	0.66	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.56	0.51	0.47
query10	0.56	0.54	0.56
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.60	0.61
query14	2.71	2.81	2.77
query15	0.91	0.83	0.82
query16	0.40	0.39	0.39
query17	1.07	1.07	1.05
query18	0.23	0.22	0.22
query19	1.98	1.88	2.02
query20	0.02	0.01	0.01
query21	15.36	0.58	0.59
query22	2.43	2.58	2.02
query23	16.82	1.22	0.88
query24	2.51	1.99	1.04
query25	0.22	0.20	0.05
query26	0.62	0.15	0.15
query27	0.06	0.06	0.04
query28	10.18	1.10	1.06
query29	12.54	3.19	3.20
query30	0.25	0.07	0.06
query31	2.86	0.38	0.38
query32	3.29	0.46	0.49
query33	2.97	3.04	3.06
query34	16.99	4.50	4.51
query35	4.55	4.55	4.54
query36	0.67	0.48	0.48
query37	0.09	0.06	0.07
query38	0.05	0.04	0.03
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.73 s
Total hot run time: 33.08 s

@wangbo
Copy link
Contributor Author

wangbo commented Nov 21, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40426 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e57a77bc04490eec88a3c6ba8558fc07280a6c8c, data reload: false

------ Round 1 ----------------------------------
q1	17576	7600	7384	7384
q2	2047	175	170	170
q3	10552	1182	1194	1182
q4	10232	759	762	759
q5	7604	2808	2746	2746
q6	243	152	153	152
q7	1003	637	620	620
q8	9252	1888	1961	1888
q9	6617	6450	6460	6450
q10	6999	2304	2281	2281
q11	466	268	269	268
q12	436	213	218	213
q13	17797	3032	3060	3032
q14	248	216	219	216
q15	571	535	519	519
q16	649	595	585	585
q17	998	562	576	562
q18	7505	6837	6678	6678
q19	1335	975	1030	975
q20	469	187	183	183
q21	3957	3282	3241	3241
q22	379	322	325	322
Total cold run time: 106935 ms
Total hot run time: 40426 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7354	7301	7275	7275
q2	333	226	236	226
q3	2950	2812	2785	2785
q4	1966	1738	1733	1733
q5	5418	5427	5443	5427
q6	221	140	142	140
q7	2128	1711	1728	1711
q8	3304	3446	3390	3390
q9	8562	8522	8519	8519
q10	3526	3445	3478	3445
q11	599	487	531	487
q12	771	580	562	562
q13	11616	3081	3061	3061
q14	289	272	262	262
q15	572	509	508	508
q16	671	625	629	625
q17	1834	1606	1599	1599
q18	7862	7544	7414	7414
q19	1683	1442	1553	1442
q20	2057	1835	1822	1822
q21	5437	5303	5207	5207
q22	619	564	540	540
Total cold run time: 69772 ms
Total hot run time: 58180 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192278 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e57a77bc04490eec88a3c6ba8558fc07280a6c8c, data reload: false

query1	983	377	363	363
query2	6541	2157	2067	2067
query3	6717	208	207	207
query4	34016	23568	23681	23568
query5	4466	473	444	444
query6	288	186	178	178
query7	4637	294	294	294
query8	294	228	224	224
query9	9615	2679	2675	2675
query10	465	252	251	251
query11	18410	15366	15303	15303
query12	166	101	100	100
query13	1678	433	407	407
query14	10401	7396	7564	7396
query15	313	186	181	181
query16	8056	436	473	436
query17	2229	592	549	549
query18	2145	292	282	282
query19	372	141	142	141
query20	120	111	107	107
query21	210	103	102	102
query22	4589	4461	4477	4461
query23	34934	34072	34175	34072
query24	10298	2461	2443	2443
query25	655	377	392	377
query26	1449	146	149	146
query27	2833	274	275	274
query28	7773	2446	2424	2424
query29	898	411	404	404
query30	300	149	143	143
query31	1048	857	843	843
query32	96	58	58	58
query33	772	291	280	280
query34	956	497	543	497
query35	880	716	715	715
query36	1095	943	979	943
query37	140	77	73	73
query38	4302	4165	4278	4165
query39	1498	1430	1419	1419
query40	269	99	99	99
query41	47	44	44	44
query42	110	94	97	94
query43	528	489	491	489
query44	1244	818	813	813
query45	181	161	169	161
query46	1138	680	688	680
query47	1953	1862	1863	1862
query48	401	312	320	312
query49	1139	380	385	380
query50	810	376	387	376
query51	7384	7062	7048	7048
query52	99	88	92	88
query53	254	181	188	181
query54	1286	404	411	404
query55	80	80	76	76
query56	261	241	253	241
query57	1326	1168	1160	1160
query58	229	223	224	223
query59	3199	3148	3112	3112
query60	293	254	247	247
query61	113	103	103	103
query62	868	672	663	663
query63	208	186	187	186
query64	5192	657	636	636
query65	3283	3208	3275	3208
query66	1443	322	325	322
query67	16249	15720	15763	15720
query68	4913	551	550	550
query69	437	240	246	240
query70	1185	1157	1167	1157
query71	348	270	252	252
query72	6438	4027	4052	4027
query73	761	355	360	355
query74	10565	9142	9154	9142
query75	3437	2671	2683	2671
query76	2929	1163	1109	1109
query77	383	272	289	272
query78	10628	9492	9455	9455
query79	2576	599	611	599
query80	1170	446	437	437
query81	553	233	246	233
query82	977	116	121	116
query83	259	155	150	150
query84	238	84	74	74
query85	1589	307	293	293
query86	486	293	295	293
query87	4693	4526	4654	4526
query88	4021	2232	2211	2211
query89	401	296	296	296
query90	2192	185	183	183
query91	131	100	103	100
query92	64	49	49	49
query93	1618	556	549	549
query94	1036	305	288	288
query95	349	245	248	245
query96	605	281	285	281
query97	2896	2708	2691	2691
query98	218	195	201	195
query99	1639	1307	1299	1299
Total cold run time: 305960 ms
Total hot run time: 192278 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.69 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e57a77bc04490eec88a3c6ba8558fc07280a6c8c, data reload: false

query1	0.03	0.03	0.02
query2	0.08	0.03	0.04
query3	0.23	0.08	0.06
query4	1.61	0.10	0.10
query5	0.43	0.43	0.41
query6	1.16	0.66	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.60	0.50	0.53
query10	0.56	0.57	0.55
query11	0.15	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.62	0.60
query14	2.72	2.79	2.72
query15	0.89	0.83	0.82
query16	0.40	0.37	0.38
query17	1.06	1.03	1.07
query18	0.23	0.22	0.22
query19	1.93	1.86	1.97
query20	0.01	0.01	0.01
query21	15.37	0.56	0.57
query22	2.59	2.94	1.67
query23	16.76	1.02	0.81
query24	3.00	1.25	1.99
query25	0.27	0.13	0.10
query26	0.56	0.13	0.12
query27	0.05	0.05	0.04
query28	9.99	1.11	1.08
query29	12.56	3.21	3.21
query30	0.25	0.06	0.06
query31	2.86	0.39	0.39
query32	3.27	0.46	0.47
query33	2.98	3.09	3.05
query34	17.21	4.45	4.46
query35	4.54	4.48	4.49
query36	0.65	0.49	0.47
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.04	0.02	0.02
query40	0.15	0.12	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.02	0.03
Total cold run time: 106.28 s
Total hot run time: 32.69 s

@wangbo
Copy link
Contributor Author

wangbo commented Nov 26, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40136 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1ce49b6e607a31a79da7105e92d126b04b1afa2c, data reload: false

------ Round 1 ----------------------------------
q1	17635	7516	7355	7355
q2	2057	167	170	167
q3	10657	1070	1270	1070
q4	10272	765	770	765
q5	7604	2729	2730	2729
q6	239	148	145	145
q7	978	615	609	609
q8	9231	1879	1895	1879
q9	6679	6458	6415	6415
q10	7015	2308	2331	2308
q11	471	261	254	254
q12	432	219	213	213
q13	17776	3014	3056	3014
q14	249	224	219	219
q15	582	537	524	524
q16	665	588	581	581
q17	986	563	521	521
q18	7450	6737	6742	6737
q19	1338	989	963	963
q20	465	178	179	178
q21	3988	3174	3334	3174
q22	391	316	317	316
Total cold run time: 107160 ms
Total hot run time: 40136 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7274	7239	7225	7225
q2	339	224	231	224
q3	2964	2824	2941	2824
q4	2058	1803	1828	1803
q5	5668	5686	5687	5686
q6	225	145	142	142
q7	2229	1834	1798	1798
q8	3455	3538	3566	3538
q9	8818	8936	8927	8927
q10	3623	3552	3584	3552
q11	619	502	506	502
q12	815	620	627	620
q13	11155	3223	3219	3219
q14	309	274	275	274
q15	568	536	524	524
q16	689	641	662	641
q17	1838	1672	1618	1618
q18	8360	7708	7851	7708
q19	1728	1546	1497	1497
q20	2121	1849	1931	1849
q21	5523	5368	5406	5368
q22	615	541	577	541
Total cold run time: 70993 ms
Total hot run time: 60080 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196348 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1ce49b6e607a31a79da7105e92d126b04b1afa2c, data reload: false

query1	1288	937	974	937
query2	6237	2075	2026	2026
query3	10806	3926	3974	3926
query4	67638	29325	23436	23436
query5	4888	465	452	452
query6	447	167	178	167
query7	5713	287	284	284
query8	304	239	213	213
query9	9126	2629	2619	2619
query10	463	244	240	240
query11	17301	15249	15835	15249
query12	162	113	106	106
query13	1574	444	427	427
query14	10167	7412	7348	7348
query15	216	187	187	187
query16	7092	473	464	464
query17	1433	599	588	588
query18	1879	304	320	304
query19	210	159	162	159
query20	120	155	110	110
query21	211	102	99	99
query22	4646	4525	4700	4525
query23	34873	34173	34575	34173
query24	5476	2511	2504	2504
query25	498	380	400	380
query26	660	154	149	149
query27	2068	277	287	277
query28	4383	2506	2475	2475
query29	686	445	420	420
query30	210	150	148	148
query31	1063	798	866	798
query32	70	52	55	52
query33	414	296	303	296
query34	923	515	531	515
query35	831	738	739	738
query36	1114	967	993	967
query37	121	70	78	70
query38	4388	4514	4390	4390
query39	1506	1499	1449	1449
query40	201	100	103	100
query41	47	42	42	42
query42	119	101	103	101
query43	534	485	477	477
query44	1207	812	814	812
query45	182	171	168	168
query46	1155	721	705	705
query47	2070	1920	1978	1920
query48	426	325	332	325
query49	733	383	393	383
query50	857	422	403	403
query51	7377	7286	6903	6903
query52	96	86	87	86
query53	254	177	181	177
query54	496	388	401	388
query55	76	80	77	77
query56	260	223	248	223
query57	1308	1192	1168	1168
query58	206	224	223	223
query59	3305	3249	3173	3173
query60	268	244	254	244
query61	112	112	107	107
query62	793	680	675	675
query63	213	186	196	186
query64	1396	665	645	645
query65	3264	3206	3197	3197
query66	705	305	314	305
query67	16170	15624	15684	15624
query68	4141	568	549	549
query69	420	250	284	250
query70	1212	1141	1159	1141
query71	359	256	245	245
query72	6469	4059	3975	3975
query73	775	358	358	358
query74	10133	8882	9124	8882
query75	3405	2709	2653	2653
query76	1852	1080	1068	1068
query77	590	286	289	286
query78	10563	9455	9442	9442
query79	2315	588	597	588
query80	1381	432	431	431
query81	532	226	223	223
query82	1287	114	115	114
query83	235	147	142	142
query84	290	84	77	77
query85	1037	303	299	299
query86	436	304	302	302
query87	4781	4495	4633	4495
query88	3539	2191	2159	2159
query89	427	288	323	288
query90	1984	192	182	182
query91	146	106	107	106
query92	69	49	50	49
query93	3130	543	528	528
query94	930	304	289	289
query95	351	250	260	250
query96	643	275	281	275
query97	2864	2723	2719	2719
query98	213	194	191	191
query99	1680	1288	1288	1288
Total cold run time: 323224 ms
Total hot run time: 196348 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.83 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1ce49b6e607a31a79da7105e92d126b04b1afa2c, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.04	0.02
query3	0.24	0.08	0.07
query4	1.61	0.10	0.10
query5	0.42	0.43	0.41
query6	1.16	0.65	0.65
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.59	0.50	0.49
query10	0.54	0.55	0.56
query11	0.15	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.61	0.59
query14	2.72	2.81	2.85
query15	0.88	0.82	0.82
query16	0.38	0.37	0.39
query17	1.05	1.04	1.08
query18	0.22	0.21	0.21
query19	1.96	1.87	2.01
query20	0.01	0.01	0.01
query21	15.35	0.58	0.58
query22	2.34	2.72	2.14
query23	17.18	0.83	0.75
query24	2.86	1.98	1.99
query25	0.34	0.05	0.18
query26	0.57	0.14	0.13
query27	0.05	0.04	0.04
query28	9.36	1.10	1.07
query29	12.55	3.24	3.21
query30	0.24	0.06	0.06
query31	2.87	0.38	0.38
query32	3.29	0.46	0.46
query33	2.99	3.03	3.11
query34	17.04	4.46	4.44
query35	4.49	4.45	4.50
query36	0.65	0.48	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.12	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.48 s
Total hot run time: 33.83 s

@wangbo
Copy link
Contributor Author

wangbo commented Nov 27, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39948 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 26973aaa2a4e61120eb4c055057490c645710ab6, data reload: false

------ Round 1 ----------------------------------
q1	17611	7502	7294	7294
q2	2051	185	180	180
q3	10565	1089	1166	1089
q4	10573	814	668	668
q5	7648	2749	2733	2733
q6	239	147	144	144
q7	974	619	597	597
q8	9217	1841	1892	1841
q9	6457	6345	6336	6336
q10	6992	2335	2320	2320
q11	456	253	251	251
q12	409	222	210	210
q13	17769	3007	3023	3007
q14	242	214	203	203
q15	574	532	514	514
q16	651	585	571	571
q17	982	519	520	519
q18	7434	6760	6752	6752
q19	1333	950	1028	950
q20	478	181	175	175
q21	3999	3330	3283	3283
q22	375	311	311	311
Total cold run time: 107029 ms
Total hot run time: 39948 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7272	7273	7217	7217
q2	331	225	228	225
q3	2947	2791	2983	2791
q4	2058	1818	1843	1818
q5	5751	5675	5636	5636
q6	229	143	148	143
q7	2242	1810	1807	1807
q8	3416	3527	3511	3511
q9	8762	8869	8912	8869
q10	3604	3576	3563	3563
q11	601	505	529	505
q12	839	616	634	616
q13	9173	3265	3203	3203
q14	308	271	278	271
q15	568	530	522	522
q16	689	668	656	656
q17	1849	1633	1601	1601
q18	8226	7781	7900	7781
q19	1807	1541	1659	1541
q20	2165	1973	1976	1973
q21	5702	5390	5526	5390
q22	663	560	588	560
Total cold run time: 69202 ms
Total hot run time: 60199 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198362 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 26973aaa2a4e61120eb4c055057490c645710ab6, data reload: false

query1	1246	965	952	952
query2	6245	2103	2063	2063
query3	11070	4211	4055	4055
query4	68366	27812	23755	23755
query5	5005	482	449	449
query6	393	197	189	189
query7	5474	303	294	294
query8	300	221	220	220
query9	8235	2756	2766	2756
query10	448	266	239	239
query11	16959	15320	15923	15320
query12	163	105	100	100
query13	1444	418	463	418
query14	10613	7277	7993	7277
query15	211	194	204	194
query16	7133	467	508	467
query17	1096	594	607	594
query18	1774	323	320	320
query19	205	168	161	161
query20	125	117	116	116
query21	211	109	109	109
query22	4818	4659	4618	4618
query23	35112	34724	34693	34693
query24	5381	2492	2551	2492
query25	510	386	391	386
query26	643	157	156	156
query27	1842	299	288	288
query28	4141	2512	2511	2511
query29	689	457	409	409
query30	210	158	154	154
query31	1002	857	857	857
query32	70	55	54	54
query33	403	302	310	302
query34	949	524	521	521
query35	913	772	764	764
query36	1111	984	990	984
query37	135	72	77	72
query38	4530	4441	4420	4420
query39	1512	1483	1476	1476
query40	210	104	105	104
query41	46	45	43	43
query42	119	107	109	107
query43	560	495	507	495
query44	1204	836	827	827
query45	192	175	167	167
query46	1186	733	713	713
query47	2021	1958	1923	1923
query48	432	338	320	320
query49	713	409	424	409
query50	852	412	428	412
query51	7431	7309	7198	7198
query52	104	87	90	87
query53	279	190	180	180
query54	510	406	409	406
query55	83	83	78	78
query56	281	240	229	229
query57	1322	1162	1148	1148
query58	230	218	234	218
query59	3269	3186	2812	2812
query60	282	252	268	252
query61	108	109	112	109
query62	796	669	675	669
query63	218	200	207	200
query64	1355	669	640	640
query65	3350	3223	3241	3223
query66	704	309	308	308
query67	16137	15592	15657	15592
query68	3951	575	567	567
query69	416	258	265	258
query70	1220	1145	1172	1145
query71	358	259	287	259
query72	6395	4035	4035	4035
query73	795	366	375	366
query74	10385	9073	9050	9050
query75	3441	2780	2705	2705
query76	1886	1070	1079	1070
query77	526	303	305	303
query78	10619	9434	9426	9426
query79	1716	615	608	608
query80	887	445	498	445
query81	514	232	239	232
query82	1291	123	120	120
query83	257	151	151	151
query84	282	75	72	72
query85	903	308	309	308
query86	353	305	310	305
query87	4748	4573	4469	4469
query88	3714	2261	2215	2215
query89	426	309	313	309
query90	2023	189	187	187
query91	144	104	104	104
query92	65	56	51	51
query93	2001	540	554	540
query94	810	303	320	303
query95	362	253	255	253
query96	632	273	286	273
query97	2902	2641	2690	2641
query98	225	197	198	197
query99	1610	1342	1321	1321
Total cold run time: 320538 ms
Total hot run time: 198362 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.65 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 26973aaa2a4e61120eb4c055057490c645710ab6, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.11
query5	0.43	0.41	0.41
query6	1.16	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.02	0.03
query9	0.58	0.52	0.52
query10	0.54	0.57	0.57
query11	0.14	0.11	0.10
query12	0.14	0.12	0.11
query13	0.62	0.60	0.60
query14	2.82	2.73	2.72
query15	0.92	0.84	0.83
query16	0.38	0.38	0.38
query17	1.05	1.07	1.05
query18	0.23	0.20	0.21
query19	1.81	1.78	2.02
query20	0.02	0.01	0.01
query21	15.39	0.59	0.59
query22	2.52	2.96	1.60
query23	17.19	0.93	0.80
query24	2.60	1.20	1.48
query25	0.24	0.04	0.15
query26	0.42	0.15	0.14
query27	0.06	0.04	0.04
query28	10.66	1.11	1.08
query29	12.56	3.25	3.25
query30	0.25	0.07	0.07
query31	2.86	0.38	0.37
query32	3.26	0.46	0.47
query33	3.01	3.05	3.04
query34	17.13	4.51	4.48
query35	4.56	4.51	4.58
query36	0.67	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.73 s
Total hot run time: 32.65 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev/2.1.x dev/3.0.x usercase Important user case type label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants