Skip to content

[featrue](Variant) limit max subcolumns in compaction#61024

Open
gary-cloud wants to merge 1 commit intoapache:masterfrom
gary-cloud:maxsubcols
Open

[featrue](Variant) limit max subcolumns in compaction#61024
gary-cloud wants to merge 1 commit intoapache:masterfrom
gary-cloud:maxsubcols

Conversation

@gary-cloud
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@gary-cloud
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 28953 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a76fed1f46c4b0d86cfd394109019e724f658f8e, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17730	4613	4330	4330
q2	q3	10704	823	524	524
q4	4734	364	259	259
q5	7899	1201	1034	1034
q6	199	177	149	149
q7	813	855	675	675
q8	10383	1475	1316	1316
q9	6623	4759	4739	4739
q10	6892	1874	1633	1633
q11	446	253	231	231
q12	733	573	467	467
q13	17847	4237	3417	3417
q14	224	224	221	221
q15	950	792	818	792
q16	758	724	655	655
q17	754	896	414	414
q18	6184	5392	5227	5227
q19	1329	996	660	660
q20	722	578	442	442
q21	4594	1939	1497	1497
q22	350	294	271	271
Total cold run time: 100868 ms
Total hot run time: 28953 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4816	4557	4536	4536
q2	q3	1806	2237	1726	1726
q4	866	1180	792	792
q5	4077	4436	4393	4393
q6	185	184	142	142
q7	1840	1673	1522	1522
q8	2512	2725	2535	2535
q9	7603	7409	7292	7292
q10	2597	2821	2515	2515
q11	509	433	412	412
q12	493	588	446	446
q13	3957	4420	3710	3710
q14	293	306	310	306
q15	858	840	787	787
q16	699	758	729	729
q17	1182	1584	1283	1283
q18	7107	6788	6558	6558
q19	912	861	857	857
q20	2059	2210	1980	1980
q21	3992	3512	3338	3338
q22	478	459	390	390
Total cold run time: 48841 ms
Total hot run time: 46249 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187508 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a76fed1f46c4b0d86cfd394109019e724f658f8e, data reload: false

query5	5114	1098	705	705
query6	436	320	253	253
query7	4266	1288	320	320
query8	425	322	300	300
query9	8865	3263	3096	3096
query10	605	444	426	426
query11	18483	19832	18712	18712
query12	234	153	159	153
query13	1322	604	420	420
query14	7292	3929	3531	3531
query14_1	3194	3166	3430	3166
query15	238	225	211	211
query16	1163	627	819	627
query17	1202	938	730	730
query18	3786	629	452	452
query19	286	291	327	291
query20	187	188	177	177
query21	247	209	153	153
query22	6081	7399	4674	4674
query23	17237	16776	16638	16638
query23_1	16668	16758	16651	16651
query24	7164	1639	1235	1235
query24_1	1233	1261	1234	1234
query25	547	457	390	390
query26	1232	267	151	151
query27	2752	459	283	283
query28	4495	1898	1864	1864
query29	792	549	452	452
query30	311	244	210	210
query31	857	731	645	645
query32	83	79	71	71
query33	514	335	282	282
query34	916	909	583	583
query35	634	695	595	595
query36	1070	1105	990	990
query37	139	97	84	84
query38	2948	2883	2937	2883
query39	881	850	870	850
query39_1	832	833	841	833
query40	225	151	144	144
query41	62	62	58	58
query42	108	107	101	101
query43	379	377	347	347
query44	
query45	203	194	183	183
query46	887	985	610	610
query47	2101	2105	2044	2044
query48	314	364	236	236
query49	629	468	380	380
query50	681	287	233	233
query51	4112	4074	4009	4009
query52	107	107	95	95
query53	294	337	292	292
query54	296	274	265	265
query55	83	82	80	80
query56	328	326	299	299
query57	1396	1335	1264	1264
query58	291	276	266	266
query59	2659	2661	2516	2516
query60	342	335	314	314
query61	152	144	140	140
query62	630	593	531	531
query63	316	277	276	276
query64	4867	1269	986	986
query65	
query66	1385	458	358	358
query67	16307	16510	16340	16340
query68	
query69	403	311	286	286
query70	985	996	968	968
query71	334	304	297	297
query72	2800	2692	2405	2405
query73	548	541	322	322
query74	10042	9940	9787	9787
query75	2867	2759	2461	2461
query76	2294	1056	693	693
query77	379	377	321	321
query78	11129	11320	10696	10696
query79	3098	824	614	614
query80	1818	620	529	529
query81	599	276	253	253
query82	953	150	119	119
query83	333	255	238	238
query84	251	126	95	95
query85	904	486	426	426
query86	496	320	303	303
query87	3109	3108	2985	2985
query88	3568	2655	2620	2620
query89	428	369	338	338
query90	2210	168	171	168
query91	159	150	143	143
query92	84	77	70	70
query93	1848	816	505	505
query94	651	328	304	304
query95	575	345	313	313
query96	643	523	225	225
query97	2468	2477	2401	2401
query98	246	229	221	221
query99	1025	971	930	930
Total cold run time: 262701 ms
Total hot run time: 187508 ms

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 50.00% (6/12) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.56% (19637/37358)
Line Coverage 36.18% (183296/506675)
Region Coverage 32.47% (142178/437857)
Branch Coverage 33.43% (61672/184489)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 50.00% (6/12) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.59% (26188/36580)
Line Coverage 54.36% (274581/505122)
Region Coverage 51.70% (228490/441994)
Branch Coverage 53.06% (98183/185053)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants