Skip to content

[fix](build) prevent math regression from global lance-c linkage - #67237

Open
linrrzqqq wants to merge 1 commit into
apache:masterfrom
linrrzqqq:opt-math-regression
Open

[fix](build) prevent math regression from global lance-c linkage#67237
linrrzqqq wants to merge 1 commit into
apache:masterfrom
linrrzqqq:opt-math-regression

Conversation

@linrrzqqq

Copy link
Copy Markdown
Collaborator

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #65304

Problem Summary:

PR #65304 added the following libraries to COMMON_THIRDPARTY:

list(APPEND COMMON_THIRDPARTY m lance_c)

This unconditionally added an early -lm to the BE link command, before Doris's own glibc-compatibility library.
In the current toolchain, libm.so is a GNU ld script:GROUP ( ./libglibc-compatibility.a ../../lib/libm.so.6 )

As a result, the early -lm resolves exp, log, and log2 from the toolchain's compatibility archive before Doris's optimized compatibility implementations are considered. This causes significant performance regressions in arithmetic benchmarks:

  • exp: approximately 220%
  • ln: approximately 167%
  • log: approximately 197%
  • log2: approximately 165%

In this pr, remove the unconditional addition of m and lance_c to COMMON_THIRDPARTY, and the imported lance_c target is retained, so future Lance consumers can link it explicitly through target_link_libraries() when the corresponding feature is enabled.

This keeps the existing glibc-compatibility link ordering unchanged and avoids
affecting unrelated BE targets.

performance

Doris> select count(exp(db)) from double_ranges;
+----------------+
| count(exp(db)) |
+----------------+
|       50000000 |
+----------------+
1 row in set (0.980 sec)

Doris> select count(exp(db)) from double_ranges;
+----------------+
| count(exp(db)) |
+----------------+
|       50000000 |
+----------------+
1 row in set (0.347 sec)


Doris> select count(ln(db)), count(ln(in_one)) from double_ranges;
+---------------+-------------------+
| count(ln(db)) | count(ln(in_one)) |
+---------------+-------------------+
|      50000000 |          50000000 |
+---------------+-------------------+
1 row in set (1.486 sec)

Doris> select count(ln(db)), count(ln(in_one)) from double_ranges;
+---------------+-------------------+
| count(ln(db)) | count(ln(in_one)) |
+---------------+-------------------+
|      50000000 |          50000000 |
+---------------+-------------------+
1 row in set (0.607 sec)


Doris> select count(log(db, db)), count(log(in_one, db)), count(log(db, in_one)), count(log(db, in_ten)) from double_ranges;
+--------------------+------------------------+------------------------+------------------------+
| count(log(db, db)) | count(log(in_one, db)) | count(log(db, in_one)) | count(log(db, in_ten)) |
+--------------------+------------------------+------------------------+------------------------+
|           50000000 |               49999990 |               50000000 |               50000000 |
+--------------------+------------------------+------------------------+------------------------+
1 row in set (5.944 sec)

Doris> select count(log(db, db)), count(log(in_one, db)), count(log(db, in_one)), count(log(db, in_ten)) from double_ranges;
+--------------------+------------------------+------------------------+------------------------+
| count(log(db, db)) | count(log(in_one, db)) | count(log(db, in_one)) | count(log(db, in_ten)) |
+--------------------+------------------------+------------------------+------------------------+
|           50000000 |               49999990 |               50000000 |               50000000 |
+--------------------+------------------------+------------------------+------------------------+
1 row in set (2.356 sec)


Doris> select count(log2(db)), count(log2(in_one)) from double_ranges;
+-----------------+---------------------+
| count(log2(db)) | count(log2(in_one)) |
+-----------------+---------------------+
|        50000000 |            50000000 |
+-----------------+---------------------+
1 row in set (1.616 sec)

Doris> select count(log2(db)), count(log2(in_one)) from double_ranges;
+-----------------+---------------------+
| count(log2(db)) | count(log2(in_one)) |
+-----------------+---------------------+
|        50000000 |            50000000 |
+-----------------+---------------------+
1 row in set (0.655 sec)

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@linrrzqqq

Copy link
Copy Markdown
Collaborator Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 17121 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5312f3b362556ee282de569ad926241b89d1b62a, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17575	3062	3107	3062
q2	2136	260	221	221
q3	10187	871	515	515
q4	4663	255	207	207
q5	7671	572	393	393
q6	147	124	98	98
q7	531	519	396	396
q8	9250	871	921	871
q9	3528	2424	2416	2416
q10	6522	841	715	715
q11	408	200	184	184
q12	607	268	202	202
q13	18103	1560	1169	1169
q14	158	153	138	138
q15	q16	435	405	368	368
q17	1411	880	840	840
q18	3142	2309	2272	2272
q19	1138	926	848	848
q20	361	294	202	202
q21	4914	1776	1939	1776
q22	322	268	228	228
Total cold run time: 93209 ms
Total hot run time: 17121 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3434	3386	3384	3384
q2	531	418	358	358
q3	2237	2356	2154	2154
q4	1247	1198	911	911
q5	2213	2140	2115	2115
q6	176	128	92	92
q7	1033	952	868	868
q8	1630	1449	1447	1447
q9	3179	3170	3141	3141
q10	1907	1820	1643	1643
q11	367	273	253	253
q12	463	444	355	355
q13	1485	1559	1143	1143
q14	176	176	163	163
q15	q16	396	399	356	356
q17	3605	3373	3218	3218
q18	4865	4492	4822	4492
q19	1013	898	881	881
q20	1034	954	849	849
q21	3912	3307	3269	3269
q22	406	355	326	326
Total cold run time: 35309 ms
Total hot run time: 31418 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 82502 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5312f3b362556ee282de569ad926241b89d1b62a, data reload: false

query5	4264	411	340	340
query6	406	139	120	120
query7	4924	405	242	242
query8	301	125	115	115
query9	8676	2935	2928	2928
query10	450	244	194	194
query11	5365	1069	923	923
query12	116	72	75	72
query13	1198	456	334	334
query14	6094	2240	2103	2103
query14_1	2008	1988	1987	1987
query15	180	120	110	110
query16	951	384	371	371
query17	827	461	388	388
query18	2346	335	243	243
query19	166	140	115	115
query20	102	73	69	69
query21	214	102	88	88
query22	5440	5490	5332	5332
query23	6876	6309	5981	5981
query23_1	6110	6026	6065	6026
query24	7268	1092	767	767
query24_1	780	778	745	745
query25	441	304	263	263
query26	1233	224	134	134
query27	2778	414	263	263
query28	4678	1502	1509	1502
query29	942	454	359	359
query30	256	162	130	130
query31	845	400	335	335
query32	131	74	79	74
query33	491	225	187	187
query34	977	821	501	501
query35	408	408	347	347
query36	564	556	500	500
query37	120	84	71	71
query38	1003	869	838	838
query39	507	492	469	469
query39_1	461	465	448	448
query40	228	95	83	83
query41	60	58	57	57
query42	78	74	74	74
query43	243	247	220	220
query44	1047	562	594	562
query45	107	105	100	100
query46	769	808	535	535
query47	767	772	738	738
query48	319	313	218	218
query49	552	234	178	178
query50	766	251	186	186
query51	8410	8250	8302	8250
query52	69	67	59	59
query53	204	220	158	158
query54	228	185	176	176
query55	71	58	53	53
query56	195	157	173	157
query57	707	664	648	648
query58	203	166	160	160
query59	1239	1260	1121	1121
query60	228	220	167	167
query61	115	115	118	115
query62	382	202	176	176
query63	177	150	143	143
query64	2715	743	618	618
query65	1593	1571	1577	1571
query66	1782	247	208	208
query67	9764	9676	9599	9599
query68	3077	1172	801	801
query69	352	228	199	199
query70	662	623	623	623
query71	255	175	166	166
query72	2337	1755	1558	1558
query73	700	622	336	336
query74	1993	1234	1149	1149
query75	1190	1110	967	967
query76	2370	737	530	530
query77	259	261	222	222
query78	3953	3855	3250	3250
query79	2786	860	572	572
query80	1616	332	283	283
query81	492	155	131	131
query82	725	121	95	95
query83	276	211	189	189
query84	298	112	92	92
query85	834	351	302	302
query86	390	180	165	165
query87	1037	987	892	892
query88	2798	2123	2095	2095
query89	286	197	182	182
query90	1936	122	131	122
query91	130	121	99	99
query92	82	70	62	62
query93	1728	1079	693	693
query94	650	245	226	226
query95	539	264	305	264
query96	809	542	273	273
query97	1014	1063	1004	1004
query98	172	153	130	130
query99	438	348	319	319
Total cold run time: 179515 ms
Total hot run time: 82502 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants