add LSX implementation for vqdmulhq_s16 - #1455
Merged
Merged
Conversation
The 128-bit form had no LSX implementation, so it fell back to
return simde_vcombine_s16(simde_vqdmulh_s16(simde_vget_low_s16(a), simde_vget_low_s16(b)),
simde_vqdmulh_s16(simde_vget_high_s16(a), simde_vget_high_s16(b)));
which splits the vector into two 64-bit halves and recombines them. LSX has
no 64-bit vector type, so every 64-bit intermediate costs a round trip
through memory or a GPR.
This matters for libjpeg-turbo: vqdmulhq_lane_s16 expands to vqdmulhq_s16 and
is the core operation of the fast DCT, used 10 times in jfdctfst and 10 times
in jidctfst, which together account for roughly 25% of the runtime of
tjbench artificial.ppm 95 -dct fast -rgb -subsamp 422
Compute the full 128-bit result directly instead:
hi = vmuh.h(a, b)
lo = vmul.h(a, b)
r = (hi + hi, saturating) | (lo >> 15)
Using a saturating add for the doubling rather than a left shift folds the
saturation in for free: the only product that reaches 2^30 is INT16_MIN *
INT16_MIN, whose low half is zero, so the OR below cannot disturb the clamped
0x7FFF. A plain left shift wraps that case to -32768 instead of saturating to
32767, so apply the same saturating doubling to the 64-bit vqdmulh_s16, which
had the same latent bug.
Measured on Loongson 3A5000HV, libjpeg-turbo built with -DWITH_SIMDE=1 -O2:
jsimd_fdct_ifast_neon 300 -> 188 instructions (-37.3%)
jsimd_idct_ifast_neon 820 -> 700 instructions (-14.6%)
compression +6.5% .. +8.1%
decompression +5.1% .. +5.6%
The qdmulh test vectors never exercised INT16_MIN * INT16_MIN, which is the only product for which the doubling overflows: (2 * a * b) >> 16 reaches 32768 only when both operands are INT16_MIN, since 2^30 is the sole factorisation of that value fitting in int16_t. The result has to saturate to INT16_MAX. qrdmulh already covers this input (see the INT16_MIN * INT16_MIN vector in test/arm/neon/qrdmulh.c), which is why its LSX implementation carries an explicit saturation step while qdmulh's did not: vqdmulh_s16 and vqdmulhq_s16 evaluated ((a * b) >> 16 << 1) | (((a * b) & 0xFFFF) >> 15) with a plain left shift on LoongArch, wrapping this input to -32768. Add the missing vectors to vqdmulh_s16 and vqdmulhq_s16, along with a few other operand extremes. With them the suite fails against the previous LSX code with assertion failed: r[0] == simde_vld1_s16(test_vec[i].r)[0] (-32768 == 32767) and passes once the doubling is performed with a saturating add.
mr-c
approved these changes
Sep 29, 2026
mr-c
enabled auto-merge (rebase)
September 29, 2026 15:57
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.