[hardware] 🐛 Fix two vfirst.m defects in the mask unit (#500) - #502
Open
renzoandri wants to merge 2 commits into
Open
renzoandri wants to merge 2 commits into
renzoandri wants to merge 2 commits into
Conversation
The MASKU index accumulator for `vfirst.m` added the leading-zero count of
every slice unconditionally:
vfirst_count_d = vfirst_count_q + vfirst_count;
`common_cells`' `lzc` documents that for an all-zero input it asserts
`empty_o` and leaves `cnt_o` at "the maximum number of zeros - 1" -- i.e.
`VfirstParallelism - 1`, not `VfirstParallelism`. Every fully-masked slice
therefore advanced the index by one less than the number of elements it
covered, and `vfirst.m` under-reported the result by exactly
`floor(idx / VfirstParallelism)`.
Observed with `VfirstParallelism = 16` (NR_LANES=2, VLEN=2048), sweeping the
position of the single set bit:
idx 16 -> 15 idx 31 -> 30 idx 33 -> 31
idx 17 -> 16 idx 32 -> 30 idx 100 -> 94 idx 255 -> 240
Positions below `VfirstParallelism` are unaffected, which is why the bug is
invisible to short vectors -- including GCC's inline RVV `strlen` expansion on
strings shorter than one slice.
Advance the index by `VfirstParallelism` when the slice is empty.
`vfirst.m` terminated the instruction as soon as the first set bit was found, through three `|| (!vfirst_empty && op == VFIRST)` terms: the two operand-ack sites and the `out_scalar_valid` site. The lanes, however, keep delivering the source-mask operand words for the rest of that `vl`. With the instruction already retired nobody acknowledged them, so they stayed in the per-lane MASKU operand queue and the *next* mask instruction consumed them as its own input. The symptom is that the first mask sequence a program runs is correct and every later one is wrong -- typically 0 -- and with enough accumulated residue the mask unit deadlocks. This is the same class of defect as pulp-platform#448. (pulp-platform#448's `vid.v` drain is not on `main` either: `masku_operands.sv` at 34bd3bc has no VID case; it arrives with PR pulp-platform#469's 69946aa.) Drop the three early-termination terms so `vfirst.m` runs to completion exactly like `vcpop.m`, which has always consumed every slice and has never shown this failure. Consuming every slice *is* the drain. A new `vfirst_found_q` flag carries "the answer is already latched" so the extra slices cannot disturb the result, and the "no set bit in vl" test becomes `!vfirst_found_d` rather than "the slice we stopped on was empty" -- with the early exit gone, the last slice of a *successful* `vfirst.m` is normally empty, so the old test would have returned -1. Cost: `vfirst.m` now always takes ceil(vl/VfirstParallelism) MASKU cycles, the same as `vcpop.m`. Keeping the early exit would need a real drain counter for the operands still in flight. Reproducer: GCC 15's inline RVV `strlen` expansion for rv64gcv (`vle8ff.v` + `vmseq.vi` + `csrr vl` + `vfirst.m`) in a loop. On ara_tb_verilator with NR_LANES=2, VLEN=2048 the first call returns the right length and every later call returns 0.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent
vfirst.mdefects inmasku.sv, one commit each, smallestfirst. Details and reproducers in #500.
lzc'scnt_ounconditionally, but
common_cells'lzcreturnsWIDTH-1(notWIDTH) foran all-zero input, so
vfirst.munder-reported by exactlyfloor(idx / VfirstParallelism). Advance byVfirstParallelismon an emptyslice.
vfirst.mterminated the instruction on the firstset bit, leaving the operand words the lanes were still delivering for the rest
of that
vlin the per-lane MASKU operand queue, where the next maskinstruction consumed them. Same class as masku_operands: drain ALU data during VID #448 — whose
vid.vdrain is not onmaineither; it arrives with 🐛 [masku] Mask-unit correctness & deadlock fixes (#446 #448 #450) #469's69946aa.vfirst.mnow runs tocompletion like
vcpop.m— consuming every slice is the drain — with avfirst_found_qflag so the extra slices cannot disturb the latched answer.Cost of 2/2:
vfirst.malways takesceil(vl/VfirstParallelism)MASKU cycles,the same as
vcpop.malready does. Keeping the early exit would need a realdrain counter for the operands still in flight.
Relation to #469 — no conflict, and #469 is the better base. The two series
touch different code: #469 touches
vfirst.monly at thevcpop_operandassignment (#446's VL trim), while these commits touch the index accumulator, the
two operand-ack conditions and the
out_scalar_validcondition. Verified ratherthan assumed —
git am -3of this series onto #469's head (aee901c7)auto-merges
masku.svwith no conflict; only theCHANGELOG.md### Fixedlistcollides, trivially.
Stacking on #469 is also the tested configuration: the tree these fixes were
characterised on already carried equivalents of #469's two mask-unit hunks (the
#446 VL trim and the #448
vid.vdrain — see the issue for the mapping), somain+ #469 + this series is what was simulated, and this series alone onmainis the untested arrangement. Both apply cleanly either way — say the wordand I'll rebase on #469, or on
mainfirst if you'd rather take theseindependently.
Not #494. That open issue also names
vfirst.m, but it is the missingvstart != 0illegal-instruction check the RVV 1.0 mandatory list requires, andit is fixed at the decode site in
ara_dispatcher.sv. This series does not touchthat file — only
masku.svandCHANGELOG.md— so the two are independent.Changelog
Fixed
vfirst.mindex under-reporting byfloor(idx/VfirstParallelism)onall-zero mask slices
vfirst.mto completion so it drains its MASKU operands, instead ofaborting on the first set bit
Checklist
Verification
ara_tb_verilator,NR_LANES=2,VLEN=2048, verilator 5.044, on the pinab4158ae(whosemasku.svis byte-identical tomainat34bd3bc) with theianfield fork's #446/#448/#451 fixes applied — two of which are #469's mask-unit
hunks, as above. These two commits have not been simulated on plain
mainwithout those, which is the other reason #469 is the base I'd suggest:
over {fault-only-first | plain load} × {mask dest overlaps load dest |
separate} × {
csrr vl| none}, plus a 10-point index sweep — goes fromFAIL/HANG to PASS on every phase, including GCC's own compiled
strlenidiom.The index sweep (set bit at 0, 1, 15, 16, 17, 31, 32, 33, 100, 255) is exact at
every point.
hello_worldrun isbit-identical at 7625 cycles / 3751 instret / IPC 0.491, and an
attention-kernel benchmark reproduces figure-for-figure (fp32 782007 scalar /
71512 vector / 10.94×, int8 94067, max|Δy| 9.28e-4, both bit-exact).
Why the in-tree suite is green today, since that is the first thing to check:
apps/riscv-tests/isa/rv64uv/vfirst.cis the only test that executesvfirst.m, and both its cases run atvl = 4— oneVfirstParallelism = 16slice (a fixed
localparam, not a function ofNrLanes), which reaches neitherdefect. Its all-zero-mask
-1expectation is the one commit 2/2 touches, and itis preserved by construction; the CI run on this PR is what exercises it.
Note the patch adds one flop and changes MASKU control, so it has not been
re-synthesised here; the numbers above are all RTL simulation.
Provenance
Prepared with AI assistance — LLM agents did the defect analysis, drafted the two
patches, and ran the simulation campaign above. I have reviewed the series and the
numbers before sending it. Nothing here rests on taking that on trust: the
reproducers, the exact commands and the pin are all in the linked issue, so every
claim is checkable. Tell me if you would rather any part of it were reworked or
rewritten.