From bb9fdfe7c6918a086be6edbfa4ba568621d04308 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 5 Nov 2026 12:12:49 -0800 Subject: [PATCH 1/4] Count a lone letter behind a particle run without the run (#1080) rules/C1's numeral site in the family part counted the pieces in front of a one-letter numeral, and an unjoined particle is a piece of its own there, so `SMITH VD V, JOHN` read family 'SMITH VD', suffix 'V', and `van der V, John` family 'van der', suffix 'V'. A run of pieces made wholly of particles right in front of the numeral is now left out of the count, as P9 (a particle that is also a suffix word, beside a family comma) already said: the letter reads as the part without the particle reads it. 1.4.0 through 2.6.0 read these as family; 2.7.0 through 2.8.2 read the suffix (#886). - P9's two deviates: #1080 markers removed - decisions/P9 entry with the grid measurement and local corpora - case rows for the fix and two boundaries (a joined particle surname, a particle behind two name words) - nameparser/AGENTS.md: a count inside a part follows C3 too Co-Authored-By: Claude Opus 5.5 --- ...lso-a-suffix-word-beside-a-family-comma.md | 2 + ...lso-a-suffix-word-beside-a-family-comma.md | 4 +- docs/release_log.rst | 1 + nameparser/AGENTS.md | 2 +- nameparser/_pipeline/_assign.py | 19 +++++++- tests/v2/cases.py | 48 +++++++++++++++++++ tests/v2/test_cases.py | 15 +++--- tools/differential/snapshot.jsonl | 4 +- 8 files changed, 83 insertions(+), 12 deletions(-) diff --git a/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md b/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md index c537ebdfc..21fcc9e95 100644 --- a/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md +++ b/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md @@ -4,6 +4,7 @@ - A particle that is also an unambiguous suffix word, with a name piece behind it in the part before a family comma, heads that name and stays family in any case (`SMITH VD MA, JOHN`) — 2026-10-02 #573. - Such a word standing between post-nominals after a family comma keeps the post-nominal reading in any case and reports the fork it decided (`suffix-or-name`); with nothing behind it, it ends the name and attaches by P6 — 2026-10-02 #573. +- A one-letter numeral behind an unjoined particle run before a family comma is counted without the run: the run is no word to spare, so `SMITH VD V, JOHN` reads as `SMITH V, JOHN` — 2026-11-05 #1080. - A never-given particle opening the given part makes the member behind it a name rather than a credential. This is rules/S5's site, indexed here because this entry is its record (decisions/S5's #573 entry points back here) — 2026-10-02 #573. Open: none in this file. The mixed-case `Doe, Jane PhD vd Ma`, which #573 left aside, was decided silent the same day (decisions/S5's #578) and that was reversed by #602 (decisions/S8's 2026-10-04 entry): it reads suffix 'PhD vd Ma'. @@ -17,3 +18,4 @@ Open: none in this file. The mixed-case `Doe, Jane PhD vd Ma`, which #573 left a NOT DECIDED HERE: the mixed-case `Doe, Jane PhD vd Ma` (middle 'vd Ma', silent), the issue's fourth bullet. Its reading is unchanged; whether the family-comma path should report the chain's fork there is a separate question. Open: #578 (decided the same day, the S2 section's #578 entry: silent, as it reads). Nor the positional read of a part before a comma followed by no name word, where the case still splits the word: `Smith vd Ma, Prof.` reads given 'Smith', family 'vd Ma', and `SMITH VD MA, PROF.` family 'SMITH', suffix 'VD MA', on this tree and on `e56dd296` alike — that part reads as the comma-less name does, and the comma-less name splits the same way. BLAST RADIUS, measured 2026-10-02 with `e56dd296` (the pre-change master) as the comparator. The gate's output is byte-identical at all five baselines apart from the tree path: no corpus name holds these shapes, the rules.md examples aside. Over a generated grid, 5,676 of 409,498 parses move, every one in one of two classes — 3,534 where a family gains the words behind its particle, 2,142 where `vd` or `mc` leaves the family for the credential run — and no list of ambiguity kinds changes; 1,310 of the movers are written in mixed case. Recompute: from the pool `vd mc de van do Ma MA PhD Jr John Berg V la Prof.`, build every `<1 or 2 words>, `, `<3 words>, <1 word>`, `Smith , John`, `Doe, Jane <2-4 words>` and `Smith, <2-4 words>`; deduplicate, then parse each as written, `.upper()` and `.lower()`, keyed by the resulting text; keep the entries whose `as_dict()` or sorted ambiguity kinds differ between the trees, and class each by whether the new family holds every old family word and more, or the old family lost only `vd`/`mc` while the suffix kept every word. Cost: the given-slot test was QUADRATIC as first written, rescanning the particle run once per member — the second review measured 29,670 → 165,591 frames on `SMITH, VD ` + `DE `×k + `MA `×k at k = 256 against `e56dd296`, where an earlier wall-clock check at k = 100 and 400 had read 3.18× and passed for linear. The run's end is now kept once per title set: 1.90–1.99× per doubling of k from 16 to 256, and `_REREAD_SHAPES["lead"]` in tests/v2/test_benchmark.py guards it (generator frames at 8 units against 32: 3.38× kept, 7.53× rescanning, bound 6.0). The frame-count reference band did not move. - 2026-10-19 — SPLIT FROM rules/P2 and rules/P6 with the rule: rules/P2's clause on the part before a family comma and rules/P6's third exception, the two sites #573 decided for a word of both vocabularies. The #573 entry above moved here from decisions/P6, unchanged but for the two site headings, which name the rule each site was in; where it says P2 or P6 elsewhere, it means the rule as it then stood. Its third site, a never-given particle opening the given part, is rules/S5's, which keeps its own cross-reference. +- 2026-11-05, #1080 — A ONE-LETTER NUMERAL BEHIND AN UNJOINED PARTICLE RUN BEFORE A FAMILY COMMA IS COUNTED WITHOUT THE RUN. rules/C1's numeral site in the family part (`_assign._family_part_numeral`, #774, added by #886 for #752's phase 2f) asked S10's words-to-spare count over the pieces in front of the numeral, and a particle that no chain had joined was still a piece of its own there, so it counted as a name word: `SMITH VD V, JOHN` read family 'SMITH VD', suffix 'V', reporting `suffix-or-name`, where `SMITH V, JOHN` keeps family 'SMITH V'; `Smith van der V, John` read family 'Smith van der' and `van der V, John` family 'van der', two particles counted as two words with no name word in front at all. 1.4.0 through 2.6.0 read each as family whole; 2.7.0 through 2.8.2 read the suffix (bisected to 1f0b07c7, #886; no entry recorded the move). DECIDED: a run of pieces made wholly of particles standing right in front of the numeral is left out of the count, as rules/C3 counts a particle run with the word it attaches to as one word, and this clause of P9 already said ("a lone letter behind it reads as it does in the same part without the particle"). The rule's examples carried `deviates: #1080` until this fix. Boundaries: a particle chain joined to a name word is a name word (`Smith van der Berg V, John` keeps suffix 'V', two words in front, as `Smith Jones V, John` does; a first draft tested the piece's first token and moved it), and a run behind two name words leaves two (`Smith Jones van V, John` keeps suffix 'V', reading as `Smith Jones V, John` does, where 1.4.0 and 2.6.0 read family 'Smith Jones van V'). A numeral of two or more letters asks no count and does not move (`Smith van VI, John` keeps suffix 'VI'); the comma-less `John van V` reads suffix 'V' as before, S10's count there being the comma-less name's own. Measured 2026-11-05 against 401bf62e (the pre-change master) on a grid of 27,216 parses: heads `Smith`, `Smith Jones`, `Dr. Smith`, `Freiherr`, `St`, `Smith van der Berg` and none, × particle runs none, `vd`, `van`, `van der`, `mc`, `de la`, `VD`, `Van`, `y`, × numerals `V v I VI Vi V. X II`, × tails `, John`, `, Jr.`, `, J.`, `, John PhD`, ` MD, John`, `, M.J.`, each joined with single spaces, parsed as written, `.upper()` and `.lower()` under all three orders: 2,135 move, every one a particle run directly in front of a one-letter numeral (I, V or X) moving from the suffix to the family, its `suffix-or-name` report dropped; no numeral of two or more letters moves. In the committed corpora only `SMITH VD V, JOHN` and `smith vd v, john` move. The local corpora `fec-individuals` (1,893,964 names, uppercase `FAMILY, GIVEN`), `ctgov-officials` and `tse-donors`, read with `snapshot.py --local --base` a checkout of 401bf62e, move none: the shape is rare in typed data, which is how two releases shipped it unnoticed. diff --git a/docs/design/rules/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md b/docs/design/rules/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md index a16c0d4cf..52512de41 100644 --- a/docs/design/rules/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md +++ b/docs/design/rules/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md @@ -10,8 +10,8 @@ ``` - **A lone letter behind it reads as it does in the same part without the particle**: a capital is a name word ('SMITH VD V, JOHN' as 'SMITH V, JOHN'), a lowercase letter in a name of mixed case the numeral ('Smith vd v, John' as 'Smith v, John'), and a name written in one case reads as its capitals (T4). ``` - "SMITH VD V, JOHN" → family="SMITH VD V" deviates: #1080 (today: family="SMITH VD") - "smith vd v, john" → family="smith vd v" deviates: #1080 (today: family="smith vd") + "SMITH VD V, JOHN" → family="SMITH VD V" + "smith vd v, john" → family="smith vd v" ``` - **In the part after a family comma**, where P6 attaches a particle ending the name to the family, a lone word of both the particle and the UNAMBIGUOUS suffix vocabulary standing INSIDE a run read as post-nominals — one in front of it and another behind, a credential or a generation, whatever made each one, and a title between it and the one in front transparent (H5) — is not the tussenvoegsel: that stands right behind the given name, so a post-nominal between them says the word is not one. It does not end the name, and keeps the post-nominal reading in any case, reported as S4's credential fork. ``` diff --git a/docs/release_log.rst b/docs/release_log.rst index 0f0383fd4..03b2e1c73 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -75,6 +75,7 @@ Release Log - **Fix the surname Mba read as the MBA degree.** ``parse("Chinedu Mba")`` gives family ``Mba``, as 1.4.0 did, where 2.0.0 through 2.8.1 gave suffix ``Mba`` and no family. ``John Smith, MBA`` and ``John Smith MBA`` still read the degree; where a name word can stand, it reads as the name, as ``MA`` does (``Smith, MBA Jane`` gives given ``MBA``), and brackets configured for maiden names read ``(MBA)`` as the maiden name. :issue:`1005` - **Fix the Filipino particle Delos, de los written as one word, reading as a middle name.** ``parse("Juan Delos Reyes")`` gives family ``Delos Reyes``, as ``Juan Dela Cruz`` gives ``Dela Cruz``, where 1.4.0 through 2.8.1 gave middle ``Delos``, family ``Reyes``. ``Delos`` is a given name too, so ``Delos Wheeler`` still gives given ``Delos``, now reporting particle-or-given. Case repair keeps both capitalized: ``parse("JUAN DELA CRUZ").capitalized()`` gives ``Juan Dela Cruz``, where 1.4.0 through 2.8.1 gave ``Juan dela Cruz``. :issue:`1037` - **Fix a family-comma name losing the given name Junior to the suffix.** ``parse("Silva, Junior Carlos")`` gives given ``Junior``, middle ``Carlos``, as 2.2.0 and 2.3.0 did, where 2.4.0 through 2.8.1 gave suffix ``Junior Carlos`` and no given name. Junior and Júnior are given names too, so a name with no word to spare keeps one as the name (``Kofi Junior`` gives family ``Junior``), while ``John Smith Junior`` still gives suffix ``Junior``. A generation word you give a name reading with ``Lexicon.add`` is weighed by the count alone, as ``Junior`` is; add it with ``title_case_is_name=True`` to have Title case read as the name, as ``fils`` does. :issue:`1039` + - **Fix a lone letter behind a surname particle before a comma reading as the generation.** ``parse("Smith van der V, John")`` gives family ``Smith van der V``, as 1.4.0 through 2.6.0 did, where 2.7.0 through 2.8.2 gave family ``Smith van der``, suffix ``V``; ``SMITH VD V, JOHN`` reads as ``SMITH V, JOHN`` does. :issue:`1080` * 2.8.2 - November 4, 2026 diff --git a/nameparser/AGENTS.md b/nameparser/AGENTS.md index 82a41a006..c0a3320f4 100644 --- a/nameparser/AGENTS.md +++ b/nameparser/AGENTS.md @@ -10,7 +10,7 @@ No comma, a FULL name before the comma, a ONE-WORD name before the comma and NOT ### Counting the one-word path -Count the one-word path the way C3 counts, which is neither tokens nor pieces: a particle run and the one word it attaches to are one word (#575), a connective join and a bound given-name pair count as their words (#613, so `Ortega y Gasset, Dr.` is a whole name), and a suffix beside the surname counts as no name word at all -- a numeral of two or more letters (`VI` to `IX`) the trailing fork reads as the generation included, as a listed one is (#893: `Dumas VI, Jr.` lost its family where `Dumas III, Jr.` kept it). #603's review found the suffix half (`Vega y Lopez Jr., PhD Jones` lost its family where `Vega y Lopez, PhD Jones` kept it); #613 then reversed the connective half, and both now read given 'Vega', middle 'y', family 'Lopez' — so test a particle surname, a connective surname and a surname with a suffix, each before a comma. A suffix word OPENING the part is the other half, and counts as a name word where one follows it: from 2.4.0 the counts passed over it, so `JP Clancy, MD` lost its given name to initials that spell a credential and `Cem Akin, MD` to a given name a credential spells, until #1017, while `Md Rahman, PhD` had read title 'Md' or family 'Md Rahman' in every release. No comma grid had put a listed suffix word at the head, and real heads are full of them, so open a head with initials (`JP`, `A.M.`), a given name a credential spells (`Cem`, `Md`), a title/suffix dual (`Sr.`) and a split credential's member (`MB ChB`), behind a title too, and put a title behind the opening word (`Md Prof., PhD`): the first build counted it as the name word there in one count and not in the other, found by review. A bound given-name pair is the same trap from the other side: a count that passes over particles reads `Abu Bakr` as ONE word, `abu` being a particle and a bound word both, where `abdul Rahman` counts two, so a rule asking whether the given name already holds two words needs a bound pair of each spelling in its grid; #1057's grid held only `abdul` until its test review found O7 completing `Abu Bakr` (`_post_rules._one_given_word` counts it now). +Count the one-word path the way C3 counts, which is neither tokens nor pieces: a particle run and the one word it attaches to are one word (#575), a connective join and a bound given-name pair count as their words (#613, so `Ortega y Gasset, Dr.` is a whole name), and a suffix beside the surname counts as no name word at all -- a numeral of two or more letters (`VI` to `IX`) the trailing fork reads as the generation included, as a listed one is (#893: `Dumas VI, Jr.` lost its family where `Dumas III, Jr.` kept it). #603's review found the suffix half (`Vega y Lopez Jr., PhD Jones` lost its family where `Vega y Lopez, PhD Jones` kept it); #613 then reversed the connective half, and both now read given 'Vega', middle 'y', family 'Lopez' — so test a particle surname, a connective surname and a surname with a suffix, each before a comma. The same holds for any count INSIDE a part, not only the comma's: #886's numeral site in the family part counted pieces, and an unjoined particle there is a piece of its own, so `Smith van der V, John` and `SMITH VD V, JOHN` read suffix 'V' from 2.7.0 until #1080, the particles counted as words to spare; put a particle run with nothing behind it but the word being counted in the grid of every count. A suffix word OPENING the part is the other half, and counts as a name word where one follows it: from 2.4.0 the counts passed over it, so `JP Clancy, MD` lost its given name to initials that spell a credential and `Cem Akin, MD` to a given name a credential spells, until #1017, while `Md Rahman, PhD` had read title 'Md' or family 'Md Rahman' in every release. No comma grid had put a listed suffix word at the head, and real heads are full of them, so open a head with initials (`JP`, `A.M.`), a given name a credential spells (`Cem`, `Md`), a title/suffix dual (`Sr.`) and a split credential's member (`MB ChB`), behind a title too, and put a title behind the opening word (`Md Prof., PhD`): the first build counted it as the name word there in one count and not in the other, found by review. A bound given-name pair is the same trap from the other side: a count that passes over particles reads `Abu Bakr` as ONE word, `abu` being a particle and a bound word both, where `abdul Rahman` counts two, so a rule asking whether the given name already holds two words needs a bound pair of each spelling in its grid; #1057's grid held only `abdul` until its test review found O7 completing `Abu Bakr` (`_post_rules._one_given_word` counts it now). ### Orders diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 25653a58a..2a608c8b8 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -775,6 +775,22 @@ def as_a_name(i: int) -> PendingReading: return order +def _names_before(fam_pieces: Sequence[Sequence[int]], + tokens: Sequence[WorkToken], lead: int, k: int) -> int: + """The name words in front of the numeral at `fam_pieces[k]`, the + title run's `lead` pieces left out: a run of particle pieces + standing right in front of the numeral would attach to it were it + a name word, so it is no word to spare (decisions/P9, #1080).""" + # rules/P9: "A lone letter behind it reads as it does in the same + # part without the particle" -- the particles are left out of the + # count, so 'SMITH VD V, JOHN' reads as 'SMITH V, JOHN' does + j = k + while j > lead and all("particle" in tokens[t].tags + for t in fam_pieces[j - 1]): + j -= 1 + return j - lead + + def _family_part_numeral(word: WorkToken, preceding: str, names: int, one_case: bool | None) -> Role | None: """The role of a roman numeral ending the family part before a @@ -1941,7 +1957,8 @@ def as_a_middle(m: int, token: WorkToken) -> PendingReading: for j in range(k + 1, len(fam_pieces))) and (numeral := _family_part_numeral( word, tokens[fam_pieces[k - 1][0]].text, - k - lead, state.one_case)) is not None): + _names_before(fam_pieces, tokens, lead, k), + state.one_case)) is not None): _set_roles(tokens, piece, numeral) # reported where taken: a name word here is the family # name the comma has just named, which nobody misreads diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 362ae3953..df71dfe64 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -2534,6 +2534,54 @@ def _check_cjk_shape_purity(self) -> None: "mixed-case row above does. 2.2.0 and 2.3.0 read family " "'SMITH MA', suffix 'VD'; 2.0.0 and 2.1.0 read as here; " "1.4.0 last 'SMITH', suffix 'VD, MA'"), + # #1080: a one-letter numeral behind such a particle, before a + # family comma. The particle run in front of it is no word to spare. + Case("a_numeral_behind_a_dual_particle_before_a_comma_is_family", + "SMITH VD V, JOHN", + {"given": "JOHN", "family": "SMITH VD V"}, + classification="parity", + notes="P9: a lone letter behind the particle reads as it does " + "in the same part without it, and 'SMITH V, JOHN' keeps " + "family 'SMITH V'. 2.7.0 through 2.8.2 read " + "family 'SMITH VD', suffix 'V', counting the " + "unjoined 'VD' as a name word to spare; 1.4.0 and 2.6.0 " + "read as here"), + Case("a_numeral_behind_a_particle_run_before_a_comma_is_family", + "Smith van der V, John", + {"given": "John", "family": "Smith van der V"}, + classification="parity", + notes="#1080: the run 'van der' would attach to the numeral " + "were it a name word, so only 'Smith' is in front of it, " + "and a one-letter numeral needs two (S10). 2.7.0 through " + "2.8.2 read family 'Smith van der', suffix " + "'V'; 1.4.0 and 2.6.0 read as here"), + Case("a_numeral_behind_only_particles_before_a_comma_is_family", + "van der V, John", + {"given": "John", "family": "van der V"}, + classification="parity", + notes="#1080: no name word stands in front of the numeral, " + "where the count had read the two particles as two. " + "2.7.0 through 2.8.2 read family 'van der', " + "suffix 'V'; 1.4.0 and 2.6.0 read as here"), + Case("a_numeral_behind_a_joined_particle_surname_keeps_the_count", + "Smith van der Berg V, John", + {"given": "John", "family": "Smith van der Berg", + "suffix": "V"}, + ambiguities=("suffix-or-name",), classification="fix(#774)", + notes="#1080's boundary: only particles standing unjoined " + "right in front of the numeral leave the count; 'van der " + "Berg' is a name word, so two stand in front, as in " + "'Smith Jones V, John'. A first draft of #1080 tested the " + "piece's first word and read family 'Smith van der Berg " + "V'. 1.4.0 and 2.6.0 read family 'Smith van der Berg V'"), + Case("a_numeral_behind_a_particle_after_two_names_keeps_the_count", + "Smith Jones van V, John", + {"given": "John", "family": "Smith Jones van", "suffix": "V"}, + ambiguities=("suffix-or-name",), classification="fix(#774)", + notes="#1080's boundary: P9 reads the letter as the part " + "without the particle does, and 'Smith Jones V, John' " + "reads suffix 'V', two name words standing in front. " + "1.4.0 and 2.6.0 read family 'Smith Jones van V'"), Case("a_comma_before_no_name_word_keeps_the_positional_read", "Berg de MA, Prof.", {"title": "Prof.", "given": "Berg", "family": "de", diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 926ea7e42..545d9b693 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -360,10 +360,12 @@ def test_the_field_sweep_sees_the_claims_it_checks() -> None: # rows, three reports each under three orders; 1609 -> 1597 the # same day, #670: six rows opening with a lowercase ambiguous # particle no longer report particle-or-given under the two - # family-first orders, the fold having taken it. - assert claims == 1597, ( - f"the field sweep checks {claims} claims, recorded as 1597 on " - f"2026-11-04") + # family-first orders, the fold having taken it; 1597 -> 1603 on + # 2026-11-05, #1080's two boundary rows, each reporting + # suffix-or-name under three orders. + assert claims == 1603, ( + f"the field sweep checks {claims} claims, recorded as 1603 on " + f"2026-11-05") #: The kinds that report a fork between two readings of a word, so carry @@ -750,8 +752,9 @@ def test_the_alternative_sweep_sees_the_fields_it_checks() -> None: # 495 -> 504 the same day, its test review's two rows, three # reports under the three orders; 504 -> 502 and 330 -> 328 the same day, # #828's 'Freiherr von Berg X.Y.Z. X.Y.Z.' joining _STAND_IN_REREADS - # under the two family-first orders - assert (checked, skipped) == (502, 328), (checked, skipped) + # under the two family-first orders; 502 -> 508 on 2026-11-05, + # #1080's two boundary rows under the three orders + assert (checked, skipped) == (508, 328), (checked, skipped) def test_every_case_name_is_unique() -> None: diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index edcca0cbc..dcd52f728 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -3324,7 +3324,7 @@ {"name": "SEÁN Ó MURCHÚ", "given": "SEÁN", "family": "Ó MURCHÚ", "initials": "S. M.", "capitalized": "Seán Ó Murchú"} {"name": "SIG.RA MARIA ROSSI", "title": "SIG.RA", "given": "MARIA", "family": "ROSSI", "initials": "M. R.", "capitalized": "Sig.ra Maria Rossi"} {"name": "SMITH VD MA, JOHN", "given": "JOHN", "family": "SMITH VD MA", "initials": "J. S. M.", "capitalized": "John Smith vd Ma"} -{"name": "SMITH VD V, JOHN", "given": "JOHN", "family": "SMITH VD", "suffix": "V", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as family (shape)"], "initials": "J. S.", "capitalized": "John Smith vd V", "generation": "V"} +{"name": "SMITH VD V, JOHN", "given": "JOHN", "family": "SMITH VD V", "initials": "J. S. V.", "capitalized": "John Smith vd V"} {"name": "SMITH VD, JOHN", "given": "JOHN", "family": "SMITH", "suffix": "VD", "initials": "J. S.", "capitalized": "John Smith VD"} {"name": "SMITH, DEAN MR.", "title": "MR.", "given": "DEAN", "family": "SMITH", "initials": "D. S.", "capitalized": "Mr. Dean Smith"} {"name": "SMITH, JOHN B. MAJ.", "title": "MAJ.", "given": "JOHN", "middle": "B.", "family": "SMITH", "initials": "J. B. S.", "capitalized": "Maj. John B. Smith"} @@ -4358,7 +4358,7 @@ {"name": "scott e. werner", "given": "scott", "middle": "e.", "family": "werner", "initials": "s. e. w.", "capitalized": "Scott E. Werner"} {"name": "scott l. werner", "given": "scott", "middle": "l.", "family": "werner", "initials": "s. l. w.", "capitalized": "Scott L. Werner"} {"name": "señora María García", "title": "señora", "given": "María", "family": "García", "initials": "M. G.", "capitalized_force": "Señora María García"} -{"name": "smith vd v, john", "given": "john", "family": "smith vd", "suffix": "v", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as family (shape)"], "initials": "j. s.", "capitalized": "John Smith vd V", "generation": "v"} +{"name": "smith vd v, john", "given": "john", "family": "smith vd v", "initials": "j. s. v.", "capitalized": "John Smith vd V"} {"name": "smith, de ma", "family": "smith de ma", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: credential as suffix (vocabulary)"], "initials": "s. m.", "capitalized": "Smith de Ma"} {"name": "smith, j y lopez", "given": "j", "middle": "y lopez", "family": "smith", "initials": "j. l. s.", "capitalized": "J y Lopez Smith"} {"name": "smith, john e phd", "given": "john", "middle": "e", "family": "smith", "suffix": "phd", "initials": "j. e. s.", "capitalized": "John E Smith PhD"} From 9e361e85d6d1cd46ddf96cc20f1c3590d596c35a Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 5 Nov 2026 12:27:55 -0800 Subject: [PATCH 2/4] Count every particle out of the family part's numeral count (#1080) Address the review of the first fix: - _names_before counts the pieces in front of the numeral that are not wholly particles, wherever they stand, so a particle opening the part is no word to spare either ('de Mesnil V, Jean', 'van Smith van V, John' keep the V in the family, as 1.4.0 through 2.6.0 read them) - S10 states the count in its family-comma bullet, with examples, and C1's sentence that the family part reads its numeral as the comma-less name does gains its exception ('John van der V, M.J.') - P9's clause is scoped to a letter ending the part, and its mixed-case example is fenced; the 'vd V Jr.' shape is recorded as not decided - decisions/P9: corrected release history (2.2.0 and 2.3.0 read 'SMITH VD V' otherwise), the grid re-measured (4,398 of 34,992 move, 1,466 per order), and two FEC names that now move recorded - case rows for a title head, a third comma part, the I spelling, a leading particle and paired initials; release-log lead says "family comma"; rule sizes, rules corpus and snapshot re-recorded Co-Authored-By: Claude Opus 5.5 --- ...lso-a-suffix-word-beside-a-family-comma.md | 4 +- docs/design/rules/C1-suffix-comma-decision.md | 3 +- ...lso-a-suffix-word-beside-a-family-comma.md | 3 +- .../rules/S10-trailing-roman-numeral.md | 5 +- docs/release_log.rst | 2 +- nameparser/AGENTS.md | 2 +- nameparser/_pipeline/_assign.py | 26 +++-- tests/v2/cases.py | 98 ++++++++++++++----- tests/v2/rules_budget.json | 6 +- tests/v2/test_cases.py | 10 +- tools/differential/corpus_rules.jsonl | 5 + tools/differential/snapshot.jsonl | 5 + 12 files changed, 115 insertions(+), 54 deletions(-) diff --git a/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md b/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md index 21fcc9e95..c787326a4 100644 --- a/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md +++ b/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md @@ -4,7 +4,7 @@ - A particle that is also an unambiguous suffix word, with a name piece behind it in the part before a family comma, heads that name and stays family in any case (`SMITH VD MA, JOHN`) — 2026-10-02 #573. - Such a word standing between post-nominals after a family comma keeps the post-nominal reading in any case and reports the fork it decided (`suffix-or-name`); with nothing behind it, it ends the name and attaches by P6 — 2026-10-02 #573. -- A one-letter numeral behind an unjoined particle run before a family comma is counted without the run: the run is no word to spare, so `SMITH VD V, JOHN` reads as `SMITH V, JOHN` — 2026-11-05 #1080. +- Before a family comma, a one-letter numeral's words to spare are counted without the part's particles, wherever they stand: a piece made wholly of particles is no word, so `SMITH VD V, JOHN` reads as `SMITH V, JOHN` and `de Mesnil V, Jean` keeps family 'de Mesnil V'. The count is rules/S10's, stated there and in rules/C1's Precedence — 2026-11-05 #1080. - A never-given particle opening the given part makes the member behind it a name rather than a credential. This is rules/S5's site, indexed here because this entry is its record (decisions/S5's #573 entry points back here) — 2026-10-02 #573. Open: none in this file. The mixed-case `Doe, Jane PhD vd Ma`, which #573 left aside, was decided silent the same day (decisions/S5's #578) and that was reversed by #602 (decisions/S8's 2026-10-04 entry): it reads suffix 'PhD vd Ma'. @@ -18,4 +18,4 @@ Open: none in this file. The mixed-case `Doe, Jane PhD vd Ma`, which #573 left a NOT DECIDED HERE: the mixed-case `Doe, Jane PhD vd Ma` (middle 'vd Ma', silent), the issue's fourth bullet. Its reading is unchanged; whether the family-comma path should report the chain's fork there is a separate question. Open: #578 (decided the same day, the S2 section's #578 entry: silent, as it reads). Nor the positional read of a part before a comma followed by no name word, where the case still splits the word: `Smith vd Ma, Prof.` reads given 'Smith', family 'vd Ma', and `SMITH VD MA, PROF.` family 'SMITH', suffix 'VD MA', on this tree and on `e56dd296` alike — that part reads as the comma-less name does, and the comma-less name splits the same way. BLAST RADIUS, measured 2026-10-02 with `e56dd296` (the pre-change master) as the comparator. The gate's output is byte-identical at all five baselines apart from the tree path: no corpus name holds these shapes, the rules.md examples aside. Over a generated grid, 5,676 of 409,498 parses move, every one in one of two classes — 3,534 where a family gains the words behind its particle, 2,142 where `vd` or `mc` leaves the family for the credential run — and no list of ambiguity kinds changes; 1,310 of the movers are written in mixed case. Recompute: from the pool `vd mc de van do Ma MA PhD Jr John Berg V la Prof.`, build every `<1 or 2 words>, `, `<3 words>, <1 word>`, `Smith , John`, `Doe, Jane <2-4 words>` and `Smith, <2-4 words>`; deduplicate, then parse each as written, `.upper()` and `.lower()`, keyed by the resulting text; keep the entries whose `as_dict()` or sorted ambiguity kinds differ between the trees, and class each by whether the new family holds every old family word and more, or the old family lost only `vd`/`mc` while the suffix kept every word. Cost: the given-slot test was QUADRATIC as first written, rescanning the particle run once per member — the second review measured 29,670 → 165,591 frames on `SMITH, VD ` + `DE `×k + `MA `×k at k = 256 against `e56dd296`, where an earlier wall-clock check at k = 100 and 400 had read 3.18× and passed for linear. The run's end is now kept once per title set: 1.90–1.99× per doubling of k from 16 to 256, and `_REREAD_SHAPES["lead"]` in tests/v2/test_benchmark.py guards it (generator frames at 8 units against 32: 3.38× kept, 7.53× rescanning, bound 6.0). The frame-count reference band did not move. - 2026-10-19 — SPLIT FROM rules/P2 and rules/P6 with the rule: rules/P2's clause on the part before a family comma and rules/P6's third exception, the two sites #573 decided for a word of both vocabularies. The #573 entry above moved here from decisions/P6, unchanged but for the two site headings, which name the rule each site was in; where it says P2 or P6 elsewhere, it means the rule as it then stood. Its third site, a never-given particle opening the given part, is rules/S5's, which keeps its own cross-reference. -- 2026-11-05, #1080 — A ONE-LETTER NUMERAL BEHIND AN UNJOINED PARTICLE RUN BEFORE A FAMILY COMMA IS COUNTED WITHOUT THE RUN. rules/C1's numeral site in the family part (`_assign._family_part_numeral`, #774, added by #886 for #752's phase 2f) asked S10's words-to-spare count over the pieces in front of the numeral, and a particle that no chain had joined was still a piece of its own there, so it counted as a name word: `SMITH VD V, JOHN` read family 'SMITH VD', suffix 'V', reporting `suffix-or-name`, where `SMITH V, JOHN` keeps family 'SMITH V'; `Smith van der V, John` read family 'Smith van der' and `van der V, John` family 'van der', two particles counted as two words with no name word in front at all. 1.4.0 through 2.6.0 read each as family whole; 2.7.0 through 2.8.2 read the suffix (bisected to 1f0b07c7, #886; no entry recorded the move). DECIDED: a run of pieces made wholly of particles standing right in front of the numeral is left out of the count, as rules/C3 counts a particle run with the word it attaches to as one word, and this clause of P9 already said ("a lone letter behind it reads as it does in the same part without the particle"). The rule's examples carried `deviates: #1080` until this fix. Boundaries: a particle chain joined to a name word is a name word (`Smith van der Berg V, John` keeps suffix 'V', two words in front, as `Smith Jones V, John` does; a first draft tested the piece's first token and moved it), and a run behind two name words leaves two (`Smith Jones van V, John` keeps suffix 'V', reading as `Smith Jones V, John` does, where 1.4.0 and 2.6.0 read family 'Smith Jones van V'). A numeral of two or more letters asks no count and does not move (`Smith van VI, John` keeps suffix 'VI'); the comma-less `John van V` reads suffix 'V' as before, S10's count there being the comma-less name's own. Measured 2026-11-05 against 401bf62e (the pre-change master) on a grid of 27,216 parses: heads `Smith`, `Smith Jones`, `Dr. Smith`, `Freiherr`, `St`, `Smith van der Berg` and none, × particle runs none, `vd`, `van`, `van der`, `mc`, `de la`, `VD`, `Van`, `y`, × numerals `V v I VI Vi V. X II`, × tails `, John`, `, Jr.`, `, J.`, `, John PhD`, ` MD, John`, `, M.J.`, each joined with single spaces, parsed as written, `.upper()` and `.lower()` under all three orders: 2,135 move, every one a particle run directly in front of a one-letter numeral (I, V or X) moving from the suffix to the family, its `suffix-or-name` report dropped; no numeral of two or more letters moves. In the committed corpora only `SMITH VD V, JOHN` and `smith vd v, john` move. The local corpora `fec-individuals` (1,893,964 names, uppercase `FAMILY, GIVEN`), `ctgov-officials` and `tse-donors`, read with `snapshot.py --local --base` a checkout of 401bf62e, move none: the shape is rare in typed data, which is how two releases shipped it unnoticed. +- 2026-11-05, #1080 — BEFORE A FAMILY COMMA, A ONE-LETTER NUMERAL'S WORDS TO SPARE ARE COUNTED WITHOUT THE PART'S PARTICLES. rules/C1's numeral site in the family part (`_assign._family_part_numeral`, #774, added by #886 for #752's phase 2f) asked S10's words-to-spare count over the pieces in front of the numeral, and a particle no chain had joined to a name word was still a piece of its own there, so it counted as a name word: `SMITH VD V, JOHN` read family 'SMITH VD', suffix 'V', reporting `suffix-or-name`, where `SMITH V, JOHN` keeps family 'SMITH V'; `Smith van der V, John` read family 'Smith van der', `van der V, John` family 'van der', and a particle opening the part counted too, `de Mesnil V, Jean` reading family 'de Mesnil', suffix 'V'. `Smith van der V, John`, `van der V, John` and `de Mesnil V, Jean` read family whole in every release from 1.4.0 through 2.6.0; `SMITH VD V, JOHN` did in 1.4.0 through 2.1.0 and 2.4.0 through 2.6.0, 2.2.0 and 2.3.0 reading family 'SMITH V', suffix 'VD' (#573's entry above), and `smith vd v, john` only in 2.5.0 and 2.6.0, 1.4.0 through 2.4.0 reading family 'smith', suffix 'vd, v' (1.4.0) or 'vd v'. 2.7.0 through 2.8.2 read the suffix for all of them (bisected to 1f0b07c7, #886; no entry recorded the move). DECIDED: a piece made wholly of particles, wherever it stands in the part, is no word to spare. The comma has named the part a surname, where a particle heads the name word behind it, the numeral included were it one, so the particle reads as part of the word it attaches to, as rules/C3 counts a particle run with that word. This is what P9's clause already said for a word of both vocabularies ("A lone letter ending the part behind it reads as it does in the same part without the particle", as the clause now reads), and what the issue's fix asked, "a particle that has a name word behind it"; its examples carried `deviates: #1080` until this fix. A first draft left out only the particles standing right in front of the numeral, and review found `de Mesnil V, Jean` and `van Smith van V, John` still reading the suffix. WHERE IT IS STATED: the count is S10's (its family-comma Precedence bullet), not P9's, since `van` and `der` are no suffix words and P9's When does not reach them; and rules/C1's sentence that "a numeral ending the family part is read as the comma-less name reads it" gains its exception, `John van der V, M.J.` reading family 'John van der V' (1.4.0 through 2.6.0 alike) where the comma-less `John van der V` keeps suffix 'V'. The comma-less name has a given name to spare that a particle may follow; a family part has none. Boundaries: a particle chain joined to a name word is one name word (`Smith van der Berg V, John` keeps suffix 'V', two words in front, as `Smith Jones V, John` does; a draft testing a piece's first token moved it), and particles behind two name words leave two (`Smith Jones van V, John` and `de Mesnil Jones V, Jean` keep suffix 'V', where 1.4.0 and 2.6.0 read `Smith Jones van V, John` family whole). A numeral of two or more letters asks no count and does not move (`Smith van VI, John` keeps suffix 'VI'); the comma-less `John van V` reads suffix 'V' as before; and a part before a suffix comma is not this site, `Smith van der V, Jr.` still reading given 'Smith', family 'van der', suffix 'V, Jr.' as 2.6.0 through 2.8.2 do (1.4.0 read family 'van der V'). NOT DECIDED: with a suffix word behind the letter, P9's first clause (the particle heads the name word behind it) reads the letter as that name word, and the part without the particle reads it as the numeral: `Mary Smith vd V Jr., John` gives family 'Mary Smith vd V', suffix 'Jr.', where `Mary Smith V Jr., John` gives suffix 'V Jr.', on this tree and on 401bf62e alike; P9's clause is scoped to a letter ending the part, and the shape is left as it reads. Measured 2026-11-05 against 401bf62e (the pre-change master; master's `_assign.py` swapped into this tree, the import checked) on a grid of 34,992 parses: heads `Smith`, `Smith Jones`, `Dr. Smith`, `Freiherr`, `St`, `Smith van der Berg`, `de Mesnil`, `van Smith` and none, × particle runs none, `vd`, `van`, `van der`, `mc`, `de la`, `VD`, `Van`, `y`, × numerals `V v I VI Vi V. X II`, × tails `, John`, `, Jr.`, `, J.`, `, John PhD`, ` MD, John`, `, M.J.`; the head, run and numeral joined with single spaces, skipping empty ones, and the tail appended as written, each parsed as written, `.upper()` and `.lower()` under all three orders. 4,398 move, 1,466 under each order, every one a one-letter numeral (I, V or X, either case) moving from the suffix into the family with one fewer `suffix-or-name` report; no numeral of two or more letters moves. In the committed corpora only `SMITH VD V, JOHN` and `smith vd v, john` move. Of the local corpora, read with `snapshot.py --local --base` a checkout of 401bf62e, `ctgov-officials` and `tse-donors` move none, and `fec-individuals` (1,893,964 names, written `FAMILY SUFFIX, GIVEN` in capitals) moves two, each a particle opening the part: `DU HART V, JOHN` from suffix 'V' to family 'DU HART V', and `VAN METER V, ISAAC C IV` from generation 'V, IV' to family 'VAN METER V', generation 'IV'. In FEC's writing both `V`s are most likely generations; the reading is the cost S10's count already takes for a one-word surname (`SMITH V, JOHN` keeps family 'SMITH V' on 401bf62e), accepted here with it, and it is 1.4.0's through 2.6.0's. The first draft, which left out only particles right in front of the numeral, moved none of the three. diff --git a/docs/design/rules/C1-suffix-comma-decision.md b/docs/design/rules/C1-suffix-comma-decision.md index 504db8cb4..18eee3dea 100644 --- a/docs/design/rules/C1-suffix-comma-decision.md +++ b/docs/design/rules/C1-suffix-comma-decision.md @@ -31,12 +31,13 @@ - **The count against written case** ("Smith, MA", "Smith Jr., Ma"): C4 counts the name words before the comma first and asks written case (S6) second, only where the count has left the word a name. - **Paired initials against the count** ("García Márquez, G.J."): C5 is the stated exception to C4's count, since two words before the comma may be one surname written in two. - **An unlisted all-caps word** ("John Smith, XYZ"): C6 decides whether it joins C4's class at all, only where the name carries a case contrast, and once it has joined, C4's count reads it. -- **A generation ending the name before the comma against paired initials after it** ("John Smith Jr., M.J."): written dotted, the initials keep C5's reading and are the given name, the generation staying the suffix; written undotted as two capitals, they are C12's credential, read as the credential they look like. A numeral ending the family part is read as the comma-less name reads it (S10), so the roman numerals S10 reads by their shape stay the suffix as the listed ones do. +- **A generation ending the name before the comma against paired initials after it** ("John Smith Jr., M.J."): written dotted, the initials keep C5's reading and are the given name, the generation staying the suffix; written undotted as two capitals, they are C12's credential, read as the credential they look like. A numeral ending the family part is read as the comma-less name reads it (S10), so the roman numerals S10 reads by their shape stay the suffix as the listed ones do, save that a particle in the part counts as no word to spare there, the comma having named it a surname (S10). ``` "John Smith Jr., M.J." → given="M.J." "John Smith Jr., MJ" → suffix="Jr., MJ" "John Smith V, M.J." → suffix="V" "John Smith VI, M.J." → suffix="VI" + "John van der V, M.J." → family="John van der V" ``` **Accepted:** diff --git a/docs/design/rules/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md b/docs/design/rules/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md index 52512de41..3dc44b40e 100644 --- a/docs/design/rules/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md +++ b/docs/design/rules/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md @@ -8,10 +8,11 @@ ``` "SMITH VD MA, JOHN" → family="SMITH VD MA" ``` -- **A lone letter behind it reads as it does in the same part without the particle**: a capital is a name word ('SMITH VD V, JOHN' as 'SMITH V, JOHN'), a lowercase letter in a name of mixed case the numeral ('Smith vd v, John' as 'Smith v, John'), and a name written in one case reads as its capitals (T4). +- **A lone letter ending the part behind it reads as it does in the same part without the particle**: a capital is a name word ('SMITH VD V, JOHN' as 'SMITH V, JOHN'), a lowercase letter in a name of mixed case the numeral ('Smith vd v, John' as 'Smith v, John'), and a name written in one case reads as its capitals (T4). It is S10's count, which leaves every particle in the part out. ``` "SMITH VD V, JOHN" → family="SMITH VD V" "smith vd v, john" → family="smith vd v" + "Smith vd v, John" → suffix="vd v" ``` - **In the part after a family comma**, where P6 attaches a particle ending the name to the family, a lone word of both the particle and the UNAMBIGUOUS suffix vocabulary standing INSIDE a run read as post-nominals — one in front of it and another behind, a credential or a generation, whatever made each one, and a title between it and the one in front transparent (H5) — is not the tussenvoegsel: that stands right behind the given name, so a post-nominal between them says the word is not one. It does not end the name, and keeps the post-nominal reading in any case, reported as S4's credential fork. ``` diff --git a/docs/design/rules/S10-trailing-roman-numeral.md b/docs/design/rules/S10-trailing-roman-numeral.md index e1c695a49..64544d177 100644 --- a/docs/design/rules/S10-trailing-roman-numeral.md +++ b/docs/design/rules/S10-trailing-roman-numeral.md @@ -92,13 +92,16 @@ "john smith v, PhD" → ambiguities=() "john j v, PhD" → family="j" ``` -- **Before a family comma** ("John Smith V, M.J."): a numeral ending the family part is read by this rule, its words to spare counted in that part (C1). Only a generation its shape alone reads reports there; a numeral the comma leaves in the family name is silent, the comma having named the family. +- **Before a family comma** ("John Smith V, M.J."): a numeral ending the family part is read by this rule, its words to spare counted in that part (C1), a piece made wholly of particles counting as no word: the comma has named the part a surname, where a particle heads the name word behind it, so a particle run reads as part of the word it attaches to, as C3 counts one, and a chain joined to a name word counts as that word. Only a generation its shape alone reads reports there; a numeral the comma leaves in the family name is silent, the comma having named the family. ``` "Smith VI, John" → suffix="VI" "Smith VI, John" → ambiguities=("suffix-or-name",) "Smith V, John" → family="Smith V" "Smith V, John" → ambiguities=() "Smith Vi, John" → ambiguities=() + "Smith van der V, John" → family="Smith van der V" + "de Mesnil V, Jean" → family="de Mesnil V" + "Smith van der Berg V, John" → suffix="V" ``` - **In the given part after a family comma** ("Smith, John VI"): the given part is not counted, the comma having settled the count there (S5), and no initial in front is asked. A numeral of two or more letters reads as the generation, reporting where its shape alone reads it, save a Title-case one in a mixed-case name, which stays a name word in silence; a one-letter numeral is the generation only where the vocabulary lists it and it ends the part (C9), so an unlisted one stays the middle initial. ``` diff --git a/docs/release_log.rst b/docs/release_log.rst index 03b2e1c73..aebcfa099 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -75,7 +75,7 @@ Release Log - **Fix the surname Mba read as the MBA degree.** ``parse("Chinedu Mba")`` gives family ``Mba``, as 1.4.0 did, where 2.0.0 through 2.8.1 gave suffix ``Mba`` and no family. ``John Smith, MBA`` and ``John Smith MBA`` still read the degree; where a name word can stand, it reads as the name, as ``MA`` does (``Smith, MBA Jane`` gives given ``MBA``), and brackets configured for maiden names read ``(MBA)`` as the maiden name. :issue:`1005` - **Fix the Filipino particle Delos, de los written as one word, reading as a middle name.** ``parse("Juan Delos Reyes")`` gives family ``Delos Reyes``, as ``Juan Dela Cruz`` gives ``Dela Cruz``, where 1.4.0 through 2.8.1 gave middle ``Delos``, family ``Reyes``. ``Delos`` is a given name too, so ``Delos Wheeler`` still gives given ``Delos``, now reporting particle-or-given. Case repair keeps both capitalized: ``parse("JUAN DELA CRUZ").capitalized()`` gives ``Juan Dela Cruz``, where 1.4.0 through 2.8.1 gave ``Juan dela Cruz``. :issue:`1037` - **Fix a family-comma name losing the given name Junior to the suffix.** ``parse("Silva, Junior Carlos")`` gives given ``Junior``, middle ``Carlos``, as 2.2.0 and 2.3.0 did, where 2.4.0 through 2.8.1 gave suffix ``Junior Carlos`` and no given name. Junior and Júnior are given names too, so a name with no word to spare keeps one as the name (``Kofi Junior`` gives family ``Junior``), while ``John Smith Junior`` still gives suffix ``Junior``. A generation word you give a name reading with ``Lexicon.add`` is weighed by the count alone, as ``Junior`` is; add it with ``title_case_is_name=True`` to have Title case read as the name, as ``fils`` does. :issue:`1039` - - **Fix a lone letter behind a surname particle before a comma reading as the generation.** ``parse("Smith van der V, John")`` gives family ``Smith van der V``, as 1.4.0 through 2.6.0 did, where 2.7.0 through 2.8.2 gave family ``Smith van der``, suffix ``V``; ``SMITH VD V, JOHN`` reads as ``SMITH V, JOHN`` does. :issue:`1080` + - **Fix a lone letter behind a surname particle before a family comma reading as the generation.** ``parse("Smith van der V, John")`` gives family ``Smith van der V``, as 1.4.0 through 2.6.0 did, where 2.7.0 through 2.8.2 gave family ``Smith van der``, suffix ``V``; ``de Mesnil V, Jean`` keeps family ``de Mesnil V`` too. With two name words in front the letter is still the generation: ``Smith Jones van V, John`` gives suffix ``V``, as ``Smith Jones V, John`` does. :issue:`1080` * 2.8.2 - November 4, 2026 diff --git a/nameparser/AGENTS.md b/nameparser/AGENTS.md index c0a3320f4..0c9a79e1d 100644 --- a/nameparser/AGENTS.md +++ b/nameparser/AGENTS.md @@ -10,7 +10,7 @@ No comma, a FULL name before the comma, a ONE-WORD name before the comma and NOT ### Counting the one-word path -Count the one-word path the way C3 counts, which is neither tokens nor pieces: a particle run and the one word it attaches to are one word (#575), a connective join and a bound given-name pair count as their words (#613, so `Ortega y Gasset, Dr.` is a whole name), and a suffix beside the surname counts as no name word at all -- a numeral of two or more letters (`VI` to `IX`) the trailing fork reads as the generation included, as a listed one is (#893: `Dumas VI, Jr.` lost its family where `Dumas III, Jr.` kept it). #603's review found the suffix half (`Vega y Lopez Jr., PhD Jones` lost its family where `Vega y Lopez, PhD Jones` kept it); #613 then reversed the connective half, and both now read given 'Vega', middle 'y', family 'Lopez' — so test a particle surname, a connective surname and a surname with a suffix, each before a comma. The same holds for any count INSIDE a part, not only the comma's: #886's numeral site in the family part counted pieces, and an unjoined particle there is a piece of its own, so `Smith van der V, John` and `SMITH VD V, JOHN` read suffix 'V' from 2.7.0 until #1080, the particles counted as words to spare; put a particle run with nothing behind it but the word being counted in the grid of every count. A suffix word OPENING the part is the other half, and counts as a name word where one follows it: from 2.4.0 the counts passed over it, so `JP Clancy, MD` lost its given name to initials that spell a credential and `Cem Akin, MD` to a given name a credential spells, until #1017, while `Md Rahman, PhD` had read title 'Md' or family 'Md Rahman' in every release. No comma grid had put a listed suffix word at the head, and real heads are full of them, so open a head with initials (`JP`, `A.M.`), a given name a credential spells (`Cem`, `Md`), a title/suffix dual (`Sr.`) and a split credential's member (`MB ChB`), behind a title too, and put a title behind the opening word (`Md Prof., PhD`): the first build counted it as the name word there in one count and not in the other, found by review. A bound given-name pair is the same trap from the other side: a count that passes over particles reads `Abu Bakr` as ONE word, `abu` being a particle and a bound word both, where `abdul Rahman` counts two, so a rule asking whether the given name already holds two words needs a bound pair of each spelling in its grid; #1057's grid held only `abdul` until its test review found O7 completing `Abu Bakr` (`_post_rules._one_given_word` counts it now). +Count the one-word path the way C3 counts, which is neither tokens nor pieces: a particle run and the one word it attaches to are one word (#575), a connective join and a bound given-name pair count as their words (#613, so `Ortega y Gasset, Dr.` is a whole name), and a suffix beside the surname counts as no name word at all -- a numeral of two or more letters (`VI` to `IX`) the trailing fork reads as the generation included, as a listed one is (#893: `Dumas VI, Jr.` lost its family where `Dumas III, Jr.` kept it). #603's review found the suffix half (`Vega y Lopez Jr., PhD Jones` lost its family where `Vega y Lopez, PhD Jones` kept it); #613 then reversed the connective half, and both now read given 'Vega', middle 'y', family 'Lopez' — so test a particle surname, a connective surname and a surname with a suffix, each before a comma. The same holds for any count INSIDE a part, not only the comma's: #886's numeral site in the family part counted pieces, and an unjoined particle there is a piece of its own, so `Smith van der V, John` and `SMITH VD V, JOHN` read suffix 'V' from 2.7.0 until #1080, the particles counted as words to spare; put a particle run in front of the word being counted, and one opening the part (`de Mesnil V, Jean`, which #1080's first fix still counted), in the grid of every count. A suffix word OPENING the part is the other half, and counts as a name word where one follows it: from 2.4.0 the counts passed over it, so `JP Clancy, MD` lost its given name to initials that spell a credential and `Cem Akin, MD` to a given name a credential spells, until #1017, while `Md Rahman, PhD` had read title 'Md' or family 'Md Rahman' in every release. No comma grid had put a listed suffix word at the head, and real heads are full of them, so open a head with initials (`JP`, `A.M.`), a given name a credential spells (`Cem`, `Md`), a title/suffix dual (`Sr.`) and a split credential's member (`MB ChB`), behind a title too, and put a title behind the opening word (`Md Prof., PhD`): the first build counted it as the name word there in one count and not in the other, found by review. A bound given-name pair is the same trap from the other side: a count that passes over particles reads `Abu Bakr` as ONE word, `abu` being a particle and a bound word both, where `abdul Rahman` counts two, so a rule asking whether the given name already holds two words needs a bound pair of each spelling in its grid; #1057's grid held only `abdul` until its test review found O7 completing `Abu Bakr` (`_post_rules._one_given_word` counts it now). ### Orders diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 2a608c8b8..c8a9cc19e 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -777,25 +777,23 @@ def as_a_name(i: int) -> PendingReading: def _names_before(fam_pieces: Sequence[Sequence[int]], tokens: Sequence[WorkToken], lead: int, k: int) -> int: - """The name words in front of the numeral at `fam_pieces[k]`, the - title run's `lead` pieces left out: a run of particle pieces - standing right in front of the numeral would attach to it were it - a name word, so it is no word to spare (decisions/P9, #1080).""" - # rules/P9: "A lone letter behind it reads as it does in the same - # part without the particle" -- the particles are left out of the - # count, so 'SMITH VD V, JOHN' reads as 'SMITH V, JOHN' does - j = k - while j > lead and all("particle" in tokens[t].tags - for t in fam_pieces[j - 1]): - j -= 1 - return j - lead + """How many name words stand in front of the numeral at + `fam_pieces[k]`, the title run's `lead` pieces left out: the pieces + that are not wholly particles, wherever they stand in the part.""" + # rules/S10: "a piece made wholly of particles counting as no word" + # -- a particle there heads the name word behind it, so it is no + # word to spare; a chain joined to a name word is one piece and + # counts (decisions/P9 (#1080)) + return sum(1 for piece in fam_pieces[lead:k] + if not all("particle" in tokens[t].tags for t in piece)) def _family_part_numeral(word: WorkToken, preceding: str, names: int, one_case: bool | None) -> Role | None: """The role of a roman numeral ending the family part before a - family comma, behind `names` pieces of that part, as the comma-less - name reads one ending it (rules/S10), or None where the numeral's + family comma, behind `names` name words of that part (`_names_before`, + which leaves its particles out), as the comma-less name reads one + ending it (rules/S10), or None where the numeral's predicate reads no numeral there (an initial in front, a letter with its period): the suffix where it reads as the generation, a one-letter numeral needing two name words in front of it, and the diff --git a/tests/v2/cases.py b/tests/v2/cases.py index df71dfe64..b8dea0c9e 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -2534,54 +2534,102 @@ def _check_cjk_shape_purity(self) -> None: "mixed-case row above does. 2.2.0 and 2.3.0 read family " "'SMITH MA', suffix 'VD'; 2.0.0 and 2.1.0 read as here; " "1.4.0 last 'SMITH', suffix 'VD, MA'"), - # #1080: a one-letter numeral behind such a particle, before a - # family comma. The particle run in front of it is no word to spare. + # #1080: a one-letter numeral before a family comma, its words to + # spare counted without the part's particles (S10). Case("a_numeral_behind_a_dual_particle_before_a_comma_is_family", "SMITH VD V, JOHN", {"given": "JOHN", "family": "SMITH VD V"}, classification="parity", - notes="P9: a lone letter behind the particle reads as it does " - "in the same part without it, and 'SMITH V, JOHN' keeps " - "family 'SMITH V'. 2.7.0 through 2.8.2 read " - "family 'SMITH VD', suffix 'V', counting the " - "unjoined 'VD' as a name word to spare; 1.4.0 and 2.6.0 " - "read as here"), + notes="P9: a lone letter ending the part behind the particle " + "reads as it does in the same part without it, and " + "'SMITH V, JOHN' keeps family 'SMITH V'. 2.7.0 through " + "2.8.2 read family 'SMITH VD', suffix 'V', counting the " + "unjoined 'VD' as a name word to spare; 2.2.0 and 2.3.0 " + "family 'SMITH V', suffix 'VD' (#573); 1.4.0 through " + "2.1.0 and 2.4.0 through 2.6.0 read as here"), Case("a_numeral_behind_a_particle_run_before_a_comma_is_family", "Smith van der V, John", {"given": "John", "family": "Smith van der V"}, classification="parity", - notes="#1080: the run 'van der' would attach to the numeral " - "were it a name word, so only 'Smith' is in front of it, " - "and a one-letter numeral needs two (S10). 2.7.0 through " - "2.8.2 read family 'Smith van der', suffix " - "'V'; 1.4.0 and 2.6.0 read as here"), + notes="S10: the run 'van der' is no word to spare, so only " + "'Smith' stands in front of the numeral, and a " + "one-letter numeral needs two. 2.7.0 through 2.8.2 read " + "family 'Smith van der', suffix 'V'; 1.4.0 through 2.6.0 " + "read as here"), Case("a_numeral_behind_only_particles_before_a_comma_is_family", "van der V, John", {"given": "John", "family": "van der V"}, classification="parity", notes="#1080: no name word stands in front of the numeral, " "where the count had read the two particles as two. " - "2.7.0 through 2.8.2 read family 'van der', " - "suffix 'V'; 1.4.0 and 2.6.0 read as here"), + "2.7.0 through 2.8.2 read family 'van der', suffix 'V'; " + "1.4.0 through 2.6.0 read as here"), + Case("a_numeral_behind_a_leading_particle_surname_is_family", + "de Mesnil V, Jean", + {"given": "Jean", "family": "de Mesnil V"}, + classification="parity", + notes="#1080: a particle opening the part heads 'Mesnil' and " + "is no word to spare either, so one name word stands in " + "front of the numeral. 2.7.0 through 2.8.2 read family " + "'de Mesnil', suffix 'V'; 1.4.0 through 2.6.0 read as " + "here"), + Case("a_numeral_behind_particles_on_both_sides_is_family", + "van Smith van V, John", + {"given": "John", "family": "van Smith van V"}, + classification="parity", + notes="#1080: a leading particle and one in front of the " + "numeral both leave the count. 2.7.0 through 2.8.2 read " + "family 'van Smith van', suffix 'V'; 1.4.0 through 2.6.0 " + "read as here"), + Case("a_numeral_behind_a_title_and_a_particle_run_is_family", + "Dr. Smith van der V, John", + {"title": "Dr.", "given": "John", "family": "Smith van der V"}, + classification="fix(#701)", + notes="#1080: the title run is left out of the count as well " + "as the particles. 2.7.0 through 2.8.2 read family " + "'Smith van der', suffix 'V'; 2.6.0 read as here; 1.4.0 " + "through 2.2.0 last 'Dr. Smith van der V', before H6 " + "peeled the title"), + Case("a_numeral_behind_a_particle_run_before_two_commas_is_family", + "Smith van der V, John, Jr.", + {"given": "John", "family": "Smith van der V", "suffix": "Jr."}, + classification="parity", + notes="#1080 with a third comma part. 2.7.0 through 2.8.2 read " + "family 'Smith van der', suffix 'V, Jr.'; 1.4.0 through " + "2.6.0 read as here"), + Case("a_numeral_i_behind_a_particle_run_before_a_comma_is_family", + "Smith van der I, John", + {"given": "John", "family": "Smith van der I"}, + classification="parity", + notes="#1080's spelling sweep: I is a numeral, an initial and " + "a pronoun. 2.7.0 through 2.8.2 read family 'Smith van " + "der', suffix 'I'; 1.4.0 through 2.6.0 read as here"), + Case("a_numeral_behind_a_particle_before_paired_initials_is_family", + "John van der V, M.J.", + {"given": "M.J.", "family": "John van der V"}, + ambiguities=("suffix-or-name",), classification="parity", + notes="C1: the family part reads its numeral as the comma-less " + "name does, save that its particles count as no word " + "(S10), so this part has one name word where 'John Smith " + "V, M.J.' has two. 2.7.0 through 2.8.2 read family 'John " + "van der', suffix 'V'; 1.4.0 through 2.6.0 read as here"), Case("a_numeral_behind_a_joined_particle_surname_keeps_the_count", "Smith van der Berg V, John", {"given": "John", "family": "Smith van der Berg", "suffix": "V"}, ambiguities=("suffix-or-name",), classification="fix(#774)", - notes="#1080's boundary: only particles standing unjoined " - "right in front of the numeral leave the count; 'van der " - "Berg' is a name word, so two stand in front, as in " - "'Smith Jones V, John'. A first draft of #1080 tested the " - "piece's first word and read family 'Smith van der Berg " - "V'. 1.4.0 and 2.6.0 read family 'Smith van der Berg V'"), + notes="#1080's boundary: a chain joined to a name word is one " + "name word, so two stand in front, as in 'Smith Jones V, " + "John'. 1.4.0 through 2.6.0 read family 'Smith van der " + "Berg V'"), Case("a_numeral_behind_a_particle_after_two_names_keeps_the_count", "Smith Jones van V, John", {"given": "John", "family": "Smith Jones van", "suffix": "V"}, ambiguities=("suffix-or-name",), classification="fix(#774)", - notes="#1080's boundary: P9 reads the letter as the part " - "without the particle does, and 'Smith Jones V, John' " - "reads suffix 'V', two name words standing in front. " - "1.4.0 and 2.6.0 read family 'Smith Jones van V'"), + notes="#1080's boundary: the part without the particle is " + "'Smith Jones V, John', which reads suffix 'V', two name " + "words standing in front. 1.4.0 through 2.6.0 read family " + "'Smith Jones van V'"), Case("a_comma_before_no_name_word_keeps_the_positional_read", "Berg de MA, Prof.", {"title": "Prof.", "given": "Berg", "family": "de", diff --git a/tests/v2/rules_budget.json b/tests/v2/rules_budget.json index 57578cb9e..9da0bf841 100644 --- a/tests/v2/rules_budget.json +++ b/tests/v2/rules_budget.json @@ -1,7 +1,7 @@ { "A1": {"words": 195, "then_words": 171, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 1, "accepted_words": 32}, "A2": {"words": 60, "then_words": 31, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, - "C1": {"words": 454, "then_words": 202, "because_sentences": 1, "except": 0, "precedence": 5, "accepted": 1, "accepted_words": 116}, + "C1": {"words": 476, "then_words": 202, "because_sentences": 1, "except": 0, "precedence": 5, "accepted": 1, "accepted_words": 116}, "C2": {"words": 525, "then_words": 432, "because_sentences": 1, "except": 2, "precedence": 0, "accepted": 1, "accepted_words": 81}, "C3": {"words": 477, "then_words": 404, "because_sentences": 2, "except": 0, "precedence": 0, "accepted": 2, "accepted_words": 229}, "C4": {"words": 529, "then_words": 393, "because_sentences": 1, "except": 0, "precedence": 2, "accepted": 0, "accepted_words": 0}, @@ -49,7 +49,7 @@ "P6": {"words": 1086, "then_words": 703, "because_sentences": 2, "except": 2, "precedence": 5, "accepted": 6, "accepted_words": 459}, "P7": {"words": 138, "then_words": 64, "because_sentences": 3, "except": 0, "precedence": 0, "accepted": 1, "accepted_words": 28}, "P8": {"words": 496, "then_words": 431, "because_sentences": 2, "except": 0, "precedence": 0, "accepted": 2, "accepted_words": 309}, - "P9": {"words": 405, "then_words": 265, "because_sentences": 1, "except": 2, "precedence": 0, "accepted": 0, "accepted_words": 0}, + "P9": {"words": 420, "then_words": 280, "because_sentences": 1, "except": 2, "precedence": 0, "accepted": 0, "accepted_words": 0}, "P10": {"words": 425, "then_words": 82, "because_sentences": 3, "except": 5, "precedence": 2, "accepted": 3, "accepted_words": 101}, "P11": {"words": 229, "then_words": 80, "because_sentences": 3, "except": 0, "precedence": 1, "accepted": 2, "accepted_words": 77}, "P12": {"words": 296, "then_words": 87, "because_sentences": 3, "except": 2, "precedence": 0, "accepted": 3, "accepted_words": 116}, @@ -73,7 +73,7 @@ "S7": {"words": 628, "then_words": 388, "because_sentences": 1, "except": 2, "precedence": 2, "accepted": 1, "accepted_words": 105}, "S8": {"words": 806, "then_words": 641, "because_sentences": 1, "except": 1, "precedence": 2, "accepted": 4, "accepted_words": 246}, "S9": {"words": 463, "then_words": 262, "because_sentences": 2, "except": 0, "precedence": 3, "accepted": 0, "accepted_words": 0}, - "S10": {"words": 665, "then_words": 246, "because_sentences": 3, "except": 4, "precedence": 3, "accepted": 3, "accepted_words": 163}, + "S10": {"words": 721, "then_words": 246, "because_sentences": 3, "except": 4, "precedence": 3, "accepted": 3, "accepted_words": 163}, "S11": {"words": 745, "then_words": 466, "because_sentences": 3, "except": 1, "precedence": 1, "accepted": 4, "accepted_words": 247}, "S12": {"words": 766, "then_words": 372, "because_sentences": 2, "except": 3, "precedence": 4, "accepted": 2, "accepted_words": 93}, "S13": {"words": 433, "then_words": 182, "because_sentences": 3, "except": 2, "precedence": 1, "accepted": 1, "accepted_words": 46}, diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 545d9b693..e55b3779c 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -360,11 +360,11 @@ def test_the_field_sweep_sees_the_claims_it_checks() -> None: # rows, three reports each under three orders; 1609 -> 1597 the # same day, #670: six rows opening with a lowercase ambiguous # particle no longer report particle-or-given under the two - # family-first orders, the fold having taken it; 1597 -> 1603 on - # 2026-11-05, #1080's two boundary rows, each reporting - # suffix-or-name under three orders. - assert claims == 1603, ( - f"the field sweep checks {claims} claims, recorded as 1603 on " + # family-first orders, the fold having taken it; 1597 -> 1606 on + # 2026-11-05, #1080's two boundary rows and its paired-initials row, + # each reporting suffix-or-name under three orders. + assert claims == 1606, ( + f"the field sweep checks {claims} claims, recorded as 1606 on " f"2026-11-05") diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 9930778e1..6fbbd90d1 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -741,6 +741,7 @@ "John van der Berg V" "John van der Berg née Jones" "John van der J. V" +"John van der V, M.J." "John von PhD Jones" "John й Smith" "John ｢Jack｣ Smith" @@ -1089,6 +1090,9 @@ "Smith and Pope" "Smith née Jones" "Smith née Jones PhD" +"Smith van der Berg V, John" +"Smith van der V, John" +"Smith vd v, John" "Smith, A.B." "Smith, Abd" "Smith, Chancellor of the Exchequer MP" @@ -1259,6 +1263,7 @@ "de Anke van y" "de Mesnil Jean" "de Mesnil Jean Paul" +"de Mesnil V, Jean" "de la Cruz Juan Carlos" "de la Torre Vega" "de la Vega" diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index dcd52f728..6d405a572 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -2101,6 +2101,7 @@ {"name": "John van der Berg V", "given": "John", "family": "van der Berg", "suffix": "V", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as family (shape)"], "initials": "J. B.", "generation": "V"} {"name": "John van der Berg née Jones", "given": "John", "family": "van der Berg", "maiden": "Jones", "initials": "J. B."} {"name": "John van der J. V", "given": "John", "family": "van der J. V", "initials": "J. J. V."} +{"name": "John van der V, M.J.", "given": "M.J.", "family": "John van der V", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: credential as suffix (shape)"], "initials": "M. J. V."} {"name": "John von PhD Jones", "given": "John", "family": "von", "suffix": "PhD Jones", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: name as family (position)"], "initials": "J. v.", "capitalized_force": "John Von PhD Jones"} {"name": "John y Jane", "given": "John", "middle": "y", "family": "Jane", "initials": "J. y. J."} {"name": "John й Smith", "given": "John", "middle": "й", "family": "Smith", "initials": "J. й. S."} @@ -3535,6 +3536,9 @@ {"name": "Smith née Jones", "family": "Smith", "maiden": "Jones", "initials": "S."} {"name": "Smith née Jones PhD", "family": "Smith", "suffix": "PhD", "maiden": "Jones", "initials": "S."} {"name": "Smith van der", "given": "Smith", "family": "van der", "initials": "S. v. d.", "capitalized_force": "Smith Van Der"} +{"name": "Smith van der Berg V, John", "given": "John", "family": "Smith van der Berg", "suffix": "V", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as family (shape)"], "initials": "J. S. B.", "generation": "V"} +{"name": "Smith van der V, John", "given": "John", "family": "Smith van der V", "initials": "J. S. V."} +{"name": "Smith vd v, John", "given": "John", "family": "Smith", "suffix": "vd v", "initials": "J. S.", "capitalized_force": "John Smith VD V", "generation": "v", "credentials": "vd"} {"name": "Smith 김민준씨, XYZ", "given": "Smith", "middle": "김", "family": "민준", "suffix": "씨, XYZ", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: name as given (position)"], "initials": "S.", "honorifics": "씨"} {"name": "Smith, A.B.", "given": "A.B.", "family": "Smith", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: credential as suffix (shape)"], "initials": "A. S."} {"name": "Smith, A.B.C.", "family": "Smith", "suffix": "A.B.C.", "initials": "S."} @@ -4140,6 +4144,7 @@ {"name": "de Mesnil Jean, Dr.", "order": "FAMILY_FIRST", "title": "Dr.", "given": "Jean", "family": "de Mesnil", "initials": "J. M."} {"name": "de Mesnil Jr.", "family": "de Mesnil", "suffix": "Jr.", "initials": "M.", "generation": "Jr."} {"name": "de Mesnil Thi Mai", "order": "FAMILY_FIRST_GIVEN_LAST", "given": "Mai", "middle": "Thi", "family": "de Mesnil", "initials": "M. T. M."} +{"name": "de Mesnil V, Jean", "given": "Jean", "family": "de Mesnil V", "initials": "J. M. V."} {"name": "de Planta, dr. Margaritta Marionna Maria jr. CF", "order": "FAMILY_FIRST", "title": "dr.", "given": "Margaritta", "middle": "Marionna Maria", "family": "de Planta", "suffix": "jr. CF", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: name as middle (position)"], "initials": "M. M. M. P.", "capitalized_force": "Dr. Margaritta Marionna Maria de Planta Jr. Cf", "generation": "jr.", "credentials": "CF"} {"name": "de Souza e Menezes Mr. Francisco Xavier Joao Anton Dk, MP", "order": "FAMILY_FIRST", "given": "Mr.", "middle": "Francisco Xavier Joao Anton Dk", "family": "de Souza e Menezes", "suffix": "MP", "initials": "M. F. X. J. A. D. S. M."} {"name": "de la Cruz Juan Carlos", "family": "de la Cruz Juan Carlos", "initials": "C. J. C."} From 91acb2761fb212b57a84c67b51b2039d61e70d69 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 5 Nov 2026 12:33:53 -0800 Subject: [PATCH 3/4] Record the FEC moves of #1080's count as decided Co-Authored-By: Claude Opus 5.5 --- ...particle-that-is-also-a-suffix-word-beside-a-family-comma.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md b/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md index c787326a4..cfc791e83 100644 --- a/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md +++ b/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md @@ -18,4 +18,4 @@ Open: none in this file. The mixed-case `Doe, Jane PhD vd Ma`, which #573 left a NOT DECIDED HERE: the mixed-case `Doe, Jane PhD vd Ma` (middle 'vd Ma', silent), the issue's fourth bullet. Its reading is unchanged; whether the family-comma path should report the chain's fork there is a separate question. Open: #578 (decided the same day, the S2 section's #578 entry: silent, as it reads). Nor the positional read of a part before a comma followed by no name word, where the case still splits the word: `Smith vd Ma, Prof.` reads given 'Smith', family 'vd Ma', and `SMITH VD MA, PROF.` family 'SMITH', suffix 'VD MA', on this tree and on `e56dd296` alike — that part reads as the comma-less name does, and the comma-less name splits the same way. BLAST RADIUS, measured 2026-10-02 with `e56dd296` (the pre-change master) as the comparator. The gate's output is byte-identical at all five baselines apart from the tree path: no corpus name holds these shapes, the rules.md examples aside. Over a generated grid, 5,676 of 409,498 parses move, every one in one of two classes — 3,534 where a family gains the words behind its particle, 2,142 where `vd` or `mc` leaves the family for the credential run — and no list of ambiguity kinds changes; 1,310 of the movers are written in mixed case. Recompute: from the pool `vd mc de van do Ma MA PhD Jr John Berg V la Prof.`, build every `<1 or 2 words>, `, `<3 words>, <1 word>`, `Smith , John`, `Doe, Jane <2-4 words>` and `Smith, <2-4 words>`; deduplicate, then parse each as written, `.upper()` and `.lower()`, keyed by the resulting text; keep the entries whose `as_dict()` or sorted ambiguity kinds differ between the trees, and class each by whether the new family holds every old family word and more, or the old family lost only `vd`/`mc` while the suffix kept every word. Cost: the given-slot test was QUADRATIC as first written, rescanning the particle run once per member — the second review measured 29,670 → 165,591 frames on `SMITH, VD ` + `DE `×k + `MA `×k at k = 256 against `e56dd296`, where an earlier wall-clock check at k = 100 and 400 had read 3.18× and passed for linear. The run's end is now kept once per title set: 1.90–1.99× per doubling of k from 16 to 256, and `_REREAD_SHAPES["lead"]` in tests/v2/test_benchmark.py guards it (generator frames at 8 units against 32: 3.38× kept, 7.53× rescanning, bound 6.0). The frame-count reference band did not move. - 2026-10-19 — SPLIT FROM rules/P2 and rules/P6 with the rule: rules/P2's clause on the part before a family comma and rules/P6's third exception, the two sites #573 decided for a word of both vocabularies. The #573 entry above moved here from decisions/P6, unchanged but for the two site headings, which name the rule each site was in; where it says P2 or P6 elsewhere, it means the rule as it then stood. Its third site, a never-given particle opening the given part, is rules/S5's, which keeps its own cross-reference. -- 2026-11-05, #1080 — BEFORE A FAMILY COMMA, A ONE-LETTER NUMERAL'S WORDS TO SPARE ARE COUNTED WITHOUT THE PART'S PARTICLES. rules/C1's numeral site in the family part (`_assign._family_part_numeral`, #774, added by #886 for #752's phase 2f) asked S10's words-to-spare count over the pieces in front of the numeral, and a particle no chain had joined to a name word was still a piece of its own there, so it counted as a name word: `SMITH VD V, JOHN` read family 'SMITH VD', suffix 'V', reporting `suffix-or-name`, where `SMITH V, JOHN` keeps family 'SMITH V'; `Smith van der V, John` read family 'Smith van der', `van der V, John` family 'van der', and a particle opening the part counted too, `de Mesnil V, Jean` reading family 'de Mesnil', suffix 'V'. `Smith van der V, John`, `van der V, John` and `de Mesnil V, Jean` read family whole in every release from 1.4.0 through 2.6.0; `SMITH VD V, JOHN` did in 1.4.0 through 2.1.0 and 2.4.0 through 2.6.0, 2.2.0 and 2.3.0 reading family 'SMITH V', suffix 'VD' (#573's entry above), and `smith vd v, john` only in 2.5.0 and 2.6.0, 1.4.0 through 2.4.0 reading family 'smith', suffix 'vd, v' (1.4.0) or 'vd v'. 2.7.0 through 2.8.2 read the suffix for all of them (bisected to 1f0b07c7, #886; no entry recorded the move). DECIDED: a piece made wholly of particles, wherever it stands in the part, is no word to spare. The comma has named the part a surname, where a particle heads the name word behind it, the numeral included were it one, so the particle reads as part of the word it attaches to, as rules/C3 counts a particle run with that word. This is what P9's clause already said for a word of both vocabularies ("A lone letter ending the part behind it reads as it does in the same part without the particle", as the clause now reads), and what the issue's fix asked, "a particle that has a name word behind it"; its examples carried `deviates: #1080` until this fix. A first draft left out only the particles standing right in front of the numeral, and review found `de Mesnil V, Jean` and `van Smith van V, John` still reading the suffix. WHERE IT IS STATED: the count is S10's (its family-comma Precedence bullet), not P9's, since `van` and `der` are no suffix words and P9's When does not reach them; and rules/C1's sentence that "a numeral ending the family part is read as the comma-less name reads it" gains its exception, `John van der V, M.J.` reading family 'John van der V' (1.4.0 through 2.6.0 alike) where the comma-less `John van der V` keeps suffix 'V'. The comma-less name has a given name to spare that a particle may follow; a family part has none. Boundaries: a particle chain joined to a name word is one name word (`Smith van der Berg V, John` keeps suffix 'V', two words in front, as `Smith Jones V, John` does; a draft testing a piece's first token moved it), and particles behind two name words leave two (`Smith Jones van V, John` and `de Mesnil Jones V, Jean` keep suffix 'V', where 1.4.0 and 2.6.0 read `Smith Jones van V, John` family whole). A numeral of two or more letters asks no count and does not move (`Smith van VI, John` keeps suffix 'VI'); the comma-less `John van V` reads suffix 'V' as before; and a part before a suffix comma is not this site, `Smith van der V, Jr.` still reading given 'Smith', family 'van der', suffix 'V, Jr.' as 2.6.0 through 2.8.2 do (1.4.0 read family 'van der V'). NOT DECIDED: with a suffix word behind the letter, P9's first clause (the particle heads the name word behind it) reads the letter as that name word, and the part without the particle reads it as the numeral: `Mary Smith vd V Jr., John` gives family 'Mary Smith vd V', suffix 'Jr.', where `Mary Smith V Jr., John` gives suffix 'V Jr.', on this tree and on 401bf62e alike; P9's clause is scoped to a letter ending the part, and the shape is left as it reads. Measured 2026-11-05 against 401bf62e (the pre-change master; master's `_assign.py` swapped into this tree, the import checked) on a grid of 34,992 parses: heads `Smith`, `Smith Jones`, `Dr. Smith`, `Freiherr`, `St`, `Smith van der Berg`, `de Mesnil`, `van Smith` and none, × particle runs none, `vd`, `van`, `van der`, `mc`, `de la`, `VD`, `Van`, `y`, × numerals `V v I VI Vi V. X II`, × tails `, John`, `, Jr.`, `, J.`, `, John PhD`, ` MD, John`, `, M.J.`; the head, run and numeral joined with single spaces, skipping empty ones, and the tail appended as written, each parsed as written, `.upper()` and `.lower()` under all three orders. 4,398 move, 1,466 under each order, every one a one-letter numeral (I, V or X, either case) moving from the suffix into the family with one fewer `suffix-or-name` report; no numeral of two or more letters moves. In the committed corpora only `SMITH VD V, JOHN` and `smith vd v, john` move. Of the local corpora, read with `snapshot.py --local --base` a checkout of 401bf62e, `ctgov-officials` and `tse-donors` move none, and `fec-individuals` (1,893,964 names, written `FAMILY SUFFIX, GIVEN` in capitals) moves two, each a particle opening the part: `DU HART V, JOHN` from suffix 'V' to family 'DU HART V', and `VAN METER V, ISAAC C IV` from generation 'V, IV' to family 'VAN METER V', generation 'IV'. In FEC's writing both `V`s are most likely generations; the reading is the cost S10's count already takes for a one-word surname (`SMITH V, JOHN` keeps family 'SMITH V' on 401bf62e), accepted here with it, and it is 1.4.0's through 2.6.0's. The first draft, which left out only particles right in front of the numeral, moved none of the three. +- 2026-11-05 (Derek), #1080 — BEFORE A FAMILY COMMA, A ONE-LETTER NUMERAL'S WORDS TO SPARE ARE COUNTED WITHOUT THE PART'S PARTICLES. rules/C1's numeral site in the family part (`_assign._family_part_numeral`, #774, added by #886 for #752's phase 2f) asked S10's words-to-spare count over the pieces in front of the numeral, and a particle no chain had joined to a name word was still a piece of its own there, so it counted as a name word: `SMITH VD V, JOHN` read family 'SMITH VD', suffix 'V', reporting `suffix-or-name`, where `SMITH V, JOHN` keeps family 'SMITH V'; `Smith van der V, John` read family 'Smith van der', `van der V, John` family 'van der', and a particle opening the part counted too, `de Mesnil V, Jean` reading family 'de Mesnil', suffix 'V'. `Smith van der V, John`, `van der V, John` and `de Mesnil V, Jean` read family whole in every release from 1.4.0 through 2.6.0; `SMITH VD V, JOHN` did in 1.4.0 through 2.1.0 and 2.4.0 through 2.6.0, 2.2.0 and 2.3.0 reading family 'SMITH V', suffix 'VD' (#573's entry above), and `smith vd v, john` only in 2.5.0 and 2.6.0, 1.4.0 through 2.4.0 reading family 'smith', suffix 'vd, v' (1.4.0) or 'vd v'. 2.7.0 through 2.8.2 read the suffix for all of them (bisected to 1f0b07c7, #886; no entry recorded the move). DECIDED: a piece made wholly of particles, wherever it stands in the part, is no word to spare. The comma has named the part a surname, where a particle heads the name word behind it, the numeral included were it one, so the particle reads as part of the word it attaches to, as rules/C3 counts a particle run with that word. This is what P9's clause already said for a word of both vocabularies ("A lone letter ending the part behind it reads as it does in the same part without the particle", as the clause now reads), and what the issue's fix asked, "a particle that has a name word behind it"; its examples carried `deviates: #1080` until this fix. A first draft left out only the particles standing right in front of the numeral, and review found `de Mesnil V, Jean` and `van Smith van V, John` still reading the suffix. WHERE IT IS STATED: the count is S10's (its family-comma Precedence bullet), not P9's, since `van` and `der` are no suffix words and P9's When does not reach them; and rules/C1's sentence that "a numeral ending the family part is read as the comma-less name reads it" gains its exception, `John van der V, M.J.` reading family 'John van der V' (1.4.0 through 2.6.0 alike) where the comma-less `John van der V` keeps suffix 'V'. The comma-less name has a given name to spare that a particle may follow; a family part has none. Boundaries: a particle chain joined to a name word is one name word (`Smith van der Berg V, John` keeps suffix 'V', two words in front, as `Smith Jones V, John` does; a draft testing a piece's first token moved it), and particles behind two name words leave two (`Smith Jones van V, John` and `de Mesnil Jones V, Jean` keep suffix 'V', where 1.4.0 and 2.6.0 read `Smith Jones van V, John` family whole). A numeral of two or more letters asks no count and does not move (`Smith van VI, John` keeps suffix 'VI'); the comma-less `John van V` reads suffix 'V' as before; and a part before a suffix comma is not this site, `Smith van der V, Jr.` still reading given 'Smith', family 'van der', suffix 'V, Jr.' as 2.6.0 through 2.8.2 do (1.4.0 read family 'van der V'). NOT DECIDED: with a suffix word behind the letter, P9's first clause (the particle heads the name word behind it) reads the letter as that name word, and the part without the particle reads it as the numeral: `Mary Smith vd V Jr., John` gives family 'Mary Smith vd V', suffix 'Jr.', where `Mary Smith V Jr., John` gives suffix 'V Jr.', on this tree and on 401bf62e alike; P9's clause is scoped to a letter ending the part, and the shape is left as it reads. Measured 2026-11-05 against 401bf62e (the pre-change master; master's `_assign.py` swapped into this tree, the import checked) on a grid of 34,992 parses: heads `Smith`, `Smith Jones`, `Dr. Smith`, `Freiherr`, `St`, `Smith van der Berg`, `de Mesnil`, `van Smith` and none, × particle runs none, `vd`, `van`, `van der`, `mc`, `de la`, `VD`, `Van`, `y`, × numerals `V v I VI Vi V. X II`, × tails `, John`, `, Jr.`, `, J.`, `, John PhD`, ` MD, John`, `, M.J.`; the head, run and numeral joined with single spaces, skipping empty ones, and the tail appended as written, each parsed as written, `.upper()` and `.lower()` under all three orders. 4,398 move, 1,466 under each order, every one a one-letter numeral (I, V or X, either case) moving from the suffix into the family with one fewer `suffix-or-name` report; no numeral of two or more letters moves. In the committed corpora only `SMITH VD V, JOHN` and `smith vd v, john` move. Of the local corpora, read with `snapshot.py --local --base` a checkout of 401bf62e, `ctgov-officials` and `tse-donors` move none, and `fec-individuals` (1,893,964 names, written `FAMILY SUFFIX, GIVEN` in capitals) moves two, each a particle opening the part: `DU HART V, JOHN` from suffix 'V' to family 'DU HART V', and `VAN METER V, ISAAC C IV` from generation 'V, IV' to family 'VAN METER V', generation 'IV'. In FEC's writing both `V`s are most likely generations; the reading is the cost S10's count already takes for a one-word surname (`SMITH V, JOHN` keeps family 'SMITH V' on 401bf62e), accepted with it, decided on review of the two moves, and it is 1.4.0's through 2.6.0's. The first draft, which left out only particles right in front of the numeral, moved none of the three. From fd66588d36a14666e6b81c1dbd9bc41e27ad7d37 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 5 Nov 2026 12:49:43 -0800 Subject: [PATCH 4/4] File #1080's count under S10 and fix its release history Address the second review of the fix commits: - decisions/S10 gains the Standing line and a Log entry for the count, which is S10's; the full record stays in decisions/P9 - head notes on decisions/C1's #774 entry and vocabulary-model's 2f sites bullet, which restate the count over every piece - release ranges: 'smith vd v, john' ('vd, v' through 2.2.0, 'vd v' in 2.3.0 and 2.4.0), 'Smith van der V, Jr.' (2.2.0 on), and the title row's note (2.5.0 and 2.6.0 read as here) - the release-log bullet says the de Mesnil reading moved, and names the generation it reads as part of a particle surname Co-Authored-By: Claude Opus 5.5 --- docs/design/decisions/C1-suffix-comma-decision.md | 2 +- ...icle-that-is-also-a-suffix-word-beside-a-family-comma.md | 4 ++-- docs/design/decisions/S10-trailing-roman-numeral.md | 2 ++ docs/design/decisions/vocabulary-model.md | 2 +- docs/release_log.rst | 2 +- tests/v2/cases.py | 6 +++--- 6 files changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/design/decisions/C1-suffix-comma-decision.md b/docs/design/decisions/C1-suffix-comma-decision.md index 19fb4064b..b84163138 100644 --- a/docs/design/decisions/C1-suffix-comma-decision.md +++ b/docs/design/decisions/C1-suffix-comma-decision.md @@ -34,5 +34,5 @@ Open: none in this file. SUPERSEDES: decisions/C4's 2026-10-01 #562 bullet's mechanism and its accepted reports (its pair report and the settled-run test); decisions/C13's 2026-10-04 #603 bullet's "segment decides" wording; and decisions/C3's #575 bullet's account of WHY segment builds its unit facts from vocabulary -- `_comma._whole_name` counts over classify's tags, and `name_word_count`, which reads the vocabulary, is asked by decide and by script_segment, the agreement sweep `test_surname_unit_tags_agree_with_classify` keeping the vocabulary view equal to the tags. - 2026-10-18 #728 — SPLIT. rules/C1 held what are now C1 and C3–C11, and this file their history; each entry moved to the file of the rule it decided, and the #429, #436/#437 and #511 entries, which decide R1's entry pass, to decisions/R1. The entries kept here moved nothing but the pointers the split broke and the measurement paragraphs, each cut to one line naming its population, counts, baseline and PR. The #611 entry stays here, an open question on #728. - 2026-10-19 (Derek), #761 — A GENERATION BEFORE THE COMMA AGAINST PAIRED INITIALS AFTER IT: DOTTED, THE INITIALS ARE THE GIVEN NAME AND THE GENERATION STAYS THE SUFFIX; UNDOTTED, THEY ARE THE CREDENTIAL. rules/C12 (a suffix word ending the name before the comma makes the comma a suffix comma) and rules/C5 (paired initials after the comma are the given name) both claim `John Smith Jr., M.J.`, and no rule text said which wins; the C section's move to the skeleton format, which gives a hub Precedence bullets, asked. Measured on 732a2716: `John Smith Jr., M.J.` reads given 'M.J.', family 'John Smith', suffix 'Jr.', and `John Smith Jr., MJ` reads given 'John', family 'Smith', suffix 'Jr., MJ', each reporting `suffix-or-name`. Decided: that reading is the rule, for every generation C12 reads, written in rules/C1's Precedence with both as examples and in C5's and C12's Precedence where they meet. It is the reading C12's Accepted clause (`Lloret Palol II, JA`) and its `Smith Jones Jr., J.A.` boundary already gave behind a two-word surname. The review found a roman numeral S10 reads by shape breaking it today: `John Smith V, M.J.` gives given 'M.J.', family 'John Smith V' and no suffix, while `John Smith V, MJ` reads suffix 'V, MJ'. So does every numeral S10 reads (`John Smith VI, M.J.`, `I`, `IX`, `X` alike). Derek ruled that a bug, so C1's Precedence carries the intended reading as two `deviates:` examples, V for the one-letter count and VI for the shape-only path, tracked by #774. The same pass collected into C1's Precedence the order among C4, C5 and C6 that their own texts already state (C4 counts name words first and asks written case second, C5 is the exception to that count, C6 gates when an unlisted all-caps word joins the class), deciding nothing new there; and the strict switch left C1 for rules/C14 (decisions/C14). -- 2026-10-25 #774 — A NUMERAL ENDING THE FAMILY PART IS READ AS THE COMMA-LESS NAME READS IT. The family part's walk took a word into the suffix by the suffix-piece test alone, so a listed generation stayed the suffix (`John Smith III, M.J.` → suffix 'III') while `V`, which the initial veto refuses, and `VI`, which no list holds, went into the family (`John Smith V, M.J.` → family 'John Smith V'), the split rules/C1's #761 Precedence example tracked as a `deviates:` row. The walk now asks the numeral's predicate (decisions/vocabulary-model's 2026-10-25 2f entry) of a single-word piece shaped as a roman numeral with nothing but suffix pieces behind it: the generation where it reads as one, a one-letter numeral needing two pieces of the part in front of it, as S10's count asks of the comma-less name's pieces, and reported where a shape reads it, as the comma-less name reports. `John Smith V, M.J.` and `John Smith VI, M.J.` keep the numeral as the suffix, `Smith VI, John` reads suffix 'VI', and `Smith V, John` keeps family 'Smith V' (one word in front), silent, the comma having named the family. The two `deviates:` markers come off. +- 2026-10-25 #774 — A NUMERAL ENDING THE FAMILY PART IS READ AS THE COMMA-LESS NAME READS IT. SUPERSEDED IN PART 2026-11-05 (#1080): before a family comma the count is over the pieces that are not wholly particles, not over every piece (decisions/S10's 2026-11-05 entry). The family part's walk took a word into the suffix by the suffix-piece test alone, so a listed generation stayed the suffix (`John Smith III, M.J.` → suffix 'III') while `V`, which the initial veto refuses, and `VI`, which no list holds, went into the family (`John Smith V, M.J.` → family 'John Smith V'), the split rules/C1's #761 Precedence example tracked as a `deviates:` row. The walk now asks the numeral's predicate (decisions/vocabulary-model's 2026-10-25 2f entry) of a single-word piece shaped as a roman numeral with nothing but suffix pieces behind it: the generation where it reads as one, a one-letter numeral needing two pieces of the part in front of it, as S10's count asks of the comma-less name's pieces, and reported where a shape reads it, as the comma-less name reports. `John Smith V, M.J.` and `John Smith VI, M.J.` keep the numeral as the suffix, `Smith VI, John` reads suffix 'VI', and `Smith V, John` keeps family 'Smith V' (one word in front), silent, the comma having named the family. The two `deviates:` markers come off. ACCEPTED, found by the PR's reviews: a part holding a connective surname reads its one-letter numeral as its comma-less twin does only where the two join the connective alike. The part before the comma joins `Vega y Lopez` into one piece where the comma-less `Vega y Lopez V` leaves it unjoined, so `Vega y Lopez V, Mary` keeps family 'Vega y Lopez V' where the twin reads suffix 'V'. Counting a join's words instead fixed that row and broke the longer joins, which both spellings join (`Smith and Jones V, Mary`, `Ortega y Gasset y Lopez V, Mary`, family 'V' comma-less and suffix 'V' with the count), so the piece count stands. Also as on master: `St Smith V, Mary` (a title-particle is no title run before a family comma, rules/H6) and `Jane nee Smith VI, Mary` (the clause keeps the numeral) differ from their twins. diff --git a/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md b/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md index cfc791e83..4975481f5 100644 --- a/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md +++ b/docs/design/decisions/P9-particle-that-is-also-a-suffix-word-beside-a-family-comma.md @@ -4,7 +4,7 @@ - A particle that is also an unambiguous suffix word, with a name piece behind it in the part before a family comma, heads that name and stays family in any case (`SMITH VD MA, JOHN`) — 2026-10-02 #573. - Such a word standing between post-nominals after a family comma keeps the post-nominal reading in any case and reports the fork it decided (`suffix-or-name`); with nothing behind it, it ends the name and attaches by P6 — 2026-10-02 #573. -- Before a family comma, a one-letter numeral's words to spare are counted without the part's particles, wherever they stand: a piece made wholly of particles is no word, so `SMITH VD V, JOHN` reads as `SMITH V, JOHN` and `de Mesnil V, Jean` keeps family 'de Mesnil V'. The count is rules/S10's, stated there and in rules/C1's Precedence — 2026-11-05 #1080. +- Before a family comma, a one-letter numeral's words to spare are counted without the part's particles, wherever they stand: a piece made wholly of particles is no word, so `SMITH VD V, JOHN` reads as `SMITH V, JOHN` and `de Mesnil V, Jean` keeps family 'de Mesnil V'. The count is rules/S10's, stated there and in rules/C1's Precedence, and decisions/S10 carries its Standing line; the record is this file's — 2026-11-05 #1080. - A never-given particle opening the given part makes the member behind it a name rather than a credential. This is rules/S5's site, indexed here because this entry is its record (decisions/S5's #573 entry points back here) — 2026-10-02 #573. Open: none in this file. The mixed-case `Doe, Jane PhD vd Ma`, which #573 left aside, was decided silent the same day (decisions/S5's #578) and that was reversed by #602 (decisions/S8's 2026-10-04 entry): it reads suffix 'PhD vd Ma'. @@ -18,4 +18,4 @@ Open: none in this file. The mixed-case `Doe, Jane PhD vd Ma`, which #573 left a NOT DECIDED HERE: the mixed-case `Doe, Jane PhD vd Ma` (middle 'vd Ma', silent), the issue's fourth bullet. Its reading is unchanged; whether the family-comma path should report the chain's fork there is a separate question. Open: #578 (decided the same day, the S2 section's #578 entry: silent, as it reads). Nor the positional read of a part before a comma followed by no name word, where the case still splits the word: `Smith vd Ma, Prof.` reads given 'Smith', family 'vd Ma', and `SMITH VD MA, PROF.` family 'SMITH', suffix 'VD MA', on this tree and on `e56dd296` alike — that part reads as the comma-less name does, and the comma-less name splits the same way. BLAST RADIUS, measured 2026-10-02 with `e56dd296` (the pre-change master) as the comparator. The gate's output is byte-identical at all five baselines apart from the tree path: no corpus name holds these shapes, the rules.md examples aside. Over a generated grid, 5,676 of 409,498 parses move, every one in one of two classes — 3,534 where a family gains the words behind its particle, 2,142 where `vd` or `mc` leaves the family for the credential run — and no list of ambiguity kinds changes; 1,310 of the movers are written in mixed case. Recompute: from the pool `vd mc de van do Ma MA PhD Jr John Berg V la Prof.`, build every `<1 or 2 words>, `, `<3 words>, <1 word>`, `Smith , John`, `Doe, Jane <2-4 words>` and `Smith, <2-4 words>`; deduplicate, then parse each as written, `.upper()` and `.lower()`, keyed by the resulting text; keep the entries whose `as_dict()` or sorted ambiguity kinds differ between the trees, and class each by whether the new family holds every old family word and more, or the old family lost only `vd`/`mc` while the suffix kept every word. Cost: the given-slot test was QUADRATIC as first written, rescanning the particle run once per member — the second review measured 29,670 → 165,591 frames on `SMITH, VD ` + `DE `×k + `MA `×k at k = 256 against `e56dd296`, where an earlier wall-clock check at k = 100 and 400 had read 3.18× and passed for linear. The run's end is now kept once per title set: 1.90–1.99× per doubling of k from 16 to 256, and `_REREAD_SHAPES["lead"]` in tests/v2/test_benchmark.py guards it (generator frames at 8 units against 32: 3.38× kept, 7.53× rescanning, bound 6.0). The frame-count reference band did not move. - 2026-10-19 — SPLIT FROM rules/P2 and rules/P6 with the rule: rules/P2's clause on the part before a family comma and rules/P6's third exception, the two sites #573 decided for a word of both vocabularies. The #573 entry above moved here from decisions/P6, unchanged but for the two site headings, which name the rule each site was in; where it says P2 or P6 elsewhere, it means the rule as it then stood. Its third site, a never-given particle opening the given part, is rules/S5's, which keeps its own cross-reference. -- 2026-11-05 (Derek), #1080 — BEFORE A FAMILY COMMA, A ONE-LETTER NUMERAL'S WORDS TO SPARE ARE COUNTED WITHOUT THE PART'S PARTICLES. rules/C1's numeral site in the family part (`_assign._family_part_numeral`, #774, added by #886 for #752's phase 2f) asked S10's words-to-spare count over the pieces in front of the numeral, and a particle no chain had joined to a name word was still a piece of its own there, so it counted as a name word: `SMITH VD V, JOHN` read family 'SMITH VD', suffix 'V', reporting `suffix-or-name`, where `SMITH V, JOHN` keeps family 'SMITH V'; `Smith van der V, John` read family 'Smith van der', `van der V, John` family 'van der', and a particle opening the part counted too, `de Mesnil V, Jean` reading family 'de Mesnil', suffix 'V'. `Smith van der V, John`, `van der V, John` and `de Mesnil V, Jean` read family whole in every release from 1.4.0 through 2.6.0; `SMITH VD V, JOHN` did in 1.4.0 through 2.1.0 and 2.4.0 through 2.6.0, 2.2.0 and 2.3.0 reading family 'SMITH V', suffix 'VD' (#573's entry above), and `smith vd v, john` only in 2.5.0 and 2.6.0, 1.4.0 through 2.4.0 reading family 'smith', suffix 'vd, v' (1.4.0) or 'vd v'. 2.7.0 through 2.8.2 read the suffix for all of them (bisected to 1f0b07c7, #886; no entry recorded the move). DECIDED: a piece made wholly of particles, wherever it stands in the part, is no word to spare. The comma has named the part a surname, where a particle heads the name word behind it, the numeral included were it one, so the particle reads as part of the word it attaches to, as rules/C3 counts a particle run with that word. This is what P9's clause already said for a word of both vocabularies ("A lone letter ending the part behind it reads as it does in the same part without the particle", as the clause now reads), and what the issue's fix asked, "a particle that has a name word behind it"; its examples carried `deviates: #1080` until this fix. A first draft left out only the particles standing right in front of the numeral, and review found `de Mesnil V, Jean` and `van Smith van V, John` still reading the suffix. WHERE IT IS STATED: the count is S10's (its family-comma Precedence bullet), not P9's, since `van` and `der` are no suffix words and P9's When does not reach them; and rules/C1's sentence that "a numeral ending the family part is read as the comma-less name reads it" gains its exception, `John van der V, M.J.` reading family 'John van der V' (1.4.0 through 2.6.0 alike) where the comma-less `John van der V` keeps suffix 'V'. The comma-less name has a given name to spare that a particle may follow; a family part has none. Boundaries: a particle chain joined to a name word is one name word (`Smith van der Berg V, John` keeps suffix 'V', two words in front, as `Smith Jones V, John` does; a draft testing a piece's first token moved it), and particles behind two name words leave two (`Smith Jones van V, John` and `de Mesnil Jones V, Jean` keep suffix 'V', where 1.4.0 and 2.6.0 read `Smith Jones van V, John` family whole). A numeral of two or more letters asks no count and does not move (`Smith van VI, John` keeps suffix 'VI'); the comma-less `John van V` reads suffix 'V' as before; and a part before a suffix comma is not this site, `Smith van der V, Jr.` still reading given 'Smith', family 'van der', suffix 'V, Jr.' as 2.6.0 through 2.8.2 do (1.4.0 read family 'van der V'). NOT DECIDED: with a suffix word behind the letter, P9's first clause (the particle heads the name word behind it) reads the letter as that name word, and the part without the particle reads it as the numeral: `Mary Smith vd V Jr., John` gives family 'Mary Smith vd V', suffix 'Jr.', where `Mary Smith V Jr., John` gives suffix 'V Jr.', on this tree and on 401bf62e alike; P9's clause is scoped to a letter ending the part, and the shape is left as it reads. Measured 2026-11-05 against 401bf62e (the pre-change master; master's `_assign.py` swapped into this tree, the import checked) on a grid of 34,992 parses: heads `Smith`, `Smith Jones`, `Dr. Smith`, `Freiherr`, `St`, `Smith van der Berg`, `de Mesnil`, `van Smith` and none, × particle runs none, `vd`, `van`, `van der`, `mc`, `de la`, `VD`, `Van`, `y`, × numerals `V v I VI Vi V. X II`, × tails `, John`, `, Jr.`, `, J.`, `, John PhD`, ` MD, John`, `, M.J.`; the head, run and numeral joined with single spaces, skipping empty ones, and the tail appended as written, each parsed as written, `.upper()` and `.lower()` under all three orders. 4,398 move, 1,466 under each order, every one a one-letter numeral (I, V or X, either case) moving from the suffix into the family with one fewer `suffix-or-name` report; no numeral of two or more letters moves. In the committed corpora only `SMITH VD V, JOHN` and `smith vd v, john` move. Of the local corpora, read with `snapshot.py --local --base` a checkout of 401bf62e, `ctgov-officials` and `tse-donors` move none, and `fec-individuals` (1,893,964 names, written `FAMILY SUFFIX, GIVEN` in capitals) moves two, each a particle opening the part: `DU HART V, JOHN` from suffix 'V' to family 'DU HART V', and `VAN METER V, ISAAC C IV` from generation 'V, IV' to family 'VAN METER V', generation 'IV'. In FEC's writing both `V`s are most likely generations; the reading is the cost S10's count already takes for a one-word surname (`SMITH V, JOHN` keeps family 'SMITH V' on 401bf62e), accepted with it, decided on review of the two moves, and it is 1.4.0's through 2.6.0's. The first draft, which left out only particles right in front of the numeral, moved none of the three. +- 2026-11-05 (Derek), #1080 — BEFORE A FAMILY COMMA, A ONE-LETTER NUMERAL'S WORDS TO SPARE ARE COUNTED WITHOUT THE PART'S PARTICLES. rules/C1's numeral site in the family part (`_assign._family_part_numeral`, #774, added by #886 for #752's phase 2f) asked S10's words-to-spare count over the pieces in front of the numeral, and a particle no chain had joined to a name word was still a piece of its own there, so it counted as a name word: `SMITH VD V, JOHN` read family 'SMITH VD', suffix 'V', reporting `suffix-or-name`, where `SMITH V, JOHN` keeps family 'SMITH V'; `Smith van der V, John` read family 'Smith van der', `van der V, John` family 'van der', and a particle opening the part counted too, `de Mesnil V, Jean` reading family 'de Mesnil', suffix 'V'. `Smith van der V, John`, `van der V, John` and `de Mesnil V, Jean` read family whole in every release from 1.4.0 through 2.6.0; `SMITH VD V, JOHN` did in 1.4.0 through 2.1.0 and 2.4.0 through 2.6.0, 2.2.0 and 2.3.0 reading family 'SMITH V', suffix 'VD' (#573's entry above), and `smith vd v, john` only in 2.5.0 and 2.6.0, 1.4.0 through 2.4.0 reading family 'smith', suffix 'vd, v' (1.4.0 through 2.2.0) or 'vd v' (2.3.0 and 2.4.0). 2.7.0 through 2.8.2 read the suffix for all of them (bisected to 1f0b07c7, #886; no entry recorded the move). DECIDED: a piece made wholly of particles, wherever it stands in the part, is no word to spare. The comma has named the part a surname, where a particle heads the name word behind it, the numeral included were it one, so the particle reads as part of the word it attaches to, as rules/C3 counts a particle run with that word. This is what P9's clause already said for a word of both vocabularies ("A lone letter ending the part behind it reads as it does in the same part without the particle", as the clause now reads), and what the issue's fix asked, "a particle that has a name word behind it"; its examples carried `deviates: #1080` until this fix. A first draft left out only the particles standing right in front of the numeral, and review found `de Mesnil V, Jean` and `van Smith van V, John` still reading the suffix. WHERE IT IS STATED: the count is S10's (its family-comma Precedence bullet), not P9's, since `van` and `der` are no suffix words and P9's When does not reach them; and rules/C1's sentence that "a numeral ending the family part is read as the comma-less name reads it" gains its exception, `John van der V, M.J.` reading family 'John van der V' (1.4.0 through 2.6.0 alike) where the comma-less `John van der V` keeps suffix 'V'. The comma-less name has a given name to spare that a particle may follow; a family part has none. Boundaries: a particle chain joined to a name word is one name word (`Smith van der Berg V, John` keeps suffix 'V', two words in front, as `Smith Jones V, John` does; a draft testing a piece's first token moved it), and particles behind two name words leave two (`Smith Jones van V, John` and `de Mesnil Jones V, Jean` keep suffix 'V', where 1.4.0 and 2.6.0 read `Smith Jones van V, John` family whole). A numeral of two or more letters asks no count and does not move (`Smith van VI, John` keeps suffix 'VI'); the comma-less `John van V` reads suffix 'V' as before; and a part before a suffix comma is not this site, `Smith van der V, Jr.` still reading given 'Smith', family 'van der', suffix 'V, Jr.' as 2.2.0 through 2.8.2 do (1.4.0 through 2.1.0 read family 'van der V'). NOT DECIDED: with a suffix word behind the letter, P9's first clause (the particle heads the name word behind it) reads the letter as that name word, and the part without the particle reads it as the numeral: `Mary Smith vd V Jr., John` gives family 'Mary Smith vd V', suffix 'Jr.', where `Mary Smith V Jr., John` gives suffix 'V Jr.', on this tree and on 401bf62e alike; P9's clause is scoped to a letter ending the part, and the shape is left as it reads. Measured 2026-11-05 against 401bf62e (the pre-change master; master's `_assign.py` swapped into this tree, the import checked) on a grid of 34,992 parses: heads `Smith`, `Smith Jones`, `Dr. Smith`, `Freiherr`, `St`, `Smith van der Berg`, `de Mesnil`, `van Smith` and none, × particle runs none, `vd`, `van`, `van der`, `mc`, `de la`, `VD`, `Van`, `y`, × numerals `V v I VI Vi V. X II`, × tails `, John`, `, Jr.`, `, J.`, `, John PhD`, ` MD, John`, `, M.J.`; the head, run and numeral joined with single spaces, skipping empty ones, and the tail appended as written, each parsed as written, `.upper()` and `.lower()` under all three orders. 4,398 move, 1,466 under each order, every one a one-letter numeral (I, V or X, either case) moving from the suffix into the family with one fewer `suffix-or-name` report; no numeral of two or more letters moves. In the committed corpora only `SMITH VD V, JOHN` and `smith vd v, john` move. Of the local corpora, read with `snapshot.py --local --base` a checkout of 401bf62e, `ctgov-officials` and `tse-donors` move none, and `fec-individuals` (1,893,964 names, written `FAMILY SUFFIX, GIVEN` in capitals) moves two, each a particle opening the part: `DU HART V, JOHN` from suffix 'V' to family 'DU HART V', and `VAN METER V, ISAAC C IV` from generation 'V, IV' to family 'VAN METER V', generation 'IV'. In FEC's writing both `V`s are most likely generations; the reading is the cost S10's count already takes for a one-word surname (`SMITH V, JOHN` keeps family 'SMITH V' on 401bf62e), accepted with it, decided on review of the two moves, and it is 1.4.0's through 2.6.0's. The first draft, which left out only particles right in front of the numeral, moved none of the three. diff --git a/docs/design/decisions/S10-trailing-roman-numeral.md b/docs/design/decisions/S10-trailing-roman-numeral.md index 0a6b08d3f..76a5049b0 100644 --- a/docs/design/decisions/S10-trailing-roman-numeral.md +++ b/docs/design/decisions/S10-trailing-roman-numeral.md @@ -3,6 +3,7 @@ ## Standing - A one-letter roman numeral is the generation only with words to spare, counted in name units as S4 counts: `John Smith V` → suffix 'V', while `John V` and `Malcolm X` keep the letter as the family; a title is no word to spare (`Dr Smith V` → family 'V') and a particle chain is one word (`Freiherr von Richthofen V` → family 'von Richthofen V') — 2026-10-17 #718. +- Before a family comma, a numeral ending the family part counts as words to spare only the pieces in front of it that are not wholly particles, wherever they stand: `SMITH VD V, JOHN`, `Smith van der V, John` and `de Mesnil V, Jean` keep the letter in the family, the comma-less name's count no longer the family part's there (rules/C1's exception) — 2026-11-05 #1080. - A lowercase numeral in a mixed-case name is the generation and reports nothing (`John Smith vi`); a name written in one case is counted as its capitals are — 2026-10-17 #718; 2026-10-18 #721. - The declined numeral reports `suffix-or-name` as the taken one does, assign alone reporting it — 2026-10-18 #718; 2026-10-18 #718's PR review. - The count is not asked in the given part after a family comma: `Smith, Freiherr von Berg V` keeps suffix 'V' — 2026-10-18 #718 review. @@ -42,3 +43,4 @@ Open: none in this file. - 2026-10-19 (Derek), #761 — THE RARITY PREMISE LEAVES THE RATIONALE, AND THE SUFFIX-COMMA PRECEDENCE NAMES C12. The Because no longer says the generation a one-letter numeral would name is rare: the one-letter reading rests on the words-to-spare count (#722 above), not on rarity. The Precedence bullet for the part before a suffix comma had said it reads as the same name written without the comma (C1); that was false for a one-case part before a capitalized credential (`john j v, PhD` reads family 'j', suffix 'v, PhD', where `john j v` reads family 'v'; 2.5.0 the same). That reading is C12 deciding the comma and written case being asked of the whole name, credential included, so the part belongs to a name of mixed case (T4); the bullet now says so and carries both shapes as examples. No reading moved. - 2026-10-25 #752 — EVERY NUMERAL RULE ASKS ONE PREDICATE, AND THE ONE-LETTER NUMERAL BEFORE A CREDENTIAL STAYS A NAME. Derek agreed on 2026-10-23 that every rule weighing a numeral asks one predicate, answering for `i` to `x` as this rule's fork does (decisions/vocabulary-model's 2026-10-23 #752 entry, item 8); the sites and the measurements are that file's 2026-10-25 #752, phase 2f entry. For this rule: the fork reads a numeral of two or more letters anywhere in the trailing run, not only as its last word, so `John Smith VI MD` reads suffix 'VI MD' as the listed `III MD` did (#784); a numeral of two or more letters keeps its reading written with a period, as `III.` did (`Georg VI.`, the German ordinal, and GND's two other regnal rows read suffix where they read family); a Title-case numeral in a mixed-case name leans to the name and reports, the third reading agreed (`Nguyen Van Vi`, 1.4.0's fields; PSC's three Vietnamese `Vi` surnames fixed, one `Viii` written as a generation broken). A numeral the vocabulary lists in two or more letters is not weighed: neither an initial in front nor Title case moves it (`John J. III`, `John Smith Iii` keep the suffix). KEPT: the #721 reading of a one-letter numeral before a credential (`John Smith V PhD` → family 'V', reported). Read with the count instead, as the predicate reads one ending the name, it moved 6 tse-donors names, all wrong (a lone letter before `JR` abbreviating a surname), and the committed rows `John Smith V PhD` and its maiden twins; so the peel reads a one-letter numeral only as the walk's last piece, as before. +- 2026-11-05 (Derek), #1080 — BEFORE A FAMILY COMMA, A PIECE MADE WHOLLY OF PARTICLES IS NO WORD TO SPARE. The count of the family part's numeral site (rules/C1, #774) counted every piece in front of a one-letter numeral, so an unjoined particle counted as a name word, from 2.7.0 (#886): `Smith van der V, John` read suffix 'V'. It now counts only the pieces that are not wholly particles, as the clause added to this rule's family-comma Precedence bullet states, and C1's sentence that the family part reads its numeral as the comma-less name does gains the exception. The full record (release history, boundaries, the grid, and the two FEC names whose `V` now reads as the family, accepted as the cost) is decisions/P9's 2026-11-05 entry, filed there because the issue arose from P9's clause. diff --git a/docs/design/decisions/vocabulary-model.md b/docs/design/decisions/vocabulary-model.md index 4a4b08cfa..aae469dea 100644 --- a/docs/design/decisions/vocabulary-model.md +++ b/docs/design/decisions/vocabulary-model.md @@ -215,7 +215,7 @@ Open: #753 (phase 3, language packs; the 16 words strict ungates, each read for - **Measured neutral** against the branch, first on 632685c8 and again on 2dd0414b once phase 2a had merged: a grid of 20,364 texts (22 heads covering no comma, a whole name, one word and nothing before a comma, a nickname-only head, a maiden clause, a family comma with a title-particle head and a quoted clause before the pair; 33 spellings including `MA`, `M.D.`, `X.Y.Z.`, `XYZ`, `Ph. D.` and its case variants, `V`, `VI`, `III`, `XI`, `M.J.`, `K.C.` and `พ.บ.`; each text also in lower and upper case), under all three orders, read identically (fields, ambiguity kinds and details, token roles and tags, `initials()`, `capitalized()`), and the same harness reported 45,871, 18, 3,974 and 2,986 differing readings with `split_pair`, C12's call to the numeral predicate, the cased half and the peel's call to it each knocked out. The old and new predicates gave the same answer on 19,769 direct calls. The committed snapshot, every CLDR, GND, Wikidata and conventions agreement record, psc-individuals' agreement record and the ctgov-officials, fec-individuals and tse-donors local snapshots (each recorded from the base with `--tree`) all match, on both bases. psc-individuals' record was itself stale against 632685c8 (3 regressions, 5 progress, 5 changed before this change), so it was re-recorded from each base first. - **Frames**, counted with `sys.setprofile` on both trees: `_CALL_BASELINE`'s reference name and `Smith, John` cost what they did (305 and 158). A word matching `SPLIT_PAIR_OPENS` (`Ph`, `ph.`) with a word after it pays one frame at each site that tests it, whether or not a `D.` follows (`John Smith Ph. D.` 233 → 234, `Smith, John Ph. D.` 239 → 241; the review counted `ph ph ph ph ph ph` 282 → 286), so the cost grows only with the number of such words. C12's evidence keeps its tag test ahead of the call, so it costs what it did (`John Smith Jr., AD` 372). That is a rare shape that no guard counts. - **2026-10-25 #752, phase 2f — EVERY NUMERAL RULE ASKS ONE PREDICATE.** (Its expectation that the split-credential sub-phase keep `John Smith V Ph. D.` as suffix 'V Ph. D.' superseded by the phase 2e entry below, decisions/S12, which reads it as `John Smith V PhD` reads, family 'V'.) Carries out item 8 of the 2026-10-23 #752 entry above (#784, #774, #819), and the list of sites in the 2026-10-25 one-predicate entry. `_vocab.reads_as_generation(text, tags, preceding, one_case)` answers for every spelling from `i` to `x` as S10's trailing fork does: the shape on the word without its trailing periods; a single letter written with its period an initial (rules/T5); a numeral the vocabulary lists in two or more letters (`ii` to `iv`, read from the token's `vocab:suffix` tag, not copied at a call site) the generation whatever stands around it; for the rest, an initial in front a name word, and a Title-case numeral in a name of mixed case declined, which `numeral_leans_to_the_name` tells apart so the decline is reported. `preceding` is None where a slot asks no initial in front, the given part after a family comma, where the predicate reads a one-letter numeral by the vocabulary. The count for one letter stays the caller's, which alone knows where the name's words are. - - **The sites.** The trailing peel asks it of a numeral anywhere in the run, behind the suffix test, so `John Smith VI MD` reads suffix 'VI MD' as `III MD` does (#784); rules/S8's run start asks it of a numeral of two or more letters (`John Smith VI Jones` → suffix 'VI Jones'); rules/C12's evidence asks it with the token's tags, its broader suffix-word test still beside the call; C1's listing form asks it of a numeral ending the family part, with S10's count over that part's pieces (`John Smith V, M.J.` and `John Smith VI, M.J.` keep the suffix, #774); the given part's walk asks it of a numeral of two or more letters (`Smith, John VI` → suffix 'VI') and of the bare one-letter numeral its last-piece test reads, and a one-letter numeral behind a suffix word in a part of suffix words asks it too; the maiden take and `tail_reading` reach it through the peel, `trailing_candidates` with no case (its superset); the report on a numeral a post-nominal follows asks it, with the Title-case decline. A numeral of two or more letters read by its shape alone reports `suffix-or-name` wherever it is taken, the run start and the given part included. + - **The sites.** SUPERSEDED IN PART 2026-11-05 (#1080): before a family comma the count is over the pieces that are not wholly particles, not over every piece (decisions/S10's 2026-11-05 entry). The trailing peel asks it of a numeral anywhere in the run, behind the suffix test, so `John Smith VI MD` reads suffix 'VI MD' as `III MD` does (#784); rules/S8's run start asks it of a numeral of two or more letters (`John Smith VI Jones` → suffix 'VI Jones'); rules/C12's evidence asks it with the token's tags, its broader suffix-word test still beside the call; C1's listing form asks it of a numeral ending the family part, with S10's count over that part's pieces (`John Smith V, M.J.` and `John Smith VI, M.J.` keep the suffix, #774); the given part's walk asks it of a numeral of two or more letters (`Smith, John VI` → suffix 'VI') and of the bare one-letter numeral its last-piece test reads, and a one-letter numeral behind a suffix word in a part of suffix words asks it too; the maiden take and `tail_reading` reach it through the peel, `trailing_candidates` with no case (its superset); the report on a numeral a post-nominal follows asks it, with the Title-case decline. A numeral of two or more letters read by its shape alone reports `suffix-or-name` wherever it is taken, the run start and the given part included. - **The three readings agreed on 2026-10-23.** #819: a lone letter ending the given part ahead of a further comma is the middle initial, the credential run in front stopping short of it (decisions/C9's 2026-10-25 #819 entry; `Smith, John PSM I, Jr.` the accepted cost). FEC's constraint: in that part a one-letter numeral is the generation only where listed, so `SMITH, JOHN X` keeps middle 'X'. A Title-case numeral in a mixed-case name leans to the name (`Nguyen Van Vi` → family 'Van Vi', reported); a numeral the vocabulary lists in two or more letters is not leaned (`John Smith Iii` keeps suffix 'Iii'), being unambiguous vocabulary as an unambiguous credential is. - **One reading kept rather than widened: a one-letter numeral before a credential stays a name word** (decisions/S10's #721 entry, `John Smith V PhD` → family 'V', reported). Reading it with the count instead moved 6 tse-donors names, every one wrong: a lone `V` or `I` behind a surname or a particle before `JR` or `JUNIOR` (shape `Aa A Aa de V JUNIOR` read family 'de', suffix 'V JUNIOR'), where the letter abbreviates a surname; and 3 committed names (`John Smith V PhD`, `Jane Doe nee Smith V PhD`, `Jane Doe nee Smith V MA`). A split credential behind such a numeral (`John Smith V Ph. D.`) is left to the split-credential sub-phase (item 6), which must keep its suffix 'V Ph. D.' by its own rule. - **Measured** (2026-10-25, against master 2af8da8e, each local baseline re-recorded from that checkout with `--tree` immediately before its check, since the shared cache's `snapshot.jsonl` is overwritten by any session measuring in parallel): the committed snapshot moves 6 names, every one intended: the three GND regnal rows (`Georg VI.`, `Konstantin VII.`, `Bonifatius VIII.`, family → suffix), `Smith VI, AD` (family 'Smith VI' → 'Smith', suffix 'VI'), and #774's `John Smith V, M.J.` and `John Smith VI, M.J.`. GND agreement goes from 618 to 621 of 801 on family and 779 to 782 on suffix, three progress rows each, and every other agreement record matches. ctgov-officials moves 1 reading (a job title ending in a level before a comma, `Aa Aa I, A` → suffix 'I', reported); fec-individuals 22, each a numeral ending the given part or the family part (`A, A VI` → suffix 'VI'; `A VI, A` → suffix 'VI'; `A A V, A` → suffix 'V'), about 14 read as generations and about 8 a middle name or a fragment read as one, all reported; tse-donors none. psc-individuals (1,039,171 records scored): 3 fixed, all Vietnamese names ending in the surname `Vi` that the Title-case lean keeps a name; 1 broken, a Title-case `Viii` written as the bearer's generation, which the lean reads as a name (the lean's cost, rules/S10); 1 changed, a record whose title box the scorer sets aside. Spike figures above (listing `vi` to `x`: 152 FEC moves) are the 2026-10-23 entry's and were not re-taken. diff --git a/docs/release_log.rst b/docs/release_log.rst index aebcfa099..0d9fe7903 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -75,7 +75,7 @@ Release Log - **Fix the surname Mba read as the MBA degree.** ``parse("Chinedu Mba")`` gives family ``Mba``, as 1.4.0 did, where 2.0.0 through 2.8.1 gave suffix ``Mba`` and no family. ``John Smith, MBA`` and ``John Smith MBA`` still read the degree; where a name word can stand, it reads as the name, as ``MA`` does (``Smith, MBA Jane`` gives given ``MBA``), and brackets configured for maiden names read ``(MBA)`` as the maiden name. :issue:`1005` - **Fix the Filipino particle Delos, de los written as one word, reading as a middle name.** ``parse("Juan Delos Reyes")`` gives family ``Delos Reyes``, as ``Juan Dela Cruz`` gives ``Dela Cruz``, where 1.4.0 through 2.8.1 gave middle ``Delos``, family ``Reyes``. ``Delos`` is a given name too, so ``Delos Wheeler`` still gives given ``Delos``, now reporting particle-or-given. Case repair keeps both capitalized: ``parse("JUAN DELA CRUZ").capitalized()`` gives ``Juan Dela Cruz``, where 1.4.0 through 2.8.1 gave ``Juan dela Cruz``. :issue:`1037` - **Fix a family-comma name losing the given name Junior to the suffix.** ``parse("Silva, Junior Carlos")`` gives given ``Junior``, middle ``Carlos``, as 2.2.0 and 2.3.0 did, where 2.4.0 through 2.8.1 gave suffix ``Junior Carlos`` and no given name. Junior and Júnior are given names too, so a name with no word to spare keeps one as the name (``Kofi Junior`` gives family ``Junior``), while ``John Smith Junior`` still gives suffix ``Junior``. A generation word you give a name reading with ``Lexicon.add`` is weighed by the count alone, as ``Junior`` is; add it with ``title_case_is_name=True`` to have Title case read as the name, as ``fils`` does. :issue:`1039` - - **Fix a lone letter behind a surname particle before a family comma reading as the generation.** ``parse("Smith van der V, John")`` gives family ``Smith van der V``, as 1.4.0 through 2.6.0 did, where 2.7.0 through 2.8.2 gave family ``Smith van der``, suffix ``V``; ``de Mesnil V, Jean`` keeps family ``de Mesnil V`` too. With two name words in front the letter is still the generation: ``Smith Jones van V, John`` gives suffix ``V``, as ``Smith Jones V, John`` does. :issue:`1080` + - **Fix a lone letter behind a surname particle before a family comma reading as the generation.** ``parse("Smith van der V, John")`` gives family ``Smith van der V``, as 1.4.0 through 2.6.0 did, where 2.7.0 through 2.8.2 gave family ``Smith van der``, suffix ``V``; ``de Mesnil V, Jean`` gives family ``de Mesnil V`` too, where 2.7.0 through 2.8.2 gave suffix ``V``. A generation written before the comma after a particle surname is read as part of it, as after a one-word surname (``Du Hart V, John`` gives family ``Du Hart V``, as ``Smith V, John`` gives ``Smith V``); with two name words in front the letter is still the generation (``Smith Jones van V, John`` gives suffix ``V``). :issue:`1080` * 2.8.2 - November 4, 2026 diff --git a/tests/v2/cases.py b/tests/v2/cases.py index b8dea0c9e..14ac664de 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -2587,9 +2587,9 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#701)", notes="#1080: the title run is left out of the count as well " "as the particles. 2.7.0 through 2.8.2 read family " - "'Smith van der', suffix 'V'; 2.6.0 read as here; 1.4.0 " - "through 2.2.0 last 'Dr. Smith van der V', before H6 " - "peeled the title"), + "'Smith van der', suffix 'V'; 2.5.0 and 2.6.0 read as " + "here; 1.4.0 through 2.4.0 family 'Dr. Smith van der V', " + "before H6 peeled the title"), Case("a_numeral_behind_a_particle_run_before_two_commas_is_family", "Smith van der V, John, Jr.", {"given": "John", "family": "Smith van der V", "suffix": "Jr."},