diff --git a/docs/design/decisions/S1-bracketed-credential-is-not-a-nickname.md b/docs/design/decisions/S1-bracketed-credential-is-not-a-nickname.md index 99b22410d..b62f59cc5 100644 --- a/docs/design/decisions/S1-bracketed-credential-is-not-a-nickname.md +++ b/docs/design/decisions/S1-bracketed-credential-is-not-a-nickname.md @@ -19,7 +19,7 @@ Elsewhere: an ambiguous member alone in the brackets staying a nickname, reporte A REPORT THE COMMA FIX REACHES: a member of the ambiguous class written with the clause's comma is a tagged member now, so a nickname clause holding one reports `suffix-or-nickname`, as S13 reports any member left in a nickname (ORCID's `Prof Anh My Chu (BEng, MEng, DEng)` and `HIKONO, Masaru (MEng, Ph.D. in Human Science, CPE)`, whose `MEng,` had carried no tag). ACCEPTED COSTS (rules/S1's Accepted bullets): a clause between name words reads otherwise than the same words bare (`W. Russell Jr. Withers` keeps middle 'Russell Jr.'); a short title word a writer's data uses as a tag reads as the title (FEC's state codes, `CASHION, JOHN(ST)` and six more entries holding `(ST)`, two holding `(ME)`, by the same recipe); a title-only clause behind a credential keeps its pair's reading even where it holds a title of the name and loses the period escape it took (`John Smith, MD (Dr.)` → nickname 'Dr.'; `Louise Dye, PhD (Professor)` stays the nickname it was); a set-apart clause in a name of no word of its own leaves the name empty (`(MPH)` → suffix 'MPH', where the escape had made it given 'MPH', H4); and `Col. (Ret.) Smith` reads suffix 'Ret.' where 1.4.0 through 2.8.0 read title 'Col. Ret.' (tests/test_suffixes.py's two "known limitation" rows, `Lon (Jr.) Williams` now suffix 'Jr.', are re-pinned). MEASURED 2026-11-04 with `snapshot.py --local --base ` (head package digest 9b70692c0bafee65): ctgov-officials 26 entries move, fec-individuals 16, psc-individuals 10, orcid-researchers 79; govuk-people, ukec-individuals, cz-candidates and dof-signatures none. By class: a bracketed title read as the title where it read as the nickname (`Dorothee Auer (Principal Investigator)`, `Prof (Dr) Ashish Gadekar` → title 'Prof Dr', `Mary (Rev Mrs) Fosu`; FEC's state codes `(ST)` and `(ME)`, the accepted cost); a credential run read as the suffix where it read as the nickname (`Brian Chirombo (MBChB, MPH)`, `F.L.J. Visseren, Professor (MD PhD)`); a bracketed generation or credential between name words ending at its brackets (`Mr Hugh (Snr) Boyle` → suffix 'Snr' where middle, `Abebe Alemu (PhD) Balcha` → family 'Balcha', `Dr (Phd) Alan Green` → given 'Alan'); credential initials inside a name moving from the middle name or the suffix run to the nickname (`FOX, (JP)JIMMY`); two ORCID nickname clauses gaining the member report above; and FEC's `TOBIN, MARY (V)` losing the suffix-or-name report #877 gives a bare letter ending the given part after a comma, which master gave it because its extract stage drops the brackets from content that is suffix-shaped, reading it bare: the clause is set apart in post_rules, after assign's walk reports, and a letter its writer bracketed is set apart from the name rather than a middle initial in doubt, so rules/C9 names it among the slots that settle the letter (the first measurement, against c6207549 before #877 merged, counted FEC 16). CLDR agreement: one Czech rendering moves, `Machová, Alexandra Zuzana (paní, Ph.D.)`, its given names agreeing now and its title gaining 'paní', which CLDR leaves out. -- 2026-11-05 (Derek), #1088 — A JOINED CREDENTIAL READS AS ITS WORDS WRITTEN APART. `John Smith (MD, PhD)` read suffix 'MD, PhD' under #792, but `John Smith (MD/PhD)` and `John Smith (MD-PhD)` read as nicknames, though S3 reads the bare `John Smith MD/PhD` as the suffix: the set-apart word asked its own copy of the unambiguous suffix sets, which S3's joiner arm is not in. #1084's simplify pass proposed asking `suffix_as_written`, the bare word's question, and refused it as a behavior change, since that predicate also reads a dotted member of the ambiguous class (`M.A.`) as a suffix, which S13 settles only after the name. DECIDED (Derek), as proposed: the word asks the bare word's question, a member of the ambiguous class asked first and left to S13, whose position test is unchanged. The member's dotted test, which post_rules had copied twice, is one predicate now (`_vocab.dotted_member`), asked by the settling and by the report alike, so the report on a member left the nickname leaves off a comma the clause keeps, as the settling did (rules/N1): `Jean (M.A,) Smith` reports suffix-or-nickname, as `Jean (M.A) Smith` does. The interior-period half of S3 (`(Msc.Ed)`, a different predicate whose any-chunk test is looser than the joiner's every-chunk) was left out, for an issue drafted the same day. +- 2026-11-05 (Derek), #1088 — A JOINED CREDENTIAL READS AS ITS WORDS WRITTEN APART. `John Smith (MD, PhD)` read suffix 'MD, PhD' under #792, but `John Smith (MD/PhD)` and `John Smith (MD-PhD)` read as nicknames, though S3 reads the bare `John Smith MD/PhD` as the suffix: the set-apart word asked its own copy of the unambiguous suffix sets, which S3's joiner arm is not in. #1084's simplify pass proposed asking `suffix_as_written`, the bare word's question, and refused it as a behavior change, since that predicate also reads a dotted member of the ambiguous class (`M.A.`) as a suffix, which S13 settles only after the name. DECIDED (Derek), as proposed: the word asks the bare word's question, a member of the ambiguous class asked first and left to S13, whose position test is unchanged. The member's dotted test, which post_rules had copied twice, is one predicate now (`_vocab.dotted_member`), asked by the settling and by the report alike, so the report on a member left the nickname leaves off a comma the clause keeps, as the settling did (rules/N1): `Jean (M.A,) Smith` reports suffix-or-nickname, as `Jean (M.A) Smith` does. The interior-period half of S3 (`(Msc.Ed)`, a different predicate whose any-chunk test is looser than the joiner's every-chunk) was left out, for #1104. THE INITIALS EXCEPT, decided after review: the first build set a joined credential apart before a name word too, where its words written apart stay the nickname under S1's initials Except, which asked the whole word, and a joined word never has the shape of initials (`John (MD/PhD) Smith` and `Justin (JP/MD) Stearns` → suffix, against `John (MD, PhD) Smith` and `Justin (JP, MD) Stearns` → nickname; the code, comment and design-docs reviews each found it, the last by the two-spelling invariant over 3,360 pairs). OPTIONS PUT: (A) the Except asks each chunk, so the two spellings read alike; (B) keep the divergence as an accepted cost. DECIDED (Derek): A. A joined credential counts as initials where any of its chunks would, as its words written apart do (`JP/MD`, `MD/PhD`, `M.D./Ph.D.`), and one whose chunks are generation words does not (`John (Jr/Sr) Smith` → suffix, as `(Jr, Sr)` reads). A credential the vocabulary lists whole is one word, not two S3 joins, and is never split (`John (NP-C) Smith` and `John (SHRM-CP) Smith` → suffix, as master reads them): the second-round code review found the first parity fix splitting it, `CP` and `C` reading as initials, and a sweep of the 70 listed words holding a joiner (two spellings each, in four positions, 560 names) reads every one as master does. tests/v2/pipeline/test_post_rules.py holds the two spellings alike over a grid of plain credential pairs in eleven shapes under the three orders; with the chunk loop off, 540 of its 2,310 rows read apart. Where the spelling written apart reads by another rule the joined word does not meet, the two still differ, each by that rule, and rules/S1 records the first two as Accepted: a word that is a title too reads as the title in front of the name's first word or in a name with no word of its own (`(PhD, Sr) John Smith` → title 'Sr', `(PhD/Sr) John Smith` → suffix 'PhD/Sr'; `(MD, MS)` → title 'MS', `(MD/MS)` → suffix 'MD/MS'); a member of the ambiguous class written as its own word is settled by S13's position test, which a joined word S3 reads whole does not take (`(M.A., PhD)` and `Smith (M.A., PhD), John` → nickname, reported, `(M.A./PhD)` and `Smith (M.A./PhD), John` → suffix, where master and 2.8.2 read both spellings as the nickname; the second-round design-docs review found it over 40 such pairs); and a clause ending in a member written with its periods takes the period escape written apart (`John (MD, M.A.) Smith` → suffix 'MD, M.A. Smith', `John (MD/M.A.) Smith` → nickname). WHAT IT MOVES, and accepted: a joined credential ending in a period is set apart before the period escape is asked, which had read it bare, so `Smith (M.D./Ph.D.), John` → suffix as 2.8 read it through the escape, `Jack (M.D./Ph.D.)` → family 'Jack', as `Jack (PhD)` reads, where 2.8 read given 'Jack' and reported given-or-family, and a period-final joined word before a name word, which the initials Except declines, keeps its pair's reading rather than the escape, a nickname no release read (`John (M.D./Ph.D.) Smith`, where 1.4.0 through 2.8 read middle 'M.D./Ph.D.', and `(M.D./Ph.D.) John Smith`, where they read given 'M.D./Ph.D.'). A clause the joined word completes is set apart whole, so a member beside it that its capitals settle no longer reports (`John Smith (MD/PhD, MBA)` → suffix 'MD/PhD, MBA', where master read the nickname and reported suffix-or-nickname, `mba` having been marked ambiguous on 2026-11-03; 1.4.0 and 2.8.2 read the nickname with no report). Not moved: a joined word with a chunk that is no credential or is a bare member (`MD/Bob`, `MA/PhD`), a hyphenated name (`Smith-Jones`, `Ae-Ra`), single-letter numerals (`I-V`) and a dangling joiner (`MD/`), each as S3 already refuses it bare. MEASURED 2026-11-05 against master 4109b026 checked out in a detached worktree: the committed snapshot moves none of its 6,945 entries (6,736 distinct names), and none of the 181 distinct names holding a pair of `DEFAULT_NICKNAME_DELIMITERS` has a word joined by a slash or a hyphen inside one, so the zero says the snapshot could not see the shape (recipe: for each name and each pair, the text between an opener and the next closer after it, matched against `\S[-/]\S`); the rule's fourteen new examples join the snapshot through corpus_rules.jsonl in the same change, recorded as new entries. A grid of 30 clause words (thirteen that move: the joined credentials `MD/PhD`, `MD-PhD`, `Md-PhD`, `md/phd`, `MD/PhD/MPH`, `M.A./PhD`, `Jr/Sr`, `MD/PhD,`, `M.D./Ph.D.`, and the clauses `MD/PhD, MBA` and `PhD, MD/MPH` holding one, and the kept-comma members `M.A,` and `M.A.,`; seventeen that do not: S3's refusals bare, a bare member as a chunk (`PhD/MBA`, `MA/PhD`, `JD/MBA`), a chunk not written with its periods (`M.A/PhD`), an unlisted chunk (`RN-BC`, `CPA/PFS`, `XY/ZW`), a name or a title as a chunk (`MD/Bob`, `Bob/MD`, `Dr/PhD`), a dangling joiner (`MD/`, `/PhD`), single-letter numerals (`I-V`) and a hyphenated name (`Smith-Jones`, `Ae-Ra`), and the members without a comma, `M.A` and `MA`) in 14 shapes, the templates exactly `John Smith (X)`, `John (X) Smith`, `(X) John Smith`, `(X)`, `Smith, John (X)`, `Smith (X), John`, `John Smith (X), Jr.`, `John Smith, Jr. (X)`, `JOHN SMITH (X)`, `Jack (X)`, `Smith, (X)`, `(X), PhD`, `John Smith "X"` and `Kim Min (X)`, each parsed by `Parser()` and by `Parser(policy=Policy(name_order=...))` under FAMILY_FIRST and FAMILY_FIRST_GIVEN_LAST, a name moving where `as_dict()` or its ambiguity kinds differ, 1,260 names: 399 move, 133 under each order, every move a joined credential read as the suffix, a period-final joined word before a name word read as the nickname (two under each order), or the member report (447, 149 under each order, before the initials Except asked each chunk). The comparator is the base tree's own readings of the same grid, parsed with each tree's `nameparser.__file__` asserted. In the local corpora (`snapshot.py --local --base `, on the first build): ctgov-officials 3 entries move, `Franck Pages, Professor (MD-PHD)` and `Franck Pagès, Professor (MD-PHD)` and `Philippe PC Courtet, Professor(MD-PhD)` → suffix where nickname; fec-individuals 1, `FRANTZ (CAPT USN-RET), JOSEPH` → title 'CAPT', suffix 'USN-RET' where nickname 'CAPT USN-RET'; psc-individuals and orcid-researchers none. The initials Except reaches only a joined word before a name word of its comma part, and none of the four stands there; each was parsed again on the final tree and reads as listed. diff --git a/nameparser/AGENTS.md b/nameparser/AGENTS.md index dbb3b08aa..d4e97a6fb 100644 --- a/nameparser/AGENTS.md +++ b/nameparser/AGENTS.md @@ -46,7 +46,7 @@ So a change that handles `V` and `III` can still miss `VI` — #610 found exactl ### Credentials and titles -The same split runs through the credentials (`MA` listed, `X.Y.Z.` by the dotted shape, `XYZ` by the caps shape, `MD/PhD` by its listed chunks and `PU-PH` by the caps shape's joined spelling, #752, each with its joiner glued, spaced and glued to one side, which are three token streams -- and a new chunk reading inherits the old one's exceptions, which #752's first draft did not (`Jack I-V` lost its family to the listed numerals, and the fix's length test counted a chunk's period, so `Jack I.-V.` did too, found by the second review), and `Ph. D.` merged by group into one flagged piece the peel's walk never holds -- #603 found #602's run-start predicate refusing it too, so `Smith, John Ph. D. Jones` kept middle 'Jones' where `Smith, John PhD Jones` read suffix 'PhD Jones') and the titles (`Prof.` period-marked and read at the end of a name, `Prof` bare and a name word there, and `Mr. and Mrs.` a run P3 joins, carrying a piece tag a one-word vocabulary test never sees -- PR #735's review found C12's hand copy of C13's title test (C8's until its 2026-10-19 split) reading `John Smith Jr., Mr. and Mrs. AD` as all suffix; ask C13's predicates in C13's order, `not is_suffix_piece` then `is_title_piece`, not the token tags, since `is_title_piece` alone reads the duals `Ms` and `Sr.` as titles). A listed acronym's CASE is a spelling too: #602's run start asked only whether the word was listed, so `Md` (Muhammad written short), `Ud`, `Bt` and `Pg` in Title case before a name word took it into the suffix (`Badrul Akmal Hisham Md Yusoff` → suffix 'Md Yusoff'), the Title case being a name word's writing, which `MD`, `md` and `M.D.` still start the run in, found by #1017's measurement and fixed by #1019; put `MD`, `Md`, `md` and `M.D.` in a credential rule's grid, each with a name word behind it, and the dotted and joined Title-case spellings (`M.d.`, `M.a.`, `Pa-c`): #1019's first build read a word's letters with its punctuation dropped, so those read as name words too, found by review. +The same split runs through the credentials (`MA` listed, `X.Y.Z.` by the dotted shape, `XYZ` by the caps shape, `MD/PhD` by its listed chunks and `PU-PH` by the caps shape's joined spelling, #752, each with its joiner glued, spaced and glued to one side, which are three token streams -- and a new chunk reading inherits the old one's exceptions, which #752's first draft did not (`Jack I-V` lost its family to the listed numerals, and the fix's length test counted a chunk's period, so `Jack I.-V.` did too, found by the second review; and the converse, a credential listed whole WITH its joiner (`NP-C`, `SHRM-CP`), is one word a chunk reading must not cut: #1088's fix asking each chunk of a joined word in S1's initials Except read the chunk `C` and turned `John (NP-C) Smith` from suffix to nickname, found by the second review, so put a listed hyphenated credential in the grid of any change that reads chunks), and `Ph. D.` merged by group into one flagged piece the peel's walk never holds -- #603 found #602's run-start predicate refusing it too, so `Smith, John Ph. D. Jones` kept middle 'Jones' where `Smith, John PhD Jones` read suffix 'PhD Jones') and the titles (`Prof.` period-marked and read at the end of a name, `Prof` bare and a name word there, and `Mr. and Mrs.` a run P3 joins, carrying a piece tag a one-word vocabulary test never sees -- PR #735's review found C12's hand copy of C13's title test (C8's until its 2026-10-19 split) reading `John Smith Jr., Mr. and Mrs. AD` as all suffix; ask C13's predicates in C13's order, `not is_suffix_piece` then `is_title_piece`, not the token tags, since `is_title_piece` alone reads the duals `Ms` and `Sr.` as titles). A listed acronym's CASE is a spelling too: #602's run start asked only whether the word was listed, so `Md` (Muhammad written short), `Ud`, `Bt` and `Pg` in Title case before a name word took it into the suffix (`Badrul Akmal Hisham Md Yusoff` → suffix 'Md Yusoff'), the Title case being a name word's writing, which `MD`, `md` and `M.D.` still start the run in, found by #1017's measurement and fixed by #1019; put `MD`, `Md`, `md` and `M.D.` in a credential rule's grid, each with a name word behind it, and the dotted and joined Title-case spellings (`M.d.`, `M.a.`, `Pa-c`): #1019's first build read a word's letters with its punctuation dropped, so those read as name words too, found by review. ### A word a rule only asks ABOUT