Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/design/mechanisms.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ This catalog converts discovery into a one-time cost — it does not remove disc
**Contract statement.** A vocabulary layer first claims words for what they ARE, wherever they sit; a positional layer then reads every unclaimed word by where it STANDS. Every rule belongs to exactly one layer — with one named exception: the leading-abbreviation shape (rule H2) fires before the suffix vocabulary is consulted, so
"Esq. Smith" reads title, not suffix.

**How it works.** The two layers compose without ordering bugs because the positional layer never overrides a vocabulary claim (rule O4 is the positional layer's contract). The exception is still ONE, re-checked 2026-09-08 against the trailing title run (rule H5), which reads title vocabulary in the trailing slot and is NOT a second one: it runs after the suffix peel rather than before it, so a post-nominal keeps its claim — `John Smith Esq.` reads suffix and `John Smith Prof. Jr.` reads suffix `Jr.`, both measured. That rule is also the worked answer to the Reach-for-it line below: it wanted a word's identity and its position at once and was split by SLOT, the front asking about shape (H2) and the back about vocabulary (H5), two questions with two criteria rather than one rule fighting both layers.
**How it works.** The two layers compose without ordering bugs because the positional layer never overrides a vocabulary claim (rule O4 is the positional layer's contract). The exception is still ONE, re-checked 2026-09-08 against the trailing title run (rule H5), which reads title vocabulary in the trailing slot and is NOT a second one: it runs after the suffix peel rather than before it, so a post-nominal keeps its claim — `John Smith Esq.` reads suffix and `John Smith Prof. Jr.` reads suffix `Jr.`, both measured. That rule is also the worked answer to the Reach-for-it line below: it wanted a word's identity and its position at once and was split by SLOT, the front asking about shape (H2) and the back about vocabulary (H5), two questions with two criteria rather than one rule fighting both layers. A claim here is a word's role, not every reading the vocabulary records: a `given` reading says only that a word may be a given name, and the rules that decide a lone word's field read past it, H1 and M4 both reading such a word as the family name (`Dr. Chas.`, `Chas. née Jones`), M4 keeping only a bound given name and an initial (decisions/M4's 2026-11-05 #815 entry).

**Lives in.** _classify/_group (vocabulary side), _assign (positional side).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@

**Except:**

- **A word the vocabulary has claimed as a given name keeps that reading, and so does a word read as an initial** (T4, T5), which is nobody's family name. Both are beyond the rule's reach for one reason: it changes what POSITION would have decided, so it cannot overrule what a word already IS (mechanisms.md#TWO-LAYER-ASSIGN).
- **A bound given name keeps the reading its position gives it, and so does a word read as an initial** (P5, T4, T5). Both are beyond the rule's reach for one reason: it changes what POSITION would have decided, so it cannot overrule what a word already IS (mechanisms.md#TWO-LAYER-ASSIGN).
```
"abd née Jones" → given="abd" · boundary
"J. née Jones Smith V" → given="J." · boundary
Expand All @@ -35,6 +35,14 @@

- **A name carrying a TITLE** is H1's rather than this rule's, H1's given-name-title carve-out included, which keeps the word a given name.

**Accepted:**

- A word the vocabulary reads as possibly a given name, without the claim a bound given name makes, is the family name all the same — a given name written short, a title or a particle that may also be a given name — since the marker names the word beside it as the surname the bearer carries now.
```
"Chas. née Jones" → family="Chas."
"Van née Jones" → family="Van"
```

---
**Signal:** count
**history:** decisions/M4 · **implemented:** nameparser/_pipeline/_post_rules.py
4 changes: 2 additions & 2 deletions nameparser/_pipeline/_assign.py
Original file line number Diff line number Diff line change
Expand Up @@ -212,8 +212,8 @@ def _absorbed(piece: Sequence[int], tokens: Sequence[WorkToken],
#: could speak, so O5's convention decided nothing. Built from M4's
#: `_NEVER_FLIPPED` pair rather than respelling it: the two sets answer
#: different questions -- may M4 retag the word, and did anything
#: decide the field -- and share the pair because a word vocabulary
#: claimed as a given name, or wrote as an initial, answers both. The
#: decide the field -- and share the pair because a bound given name,
#: or a word written as an initial, answers both. The
#: third, `particle`, is here alone because a lone particle's reading
#: is P4's. None is a predicate this emitter owns: they are read off
#: the tags classify already recorded (mechanisms.md#TWO-LAYER-ASSIGN).
Expand Down
4 changes: 2 additions & 2 deletions nameparser/_pipeline/_post_rules.py
Original file line number Diff line number Diff line change
Expand Up @@ -894,8 +894,8 @@ def post_rules(state: ParseState) -> ParseState:
# another rule counts them, so the two disagree; widening moves
# no corpus name and is not this change's to make
# (decisions/M4).
# rules/M4: "a word the vocabulary has claimed as a given name
# keeps that reading, and so does a word read as an initial"
# rules/M4: "a bound given name keeps the reading its position
# gives it, and so does a word read as an initial"
# -- read off the tags classify already recorded rather than a
# predicate of this rule's own, because this rule changes what
# POSITION decided and must not reach what a word IS
Expand Down
2 changes: 1 addition & 1 deletion tests/v2/rules_budget.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
"M1": {"words": 239, "then_words": 119, "because_sentences": 2, "except": 1, "precedence": 2, "accepted": 0, "accepted_words": 0},
"M2": {"words": 755, "then_words": 435, "because_sentences": 1, "except": 4, "precedence": 1, "accepted": 6, "accepted_words": 204},
"M3": {"words": 239, "then_words": 84, "because_sentences": 2, "except": 2, "precedence": 2, "accepted": 1, "accepted_words": 93},
"M4": {"words": 331, "then_words": 168, "because_sentences": 2, "except": 1, "precedence": 1, "accepted": 0, "accepted_words": 0},
"M4": {"words": 325, "then_words": 168, "because_sentences": 2, "except": 1, "precedence": 1, "accepted": 1, "accepted_words": 59},
"N1": {"words": 95, "then_words": 62, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0},
"N2": {"words": 131, "then_words": 79, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0},
"N3": {"words": 156, "then_words": 34, "because_sentences": 1, "except": 1, "precedence": 1, "accepted": 5, "accepted_words": 249},
Expand Down
2 changes: 2 additions & 0 deletions tools/differential/corpus_rules.jsonl
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@
"Chancellor Smith"
"Chas. Smith"
"Chas. Smith, Ed"
"Chas. née Jones"
"Cheikh Anta Diop"
"Chief Dean and Mary Smith"
"Chief Justice John Roberts"
Expand Down Expand Up @@ -1264,6 +1265,7 @@
"Van Gogh Vincent"
"Van Johnson"
"Van der Merwe Jan"
"Van née Jones"
"Vega, Juan de la"
"Vincent van Gogh van Beethoven"
"W. Russell (Jr.) Withers"
Expand Down
2 changes: 2 additions & 0 deletions tools/differential/snapshot.jsonl
Original file line number Diff line number Diff line change
Expand Up @@ -592,6 +592,7 @@
{"name": "Charles van der van der Berg", "given": "Charles", "family": "van der van der Berg", "initials": "C. B."}
{"name": "Chas. Smith", "given": "Chas.", "family": "Smith", "initials": "C. S."}
{"name": "Chas. Smith, Ed", "given": "Chas.", "family": "Smith", "suffix": "Ed", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: name as given (declared)"], "initials": "C. S.", "capitalized_force": "Chas. Smith, ED"}
{"name": "Chas. née Jones", "family": "Chas.", "maiden": "Jones", "initials": "C."}
{"name": "Chattel, Frederik Jacobus van Rossum", "given": "Frederik", "middle": "Jacobus van Rossum", "family": "Chattel", "initials": "F. J. R. C."}
{"name": "Cheikh Anta Diop", "given": "Cheikh", "middle": "Anta", "family": "Diop", "initials": "C. A. D."}
{"name": "Chemistry Jane Smith", "given": "Chemistry", "middle": "Jane", "family": "Smith", "initials": "C. J. S."}
Expand Down Expand Up @@ -3934,6 +3935,7 @@
{"name": "Van der Merwe Jan Koos", "order": "FAMILY_FIRST", "given": "der Merwe Jan Koos", "family": "Van", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: particle as family (vocabulary)"], "initials": "d. M. J. K. V.", "capitalized_force": "Van Der Merwe Jan Koos"}
{"name": "Van der Merwe, Jan Koos", "given": "Jan", "middle": "Koos", "family": "Van der Merwe", "initials": "J. K. M.", "capitalized_force": "Jan Koos van der Merwe"}
{"name": "Van der Walt", "given": "Van", "family": "der Walt", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: particle as family (vocabulary)"], "initials": "V. W."}
{"name": "Van née Jones", "family": "Van", "maiden": "Jones", "initials": "V."}
{"name": "Varano, Costanza da", "given": "Costanza", "family": "da Varano", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: name as middle (declared)"], "initials": "C. V."}
{"name": "Vatson Con Hamiş", "order": "FAMILY_FIRST", "given": "Con", "middle": "Hamiş", "family": "Vatson", "initials": "C. H. V."}
{"name": "Vega Smith", "given": "Vega", "family": "Smith", "initials": "V. S."}
Expand Down
Loading