Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions AGENTS.md

Large diffs are not rendered by default.

8 changes: 6 additions & 2 deletions docs/concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -77,15 +77,19 @@ and ``script_orders`` change what "first" and "last" mean here, as
does the opt-in ``two_family_names``, which takes the last two name
words as the family (:ref:`two-family-names`), and the opt-in
``two_given_names``, which gives the given name the middle name beside
it (:ref:`two-given-names`).
it (:ref:`two-given-names`). A name of one name word has no first and
last to tell apart: it is the given name under every ``name_order``, or the
family name where ``lone_name_word`` says so, where a title beside it
makes it one (``"Mr. Johnson"`` is family ``Johnson``), or where its
script settles the order (``"王"``) (:ref:`lone-name-word`).
``script_orders`` covers the East Asian writing that settles its own
order: a name wholly in Han or hangul, or a Japanese name in kanji and
kana — any of them, hiragana alone included — unless it is katakana
alone, which is how a foreign name is transcribed. The Chinese
interpunct ``·`` dividing such a name walks that back, marking a
transcription that follows ``name_order`` like any other name. Three rules can still
override the arrangement: a lone name word beside a given-name title
such as ``Sir`` is the given name under every order, a particle that can never be a given name,
such as ``Sir`` is the given name whatever ``lone_name_word`` says, a particle that can never be a given name,
opening the name, opens the surname under every order rather than
becoming the given name (both in :ref:`order-vocabulary-first`), and the
opt-in ``patronymic_rules`` reorder, under the default order, a name
Expand Down
8 changes: 5 additions & 3 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -854,7 +854,8 @@ listed below.
``False``.
* - ``two_family_names``
- ``bool``
- Reads the last two name words as the family name, for data
- Reads the last two name words as the family name (the first
two under a family-first order), for data
where people carry two family names (Spain, most of Latin
America). Defaults to ``False``. See :ref:`two-family-names`.
* - ``two_given_names``
Expand Down Expand Up @@ -1182,8 +1183,9 @@ holds family names. For data whose lone words are surnames, set

Anything that does decide the word still decides it first, whatever the
switch says: a title (``"Mr. Johnson"`` is family ``Johnson``, ``"Sir
John"`` given ``John``), a nickname or maiden name beside it, and a
Han or Hangul name, which its script reads family-first. Through 2.x
John"`` given ``John``), a nickname or maiden name beside it, a family
comma (``"Anh, Jr."`` is family ``Anh``, ``", Anh"`` given ``Anh``), and a name its script reads family-first: Han, Hangul, or kana other than
katakana alone (``"みなみ"`` is family ``みなみ``). Through 2.x
the field followed ``name_order``, so a family-first order read a
lone word as the family name.

Expand Down
2 changes: 1 addition & 1 deletion docs/design/decisions/vocabulary-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -586,7 +586,7 @@ Open: #753 (phase 3, language packs; the 16 words strict ungates, each read for
- **2026-11-05 #1108 (its `default = false` on `ths`, `đbqh` and `dbqh` superseded by the 2026-11-06 #1115, the pack titles entry; `ts`, `gs` and `pgs` stay the choice's, written as initials too; its accepted stacked title ending a name, given 'PGS.TS.', superseded by decisions/H5's 2026-11-06 #1116 entry, which reads it as the title; its accepted lone name word read as the family under the pack, `Anh` → family 'Anh', superseded by decisions/O5's 2026-11-06 #1111 entry: a lone word reads given under every order, the pack's included) — `locales.VI`: A PLAIN CHOICE OF `vi`, vi.toml'S `[rules]` ORDER (`name_order = "family-first-given-last"`), AND A LEXICON FRAGMENT THAT REMOVES THE READINGS OTHER FILES GIVE THE SYLLABLES VIETNAMESE NAMES ARE WRITTEN WITH: PARTICLE (`van`, `le`, `do`, `du`, `di`, `la`, `tho`, `da`, `san`), CONNECTIVE (`the`, `y`) AND CREDENTIAL (`do`, `ba`, `chi`); THE ONE PACK FRAGMENT THAT REMOVES; NO STRICT MODE; THE DEFAULT DOES NOT MOVE.** Decided in chat on 2026-11-05 ("i agree with route A"; the word list "all as recommended"; the credentials "yes, send that", and `ma` left out, confirmed in chat), over (B) a pack of the order and titles alone, documented with `strict=True`, and (C) a `strict` field on `Locale`. #1108 proposed `given` readings of `văn`/`van` and `thị`/`thi` in vi.toml, on the premise that a reading turned on also prefers. It does not: `choose` prefers only by a vocabulary reading, and a name reading never settles (decision 15), so under a plain choice `Nguyen Van Anh` read given 'Van Anh', the Dutch particle chaining `Van` into the given name, exactly as the order alone reads it. No kind a file can write would make `van` a name word, so the settling is done by the pack's fragment, whose `remove` calls apply after the choice as a caller's own do. It bends decisions 6 and 15, which bind the default and the packs, for this pack alone, on the ground that its order commits a caller to Vietnamese names: a name written without a comma that the removal could misread (`Ludwig van Beethoven`) is misread by the order first. A family comma fixes the order, and the removals still act there, a cost the pack adds on its own, accepted below. `mechanisms.md#LOCALE-PACKS-PURE-DATA`'s contract names the exception. No compiled set of the default moves (`tools/vocab/build.py` reports `_compiled.py` current), and `snapshot.jsonl` is unchanged.
- **The measurement**, on the local corpora #1110 added (`tools/differential/local_uit.py`, UIT-ViNames; `tools/differential/local_vnassembly.py`, the National Assembly deputies), each record's parts derived from its order (the first word the family, the last the given, the words between the middle), measured 2026-11-05 and re-taken on the commit adding the credential removal, which moved none of them (default / the order alone / VI / strict VI): UIT-ViNames, 21,090 names written with diacritics, 13 / 21,065 / 21,085 / 21,089; the same names with the marks stripped (đ as d), standing in for romanized writing, 89 / 19,221 / 21,082 / 21,089 (the order to VI +1,861/−0); the deputies, 499 with the titles composed from their education, 0 / 483 / 499 / 499, stripped 0 / 404 / 499 / 499 (the order to VI +16, the stacked `PGS.TS.` and `GS.TS.`, and +95 stripped). The Vietnamese nationals of `psc-individuals`, filed in Western order, are the contrast: 1,097 of 1,494 by default and 1 under the order, the pack adding no loss beyond the order's. Per word, the records VI loses with the word left out of the fragment (UIT stripped / deputies stripped): `van` 1,221 / 74, `le` 398 / 2, `the` 163 / 3, `do` 40 / 0, `du` 15, `da` 11, `y` 10, `di` 9, `la` 3, `san` 3 (and 3 written with diacritics), `tho` 2 / 1; with diacritics only `du` 7, `di` 4, `la` 3, `san` 3, `da` 2 and `the` 1 fire. `mac`, in the first list, moved no Vietnamese-order name in any corpus (2 PSC Western-order names, losing nothing) and was dropped; `san`, the saint particle chaining a given name (`… San Hà`), was added (+3/−0 either way). Earlier and not re-taken: ORCID's Vietnam records composed family-first, 43,790 with both boxes, counting a middle the filer put in the family box as agreeing, read 89.95% under the order, 94.72% with the first fragment (`mac` in, `san` out), 95.43% strict.
- **The credentials**, removed after the code review found `NGUYEN VAN DO` reading suffix 'DO' under the pack: removing `van`'s particle had left en.toml's credential `DO` to take the given name of a name in one case, and `BA` and `CHI` did so by default. The comparator is VI with the credential readings kept. GAINED: the removal moves a role only in a name written in capitals, the syllable read as the given name where the credential took it: UIT-ViNames stripped and in capitals 108 names (the recipe prints 109 moved, one moving only its report, `KA BA`) (`chi` 82, `do` 20, `ba` 6), in capitals with its diacritics 68 (`chi` 65, `ba` 3), the deputies in capitals 4 (`chi` 2, `ba` 2), every one a gain by the corpora's own parts, no loss; as written and stripped in mixed case it moves only the `suffix-or-name` report (69 UIT names and 4 deputies' names as written, 110 and 4 stripped). FEC's donors with a common Vietnamese surname gain too, a given name the default reads as a credential: `chi` 5 (`NGUYEN, VY CHI` → middle 'CHI'), `ba` 2. COST: among names with a common Vietnamese surname (1,200 in ctgov-officials, 3,294 in FEC), a real credential stops being a suffix in `do` 4 (`Ryan Nguyen, DO`, `Thanh D Hoang, DO`, `Thao L Nguyen, DO, FAAP`; FEC `NGUYEN, MINH QUANG LCDR DO, MPH,`) and `ba` 1 (`Phuong T Vo, BA`), `chi` none. Every ctgov one is written given-first for English-language readers, which VI misreads with the credential kept (`Ryan Nguyen, DO` → family 'Ryan') and the default reads right. Measured 2026-11-05. Recompute the gains with `uv run python tools/differential/config_diff.py diff 'expr:parser_for(Locale(code="vi", lexicon=Lexicon.empty().remove("da","di","do","du","la","le","san","tho","van", kind="particle").remove("the","y", kind="conjunction"), policy=locales.VI.policy, languages=locales.VI.languages))' plain:VI --local uit-vinames --spelling stripped,upper` (`--spelling` added for this; `upper` alone, or `--local vn-assembly`, for the other rows); the costs by the same two configurations over the ctgov and FEC names holding one of `STRONG`'s surnames (the session notes' `credcost.py`, not in a tree).
- **Accepted for the credentials:** `DO` or `BA` written as a credential after a Vietnamese name reads as a name word, with a comma or without, dotted (reported `suffix-or-name`), in brackets or behind another credential (`Nguyen Van Anh, DO` → family 'Nguyen Van Anh', given 'DO'; `Nguyen Van Anh DO`, `Nguyen Van Anh D.O.` → given 'DO'; `Nguyen Van Anh (DO)` → nickname 'DO'; `Nguyen Van Anh MD, DO` → given 'DO'); past a second comma it stays a suffix (`Nguyen, Van Anh, DO`). `CHI` reads so without a comma (`Nguyen Van Anh CHI` → given 'CHI'); after one the capitals shape (rules/C6) takes a three-letter word in capitals as a suffix whatever its readings, so `Nguyen Van Anh, CHI` keeps suffix 'CHI', reported. Left out: `ma`, which gained no corpus name and cost one (`Linda Trinh, MA`), so `TRAN THI MA` stays a known gap (suffix 'MA'). Not removable: `VI`, a given name read as the numeral by its shape (`NGUYEN THI VI`). Declined for 3.0: keeping the credential readings and declining only the bare word ending the core of a one-case name, which would keep the comma costs away but loses FEC's family-comma gains, and needs a new `Policy` field set from vi.toml's `[rules]` (packs carry no code, and `leading_only`/`trailing_only` are reserved, unbuilt, for 3.x), amending rules/S4 (an ambiguous acronym needs words to spare) at 3.0 final.
- **Accepted for the credentials:** `DO` or `BA` written as a credential after a Vietnamese name reads as a name word, with a comma or without, dotted (reported `suffix-or-name`), in brackets or behind another credential (`Nguyen Van Anh, DO` → family 'Nguyen Van Anh', given 'DO'; `Nguyen Van Anh DO`, `Nguyen Van Anh D.O.` → given 'DO'; `Nguyen Van Anh (DO)` → nickname 'DO'; `Nguyen Van Anh MD, DO` → given 'DO'); past a second comma it stays a suffix (`Nguyen, Van Anh, DO`). `CHI` reads so without a comma (`Nguyen Van Anh CHI` → given 'CHI'); in the part right after one, the capitals shape (rules/C6) takes an unlisted word of three letters or more in capitals as a suffix behind two name words or more (C4) where the name's case contrasts, so `Nguyen Van Anh, CHI` keeps suffix 'CHI', reported, while a name written wholly in capitals gives the shape nothing to contrast with (`NGUYEN VAN ANH, CHI` → given 'CHI', unreported), and one name word before the comma leaves it nothing to spare (`Anh, CHI` → given 'CHI'); past a second comma it stays a suffix in any case, as `DO` does (rules/C2: `NGUYEN, VAN ANH, CHI` → suffix 'CHI'). (Corrected 2026-11-07: the entry said the shape takes the word whatever its readings, after any comma; it takes `CHI` only because the pack removes its credential reading, and only where the name's case contrasts, as C6 states.) Left out: `ma`, which gained no corpus name and cost one (`Linda Trinh, MA`), so `TRAN THI MA` stays a known gap (suffix 'MA'). Not removable: `VI`, a given name read as the numeral by its shape (`NGUYEN THI VI`). Declined for 3.0: keeping the credential readings and declining only the bare word ending the core of a one-case name, which would keep the comma costs away but loses FEC's family-comma gains, and needs a new `Policy` field set from vi.toml's `[rules]` (packs carry no code, and `leading_only`/`trailing_only` are reserved, unbuilt, for 3.x), amending rules/S4 (an ambiguous acronym needs words to spare) at 3.0 final.
- **Accepted for comma names:** after a family comma, where the order reads nothing differently, the removals still act, so a particle or a credential there reads as a name word: `Beethoven, Ludwig van` → family 'Beethoven', middle 'van', where the default gives family 'van Beethoven'; `Smith, John Do` → middle 'Do'. UIT-ViNames and the deputies hold no comma names, so no corpus measures it.
- **Recompute the four-parser table** with `local_agreement.read` over the uit-vinames and vn-assembly records under `Parser()`, `Parser(policy=Policy(name_order=FAMILY_FIRST_GIVEN_LAST))`, `parser_for(locales.VI)` and `parser_for(locales.VI, strict=True)` (the session notes' `vi_measure.py`, not in a tree; `local_agreement.py --locale vi --base` scores the base under the pack too, so it cannot give the default against it); `config_diff.py diff default plain:VI --local uit-vinames`, with `--spelling stripped` for the romanized view, gives the moves, and an `expr:` fragment leaving one word out each word's.
- **The titles.** vi.toml turns on, `default = false`, the academic titles written before a name (`ts`, `ths`, `gs`, `pgs`) and the office `đbqh` (đại biểu Quốc hội, National Assembly deputy) with its ASCII spelling `dbqh`. A dotted title is one at the head by shape already (rules/H2), so a reading is what reads the bare form (`Ts Nguyen Van Anh`) and the stacked one (`PGS.TS.`, each chunk a listed title, `PGS.TS.BS.` included, `bs` being en.toml's credential): the deputies' composed titles are ThS. 241, TS. 142, PGS.TS. 9 and GS.TS. 7, and the 16 stacked ones are exactly what the order alone misses. None of the four has a Vietnamese bearer (tools/name_usage.py, the counts on each word). `đbqh` is in none of the three corpora; CLDR 48's vi file writes it before a surname-first name (`ĐBQH Nguyễn Văn Anh`, which now reads family 'Nguyễn'). The office makes title phrases with `chief` and `deputy`, which attach to every office, so both join the choice's derived surface; neither reads differently. Accepted: a stacked title ending a name reads as the given name (`Nguyen Van Anh PGS.TS.` → given 'PGS.TS.'), where the default reads suffix 'PGS.TS.' by rules/S3's unlisted dotted suffix, which no longer takes it once `pgs` and `ts` are listed, and rules/H5 takes only single words; no local or committed corpus writes one trailing (all 16 `PGS.TS.` and `GS.TS.` lead, in vn-assembly, measured 2026-11-05), and keeping the readings to the leading position waits for `leading_only`, reserved for 3.x. Declined: `bs` (bác sĩ, physician), whose title reading, preferred under the choice, evicted en.toml's credential `BS`, so `Smith, John BS` read middle 'BS'; excluded in vi.toml with that reason. Not proposed: the military ranks, which the deputies' data writes in the position, never the name, and the monks' titles (Hòa thượng, Thượng tọa), full words found only in aliases.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

- **In the default given-first order** the first name word is the given name, the last is the family name, and everything between is middle names.
- **In a family-first order** the first name word is the family; in family-first-given-last the given name comes from the end, the middles from between.
- **A name of one name word** is O5's under every order.

**Because:** what no vocabulary claims can only be read by where it stands, under the order the caller declared.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@
"Jane Smith OTR/L/X" → family="OTR/L/X" · boundary
"Jane Smith-NP-C" → family="Smith-NP-C" · boundary
```
- **A member of the ambiguous class (S4) is a chunk too, however it is written, where the word holds a chunk that settles it**: a credential that is no name, or a member written with a period inside it, and not a generation, a numeral, an honorific or a word that is a title too, which say nothing about the member, as in a clause (S1), nor a member's case, which a chunk cannot be read by. Written alone, such a member may be a name or the initials it looks like (`John Smith MA` and `John Smith M.A` keep the family), but a credential joined to it says what the whole word is, since nobody joins a name or initials to a credential. A word of bare members alone has nothing to settle it, and a name word beside a member still refuses the word.
- **A member of the ambiguous class (S4) is a chunk too, however it is written, where the word holds a chunk that settles it**: a credential that is no name, or a member written with a period inside it, and not a generation, a numeral, an honorific or a word that is a title too, which say nothing about the member, as in a clause (S1), nor a member's case, which a chunk cannot be read by. Written alone, such a member may be a name or the initials it looks like (`John Smith Ma` and `John Smith M.A` keep the family), but a credential joined to it says what the whole word is, since nobody joins a name or initials to a credential. A word of bare members alone has nothing to settle it, and a name word beside a member still refuses the word.
```
"John Smith MA/PhD" → suffix="MA/PhD"
"John Smith M.A/PhD" → suffix="M.A/PhD"
Expand Down
6 changes: 4 additions & 2 deletions docs/locales.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1083,8 +1083,10 @@ name words are name words there too (``Beethoven, Ludwig van`` gives
middle ``van``). ``DO`` or ``BA`` written after a Vietnamese name as a
credential, for English-language readers, reads as a name word, with a
comma or without (``Nguyen Van Anh, DO`` gives family ``Nguyen Van
Anh``, given ``DO``), and ``CHI`` without a comma; other credentials,
and ``CHI`` after a comma, stay suffixes (``Nguyen Van Anh, PhD``).
Anh``, given ``DO``), and ``CHI`` without a comma or in a name written
wholly in capitals (``NGUYEN VAN ANH, CHI`` gives given ``CHI``); other
credentials, and ``CHI`` after a comma in a name written in mixed case,
stay suffixes (``Nguyen Van Anh, PhD``).

.. _segmenter-contract:

Expand Down
Loading
Loading