Repository navigation
Should a word glued to the next at its period split there? (Dr.John Smith, Ц.Сандаг-Очир, Yu. A. Chibrikova) #1255
Description
A period with no space after it joins two words into one token, and several shapes lose a name part that way:
| Input | today |
|---|---|
Dr.John Smith |
title Dr.John, no given name |
Ц.Сандаг-Очир |
given Ц.Сандаг-Очир, no family name |
J.Smith |
the same shape in Latin (#998 counted 1,179 such names) |
Mark Jhon Sta.Ana |
a saint particle glued to its surname |
Yu. A. Chibrikova |
Yu. reads as a title by H2 (the leading abbreviation title)'s two-letter shape, not as a two-letter initial |
period_joined_vocab already resolves some of these tokens by vocabulary (Lt.Gov.). What's missing is a rule for when the shape alone splits a token: an initial or a listed title before the period, and a name word after it. That has to sit alongside #998's credentials that keep the period inside (B.Pharm, M.Kes).
The shape question: does tokenize, or an early stage, split a period-joined token by the shape of its halves? If so, which halves qualify, and how does that interact with H2's shape for two-letter Cyrillic-transliterated initials (Yu., Zh.)?
Sub-issues: #1230, #1235, #1136, #1071.
Bordering: #998 (credentials glued at the period), #1166 and #317 (Thai titles glued to the name, with no period).
Activity
Dr.John Smithgives title 'Dr.John', no given name) #1230Ц.Сандаг-Очирgives given 'Ц.Сандаг-Очир', no family) #1235Yu. A. ChibrikovareadYu.as a given-name initial? Two-letter initials at the start of a name read as titles #1136derek73 commentedon Nov 9, 2026
Two more given-name shapes of this kind came up while building name matching (#1221, #1250). Name matching compares given names word by word, so it needs the token split the way the name is meant before it can compare them. Matching sets these aside until this issue is decided.
Two words joined at the period, a capital after it. Each reads as one given word, where the same name written with a space reads as two:
Ma.Teresa CruzMa.Teresa(Ma. Teresa Cruzreads givenMa., middleTeresa)M.Teresa RossiM.TeresaGio.Batta RossiGio.Batta(Giovanni Battista)One word contracted, lowercase after the period. The Italian start-plus-ending abbreviation. These should not split:
G.ppe VerdiG.ppeAnt.o BianchiAnt.oFr.co RossiFr.co, no given nameSo the case after the period is a candidate signal beside the ones above. A capital starts a second word; lowercase continues the same word.
Fr.coreading as a title is a separate gap: its first half is read asFr.(Father).Bordering, on the
Yu.row's H2 question.Alex. Smithreads titleAlex., by the same leading period-abbreviation shape. That shape yields only to a listed given abbreviation (same_as), so every unlisted given name written short at the front of a name reads as a title. That's more than two-letter initials. Matching's planned prefix rule (Jo.,Geo.,Benj.against the name they begin) can only act where the stub reaches the given role.🤖 Generated with Claude Code