Skip to content

Let the glossary decide which single words L02 leaves alone - #57

Merged
tamnd merged 1 commit into
mainfrom
l02-exempts-kept-terms
Aug 18, 2026
Merged

Let the glossary decide which single words L02 leaves alone#57
tamnd merged 1 commit into
mainfrom
l02-exempts-kept-terms

Conversation

@tamnd

@tamnd tamnd commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Closes #49, the tool half of it. The glossary half is content tamnd/python-docs-vi-machine-translation#54.

Narrowing the identifier rule in #47 put 6,558 single-word entries into Corpus.prose() for the first time and took L02 from 10 findings to 144. A third of those were entries reading sys, builtins, import, exec, NaN and Infinity. Each one is a Sphinx index entry naming a module or a statement, and a reviewer left it in English because that is what a Vietnamese programmer calls it.

Nothing in the string separates those from module, object and type, which are the other side of the same list: ordinary English words used as index categories, sitting untranslated. sys and Notes are the same shape, and that is precisely the discrimination #47 established the classifier cannot make.

So the decision is made once, by hand, in the glossary, where keep_en already means it. L02 now skips an entry whose whole msgid is a kept term, and G03 checks that term in both directions instead. A written decision rather than an exception buried in a check.

Matched on the whole msgid, not on a substring: a kept term inside a sentence says nothing about whether the sentence was translated.

Measured on the real corpus

check before after
L02 no entry is the English verbatim 139 85
G03 kept terms stayed in English 81 95
G02 glossary renderings are used 191 190
G04 no untranslated glossary term 143 142

54 fewer L02 findings, not the 38 the issue predicted, and the extra 16 are worth naming. The glossary already held type and Boolean as kept, and the issue's triage had counted those 16 entries as real findings. The glossary and the triage disagree, the glossary wins because it is the written decision, and the disagreement is now visible instead of being an argument nobody was having. Filed as #58.

The 14 new G03 findings are the other direction: entries where a kept term appears in prose and the translation rendered it in Vietnamese anyway. sys is hệ thống in some entries and sys in others, so the corpus says both, and G03 is now the place that says so.

Checks

make check green, 1,403 tests, 97.45 per cent coverage. H03 clean.

L02 asks whether a translation is the English handed back, and for an entry
reading `sys` the answer is yes and it is the right answer. Narrowing the
identifier rule put 6 558 single-word entries into prose for the first time
and took this check from 10 findings to 144, a third of which were index
entries naming a module or a statement that a reviewer correctly left in
English.

Nothing in the string separates those from `module`, `object` and `type`,
which are the other side of the same list, ordinary English words used as
index categories and sitting untranslated. `sys` and `Notes` are the same
shape. So the decision is made once in the glossary, where `keep_en` already
means it, and L02 skips an entry whose whole msgid is a kept term. G03 then
checks that term in both directions.

Matched on the whole msgid rather than on a substring. A kept term inside a
sentence says nothing about whether the sentence was translated.

On the corpus: L02 139 to 85, G03 81 to 95, G02 191 to 190, G04 143 to 142.

54 fewer rather than the 38 the issue predicted. The glossary already held
`type` and `Boolean` as kept and the issue's triage had counted those 16 as
real findings. The glossary wins because it is the written decision, and the
disagreement is now visible rather than being an argument nobody was having.
@tamnd
tamnd merged commit b596306 into main Aug 18, 2026
6 of 7 checks passed
@tamnd
tamnd deleted the l02-exempts-kept-terms branch August 18, 2026 08:20
@tamnd

tamnd commented Aug 18, 2026

Copy link
Copy Markdown
Owner Author

Correcting the table in the description. Those numbers were measured with an Infinity row in the glossary proposal, and that row was dropped before the content side shipped: vô cực is right for the 80 prose entries that mean the mathematical concept and wrong for the 1 index entry, and a row cannot say both.

Measured on what actually shipped, content tamnd/python-docs-vi-machine-translation#54:

check before after
L02 no entry is the English verbatim 139 86
G03 kept terms stayed in English 81 94
G02 glossary renderings are used 191 191
G04 no untranslated glossary term 143 143

53 fewer L02 findings rather than 54, and the single Infinity entry stays reported. G02 and G04 do not move at all, which the earlier table had wrong.

The rest of the description holds. type and Boolean are exempt because the glossary already kept them, the issue's triage had counted them as real, and that disagreement is #58.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

L02 cannot tell a kept identifier from an untranslated word

1 participant