Skip to content

Commit c15cb8d

Browse files
ronaldtseronaldtse
committed
blog: the phonological layer — what shipped, what collapsed, the mojibake saga, and the beam correction
1 parent 7ebb486 commit c15cb8d

1 file changed

Lines changed: 133 additions & 0 deletions

File tree

‎src/content/blog/2026-08-24-phonological-layer-shipped.adoc‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
= The phonological layer: ten models, three wrong turns, and one correction
2+
Ronald Tse
3+
v1.0, 2026-08-24
4+
:doctype: book
5+
:docinfo:
6+
7+
== Introduction
8+
9+
Interscript has always been about deterministic transliteration: 289
10+
authority-backed systems, one map format, byte-identical output from
11+
every runtime. But some conversions have no committee. Restoring the
12+
haraqat a scribe left out. Turning unwritten Thai into phonemes.
13+
Reading Urdu aloud. Those are learned — and for two years they lived
14+
in a research branch, promising and unshipped.
15+
16+
This August we shipped the phonological layer: ten neural models under
17+
the same discipline as the maps — one artifact format, checksums
18+
verified on every load, identical output from Ruby, Python, and
19+
TypeScript, and a measured number next to every claim. This post is
20+
the story of what shipped, what didn't, and the two weeks of debugging
21+
that taught us our own decode was lying to us.
22+
23+
== What shipped
24+
25+
The catalog now spans four languages and two tiers:
26+
27+
[cols="1,1,3,1"]
28+
|===
29+
|Model |Task |Measured |Artifact
30+
31+
|khm-latn-1.0 |transliteration |CER 27.42 |fp32, 1.3 GiB
32+
|urd-g2p-1.0 |grapheme→phoneme |CER 14.77 |fp32, 1.3 GiB
33+
|urd-diac-1.0 |diacritization |CER 3.74 |fp32, 1.3 GiB
34+
|heb-diac-1.0 |diacritization |DER 29.0 greedy / 17.5 beam-4 |fp32, parts
35+
|tha-g2p-base-1.0 |grapheme→phoneme |PER 9.19 (teacher 4.43) |fp32, parts
36+
|fas-g2p-1.0 |grapheme→phoneme |CER ≈1.6 — above published SOTA on SentenceBench |fp32, parts
37+
|tha-g2p-small-1.0 |grapheme→phoneme |PER 2.85 greedy |int8, 246 MiB
38+
|tha-g2p-small-1.0-int4 |same student, 4-bit |byte-identical decode |int4, 193 MiB
39+
|heb-diac-small-1.0 |diacritization |DER 30.37 (teacher 24.79) |fp32, 1.3 GiB
40+
|===
41+
42+
Every student was distilled from a frozen, independently evaluated
43+
teacher, gated at a pre-agreed error budget before release. Every
44+
artifact is a zip — metadata, ONNX graphs, per-member SHA-256 — that
45+
any runtime can verify and serve. Models above GitHub's 2 GiB asset
46+
cap ship as checksummed parts the runtimes reassemble transparently.
47+
48+
== Wrong turn 1: the 30 MB tier that wouldn't exist
49+
50+
The obvious request: make the client models *small*. We tried
51+
everything. Custom 33M and 70M byte-level students from scratch.
52+
Microkimi-style linear bridges that map teacher activations into the
53+
student's geometry. Depth-pruning ByT5. The bridges helped structure
54+
(75.8 → 71.1 PER on Thai) but nothing rescued accuracy: a randomly
55+
initialized sub-100M byte model simply cannot generalize, and ByT5's
56+
smallest pretrained rung (300M) has no useful prune below it — its
57+
quality lives in its width.
58+
59+
We re-tested the hypothesis on Arabic with clean labels and a
60+
pretrained backbone at the small rung: the from-scratch 33M student
61+
fits its training data perfectly and then emits nothing at inference —
62+
an immediate end-of-sequence, scoring the bare-text error constant.
63+
The conclusion, now measured in two task families: **a pretrained
64+
backbone is non-negotiable.** The client tier ships at ByT5-small,
65+
quantized — 246 MiB at int8, 193 MiB at int4, with the 4-bit drop
66+
costing 0.17 points of character error and a parity gate to prove it.
67+
68+
== Wrong turn 2: the bug that looked like infrastructure
69+
70+
For two days, every Arabic training run reported its labels file
71+
"torn" — two to six valid pairs surviving out of eleven thousand, on a
72+
file that read perfectly from a laptop. We blamed volume replication,
73+
wrote retry loops, shipped labels inside container images. All of it
74+
was misdirection for three stacked bugs, each masking the next:
75+
76+
. `str.splitlines()` is not a line split — it fragments on code points
77+
that legitimately appear inside Arabic text, shredding healthy files
78+
mid-line.
79+
. The platform's file-transfer layers re-encode non-ASCII text from
80+
this workstation — our labels were double-encoded in transit, and the
81+
"corruption on the volume" was corruption at the door.
82+
. A hardcoded 384-byte label filter silently discarded nearly every
83+
Arabic label, because Arabic trains on 1,450-byte windows.
84+
85+
The lesson generalizes: when a file "reads corrupt" remotely and clean
86+
locally, suspect the parser and the transport before the storage. Our
87+
labels now travel as gzip+base64 — pure ASCII, immune to every
88+
encoding layer — and are split on plain newlines.
89+
90+
== The correction: our decode was lying
91+
92+
The deepest problem was the one we trusted most. Evaluation harnesses
93+
inherited beam-4 decoding from teacher-side convention; the runtimes
94+
ship greedy. When we finally ported beam search into the runtimes and
95+
measured both decodes on the *shipped artifact*, through the exact
96+
ONNX path users get, the numbers disagreed absurdly: the published
97+
12.06% PER was really **2.85%** — a 4× error — with nearly identical
98+
exact-match.
99+
100+
The cause is statistical, not mechanical. Distilled byte-level
101+
students have flat per-token distributions — the top prediction is
102+
barely above uniform. Greedy rides those tiny margins consistently.
103+
Beam search with length normalization systematically prefers *long*
104+
outputs on flat distributions; with raw scores, *short* ones. Either
105+
way, every sentence the model didn't get exactly right is mangled, and
106+
edit distance explodes while exact-match barely moves.
107+
108+
So: greedy is the runtime protocol, greedy is what we publish, and the
109+
beam-4 figures stand only as measurements under that decode. The
110+
correction rippled through the whole provenance chain — results log,
111+
model metadata, the index — because a number you can't trace to a
112+
protocol is a number you can't trust.
113+
114+
== The discipline
115+
116+
Everything above survives because of one contract: no number ships
117+
without a harness, and no harness without a public protocol. Teachers
118+
are frozen before distillation and never LLM-generated — language
119+
models hallucinate haraqat, and a poisoned teacher poisons every
120+
student. Students are gated against their teacher on the same test
121+
set. Release artifacts carry their own provenance, and the index
122+
refuses entries whose metrics don't trace to the results log.
123+
124+
The two-layer stack is now complete: 289 deterministic maps for the
125+
conversions authorities have published, and a measured neural layer
126+
for the ones they haven't. Same discipline, same checksums, same bytes
127+
out of every runtime.
128+
129+
Try it: `pip install secryst` —
130+
`Secryst::Model.load("tha-g2p-small-1.0")` in Ruby, `Model.load` in
131+
Python, the same in TypeScript. The catalog lives at
132+
https://github.com/interscript/interscript-ml[interscript-ml]; the
133+
measured story behind every number lives in its results log.

0 commit comments

Comments
 (0)