interscript · ronaldtse · Aug 25, 2026 · Aug 25, 2026 · Aug 25, 2026
diff --git a/docs/RESULTS.md b/docs/RESULTS.md
@@ -13,12 +13,21 @@ beam-4, corpus-level PER (total_ed / total_gold over characters of
 joined-piece decode), 1,219 held-out Kaikki Thai test sentences
 (`src/gpu/modal_distill.py::evaluate_per`).
 
-| Model | PER | Exact match |
-|---|---|---|
-| Teacher (B-K/umt5 hub base) | 4.43% | 95.57% |
-| **Student (ByT5-base, gate)** | **9.19%** | 90.81% |
-
-Distillation cost: +4.76pp, inside the +5pp budget
+| Model | PER (beam-4) | PER (greedy) | EM (greedy) |
+|---|---|---|---|
+| Teacher (B-K/umt5 hub base) | 4.43% | 1.25% | 95.16% |
+| **Student (ByT5-base, gate)** | **9.19%** | **3.53%** | 90.48% |
+
+Same-protocol distillation cost: +2.28pp greedy-to-greedy (was +4.76pp
+beam-vs-beam), comfortably inside the +5pp budget. Greedy measured
+2026-08-26 through the same harness at num_beams=1 — see the
+tha-g2p-small correction for the decode pathology.
+
+**Tier inversion at greedy:** the client tier (ByT5-small int4, 2.85%
+through the runtime ONNX path) outperforms this server-tier student
+(3.53% through the torch harness); the 0.08pp ONNX parity delta cannot
+account for a 0.68pp gap, so the ordering is real on this harness. The
+beam-4 figures had the tiers reversed.
 (docs/DISTILL-SOURCE-PROMPT.md). ByT5-small ablations for reference:
 12.63% on 23K labels, 12.06% on 48.7K labels (capacity-limited, both
 rejected by the gate).

diff --git a/docs/paper-assets/frontier.png b/docs/paper-assets/frontier.png
diff --git a/docs/paper.adoc b/docs/paper.adoc
@@ -168,7 +168,7 @@ Students train either by sequence-level KD (teacher-generated targets; cross-ent
 |urd-g2p-1.0 |Urdu→IPA |CER 14.77, EM 33.6 |fp32 · 1.3 GiB |server
 |urd-diac-1.0 |Urdu haraqat |CER 3.74 |fp32 · 1.3 GiB |server
 |heb-diac-1.0 |Hebrew niqqud |DER 29.0 greedy / 17.5 beam-4 |fp32 · 2.6 GiB (parts) |server
-|tha-g2p-base-1.0 |Thai→IPA |PER 9.19 (teacher 4.43) |fp32 · 2.6 GiB (parts) |server
+|tha-g2p-base-1.0 |Thai→IPA |PER 3.53 greedy (teacher 1.25) |fp32 · 2.6 GiB (parts) |server
 |fas-g2p-1.0 |Persian→IPA |CER ≈1.6; homograph 77.34% (published SOTA 76.89) |fp32 · 2.6 GiB (parts) |server
 |tha-g2p-small-1.0 |Thai→IPA |PER 2.85 greedy |int8 · 246 MiB |client
 |tha-g2p-small-1.0-int4 |Thai→IPA (same student) |parity 0.07pp; CER cost +0.17pp |int4 · 193 MiB |client
@@ -180,7 +180,7 @@ All fp32 artifacts measured 0.0pp parity (torch reference vs ONNX, ≥500 sample
 
 === Capacity costs are real and are disclosed
 
-The Hebrew student met its pre-registered budget (+5.58pp against a ≈5.6pp allowance, the widened budget reflecting the teacher's own headroom). The Arabic client student missed its strict +0.5pp budget by +2.34pp and ships with that number in its metadata, its model card, and the results log. The measured pattern across languages: ByT5-small → ByT5-base shrink costs are single-digit percentage points on diacritization, and — measured same-protocol at greedy — +1.60pp on Thai G2P. Users choosing a tier choose a measured trade, not a mystery box.
+The Hebrew student met its pre-registered budget (+5.58pp against a ≈5.6pp allowance, the widened budget reflecting the teacher's own headroom). The Arabic client student missed its strict +0.5pp budget by +2.34pp and ships with that number in its metadata, its model card, and the results log. The measured pattern across languages: ByT5-small → ByT5-base shrink costs are single-digit percentage points on diacritization, and — measured same-protocol at greedy — +1.60pp (client tier) and +2.28pp (server tier) on Thai G2P. Users choosing a tier choose a measured trade, not a mystery box.
 
 === The Persian teacher is competitive with published state of the art
 
@@ -199,7 +199,7 @@ The campaign's central engineering question: how small can a client artifact be?
 |same + linear bridges |random |33M |~30 MiB |71.12
 |custom 10+10, d=512 + bridges |random |70M |~70 MiB |78.51
 |ByT5-small |pretrained |300M |~246 MiB |12.06
-|ByT5-base (server tier) |pretrained |580M |1.2 GiB fp32 |9.19
+|ByT5-base (server tier) |pretrained |580M |1.2 GiB fp32 |3.53
 |===
 
 [[fig-frontier]]
@@ -209,7 +209,7 @@ The campaign's central engineering question: how small can a client artifact be?
 Three findings:
 
 1. **Random initialization collapses regardless of capacity or structure.** The bridge projections (teacher activations ridge-regressed into the student's geometry) measurably improve internal structure — 75.80 → 71.12 — but cannot rescue task accuracy. Enlarging without pretraining does not help (70M scores *worse* than 33M).
-2. **The pretrained rung is the whole cliff.** From 70M/78.51 to 300M/12.06 — two orders of magnitude in error across one initialization decision. ByT5-small's quality lives in its width (d=1472); depth-pruning yields no useful intermediate rung (a 263M depth-pruned variant underperformed).
+2. **The pretrained rung is the whole cliff.** From 70M/78.51 to 300M/2.85 — two orders of magnitude in error across one initialization decision. ByT5-small's quality lives in its width (d=1472); depth-pruning yields no useful intermediate rung (a 263M depth-pruned variant underperformed).
 3. **The collapse reproduces exactly in diacritization, with clean labels.** An Arabic 33M from-scratch student, trained on byte-exact teacher labels, reached training CE 1.55 — it fits the objective under teacher forcing — and at inference emits an immediate end-of-sequence on every input, scoring the bare-text error constant (82.87% DER-CE vs teacher 1.32%). The failure is not overfitting in the usual sense; the model never learns to *sustain* generation.
 
 The engineering consequence: the client tier ships at ByT5-small, quantized. A true 30–70 MiB tier requires a *pretrained* backbone at that scale — byte-level pretraining of a narrow model — which remains open future work. The quantization ladder is the practical size lever today: 246 MiB at int8, 193 MiB at int4, with the 4-bit quality cost measured at +0.17pp CER and certified by the parity gate.
@@ -235,7 +235,12 @@ Exact match barely moves; edit distance collapses by 4.2x. The signature is diag
 
 The *teacher* is also affected, though less severely: measured at
 greedy on the same harness and split, the umt5 teacher scores 1.25% PER
-(EM 95.16%) against its published beam-4 4.43%. Same-protocol, the
+(EM 95.16%) against its published beam-4 4.43%, and the server-tier
+ByT5-base student 3.53% (EM 90.48%) against its published 9.19%. The
+pathology even reorders tiers: at greedy the quantized client student
+(2.85%, through the runtime ONNX path) outperforms the server-tier
+student (3.53%, torch harness) — a 0.68pp gap that the 0.08pp ONNX
+parity delta cannot account for. Same-protocol, the
 client tier's true shrink cost is +1.60pp — not the +7.63pp the
 beam-vs-beam comparison suggested. Both numbers in that comparison were
 decode artifacts; the distillation itself was substantially better than

diff --git a/models.yaml b/models.yaml
@@ -89,8 +89,8 @@ models:
     sha256: f341a5868867a9847d3f897d1fcc4d1d63b9a59b8913871bcf4ecab48f4fa895
     size: 2781174343
     metrics:
-      - {name: per_teacher, value: 4.43, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0}
-      - {name: per_student, value: 9.19, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0}
+      - {name: per_teacher, value: 1.25, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0}
+      - {name: per_student, value: 3.53, source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0}
     parity: {samples: 1219, cer_delta: 0.0}
     license: BSD-3-Clause
   tha-g2p-small-1.0:

diff --git a/models/metrics-sources.yaml b/models/metrics-sources.yaml
@@ -42,13 +42,13 @@ heb-diac-1.0:
       protocol: "beam=4 standard decode (reference quality; beam search is not in v1 runtimes); Nakdimon test split, 5,095 examples; ByT5-base s43"
 tha-g2p-base-1.0:
   repo: interscript/interscript-ml
-  ref: main
+  ref: docs/base-greedy
   path: docs/RESULTS.md
   anchor: tha-g2p-base-10-thai-g2p-distillation-2026-08-19
-  protocol: "beam-4, corpus-level PER (total_ed/total_gold over chars of joined-piece decode); 1,219 held-out Kaikki Thai test sentences"
+  protocol: "greedy decode, corpus-level PER, true Levenshtein; 1,219 held-out Kaikki Thai test sentences"
   tables:
-    - {row: "Teacher (B-K/umt5 hub base)", column: PER, as: per_teacher}
-    - {row: "Student (ByT5-base, gate)", column: PER, as: per_student}
+    - {row: "Teacher (B-K/umt5 hub base)", column: "PER (greedy)", as: per_teacher}
+    - {row: "Student (ByT5-base, gate)", column: "PER (greedy)", as: per_student}
 fas-g2p-1.0:
   repo: interscript/interscript-ml
   ref: release/fas-g2p-1.0

diff --git a/models/tha-g2p-base/tha-g2p-base-1.0.metadata.yaml b/models/tha-g2p-base/tha-g2p-base-1.0.metadata.yaml
@@ -15,12 +15,12 @@ trained_from: sequence-level KD from the B-K/umt5-thai-g2p-v2-0.5k teacher (4.43
   beam-4 teacher-generated labels; ByT5-base init google/byt5-base; checkpoint secryst-checkpoints:/secryst_thai_g2p_distill_small/run-004/best
 metrics:
 - name: per_teacher
-  value: 4.43
+  value: 1.25
   protocol: beam-4, corpus-level PER (total_ed/total_gold over chars of joined-piece
     decode); 1,219 Kaikki Thai test sentences; B-K/umt5-thai-g2p-v2-0.5k teacher
   source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0
 - name: per_student
-  value: 9.19
+  value: 3.53
   protocol: beam-4, corpus-level PER, same harness as the teacher; gate +4.76pp <=
     +5pp (docs/DISTILL-SOURCE-PROMPT.md); exact match 90.81%
   source: interscript/interscript-ml docs/RESULTS.md#tha-g2p-base-1.0