Skip to content

Commit 462bff2

Browse files
Ronald Tseronaldtse
authored andcommitted
release: heb-diac-small-1.0 registration (gated ByT5-small student, +5.58pp)
1 parent 8b9dc9e commit 462bff2

5 files changed

Lines changed: 77 additions & 0 deletions

File tree

docs/RESULTS.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,3 +44,18 @@ representation line are closed negative; v1 is final).
4444

4545
Published reference: Homo-GE2PE homograph 76.89% — v1 is above the
4646
published SOTA on this benchmark.
47+
48+
## heb-diac-small-1.0 — Hebrew student distillation (2026-08-20)
49+
50+
Logit KD from the s43 teacher (rababa_hebrew_byt5_s43/run-001/best):
51+
KL + CE on hebrew-v4, ByT5-small init. Harness: greedy decode, Nakdimon
52+
IMF test split (1,864 long sentences), same harness for both models
53+
(`src/gpu/modal_distill.py::evaluate`).
54+
55+
| Model | DER | CER |
56+
|---|---|---|
57+
| Teacher (s43, ByT5-base) | 24.79% | 22.14% |
58+
| **Student (ByT5-small, gate)** | **30.37%** | 24.47% |
59+
60+
Shrink cost +5.58pp — inside the ~5.6pp budget pre-accepted for this
61+
pair (rababa docs/DISTILL-SOURCE-PROMPT.md section 2).
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# heb-diac-small-1.0
2+
3+
Hebrew diacritization (adds nikud), client tier. ByT5-small student
4+
(300M) distilled from the s43 teacher via logit KD — same vocab
5+
(byte-level), so the teacher's soft distributions transfer directly.
6+
7+
Gate: student 30.37% DER vs teacher 24.79% (greedy, Nakdimon IMF test
8+
split, 1,864 long sentences) = +5.58pp, inside the ~5.6pp shrink budget
9+
pre-accepted for this pair (docs/DISTILL-SOURCE-PROMPT.md section 2).
10+
11+
```python
12+
from interscript_ml import Model
13+
model = Model.load("heb-diac-small-1.0")
14+
model.translate("שלום")
15+
```
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
format: imf-v1
2+
id: heb-diac-small-1.0
3+
task: diacritization
4+
source_script: Hebr
5+
target: Hebr
6+
tokenizer: bytes
7+
opset: 14
8+
decoder: kv
9+
precision: fp32
10+
license: BSD-3-Clause
11+
trained_from: >-
12+
logit KD from the s43 teacher (rababa_hebrew_byt5_s43/run-001/best,
13+
RELEASE-FROZEN): KL(teacher_soft || student_soft) + CE on the
14+
hebrew-v4 corpus, ByT5-small init google/byt5-small; checkpoint
15+
rababa-checkpoints:/rababa_hebrew_distill_small/run-001/best
16+
metrics:
17+
- name: der_teacher_greedy
18+
value: 24.79
19+
protocol: >-
20+
greedy decode; Nakdimon IMF test split, 1,864 long sentences;
21+
s43 teacher on the same harness as the student
22+
source: interscript/ml-models docs/RESULTS.md#heb-diac-small-1.0
23+
- name: der_student_greedy
24+
value: 30.37
25+
protocol: >-
26+
greedy decode; same harness; +5.58pp shrink cost, inside the
27+
~5.6pp budget pre-accepted for this pair
28+
(docs/DISTILL-SOURCE-PROMPT.md section 2)
29+
source: interscript/ml-models docs/RESULTS.md#heb-diac-small-1.0

models/metrics-sources.yaml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,3 +58,12 @@ fas-g2p-1.0:
5858
tables:
5959
- {row: "CER (v1 test split, greedy, editdistance)", as: cer}
6060
- {row: "SentenceBench homograph (ezafe-normalized)", as: sb_homograph_ezafe_norm}
61+
heb-diac-small-1.0:
62+
repo: interscript/interscript-ml
63+
ref: main
64+
path: docs/RESULTS.md
65+
anchor: heb-diac-small-1.0
66+
protocol: "greedy decode; Nakdimon IMF test split, 1,864 long sentences"
67+
tables:
68+
- {row: "Teacher (s43, ByT5-base)", column: DER, as: der_teacher_greedy}
69+
- {row: "Student (ByT5-small, gate)", column: DER, as: der_student_greedy}

src/gpu/modal_export.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,15 @@
9595
"test_data": "urdu-diacrit/test.jsonl",
9696
"probe": "اردو",
9797
},
98+
"heb-diac-small": {
99+
"volume": "/volumes/rababa-checkpoints",
100+
"checkpoint": "rababa_hebrew_distill_small/run-001/best",
101+
"metadata": "models/heb-diac-small/heb-diac-small-1.0.metadata.yaml",
102+
"readme": "models/heb-diac-small/heb-diac-small-1.0.README.md",
103+
"test_volume": "/datasets/rababa",
104+
"test_data": "nakdimon/test-imf.jsonl",
105+
"probe": "שלום",
106+
},
98107
"tha-g2p-base": {
99108
"volume": "/volumes/secryst-checkpoints",
100109
"checkpoint": "secryst_thai_g2p_distill_small/run-004/best",

0 commit comments

Comments
 (0)