Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/EXPERIMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,11 @@ All rows passed the CER parity gate at release. Readings:

## E4 — ara-diac-small-2.0 candidate (run-006-r7-muon)

- **Status:** queued (registered 2026-08-28 before launch).
- **Status:** COMPLETE (2026-08-29). **PASSED — 4.8218** full-set
windowed DER (gate ≤ 6.26; registered prediction 4.3–5.0; teacher r7
reproduces 2.289 vs documented 2.2864). −3.44pp / 42% relative vs the
shipped 1.0 at identical architecture and size; matches the PKM arm's
4.829 without memory layers. Released as ara-diac-small-2.0.
- **Hypothesis:** the two measured wins compound — the r7 canonical
teacher (better labels; ID 2.2864 vs 2.5793) plus the E3-adopted
Muon optimizer (−2.727pp on r6 labels) — moving the client rung far
Expand Down
19 changes: 19 additions & 0 deletions docs/RESULTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,25 @@ gap) — roughly additive, slightly sub-additive on memory. Residual
~2.2pp is domain coverage. Optimization is the dominant recoverable
term of the distillation gap at the ByT5-small rung.

### run-006-r7-muon — E4: the 2.0 release candidate (2026-08-29)

The two measured wins compounded on the vanilla architecture: r7
canonical teacher (fresh greedy labels) + Muon optimizer, same
corpus/limits/seed family. Pre-registered E4 gate ≤ 6.26 (prediction
4.3–5.0):

| Model | DER-CE (full 1,200) |
|---|---|
| Teacher (r7, in-run) | 2.2890% |
| **ByT5-small, r7 labels + Muon (run-006)** | **4.8218%** |
| ByT5-small, r6 labels + Muon (run-005) | 5.2945% |
| ByT5-small, r6 labels + AdamW (run-002, shipped 1.0) | 8.2590% |

**4.8218 — gate passed; −3.44pp / 42% relative vs the shipped 1.0** at
identical architecture and artifact size. Matches the PKM arm's 4.829
without the memory layers. Release: ara-diac-small-2.0 (run-006
checkpoint; strict teacher+0.5pp still missed at +2.53pp, disclosed).

Leaderboard context (SadeedDiac-25, Misraj evaluator, zero-skip,
harakat-projected DER-CE): the teacher tier (r6, 580M) at 2.5793
(reproduced at 2.5815, 2026-08-26) is the best dedicated model measured
Expand Down
19 changes: 10 additions & 9 deletions models/ara-diac-small/ara-diac-small-2.0.README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,18 @@
# ara-diac-small-2.0

Arabic diacritization (haraqat restoration). Client-tier ByT5-small
student, identical corpus/labels/epochs to ara-diac-small-1.0 with one
variable changed: **the Muon optimizer** (E3 factorial). That single
change closes 2.96pp of the 5.68pp teacher-student gap:
student — the two measured wins of the campaign compounded on the same
architecture and artifact size as 1.0: the **r7 canonical teacher**
(2.2864; fresh greedy labels) and the **Muon optimizer** (E3-adopted).

- 1.0 (AdamW): 8.26 full-set windowed DER-CE
- **2.0 (Muon): 5.29** (teacher r6: 2.58; in-run reproduction 2.60)
- 1.0 (r6 labels, AdamW): 8.26 full-set windowed DER-CE
- 2.0 (r7 labels, Muon): **4.82** (teacher r7 in-run: 2.289)

The strict teacher+0.5pp gate is still missed; the residual decomposes
as ~0.70pp capacity + ~2.25pp domain coverage (E2/E3 factorial), and an
r7-teacher re-distillation is in flight. Identical IMF v1 contract:
dynamic fetch, sha256-verified, KV decode, margins JSON alongside.
A 42% error reduction, pre-registered as E4 (gate ≤ 6.26; prediction
4.3–5.0 — landed at 4.82). The strict teacher+0.5pp gate is still
missed (+2.53pp, disclosed); the E2/E3 factorial attributes the
residual to domain coverage. Identical IMF v1 contract: dynamic fetch,
sha256-verified, KV decode, margins JSON alongside.

```python
from interscript_ml import Model
Expand Down
35 changes: 18 additions & 17 deletions models/ara-diac-small/ara-diac-small-2.0.metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,26 @@ opset: 14
decoder: kv
precision: fp32
license: BSD-3-Clause
trained_from: 'sequence-level KD from the r6 teacher (rababa_arabic_byt5/run-006-morph/best,
2.5793 windowed DER-CE full protocol): the same 29,322 greedy teacher labels as
ara-diac-small-1.0, with the Muon optimizer (E3 factorial, vanilla student arm).
Checkpoint rababa-checkpoints:/rababa_arabic_distill_small/run-005-muon/best. The
optimizer alone closes 2.96pp of the 5.68pp teacher-student gap measured for the
1.0 (AdamW) release: 8.259 -> 5.2945 full-set windowed DER-CE (teacher reproduces
2.5997 in-run). Still misses the strict teacher+0.5pp gate — the residual decomposes
as ~0.70pp capacity + ~2.25pp domain coverage per the E2/E3 factorial; miss disclosed.
An r7-teacher re-distillation is in flight and expected to supersede as a later rung.'
trained_from: 'sequence-level KD from the r7 canonical teacher (rababa_arabic_byt5/run-007-news/best,
2.2864 windowed DER-CE full protocol): fresh greedy r7 labels on the same r5-units
corpus/limits as ara-diac-small-1.0, Muon optimizer (E3-adopted), vanilla ByT5-small
(E4, pre-registered gate <= 6.26). Checkpoint
rababa-checkpoints:/rababa_arabic_distill_small/run-006-r7-muon/best. The two
measured wins compound: 8.259 -> 4.8218 full-set windowed DER-CE (teacher reproduces
2.289 in-run vs documented 2.2864) — a 42% error reduction on the 1.0 release at the
same architecture and artifact size. Still misses the strict teacher+0.5pp gate
(+2.53pp; miss disclosed); the E2/E3 factorial attributes the residual to domain
coverage.'
metrics:
- name: der_teacher_fullset
value: 2.5997
value: 2.289
protocol: windowed DER-CE (1400-byte windows, word-boundary split, greedy,
haraqat-projected, Misraj evaluator); full 1,200-paragraph SadeedDiac-25;
in-run reproduction of the documented 2.5815/2.5793
source: interscript/interscript-ml docs/RESULTS.md#run-005-muon
in-run reproduction of the documented 2.2864 (r7 canonical teacher)
source: interscript/interscript-ml docs/RESULTS.md#run-006-r7-muon
- name: der_student_fullset
value: 5.2945
protocol: same full-set harness; +2.69pp over the in-run teacher — the Muon
arm of the E3 2x2 factorial (vanilla student, optimizer as the only variable
vs the 8.259 AdamW release)
source: interscript/interscript-ml docs/RESULTS.md#run-005-muon
value: 4.8218
protocol: same full-set harness; E4 (r7 teacher labels + Muon, vanilla
ByT5-small) vs the 8.259 AdamW/r6-labels 1.0 release — a 42% error
reduction at identical architecture and artifact size
source: interscript/interscript-ml docs/RESULTS.md#run-006-r7-muon
2 changes: 1 addition & 1 deletion src/gpu/modal_export.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@
},
"ara-diac-small-2": {
"volume": "/volumes/rababa-checkpoints",
"checkpoint": "rababa_arabic_distill_small/run-005-muon/best",
"checkpoint": "rababa_arabic_distill_small/run-006-r7-muon/best",
"metadata": "models/ara-diac-small/ara-diac-small-2.0.metadata.yaml",
"readme": "models/ara-diac-small/ara-diac-small-2.0.README.md",
"test_volume": "/datasets/rababa",
Expand Down
Loading