diff --git a/docs/EXPERIMENTS.md b/docs/EXPERIMENTS.md index 0eca8e0..6b522de 100644 --- a/docs/EXPERIMENTS.md +++ b/docs/EXPERIMENTS.md @@ -152,7 +152,11 @@ All rows passed the CER parity gate at release. Readings: ## E4 — ara-diac-small-2.0 candidate (run-006-r7-muon) -- **Status:** queued (registered 2026-08-28 before launch). +- **Status:** COMPLETE (2026-08-29). **PASSED — 4.8218** full-set + windowed DER (gate ≤ 6.26; registered prediction 4.3–5.0; teacher r7 + reproduces 2.289 vs documented 2.2864). −3.44pp / 42% relative vs the + shipped 1.0 at identical architecture and size; matches the PKM arm's + 4.829 without memory layers. Released as ara-diac-small-2.0. - **Hypothesis:** the two measured wins compound — the r7 canonical teacher (better labels; ID 2.2864 vs 2.5793) plus the E3-adopted Muon optimizer (−2.727pp on r6 labels) — moving the client rung far diff --git a/docs/RESULTS.md b/docs/RESULTS.md index 61a74c4..b711e31 100644 --- a/docs/RESULTS.md +++ b/docs/RESULTS.md @@ -315,6 +315,25 @@ gap) — roughly additive, slightly sub-additive on memory. Residual ~2.2pp is domain coverage. Optimization is the dominant recoverable term of the distillation gap at the ByT5-small rung. +### run-006-r7-muon — E4: the 2.0 release candidate (2026-08-29) + +The two measured wins compounded on the vanilla architecture: r7 +canonical teacher (fresh greedy labels) + Muon optimizer, same +corpus/limits/seed family. Pre-registered E4 gate ≤ 6.26 (prediction +4.3–5.0): + +| Model | DER-CE (full 1,200) | +|---|---| +| Teacher (r7, in-run) | 2.2890% | +| **ByT5-small, r7 labels + Muon (run-006)** | **4.8218%** | +| ByT5-small, r6 labels + Muon (run-005) | 5.2945% | +| ByT5-small, r6 labels + AdamW (run-002, shipped 1.0) | 8.2590% | + +**4.8218 — gate passed; −3.44pp / 42% relative vs the shipped 1.0** at +identical architecture and artifact size. Matches the PKM arm's 4.829 +without the memory layers. Release: ara-diac-small-2.0 (run-006 +checkpoint; strict teacher+0.5pp still missed at +2.53pp, disclosed). + Leaderboard context (SadeedDiac-25, Misraj evaluator, zero-skip, harakat-projected DER-CE): the teacher tier (r6, 580M) at 2.5793 (reproduced at 2.5815, 2026-08-26) is the best dedicated model measured diff --git a/models/ara-diac-small/ara-diac-small-2.0.README.md b/models/ara-diac-small/ara-diac-small-2.0.README.md index a4ec7bf..e0076ab 100644 --- a/models/ara-diac-small/ara-diac-small-2.0.README.md +++ b/models/ara-diac-small/ara-diac-small-2.0.README.md @@ -1,17 +1,18 @@ # ara-diac-small-2.0 Arabic diacritization (haraqat restoration). Client-tier ByT5-small -student, identical corpus/labels/epochs to ara-diac-small-1.0 with one -variable changed: **the Muon optimizer** (E3 factorial). That single -change closes 2.96pp of the 5.68pp teacher-student gap: +student — the two measured wins of the campaign compounded on the same +architecture and artifact size as 1.0: the **r7 canonical teacher** +(2.2864; fresh greedy labels) and the **Muon optimizer** (E3-adopted). -- 1.0 (AdamW): 8.26 full-set windowed DER-CE -- **2.0 (Muon): 5.29** (teacher r6: 2.58; in-run reproduction 2.60) +- 1.0 (r6 labels, AdamW): 8.26 full-set windowed DER-CE +- 2.0 (r7 labels, Muon): **4.82** (teacher r7 in-run: 2.289) -The strict teacher+0.5pp gate is still missed; the residual decomposes -as ~0.70pp capacity + ~2.25pp domain coverage (E2/E3 factorial), and an -r7-teacher re-distillation is in flight. Identical IMF v1 contract: -dynamic fetch, sha256-verified, KV decode, margins JSON alongside. +A 42% error reduction, pre-registered as E4 (gate ≤ 6.26; prediction +4.3–5.0 — landed at 4.82). The strict teacher+0.5pp gate is still +missed (+2.53pp, disclosed); the E2/E3 factorial attributes the +residual to domain coverage. Identical IMF v1 contract: dynamic fetch, +sha256-verified, KV decode, margins JSON alongside. ```python from interscript_ml import Model diff --git a/models/ara-diac-small/ara-diac-small-2.0.metadata.yaml b/models/ara-diac-small/ara-diac-small-2.0.metadata.yaml index 4ecb408..79af49c 100644 --- a/models/ara-diac-small/ara-diac-small-2.0.metadata.yaml +++ b/models/ara-diac-small/ara-diac-small-2.0.metadata.yaml @@ -8,25 +8,26 @@ opset: 14 decoder: kv precision: fp32 license: BSD-3-Clause -trained_from: 'sequence-level KD from the r6 teacher (rababa_arabic_byt5/run-006-morph/best, - 2.5793 windowed DER-CE full protocol): the same 29,322 greedy teacher labels as - ara-diac-small-1.0, with the Muon optimizer (E3 factorial, vanilla student arm). - Checkpoint rababa-checkpoints:/rababa_arabic_distill_small/run-005-muon/best. The - optimizer alone closes 2.96pp of the 5.68pp teacher-student gap measured for the - 1.0 (AdamW) release: 8.259 -> 5.2945 full-set windowed DER-CE (teacher reproduces - 2.5997 in-run). Still misses the strict teacher+0.5pp gate — the residual decomposes - as ~0.70pp capacity + ~2.25pp domain coverage per the E2/E3 factorial; miss disclosed. - An r7-teacher re-distillation is in flight and expected to supersede as a later rung.' +trained_from: 'sequence-level KD from the r7 canonical teacher (rababa_arabic_byt5/run-007-news/best, + 2.2864 windowed DER-CE full protocol): fresh greedy r7 labels on the same r5-units + corpus/limits as ara-diac-small-1.0, Muon optimizer (E3-adopted), vanilla ByT5-small + (E4, pre-registered gate <= 6.26). Checkpoint + rababa-checkpoints:/rababa_arabic_distill_small/run-006-r7-muon/best. The two + measured wins compound: 8.259 -> 4.8218 full-set windowed DER-CE (teacher reproduces + 2.289 in-run vs documented 2.2864) — a 42% error reduction on the 1.0 release at the + same architecture and artifact size. Still misses the strict teacher+0.5pp gate + (+2.53pp; miss disclosed); the E2/E3 factorial attributes the residual to domain + coverage.' metrics: - name: der_teacher_fullset - value: 2.5997 + value: 2.289 protocol: windowed DER-CE (1400-byte windows, word-boundary split, greedy, haraqat-projected, Misraj evaluator); full 1,200-paragraph SadeedDiac-25; - in-run reproduction of the documented 2.5815/2.5793 - source: interscript/interscript-ml docs/RESULTS.md#run-005-muon + in-run reproduction of the documented 2.2864 (r7 canonical teacher) + source: interscript/interscript-ml docs/RESULTS.md#run-006-r7-muon - name: der_student_fullset - value: 5.2945 - protocol: same full-set harness; +2.69pp over the in-run teacher — the Muon - arm of the E3 2x2 factorial (vanilla student, optimizer as the only variable - vs the 8.259 AdamW release) - source: interscript/interscript-ml docs/RESULTS.md#run-005-muon + value: 4.8218 + protocol: same full-set harness; E4 (r7 teacher labels + Muon, vanilla + ByT5-small) vs the 8.259 AdamW/r6-labels 1.0 release — a 42% error + reduction at identical architecture and artifact size + source: interscript/interscript-ml docs/RESULTS.md#run-006-r7-muon diff --git a/src/gpu/modal_export.py b/src/gpu/modal_export.py index 4d28407..513931e 100644 --- a/src/gpu/modal_export.py +++ b/src/gpu/modal_export.py @@ -116,7 +116,7 @@ }, "ara-diac-small-2": { "volume": "/volumes/rababa-checkpoints", - "checkpoint": "rababa_arabic_distill_small/run-005-muon/best", + "checkpoint": "rababa_arabic_distill_small/run-006-r7-muon/best", "metadata": "models/ara-diac-small/ara-diac-small-2.0.metadata.yaml", "readme": "models/ara-diac-small/ara-diac-small-2.0.README.md", "test_volume": "/datasets/rababa",