Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions models/ara-diac-layerdrop/ara-diac-layerdrop-1.0.README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# ara-diac-layerdrop-1.0

The depth-cut client rung: ByT5-small with the encoder halved
(layers copied verbatim — width survives, depth is the compressible
axis). Muon + r7 labels + 6 epochs, the full lever set.

- Full-set DER: **5.78** (delta CI [3.03, 3.49]; full-depth peer 4.57)
- ~190M parameters; int8 ~190MB; **int4 ~95MB — the browser budget**
- The capacity law in one artifact: halving depth costs 1.21pp where
every width surgery collapsed

```python
from interscript_ml import Model
model = Model.load("ara-diac-layerdrop-1.0")
model.translate("قوله فحكمها في الوفاة")
```
30 changes: 30 additions & 0 deletions models/ara-diac-layerdrop/ara-diac-layerdrop-1.0.metadata.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
format: imf-v1
id: ara-diac-layerdrop-1.0
task: diacritization
source_script: Arab
target: Arab
tokenizer: bytes
opset: 14
decoder: kv
precision: fp32
license: BSD-3-Clause
trained_from: 'the depth-cut rung: ByT5-small with the encoder halved
12->6 (surviving layers copied VERBATIM from pretraining - both width-cut
approaches collapsed at 74.68/82.96 while this works), Muon optimizer, r7
teacher labels (the 2.0 recipe), 6 epochs. ~190M parameters (63% of
ByT5-small). Full-set windowed DER-CE 5.784, delta CI [3.033, 3.491] vs
in-run teacher 2.2921 — the depth premium over the full-depth G2a peer
(4.5701, CI [1.911, 2.352]) is 1.21pp with non-overlapping intervals.
The int4 variant (~95MB) is the browser-budget tier; margin-gated before
ship. Checkpoint
rababa-checkpoints:/rababa_arabic_distill_small/run-009-layerdrop-6ep/best.'
metrics:
- name: der_teacher_fullset
value: 2.2921
protocol: windowed DER-CE (1400-byte windows, word-boundary split, greedy,
haraqat-projected, Misraj evaluator); full 1,200-paragraph SadeedDiac-25
source: interscript/interscript-ml docs/RESULTS.md#layerdrop-6ep
- name: der_student_fullset
value: 5.784
protocol: same harness; paired bootstrap delta 3.2455 [3.033, 3.491]
source: interscript/interscript-ml docs/RESULTS.md#layerdrop-6ep
9 changes: 9 additions & 0 deletions src/gpu/modal_export.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,15 @@
"test_data": "arabic-sadeed-imf/test.jsonl",
"probe": "قوله",
},
"ara-diac-layerdrop": {
"volume": "/volumes/rababa-checkpoints",
"checkpoint": "rababa_arabic_distill_small/run-009-layerdrop-6ep/best",
"metadata": "models/ara-diac-layerdrop/ara-diac-layerdrop-1.0.metadata.yaml",
"readme": "models/ara-diac-layerdrop/ara-diac-layerdrop-1.0.README.md",
"test_volume": "/datasets/rababa",
"test_data": "arabic-sadeed-imf/test.jsonl",
"probe": "قوله",
},
"urd-diac": {
"volume": "/volumes/urdu-diacrit-checkpoints",
"checkpoint": "urdu_diacrit/run-001/best",
Expand Down
Loading