Skip to content

Commit c80a78e

Browse files
authored
Merge pull request #23 from interscript/feat/tiny-tier-reconstruction
feat(distill): tiny client tier — byte-vocab students + microkimi bridge KD
2 parents 462bff2 + 2e97c00 commit c80a78e

42 files changed

Lines changed: 671 additions & 147 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/release.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ jobs:
133133
uses: actions/attest-build-provenance@v1
134134
continue-on-error: true
135135
with:
136-
subject-name: github.com/interscript/ml-models
136+
subject-name: github.com/interscript/interscript-ml
137137
subject-digest: sha256:${{ github.sha }}
138138

139139
publish-hf:

TODO.distribution/00-overview.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,23 +18,23 @@ zero manual steps after a `git tag`.
1818
## Channel architecture (hybrid, MECE)
1919

2020
```
21-
Source code → github.com/interscript/ml-models
21+
Source code → github.com/interscript/interscript-ml
2222
(this repo: framework, configs, tests, CI/CD,
2323
release changelog, issue tracking)
2424
2525
Trained weights → huggingface.co/interscript/<task_name>
2626
(PyTorch, full) (model card, datasets, transformers.js hook,
2727
free Cloudflare-backed CDN, researcher entry)
2828
29-
Browser-ready ONNX → github.com/interscript/ml-models/releases/tag/<task>-v<X.Y.Z>
29+
Browser-ready ONNX → github.com/interscript/interscript-ml/releases/tag/<task>-v<X.Y.Z>
3030
(one .onnx asset per release; checksums attached;
3131
stable immutable URLs; programmatic via `gh`)
3232
33-
CDN mirror → jsdelivr.net/gh/interscript/ml-models@<tag>/...
33+
CDN mirror → jsdelivr.net/gh/interscript/interscript-ml@<tag>/...
3434
(global edge cache of the GH release assets;
3535
CORS-friendly; no rate limits for end users)
3636
37-
JS glue package → npm: @interscript/models
37+
JS glue package → npm: npm `secryst` (manifest now the models.yaml index)
3838
(tiny manifest of current versions per task;
3939
consumed by interscript-ts to resolve URLs)
4040
@@ -59,7 +59,7 @@ Each channel serves a real audience. None is redundant.
5959
- `01-github-releases.md` — release asset layout, tag conventions, asset naming
6060
- `02-huggingface-hub.md` — HF org, model cards, datasets, auto-conversion
6161
- `03-cdn-strategy.md` — jsdelivr + GH releases URL convention, fallback chain
62-
- `04-npm-packages.md``@interscript/models` manifest package
62+
- `04-npm-packages.md``npm `secryst` (manifest now the models.yaml index)` manifest package
6363
- `05-ruby-model-cache.md` — gem-side download + verify + atomic write
6464
- `06-versioning-scheme.md` — per-task semver, breaking-change rules
6565
- `07-ci-cd.md``.github/workflows/release.yml` full design

TODO.distribution/01-github-releases.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ Asset filenames follow `<task>-<variant>.<ext>`:
5757
The CDN URL is computable from `<task>` + `<version>` + `<variant>`:
5858

5959
```
60-
https://github.com/interscript/ml-models/releases/download/<task>-v<ver>/<task>-<variant>.onnx
60+
https://github.com/interscript/interscript-ml/releases/download/<task>-v<ver>/<task>-<variant>.onnx
6161
```
6262

6363
No central manifest needed to compute URLs. The convention IS the API.

TODO.distribution/03-cdn-strategy.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,12 @@ Zero cost to us.
1313

1414
Primary (CDN-cached):
1515
```
16-
https://cdn.jsdelivr.net/gh/interscript/ml-models@<task>-v<x.y.z>/<task>.onnx
16+
https://cdn.jsdelivr.net/gh/interscript/interscript-ml@<task>-v<x.y.z>/<task>.onnx
1717
```
1818

1919
Fallback (direct GH):
2020
```
21-
https://github.com/interscript/ml-models/releases/download/<task>-v<x.y.z>/<task>.onnx
21+
https://github.com/interscript/interscript-ml/releases/download/<task>-v<x.y.z>/<task>.onnx
2222
```
2323

2424
jsdelivr mirrors GitHub Releases content via the `@<tag>` syntax. The
@@ -89,19 +89,19 @@ Result: second visit loads instantly. Models cached indefinitely
8989
becomes:
9090

9191
```typescript
92-
const DEFAULT_MODEL_BASE = "https://cdn.jsdelivr.net/gh/interscript/ml-models@"
92+
const DEFAULT_MODEL_BASE = "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@"
9393
```
9494

9595
`loadModel({ kind: "rababa", id: "default" })` resolves to:
9696
1. Look up `<task>` from id (e.g. `"default"``"rababa_arabic"`)
97-
2. Look up `<version>` from `@interscript/models` npm package
97+
2. Look up `<version>` from `npm `secryst` (manifest now the models.yaml index)` npm package
9898
3. Build URL: `${DEFAULT_MODEL_BASE}<task>-v<version>/<task>.onnx`
9999
4. Fetch with fallback chain
100100
5. Cache in IndexedDB
101101

102102
## Version manifest (npm)
103103

104-
The `@interscript/models` npm package exposes a tiny JSON manifest:
104+
The `npm `secryst` (manifest now the models.yaml index)` npm package exposes a tiny JSON manifest:
105105

106106
```json
107107
{
@@ -114,7 +114,7 @@ The `@interscript/models` npm package exposes a tiny JSON manifest:
114114
`interscript-ts` reads this at startup. Users can pin a version via:
115115

116116
```typescript
117-
setModelBase("https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0/")
117+
setModelBase("https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0/")
118118
```
119119

120120
## Acceptance

TODO.distribution/04-npm-packages.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ go live. Adds:
2222
- `transliterateAsync()` for ML funcalls
2323
- Model provisioning layer
2424

25-
### 2. `@interscript/models` (new, tiny)
25+
### 2. `npm `secryst` (manifest now the models.yaml index)` (new, tiny)
2626

2727
A pure-JSON manifest package. No JS code. Just a version index:
2828

@@ -34,8 +34,8 @@ A pure-JSON manifest package. No JS code. Just a version index:
3434
"version": "1.0.0",
3535
"size_bytes": 6197600,
3636
"sha256": "3a7f2b...",
37-
"url": "https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0/rababa_arabic.onnx",
38-
"vocab_url": "https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0/rababa_arabic-vocab.json"
37+
"url": "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0/rababa_arabic.onnx",
38+
"vocab_url": "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0/rababa_arabic-vocab.json"
3939
},
4040
"rababa_hebrew": { ... },
4141
"secryst_thai_ipa": { ... }
@@ -72,8 +72,8 @@ Same pattern for `@interscript/model-rababa-hebrew`,
7272

7373
```
7474
1. CI builds ONNX, attaches to GH release
75-
2. CI updates @interscript/models manifest
76-
3. CI runs: npm version patch && npm publish for @interscript/models
75+
2. CI updates npm `secryst` (manifest now the models.yaml index) manifest
76+
3. CI runs: npm version patch && npm publish for npm `secryst` (manifest now the models.yaml index)
7777
4. Users running `interscript-ts@0.2.x` see the new version on next
7878
app restart (model cache invalidates by URL change)
7979
5. Users running `@interscript/model-rababa-arabic` opt-in via
@@ -82,16 +82,16 @@ Same pattern for `@interscript/model-rababa-hebrew`,
8282

8383
## Why three packages (not one)
8484

85-
- **MECE.** Runtime code (interscript-ts) vs version manifest (@interscript/models) vs bundled binary (@interscript/model-*). Three concerns, three packages.
86-
- **Bundle size.** Browser apps want interscript-ts + @interscript/models (small). They fetch ONNX at runtime from CDN. Bundle stays small.
85+
- **MECE.** Runtime code (interscript-ts) vs version manifest (npm `secryst` (manifest now the models.yaml index)) vs bundled binary (@interscript/model-*). Three concerns, three packages.
86+
- **Bundle size.** Browser apps want interscript-ts + npm `secryst` (manifest now the models.yaml index) (small). They fetch ONNX at runtime from CDN. Bundle stays small.
8787
- **Air-gap friendliness.** Enterprises install @interscript/model-* via private npm mirror. No CDN calls.
8888
- **Versioning independence.** Runtime API breaks != model version bumps != model retrain.
8989

90-
## Why `@interscript/models` not embedded in `interscript-ts`
90+
## Why `npm `secryst` (manifest now the models.yaml index)` not embedded in `interscript-ts`
9191

9292
If manifest lives in `interscript-ts`, every model release forces a
9393
runtime bump. Decouples release cadence:
94-
- Model retrain: bump `@interscript/models` only.
94+
- Model retrain: bump `npm `secryst` (manifest now the models.yaml index)` only.
9595
- Runtime API change: bump `interscript-ts` only.
9696

9797
## Install ergonomics
@@ -104,12 +104,12 @@ npm install interscript-ts
104104
npm install interscript-ts @interscript/model-rababa-arabic
105105

106106
# Pin to specific model version:
107-
npm install @interscript/models@1.2.0
107+
npm install npm `secryst` (manifest now the models.yaml index)@1.2.0
108108
```
109109

110110
## Versioning
111111

112-
`@interscript/models` uses CalVer-style: `1.<month>.0` so users can
112+
`npm `secryst` (manifest now the models.yaml index)` uses CalVer-style: `1.<month>.0` so users can
113113
see at a glance when the manifest was last updated. (e.g. `1.8.0` =
114114
August 2026 release.)
115115

@@ -118,7 +118,7 @@ August 2026 release.)
118118

119119
## Acceptance
120120

121-
- [ ] `@interscript/models` published (placeholder manifest)
121+
- [ ] `npm `secryst` (manifest now the models.yaml index)` published (placeholder manifest)
122122
- [ ] `@interscript/model-rababa-arabic` published (placeholder)
123-
- [ ] `interscript-ts@0.2.0` released depending on `@interscript/models`
123+
- [ ] `interscript-ts@0.2.0` released depending on `npm `secryst` (manifest now the models.yaml index)`
124124
- [ ] README documents install patterns for all three combos

TODO.distribution/05-ruby-model-cache.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ rababa_arabic:
7474
version: "1.0.0"
7575
sha256: "3a7f2b..."
7676
size_bytes: 6197600
77-
url_base: "https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0"
77+
url_base: "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0"
7878
```
7979
8080
Override at runtime:

TODO.distribution/06-versioning-scheme.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -65,8 +65,8 @@ Following SemVer pre-release syntax:
6565
Pre-releases are:
6666
- Uploaded to GH Releases as drafts
6767
- Published to HF Hub with `stage: prerelease` tag
68-
- **NOT** added to `@interscript/models` manifest until promoted
69-
- Available via explicit URL: `cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0-rc.1/`
68+
- **NOT** added to `npm `secryst` (manifest now the models.yaml index)` manifest until promoted
69+
- Available via explicit URL: `cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0-rc.1/`
7070

7171
## Versioning the framework itself
7272

@@ -79,7 +79,7 @@ Framework: `0.1.0` → `0.2.0` → ... → `1.0.0` (after first production task)
7979

8080
When a task version is superseded:
8181
1. New release tag pushes normally.
82-
2. `@interscript/models` manifest points `default` → new version.
82+
2. `npm `secryst` (manifest now the models.yaml index)` manifest points `default` → new version.
8383
3. Old version stays in GH Releases (immutable).
8484
4. Old version's HF model card gets a banner: "Superseded by v1.1.0".
8585
5. `Interscript.clear_cache!(older_than: 90.days)` eventually prunes.

TODO.distribution/07-ci-cd.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
3. Benchmarks (DER/PER, latency)
1212
4. GH Release with auto-generated notes
1313
5. HF Hub upload (PyTorch + ONNX + card)
14-
6. `@interscript/models` npm manifest bump
14+
6. `npm `secryst` (manifest now the models.yaml index)` npm manifest bump
1515
7. (Optional) `@interscript/model-<task>` npm publish
1616

1717
Zero manual steps after the tag.

TODO.distribution/08-supply-chain.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -60,12 +60,12 @@ Every release asset has `<asset>.provenance.json`:
6060
"runner": "ubuntu-latest"
6161
},
6262
"source": {
63-
"repository": "github.com/interscript/ml-models",
63+
"repository": "github.com/interscript/interscript-ml",
6464
"ref": "refs/tags/rababa_arabic-v1.0.0",
6565
"commit_sha": "abc123def456..."
6666
},
6767
"materials": [
68-
{ "uri": "git+github.com/interscript/ml-models", "digest": { "sha1": "abc123..." } },
68+
{ "uri": "git+github.com/interscript/interscript-ml", "digest": { "sha1": "abc123..." } },
6969
{ "uri": "pypi://torch@2.2.0", "digest": { "sha256": "..." } },
7070
{ "uri": "pypi://transformers@4.42.0", "digest": { "sha256": "..." } }
7171
],
@@ -84,7 +84,7 @@ Consumers can verify with `gh attestation verify`:
8484

8585
```bash
8686
gh attestation verify rababa_arabic.onnx \
87-
--repo interscript/ml-models \
87+
--repo interscript/interscript-ml \
8888
--predicate-type https://slsa.dev/provenance/v1
8989
```
9090

@@ -114,16 +114,16 @@ private key to manage (keyless signing via short-lived cert).
114114

115115
## Manifest package ties it together
116116

117-
`@interscript/models` manifest carries all three:
117+
`npm `secryst` (manifest now the models.yaml index)` manifest carries all three:
118118

119119
```json
120120
{
121121
"rababa_arabic": {
122122
"version": "1.0.0",
123-
"url": "https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0/rababa_arabic.onnx",
123+
"url": "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0/rababa_arabic.onnx",
124124
"sha256": "3a7f2b...",
125-
"provenance_url": "https://github.com/interscript/ml-models/attestations/...",
126-
"sigstore_bundle_url": "https://cdn.jsdelivr.net/gh/interscript/ml-models@rababa_arabic-v1.0.0/rababa_arabic.onnx.bundle"
125+
"provenance_url": "https://github.com/interscript/interscript-ml/attestations/...",
126+
"sigstore_bundle_url": "https://cdn.jsdelivr.net/gh/interscript/interscript-ml@rababa_arabic-v1.0.0/rababa_arabic.onnx.bundle"
127127
}
128128
}
129129
```
@@ -148,7 +148,7 @@ Three threats, three mitigations. MECE.
148148

149149
If our signing identity is compromised:
150150
1. Revoke the cert via Sigstore Rekor entry (public log).
151-
2. Yank the manifest entry (publish new `@interscript/models`).
151+
2. Yank the manifest entry (publish new `npm `secryst` (manifest now the models.yaml index)`).
152152
3. Cut new release with new signing identity.
153153
4. Announce via RSS / mailing list / model card banner.
154154

TODO.distribution/09-model-cards.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ Interscript.transliterate("<map_code>", "<input>")
110110
### Direct ONNX (any language)
111111
112112
\`\`\`bash
113-
curl -LO https://github.com/interscript/ml-models/releases/download/<task>-v<VERSION>/<task>.onnx
113+
curl -LO https://github.com/interscript/interscript-ml/releases/download/<task>-v<VERSION>/<task>.onnx
114114
# Use onnxruntime to load + run
115115
\`\`\`
116116
@@ -119,7 +119,7 @@ curl -LO https://github.com/interscript/ml-models/releases/download/<task>-v<VER
119119
\`\`\`bibtex
120120
@software{interscript_<task>,
121121
title = {Interscript <task_full_name>},
122-
url = {https://github.com/interscript/ml-models},
122+
url = {https://github.com/interscript/interscript-ml},
123123
version = {<VERSION>}
124124
}
125125
\`\`\`

0 commit comments

Comments
 (0)