Skip to content

base student at greedy: 3.53% — tier inversion - #50

Merged
ronaldtse merged 2 commits into
mainfrom
docs/base-greedy
Aug 25, 2026
Merged

base student at greedy: 3.53% — tier inversion#50
ronaldtse merged 2 commits into
mainfrom
docs/base-greedy

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

Summary

  • base student measured at greedy: 3.53% PER / 90.48% EM — same-protocol picture complete: teacher 1.25 / base 3.53 / small 2.85
  • tier inversion at greedy: the quantized client student outperforms the server-tier student (0.68pp gap, beyond the 0.08pp parity delta); beam-4 had the tiers reversed
  • RESULTS, metadata, models.yaml, metrics chain, frontier figure, paper catalog/decode section updated

Test plan

  • CI green (metrics validated locally)

Ronald Tse added 2 commits August 26, 2026 06:46
Same-protocol picture complete for Thai: teacher 1.25 / base 3.53 /
small 2.85. The client tier outperforms the server tier at greedy
(0.68pp gap, beyond the 0.08pp parity delta); the beam-4 figures had
the tiers reversed. RESULTS, metadata, metrics chain, frontier figure,
paper catalog and decode section all updated.
@ronaldtse
ronaldtse merged commit 6a55c87 into main Aug 25, 2026
10 checks passed
@ronaldtse ronaldtse mentioned this pull request Aug 25, 2026
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant