Skip to content

docs: GKD negative verdict #194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ronaldtse merged 1 commit into from
Sep 6, 2026
Merged

docs: GKD negative verdict #194

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/RESULTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -582,3 +582,23 @@ Consequences, stated plainly:
Predictions for all five frontier runs publish alongside this entry
(release frontier-predictions-v1) so the numbers above are re-derivable
by anyone.

## ara-diac-small-2-gkd — on-policy distillation NEGATIVE: 6.0036 (2026-09-06)

The on-policy rung (GKD: student-generated mistakes scored by the
teacher, 10,995 steps, labels sha e70ce991): teacher reproduces 2.289,
student **6.0036** full-set, paired bootstrap delta 3.4083 [3.109,
3.743], p=0 — gate FAILED, and 1.43pp WORSE than the off-policy
sequence-KD rung it was meant to improve (G2a 4.5701).

This completes the residual-attribution program with a clean pattern:
every lever tested fails to close the teacher-student gap —
* classical corpus add (G2b): flat-negative
* register swap (E6): negative
* on-policy distillation (GKD): negative, worse than off-policy
* memory layers (PKM): real but small (0.70pp), below bar
* epochs: small (0.25pp)
The residual is not data, not domain coverage, not an off/on-policy
deficit. It is a property of the compression itself at this rung —
the honest open question for the paper. On-policy stays listed in
Paper B's future work with its measured negative attached.
Loading