Skip to content

spec(ml): train_hslm -- main() trains the whole 3-block HSLM with STE and LAMB, writes MSLH checkpoints (Refs #10372) - #10387

Merged
gHashTag merged 18 commits into
masterfrom
feat/train-hslm
Oct 19, 2026
Merged

gHashTag merged 18 commits into
masterfrom
feat/train-hslm

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Refs #10372

What

specs/ml/train_hslm.t27. Its main() trains the whole 3-block HSLM from nn/hslm (ctx 27, vocab 729, embed 243, ff 729, 3 heads), on the real HTOK corpus from #10380.

It reuses these through use, without copying them:

  • nn/hslm: forward, backward and cross-entropy
  • nn/ternary_ste: quantize_twn
  • nn/hslm_data: the MSLH header and layout

What the spec adds:

  • STE. There are 2130138 f64 parameters in 30 segments. All 19 ternary matrices keep an f64 shadow, requantized by TWN after every update with one alpha per matrix. The embedding table, RMSNorm gains and biases are trained directly.
  • Optimizer: LAMB. beta1 0.9, beta2 0.999, eps 1e-6. Decoupled weight decay 0.01 applies to the matrices and the embedding, not to gains or biases. The per-segment trust ratio ||w||/||r|| is clamped to 10. This follows trinity-training lambStepSlice.
  • Schedule. lr 1e-3, linear warmup over 2000 updates, then cosine down to 1e-6 at total. Global-norm clip 1.0. Batch is 66 windows of 27 tokens, with starts drawn from a pinned-seed xorshift64.
  • Config. /tmp/hslm/train.cfg is optional and holds key=value lines. Keys: steps total warmup eval_every ckpt_every seed speed resume synth eval_windows loss_positions gradcheck. Defaults: 32000 / 32000 / 2000 / 500 / 2000 / 1729 / 2 / 0 / 0 / 243 / 27 / 0.
  • Eval line. Every eval_every steps, one line goes to stdout. It carries the step, the minibatch train loss and its ppl, the mean train loss since the last line, the held-out loss over 243 fixed evenly spaced valid.tok windows and its ppl, tok/s, s/step, lr and the pre-clip gnorm.
  • Checkpoints. One is written every ckpt_every steps and at the end:
    • /tmp/hslm/ckpt_<step>.bin is MSLH: the nn/hslm_data header plus its layout, 4971796 bytes.
    • /tmp/hslm/ckpt_<step>.state holds the full resumable state, because MSLH has no attention matrices, embedding or gains. It is a 64-byte T27S header (step, rng, beta powers, loss), then p, m and v as f64 LE: 51123376 bytes.
    • Both files are read back and compared byte by byte. Any mismatch makes main() assert, which gives a nonzero exit.
  • Resume. resume=<step> restarts from the .state file and continues bit-identically.

Speed: same model, honest kernels

speed what changes s/step tok/s
0 the nn/hslm reference kernels as they are 19.91 89.5
1 ternary_matmul as a branch-free row axpy (bit-identical); trit_int and outer_add branch-free; ternary_matmul_t_add with 4 partial sums (backward only) 2.05 868
2 (default) level 1, plus positions 0..25 skip the head and, in the last block, everything except the K/V that position 26 reads 1.49 1196
  • The tests check that speed 1 and speed 2 give speed-0 logits to the bit.
  • On every step of the runs below, the losses at speeds 0, 1 and 2 are identical.
  • 32000 steps × 1.49 s comes to about 13.2 h on one core, inside the 48 h budget.

50-step measurement (real tokens, speed 2, seed 1729)

Data: train.tok has 8557893 tokens and valid.tok has 7689354.

step 0  valid_loss 6.5952 valid_ppl 731.56
step 10 train_loss 6.6072 avg_loss 6.6068 valid_loss 6.5952 tok_per_s 1229.8 s_per_step 1.4490 lr 0.00000500 gnorm 0.1436
step 20 train_loss 6.5829 avg_loss 6.6056 valid_loss 6.5949 tok_per_s 1205.7 s_per_step 1.4780 lr 0.00001000 gnorm 0.1386
step 30 train_loss 6.5933 avg_loss 6.6113 valid_loss 6.5944 tok_per_s 1186.4 s_per_step 1.5020 lr 0.00001500 gnorm 0.1513
step 40 train_loss 6.6336 avg_loss 6.6139 valid_loss 6.5935 tok_per_s 1182.5 s_per_step 1.5070 lr 0.00002000 gnorm 0.1490
step 50 train_loss 6.5931 avg_loss 6.5960 valid_loss 6.5916 valid_ppl 728.95 tok_per_s 1174.7 s_per_step 1.5170 lr 0.00002500 gnorm 0.1582
done steps 50 train_s 74.53 s_per_step 1.4906 tok_per_s 1195.5 io_fail 0 rc 0
  • I rebuilt at 47aff93 (the duplicate-body fix) and reran the 50 steps. Every printed loss matched to the last digit. This run measured 1.534 s/step and 1161 tok/s while test-report shared the box.

  • After 50 steps the learning rate is still only 2.5e-5 of a 2000-step warmup, so the held-out loss moves just 6.5952 -> 6.5916; ln 729 = 6.5917.

  • The checkpoints at steps 25 and 50 were written and read back byte-equal.

  • Resume check: I ran again with resume=25 to step 50. The resulting ckpt_50.bin and ckpt_50.state matched the uninterrupted run under cmp.

Tests

t27c test-report specs/ml/train_hslm.t27 gives 20/20 pass, 0 vacuous: the 14 below plus the 6 for the 27-position objective (next section). The original 14:

  • sampling is deterministic for a seed, and window starts leave room for the target
  • the LR at step 0, at the end of warmup, at the midpoint, at the end and after the end
  • clipping bounds the global norm and leaves a small gradient alone
  • the parameter count
  • an MSLH buffer round trip is byte-equal and parses (step, loss, size), and one flipped byte is found
  • the state stream round trips bit-exactly
  • speed 1 and speed 2 logits equal speed 0, and the speed-2 input gradient matches within 1e-12
  • two LAMB steps on a tiny fixed slice lower the loss

Differences from R5 / the Zig trainer (trinity-training @ 3e9648c)

  • f64 instead of f32.
  • LAMB instead of AdamW.
  • One global clip instead of a per-tensor clip.
  • Warmup plus cosine instead of the phi schedule.
  • The embedding table and the gains are trained.
  • The gradient is that of the mean loss over the last loss_positions positions (default 27; loss_positions=1 is R5's last-position objective), each position's with earlier K/V treated as constants (nn/hslm section 5).
  • Alphas are recomputed by TWN, not learned.
  • The VSA table is nn/hslm's hash.
  • MSLH body floats are RNE f32 casts; the header loss is encode_f32.

Lab commands (Railway fs-lab, t27c at /root/t27, zig 0.16.0)

cd /root/t27-hslm-train
PATH=/opt/zig:$PATH /root/t27/target/release/t27c compile specs/ml/train_hslm.t27 --backend ml
/opt/zig/zig build-exe -O ReleaseFast specs/ml/train_hslm.zig -femit-bin=/root/t27-hslm-train/train_hslm
printf "steps=50\neval_every=10\nckpt_every=25\neval_windows=243\nspeed=2\n" > /tmp/hslm/train.cfg
nohup /root/t27-hslm-train/train_hslm > run50_speed2.log 2>&1 &
PATH=/opt/zig:$PATH /root/t27/target/release/t27c test-report specs/ml/train_hslm.t27 --verbose
  • The ReleaseFast binary is at /root/t27-hslm-train/train_hslm (sha256 549bb065...334c, built from 47aff93).
  • I did not start the long run. With no train.cfg, the defaults run all 32000 steps.

The 27-position objective (3b5c87a, owner's choice 2026-10-18)

Why. A held-out eval (specs/ml/eval_hslm.t27, #10467) showed that the last-position objective made the full-sequence loss worse. Between steps 2000 and 4000, last_loss went 5.75 -> 4.50 while full_loss rose 6.20 -> 6.54.

What changed.

  • The new cfg key loss_positions takes 1..27, default 27.
  • Each of the last k positions gets a full forward, a cross-entropy against its next token, and a backward with dlogits scaled by 1/k. The window loss is the mean of the k losses.
  • loss_positions=1 is the old step, to the bit.
  • The eval line names the objective: objective last27 train_loss_last27 .. train_ppl_last27 .. avg_loss_last27 ...
    • valid_loss / valid_ppl are still the last-position number, so they compare with run 1.
    • valid_full_loss / valid_full_ppl are new: the mean over all 27 positions of the same 243 held-out windows.
  • tr_window_forward, tr_window_loss, tr_valid_loss and the checkpoint reader are unchanged. eval_hslm.t27 at a6d8af0 still gives 10/10 tests against the new train_hslm and builds with --backend ml.

New tests (6).

  • loss_positions_one_is_the_last_position_step_to_the_bit: a k=1 step equals the old step, with 0 of 2130138 parameters differing. A k=27 step moves more than 10^6 of them.
  • the_27_position_loss_is_the_mean_of_27_forward_only_cross_entropies: checked for k = 27, 7 and 1, with gap < 1e-12. The measured gap is 0.
  • uniform_logits_give_ln_729_at_every_position: with the head zeroed, the loss equals ln 729 = 6.591673732009 within 1e-12. The output-bias gradient equals 1/729 - count_j/27 within 1e-15 (measured 3.5e-17).
  • Three finite-difference tests, described next.

Gradient check. It uses central differences with h = 1e-4 on window 700 of a seed-5 synthetic corpus, at speed 2. The bound is relative error < 1e-4, and the gradient magnitude must be > 1e-7 so the check is not vacuous.

fd_stop moves the parameter only inside the scored position's forward, which matches the documented gradient (earlier K/V held). fd_true moves it for the whole window.

parameter k analytic fd_stop rel_stop rel_true
W_out [5,17] 27 7.815785016e-06 7.815785221e-06 2.6e-08 9.5e-07
b_out [17] 27 1.430989022e-03 1.430989025e-03 1.7e-09 5.1e-09
W_out [5,17] 7 7.909537245e-06 7.909537152e-06 1.2e-08 3.1e-07
W_down block 2 27 2.294110614e-07 2.294122094e-07 5.0e-06 2.4e-05
W_q block 2 27 1.205906677e-05 1.205906617e-05 4.9e-08 1.3e-06
W_v block 0 27 -4.407090518e-06 -4.407090659e-06 3.2e-08 1.94
W_k block 1 27 2.590987749e-06 2.590987836e-06 3.4e-08 1.93
W_up block 0 27 9.631424154e-06 9.631424547e-06 4.1e-08 0.74
embed [tok, 10] 27 5.978897969e-04 5.978897968e-04 2.7e-10 0.13
  • The head and the last block's Q / down have no K/V cache downstream, and they match the true gradient within 1e-4. The tests assert this.
  • Deep parameters match the stop-gradient differences within 1e-4. That is the gradient the step computes.
  • Finding, not fixed here: for deep parameters the true full-window gradient differs from the stop-gradient one by order 1, including sign flips for W_v block 0 and W_k block 1.
    • This is the pre-existing "earlier K/V as constants" design of nn/hslm section 5.
    • Under the 27-position objective, earlier positions' K/V feed 26 later losses. The dropped term is therefore no longer small, and it is a candidate next step.

Negative controls. Each mutates one line of tr_window_grad_k. For each, test-report ran and the file was restored (sha256 checked).

mutation tests failing caught by
target tokens[start+p+1] -> tokens[start+p] 5 of 20 k=1 bit-identity, loss-mean, uniform ln 729, last-block FD, deep FD (not the head FD: vocab entry 17 is not a target in that window)
drop the 1/k scale on dlogits 4 of 20 uniform bias gradient, head FD, last-block FD, deep FD
loop bound p + k >= 27 -> p + k > 27 5 of 20 k=1 bit-identity, loss-mean, uniform ln 729, head FD, deep FD

Seal. t27c seal --save wrote .trinity/seals/ml_train_hslm.json (spec sha256 fa8490d8...). seal --verify reports all hashes MATCH: spec, zig, verilog, c, rust.

Speed. This was measured over 50 real-token steps on the lab, seed 1729, speed 2, while run 1 shared the box.

objective s_per_step tok_per_s
last position (run 1 log, steps 5000-6000) 1.52-1.93 1170-920
27 positions 5.84 305

At 50 steps of the 27-position run, valid_loss was 6.5930 and valid_full_loss 6.6055. Step 0 gave the same 6.5952 as run 1.

Run 2 (started 2026-10-18 14:03:44 UTC):

  • Config: loss_positions=27 steps=8000 total=8000 seed=1729 eval_every=500 ckpt_every=1000; warmup, lr and batch are unchanged.
  • 8000 x 5.84 s is about 13.0 h, inside 14 h with room for the measured speed spread.
  • ckpt_every=1000 gives 8 checkpoints. 2000 would give only 4.
  • Run 1's run.log, ckpt_{2000,4000,6000} and eval_{2000,4000}.txt are kept in /tmp/hslm/run1_lastpos/.
  • Data is unchanged: train.tok sha256 9631d315...27d5, valid.tok d076d9e8...4d91.

Nothing needed a compiler change. Host I/O uses the existing host_open/read/write/close, with a NUL-terminated path buffer for the checkpoint names, and the clock is read from /proc/uptime.

🤖 Generated with Claude Code

gHashTag and others added 2 commits October 18, 2026 17:32
… and LAMB, writes MSLH checkpoints (Refs #10372)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the flat vector, xorshift guards zero (Refs #10372)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag added a commit that referenced this pull request Oct 18, 2026
…d specs (Refs #10387)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-18 10:50:47 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 50
PRs with All Checks Green 0
READY 0
FAILING 50
PENDING 0
NO CHECKS YET 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=fad12492937a != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-18 10:54:52 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 50
PRs with All Checks Green 0
READY 0
FAILING 50
PENDING 0
NO CHECKS YET 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=fad12492937a != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

…efs #10372)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-18 11:01:23 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 49
PRs with All Checks Green 1
READY 0
FAILING 49
PENDING 0
NO CHECKS YET 0

These columns do not partition: 0 + 49 + 0 + 0 = 49, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=fad12492937a != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Oct 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-19 02:00:14 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 45
PRs with All Checks Green 5
READY 2
FAILING 45
PENDING 0
NO CHECKS YET 0

These columns do not partition: 2 + 45 + 0 + 0 = 47, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=fad12492937a != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Oct 19, 2026
@gHashTag
gHashTag marked this pull request as ready for review October 19, 2026 03:33
@gHashTag
gHashTag merged commit 7e39315 into master Oct 19, 2026
34 of 36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-auto-merge Opt out of the issue gate's default squash auto-merge (issue-gate.yml)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant