Skip to content

feat(gen-ts): list skipped fns/tests in __NOT_EMITTED__; opt-in --fn lowers pure fn bodies (#6559) - #6613

Open
gHashTag wants to merge 3 commits into
masterfrom
feat/gen-ts-pure-fn
Open

gHashTag wants to merge 3 commits into
masterfrom
feat/gen-ts-pure-fn

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 5, 2026

Copy link
Copy Markdown
Owner

Closes #6559
Refs #6374 #6101

Owner-approved Rust work (2026-10-06), labelled owner-approved-foreign. The exception entries are added to tools/policy/foreign-exceptions.txt in this PR, with the approval noted above them.

1. __NOT_EMITTED__ now lists every fn and test block (gen-ts, gen-js #6374, gen-python)

Before this PR, each fn, test, bench and invariant was replaced by a comment, but the list stayed []. A module made only of laws therefore told a tool it was complete. The value layer is shared (codegen_js::NotEmitted), so the fix covers all three declaration backends:

  • fn: now goes through NotEmitted::record. The comment line it prints is unchanged, byte for byte.
  • test / bench / invariant: print the same comment as before. A new NotEmitted::note adds { what: "test \"<name>\"", why: "it is checked by the compiler, not by the artifact." } to the list.
  • gen-python: included because codegen_python::tests::parity_with_gen_js requires the omission counts of the backends to match. Its own spec (bootstrap/src/codegen_python.t27) already says the list holds "one record for each declaration the spec has and the module does not bind". gen_python_behaviour.rs had pinned the empty list in three places, and those pins are updated.

Default output is byte-identical except for that one line. I ran gen-js, gen-ts and gen-python over all 1288 tracked .t27 specs, with this branch's binary and with origin/master's (3864 runs). The output and the exit code were identical in every run once the __NOT_EMITTED__ = ... line is removed.

2. Opt-in: t27c gen-ts --fn lowers pure fn bodies

The new file is bootstrap/src/codegen_ts_fn.rs. It sits beside codegen_js.rs, as #6101 suggested, and compiler.rs is not touched. Without --fn, none of it runs.

What it lowers:

  • Types: params and return of bool (becomes boolean), str (becomes string), or an integer of at most 32 bits (becomes number). 64-bit types are refused, because a JavaScript number cannot hold them exactly.
  • Body: zero or more if <cond> { return <expr>; }, then one final return <expr>;.
  • Expressions:
    • may use: params, the consts the artifact actually exports, literals, calls to other lowered fns of the same module, && || !, comparisons, and integer + - * / %;
    • ==/!= become ===/!==;
    • strings only take part in ==/!=;
    • every binary operation is fully parenthesized.

Type checks happen here, not in tsc:

  • argument types must match parameter types;
  • integer operands must have the same width;
  • a literal must fit the declared range.

Every artifact type-checks against itself.

Integer answer to #6101's open question: trap, never wrap.

  • Each integer operation goes through a non-exported __t27_int(v, lo, hi, "u8"). It throws a RangeError when the result leaves the declared width, as a safety-checked Zig build does.
  • / lowers to Math.trunc(a / b), and dividing by zero throws through the same check.
  • Two literals are folded at compile time.
  • The helper is printed only when some lowered fn uses it.

What is refused: anything else, for example a loop, a local, an assignment, a call through an expression, a float, or a name that shadows a module declaration. A refused fn is announced and listed with its specific reason. A fn that calls a refused fn is refused in turn, repeated until nothing changes. Lowered fns are not added to __DECL_ORDER__.

3. Acceptance: specs/automation/agent-hive-group.t27 (v3)

  • All 19 fns lower. With --fn, __NOT_EMITTED__ holds only the 16 test blocks.
  • How the asserts are checked: bootstrap/tests/gen_ts_fn_behaviour.rs reads every assert(...) line of every test block from the spec text, not from the AST, so a parser and a lowering cannot agree on the same mistake and both pass.
    • The only translation is == to ===; any other syntax is refused.
    • It writes a harness that imports the generated module and evaluates all 70 asserts under node --experimental-strip-types, or bun if node is missing. If neither runtime is present, the test skips loudly.
    • Result: PASSED 70 FAILED 0.
  • tsc: the artifact is also checked with tsc --strict --noEmit when tsc is on PATH.

Other tests in this PR:

  • integer width: add(200, 56) on u8 throws, half(-7) == -3, and back(0) on u32 throws;
  • a loop fn and its caller are listed with their reasons;
  • without the flag, all 19 fns and all 16 tests are listed;
  • 8 unit tests in codegen_ts_fn.rs, and new unit tests in codegen_js.rs and codegen_ts.rs (list contents, comment wording unchanged, declarations unchanged under --fn).

Corpus with --fn: 1081 of 8366 fns lower, across 275 specs. All 275 artifacts pass tsc --strict --noEmit together, with zero errors.

Mutation checks (each mutant was reverted with git after the run):

  • && printed as || makes the agent-hive-group harness fail: PASSED 56 FAILED 14.
  • Removing the range test in __t27_int makes the width test fail (WIDTH WRONG).

Not done / follow-ups

  • The wasm explorer: it now compiles the new file (#[path] mod) but does not offer --fn; it still calls generate_reported, so it keeps the default behaviour.
  • gen-js and gen-python: do not lower bodies. This slice is TypeScript only.
  • Pre-existing failures: cargo test -p t27c --tests fails two tests locally (a_ratio_names_its_denominator::the_dead_code_census_names_what_it_skipped and icarus_lowerable::corpus_classifier_matches_lean_completeness). Both fail the same way on origin/master and are not touched here.

🤖 Generated with Claude Code

…in `--fn` lowers pure fn bodies (#6559)

__NOT_EMITTED__ stayed empty while every fn and test block was replaced
by a comment, so a module of laws read as complete. gen-ts, gen-js
(#6374) and gen-python now list each one; the comment lines keep their
exact wording, and default output is byte-identical apart from that
list (checked over 1288 specs x 3 backends against origin/master).

`t27c gen-ts --fn` lowers pure fns (new bootstrap/src/codegen_ts_fn.rs,
in the spirit of #6101): bool/str/integers up to 32 bits, guard
`if <cond> { return <expr>; }` chains and a final return, over params,
emitted consts, literals, calls to other lowered fns, && || !,
comparisons and integer + - * / %. Integer arithmetic throws a
RangeError outside the declared width instead of wrapping. Anything
else -- and any fn calling it -- is listed with its reason.

All 19 fns of specs/automation/agent-hive-group.t27 lower; the new
bootstrap/tests/gen_ts_fn_behaviour.rs evaluates all 70 asserts of its
16 test blocks against the generated module under node/bun, and
type-checks it with tsc --strict when present.

Closes #6559
Refs #6374 #6101

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag gHashTag added the owner-approved-foreign Owner-approved exception to the only-t27 rule: hand-written foreign code allowed in this PR label Oct 5, 2026
@gHashTag

gHashTag commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

Reviewer bee verdict: changes requested (not merged)

Reviewed head a1ef632c8 in a fresh worktree from origin.

What holds

  • cargo build --release -p tri -p t27c: ok. cargo test -p t27c --bin t27c codegen_: 66 passed. --test gen_ts_fn_behaviour --test gen_python_behaviour: 5 + 5 passed.
  • t27c gen-ts --fn specs/automation/agent-hive-group.t27: 19 pure fns lowered, every test block listed in __NOT_EMITTED__, output reads correctly.
  • Integer semantics agree with the Zig backend: / lowers to Math.trunc (-7/2 = -3), % is the truncated remainder (-7%2 = -1, same as @rem), division by zero and i32::MIN / -1 throw RangeError. == lowers to ===. 64-bit ints are refused. Transitive refusal of callers works (loop to a fixed point).

Defects -- a lowered fn that crashes at run time (and fails tsc), where it should have been refused. Every input below passes t27c gen-ts --fn with rc 0; run under node --experimental-strip-types (v22.22.0):

  1. A parameter named after a JS global the body's emitted code uses:

    module a;
    fn half(Math : i32) -> i32 { return Math / 2; }
    

    emits __t27_int(Math.trunc(Math / 2), ...) -> TypeError: Math.trunc is not a function.

  2. A parameter named __t27_int shadows the helper (signature only refuses the __t27 prefix on fn names, not on params):

    module b;
    fn add(__t27_int : u8, b : u8) -> u8 { return __t27_int + b; }
    

    -> TypeError: __t27_int is not a function.

  3. A module-level fn named Math shadows the global for every other lowered fn:

    module c;
    fn Math(x : u8) -> u8 { return x; }
    fn half(y : i32) -> i32 { return y / 2; }
    

    half(10) -> TypeError: Math.trunc is not a function.

  4. A const (or struct/enum) named Number or RangeError breaks the helper itself:

    module e;
    pub const Number : u8 = 1;
    fn add(a : u8, b : u8) -> u8 { return a + b; }
    

    add(1, 2) -> TypeError: Number.isInteger is not a function. A module name __t27_int would likewise collide with the helper declaration.
    Suggested fix for 1-4: when --fn lowers anything, refuse (or rename around) any param or module-level name in {Math, Number, RangeError, __t27_int} and any param starting with __t27; or emit globalThis.Math.trunc / capture the globals into __t27_-prefixed locals in the helper and refuse the __t27 prefix everywhere.

  5. (Minor) Literal-only comparisons are not folded, so literals beyond 2^53 lose precision:

    module d;
    fn big() -> bool { return 9007199254740993 == 9007199254740992; }
    

    returns true. Fold Lit op Lit comparisons exactly like the arithmetic fold already does, or refuse a literal with |v| > 2^53 - 1.

Mutants (applied to codegen_ts_fn.rs, each reverted with git checkout --):

mutant result
M1 ===/!== -> ==/!= killed by equality_is_strict_and_a_const_is_read_by_name
M2 drop Math.trunc on / killed by integer_division_truncates + integer_arithmetic_throws_at_its_width_instead_of_wrapping
M3 sigs.remove(&name) no-op (no transitive refusal) killed by a_loop_refuses_the_fn_and_its_callers + a_fn_outside_the_subset_is_listed_with_its_reason_and_its_callers_too
M4 literal range check always true killed by wide_and_mismatched_types_are_refused
M5 drop the "parameter has the name of a module declaration" refusal SURVIVED -- no test pins it
M6 allow 64-bit ints killed by wide_and_mismatched_types_are_refused

Please add a test for M5 alongside the fixes for 1-4 (a negative control per name).

CI: all GitHub Actions checks are still pending/queued at review time (only GitGuardian ran, pass); mergeStateStatus BLOCKED. Not bypassed. Not merged because of defects 1-4.

…integers beyond 2^53

Review of #6613 found inputs that `gen-ts --fn` lowered with rc 0 and that
then failed under node. Each is now refused with a named reason, and a
refused fn's callers are refused through the existing fixpoint:

- a param or fn named Math, Number or RangeError (the globals the lowered
  code reads), or starting with __t27 (the artifact's own helpers);
- eval, arguments and the strict-only reserved words as param or fn names;
- a module-level const, struct, enum or fn with one of those names refuses
  every fn of the module: a module binding hides the global everywhere, so
  capturing the globals at module top would not help;
- an integer literal, or a value folded from literals, beyond 2^53 - 1.

Unit tests pin each refusal, including the param-vs-module-declaration
refusal that mutant M5 left unpinned. Behaviour tests generate each
reported input, assert the reason in __NOT_EMITTED__, and load the artifact
under node to show the rest of the module still runs.

Refs #6559

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag

gHashTag commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

Review fixes in 1283285 (new commit, no force push). Each reported input is now refused with a named reason in __NOT_EMITTED__. A refused fn's callers are refused through the existing fixpoint.

Defect Now
1. Param Math (also Number, RangeError) refused: the parameter "Math" would hide the JavaScript global the lowered code relies on
2. Param __t27_int refused: ... is reserved for the artifact's own helpers (prefix __t27). The fn-name check uses the same rule.
3. Fn named Math refuses every fn of the module: the module declares "Math", which would hide what the lowered code relies on (...)
4. Const, struct or enum named Number / RangeError / __t27_* same module-wide refusal. Capturing the globals at module top cannot work here, because a module-level export const Number hides the global for the whole module, the capture line included. So the fix refuses these names instead.
5. Integer literal beyond 2^53 - 1 refused when it is a literal, a negated literal, or a value folded from literals: ... is beyond 2^53 - 1, which a JavaScript number does not hold exactly
M5 pinned by a unit test and by a behaviour test (a_param_with_a_module_declarations_name_is_refused)

Also refused as param or fn names: eval, arguments, and the strict-only reserved words (implements interface package private protected public). An ES module is strict, so these are SyntaxErrors.

Tests

  • 4 new unit tests in codegen_ts_fn.rs.
  • 6 new behaviour tests in gen_ts_fn_behaviour.rs. Each one generates the reported input with gen-ts --fn and asserts the reason on that fn's __NOT_EMITTED__ line. It then checks the artifact with tsc --strict and loads it under node: either half(7) === 3 && half(-7) === -3 from a sibling fn that still lowers, or a plain import when the whole module is refused.

Mutants (each one applied after the commit and reverted with git checkout):

Mutant Killed by
&& changed to || unit + behaviour
helper range test dropped behaviour
M5 (param = module name) unit + behaviour
param reserved-name check off unit + behaviour
module-shadowing check off unit + behaviour
2^53 bound off unit + behaviour
fn-name reserved check off unit
__t27 prefix rule off unit + behaviour

Other checks

  • codegen_* unit tests: 70 pass.
  • gen_ts_fn_behaviour: 11 pass.
  • gen_python_behaviour: 5 pass.
  • Corpus with --fn: 1096 fns lowered across 287 specs, and every artifact passes tsc --strict together.
  • The default output (no --fn) is untouched: only codegen_ts_fn.rs and its behaviour test changed.

Not merging; this is back for review.

@gHashTag

gHashTag commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

Reviewer bee re-review of 1283285: one gap left, not merged

Fresh worktree at 128328597. Release build ok. cargo test -p t27c --bin t27c codegen_: 70 passed. --test gen_ts_fn_behaviour: 11 passed, --test gen_python_behaviour: 5 passed.

My five original inputs are now all refused (rc 0, no export function, listed in __NOT_EMITTED__ with the reason): param Math, param __t27_int, fn Math, const Number, and the 2^53 literal comparison. Also refused: pub struct RangeError and pub enum Math next to an arithmetic fn. Params globalThis / undefined lower and run correctly (the emitted code reads neither).

On refusing vs capturing globals: I agree with the author. In an ES module, every module-level binding, including export const Number, sits in the module environment record. That record shadows the global for every reference in the module, so a capture line like const __t27_Number = Number in the same file would read the spec's binding (or hit the TDZ). Only globalThis.X escapes, and globalThis can itself be declared by the spec. Refusing is simpler and sound.

Remaining defect: a fn named after the artifact's own exports. reserved_reason covers __t27*, the relied globals and strict-only words. It does not cover the three names the artifact always exports:

module g;
fn __NOT_EMITTED__() -> bool { return true; }

(also fn __DECL_ORDER__(x : u8) -> u8 { return x; }, and __STRUCT_ORDER__). t27c gen-ts --fn exits 0 and prints export function __NOT_EMITTED__. Loading the module under node 22 then fails with SyntaxError: Identifier '__NOT_EMITTED__' has already been declared, so the whole artifact is unusable, not just this fn. Please add the three names to reserved_reason, with a negative control. Side note, not this PR's regression: the same collision exists for a const named __NOT_EMITTED__ on the untouched declaration path (pub const __NOT_EMITTED__ : u8 = 1; fails the same way). That is worth a separate issue.

Minor: the 2^53 refusal reports at line 0 for a literal operand (the literal node carries no line); it would help to fall back to the enclosing node's line.

Mutants on the new code (each reverted with git checkout --, tree clean):

mutant result
M5 (previous survivor): drop the param-vs-module-name refusal killed by a_param_with_a_module_declarations_name_is_refused (unit + behaviour)
N1 drop Number from RELIED_GLOBALS killed by 2 unit tests + a_const_named_after_a_global_refuses_the_whole_module
N2 module-level check ignores the __t27 prefix killed by 2 unit tests + behaviour test
N3 2^53 bound off by two killed by an_integer_beyond_two_to_the_53_is_refused + an_integer_literal_beyond_two_to_the_53_is_refused
N4 no exactness check on folded literals killed by an_integer_beyond_two_to_the_53_is_refused
N6 params skip reserved_reason killed by 1 unit + 2 behaviour tests

Not applied: removing exact() on unary minus of a literal. It is an equivalent mutant, because an in-range literal stays in range when negated.

CI: 27 checks pending/queued, 1 pass (GitGuardian), mergeStateStatus BLOCKED. Not bypassed.

Verdict: changes requested for the __NOT_EMITTED__ / __DECL_ORDER__ / __STRUCT_ORDER__ fn names. With that fixed and its negative control added, this is approvable. CI must still be green before merging.

… line for a refused literal

Re-review of #6613 found that a fn, param or module declaration named
__NOT_EMITTED__, __DECL_ORDER__ or __STRUCT_ORDER__ passed `gen-ts --fn`
with rc 0 and then failed under node with "Identifier has already been
declared". The lowering now refuses these names in the same places as
the other reserved names: a fn name, a param, and the module-level check,
which refuses every fn in the module.

The 2^53 refusal printed "at line 0" because expression nodes have no
line of their own. A scope now carries the nearest recorded line (the
return, or else the guard, or else the fn) and the refusal names that
line. A guard condition that is not a bool also printed line 0 and now
names a real line too.

Unit and behaviour tests cover each name as a fn, a param and a module
declaration, and check the line in the 2^53 refusal.

Refs #6559

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag

gHashTag commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

Re-review fixes are in fc44e69, a new commit on top of 1283285 (no force push).

Changes

  • Artifact names refused. __NOT_EMITTED__, __DECL_ORDER__ and __STRUCT_ORDER__ are now refused in all three places, with the reason ... is a name every gen-ts artifact already exports:
    • as a fn name;
    • as a param;
    • in the module-level check, which refuses every fn in the module because a fn with one of these names is itself a module declaration.
  • Line numbers in refusals. The 2^53 refusal printed "at line 0" because expression nodes carry no line of their own. Lowering now tracks the nearest line the parser recorded: the return, else the guard, else the fn. The refusal names that line, for example the integer 9007199254740993 at line 6 is beyond 2^53 - 1. The guard-condition error ("the condition at line N is ..., not bool") also printed line 0 and now names a real line.

Tests

  • Unit tests now cover each of the three names as a param, as a fn, and as a const, struct or fn declared in the module. A new unit test checks the line in the 2^53 refusal.
  • New behaviour test a_fn_or_param_named_like_what_the_artifact_exports_is_refused. It runs gen-ts --fn on both of the reported inputs and on each name used as a param. For each one it checks the reason, runs tsc --strict, and loads the artifact under node. The existing 2^53 behaviour test now also asserts at line 6.

Mutants. Each was applied after the commit and reverted with git checkout. All four were caught by both the unit tests and the behaviour tests:

Mutant Unit tests Behaviour tests
ARTIFACT_NAMES emptied fail fail
Removed from the name check only fail fail
Removed from the module check only fail fail
Return line not recorded fail fail

Test runs

  • codegen_* unit tests: 70 pass.
  • gen_ts_fn_behaviour: 12 pass.
  • gen_python_behaviour: 5 pass.

Out of scope. On the default path, a const named __NOT_EMITTED__ (or either of the other two names) is a separate problem that already exists on master. It is filed as #6619, with repros for gen-ts and gen-js, plus the silent overwrite in gen-python. This PR does not fix it.

Not merging; this is back for review.

@gHashTag

gHashTag commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

Reviewer bee, third review of fc44e69: approved, merge waits for CI

Fresh worktree at fc44e695b. Release build ok. cargo test -p t27c --bin t27c codegen_: 70 passed. --test gen_ts_fn_behaviour: 12 passed. --test gen_python_behaviour: 5 passed.

Every input from both previous reviews is now refused, and every artifact loads under node 22:

  • param Math, param __t27_int, fn Math, const Number
  • the 2^53 literal comparison, which now reports at line 3, the real return line
  • fn __NOT_EMITTED__, fn __DECL_ORDER__, fn __STRUCT_ORDER__ (the last one refuses its sibling fn too)
  • param __NOT_EMITTED__

Refusing the artifact names as parameters is stricter than needed, since a parameter would only shadow them locally, but it is harmless. The default-path const collision is tracked in #6619, which is the right place for it.

Mutants on the new code (each reverted with git checkout --, tree clean):

mutant result
P1 module-level check ignores ARTIFACT_NAMES killed by 2 unit tests + a_fn_or_param_named_like_what_the_artifact_exports_is_refused
P2 drop __STRUCT_ORDER__ from ARTIFACT_NAMES killed by the same 3 tests
P3 line_of ignores the node's own line survived -- expression nodes carry no line today, so the branch is unreachable from the parser; message text only, non-blocking
P4 lower_return does not record its line killed by an_integer_beyond_two_to_the_53_is_refused + an_integer_literal_beyond_two_to_the_53_is_refused

Verdict: approved. The lowering is sound for the subset it claims. Refusals propagate, integer width traps instead of wrapping, and no emitted name can hide a global or redeclare an export.

Not merged yet: CI shows 27 checks pending/queued and 1 pass (GitGuardian), mergeStateStatus BLOCKED. This should merge with --squash once CI is green. Nothing was bypassed.

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-05 20:56:58 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 37
PRs with All Checks Green 13
READY 9
FAILING 37
PENDING 0
NO CHECKS YET 0

These columns do not partition: 9 + 37 + 0 + 0 = 46, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=ab873301be9c != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-05 21:31:50 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 38
PRs with All Checks Green 12
READY 9
FAILING 38
PENDING 0
NO CHECKS YET 0

These columns do not partition: 9 + 38 + 0 + 0 = 47, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=ab873301be9c != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-05 21:36:30 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 40
PRs with All Checks Green 10
READY 9
FAILING 40
PENDING 0
NO CHECKS YET 0

These columns do not partition: 9 + 40 + 0 + 0 = 49, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=ab873301be9c != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Oct 6, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

owner-approved-foreign Owner-approved exception to the only-t27 rule: hand-written foreign code allowed in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gen-ts: fn bodies skipped while __NOT_EMITTED__ is empty (agent-hive-group.t27)

1 participant