Skip to content

lua, c: the VM bridge and the C program host speak kore-rt's bridge - #34

Open
SnowCheetos wants to merge 10 commits into
ft/fervent-shannon-egp5ptfrom
ft/festive-euler-sghq2d
Open

SnowCheetos wants to merge 10 commits into
ft/fervent-shannon-egp5ptfrom
ft/festive-euler-sghq2d

Conversation

@SnowCheetos

@SnowCheetos SnowCheetos commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

kore-lua's VM bridge and kore-c's program host now speak kore-rt's bridge, the one seam for a synchronous engine on a worker thread served by its async host. Kore's tcc and its Lua REPL speak the same seam upstream. Each engine keeps only its payload and its dispatch. The word's states, the park-and-kick ordering, the cell's hand-over and the worker's end are kore-rt's.

This is stacked on ft/fervent-shannon-egp5pt (kore-c, libkore.so), which is #32; this branch merges it at 98186bf. This lands after it, or this pull request's base moves to alpha to land both. The Kore half is MicroPerceptron/kore#1536, stacked on MicroPerceptron/kore#1494, which pins this branch at e4c3905.

Why

The two copies here had drifted from each other and from TCC's:

  • kore-lua:
    • A VM that could not be built stored WORKER_GONE over a host-held word, which the first evaluation could overwrite. Thread 0 then parked forever.
    • A park refused mid-call returned into Lua while thread 0 still read the call's input, a borrow of the Lua frame.
    • A VM-thread panic was never reported to thread 0.
    • eval handed the VM thread a pointer into the caller's script. A dropped eval future ended that borrow while the VM went on compiling from it: a use-after-free reachable from safe code.
  • kore-c: it kept its own session states, park loops, race and fatal path beside them.

What changed

  • native/ is re-exported byte for byte with tools/sync-native.py at Kore 6a8c754bc959e9489a6fb18be642fcc881a65c48. That commit carries the bridge and merges MicroPerceptron/kore#1494's branch at f2644e5b, so the export also has everything the kore-c branch exported since: the parking controller, threads that found taking a founder's arena, Kore's C library, and the link window's free span. Against the base's export, the closure keeps 27 packages and gains kore-rt's bridge (402 files), with:

    • ThreadBlock's hosted_bridge tail slot;
    • a bridge-worker notice in kore-rt's panic paths;
    • the worker's exit hook, so a worker that ends its own thread without its last word has it said for it;
    • a side source the host serves beside the worker (Side, serve_beside), armed and re-read with the word;
    • a fix to a flaky kore-rt test (6a8c754b). The futex sweep test judged every row of the wait table while the suite's other tests held rows of their own. It failed 8 runs in 300 of futex::tests, and once in this branch's make test. MicroPerceptron/kore#1534 (a4ab7234), which ffi: a cancellation wakes its target, and no cancellable wait polls #33 exports, carries the same fix.

    No exported manifest changed, so native/Cargo.lock did not.

  • kore-lua:

    • The VM is the Interpreter engine. Its first item carries E::LIBRARIES and E::POLICY, so the engine is not generic over the Extension. That keeps Extension free of a 'static bound, as its contract requires.
    • Host keeps its API (start::<E>, eval::<E>, quit).
    • A worker gone for any reason but a failed start is now Fatal. It used to answer the last evaluation's stale outcome.
    • eval is sound under cancellation (57c57d7). The evaluation runs on its own copy of the script, in a buffer reused across evaluations: one bounded memcpy, or Outcome::ResourceRefused with nothing posted. The next eval finishes an abandoned one first. The bridge cannot close this from below, since a host future may always be dropped.
  • kore-c:

    • The main thread is the bridge's worker, and its one item is the program. It is founded through WorkerHost::spawn, holding the channel-space grant b9eb3c2 gave it.

    • What the program's other threads hand thread 0 is the bridge's side source. Thread 0 serves it between the main thread's sessions and beside one in flight:

      • the output ring 0460f5c brought, drained a pass at a time. The stream is finished before the main thread is retired, since the main thread may still be waiting for room;
      • a thread's mmap or munmap, which c876e43 brought. Thread 0 carries it out, since it holds the supply's root. A pass answers it before it drains the ring, so a pass dropped beside a session cannot cut the answer short. Only the ring is armed, since a mapping's caller always kicks.

      The base's own park, race and end-wait are gone, since the bridge owns those.

    • Thread 0 serves the session with serve_beside, stopping on the exit word, so an end requested mid-session still does not wait for the session.

    • mmap refuses a program with no host through program::hosted, which is the bridge's is_hosted: true from the founding's claim until the host joins the main thread.

    • The exit word, the roles and the join-or-abandon decision stay this crate's, since they are the program's lifecycle. A main thread that ends itself also leaves the bridge, so the host can join it for a clean barrier.

  • make miri also runs kore-rt's bridge suite, which drives the state machine on real threads, so Miri checks every hand-over.

  • Docs: docs/languages.md, crates/lua/src/lib.rs's Extension contract, and docs/provenance.md.

History

The branch merges its moved base instead of rebasing onto it, since its history was already pushed. Each merge re-exports native/ at the Kore commit that carries both halves:

  • 9d1d43d superseded a0e452e's export at 61ae1ace.
  • 26115b2 took the base at 0460f5c, after it had moved through exports at Kore 2a0730dd, 6c57f344 and 2d4998ba.
  • e4c3905 takes the base at 98186bf: an export at Kore 10c7b400, c876e43's descriptors and mmap, and rustix 1.1.6 (RUSTSEC-2026-0486).

Each export's Kore commits are on MicroPerceptron/kore#1494's branch, so each export carries them beside the bridge.

Verification

Locally, with the pinned nightly:

  • make fmt test check docs owners cross miri bindings-check: PASS at e4c3905, on the export at Kore 6a8c754b. That covers:

    • the native kore-rt suite (477, the bridge's 13 among them), kore-dagfs and kore-vol;
    • clippy and rustdoc with warnings denied;
    • cross: the three Kore user-target checks, which compile kore-lua's and kore-c's bridges and the metal half of kore-rt's for all three ISAs;
    • check-program: libkore.so links for all three ISAs with exactly its declared 129 exports, the five c876e43 brought (open, close, mmap, munmap, kore_fd_describe) among them;
    • Miri over the bridge suite (14 tests with erased_poll_and_drop);
    • cbindgen and Zig for the bindings.
  • The futex fix: 8 of 300 runs of futex::tests failed before it, and 0 of 1000 after.

  • This repo's CI on e4c3905:

    • host and miri pass.
    • The three targets jobs passed every build and check up to apt-get install llvm, which failed both times they ran: their runners could not resolve Ubuntu's archives. So CI has not yet run the allocator-symbol check or the ABI check after it, which pass locally (make cross).
  • On Kore's side, MicroPerceptron/kore#1536 pins this branch at e4c3905 (6187e3ea), sealed in c27b20ba. Its sealed production x86 image passed these under QEMU 8.2 TCG, booting the seal's own bytes:

    • make keytest-tcc-x86:
      • its libkore tour runs to exit 42 on kore-c's host over the bridge;
      • hello.c prints every line through the output ring the side source drains;
      • memory.c opens /kore/memory, maps a zeroed megabyte, maps a second one elsewhere, reuses an unmapped region, and has its meaningless requests refused. Each of its mmap and munmap calls reaches thread 0 through the side source.
    • make keytest-lua-x86: Kore's own REPL, the same seam.
    • make exec-boot-gate-x86: all 19 services founded.

    lua-tool-worker, which runs kore-lua, builds from this revision and drops its own copy of the script; no keytest drives it. Kore's make test passes (223 cargo suites, 8436 tests), and make check-x86, check-aarch64 and check-riscv report 0 errors.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi

## Why

kore-lua's VM bridge and kore-c's program host each carried their own copy
of the worker/host exchange: its word, its states, its park-and-kick
ordering and its worker-gone handling. Kore's `tcc` and its Lua REPL carried
two more. The copies had drifted where the discipline is subtle. kore-rt now
owns that exchange once, as `bridge`, and this export brings it here
through the exporter, not as an edit to the snapshot.

The previous export named Kore `eab77f8d`, which never reached GitHub. It
was MicroPerceptron/kore#1494's `8bcaa7d3` plus one kore-rt commit, which
Kore's `ecff66ad` now carries byte for byte. This export's crash, startup,
thread and thread-controller sources are identical to the previous
export's, which proves that.

## What

- `native/` is re-exported byte for byte at
  `61ae1aced5113de12aefdcf658bc122b9a46dfbf` with `tools/sync-native.py`.
  That commit heads Kore's `ft/festive-euler-sghq2d`, stacked on #1494's
  branch. The closure keeps its 26 packages and gains one file (383):
  - `kore-rt`'s `bridge`: `Bridge<P>`, the `Engine` trait, `Work` and
    `WorkerHost`, with its host suite;
  - `ThreadBlock`'s `hosted_bridge` tail slot, which names a worker's bridge;
  - a bridge-worker notice in `entry::panic_refuse` and `crash::report`.
- No exported manifest changed, so `native/Cargo.lock` did not.
- `make miri` also runs kore-rt's `bridge` suite, which drives the
  state machine on real threads, so Miri checks every hand-over.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi
The VM's bridge kept its own copy of the word, its six states, the host
call's ordering and thread 0's park loop. Two of its paths could hang
thread 0 or race it:
- A VM that could not be built stored WORKER_GONE with a plain store over
  the host-held word, which the first evaluation's EVAL_POSTED could
  overwrite. Thread 0 then parked for a kick that never came.
- A park refused mid-call returned an error into Lua while thread 0 still
  held the cell and read the call's input, a borrow of the Lua frame the
  worker had just left.
A VM-thread panic was never reported to thread 0 at all.

It now runs over `kore::runtime::bridge`, which owns the exchange and its
worker-gone handling, and keeps what crosses it and the dispatch that
answers it.

- The worker is the `Interpreter` engine. Its first item, handed over with
  its founding, carries `E::LIBRARIES` and `E::POLICY`, so the engine is not
  generic over the `Extension`. That keeps `Extension` free of a `'static`
  bound, as its contract requires. The first item builds the VM and
  `fail`s with `Outcome::Startup` when it cannot. After that, every item is
  an evaluation, until thread 0 quits it.
- `kore_lua_host_require`, `kore_lua_host_call` and `kore_lua_host_release`
  are `Bridge::call` and `Bridge::with_work`. A call from outside an
  evaluation, or from inside another bridge borrow, is refused with a text
  before anything is posted.
- `Host` keeps its API (`start::<E>`, `eval::<E>`, `quit`) over a
  `WorkerHost<Interpreter>`. A worker gone for any reason but a failed
  start is now `Fatal`. It used to answer the last evaluation's stale
  outcome.
- The dispatch (`serve_call`, `serve_require`, `serve_value`, `serve_fs`)
  is unchanged except that it receives the cell. A require's name is
  served from the cell rather than copied out first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi
`kore-c`'s host kept its own session cell (IDLE/POSTED/SERVED), the main
thread's park, thread 0's park loop, the race of a session against an end,
and a `fatal` path for a main thread that cannot park. It now runs over
`kore_std::runtime::bridge`, and keeps what no other engine shares: the
program's lifecycle.

- The main thread is the bridge's worker (`Main`). Its one work item is
  the program, which ends the process rather than finishing.
  `kore_fs_run`'s half is `Bridge::call`, refused, as before, on any thread
  but the main thread of a hosted program. `HOSTED` is the bridge's
  `is_hosted`.
- Thread 0 serves the session with `WorkerHost::serve_until`, stopping on
  the exit word, so an end requested mid-session still does not wait for
  the session. The exit word stays this crate's. Any thread posts it and
  kicks the host's wake channel. The main thread's exit also leaves the
  bridge (`Bridge::leave`), so the host can wait for it to end and join it
  for a clean barrier.
- A main thread that panics or loses its park is reported by the bridge
  itself, after the panic record is published. The host ends with PANIC
  or INTERNAL, joining it, where `fatal()` used to post the end by hand.
- `docs/languages.md` describes the seam.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi
`docs/provenance.md` records the export at Kore `61ae1ace`, where kore-rt
owns the worker/host bridge, and why `eab77f8d`, which the previous entry
named, is now `ecff66ad`. It records the libkore gates that passed on it and
the Kore keytests that booted the same seam under QEMU. It also says what
nothing has run yet: `kore-c`'s host, which no Kore image ships.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi
…after

`Host::eval` is a safe `async fn`, and it handed the VM thread a pointer
into the caller's `script`. A caller that dropped the future
mid-evaluation (a `select`, a deadline) ended that borrow while the VM
thread went on compiling from it: a use-after-free reachable from safe
code. The bridge cannot close it from below. It keeps a *worker's* call
borrows alive by keeping the worker parked, but a *host* future can always
be dropped.

- The evaluation runs on its own copy. `eval` copies the script into the
  cell's reusable buffer (`try_reserve` and one bounded memcpy) before it
  posts. The worker takes the buffer for the evaluation and hands it back
  with the results, so the next evaluation reuses its capacity. A script
  the heap cannot hold a copy of is `Outcome::ResourceRefused`, and
  nothing is posted.
- A dropped `eval` leaves an evaluation running. The next `eval` first
  finishes it, serving its remaining calls and discarding its output,
  before it posts its own. `eval`'s documentation now says so.
- The copy is the price of soundness under cancellation. The alternative,
  an `unsafe` `eval` whose future must be driven to completion, would put
  the same obligation on every embedder and make no one safer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi
The kore-c branch this one stacks on moved by two commits:
- `3a373a2` re-exports `native/` at Kore `0018956a`, where the thread
  controller parks on the futex;
- `b9eb3c2` makes `kore-ffi`'s join park on its target's retirement
  word, and founds `kore-c`'s main thread holding a channel-space grant.

- `native/` is re-exported with `tools/sync-native.py` at Kore
  `280917e44f74cf956e676cc493bbe35173216758`. That is Kore's
  `ft/festive-euler-sghq2d`, which merges the same base, so the export
  carries both the parking controller and the bridge. It has 26 packages
  and 383 files, and `sync-native.py --check` matches it.
- `kore-c` founds its main thread through `WorkerHost::spawn`, handing it
  the grant as the base does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi
The bridge's section now records the export at Kore `280917e4`. That
commit merges #1494's branch, so it carries both the parking controller
and the bridge, along with what changed since the previous export:
- the bridge;
- the runtime block's slot;
- the panic notice;
- the worker's exit hook;
- five services' boot digests from #1494's reseal, which libkore never
  reads.

The section drops the account of `eab77f8d`, which has since reached
#1494's branch.

Verified on this export: `make fmt test check docs owners cross miri
bindings-check` passes.
- The native `kore-rt` suite: 473 tests, 13 of them the bridge's.
- `check-program`: `libkore.so` has exactly its 85 declared exports for
  each ISA.
- Miri over the bridge suite: 14 tests, with `erased_poll_and_drop`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi
The kore-c branch moved by four commits:
- exports at Kore 2a0730dd, where threads that found take a founder's
  arena;
- exports at 6c57f344, where a founding's staged space waits for the
  reap;
- an export at 2d4998ba, with Kore's C library;
- 0460f5c, where C programs write stdout and stderr from every thread
  through the host.

- `native/` is re-exported with `tools/sync-native.py` at Kore
  `6b1d4d871cbc1cc2ee4106e8385777affc6868cc`. That is Kore's
  `ft/festive-euler-sghq2d`, which merges Kore's TCC branch (`ft/fervent-shannon-egp5pt`)
  at `0de59be2`,
  so the export carries the base's three Kore commits and the bridge
  together. It has 27 packages and 402 files, and `sync-native.py
  --check` matches it.
- `kore-c`'s host keeps the bridge and gains the base's streams:
  - it installs the sink before the main thread is founded, and
    abandons it if the founding refuses;
  - it serves the output ring as the bridge's side source
    (`serve_beside`), so passes run between the main thread's sessions
    and beside one in flight;
  - it finishes the stream before it retires the main thread, which may
    still be waiting for room;
  - it relays what the launcher has room for when thread 0 ends itself.
  The base's own park, race and end-wait are gone, since the bridge
  owns those. `abort`, the `END` it commits through, and the role
  helpers `stdio` reads come from the base unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi
The bridge's section now records the export at Kore `6b1d4d87`. That
commit merges the TCC branch at `0de59be2`, so it carries that branch's
three later Kore commits as well as the bridge:
- `2a0730dd`;
- `6c57f344`;
- `2d4998ba`, which brings Kore's C library.

The section also notes:
- the bridge's side source (`Side`, `serve_beside`);
- that `kore-c` serves its output ring through it.

Verified on this export: `make fmt test check docs owners cross miri
bindings-check` passes.
- The native `kore-rt` suite: 477 tests, 13 of them the bridge's.
- `check-program`: `libkore.so` has exactly its 124 declared exports for
  each ISA.
- Miri over the bridge suite: 14 tests, with `erased_poll_and_drop`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi
The kore-c branch moved by three commits:
- an export at Kore 10c7b400, with the link window's free span and
  three more C errors;
- c876e43, where C programs open the capabilities they hold as
  descriptors, and map memory from them;
- 98186bf, rustix 1.1.6 in the lock (RUSTSEC-2026-0486).

- `native/` is re-exported with `tools/sync-native.py` at Kore
  `6a8c754bc959e9489a6fb18be642fcc881a65c48`. That is Kore's
  `ft/festive-euler-sghq2d`, which merges Kore's TCC branch at
  `f2644e5b` (`eca08c04`), so the export carries 10c7b400 and the
  bridge together. It has 27 packages and 402 files, and
  `sync-native.py --check` matches it.
- That commit also fixes a flaky kore-rt test: the futex sweep test
  judged every row of the wait table, while the suite's other tests
  held rows of their own meanwhile. It failed 8 runs in 300 of
  `futex::tests`, and once in this merge's `make test`.
- `kore-c`'s host serves a thread's `mmap` and `munmap` as part of the
  bridge's side source, beside the output ring:
  - the side has work while the mapping cell holds a request, and a
    pass answers it before it drains the ring once, so a pass dropped
    beside a session cannot cut the answer short;
  - only the ring is armed, since a mapping's caller always kicks;
  - an answered mapping kicks the main thread, which may be its caller.
  The base's own park and race over the cell are gone, since the
  bridge owns those.
- `program::hosted`, which `mmap` reads to refuse a program with no
  host, is the bridge's `is_hosted`: true from the founding's claim
  until the host joins the main thread.
- `docs/languages.md` says the cell is served beside a session in
  flight. `docs/provenance.md` records the export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7fL57uG16rZrxzpedi

SnowCheetos commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor Author

targets (x86_64), targets (aarch64) and targets (riscv64gc) failed on e4c3905 because their runners lost DNS while installing llvm. It is not this branch's code.

  • Every build and check in the three jobs passed up to that step. riscv64gc, for instance, reached entry: riscv64gc: 7 linked programs; 3 invalid return types rejected.
  • The next step, sudo apt-get install -y llvm for tools/check-alloc-symbols.py, failed in each job with Temporary failure resolving 'security.ubuntu.com' (and archive.ubuntu.com, azure.archive.ubuntu.com), exit 100. So the allocator-symbol check, and the libkore.so ABI check after it, never ran.
  • The base's run on 98186bf (13:11–13:22 UTC) installed llvm in the same step without trouble, so the outage began after it.
  • Locally, make cross passed on this head for all three ISAs. It covers the allocator-symbol check, and check-program linked libkore.so for each ISA with exactly its declared 129 exports.

Update: I re-ran the failed jobs once (attempt 2, 14:15–14:22 UTC). All three failed again at the same step with the same resolver errors, while host and miri stayed green. That was this PR's one re-run, so I am not re-running again. The jobs will run again with this branch's next push. A re-run once the runners resolve Ubuntu's archives again would also settle it.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants