From 352478500cd8a40ec0ef22d7fae74af8b452842a Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 31 Aug 2026 01:21:40 +0000 Subject: [PATCH] feat: the agent's machine carries its own Rust, built rather than borrowed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On 2026-08-30 a paid benchmark run watched Claude, inside the machine, choose exactly the right primitive — ask what a symbol is, then rename it — and be answered: context: source: index, resolution: matched, analyzer_starts: 0 rename: ok: false, error: unresolved, message: there is no `cargo` on this machine Everything after that in the transcript is a consequence. The preflight had said READY, and it was right about what it asked: the machine was alive and holding the right tree. Neither is the capability a Rust task needs, and there was nothing between the money and finding that out. The first fix attempted was to copy the host's ~/.rustup onto the store. Cesar stopped it and he was right: Filosofia-Fundacional.md says Thalyx is the whole system, so a programming face that only works because Fedora has rustup installed is a face that belongs to Fedora. Move the disk to another x86_64 box and the semantic provider would vanish. So the runtime is Thalyx's own artifact. dev/build-rust-runtime.sh builds it from digest-checked upstream tarballs — Rust's official musl host tools, whose sha256 come from Rust's own channel manifest — plus the two files Rust does not publish: musl's loader, compiled here from musl's release tarball, and libgcc_s.so.1, linked out of the libunwind.a that ships inside rust-std itself. Nothing is copied from the machine that builds it. 644 MB, on the store, never in the initramfs — `make count` still says the kernel and one program, and there is now a test that arms the archive and counts it. Why musl and not the ordinary toolchain was a measurement taken before anything was written: the GNU one needs glibc's loader, libc, libm, libdl, librt, libpthread, libgcc_s and libz from the host plus a separate 191 MB libLLVM; the musl one needs exactly two files, and LLVM is inside librustc_driver. Two missing files is a problem a person can close. That the unwinder-only libgcc_s is enough is also a measurement: of the 883 undefined symbols across cargo, rustc, rust-analyzer, the proc-macro server and librustc_driver, everything resolves against musl and librustc_driver except 29, and the only ones of those that are not weak are the fifteen _Unwind_*, all of which libunwind.a defines. Then it was run rather than argued. What is checked, and how, because `ldd` is the wrong instrument: it starts the real loader against the real /lib, so it says what *this* Fedora would resolve. An artifact missing a library that happens to live in the builder's /usr/lib looks perfect on the builder and is a directory of dead ELF files inside Thalyx, whose /lib is empty — and the failure surfaces as ENOENT from execve, which reads as "there is no cargo on this machine" about a cargo that is right there. So `thalyx dev rust-runtime` reads PT_INTERP and every DT_NEEDED out of the headers and asks whether they are inside the artifact. Also here: - discovery puts Thalyx's own runtime second, behind only a variable that names a file. When Thalyx carries a compiler, that is the compiler; - PID 1 makes /lib/ld-musl-x86_64.so.1 point at the artifact's loader after mounting the store, which is what the kernel reads out of PT_INTERP; - a read-only `toolchain` verb that runs cargo --version and rust-analyzer --version *inside* the machine and reads the workspace's manifests, and `thalyx-mcp --preflight --needs-rust`, which refuses to be READY without it. The benchmark asks for it whenever the project has a Cargo.toml — derived from the tree, because a flag somebody has to remember is the flag that was missing on the run that needed it; - `make -C image agent PROJECT=…` stages it automatically for a Cargo workspace, prints where it came from and how much, and refuses — deleting what it copied — if the artifact is not closed. Proven physically in this container, not just in tests: inside a chroot holding the artifact, /proc and three device nodes — no shell, no /usr, no /lib64 — cargo, rustc and rust-analyzer start, cargo metadata describes a synthetic workspace, and a full rust-analyzer session resolved a definition and returned four real rename edits. What is *not* proven here is a booted Thalyx doing it; that needs the hardware, and dev/verify-agent-rust.sh is the one command for it. Two rules learned and written into Estrategia-de-Pruebas.md: a check that asks the host answers about the host, and musl resolves $ORIGIN for the main program by reading /proc/self/exe — the same binary starts with /proc mounted and dies without it, complaining about a library that is right where it says. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01CvoEad9oWgX1Y594uirzbn --- crates/thalyx-cli/src/catalogue.rs | 29 +- crates/thalyx-cli/src/dev.rs | 109 ++++ crates/thalyx-cli/src/exec.rs | 2 +- crates/thalyx-cli/src/external.rs | 8 + crates/thalyx-cli/src/image.rs | 42 ++ crates/thalyx-cli/src/init.rs | 4 + crates/thalyx-cli/src/main.rs | 1 + crates/thalyx-cli/src/semantic.rs | 2 +- crates/thalyx-cli/src/session.rs | 8 +- crates/thalyx-cli/src/store_disk.rs | 56 ++ crates/thalyx-cli/src/toolchain.rs | 240 ++++++++ .../tests/an_external_agent_is_confined.rs | 5 + .../the_machine_says_what_it_can_resolve.rs | 165 ++++++ crates/thalyx-mcp/src/main.rs | 78 ++- crates/thalyx-rust/src/elf.rs | 166 ++++++ crates/thalyx-rust/src/lib.rs | 2 + crates/thalyx-rust/src/runtime.rs | 541 ++++++++++++++++++ crates/thalyx-rust/src/toolchain.rs | 298 +++++++++- ..._runtime_thalyx_carries_runs_on_its_own.rs | 271 +++++++++ dev/bench-external-agent.sh | 24 +- dev/bench-summary.py | 88 ++- dev/build-rust-runtime.sh | 315 ++++++++++ dev/rust-corpus/Cargo.toml | 14 + dev/rust-corpus/harbour/Cargo.toml | 7 + dev/rust-corpus/harbour/src/lib.rs | 19 + dev/rust-corpus/lantern/Cargo.toml | 4 + dev/rust-corpus/lantern/src/lib.rs | 30 + dev/verify-agent-rust.sh | 287 ++++++++++ dev/verify.sh | 56 ++ image/Makefile | 67 ++- vault/06-Pendientes/Punto-Actual.md | 50 +- vault/06-Pendientes/Tareas-Pendientes.md | 19 + .../Estado-de-Implementacion.md | 14 + .../Estrategia-de-Pruebas.md | 46 ++ .../09-Notas-Tecnicas/Runtime-Rust-Agente.md | 228 ++++++++ 35 files changed, 3253 insertions(+), 42 deletions(-) create mode 100644 crates/thalyx-cli/src/toolchain.rs create mode 100644 crates/thalyx-cli/tests/the_machine_says_what_it_can_resolve.rs create mode 100644 crates/thalyx-rust/src/elf.rs create mode 100644 crates/thalyx-rust/src/runtime.rs create mode 100644 crates/thalyx-rust/tests/the_runtime_thalyx_carries_runs_on_its_own.rs create mode 100755 dev/build-rust-runtime.sh create mode 100644 dev/rust-corpus/Cargo.toml create mode 100644 dev/rust-corpus/harbour/Cargo.toml create mode 100644 dev/rust-corpus/harbour/src/lib.rs create mode 100644 dev/rust-corpus/lantern/Cargo.toml create mode 100644 dev/rust-corpus/lantern/src/lib.rs create mode 100755 dev/verify-agent-rust.sh create mode 100644 vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md diff --git a/crates/thalyx-cli/src/catalogue.rs b/crates/thalyx-cli/src/catalogue.rs index 13905c5..d8d4311 100644 --- a/crates/thalyx-cli/src/catalogue.rs +++ b/crates/thalyx-cli/src/catalogue.rs @@ -485,6 +485,21 @@ pub const VERBS: &[Verb] = &[ summary: "What a name is — kind, crate, signature, where, how many uses — small \ enough to read, with a handle that fetches the exact lines on demand.", }, + // Beside `context` because it answers the question `context` cannot: not + // *what is this name* but *can this machine resolve a name at all*. A run + // that asks the first without ever asking the second is the 2026-08-30 + // benchmark, which paid for a machine that had no compiler on it. + Verb { + id: "toolchain", + names: &["herramientas", "toolchain"], + takes: &[], + flags: &[], + answers: Some("toolchain"), + changes: false, + errors: &[], + summary: "Whether this machine can resolve Rust names: which cargo and \ + rust-analyzer it has, whether they ran, and whose they are.", + }, Verb { id: "rename", names: &["renombrar-simbolo", "renombrar-símbolo", "rename"], @@ -946,7 +961,19 @@ mod tests { // it had. They are the frontier agent's verbs — typed, or sent over the // external surface, where the caller has already read the exact name // out of an answer this machine gave it. - const NOT_A_SENTENCE: [&str; 4] = ["exec", "evidence", "context", "rename"]; + // `toolchain` joins them for a different reason from the other four: + // not that a sentence cannot spell its argument — it takes none — but + // that it is a question about the machine's own equipment rather than + // about the human's request. Its callers are a preflight deciding + // whether to spend money and a person typing `herramientas`; teaching + // the local grammar a verb neither of those goes through would spend a + // tiny model's tokens on a word nobody says to it. + // + // Written down as a pendiente rather than as a closed question: + // «¿puedes renombrar símbolos aquí?» is a thing Cesar might well say + // out loud, and the day it is worth answering, this list is where the + // decision is recorded. + const NOT_A_SENTENCE: [&str; 5] = ["exec", "evidence", "context", "rename", "toolchain"]; let unaskable: Vec<&&str> = ops .difference(&proposable) diff --git a/crates/thalyx-cli/src/dev.rs b/crates/thalyx-cli/src/dev.rs index fc7bdd4..47ef7a5 100644 --- a/crates/thalyx-cli/src/dev.rs +++ b/crates/thalyx-cli/src/dev.rs @@ -36,6 +36,19 @@ pub enum DevCommand { /// Show what a bundle contains, without installing it Inspect { bundle: PathBuf }, + /// Look at a staged Rust runtime and say whether a machine could use it + /// + /// `vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md`. Two questions, and + /// they are not the same one: whether the artifact holds what it must and + /// nothing it must not, and whether every library its own programs name is + /// *inside it*. The second is the one `ldd` cannot answer, because `ldd` + /// asks the host — and on the host that built it, an artifact that would + /// be broken inside the machine looks perfect. + RustRuntime { + /// The artifact directory + artifact: PathBuf, + }, + /// Build the machine's root filesystem, or count what is in one /// /// The image is the Linux kernel and one program. This is what makes the @@ -125,6 +138,7 @@ pub fn run(command: DevCommand) -> Fallible { out, } => pack(&source, &manifest, &key, &out), DevCommand::Inspect { bundle } => inspect(&bundle), + DevCommand::RustRuntime { artifact } => rust_runtime(&artifact), DevCommand::Image { binary, out, list } => match (binary, out, list) { (_, _, Some(archive)) => crate::image::list(&archive), (Some(binary), Some(out), None) => crate::image::build(&binary, &out), @@ -528,3 +542,98 @@ fn append_bytes( builder.append_data(&mut header, name, contents)?; Ok(()) } + +/// Say whether a staged Rust runtime is one. +/// +/// Printed as facts and not as a verdict word alone: a staging step that says +/// only `FAILED` sends whoever ran it to read this source, and the thing they +/// need is which file is missing. +fn rust_runtime(artifact: &Path) -> Fallible { + use thalyx_rust::runtime; + + println!(" {}", artifact.display()); + let report = runtime::inspect(artifact); + let closure = runtime::closure(artifact); + + if let Some(runtime) = runtime::read(artifact) { + println!(" is {}", runtime.describe()); + } + println!( + " size {} in {} program(s)", + directory_size(artifact), + closure.programs.len() + ); + for interpreter in &closure.interpreters { + println!(" loader {interpreter}"); + } + for asked in &closure.programs { + let name = asked + .program + .file_name() + .map(|name| name.to_string_lossy().into_owned()) + .unwrap_or_default(); + let named: Vec = asked + .libraries + .iter() + .map(|(library, here)| format!("{library}{}", if *here { "" } else { " ← MISSING" })) + .collect(); + println!(" {name:<32}{}", named.join(", ")); + } + + let mut wrong = Vec::new(); + for missing in &report.missing { + wrong.push(format!("{missing} is not in the artifact")); + } + for forbidden in &report.forbidden { + wrong.push(format!( + "{forbidden} is in the artifact — a whole toolchain was copied, not a runtime assembled" + )); + } + for target in &report.other_targets { + wrong.push(format!( + "lib/rustlib/{target} is a standard library for a machine this is not" + )); + } + for (program, library) in &closure.unresolved { + wrong.push(format!( + "{program} asks for {library} and the artifact does not carry it, so it would \ + resolve against whatever the machine running it happens to have" + )); + } + if !closure.interpreter_inside { + wrong.push( + "the artifact does not carry the loader its own programs ask the kernel for" + .to_string(), + ); + } + + if wrong.is_empty() { + println!(); + println!(" closed: every library these programs name is inside the artifact."); + return Ok(()); + } + println!(); + for line in &wrong { + println!(" no {line}"); + } + Err(format!("{} is not a usable Rust runtime", artifact.display()).into()) +} + +/// How big a directory is, for a line a human reads. +fn directory_size(root: &Path) -> String { + fn walk(path: &Path) -> u64 { + let Ok(entries) = std::fs::read_dir(path) else { + return 0; + }; + entries + .flatten() + .map(|entry| match entry.file_type() { + Ok(kind) if kind.is_dir() => walk(&entry.path()), + Ok(kind) if kind.is_file() => entry.metadata().map(|m| m.len()).unwrap_or(0), + _ => 0, + }) + .sum() + } + let bytes = walk(root); + format!("{} MB", bytes / 1_000_000) +} diff --git a/crates/thalyx-cli/src/exec.rs b/crates/thalyx-cli/src/exec.rs index 5473fdd..65cd3e4 100644 --- a/crates/thalyx-cli/src/exec.rs +++ b/crates/thalyx-cli/src/exec.rs @@ -1661,7 +1661,7 @@ fn rust_check( // change not compiling. let environment: Vec<(String, String)> = thalyx_rust::toolchain::environment() .into_iter() - .map(|(name, path)| (name.to_string(), path.display().to_string())) + .map(|(name, value)| (name.to_string(), value)) .collect(); let outcome = run_confined( asked, diff --git a/crates/thalyx-cli/src/external.rs b/crates/thalyx-cli/src/external.rs index 4fa38c4..6c2d9c4 100644 --- a/crates/thalyx-cli/src/external.rs +++ b/crates/thalyx-cli/src/external.rs @@ -201,6 +201,14 @@ pub const EXPOSED: &[Exposed] = &[ repeating: MORE_OPTIONS, verbatim_from: QUOTED, }, + // Read-only, takes nothing, and it is what a harness asks before it spends + // money on a machine that may not have a compiler on it. + Exposed { + verb: "toolchain", + slots: &[], + repeating: NOTHING_MORE, + verbatim_from: QUOTED, + }, Exposed { verb: "rename", // What to rename and what to call it. Both text: the first may be a diff --git a/crates/thalyx-cli/src/image.rs b/crates/thalyx-cli/src/image.rs index 15c1bb8..5b4d3cb 100644 --- a/crates/thalyx-cli/src/image.rs +++ b/crates/thalyx-cli/src/image.rs @@ -366,6 +366,48 @@ mod tests { use super::*; use std::io::Write; + #[test] + fn the_rust_runtime_is_not_in_the_image() { + // `Construccion-del-ISO.md`: the image is the Linux kernel and one + // program, and the decree is written to be *countable* rather than + // argued. On 2026-08-31 six hundred megabytes of Rust toolchain + // arrived for the agent to program with, and the one way that could + // have gone wrong quietly was for it to be put in here — where it + // would double the boot's memory before the machine said a word, and + // where `make count` would stop saying what it says. + // + // It goes on the store, like the engine and the model, because that is + // the difference between what Thalyx *is* and what has been installed + // on it. This asserts the archive's own list, which is the same list + // the builder writes. + for directory in DIRECTORIES { + assert!( + !directory.contains("toolchain") && !directory.contains("rust"), + "the image's directory list has grown a {directory}" + ); + } + let held = tempfile::tempdir().expect("a temp dir"); + let binary = fake_binary(held.path()); + let archive = held.path().join("initramfs.cpio"); + build(&binary, &archive).expect("building the archive"); + let bytes = std::fs::read(&archive).expect("reading the archive"); + let entries = parse(&bytes).expect("parsing the archive"); + assert_eq!( + entries.iter().filter(|entry| entry.is_program()).count(), + 1, + "the image is the kernel and one program" + ); + for entry in &entries { + assert!( + !entry.name.contains("cargo") + && !entry.name.contains("rust") + && !entry.name.contains("toolchain"), + "{} is in the image and belongs on the store", + entry.name + ); + } + } + fn fake_binary(dir: &Path) -> std::path::PathBuf { let path = dir.join("thalyx"); let mut file = std::fs::File::create(&path).unwrap(); diff --git a/crates/thalyx-cli/src/init.rs b/crates/thalyx-cli/src/init.rs index 0dbadd3..92ecbce 100644 --- a/crates/thalyx-cli/src/init.rs +++ b/crates/thalyx-cli/src/init.rs @@ -392,6 +392,10 @@ pub fn run() -> Fallible { // unmounted /opt/thalyx would report an empty machine rather than an // unmounted one. crate::store_disk::mount().report(); + // After the store, because the runtime it points at is on the store, and + // before the session, because the session is what answers `context` and + // `rename`. + crate::store_disk::link_runtime_loader(); match attach_lsm() { Ok(detail) => println!(" ok thalyx-lsm {detail}"), diff --git a/crates/thalyx-cli/src/main.rs b/crates/thalyx-cli/src/main.rs index 6949541..76ab35b 100644 --- a/crates/thalyx-cli/src/main.rs +++ b/crates/thalyx-cli/src/main.rs @@ -42,6 +42,7 @@ mod session; mod snapshot; mod store_disk; mod term; +mod toolchain; mod words; use clap::{Parser, Subcommand}; diff --git a/crates/thalyx-cli/src/semantic.rs b/crates/thalyx-cli/src/semantic.rs index 8173cd0..85b6652 100644 --- a/crates/thalyx-cli/src/semantic.rs +++ b/crates/thalyx-cli/src/semantic.rs @@ -300,7 +300,7 @@ fn provider_for(store_root: &Path, tree: &Path) -> Provider { thalyx_rust::toolchain::readable(), thalyx_rust::toolchain::environment() .into_iter() - .map(|(name, path)| (name.to_string(), path.display().to_string())) + .map(|(name, value)| (name.to_string(), value)) .collect(), ); match Store::open(store_root) { diff --git a/crates/thalyx-cli/src/session.rs b/crates/thalyx-cli/src/session.rs index 57f561e..dca74a9 100644 --- a/crates/thalyx-cli/src/session.rs +++ b/crates/thalyx-cli/src/session.rs @@ -1480,7 +1480,7 @@ pub fn run(store: &Store, once: bool) -> Fallible { println!(" `buscar `, `encontrar `, `contenido `,"); println!(" `historia`, `intento`, `cambios`,"); println!(" `contexto [presupuesto=N|usos=N|expandir=asa]`,"); - println!(" `renombrar-simbolo `,"); + println!(" `renombrar-simbolo `, `herramientas`,"); println!(" `hacer `, `evidencia `,"); println!(" `procesos [patrón]`, `memoria`, `matar [forzar]`,"); println!(" `disponibles`, `instalar `, `modulos`, `correr `,"); @@ -2153,6 +2153,12 @@ fn dispatch_asking( "contexto" | "context" => { crate::semantic::context(store.root(), here, "", face)?; } + // What the machine can resolve *with*, which is a different question + // from what a name is, and the one nobody asked before paying for a + // run on a machine with no compiler. + "herramientas" | "toolchain" => { + crate::toolchain::act(here, face)?; + } // `renombrar-simbolo` and not `renombrar`: `renombrar` has meant // *move this file* since the file verbs existed, and stealing the word // would silently change what an existing caller's line does. Two things diff --git a/crates/thalyx-cli/src/store_disk.rs b/crates/thalyx-cli/src/store_disk.rs index 0775676..74344e8 100644 --- a/crates/thalyx-cli/src/store_disk.rs +++ b/crates/thalyx-cli/src/store_disk.rs @@ -300,6 +300,62 @@ pub fn mount() -> Store { } } +/// Make the name the Rust runtime's own binaries were compiled with resolve. +/// +/// ## Why PID 1 does this and why it is a symlink +/// +/// Every program in `toolchains/rust//` carries +/// `PT_INTERP: /lib/ld-musl-x86_64.so.1` in its ELF header. That is not a +/// choice anybody here made — it is what the upstream artifact was linked +/// with, and the *kernel* reads it, before the process exists, as an absolute +/// path. A machine without that name answers `execve` with `ENOENT`, and what +/// the human sees is "there is no cargo on this machine" about a cargo that is +/// right there. That sentence, from inside a benchmark run on 2026-08-30, is +/// what this whole change exists to stop. +/// +/// A symlink and not a bind mount, for a reason that would only show up later: +/// a confined program's root filesystem binds `/lib`, and while that bind is +/// recursive today, a symlink is a directory entry in the initramfs root and +/// travels with **any** bind, recursive or not. The cheaper thing is also the +/// one a change somewhere else cannot quietly break. +/// +/// Nothing happens on a machine with no runtime staged, which is every machine +/// that is not preparing an agent. Reported either way: a store that carries a +/// compiler and a `/lib` that does not point at it is exactly the failure +/// above, and it must not be something a person has to go looking for. +pub fn link_runtime_loader() { + let staged = thalyx_rust::runtime::staged(Path::new(thalyx_rust::runtime::STORE_ROOT)); + let Some(runtime) = staged.into_iter().next() else { + return; + }; + let name = Path::new(thalyx_rust::runtime::LOADER); + if let Some(parent) = name.parent() { + let _ = std::fs::create_dir_all(parent); + } + // Removed first: a previous boot may have left one pointing at a runtime + // that is no longer the one on the disk, and a stale loader is one from + // another Rust release — which fails at a relocation deep inside the first + // analysis rather than at the link, where it would be readable. + let _ = std::fs::remove_file(name); + match std::os::unix::fs::symlink(runtime.loader(), name) { + Ok(()) => println!( + " ok rust {} — {} points at it", + runtime.describe(), + name.display() + ), + Err(error) => { + println!( + " no rust {} is staged and unusable", + runtime.identity + ); + println!(" {} could not be made: {error}", name.display()); + println!(" Every program in it asks the kernel for that name, so cargo"); + println!(" and rust-analyzer will not start, and the machine will say"); + println!(" there is no cargo on it."); + } + } +} + impl Store { /// Print what happened, in the shape the rest of the boot uses. /// diff --git a/crates/thalyx-cli/src/toolchain.rs b/crates/thalyx-cli/src/toolchain.rs new file mode 100644 index 0000000..3b4d9df --- /dev/null +++ b/crates/thalyx-cli/src/toolchain.rs @@ -0,0 +1,240 @@ +//! Can this machine actually resolve a Rust name, asked before anybody pays to +//! find out. +//! +//! ## The failure this file exists to stop +//! +//! On 2026-08-30 the benchmark's preflight said `READY` and the run was paid +//! for. Inside the machine, the first thing the agent did was the right thing +//! — ask what a symbol is, then rename it — and the machine answered +//! `source: index`, `analyzer_starts: 0`, and +//! +//! ```text +//! rename: { ok: false, error: unresolved, +//! message: "there is no `cargo` on this machine" } +//! ``` +//! +//! The preflight had checked that the machine was **alive** and holding the +//! **right tree**. Both were true. Neither is the capability the task needed, +//! and there was nothing between the money and finding that out. +//! +//! So this verb answers the question the preflight could not ask: *is there a +//! compiler here, and did it run*. It is free, it is read-only, and it is the +//! same code path `context` and `rename` use to find their tools — a probe +//! written against a copy of that search would prove the copy. +//! +//! ## What it does not do +//! +//! It does not rename anything, write anything, or start rust-analyzer. The +//!2026-08-29 lesson was a probe that changed the starting state of the run it +//! was clearing, and `cargo metadata --no-deps --offline` — the heaviest thing +//! here — reads manifests and resolves nothing, so it writes no `Cargo.lock`. +//! +//! ## Why running `--version` is the whole point +//! +//! `~/.cargo/bin/rust-analyzer` exists on every rustup install and answers +//! `error: Unknown binary`. A staged runtime whose loader is missing is a +//! directory full of perfectly good ELF files that cannot start. In both cases +//! the file is there and the program is not, and only one of those two facts is +//! the one a benchmark needs. [`thalyx_rust::toolchain`] has had that rule +//! since it was written: a candidate becomes the answer after it answers +//! `--version`, so a path coming back from it *is* the evidence that something +//! ran. + +use crate::files::{Face, Where}; +use serde_json::{Value, json}; +use std::path::Path; +use thalyx_rust::toolchain::{Found, Kind}; + +type Fallible = Result<(), Box>; + +pub const OP: &str = "toolchain"; + +/// What one tool answered when it was asked what it is. +/// +/// Run a second time rather than remembered from the search, because the +/// search only kept *whether* the exit status was zero. The string is what +/// makes an answer worth reading: "cargo 1.90.0" and "cargo is somewhere" are +/// different amounts of knowing. +fn version_of(path: &Path) -> Option { + let output = std::process::Command::new(path) + .arg("--version") + .stdin(std::process::Stdio::null()) + .output() + .ok()?; + if !output.status.success() { + return None; + } + let said = String::from_utf8_lossy(&output.stdout).trim().to_string(); + (!said.is_empty()).then_some(said) +} + +/// One tool, as the answer carries it. +fn tool(found: &Found) -> Value { + match &found.path { + Some(path) => json!({ + "path": path.display().to_string(), + // `thalyx` when it is the runtime on the store, `host` when it is + // one somebody installed, `named` when a variable said so. The + // distinction is the decree of 2026-08-31 and it is the field + // worth reading first. + "from": found.kind.map(Kind::as_str).unwrap_or("unknown"), + "version": version_of(path), + }), + None => json!({ + "path": Value::Null, + "from": Value::Null, + // Where it looked, so a refusal is something a person can act on + // rather than a sentence they have to believe. + "looked_at": found + .looked_at + .iter() + .map(|path| json!(path.display().to_string())) + .collect::>(), + }), + } +} + +/// The answer, and whether the machine can do semantics at all. +pub fn report(here: &Where) -> (Value, bool, Vec) { + let cargo = thalyx_rust::toolchain::cargo(); + let analyzer = thalyx_rust::toolchain::rust_analyzer(); + let tree = crate::semantic::tree_of(here); + + let mut because: Vec = Vec::new(); + if cargo.path.is_none() { + because.push(cargo.why_not( + "cargo that runs", + "A machine meant to program Rust is prepared with \ + `make -C image agent PROJECT=… RUST=1`.", + )); + } + if analyzer.path.is_none() { + because.push(analyzer.why_not( + "rust-analyzer that runs", + "Without it every answer comes from the scan, which matches names \ + rather than resolving them.", + )); + } + + let runtime = thalyx_rust::toolchain::managed_runtime(); + let is_rust = tree.join("Cargo.toml").is_file(); + + // Read-only and cheap: `--no-deps` resolves nothing, so no lockfile is + // written and the starting state of whatever runs next is untouched. It is + // the difference between "a cargo exists" and "a cargo can read *this* + // workspace", which is the question a run about this workspace has. + let workspace = if is_rust && cargo.path.is_some() { + match thalyx_rust::Workspace::read(&tree) { + Ok(workspace) => json!({ + "rust": true, + "root": workspace.root.display().to_string(), + "packages": workspace.packages.len(), + }), + Err(why) => { + because.push(format!("cargo could not read this workspace: {why}")); + json!({"rust": true, "root": tree.display().to_string(), "read": false}) + } + } + } else { + json!({"rust": is_rust, "root": tree.display().to_string()}) + }; + + let ready = because.is_empty() && is_rust; + if !is_rust { + // Not a fault. A machine holding a tree that is not a Cargo workspace + // has nothing to be ready *for*, and saying so is different from + // saying the toolchain is missing. Rule 10. + because.push(format!( + "{} is not a Cargo workspace, so there is nothing here for a Rust \ + semantic provider to answer about", + tree.display() + )); + } + + let answer = json!({ + "cargo": tool(cargo), + "rust_analyzer": tool(analyzer), + "runtime": match &runtime { + Some(runtime) => json!({ + "identity": runtime.identity, + "rust": runtime.rust, + "musl": runtime.musl, + "root": runtime.root.display().to_string(), + }), + None => Value::Null, + }, + "workspace": workspace, + "semantic_ready": ready, + "because": because.clone(), + }); + (answer, ready, because) +} + +/// The verb. +pub fn act(here: &Where, face: Face) -> Fallible { + let (answer, ready, because) = report(here); + + if face.is_machine() { + let carried: Vec<(&'static str, Value)> = answer + .as_object() + .map(|fields| { + fields + .iter() + .map(|(name, value)| (leak(name), value.clone())) + .collect() + }) + .unwrap_or_default(); + face.say(thalyx_files::machine::answer(OP, carried)); + return Ok(()); + } + + println!(); + for (name, value) in [ + ("cargo", &answer["cargo"]), + ("rust-analyzer", &answer["rust_analyzer"]), + ] { + match value["path"].as_str() { + Some(path) => println!( + " {name:<15}{path}\n {:<15}{} ({})", + "", + value["version"].as_str().unwrap_or("no version"), + value["from"].as_str().unwrap_or("unknown"), + ), + None => println!(" {name:<15}none of the places this machine looks holds one"), + } + } + if let Some(identity) = answer["runtime"]["identity"].as_str() { + println!(" {:<15}{identity}", "runtime"); + } + println!( + " {:<15}{}", + "semantic", + if ready { + "ready — names can be resolved here" + } else { + "NOT ready" + } + ); + for line in &because { + println!(" {line}"); + } + println!(); + Ok(()) +} + +/// The field names of a fixed object, as the `'static` strings the machine +/// face takes. +/// +/// The set is closed and written three screens above; leaking it is bounded by +/// the number of keys in that literal, not by anything a caller controls. +fn leak(name: &str) -> &'static str { + match name { + "cargo" => "cargo", + "rust_analyzer" => "rust_analyzer", + "runtime" => "runtime", + "workspace" => "workspace", + "semantic_ready" => "semantic_ready", + "because" => "because", + _ => "extra", + } +} diff --git a/crates/thalyx-cli/tests/an_external_agent_is_confined.rs b/crates/thalyx-cli/tests/an_external_agent_is_confined.rs index 27ea0da..014cf60 100644 --- a/crates/thalyx-cli/tests/an_external_agent_is_confined.rs +++ b/crates/thalyx-cli/tests/an_external_agent_is_confined.rs @@ -418,6 +418,11 @@ fn every_verb_the_session_offers_answers_with_exactly_one_object() { // A name and a new name, which on a tree with no Cargo manifest // reaches `unresolved`. Still one object, which is the claim. "rename" => vec!["greet", "salute"], + // Nothing at all: it is a question about the machine's own + // equipment. Whether this container turns out to have a compiler + // is not this test's question — one line in, exactly one object + // out, through the real bridge, is. + "toolchain" => vec![], other => panic!("`{other}` is offered and this test does not know how to call it"), }; let answer = wire.ask(verb, &arguments); diff --git a/crates/thalyx-cli/tests/the_machine_says_what_it_can_resolve.rs b/crates/thalyx-cli/tests/the_machine_says_what_it_can_resolve.rs new file mode 100644 index 0000000..83738c6 --- /dev/null +++ b/crates/thalyx-cli/tests/the_machine_says_what_it_can_resolve.rs @@ -0,0 +1,165 @@ +//! Before anybody pays for a run, the machine is asked whether it has a +//! compiler — and the answer has to be a thing a program can read. +//! +//! ## The failure this file exists to stop +//! +//! `vault/07-Adopcion-y-Fases/Evidencia-de-Agentes.md`, 2026-08-30. The +//! benchmark's preflight said `READY`, the run was paid for, and the first +//! thing the agent did inside the machine came back +//! +//! ```text +//! rename: { ok: false, error: unresolved, +//! message: "there is no `cargo` on this machine" } +//! ``` +//! +//! Everything the preflight had checked was true: the machine answered, and it +//! was holding the right tree. Neither is the capability a Rust task needs, and +//! there was nothing between the money and finding that out. +//! +//! `thalyx-mcp --preflight --needs-rust` now asks this verb and refuses to be +//! READY when the answer is no. That refusal is only worth anything if the +//! fields it reads exist, in the machine face, with those names — which is what +//! this test is. The decision itself is tested separately and for free in +//! `dev/bench-summary.py --self-test`, against a machine with a compiler, one +//! without, and one too old to be asked. + +use std::io::Write; +use std::path::Path; +use std::process::{Command, Output, Stdio}; + +fn thalyx() -> &'static str { + env!("CARGO_BIN_EXE_thalyx") +} + +/// Type at the prompt down a plain pipe, which is how a program drives Thalyx. +fn piped(root: &Path, lines: &[&str]) -> Output { + let mut child = Command::new(thalyx()) + .arg("session") + .env("THALYX_ROOT", root) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .expect("the session"); + let mut typed = String::new(); + for line in lines { + typed.push_str(line); + typed.push('\n'); + } + child + .stdin + .take() + .expect("stdin") + .write_all(typed.as_bytes()) + .expect("feeding the session"); + child.wait_with_output().expect("waiting for the session") +} + +fn answer(output: &Output, op: &str) -> serde_json::Value { + String::from_utf8_lossy(&output.stdout) + .replace('\r', "") + .lines() + .filter_map(|line| serde_json::from_str::(line.trim()).ok()) + .find(|value| value["op"] == serde_json::json!(op)) + .unwrap_or_else(|| { + panic!( + "nothing answered `{op}`:\n{}", + String::from_utf8_lossy(&output.stdout) + ) + }) +} + +#[test] +fn the_machine_answers_whether_it_can_resolve_a_rust_name() { + let root = tempfile::tempdir().expect("a store"); + let said = piped(root.path(), &["structured on", "toolchain", "salir"]); + let answer = answer(&said, "toolchain"); + + // Every field the preflight reads, present whatever the machine turned out + // to have. An absent key and a false one are the same shape and different + // facts, and a harness that had to tell them apart by `in` would be reading + // the wrong thing on the day it mattered. Rule 10. + for field in [ + "cargo", + "rust_analyzer", + "runtime", + "workspace", + "semantic_ready", + "because", + ] { + assert!( + answer.get(field).is_some(), + "the answer has no `{field}`, which the preflight reads:\n{answer:#}" + ); + } + assert!( + answer["semantic_ready"].is_boolean(), + "`semantic_ready` has to be a yes or a no, not {:?}", + answer["semantic_ready"] + ); + assert!( + answer["because"].is_array(), + "`because` has to be a list, so a refusal can be printed line by line" + ); + + // Whichever way this machine went, the answer says *why* and *whose*. A + // machine that said `false` with an empty `because` would be the 2026-08-30 + // failure again with a different word on it. + if answer["semantic_ready"] == serde_json::json!(false) { + assert!( + !answer["because"].as_array().expect("a list").is_empty(), + "not ready, and it did not say why:\n{answer:#}" + ); + } else { + for tool in ["cargo", "rust_analyzer"] { + assert!( + answer[tool]["path"].is_string(), + "ready, and {tool} has no path:\n{answer:#}" + ); + assert!( + answer[tool]["from"].is_string(), + "ready, and nothing says whose {tool} it is — `thalyx`, `host` or \ + `named` is the field that decides whether this machine was autonomous" + ); + } + } +} + +#[test] +fn a_tool_that_is_not_there_is_reported_with_the_places_it_was_looked_for() { + // "There is no rust-analyzer" is a sentence nobody can act on. The whole + // reason `Found` carries `looked_at` is that "there is none at these four + // paths" tells a person which home the search was in — which, under `sudo`, + // is the entire problem. + let root = tempfile::tempdir().expect("a store"); + let said = piped(root.path(), &["structured on", "toolchain", "salir"]); + let answer = answer(&said, "toolchain"); + + for tool in ["cargo", "rust_analyzer"] { + if answer[tool]["path"].is_null() { + let looked = answer[tool]["looked_at"] + .as_array() + .unwrap_or_else(|| panic!("{tool} is absent and did not say where it looked")); + assert!( + !looked.is_empty(), + "{tool} is absent and the list of places is empty" + ); + let why = answer["because"] + .as_array() + .expect("a list") + .iter() + .filter_map(serde_json::Value::as_str) + .collect::>() + .join(" "); + assert!( + why.contains("place(s) this machine looks"), + "the reason does not name where it looked: {why}" + ); + return; + } + } + println!( + "NOT PROVEN: this machine has both tools, so the shape of the refusal was not \ + exercised here. `crates/thalyx-rust/src/toolchain.rs` tests it directly." + ); +} diff --git a/crates/thalyx-mcp/src/main.rs b/crates/thalyx-mcp/src/main.rs index 283df3e..f1f6c2d 100644 --- a/crates/thalyx-mcp/src/main.rs +++ b/crates/thalyx-mcp/src/main.rs @@ -78,6 +78,21 @@ struct Cli { #[arg(long)] preflight: bool, + /// Refuse to be READY unless the machine can resolve Rust names + /// + /// The 2026-08-30 run: the preflight said READY, the run was paid for, and + /// the first thing the agent did inside the machine came back + /// `there is no cargo on this machine`. Aliveness and the right tree were + /// both true and neither was the capability the task needed. + /// + /// A flag rather than always-on, because most tasks are not Rust and a + /// preflight that demanded a compiler of every machine would be a + /// preflight that fails on the ones it was written for. Rule 3's shape: + /// one switch per requirement, so a run that needs a compiler can demand + /// one and a run that does not is unaffected. + #[arg(long)] + needs_rust: bool, + /// Which set of tools to offer the model /// /// `compact` — the default — offers three: what a name is, do a stretch of @@ -140,7 +155,7 @@ fn main() { ); if cli.preflight { - let (report, ready) = preflight(&mut machine, &greeting, offered.len()); + let (report, ready) = preflight(&mut machine, &greeting, offered.len(), cli.needs_rust); println!("{report}"); std::process::exit(i32::from(!ready)); } @@ -233,7 +248,12 @@ fn usable(verbs: &[String], whole_catalogue: bool) -> Vec<&'static tools::Tool> /// would have changed the starting state of the very run it was clearing — /// and the `reversible` task's whole verdict is a comparison against that /// starting state. -fn preflight(machine: &mut Machine, greeting: &machine::Greeting, offered: usize) -> (Value, bool) { +fn preflight( + machine: &mut Machine, + greeting: &machine::Greeting, + offered: usize, + needs_rust: bool, +) -> (Value, bool) { let mut report = json!({ "protocol": thalyx_bridge::PROTOCOL, "thalyx": greeting.thalyx, @@ -283,6 +303,60 @@ fn preflight(machine: &mut Machine, greeting: &machine::Greeting, offered: usize Err(why) => trouble.push(format!("`list .` did not answer: {why}")), } + // ── and, when the task is a Rust one, whether it can resolve a name ── + // + // Asked of the machine, over the same channel, with the same adapter, and + // it costs nothing: `toolchain` runs `cargo --version` and + // `rust-analyzer --version` inside the machine and reads the workspace's + // manifests. It writes nothing — the 2026-08-29 lesson was a probe that + // changed the starting state of the run it was clearing. + let has_verb = greeting.verbs.iter().any(|verb| verb == "toolchain"); + if needs_rust { + if !has_verb { + // Rule 10: a machine too old to answer is not a machine without a + // compiler, and the remedy is different. Still not READY — a run + // that needs the capability cannot be told "probably". + trouble.push( + "this machine has no `toolchain` verb, so whether it can resolve Rust \ + names cannot be established. It is running a Thalyx from before the \ + managed Rust runtime existed" + .into(), + ); + } else { + match machine.ask("toolchain", vec![]) { + Ok(answer) => { + let ready = answer + .get("semantic_ready") + .and_then(Value::as_bool) + .unwrap_or(false); + if !ready { + let said = answer + .get("because") + .and_then(Value::as_array) + .map(|lines| { + lines + .iter() + .filter_map(Value::as_str) + .collect::>() + .join("; ") + }) + .unwrap_or_else(|| "the machine did not say why".to_string()); + trouble.push(format!( + "this machine cannot resolve Rust names, and the task needs it: {said}" + )); + } + report["toolchain"] = answer; + } + Err(why) => trouble.push(format!("`toolchain` did not answer: {why}")), + } + } + } else if has_verb && let Ok(answer) = machine.ask("toolchain", vec![]) { + // Reported even when it is not required, because a run that turns out + // to have been about Rust after all should not have to guess + // afterwards what the machine had on it. + report["toolchain"] = answer; + } + let ready = trouble.is_empty(); report["ready"] = json!(ready); // Always present, never only on failure: a caller that reads `because` to diff --git a/crates/thalyx-rust/src/elf.rs b/crates/thalyx-rust/src/elf.rs new file mode 100644 index 0000000..60bcf80 --- /dev/null +++ b/crates/thalyx-rust/src/elf.rs @@ -0,0 +1,166 @@ +//! Just enough ELF to ask a program what it will need before it is asked to +//! run. +//! +//! ## Why this is here and not `ldd` +//! +//! `ldd` answers the question *on the machine running it*: it starts the real +//! loader against the real `/lib`, so it says what a Fedora would resolve, not +//! what a Thalyx will. The question this crate has is the opposite one — does +//! the **artifact** carry everything its own programs ask for, on a machine +//! that has nothing else at all. Reading the headers answers that without a +//! machine to answer it on, which is also what makes it testable in a +//! container. +//! +//! The reader is small on purpose: `PT_INTERP`, and `DT_NEEDED` out of +//! `PT_DYNAMIC`. Nothing else about ELF is any of this crate's business. +//! `thalyx-bpf` has its own reader for the *relocatable* objects clang emits; +//! this one reads executables and shared objects, and they overlap in nothing +//! but the magic number. + +use std::path::Path; + +/// What a program says it will need. +#[derive(Debug, Clone, PartialEq, Eq, Default)] +pub struct Needs { + /// The dynamic loader, from `PT_INTERP`. `None` for a static binary and + /// for a shared object, which are different things and are both legal. + pub interpreter: Option, + /// Every `DT_NEEDED`, in the order the header lists them. + pub libraries: Vec, +} + +fn u16_at(bytes: &[u8], at: usize) -> Option { + Some(u16::from_le_bytes(bytes.get(at..at + 2)?.try_into().ok()?)) +} + +fn u32_at(bytes: &[u8], at: usize) -> Option { + Some(u32::from_le_bytes(bytes.get(at..at + 4)?.try_into().ok()?)) +} + +fn u64_at(bytes: &[u8], at: usize) -> Option { + Some(u64::from_le_bytes(bytes.get(at..at + 8)?.try_into().ok()?)) +} + +/// A NUL-terminated string at an offset, without running off the end. +fn string_at(bytes: &[u8], at: usize) -> Option { + let rest = bytes.get(at..)?; + let end = rest.iter().position(|byte| *byte == 0)?; + Some(String::from_utf8_lossy(&rest[..end]).into_owned()) +} + +const PT_LOAD: u32 = 1; +const PT_DYNAMIC: u32 = 2; +const PT_INTERP: u32 = 3; +const DT_NULL: u64 = 0; +const DT_NEEDED: u64 = 1; +const DT_STRTAB: u64 = 5; + +/// What one file will ask the loader for. +/// +/// `None` when the bytes are not an ELF64 little-endian file this can read. +/// Not an error type: every caller's next move is the same — a file that is +/// not an ELF is not a program whose libraries anybody has to find. +pub fn needs(bytes: &[u8]) -> Option { + if bytes.len() < 64 || &bytes[..4] != b"\x7fELF" || bytes[4] != 2 || bytes[5] != 1 { + return None; + } + let phoff = u64_at(bytes, 0x20)? as usize; + let phentsize = u16_at(bytes, 0x36)? as usize; + let phnum = u16_at(bytes, 0x38)? as usize; + + let mut needs = Needs::default(); + let mut loads: Vec<(u64, u64, u64)> = Vec::new(); + let mut dynamic: Option<(usize, usize)> = None; + + for index in 0..phnum { + let at = phoff.checked_add(index.checked_mul(phentsize)?)?; + let kind = u32_at(bytes, at)?; + let offset = u64_at(bytes, at + 0x08)?; + let vaddr = u64_at(bytes, at + 0x10)?; + let filesz = u64_at(bytes, at + 0x20)?; + match kind { + PT_LOAD => loads.push((vaddr, offset, filesz)), + PT_INTERP => needs.interpreter = string_at(bytes, offset as usize), + PT_DYNAMIC => dynamic = Some((offset as usize, filesz as usize)), + _ => {} + } + } + + // A virtual address is not a file offset. Every string the dynamic section + // names lives at a *vaddr*, and the only thing that translates one is the + // load map — which is why this walks `PT_LOAD` rather than assuming the + // two coincide. They do coincide in most binaries, and a reader that + // assumed it would be right until the first file where they do not. + let file_offset = |vaddr: u64| -> Option { + loads + .iter() + .find(|(base, _, size)| vaddr >= *base && vaddr < base + size) + .map(|(base, offset, _)| (offset + (vaddr - base)) as usize) + }; + + if let Some((offset, size)) = dynamic { + let mut strtab = None; + let mut wanted: Vec = Vec::new(); + let mut at = offset; + while at + 16 <= offset + size { + let tag = u64_at(bytes, at)?; + let value = u64_at(bytes, at + 8)?; + match tag { + DT_NULL => break, + DT_NEEDED => wanted.push(value), + DT_STRTAB => strtab = Some(value), + _ => {} + } + at += 16; + } + if let Some(strtab) = strtab.and_then(file_offset) { + for offset in wanted { + if let Some(name) = string_at(bytes, strtab + offset as usize) { + needs.libraries.push(name); + } + } + } + } + Some(needs) +} + +/// The same, for a file on disk. +pub fn needs_of(path: &Path) -> Option { + needs(&std::fs::read(path).ok()?) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_file_that_is_not_an_elf_is_not_read_as_one() { + assert_eq!(needs(b"#!/bin/sh\necho hello\n"), None); + assert_eq!(needs(&[]), None); + // Long enough not to be rejected on length, and still not an ELF. + assert_eq!(needs(&[0u8; 128]), None); + } + + #[test] + fn a_real_binary_names_its_loader_and_its_libraries() { + // Rule 6 of `Estrategia-de-Pruebas.md`, applied one level down: a + // hand-written ELF would prove this reader matches somebody's model of + // the format. The one real ELF every test run is guaranteed to have is + // the test binary itself. + let me = std::env::current_exe().expect("this test's own binary"); + let needs = needs_of(&me).expect("the test binary is an ELF64"); + // Statically linked test binaries exist — this asserts the reader got + // a coherent answer, not that the host links dynamically. + if let Some(interpreter) = &needs.interpreter { + assert!(interpreter.starts_with('/'), "{needs:?}"); + assert!(!needs.libraries.is_empty(), "{needs:?}"); + assert!( + needs.libraries.iter().any(|name| name.contains("libc")), + "{needs:?}" + ); + } + for name in &needs.libraries { + assert!(!name.is_empty(), "{needs:?}"); + } + } +} diff --git a/crates/thalyx-rust/src/lib.rs b/crates/thalyx-rust/src/lib.rs index 36dd1d7..e64e801 100644 --- a/crates/thalyx-rust/src/lib.rs +++ b/crates/thalyx-rust/src/lib.rs @@ -29,7 +29,9 @@ pub mod affected; pub mod analyzer; pub mod edits; +pub mod elf; pub mod metadata; +pub mod runtime; pub mod toolchain; pub use affected::{Affected, affected}; diff --git a/crates/thalyx-rust/src/runtime.rs b/crates/thalyx-rust/src/runtime.rs new file mode 100644 index 0000000..0d70572 --- /dev/null +++ b/crates/thalyx-rust/src/runtime.rs @@ -0,0 +1,541 @@ +//! The Rust runtime Thalyx carries, as opposed to the one a host happens to +//! have. +//! +//! ## The failure this file exists to stop +//! +//! On 2026-08-30 a paid benchmark watched Claude, inside the machine, choose +//! exactly the right primitive — `thalyx.context`, then `thalyx.rename` — and +//! receive `source: index`, `analyzer_starts: 0`, and +//! +//! ```text +//! rename: { ok: false, error: unresolved, +//! message: "there is no `cargo` on this machine" } +//! ``` +//! +//! Everything the agent did afterwards was a consequence of a promise the +//! machine could not keep. [`crate::toolchain`] knows how to find a toolchain +//! that somebody installed; inside Thalyx nobody installed one, and nobody is +//! going to — `Filosofia-Fundacional.md` says Thalyx is the whole system, so a +//! programming face that only works because the host has rustup is a +//! programming face that belongs to the host. +//! +//! So the runtime is **an artifact of Thalyx's own**, built by +//! `dev/build-rust-runtime.sh` from digest-checked upstream tarballs, staged +//! onto the store, and found here. Move the disk to another x86_64 machine and +//! the semantic provider moves with it. +//! +//! ## Where it lives, and why the store rather than the image +//! +//! `/toolchains/rust//`, which is +//! `/opt/thalyx/toolchains/rust/rust--/` on a running machine. +//! +//! Never the initramfs. `make -C image count` says the image is the Linux +//! kernel and one program, and that is the decree it exists to keep countable +//! — six hundred megabytes of compiler is *software installed on Thalyx*, +//! which is the whole distinction between what Thalyx is and what has been put +//! on it. The engine and its weights already live on the store for the same +//! reason. +//! +//! ## What the artifact is made of +//! +//! The upstream `x86_64-unknown-linux-musl` host tools, which need exactly two +//! files Rust does not publish: musl's loader — compiled from musl's own +//! release tarball by the build script — and `libgcc_s.so.1`, linked out of +//! the `libunwind.a` that ships inside `rust-std` itself. Neither is copied +//! from the machine that built it. The reasoning, and the measurement behind +//! choosing musl over the ordinary GNU toolchain, is in +//! `dev/build-rust-runtime.sh` and in +//! `vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md`. + +use std::path::{Path, PathBuf}; + +/// Where runtimes live under a store root. +/// +/// One string, used by the build script's `INSIDE`, by PID 1, and by +/// discovery. Two places spelling this is two answers to where the toolchain +/// is, and the second one is always the empty directory somebody is confused +/// by. +pub const UNDER: &str = "toolchains/rust"; + +/// The store root a machine uses when nothing says otherwise. +/// +/// The same default `thalyx-cli` has. Repeated rather than depended on because +/// this crate must not depend on the CLI, and a *wrong* default here would be +/// a silent failure to find a toolchain that is right there. +pub const STORE_ROOT: &str = "/opt/thalyx"; + +/// The file the loader is asked for by every program in the artifact. +/// +/// It is in the ELF header of the binaries themselves — `PT_INTERP` — so it is +/// not a choice anybody here gets to make. PID 1 makes the name resolve. +pub const LOADER: &str = "/lib/ld-musl-x86_64.so.1"; + +/// Everything the artifact must contain to be one, as paths relative to its +/// root. +/// +/// A list of what is there rather than a list of what is not: an exclusion +/// list is a claim about everything nobody thought of. +/// +/// `lib/rustlib/src` is on it because it was **measured** to be required, not +/// because it seemed thorough: without it rust-analyzer says `can't load +/// standard library, try installing rust-src` and then dies partway through +/// the first analysis. +pub const NEEDED: &[&str] = &[ + "bin/cargo", + "bin/rustc", + "bin/rust-analyzer", + "libexec/rust-analyzer-proc-macro-srv", + "lib/libc.so", + "lib/ld-musl-x86_64.so.1", + "lib/libgcc_s.so.1", + "lib/rustlib/src", + "runtime.json", +]; + +/// Things whose presence means somebody copied a whole toolchain in. +/// +/// Checked rather than trusted, because the way this goes wrong is not a crash +/// — it is a store that quietly grew by a gigabyte of manual pages and +/// documentation nothing inside the machine can read, and nobody noticing +/// until the disk is full. +pub const FORBIDDEN: &[&str] = &[ + "share", + "bin/rustdoc", + "bin/rustfmt", + "bin/cargo-clippy", + "bin/rustup", + "lib/rustlib/etc", +]; + +/// A staged runtime, and what it says about itself. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Runtime { + /// Where the artifact is, on the machine asking. + pub root: PathBuf, + /// `rust--`, which is the directory's own name. + pub identity: String, + /// The Rust release, from `runtime.json`. + pub rust: Option, + /// The musl release the loader was built from, from `runtime.json`. + pub musl: Option, +} + +impl Runtime { + pub fn cargo(&self) -> PathBuf { + self.root.join("bin/cargo") + } + pub fn rustc(&self) -> PathBuf { + self.root.join("bin/rustc") + } + pub fn rust_analyzer(&self) -> PathBuf { + self.root.join("bin/rust-analyzer") + } + pub fn lib(&self) -> PathBuf { + self.root.join("lib") + } + /// The loader, at the path inside the artifact — which is the file PID 1 + /// makes [`LOADER`] point at. + pub fn loader(&self) -> PathBuf { + self.root.join("lib/libc.so") + } + /// One line naming what this is, for an answer that has to say which + /// toolchain produced it. + pub fn describe(&self) -> String { + match (&self.rust, &self.musl) { + (Some(rust), Some(musl)) => { + format!( + "Thalyx runtime {} (Rust {rust}, musl {musl})", + self.identity + ) + } + _ => format!("Thalyx runtime {}", self.identity), + } + } +} + +/// The directory runtimes are staged into, under a store root. +pub fn directory(store_root: &Path) -> PathBuf { + store_root.join(UNDER) +} + +/// The store root this process should look under. +/// +/// `THALYX_ROOT` first, because that is how every test and every stage of +/// `verify.sh` moves the store somewhere it can be written without becoming +/// the machine's real one. +pub fn store_root() -> PathBuf { + std::env::var_os("THALYX_ROOT") + .map(PathBuf::from) + .unwrap_or_else(|| PathBuf::from(STORE_ROOT)) +} + +/// Every runtime staged under a store root, in a fixed order. +/// +/// Sorted by directory name, so a store that somehow holds two of them picks +/// the same one on every boot. A verdict whose meaning depends on `read_dir` +/// order is a verdict nobody can reproduce — and `read_dir` is not sorted on +/// any filesystem this runs on. +pub fn staged(store_root: &Path) -> Vec { + let Ok(entries) = std::fs::read_dir(directory(store_root)) else { + return Vec::new(); + }; + let mut roots: Vec = entries + .flatten() + .map(|entry| entry.path()) + .filter(|path| path.is_dir()) + .collect(); + roots.sort(); + roots.iter().filter_map(|root| read(root)).collect() +} + +/// Read one artifact directory, if it looks like an artifact at all. +/// +/// `None` when the required files are not all there. That is deliberate and it +/// is rule 9: a half-staged toolchain — an interrupted copy, a disk that +/// filled — must not be discovered, because what it produces is not a refusal +/// but a rust-analyzer that starts and dies, which reads as the semantic +/// provider being broken rather than as a store that was never finished. +pub fn read(root: &Path) -> Option { + if !NEEDED.iter().all(|needed| root.join(needed).exists()) { + return None; + } + let identity = root.file_name()?.to_string_lossy().into_owned(); + let (rust, musl) = match std::fs::read_to_string(root.join("runtime.json")) { + Ok(text) => (field(&text, "rust"), field(&text, "musl")), + // A runtime.json that cannot be read is not a runtime that is not + // there: the binaries are all present and they will run. Rule 10 — say + // which one happened — so the version comes back unknown and the + // toolchain is still found. + Err(_) => (None, None), + }; + Some(Runtime { + root: root.to_path_buf(), + identity, + rust, + musl, + }) +} + +/// One string field out of `runtime.json`, without a JSON dependency. +/// +/// This crate is read by PID 1's own binary and the file is written by a shell +/// script three lines above where it is read; a parser here would be a +/// dependency taken on to read two version numbers that are only ever used in +/// a sentence a human reads. +fn field(text: &str, name: &str) -> Option { + let needle = format!("\"{name}\""); + let after = text.split_once(&needle)?.1; + let after = after.split_once(':')?.1; + let after = after.trim_start(); + let rest = after.strip_prefix('"')?; + let (value, _) = rest.split_once('"')?; + Some(value.to_string()) +} + +/// The runtime this machine has, if it has one. +pub fn managed() -> Option { + staged(&store_root()).into_iter().next() +} + +// ── is a staged tree the thing it claims to be ─────────────────────────────── + +/// What a directory has and has not, said in the terms that decide what to do. +#[derive(Debug, Clone, PartialEq, Eq, Default)] +pub struct Report { + /// Required paths that are not there. + pub missing: Vec, + /// Paths that are there and should not be — the sign that a whole + /// toolchain was copied rather than a runtime assembled. + pub forbidden: Vec, + /// Target directories under `lib/rustlib` other than this artifact's own. + pub other_targets: Vec, +} + +impl Report { + pub fn is_complete(&self) -> bool { + self.missing.is_empty() && self.forbidden.is_empty() && self.other_targets.is_empty() + } +} + +/// Look at a staged tree and say what is wrong with it. +/// +/// Separate from [`read`] because they answer different questions: `read` asks +/// "can this be used", which a machine needs at every boot, and this asks "was +/// this assembled correctly", which the build and its tests need once. +pub fn inspect(root: &Path) -> Report { + let mut report = Report::default(); + for needed in NEEDED { + if !root.join(needed).exists() { + report.missing.push((*needed).to_string()); + } + } + for forbidden in FORBIDDEN { + if root.join(forbidden).exists() { + report.forbidden.push((*forbidden).to_string()); + } + } + // The target the artifact says it is for is the only one that may be under + // `rustlib`. `lib/rustlib/x86_64-unknown-linux-gnu` in a musl artifact is + // 220 MB of a standard library for a machine this is not. + // + // Read from `runtime.json` and only then from the directory's name: a + // staging step copies the artifact under whatever name it likes, and an + // artifact that reported a problem because somebody renamed its directory + // would be reporting on the copy rather than on the toolchain. Found by + // staging one as `broken/` on purpose and watching it accuse its own + // standard library. + let ours = std::fs::read_to_string(root.join("runtime.json")) + .ok() + .and_then(|text| field(&text, "target")) + .or_else(|| { + root.file_name() + .map(|name| name.to_string_lossy().into_owned()) + }) + .unwrap_or_default(); + if let Ok(entries) = std::fs::read_dir(root.join("lib/rustlib")) { + for entry in entries.flatten() { + let name = entry.file_name().to_string_lossy().into_owned(); + if name == "src" || !entry.path().is_dir() { + continue; + } + if !ours.ends_with(&name) { + report.other_targets.push(name); + } + } + } + report +} + +// ── does the artifact carry what its own programs ask for ──────────────────── + +/// One program of the artifact and what it will ask the loader for. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Asked { + pub program: PathBuf, + pub interpreter: Option, + /// Every `DT_NEEDED`, paired with whether this artifact has it. + pub libraries: Vec<(String, bool)>, +} + +/// Whether the artifact is closed: everything its programs name is inside it. +#[derive(Debug, Clone, PartialEq, Eq, Default)] +pub struct Closure { + pub programs: Vec, + /// `(program, library)` for every name nothing in the artifact provides. + pub unresolved: Vec<(String, String)>, + /// The interpreters the programs ask the kernel for, deduplicated. + /// + /// Carried rather than checked against a constant, because the constant is + /// derived *from* this: [`LOADER`] is what the binaries say, not what + /// anybody chose. + pub interpreters: Vec, + /// Whether the artifact carries a file with the interpreter's own name. + pub interpreter_inside: bool, +} + +impl Closure { + /// Nothing is missing and the loader travels with the artifact. + pub fn is_closed(&self) -> bool { + self.unresolved.is_empty() && self.interpreter_inside && !self.programs.is_empty() + } +} + +/// Read every program in the artifact and say whether the artifact is enough. +/// +/// This is the check that distinguishes "the files were copied" from "the +/// machine can run them", and it is the whole point of the exercise: a runtime +/// that resolves against the *building* host looks identical, on the building +/// host, to one that resolves against itself. `ldd` cannot tell them apart +/// because `ldd` asks the host. This asks the artifact. +/// +/// Symbols are not its business — a name that resolves to a library which is +/// missing a function is a different failure, and the build script catches +/// that one by running the programs. +pub fn closure(root: &Path) -> Closure { + let mut closure = Closure::default(); + let lib = root.join("lib"); + for relative in NEEDED { + if !relative.starts_with("bin/") && !relative.starts_with("libexec/") { + continue; + } + let program = root.join(relative); + let Some(needs) = crate::elf::needs_of(&program) else { + continue; + }; + if let Some(interpreter) = &needs.interpreter + && !closure.interpreters.contains(interpreter) + { + closure.interpreters.push(interpreter.clone()); + } + let mut libraries = Vec::new(); + for name in &needs.libraries { + let here = lib.join(name).exists(); + if !here { + closure + .unresolved + .push(((*relative).to_string(), name.clone())); + } + libraries.push((name.clone(), here)); + } + closure.programs.push(Asked { + program, + interpreter: needs.interpreter, + libraries, + }); + } + // The kernel reads `PT_INTERP` as an absolute path, so what matters is + // that the artifact carries a file of that name for PID 1 to point the + // path at. Derived from what the binaries said rather than from `LOADER`, + // so an artifact built for some other interpreter is reported honestly + // instead of silently checked against the wrong name. + closure.interpreter_inside = !closure.interpreters.is_empty() + && closure.interpreters.iter().all(|interpreter| { + Path::new(interpreter) + .file_name() + .is_some_and(|name| lib.join(name).exists()) + }); + closure +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A tree shaped like a finished artifact, with empty files. + fn staged_tree(identity: &str) -> tempfile::TempDir { + let directory = tempfile::tempdir().expect("a temp dir"); + let root = directory.path().join(identity); + for needed in NEEDED { + let path = root.join(needed); + if *needed == "lib/rustlib/src" { + std::fs::create_dir_all(&path).expect("the source directory"); + continue; + } + std::fs::create_dir_all(path.parent().expect("a parent")).expect("the directory"); + std::fs::write(&path, b"").expect("the file"); + } + std::fs::create_dir_all(root.join("lib/rustlib/x86_64-unknown-linux-musl/lib")) + .expect("the sysroot"); + std::fs::write( + root.join("runtime.json"), + br#"{"identity": "x", "rust": "1.90.0", "musl": "1.2.4", "target": "x86_64-unknown-linux-musl"}"#, + ) + .expect("the description"); + directory + } + + #[test] + fn an_artifact_with_everything_it_needs_is_complete() { + let held = staged_tree("rust-1.90.0-x86_64-unknown-linux-musl"); + let root = held.path().join("rust-1.90.0-x86_64-unknown-linux-musl"); + let report = inspect(&root); + assert!(report.is_complete(), "{report:?}"); + } + + #[test] + fn a_copy_of_a_whole_toolchain_is_not_an_artifact() { + // The mistake this catches is not a crash: it is a store that grew by + // most of a gigabyte of manual pages, and nobody noticing until the + // disk filled. + let held = staged_tree("rust-1.90.0-x86_64-unknown-linux-musl"); + let root = held.path().join("rust-1.90.0-x86_64-unknown-linux-musl"); + std::fs::create_dir_all(root.join("share/doc")).expect("the documentation"); + std::fs::write(root.join("bin/rustdoc"), b"").expect("rustdoc"); + let report = inspect(&root); + assert!(!report.is_complete()); + assert!( + report.forbidden.contains(&"share".to_string()), + "{report:?}" + ); + assert!( + report.forbidden.contains(&"bin/rustdoc".to_string()), + "{report:?}" + ); + } + + #[test] + fn a_standard_library_for_another_machine_is_not_wanted() { + let held = staged_tree("rust-1.90.0-x86_64-unknown-linux-musl"); + let root = held.path().join("rust-1.90.0-x86_64-unknown-linux-musl"); + std::fs::create_dir_all(root.join("lib/rustlib/x86_64-unknown-linux-gnu/lib")) + .expect("the other target"); + let report = inspect(&root); + assert_eq!( + report.other_targets, + vec!["x86_64-unknown-linux-gnu".to_string()] + ); + assert!(!report.is_complete()); + } + + #[test] + fn the_standard_library_sources_are_required_because_the_analyzer_dies_without_them() { + // Measured, not guessed: with `lib/rustlib/src` absent, rust-analyzer + // logs `can't load standard library, try installing rust-src` and + // aborts partway through its first analysis. + assert!(NEEDED.contains(&"lib/rustlib/src")); + let held = staged_tree("rust-1.90.0-x86_64-unknown-linux-musl"); + let root = held.path().join("rust-1.90.0-x86_64-unknown-linux-musl"); + std::fs::remove_dir_all(root.join("lib/rustlib/src")).expect("removing the sources"); + assert!( + inspect(&root) + .missing + .contains(&"lib/rustlib/src".to_string()) + ); + } + + #[test] + fn renaming_the_directory_does_not_make_the_artifact_wrong() { + // Found by staging one under the name `broken/` on purpose: the target + // used to be read from the directory's name, so an artifact copied + // under any other name accused its own standard library of being for a + // machine it is not. The check was reporting on the copy instead of on + // the toolchain. + let held = staged_tree("something-somebody-renamed"); + let root = held.path().join("something-somebody-renamed"); + assert!(inspect(&root).other_targets.is_empty(), "{root:?}"); + } + + #[test] + fn a_half_staged_runtime_is_not_discovered() { + // An interrupted copy leaves a directory that looks finished. What it + // produces is not a refusal but a rust-analyzer that starts and dies, + // which reads as a broken provider rather than an unfinished store. + let held = staged_tree("rust-1.90.0-x86_64-unknown-linux-musl"); + let root = held.path().join("rust-1.90.0-x86_64-unknown-linux-musl"); + std::fs::remove_file(root.join("lib/libgcc_s.so.1")).expect("removing the unwinder"); + assert_eq!(read(&root), None); + assert!(staged(held.path()).is_empty()); + } + + #[test] + fn a_staged_runtime_says_which_rust_and_which_musl_it_is() { + let held = staged_tree("rust-1.90.0-x86_64-unknown-linux-musl"); + let root = held.path().join("rust-1.90.0-x86_64-unknown-linux-musl"); + let runtime = read(&root).expect("a runtime"); + assert_eq!(runtime.rust.as_deref(), Some("1.90.0")); + assert_eq!(runtime.musl.as_deref(), Some("1.2.4")); + assert!(runtime.describe().contains("musl 1.2.4"), "{runtime:?}"); + } + + #[test] + fn two_staged_runtimes_are_chosen_between_the_same_way_every_time() { + let directory = tempfile::tempdir().expect("a temp dir"); + let under = directory.path().join(UNDER); + for identity in ["rust-1.91.0-x86_64-unknown-linux-musl", "rust-1.90.0-x"] { + for needed in NEEDED { + let path = under.join(identity).join(needed); + if *needed == "lib/rustlib/src" { + std::fs::create_dir_all(&path).expect("sources"); + continue; + } + std::fs::create_dir_all(path.parent().expect("a parent")).expect("a directory"); + std::fs::write(&path, b"{}").expect("a file"); + } + } + let once = staged(directory.path()); + let twice = staged(directory.path()); + assert_eq!(once, twice); + assert_eq!(once.len(), 2); + assert_eq!(once[0].identity, "rust-1.90.0-x"); + } +} diff --git a/crates/thalyx-rust/src/toolchain.rs b/crates/thalyx-rust/src/toolchain.rs index 185d712..12e6597 100644 --- a/crates/thalyx-rust/src/toolchain.rs +++ b/crates/thalyx-rust/src/toolchain.rs @@ -26,18 +26,30 @@ //! 1. **What somebody said explicitly.** `THALYX_CARGO` and //! `THALYX_RUST_ANALYZER` name a file. A run whose meaning has to be //! reproducible is a run whose tools were named, not found. -//! 2. **What rustup itself was told.** `CARGO_HOME` and `RUSTUP_HOME` are +//! 2. **Thalyx's own runtime**, staged on the store — see [`crate::runtime`]. +//! Added 2026-08-31, after a paid benchmark run inside the machine came +//! back with `there is no cargo on this machine` for a workspace Thalyx had +//! promised it could rename symbols in. It is second and not fifth on +//! purpose: `Filosofia-Fundacional.md` says Thalyx is the whole system, so +//! when Thalyx carries a compiler that is *the* compiler, and a host's +//! installed one is the fallback rather than the other way round. Only a +//! person naming a file outright outranks it. +//! 3. **What rustup itself was told.** `CARGO_HOME` and `RUSTUP_HOME` are //! rustup's own variables, they survive `sudo -E`, and `verify.sh` sets //! them from `$SUDO_USER`'s home precisely so that root can use the //! person's toolchain on purpose. Reading them is not a workaround: it is //! reading the configuration. -//! 3. **The invoking user's home, when `sudo` says who that was.** `SUDO_USER` +//! 4. **The invoking user's home, when `sudo` says who that was.** `SUDO_USER` //! plus the passwd entry, so a root shell finds the toolchain of the person //! who asked for it. -//! 4. **`HOME`.** The ordinary case, where nobody is pretending to be anybody. -//! 5. **Named system locations.** `/usr/local/bin`, `/usr/bin`. Two paths, in +//! 5. **`HOME`.** The ordinary case, where nobody is pretending to be anybody. +//! 6. **Named system locations.** `/usr/local/bin`, `/usr/bin`. Two paths, in //! a fixed order. //! +//! Steps 3 to 6 are how `dev/verify.sh` and every developer machine still +//! work: they have no store and therefore no step 2, and nothing about them +//! changed. +//! //! ## And never a walk of `PATH` //! //! A validation that ran whichever `cargo` came first on a caller's `PATH` is @@ -64,6 +76,32 @@ use std::path::{Path, PathBuf}; use std::process::{Command, Stdio}; use std::sync::OnceLock; +/// Whose toolchain answered. +/// +/// Reported rather than inferred from the path, because the whole point of +/// `vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md` is a machine that can say +/// **it is using its own** — and "the path starts with /opt/thalyx" is a guess +/// that is right until somebody mounts a store somewhere else. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Kind { + /// A variable named this file outright. + Named, + /// Thalyx's own runtime artifact, staged on the store. + Managed, + /// A toolchain somebody installed on the machine this is running on. + Installed, +} + +impl Kind { + pub fn as_str(self) -> &'static str { + match self { + Kind::Named => "named", + Kind::Managed => "thalyx", + Kind::Installed => "host", + } + } +} + /// A tool that was looked for, and what was found. /// /// Two fields and not an `Option`, because "there is no cargo here" and "here @@ -82,6 +120,8 @@ pub struct Found { pub looked_at: Vec, /// The variable that named it, when one did. pub named_by: Option<&'static str>, + /// Whose toolchain it turned out to be. `None` when nothing was found. + pub kind: Option, } impl Found { @@ -167,6 +207,24 @@ fn toolchain_bins(rustup_home: &Path) -> Vec { bins } +/// The `bin` of Thalyx's own runtime artifact, when the store carries one. +/// +/// **Ahead of every installed toolchain**, and that ordering is the decree of +/// 2026-08-31 rather than a preference: a Thalyx that resolved names with the +/// host's compiler would be a Thalyx whose programming face belongs to the +/// host. Only a variable that names a file outright comes before it, because a +/// person saying which compiler to use is the one thing more explicit than +/// Thalyx's own. +/// +/// On a machine with no store — this container, a laptop running the tests — +/// there is nothing here and the search continues exactly as it did. +fn managed_places_under(store_root: &Path) -> Vec { + crate::runtime::staged(store_root) + .into_iter() + .map(|runtime| runtime.root.join("bin")) + .collect() +} + /// Where an installed toolchain's binaries could be, most explicit first. fn toolchain_places() -> Vec { let mut places = Vec::new(); @@ -213,7 +271,10 @@ fn answers(candidate: &Path) -> bool { } /// Look for one binary in the named places, running each candidate. -fn look_for(binary: &str, named_by: &'static str, places: Vec) -> Found { +/// +/// Each place carries whose it is, so the answer can say which toolchain +/// produced it rather than leaving a caller to guess from the path. +fn look_for(binary: &str, named_by: &'static str, places: Vec<(PathBuf, Kind)>) -> Found { let mut looked_at = Vec::new(); let mut named = None; @@ -226,6 +287,7 @@ fn look_for(binary: &str, named_by: &'static str, places: Vec) -> Found path: Some(path), looked_at, named_by: Some(named_by), + kind: Some(Kind::Named), }; } // Named and wrong is worth saying. A variable pointing at a file @@ -236,7 +298,7 @@ fn look_for(binary: &str, named_by: &'static str, places: Vec) -> Found } } - for place in places { + for (place, kind) in places { let candidate = place.join(binary); if looked_at.contains(&candidate) { continue; @@ -247,6 +309,7 @@ fn look_for(binary: &str, named_by: &'static str, places: Vec) -> Found path: Some(candidate), looked_at, named_by: None, + kind: Some(kind), }; } } @@ -255,9 +318,35 @@ fn look_for(binary: &str, named_by: &'static str, places: Vec) -> Found path: None, looked_at, named_by: named, + kind: None, } } +/// Every place a tool could be, in the order authority says to look. +/// +/// One list, built once, so `cargo` and `rust-analyzer` cannot disagree about +/// which toolchain this machine is using — a machine whose cargo is Thalyx's +/// and whose rust-analyzer is the host's is two machines wearing one name. +fn places() -> Vec<(PathBuf, Kind)> { + places_under(&crate::runtime::store_root()) +} + +/// The same, for a named store — which is what makes the *order* testable +/// without a test having to change the machine it is measuring. Rule 11. +fn places_under(store_root: &Path) -> Vec<(PathBuf, Kind)> { + let mut places: Vec<(PathBuf, Kind)> = managed_places_under(store_root) + .into_iter() + .map(|path| (path, Kind::Managed)) + .collect(); + places.extend( + toolchain_places() + .into_iter() + .chain(shim_places()) + .map(|path| (path, Kind::Installed)), + ); + places +} + /// The environment variable that names a cargo outright. pub const CARGO_VARIABLE: &str = "THALYX_CARGO"; @@ -267,21 +356,13 @@ pub const ANALYZER_VARIABLE: &str = "THALYX_RUST_ANALYZER"; /// Where this machine's `cargo` is, and where it was looked for. pub fn cargo() -> &'static Found { static ASKED: OnceLock = OnceLock::new(); - ASKED.get_or_init(|| { - let mut places = toolchain_places(); - places.extend(shim_places()); - look_for("cargo", CARGO_VARIABLE, places) - }) + ASKED.get_or_init(|| look_for("cargo", CARGO_VARIABLE, places())) } /// Where this machine's `rust-analyzer` is, and where it was looked for. pub fn rust_analyzer() -> &'static Found { static ASKED: OnceLock = OnceLock::new(); - ASKED.get_or_init(|| { - let mut places = toolchain_places(); - places.extend(shim_places()); - look_for("rust-analyzer", ANALYZER_VARIABLE, places) - }) + ASKED.get_or_init(|| look_for("rust-analyzer", ANALYZER_VARIABLE, places())) } /// The `cargo` to run, falling back to the bare name. @@ -305,8 +386,45 @@ pub fn cargo_command() -> PathBuf { /// would change what every other part of Thalyx thinks the machine is, which /// is rule 11 — a global switch with no owner, whose value is some other /// check's precondition. -pub fn environment() -> Vec<(&'static str, PathBuf)> { - let mut environment = Vec::new(); +/// +/// ## Two machines, two answers, and the difference is the whole decree +/// +/// When the toolchain is **Thalyx's own**, this must not name the host's +/// `RUSTUP_HOME` or `CARGO_HOME`. Handing a managed cargo the registry of +/// whoever built the disk is exactly the borrowing that +/// `vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md` exists to end — and it +/// would be invisible, because on the machine that built the store it works. +/// So a managed toolchain gets a `CARGO_HOME` **on the store**, beside the +/// rest of Thalyx's state, and nothing pointing outward. +/// +/// `CARGO_NET_OFFLINE` because the semantic provider has no network by +/// construction, and a Cargo that does not know that spends its timeout +/// finding out. Failing closed is the fast answer here as well as the correct +/// one. +/// +/// `LD_LIBRARY_PATH` because the toolchain's binaries carry +/// `RPATH: [$ORIGIN/../lib]`, and **musl resolves `$ORIGIN` for the main +/// program by reading `/proc/self/exe`** — measured on 2026-08-31, where the +/// same binary ran with `/proc` mounted and failed without it with +/// `Error loading shared library librustc_driver-.so`. Naming the +/// directory outright makes the toolchain independent of whether whoever +/// starts it remembered `/proc`. It reaches nothing new: the directory is +/// inside the artifact [`readable`] already grants. +pub fn environment() -> Vec<(&'static str, String)> { + let mut environment: Vec<(&'static str, String)> = Vec::new(); + + if let Some(runtime) = managed_runtime() { + environment.push((LOADER_PATH_VARIABLE, runtime.lib().display().to_string())); + // Under the store, so it survives a reboot and belongs to Thalyx. Made + // here rather than left to Cargo: a directory a grant names has to + // exist before the grant can be given. + let home = crate::runtime::store_root().join("state").join("cargo"); + let _ = std::fs::create_dir_all(&home); + environment.push(("CARGO_HOME", home.display().to_string())); + environment.push(("CARGO_NET_OFFLINE", "true".to_string())); + return environment; + } + let rustup = std::env::var_os("RUSTUP_HOME") .map(PathBuf::from) .or_else(|| { @@ -316,7 +434,7 @@ pub fn environment() -> Vec<(&'static str, PathBuf)> { .find(|path| path.is_dir()) }); if let Some(rustup) = rustup { - environment.push(("RUSTUP_HOME", rustup)); + environment.push(("RUSTUP_HOME", rustup.display().to_string())); } let cargo_home = std::env::var_os("CARGO_HOME") .map(PathBuf::from) @@ -327,11 +445,26 @@ pub fn environment() -> Vec<(&'static str, PathBuf)> { .find(|path| path.is_dir()) }); if let Some(cargo_home) = cargo_home { - environment.push(("CARGO_HOME", cargo_home)); + environment.push(("CARGO_HOME", cargo_home.display().to_string())); } environment } +/// The runtime this machine's tools actually came out of, or `None`. +/// +/// Asked of [`cargo`] rather than of the store, and the difference matters: a +/// store can hold an artifact that does not run — built for another +/// architecture, half copied, staged on a host with no musl loader — and in +/// that case [`cargo`] has already fallen through to an installed toolchain. +/// Reading the store would then describe a toolchain nothing is using. +pub fn managed_runtime() -> Option { + if cargo().kind != Some(Kind::Managed) { + return None; + } + let bin = cargo().path.as_ref()?.parent()?; + crate::runtime::read(bin.parent()?) +} + /// The environment variable that names where the loader looks first. pub const LOADER_PATH_VARIABLE: &str = "LD_LIBRARY_PATH"; @@ -389,8 +522,19 @@ pub fn readable() -> Vec { readable.push(path); } }; - for (_, path) in environment() { - push(path); + // Thalyx's own runtime first, and it is the only entry that matters + // inside the machine: the artifact holds the compiler, the standard + // library, the standard library's sources and the loader, and a confined + // provider that cannot read it cannot start. + if let Some(runtime) = managed_runtime() { + push(runtime.root.clone()); + push(crate::runtime::store_root().join("state").join("cargo")); + } + if let Some(named) = std::env::var_os("RUSTUP_HOME") { + push(PathBuf::from(named)); + } + if let Some(named) = std::env::var_os("CARGO_HOME") { + push(PathBuf::from(named)); } for home in homes() { push(home.join(".cargo")); @@ -420,7 +564,7 @@ mod tests { let found = look_for( "cargo", "THALYX_TEST_NOTHING_NAMES_THIS", - vec![PathBuf::from("/nonexistent/place")], + vec![(PathBuf::from("/nonexistent/place"), Kind::Installed)], ); assert_eq!(found.path, None); assert!( @@ -449,7 +593,7 @@ mod tests { let found = look_for( "rust-analyzer", "THALYX_TEST_NOTHING_NAMES_THIS", - vec![directory.path().to_path_buf()], + vec![(directory.path().to_path_buf(), Kind::Installed)], ); assert_eq!(found.path, None, "a shim that fails was taken for the tool"); assert_eq!(found.looked_at, vec![impostor]); @@ -464,8 +608,8 @@ mod tests { "rust-analyzer", "THALYX_TEST_NOTHING_NAMES_THIS", vec![ - PathBuf::from("/nonexistent/one"), - PathBuf::from("/none/two"), + (PathBuf::from("/nonexistent/one"), Kind::Installed), + (PathBuf::from("/none/two"), Kind::Installed), ], ); let why = found.why_not( @@ -477,6 +621,108 @@ mod tests { assert!(why.contains("rustup component add"), "{why}"); } + /// A staged runtime whose `bin/` is a script that answers. + /// + /// A fake, and it models the property under test rather than standing in + /// for the whole thing: the question here is *which place is looked at + /// first and does a candidate that answers win*, and for that a script + /// that prints a version is exactly as good as six hundred megabytes of + /// compiler. Rule 8 — a fake must model the property, and this one does. + fn a_runtime_that_answers(store_root: &Path, identity: &str) -> PathBuf { + let root = crate::runtime::directory(store_root).join(identity); + for needed in crate::runtime::NEEDED { + let path = root.join(needed); + if *needed == "lib/rustlib/src" { + std::fs::create_dir_all(&path).expect("the sources"); + continue; + } + std::fs::create_dir_all(path.parent().expect("a parent")).expect("a directory"); + std::fs::write(&path, b"{}").expect("a file"); + } + for name in ["cargo", "rust-analyzer"] { + let path = root.join("bin").join(name); + std::fs::write(&path, format!("#!/bin/sh\necho '{name} 0.0.0 (thalyx)'\n")) + .expect("the program"); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o755)) + .expect("executable"); + } + } + root + } + + #[test] + fn the_machines_own_runtime_is_looked_at_before_anything_installed() { + // The decree of 2026-08-31: when Thalyx carries a compiler, that is + // *the* compiler. An installed one is the fallback and not the other + // way round — otherwise the machine's programming face belongs to + // whatever host it happens to be booted on. + let store = tempfile::tempdir().expect("a temp store"); + a_runtime_that_answers(store.path(), "rust-1.90.0-x86_64-unknown-linux-musl"); + let places = places_under(store.path()); + assert_eq!(places.first().map(|(_, kind)| *kind), Some(Kind::Managed)); + assert!( + places[0] + .0 + .ends_with("toolchains/rust/rust-1.90.0-x86_64-unknown-linux-musl/bin"), + "{places:?}" + ); + assert!( + places[1..].iter().all(|(_, kind)| *kind == Kind::Installed), + "{places:?}" + ); + } + + #[test] + fn a_machine_with_no_store_looks_exactly_where_it_always_did() { + // The other half of the same claim, and the one that keeps every + // developer machine and `dev/verify.sh` working: with nothing staged + // there is no managed place at all, and the list is what it was. + let empty = tempfile::tempdir().expect("a temp store"); + let places = places_under(empty.path()); + assert!( + places.iter().all(|(_, kind)| *kind == Kind::Installed), + "{places:?}" + ); + } + + #[test] + fn the_toolchain_thalyx_carries_is_the_one_that_answers() { + // End of the ordering claim: not merely that the managed place is + // first in a list, but that the search returns it and says whose it + // is. `from: "thalyx"` is the field the preflight prints, and a run + // that cannot tell whose compiler answered cannot tell whether the + // machine was autonomous. + let store = tempfile::tempdir().expect("a temp store"); + let root = a_runtime_that_answers(store.path(), "rust-1.90.0-x86_64-unknown-linux-musl"); + let found = look_for( + "cargo", + "THALYX_TEST_NOTHING_NAMES_THIS", + places_under(store.path()), + ); + assert_eq!( + found.path.as_deref(), + Some(root.join("bin/cargo").as_path()) + ); + assert_eq!(found.kind, Some(Kind::Managed)); + } + + #[test] + fn a_half_staged_runtime_is_not_offered_as_a_toolchain() { + // Rule 9. An interrupted copy leaves `bin/cargo` sitting there looking + // finished; what it produces is a rust-analyzer that starts and dies, + // which reads as a broken provider rather than an unfinished store. + let store = tempfile::tempdir().expect("a temp store"); + let root = a_runtime_that_answers(store.path(), "rust-1.90.0-x86_64-unknown-linux-musl"); + std::fs::remove_file(root.join("lib/libc.so")).expect("removing the loader"); + assert!( + managed_places_under(store.path()).is_empty(), + "a runtime with no loader was offered as one" + ); + } + #[test] fn a_home_is_never_guessed_from_a_user_name() { // `/home/` exists on most machines and belongs to the wrong diff --git a/crates/thalyx-rust/tests/the_runtime_thalyx_carries_runs_on_its_own.rs b/crates/thalyx-rust/tests/the_runtime_thalyx_carries_runs_on_its_own.rs new file mode 100644 index 0000000..50d5174 --- /dev/null +++ b/crates/thalyx-rust/tests/the_runtime_thalyx_carries_runs_on_its_own.rs @@ -0,0 +1,271 @@ +//! The artifact is asked to be a machine's whole Rust toolchain, on a host that +//! is not the one that built it. +//! +//! `vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md`. The failure these are +//! written against is not a crash: it is an artifact that works perfectly on +//! the machine that assembled it, because every library it forgot to carry was +//! sitting in that machine's `/usr/lib`. On the machine it is *for* — a Thalyx, +//! whose `/lib` is empty — the same artifact is a directory of ELF files that +//! cannot start, and what the human reads is `there is no cargo on this +//! machine`. +//! +//! ## Why these can run anywhere, including a host with no musl +//! +//! musl's `libc.so` **is** the dynamic loader, and a loader can be invoked +//! directly with the program as its argument. So +//! `/lib/libc.so /bin/cargo` runs the staged cargo using +//! the staged loader, on any x86_64 Linux, without the host having musl and +//! without anybody writing to `/lib`. Rule 11: a test that wrote a machine-wide +//! symlink would have changed the machine it was measuring. +//! +//! ## And why they skip loudly +//! +//! Building the artifact downloads about 170 MB and compiles a C library, so +//! it is not a thing every `cargo test` should do. Rule 3: a test that skips +//! says `NOT PROVEN`, and `THALYX_REQUIRE_RUST_RUNTIME=1` turns the skip into +//! a failure, so a machine that has one can demand that it was used. + +use std::path::{Path, PathBuf}; +use std::process::Command; + +/// Where the artifact is, if this machine has one. +/// +/// `THALYX_RUST_RUNTIME` names it outright; otherwise the place +/// `make -C image rust-runtime` leaves it, which is where a developer who ran +/// the documented command will have it. +fn artifact() -> Option { + if let Some(named) = std::env::var_os("THALYX_RUST_RUNTIME") { + let path = PathBuf::from(named); + return path.is_dir().then_some(path); + } + let built = Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../../image/build/rust-runtime") + .canonicalize() + .ok()?; + std::fs::read_dir(built) + .ok()? + .flatten() + .map(|entry| entry.path()) + .find(|path| thalyx_rust::runtime::read(path).is_some()) +} + +/// The artifact, or a skip that says so in the words rule 3 requires. +fn artifact_or_skip(claim: &str) -> Option { + match artifact() { + Some(path) => Some(path), + None => { + let demanded = std::env::var("THALYX_REQUIRE_RUST_RUNTIME").as_deref() == Ok("1"); + assert!( + !demanded, + "THALYX_REQUIRE_RUST_RUNTIME=1 and there is no artifact to test: {claim}. \ + Build one with `make -C image rust-runtime`, or name one with \ + THALYX_RUST_RUNTIME." + ); + println!( + "NOT PROVEN: {claim} — no Rust runtime artifact on this machine. \ + `make -C image rust-runtime` builds one; THALYX_REQUIRE_RUST_RUNTIME=1 \ + makes this a failure instead of a skip." + ); + None + } + } +} + +/// Run a program of the artifact through the artifact's own loader. +/// +/// The environment is emptied first, and that is the whole test rather than +/// tidiness: an inherited `LD_LIBRARY_PATH`, `RUSTUP_HOME` or `PATH` is a way +/// for the host to help, and the claim is that it cannot. `HOME` is named at a +/// directory the caller controls because a Cargo with no home writes into the +/// one it finds. +fn through_its_own_loader(artifact: &Path, program: &str, arguments: &[&str]) -> Command { + let mut command = Command::new(artifact.join("lib/libc.so")); + command.arg(artifact.join(program)); + command.args(arguments); + command.env_clear(); + command +} + +#[test] +fn an_artifact_carries_the_loader_and_every_library_its_own_programs_name() { + let Some(artifact) = artifact_or_skip("the artifact is closed") else { + return; + }; + let report = thalyx_rust::runtime::inspect(&artifact); + assert!( + report.missing.is_empty(), + "the artifact is missing {:?}", + report.missing + ); + assert!( + report.forbidden.is_empty(), + "a whole toolchain was copied rather than a runtime assembled: {:?}", + report.forbidden + ); + assert!( + report.other_targets.is_empty(), + "it carries a standard library for a machine this is not: {:?}", + report.other_targets + ); + + let closure = thalyx_rust::runtime::closure(&artifact); + assert!( + closure.unresolved.is_empty(), + "these would resolve against whatever the host happens to have: {:?}", + closure.unresolved + ); + assert!( + closure.interpreter_inside, + "the artifact does not carry the loader its programs ask the kernel for: {:?}", + closure.interpreters + ); + assert!( + closure.programs.len() >= 3, + "an artifact with fewer than three programs is not a toolchain: {:?}", + closure.programs + ); +} + +#[test] +fn nothing_in_the_artifact_points_at_the_machine_that_built_it() { + // The shape of the failure: a staging step that "worked" because it left + // symlinks into `~/.rustup`. Every check passes on the machine that ran + // it, and the store is useless the moment it is carried anywhere else — + // which is the entire property this artifact exists to have. + let Some(artifact) = artifact_or_skip("the artifact is self-contained") else { + return; + }; + + fn walk(path: &Path, artifact: &Path, escaping: &mut Vec) { + let Ok(entries) = std::fs::read_dir(path) else { + return; + }; + for entry in entries.flatten() { + let here = entry.path(); + match entry.file_type() { + Ok(kind) if kind.is_symlink() => { + let Ok(target) = std::fs::read_link(&here) else { + continue; + }; + let resolved = if target.is_absolute() { + target.clone() + } else { + here.parent().unwrap_or(artifact).join(&target) + }; + // Canonicalised, because `../..` inside the artifact is + // still inside the artifact and a string comparison would + // call it an escape. + let inside = resolved + .canonicalize() + .map(|path| path.starts_with(artifact)) + .unwrap_or(false); + if !inside { + escaping.push(format!("{} → {}", here.display(), target.display())); + } + } + Ok(kind) if kind.is_dir() => walk(&here, artifact, escaping), + _ => {} + } + } + } + + let canonical = artifact.canonicalize().expect("the artifact's real path"); + let mut escaping = Vec::new(); + walk(&canonical, &canonical, &mut escaping); + assert!( + escaping.is_empty(), + "the artifact links out of itself, so it is not a thing that can be carried: {escaping:?}" + ); + + // And what it says about itself names no home either. A `runtime.json` + // carrying `/home/` would mean the description was written from + // the builder's paths rather than from the pins. + let described = std::fs::read_to_string(canonical.join("runtime.json")).expect("runtime.json"); + for shape in ["/home/", "/root/", ".rustup", ".cargo"] { + assert!( + !described.contains(shape), + "runtime.json mentions {shape}, so it was written from the builder's machine:\n{described}" + ); + } +} + +#[test] +fn the_staged_cargo_and_rust_analyzer_run_with_nothing_from_the_host() { + let Some(artifact) = artifact_or_skip("the staged programs run") else { + return; + }; + for (program, expected) in [ + ("bin/cargo", "cargo "), + ("bin/rustc", "rustc "), + ("bin/rust-analyzer", "rust-analyzer "), + ] { + let output = through_its_own_loader(&artifact, program, &["--version"]) + .output() + .unwrap_or_else(|error| panic!("{program} could not be started at all: {error}")); + let said = String::from_utf8_lossy(&output.stdout); + assert!( + output.status.success(), + "{program} did not answer --version with an empty environment: {}", + String::from_utf8_lossy(&output.stderr) + ); + assert!( + said.starts_with(expected), + "{program} answered {said:?}, which is not a version" + ); + } +} + +#[test] +fn the_staged_cargo_can_read_a_workspace_it_has_never_seen() { + // A version string proves a program starts. This proves it *works*: the + // question a run about a Rust tree has is not whether cargo exists but + // whether cargo can describe this tree, and `--no-deps` resolves nothing, + // so it writes no lockfile and needs no registry and no network. + let Some(artifact) = artifact_or_skip("the staged cargo reads a workspace") else { + return; + }; + let tree = tempfile::tempdir().expect("a temp workspace"); + std::fs::create_dir_all(tree.path().join("src")).expect("src"); + std::fs::write( + tree.path().join("Cargo.toml"), + b"[package]\nname = \"beacon\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[dependencies]\n", + ) + .expect("the manifest"); + std::fs::write( + tree.path().join("src/lib.rs"), + b"pub struct LanternRegistry { pub lit: u32 }\n", + ) + .expect("the source"); + + let manifest = tree.path().join("Cargo.toml"); + let output = through_its_own_loader( + &artifact, + "bin/cargo", + &[ + "metadata", + "--no-deps", + "--offline", + "--format-version", + "1", + "--manifest-path", + &manifest.display().to_string(), + ], + ) + .env("HOME", tree.path()) + .env("CARGO_HOME", tree.path().join("cargo-home")) + .env("CARGO_NET_OFFLINE", "true") + .output() + .expect("starting the staged cargo"); + + let said = String::from_utf8_lossy(&output.stdout); + assert!( + output.status.success(), + "the staged cargo could not read a workspace with nothing from the host:\n{}", + String::from_utf8_lossy(&output.stderr) + ); + assert!( + said.contains("\"name\":\"beacon\""), + "cargo answered something that is not this workspace: {}", + &said[..said.len().min(400)] + ); +} diff --git a/dev/bench-external-agent.sh b/dev/bench-external-agent.sh index ff1e5d5..77b3973 100755 --- a/dev/bench-external-agent.sh +++ b/dev/bench-external-agent.sh @@ -457,10 +457,26 @@ parity_gate() { say "the two arms were staged from the same tree" } +# Whether this run is about Rust, decided from the tree rather than from a flag. +# +# The task's own workspace answers it: a `Cargo.toml` at the root means the +# agent will be asked to resolve Rust names, which means a machine with no +# compiler cannot do the task however alive it is. Derived rather than typed, +# because a flag somebody has to remember is a flag that is missing on the run +# that needed it — which is 2026-08-30 exactly. +needs_rust() { + [ -f "$PROJECT/Cargo.toml" ] +} + preflight_b() { local report="$OUT/preflight-b.json" + local rust="" + if needs_rust; then rust="--needs-rust"; fi rm -f "$report" "$OUT/preflight-b.verdict.json" say "arm B: asking the machine whether it is there (this costs nothing)" + if [ -n "$rust" ]; then + say "arm B: and whether it can resolve a Rust name, which this task needs" + fi if [ -n "${THALYX_BENCH_PREFLIGHT_CMD:-}" ]; then # shellcheck disable=SC2086 $THALYX_BENCH_PREFLIGHT_CMD > "$report" 2> "$OUT/preflight-b.err" || true @@ -471,7 +487,8 @@ preflight_b() { printf '{"ready":false,"because":["there is no socket at %s — is the machine up? `make -C image run-agent`"]}\n' \ "$SOCKET" > "$report" else - "$MCP" --connect "$SOCKET" --preflight \ + # shellcheck disable=SC2086 + "$MCP" --connect "$SOCKET" --preflight $rust \ --wait "${THALYX_BENCH_PREFLIGHT_WAIT:-20}" \ > "$report" 2> "$OUT/preflight-b.err" || true fi @@ -481,8 +498,11 @@ preflight_b() { # before the block below could say why. The fifth entry in # `Estrategia-de-Pruebas.md`: the instrument includes the harness, and the # self-test that caught this was written before the code it caught. + local require="" + if needs_rust; then require="--require-rust"; fi + # shellcheck disable=SC2086 python3 "$ROOT/dev/bench-summary.py" --preflight-verdict "$report" \ - --project "$PROJECT" > "$OUT/preflight-b.verdict.json" 2>&1 || true + --project "$PROJECT" $require > "$OUT/preflight-b.verdict.json" 2>&1 || true } # ── the three prompts, which are one prompt ────────────────────────────────── diff --git a/dev/bench-summary.py b/dev/bench-summary.py index 5355e7a..4b13cb3 100755 --- a/dev/bench-summary.py +++ b/dev/bench-summary.py @@ -1319,7 +1319,7 @@ def project_top_level(root): ) -def preflight_verdict(report, project=None): +def preflight_verdict(report, project=None, require_rust=False): """Whether arm B is ready to be paid for, decided outside the probe. The probe (`thalyx-mcp --preflight`) talks to the machine; this decides what @@ -1360,6 +1360,42 @@ def preflight_verdict(report, project=None): else: verdict["top_level_matches"] = True + # ── the capability the task needs, not just a machine that answers ── + # + # The 2026-08-30 run: READY, paid for, and then `there is no cargo on this + # machine` from inside. Aliveness and the right tree were both true and + # neither was the thing a Rust task needs. Decided here rather than in the + # probe for the same reason the rest of this function is here: it can then + # be tested against a machine with a compiler, one without, and one too old + # to be asked, all three for free and none of them needing a VM. + if require_rust: + toolchain = report.get("toolchain") + verdict["toolchain"] = toolchain + if not isinstance(toolchain, dict): + verdict["because"].append( + "this run needs Rust semantics and the machine said nothing about a " + "toolchain. Either it is a Thalyx from before the managed runtime, or " + "--needs-rust was not passed to the probe" + ) + elif toolchain.get("semantic_ready") is not True: + said = toolchain.get("because") or ["the machine did not say why"] + verdict["because"].append( + "this run needs Rust semantics and the machine cannot resolve names: " + + "; ".join(said if isinstance(said, list) else [str(said)]) + ) + else: + # Written down because it is the fact a later reader will want: + # which compiler produced the run's answers, and whose it was. + cargo = toolchain.get("cargo") or {} + analyzer = toolchain.get("rust_analyzer") or {} + verdict["rust"] = { + "cargo": cargo.get("version"), + "cargo_from": cargo.get("from"), + "rust_analyzer": analyzer.get("version"), + "rust_analyzer_from": analyzer.get("from"), + "runtime": (toolchain.get("runtime") or {}).get("identity"), + } + verdict["ready"] = not verdict["because"] return verdict @@ -2839,6 +2875,49 @@ def channel(name, calls, cwd=infrastructure): ok("a machine that answered and is holding this project is READY", alive["ready"], True) + # ── the capability, which is what the 2026-08-30 run was missing ── + healthy = { + "ready": True, "thalyx": "0.1.0", "workspace": "/workspace", + "tools_offered": 11, "top_level": ["Cargo.toml", "crates", "target"], + } + no_compiler = dict(healthy, toolchain={ + "semantic_ready": False, + "because": ["there is no cargo that runs at any of the 4 places"], + "cargo": {"path": None, "from": None}, + "rust_analyzer": {"path": None, "from": None}, + }) + ok("a machine with no cargo is not READY for a Rust run", + preflight_verdict(no_compiler, workspace, require_rust=True)["ready"], False) + ok("...and the same machine is still READY for a run that is not about Rust", + preflight_verdict(no_compiler, workspace)["ready"], True) + + no_analyzer = dict(healthy, toolchain={ + "semantic_ready": False, + "because": ["there is no rust-analyzer that runs at any of the 4 places"], + "cargo": {"path": "/opt/thalyx/toolchains/rust/x/bin/cargo", + "from": "thalyx", "version": "cargo 1.90.0"}, + "rust_analyzer": {"path": None, "from": None}, + }) + ok("a machine with cargo and no rust-analyzer is not READY for a Rust run", + preflight_verdict(no_analyzer, workspace, require_rust=True)["ready"], False) + + ok("a machine too old to be asked is not READY for a Rust run either", + preflight_verdict(healthy, workspace, require_rust=True)["ready"], False) + + both = dict(healthy, toolchain={ + "semantic_ready": True, "because": [], + "cargo": {"path": "/opt/thalyx/toolchains/rust/x/bin/cargo", + "from": "thalyx", "version": "cargo 1.90.0"}, + "rust_analyzer": {"path": "/opt/thalyx/toolchains/rust/x/bin/rust-analyzer", + "from": "thalyx", "version": "rust-analyzer 1.90.0"}, + "runtime": {"identity": "rust-1.90.0-x86_64-unknown-linux-musl"}, + }) + settled = preflight_verdict(both, workspace, require_rust=True) + ok("a machine whose own runtime answered is READY for a Rust run", + settled["ready"], True) + ok("...and the verdict records whose compiler it was", + settled.get("rust", {}).get("cargo_from"), "thalyx") + # ── 5. the two arms were given the same thing ── same = { "source_commit": "abc123", "exclusions": list(OUTSIDE_THE_WORKSPACE), @@ -3861,6 +3940,11 @@ def main(): ) parser.add_argument("--project", type=pathlib.Path, help="the tree --preflight-verdict compares the machine against") + parser.add_argument( + "--require-rust", action="store_true", + help="with --preflight-verdict: a machine that cannot resolve Rust names is " + "not READY, however alive it is", + ) parser.add_argument( "--import-stamp", type=pathlib.Path, metavar="DIR", help="print, as JSON, what a tree is at the moment it is imported: where it " @@ -4099,7 +4183,7 @@ def main(): except (OSError, json.JSONDecodeError) as why: report = None print(f" the preflight probe left nothing readable: {why}", file=sys.stderr) - verdict = preflight_verdict(report, given.project) + verdict = preflight_verdict(report, given.project, given.require_rust) print(json.dumps(verdict, indent=2)) sys.exit(0 if verdict["ready"] else 1) diff --git a/dev/build-rust-runtime.sh b/dev/build-rust-runtime.sh new file mode 100755 index 0000000..b5be717 --- /dev/null +++ b/dev/build-rust-runtime.sh @@ -0,0 +1,315 @@ +#!/usr/bin/env bash +# +# The Rust runtime the agent inside Thalyx programs with — built, not borrowed. +# +# dev/build-rust-runtime.sh [output-directory] +# +# Leaves a finished artifact at // and prints the path on the +# last line. `make -C image store-stage RUST=1` copies that onto the store; see +# `image/Makefile`. +# +# ───────────────────────────────────────────────────────────────────────────── +# WHY THIS EXISTS +# +# On 2026-08-30 a paid benchmark run watched Claude pick exactly the right +# primitive inside Thalyx — +# +# const def = thalyx.context('…'); +# const r1 = thalyx.rename('…', '…'); +# +# — and get back `source: index`, `analyzer_starts: 0`, and +# +# rename: { ok: false, error: unresolved, +# message: "there is no `cargo` on this machine" } +# +# The machine had been told it could resolve names and it could not. Everything +# after that in the transcript is a consequence. +# +# The first fix attempted was to copy the host's `~/.rustup` into the store. +# Cesar stopped it, and he was right: `Filosofia-Fundacional.md` says Thalyx is +# the whole system, and a Thalyx whose programming face only works because +# Fedora happens to have rustup installed is a Thalyx that borrows its most +# important capability from the machine it claims to replace. Move the disk to +# another x86_64 box and the semantic provider would vanish. +# +# So: **the host provides nothing at agent runtime.** Not cargo, not rustc, not +# rust-analyzer, not the standard library, not the dynamic loader. This script +# builds all of it from artifacts that are named, versioned and digest-checked, +# and after it has run the host is out of the picture. +# +# ───────────────────────────────────────────────────────────────────────────── +# WHY THE musl TOOLCHAIN AND NOT THE ORDINARY ONE +# +# Measured on 2026-08-31, both ways, before anything was written: +# +# x86_64-unknown-linux-gnu needs glibc's loader, libc, libm, libdl, librt, +# libpthread, libgcc_s and libz from the host, and +# carries a separate 191 MB libLLVM.so. +# +# x86_64-unknown-linux-musl needs exactly two files that Rust does not ship: +# musl's loader and libgcc_s.so.1. LLVM is linked +# inside librustc_driver, so there is no libLLVM. +# +# Rust publishes host tools for `x86_64-unknown-linux-musl` officially, with a +# sha256 for every file in its own channel manifest. Two missing files is a +# problem a person can close; most of a distribution is not. That is the whole +# reason for the choice — see `vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md`. +# +# ───────────────────────────────────────────────────────────────────────────── +# WHERE THE TWO MISSING FILES COME FROM, AND WHY NEITHER IS COPIED +# +# **The loader** is musl's `libc.so`, compiled here from musl's own release +# tarball with the digest below. The same arrangement the Linux kernel already +# has in `image/Makefile`: a pinned tarball, checked before it is believed, +# built by us. It is about a megabyte and it builds in seconds. +# +# **libgcc_s.so.1** is linked here out of `libunwind.a`, which Rust ships inside +# `rust-std`'s `self-contained/` directory — so it comes from the same +# digest-checked artifact as the compiler and introduces nothing new. +# +# That it is enough is a measurement, not a hope. Of the 883 undefined symbols +# across cargo, rustc, rust-analyzer, the proc-macro server and +# librustc_driver, everything resolves against musl and librustc_driver except +# 29, and of those 29 the only ones that are not weak are the fifteen +# `_Unwind_*` — every one of which `libunwind.a` defines. The rest are the +# transactional-memory stubs, `__register_frame_info`, and the two `pidfd_*` +# functions Rust's std weak-links for newer musl. +# +# It was then run rather than argued: a chroot holding this artifact, `/proc` +# and three device nodes — no shell, no /usr, no /lib64, nothing else — +# answered `cargo --version`, `cargo metadata`, `rustc --emit=metadata`, and a +# full rust-analyzer session that resolved a definition and returned four +# rename edits. +# +# ───────────────────────────────────────────────────────────────────────────── +# WHAT IS DELIBERATELY NOT IN IT +# +# share/, man pages, docs never runs, and it is 13 MB of the rustc +# component alone +# lib/rustlib//bin/ rust-lld, wasm-component-ld, rust-objcopy — +# 174 MB of *linkers*. The semantic provider +# never links: `cargo metadata` and +# rust-analyzer's analysis do not. Leaving them +# out is also the honest thing, because +# `rust-lld` wants libgcc's integer builtins, +# which the unwinder-only libgcc_s above does +# not have — shipping a linker that cannot +# start is worse than shipping none. +# every other target one host, one target +# rustdoc, rustfmt, clippy not what a semantic provider is for +# +# So this artifact **resolves and renames**; it does not build. When something +# inside Thalyx needs to compile a proc-macro or a build script, that is the +# next known cause, and it gets its own change with its own evidence. +# +# ───────────────────────────────────────────────────────────────────────────── +# THE PINS +# +# Every URL below has a digest, and a digest that does not match stops the +# build. A tarball fetched over TLS proves who served the bytes, not what the +# bytes were — the same reasoning the kernel pin carries in `image/Makefile`. +# +# The Rust digests are not ours: they are the values in Rust's own channel +# manifest, `https://static.rust-lang.org/dist/channel-rust-1.90.0.toml`, which +# is how rustup itself decides whether a download is the file it asked for. +# +# dev/rust-runtime-pins.sh prints the commands that re-derive them. +# +# musl 1.2.4 and not 1.2.5 for one reason: 1.2.4 is the one that was built and +# run end to end under the toolchain above on 2026-08-31. Rule 12 — +# `vault/09-Notas-Tecnicas/Estrategia-de-Pruebas.md` — the thing that gets +# verified has to be the thing that ships. Bumping it is one line here and the +# same physical test again. + +set -euo pipefail + +RUST_VERSION="1.90.0" +RUST_DIST_DATE="2025-09-18" +TARGET="x86_64-unknown-linux-musl" +DIST="https://static.rust-lang.org/dist/$RUST_DIST_DATE" + +# component:filename:sha256 — the sha256 of the .tar.xz, from Rust's manifest. +COMPONENTS=( + "cargo:cargo-$RUST_VERSION-$TARGET.tar.xz:dddd1ee3da59440d5aa4d149ebb5fbbe0d7252dd94e171d5f2b071d7354f9b3a" + "rustc:rustc-$RUST_VERSION-$TARGET.tar.xz:993cb26cee9525b1553d82b8fc2b6ddffd50a0a561cd896e3daa3a9f8ae65949" + "rust-std:rust-std-$RUST_VERSION-$TARGET.tar.xz:38490d575786f4688e83b357baeb022d8dde0ace2cb8c1357e060c76644fc56a" + "rust-analyzer:rust-analyzer-$RUST_VERSION-$TARGET.tar.xz:cc5d529f84710b8f4439bd457d7cdc432f0dc616203a5d51af895d5ded8ed691" + "rust-src:rust-src-$RUST_VERSION.tar.xz:cde088d57064d151b2236f4619aea4a8207e0709eb3035ddc6617d609ab7d453" +) + +MUSL_VERSION="1.2.4" +MUSL_URL="https://musl.libc.org/releases/musl-$MUSL_VERSION.tar.gz" +MUSL_SHA256="7a35eae33d5372a7c0da1188de798726f68825513b7ae3ebe97aaaa52114f039" + +IDENTITY="rust-$RUST_VERSION-$TARGET" +# Where the artifact lands inside the machine. Spelled the same way in +# `crates/thalyx-rust/src/runtime.rs`, which is what discovery looks at: two +# places computing this is two answers to where the toolchain is. +INSIDE="/opt/thalyx/toolchains/rust/$IDENTITY" + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +OUT="${1:-${TMPDIR:-/tmp}/thalyx-rust-runtime}" +CACHE="${THALYX_RUST_DIST_CACHE:-$OUT/dist}" +WORK="$OUT/work" +ARTIFACT="$OUT/$IDENTITY" + +say() { printf ' %s\n' "$*" >&2; } +die() { printf '\n %s\n\n' "$*" >&2; exit 1; } + +for tool in curl tar sha256sum cc make ld; do + command -v "$tool" > /dev/null \ + || die "no $tool. Building the loader needs a C compiler and binutils; \ +the same ones image/Makefile already needs for the kernel." +done + +mkdir -p "$CACHE" "$WORK" + +# Fetch and verify. `THALYX_RUST_DIST_CACHE` lets a machine that already has the +# bytes skip the download — the digest is checked either way, so a cached file +# that is not the file is caught exactly like a bad download. +fetch() { + local url="$1" name="$2" want="$3" path="$CACHE/$2" + if [ ! -s "$path" ]; then + say "fetching $name" + curl -fsSL --retry 3 -o "$path.part" "$url" \ + || die "could not fetch $url" + mv "$path.part" "$path" + fi + local got + got="$(sha256sum "$path" | cut -d' ' -f1)" + [ "$got" = "$want" ] || die "$name is not the file it should be. + expected $want + got $got + Nothing was used. If the pin is out of date, re-derive it — dev/rust-runtime-pins.sh + prints how — rather than editing the digest to match whatever arrived." + say "verified $name" +} + +say "" +say "Rust runtime for the agent — $IDENTITY" +say "" + +for entry in "${COMPONENTS[@]}"; do + IFS=: read -r _component file digest <<< "$entry" + fetch "$DIST/$file" "$file" "$digest" +done +fetch "$MUSL_URL" "musl-$MUSL_VERSION.tar.gz" "$MUSL_SHA256" + +rm -rf "$ARTIFACT" "$WORK" +mkdir -p "$ARTIFACT/bin" "$ARTIFACT/lib/rustlib" "$ARTIFACT/libexec" "$WORK" + +unpack() { + local file="$1" + say "unpacking $file" + tar -C "$WORK" -xf "$CACHE/$file" +} +for entry in "${COMPONENTS[@]}"; do + IFS=: read -r _component file _digest <<< "$entry" + unpack "$file" +done + +C="$WORK/cargo-$RUST_VERSION-$TARGET/cargo" +R="$WORK/rustc-$RUST_VERSION-$TARGET/rustc" +S="$WORK/rust-std-$RUST_VERSION-$TARGET/rust-std-$TARGET" +A="$WORK/rust-analyzer-$RUST_VERSION-$TARGET/rust-analyzer-preview" +SRC="$WORK/rust-src-$RUST_VERSION/rust-src" + +# The selection, spelled as paths rather than as an exclusion list. An +# exclusion list is a claim about everything you did not think of; this is a +# claim about what is there. +cp "$C/bin/cargo" "$ARTIFACT/bin/cargo" +cp "$R/bin/rustc" "$ARTIFACT/bin/rustc" +cp "$A/bin/rust-analyzer" "$ARTIFACT/bin/rust-analyzer" +cp "$R/libexec/rust-analyzer-proc-macro-srv" "$ARTIFACT/libexec/" +cp "$R"/lib/librustc_driver-*.so "$ARTIFACT/lib/" +mkdir -p "$ARTIFACT/lib/rustlib/$TARGET" +cp -a "$S/lib/rustlib/$TARGET/lib" "$ARTIFACT/lib/rustlib/$TARGET/lib" +cp -a "$SRC/lib/rustlib/src" "$ARTIFACT/lib/rustlib/src" + +# ── the loader, compiled here ──────────────────────────────────────────────── +# +# `--prefix=/` and nothing installed: the one file wanted is `lib/libc.so`, +# which under musl *is* the dynamic linker as well as the C library. The +# binaries above ask the kernel for `/lib/ld-musl-x86_64.so.1`; PID 1 makes that +# name resolve to this file once the store is mounted — see +# `crates/thalyx-cli/src/store_disk.rs`. +say "building musl $MUSL_VERSION" +tar -C "$WORK" -xf "$CACHE/musl-$MUSL_VERSION.tar.gz" +( + cd "$WORK/musl-$MUSL_VERSION" + ./configure --prefix=/ --disable-static --enable-shared --disable-gcc-wrapper \ + > configure.log 2>&1 || { tail -20 configure.log >&2; die "musl configure failed"; } + make -j"$(nproc 2>/dev/null || echo 2)" > build.log 2>&1 \ + || { tail -30 build.log >&2; die "musl did not build"; } +) +cp "$WORK/musl-$MUSL_VERSION/lib/libc.so" "$ARTIFACT/lib/libc.so" +chmod 755 "$ARTIFACT/lib/libc.so" +# Both names, because two different things ask for them: the kernel reads +# PT_INTERP and wants `ld-musl-x86_64.so.1`, and the loader resolves the +# `libc.so` in every DT_NEEDED. Relative, so the artifact can be built at one +# path and mounted at another. +ln -sf libc.so "$ARTIFACT/lib/ld-musl-x86_64.so.1" + +# ── the unwinder, linked out of Rust's own ─────────────────────────────────── +say "linking libgcc_s.so.1 from the toolchain's own libunwind.a" +UNWIND="$ARTIFACT/lib/rustlib/$TARGET/lib/self-contained/libunwind.a" +[ -f "$UNWIND" ] || die "no libunwind.a in rust-std's self-contained directory" +ld -shared -o "$ARTIFACT/lib/libgcc_s.so.1" \ + -soname libgcc_s.so.1 --eh-frame-hdr -z noexecstack \ + --whole-archive "$UNWIND" --no-whole-archive \ + || die "libgcc_s.so.1 did not link" + +# ── does it run? ───────────────────────────────────────────────────────────── +# +# Asked, never assumed, and asked **through the artifact's own loader** — +# `lib/libc.so ` is how musl's ldso is invoked directly. That is what +# makes this checkable on a host that has no musl at all, which is every host +# this will ever be built on. A `-f` on the file would prove the copy worked; +# this proves the closure closed. +runs() { + "$ARTIFACT/lib/libc.so" "$1" --version > /dev/null 2>&1 +} +for program in bin/cargo bin/rustc bin/rust-analyzer; do + runs "$ARTIFACT/$program" \ + || die "$program does not run under this artifact's own loader. The runtime is + incomplete, and staging it would put a machine on the store that says it can + resolve names and cannot. Nothing was staged." +done +say "cargo, rustc and rust-analyzer all answered --version through lib/libc.so" + +# ── what it is, written down beside it ─────────────────────────────────────── +# +# Read by `thalyx-rust`'s discovery, and by the preflight, so that a machine can +# say *which* toolchain it is using rather than that it has one. +{ + printf '{\n' + printf ' "identity": "%s",\n' "$IDENTITY" + printf ' "rust": "%s",\n' "$RUST_VERSION" + printf ' "target": "%s",\n' "$TARGET" + printf ' "dist": "%s",\n' "$DIST" + printf ' "musl": "%s",\n' "$MUSL_VERSION" + printf ' "inside": "%s",\n' "$INSIDE" + printf ' "built": "%s",\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" + printf ' "components": {\n' + last=$((${#COMPONENTS[@]} - 1)); i=0 + for entry in "${COMPONENTS[@]}"; do + IFS=: read -r component file digest <<< "$entry" + printf ' "%s": {"file": "%s", "sha256": "%s"}' "$component" "$file" "$digest" + [ "$i" -lt "$last" ] && printf ',' + printf '\n'; i=$((i + 1)) + done + printf ' },\n' + printf ' "musl_sha256": "%s"\n' "$MUSL_SHA256" + printf '}\n' +} > "$ARTIFACT/runtime.json" + +say "" +say " from: $DIST (Rust $RUST_VERSION, $TARGET)" +say " $MUSL_URL" +say " size: $(du -sh "$ARTIFACT" | cut -f1)" +say " bin $(du -sh "$ARTIFACT/bin" | cut -f1)" +say " lib $(du -sh "$ARTIFACT/lib" | cut -f1)" +say " libexec $(du -sh "$ARTIFACT/libexec" | cut -f1)" +say " inside: $INSIDE" +say "" +echo "$ARTIFACT" diff --git a/dev/rust-corpus/Cargo.toml b/dev/rust-corpus/Cargo.toml new file mode 100644 index 0000000..ea0cceb --- /dev/null +++ b/dev/rust-corpus/Cargo.toml @@ -0,0 +1,14 @@ +# A synthetic Rust workspace, small on purpose. +# +# It exists so the machine's semantic provider can be exercised without the +# benchmark's corpus and without its symbols: `vault/09-Notas-Tecnicas/ +# Runtime-Rust-Agente.md` is about whether Thalyx carries a compiler, and a +# check written over the benchmark's own tree would be measuring two things at +# once. +# +# No dependencies at all, and that is deliberate: the semantic provider has no +# network by construction, so a workspace with registry dependencies would be +# testing whether Cargo can fetch rather than whether the toolchain works. +[workspace] +resolver = "2" +members = ["lantern", "harbour"] diff --git a/dev/rust-corpus/harbour/Cargo.toml b/dev/rust-corpus/harbour/Cargo.toml new file mode 100644 index 0000000..3127d85 --- /dev/null +++ b/dev/rust-corpus/harbour/Cargo.toml @@ -0,0 +1,7 @@ +[package] +name = "harbour" +version = "0.1.0" +edition = "2021" + +[dependencies] +lantern = { path = "../lantern" } diff --git a/dev/rust-corpus/harbour/src/lib.rs b/dev/rust-corpus/harbour/src/lib.rs new file mode 100644 index 0000000..8d0099d --- /dev/null +++ b/dev/rust-corpus/harbour/src/lib.rs @@ -0,0 +1,19 @@ +//! A second crate that uses the name, so a rename has to cross a file. +//! +//! Two crates and not one file with two mentions: `edits_by_file` is a claim +//! about *per file* counts, and a fixture that only ever had one file could +//! not tell that claim from a total. + +use lantern::LanternRegistry; + +pub fn open() -> LanternRegistry { + LanternRegistry::new() +} + +pub fn count(registry: &LanternRegistry) -> u32 { + registry.lit() +} + +/// An alias, because a rename that only rewrote the plain mentions would pass +/// a test that had none. +pub type Registry = LanternRegistry; diff --git a/dev/rust-corpus/lantern/Cargo.toml b/dev/rust-corpus/lantern/Cargo.toml new file mode 100644 index 0000000..83a37e9 --- /dev/null +++ b/dev/rust-corpus/lantern/Cargo.toml @@ -0,0 +1,4 @@ +[package] +name = "lantern" +version = "0.1.0" +edition = "2021" diff --git a/dev/rust-corpus/lantern/src/lib.rs b/dev/rust-corpus/lantern/src/lib.rs new file mode 100644 index 0000000..7f4a953 --- /dev/null +++ b/dev/rust-corpus/lantern/src/lib.rs @@ -0,0 +1,30 @@ +//! Where the name under test is declared. + +/// The symbol a rename is asked about. +/// +/// Named nothing like anything in the benchmark's corpus on purpose: a check +/// that shares a symbol with the thing it is meant to be independent of is not +/// independent of it. +pub struct LanternRegistry { + lit: u32, +} + +impl LanternRegistry { + pub fn new() -> Self { + LanternRegistry { lit: 0 } + } + + pub fn light(&mut self) { + self.lit += 1; + } + + pub fn lit(&self) -> u32 { + self.lit + } +} + +impl Default for LanternRegistry { + fn default() -> Self { + Self::new() + } +} diff --git a/dev/verify-agent-rust.sh b/dev/verify-agent-rust.sh new file mode 100755 index 0000000..8aa0ad3 --- /dev/null +++ b/dev/verify-agent-rust.sh @@ -0,0 +1,287 @@ +#!/usr/bin/env bash +# Ask a running Thalyx machine, over the same socket Claude Code would use, +# whether it can really resolve a Rust name. +# +# dev/verify-agent-rust.sh [socket] +# +# ───────────────────────────────────────────────────────────────────────────── +# WHAT THIS IS FOR +# +# `vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md`. On 2026-08-30 a paid run +# was answered `there is no cargo on this machine` by a machine that had said +# READY. The runtime that closes that is built, staged and checked by code with +# tests — and none of that is the claim. The claim is that a **booted Thalyx**, +# asked through its agent channel, resolves a symbol with rust-analyzer and +# renames it. +# +# No shell in a container can make that claim. This is the thing somebody runs +# on the machine that can. +# +# ## What it deliberately does not do +# +# It does not invoke Claude Code, it does not run the A/B benchmark, and it +# costs nothing. It also leaves the workspace exactly as it found it: the rename +# runs inside a program with `on_success: "rollback"`, so the work really +# happens, `edits_by_file` is really counted, and the tree is put back because +# the caller asked rather than because anything failed. +# +# ## And it does not believe the machine's summary +# +# Every check reads the field it is about out of the machine's own answer — +# `source`, `analyzer_starts`, `edits_by_file`, `tree` — rather than looking for +# a word in a sentence. Rule 10 of `Estrategia-de-Pruebas.md` has been paid for +# twice by greps that kept passing after the sentence changed. + +set -uo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +SOCKET="${1:-$ROOT/image/build/agent.sock}" +MCP="$ROOT/target/release/thalyx-mcp" +SYMBOL="${THALYX_RUST_SYMBOL:-LanternRegistry}" +RENAMED="${THALYX_RUST_RENAMED:-BeaconRegistry}" + +PROVEN=0 +UNPROVEN=0 +FAILED=0 +proven() { printf ' \033[32mPROVEN\033[0m %s\n' "$*"; PROVEN=$((PROVEN + 1)); } +unproven() { printf ' \033[33mNOT PROVEN\033[0m %s\n' "$*"; UNPROVEN=$((UNPROVEN + 1)); } +failed() { printf ' \033[31mFAILED\033[0m %s\n' "$*"; FAILED=$((FAILED + 1)); } +say() { printf ' %s\n' "$*"; } + +echo +echo " Can the machine behind $SOCKET resolve a Rust name?" +echo + +if [ ! -S "$SOCKET" ]; then + failed "there is no agent channel at $SOCKET — the machine is not running with one" + say + say " make -C image agent PROJECT=$ROOT/dev/rust-corpus" + say + exit 1 +fi + +if [ ! -x "$MCP" ]; then + say "building thalyx-mcp for this host" + ( cd "$ROOT" && cargo build --release -p thalyx-mcp ) > /dev/null 2>&1 \ + || { failed "thalyx-mcp did not build"; exit 1; } +fi + +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +# ── 1. the preflight, which is the thing that failed to fail on 2026-08-30 ── +"$MCP" --connect "$SOCKET" --preflight --needs-rust --wait 30 \ + > "$WORK/preflight.json" 2> "$WORK/preflight.err" +PREFLIGHT_STATUS=$? + +python3 - "$WORK/preflight.json" "$PREFLIGHT_STATUS" > "$WORK/preflight.lines" <<'PY' +import json, sys +try: + report = json.load(open(sys.argv[1])) +except Exception as error: + print(f" \033[31mFAILED\033[0m the preflight printed nothing readable: {error}") + sys.exit(1) +ready = report.get("ready") is True +tool = report.get("toolchain") or {} +cargo = tool.get("cargo") or {} +analyzer = tool.get("rust_analyzer") or {} +runtime = tool.get("runtime") or {} +mark = "\033[32mPROVEN\033[0m" if ready else "\033[31mFAILED\033[0m" +print(f" {mark} the machine says it is ready for a Rust task" + if ready else + f" {mark} the machine is NOT ready: " + "; ".join(report.get("because") or [])) +for name, tool_report in (("cargo", cargo), ("rust-analyzer", analyzer)): + where = tool_report.get("from") + version = tool_report.get("version") + if tool_report.get("path"): + owner = {"thalyx": "Thalyx's own runtime", + "host": "a toolchain installed on the host", + "named": "a file a variable named"}.get(where, where) + print(f" \033[32mPROVEN\033[0m {name} ran inside the machine: {version} — {owner}") + print(f" {tool_report['path']}") + else: + print(f" \033[31mFAILED\033[0m {name} is not on this machine at all") +if runtime.get("identity"): + print(f" runtime {runtime['identity']}" + f" (Rust {runtime.get('rust')}, musl {runtime.get('musl')})") +sys.exit(0 if ready else 1) +PY +READY=$? +cat "$WORK/preflight.lines" +# Counted from the lines that were printed, never from how many blocks ran: a +# summary that assumed its own arithmetic is a summary that stops matching the +# checks the first time one is added. +tally() { + PROVEN=$((PROVEN + $(grep -c 'PROVEN' "$1"))) + FAILED=$((FAILED + $(grep -c 'FAILED' "$1"))) + UNPROVEN=$((UNPROVEN + $(grep -c 'NOT PROVEN' "$1"))) + # `NOT PROVEN` contains `PROVEN`, so it was counted twice above; take it + # back once. Found by a summary that reported four proven checks on a run + # with two. + PROVEN=$((PROVEN - $(grep -c 'NOT PROVEN' "$1"))) +} +tally "$WORK/preflight.lines" + +if [ "$READY" -ne 0 ]; then + say + say "Nothing further was asked: a machine that cannot resolve names has" + say "nothing to be asked about. The preflight's own words:" + sed 's/^/ /' "$WORK/preflight.json" | head -40 + [ -s "$WORK/preflight.err" ] && sed 's/^/ /' "$WORK/preflight.err" + exit 1 +fi + +# ── 2 and 3. a real question and a real rename, over the real channel ── +# +# Driven as MCP, because that is the surface Claude Code uses and a check that +# went in some other way would be checking some other thing. +python3 - "$MCP" "$SOCKET" "$SYMBOL" "$RENAMED" <<'PY' > "$WORK/semantic.txt" 2>&1 +import json, subprocess, sys, time + +mcp, socket, symbol, renamed = sys.argv[1:5] +server = subprocess.Popen( + [mcp, "--connect", socket, "--wait", "30"], + stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, + text=True, +) + +def call(method, params, request_id): + server.stdin.write(json.dumps( + {"jsonrpc": "2.0", "id": request_id, "method": method, "params": params}) + "\n") + server.stdin.flush() + deadline = time.time() + 600 + while time.time() < deadline: + line = server.stdout.readline() + if not line: + raise SystemExit("the adapter stopped answering") + try: + message = json.loads(line) + except ValueError: + continue + if message.get("id") == request_id: + return message + raise SystemExit(f"{method} did not answer in ten minutes") + +def tool(name, arguments, request_id): + reply = call("tools/call", {"name": name, "arguments": arguments}, request_id) + text = ((reply.get("result") or {}).get("content") or [{}])[0].get("text", "") + try: + return json.loads(text) + except ValueError: + return {"unparsed": text} + +call("initialize", {"protocolVersion": "2025-06-18", "capabilities": {}, + "clientInfo": {"name": "verify-agent-rust", "version": "1"}}, 1) +server.stdin.write(json.dumps({"jsonrpc": "2.0", "method": "notifications/initialized"}) + "\n") +server.stdin.flush() + +# The first question. Starting rust-analyzer on a cold workspace takes tens of +# seconds; the timeout above is generous for that reason and not because +# anything here is slow. +context = tool("thalyx_context", {"query": symbol}, 2) +print("CONTEXT " + json.dumps(context)) + +# The rename, inside a program that puts the tree back **because it was asked +# to**, not because anything failed. So the work really happens, the counts are +# really counted, and the machine is left exactly as it was found. +program = ( + f"const before = thalyx.context({symbol!r});\n" + f"const done = thalyx.rename({symbol!r}, {renamed!r});\n" + "return {source: before.source, ok: done.ok, error: done.error,\n" + " definition: done.definition ?? null,\n" + " edits_by_file: done.edits_by_file ?? null};\n" +) +outcome = tool("thalyx_exec", {"label": "verify the machine's own Rust", + "run": program, "on_success": "rollback"}, 3) +print("EXEC " + json.dumps(outcome)) +server.kill() +PY + +CONTEXT_LINE=$(grep '^CONTEXT ' "$WORK/semantic.txt" | head -1 | cut -d' ' -f2-) +EXEC_LINE=$(grep '^EXEC ' "$WORK/semantic.txt" | head -1 | cut -d' ' -f2-) + +if [ -z "$CONTEXT_LINE" ]; then + failed "the machine never answered a context question; see below" + sed 's/^/ /' "$WORK/semantic.txt" | head -30 +else + printf '%s' "$CONTEXT_LINE" > "$WORK/context.json" + SOURCE=$(python3 -c 'import json,sys;print((json.load(open(sys.argv[1])) or {}).get("source",""))' "$WORK/context.json") + STARTS=$(python3 -c 'import json,sys;print((json.load(open(sys.argv[1])) or {}).get("analyzer_starts",""))' "$WORK/context.json") + CONFINED=$(python3 -c 'import json,sys;print((json.load(open(sys.argv[1])) or {}).get("analyzer_confined",""))' "$WORK/context.json") + if [ "$SOURCE" = "rust-analyzer" ]; then + proven "context('$SYMBOL') came from rust-analyzer, not from the scan (source=rust-analyzer, analyzer_starts=$STARTS)" + else + failed "context('$SYMBOL') answered source=$SOURCE — the machine matched a name instead of resolving one" + sed 's/^/ /' "$WORK/context.json" + fi + case "$CONFINED" in + True|true) proven "and the provider that answered was confined by Thalyx" ;; + False|false) unproven "the provider ran as an ordinary process on this machine — load the LSM (make -C lsm load) to close that half" ;; + *) unproven "the answer did not say whether the provider was confined" ;; + esac +fi + +if [ -z "$EXEC_LINE" ]; then + failed "the machine never ran the rename program; see below" + sed 's/^/ /' "$WORK/semantic.txt" | head -40 +else + printf '%s' "$EXEC_LINE" > "$WORK/exec.json" + python3 - "$WORK/exec.json" "$SYMBOL" > "$WORK/exec.lines" <<'PY' +import json, sys +answer = json.load(open(sys.argv[1])) +symbol = sys.argv[2] +# `returned`, which is what the program handed back — the field name is +# asserted in `exec.rs`'s own tests, so a rename there breaks this loudly +# instead of turning every check below into a silent "not present". +value = answer.get("returned") or {} +def proven(text): print(f" \033[32mPROVEN\033[0m {text}") +def failed(text): print(f" \033[31mFAILED\033[0m {text}") +if value.get("ok") is True: + proven(f"rename('{symbol}', …) resolved and rewrote every place that refers to it") +else: + failed(f"the rename did not work: {value.get('error')} — {json.dumps(value)[:300]}") +# A list of `{path, edits}`, which is the shape `semantic.rs` builds. Counted +# per file rather than totalled, because the whole reason the field exists is +# that a total cannot tell a caller which file moved three times and which +# moved once. +edits = value.get("edits_by_file") +if isinstance(edits, list) and edits: + per_file = ", ".join( + f"{entry.get('path')}: {entry.get('edits')}" for entry in edits + if isinstance(entry, dict)) + total = sum(entry.get("edits", 0) for entry in edits if isinstance(entry, dict)) + proven(f"edits_by_file came back per file — {len(edits)} file(s), {total} edit(s): {per_file}") + if len(edits) < 2: + print(" \033[33mNOT PROVEN\033[0m only one file moved, so this run cannot tell a " + "per-file count from a total. The corpus in dev/rust-corpus has the " + "symbol in two crates") +else: + failed("edits_by_file is not in the answer, so nothing says how much of each file moved") +if value.get("definition"): + proven(f"and it named the definition it had resolved: {json.dumps(value['definition'])[:200]}") +tree = answer.get("tree") +if tree == "restored" and answer.get("succeeded") is True: + proven("the workspace was put back byte for byte because the caller asked, not because anything failed (succeeded=true, tree=restored)") +elif tree == "restored": + failed("the tree was restored because something failed, which is not what this asked for") +else: + failed(f"the workspace was left changed (tree={tree!r}); put it back before doing anything else") +PY + cat "$WORK/exec.lines" + tally "$WORK/exec.lines" +fi + +echo +echo " ════════════════════════════════════════════════════════════" +printf ' proven %d\n' "$PROVEN" +printf ' not proven %d\n' "$UNPROVEN" +printf ' failed %d\n' "$FAILED" +echo " ════════════════════════════════════════════════════════════" +echo +if [ "$FAILED" -gt 0 ]; then + echo " The machine does not do what Thalyx says it does." + echo " The whole exchange is in $WORK — copy it before this exits." + exit 1 +fi +echo " The machine resolved a Rust name with its own compiler, renamed it," +echo " and gave the tree back exactly as it found it." diff --git a/dev/verify.sh b/dev/verify.sh index 2223474..f753b63 100755 --- a/dev/verify.sh +++ b/dev/verify.sh @@ -8703,6 +8703,62 @@ fi # seventeenth. stage_60 +step "61. the Rust the agent programs with belongs to Thalyx, not to this machine" + +# `vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md`, 2026-08-31. A paid run on +# 2026-08-30 watched Claude, inside the machine, choose the right primitive and +# be answered `there is no cargo on this machine`. The machine had promised a +# semantic rename with no compiler on it. +# +# ## What this stage is for, and why it is not the VM +# +# The physical claim — a booted Thalyx resolving a name — is `make -C image +# agent` and the commands in the note; no shell here can make it. What this +# stage establishes is the property that decides whether that claim survives +# being carried anywhere: **the artifact is closed**. Every library its own +# programs name is inside it, the loader travels with it, nothing in it points +# at the machine that built it, and the staged cargo can read a workspace with +# an empty environment. +# +# That is exactly the thing a check on the *building* machine gets wrong by +# default: an artifact that quietly resolves against this Fedora's `/usr/lib` +# looks perfect here and is a directory of dead ELF files inside Thalyx, whose +# `/lib` is empty. `ldd` cannot tell them apart, because `ldd` asks this host. +# `thalyx dev rust-runtime` reads the headers and asks the artifact. +RUNTIME_DIR="${THALYX_RUST_RUNTIME:-}" +if [ -z "$RUNTIME_DIR" ]; then + RUNTIME_DIR=$(ls -d "$ROOT"/image/build/rust-runtime/rust-* 2>/dev/null | head -1 || true) +fi +if [ -z "$RUNTIME_DIR" ] || [ ! -d "$RUNTIME_DIR" ]; then + if [ "${THALYX_REQUIRE_RUST_RUNTIME:-0}" = 1 ]; then + failed "THALYX_REQUIRE_RUST_RUNTIME=1 and there is no artifact: make -C image rust-runtime" + else + unproven "there is no Rust runtime artifact here, so nothing about the agent's toolchain was checked. Build one with: make -C image rust-runtime (about 170 MB, once)" + fi +else + RUNTIME_LOG="$WORK/rust-runtime.log" + if "$THALYX" dev rust-runtime "$RUNTIME_DIR" > "$RUNTIME_LOG" 2>&1; then + proven "$(basename "$RUNTIME_DIR") is closed: every library cargo, rustc, rust-analyzer and the proc-macro server name is inside the artifact, and so is the loader they ask the kernel for" + else + failed "the Rust runtime artifact would not work inside a machine; see $RUNTIME_LOG" + excerpt "$RUNTIME_LOG" + fi + + RUNTIME_TESTS="$WORK/rust-runtime-tests.log" + if ( cd "$ROOT" && env THALYX_REQUIRE_RUST_RUNTIME=1 "THALYX_RUST_RUNTIME=$RUNTIME_DIR" cargo test -p thalyx-rust --test the_runtime_thalyx_carries_runs_on_its_own ) > "$RUNTIME_TESTS" 2>&1; then + # `4 passed` and not merely exit 0: these skip loudly on a machine with + # no artifact, and rule 3 says a skip exits successfully. + if grep -q "4 passed" "$RUNTIME_TESTS" && ! grep -q "NOT PROVEN" "$RUNTIME_TESTS"; then + proven "the staged cargo, rustc and rust-analyzer all answered through the artifact's own musl loader with an empty environment, and the staged cargo described a workspace it had never seen — no PATH, no RUSTUP_HOME, nothing from this machine" + else + unproven "the artifact's programs were not exercised; see $RUNTIME_TESTS" + fi + else + failed "the staged toolchain does not run out of its own artifact; see $RUNTIME_TESTS" + excerpt "$RUNTIME_TESTS" + fi +fi + # ------------------------------------------------- the machine, as it is left # # The last stage that arms the machine has no stage after it, so `step()` never diff --git a/image/Makefile b/image/Makefile index 86212d3..9bd14fe 100644 --- a/image/Makefile +++ b/image/Makefile @@ -8,6 +8,7 @@ # make store-stage build what goes on the store disk # sudo make store format the disk and copy that onto it # make run boot it in QEMU +# make rust-runtime build the Rust toolchain the agent programs with # make count list what is really inside the image # sudo make installed partition a disk image and install Thalyx onto it # make run-installed boot that disk, with no medium attached @@ -172,6 +173,32 @@ AGENTMARK := $(BUILD)/agent-workspace # so writing it cannot move a restore verdict. AGENTIMPORT := $(BUILD)/agent-import.json +# ─────────────────────────────────────── the Rust runtime the agent programs with +# +# `vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md`. On 2026-08-30 a paid +# benchmark watched Claude, inside the machine, pick exactly the right primitive +# and be told `there is no cargo on this machine`. The machine had promised a +# semantic rename and had no compiler to keep the promise with. +# +# The fix is not to lend it the host's: `Filosofia-Fundacional.md` says Thalyx is +# the whole system, and a programming face that only works because Fedora has +# rustup installed is a face that belongs to Fedora. So `dev/build-rust-runtime.sh` +# builds an artifact out of digest-checked upstream tarballs — Rust's own musl +# host tools, and a musl loader compiled from musl's own release — and it goes on +# the **store**, never in the image. `make count` still says the kernel and one +# program. +# +# RUST=auto the default: on when PROJECT is a Cargo workspace, off otherwise +# RUST=1 always stage it, whatever the project is +# RUST=0 never, even for a Rust workspace +# +# `RUSTRUNTIME` names an artifact that was already built, so a person preparing +# several machines pays the download once. Left empty, the build directory is the +# cache and a second `make` finds it there. +RUST ?= auto +RUSTRUNTIME ?= +RUSTBUILD := $(BUILD)/rust-runtime + # The first module. It is not in the image and must not be: the image carries # the kernel and one program. A module lives on the store, which is the whole # distinction between what Thalyx *is* and what has been installed on it. @@ -218,7 +245,7 @@ ENGINE_RUN := $(ENGINE_DATA)/run store store-stage greeter engine engine-stage lsm doctor hook-check boot pin-kernel \ boot-graphical run-serial \ esp run-uefi initramfs-check installed run-installed run-hardware \ - run-agent boot-agent agent agent-export + run-agent boot-agent agent agent-export rust-runtime rust-stage all: doctor kernel binary image hook-check @echo @@ -888,6 +915,7 @@ store-stage: binary greeter fi @$(MAKE) --no-print-directory engine-stage @$(MAKE) --no-print-directory project-stage + @$(MAKE) --no-print-directory rust-stage @# Written last, and only on success. `store` looks for this and not for @# the directory: an interrupted stage leaves the directory sitting there @# looking finished, and a disk built from it would be missing whatever @@ -951,6 +979,33 @@ project-stage: echo " workspace: $(WORKSPACE) — a copy of $(PROJECT), $$(du -sh "$(STAGE)/user/$(PROJECTNAME)" | cut -f1)"; \ echo " imported: $(AGENTIMPORT)" +# The Rust runtime as an artifact on this host, built once and cached. +# +# Separate from everything else and never an implicit prerequisite of anything +# but `rust-stage`: it downloads about 170 MB and compiles a C library, and a +# `make` that did that every time would make every other target unusable. The +# same arrangement `engine` has, for the same reason. +rust-runtime: + @$(ROOT)/dev/build-rust-runtime.sh $(RUSTBUILD) + +# That artifact onto the stage, so the machine boots able to resolve a name. +# +# It lands under `system/`, which is `/opt/thalyx` inside the machine, and +# therefore on the store — never in the initramfs. `make count` says the image +# is the Linux kernel and one program and it has to keep saying that; six +# hundred megabytes of compiler is *software installed on Thalyx*, which is the +# whole distinction between what Thalyx is and what has been put on it. The +# engine and its weights are on the store for the same reason. +# +# Checked after it is copied, by Thalyx itself, and the check is not "the files +# arrived": `thalyx dev rust-runtime` reads the ELF headers of the staged +# programs and asks whether every library they name is inside the artifact. An +# artifact that would resolve against the *building* host looks perfect on the +# building host, and that is exactly the failure this whole change exists to +# end. +rust-stage: + @set -e; wanted="$(RUST)"; case "$$wanted" in 0|no|off) exit 0 ;; auto) if [ -n "$(PROJECT)" ] && [ -f "$(PROJECT)/Cargo.toml" ]; then wanted=1; else exit 0; fi ;; esac; artifact="$(RUSTRUNTIME)"; if [ -z "$$artifact" ]; then echo " rust: building or reusing the runtime artifact"; artifact="$$($(ROOT)/dev/build-rust-runtime.sh $(RUSTBUILD) | tail -1)"; fi; test -d "$$artifact" || { echo " a Rust runtime was asked for and there is none at $$artifact."; echo " Build one first: make -C image rust-runtime"; exit 1; }; identity="$$(basename "$$artifact")"; rm -rf "$(STAGE)/system/toolchains/rust/$$identity"; mkdir -p "$(STAGE)/system/toolchains/rust"; cp -a "$$artifact" "$(STAGE)/system/toolchains/rust/$$identity"; $(THALYX) dev rust-runtime "$(STAGE)/system/toolchains/rust/$$identity" || { echo " the staged runtime is not usable, so nothing was staged"; rm -rf "$(STAGE)/system/toolchains/rust/$$identity"; exit 1; }; echo " rust: from $$artifact"; echo " $$(du -sh "$(STAGE)/system/toolchains/rust/$$identity" | cut -f1) staged"; echo " inside Thalyx: /opt/thalyx/toolchains/rust/$$identity" + engine-stage: @$(ROOT)/dev/stage-engine.sh "$(STAGE)" "$(BUILD)" "$(THALYX)" "$(ENGINE)" "$(MODEL)" "$(ENGINE_ID)" "$(ENGINE_TIER)" "$(ENGINE_MODELS)" "$(ENGINE_RUN)" @@ -1242,6 +1297,14 @@ boot-agent: # # make -C image agent PROJECT=~/code/mi-proyecto # +# **A Cargo workspace gets a Rust toolchain without anybody asking for one.** +# `RUST=auto` — the default — sees `PROJECT/Cargo.toml` and stages the runtime +# artifact, so `context` and `rename` inside the machine resolve names instead +# of matching them. That is `vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md`, +# written the day after a paid run was answered `there is no cargo on this +# machine`. The first one builds the artifact and costs about 170 MB of +# download; every one after it reuses `build/rust-runtime/`. +# # The one that needs root is `store`, and it is the only one — same rule as # everywhere else here: the privilege boundary is the target boundary, and a # `make` that quietly asked for a password is a `make` nobody can put in a @@ -1256,7 +1319,7 @@ agent: echo " touched, and never reachable from inside the machine."; \ exit 1; \ } - $(MAKE) --no-print-directory store-stage PROJECT=$(PROJECT) + $(MAKE) --no-print-directory store-stage PROJECT=$(PROJECT) RUST=$(RUST) @echo @echo " the disk needs root: it formats a loop device and mounts it." sudo $(MAKE) --no-print-directory store diff --git a/vault/06-Pendientes/Punto-Actual.md b/vault/06-Pendientes/Punto-Actual.md index fd554c9..1932368 100644 --- a/vault/06-Pendientes/Punto-Actual.md +++ b/vault/06-Pendientes/Punto-Actual.md @@ -1,7 +1,7 @@ --- tipo: estado-vivo estado: activo -fecha-actualizacion: 2026-08-30 +fecha-actualizacion: 2026-08-31 tags: [continuidad, punto-actual, sesiones] --- @@ -14,10 +14,54 @@ tags: [continuidad, punto-actual, sesiones] > > Para *cómo* trabajar en el proyecto, ver `CLAUDE.md` en la raíz del repo. -## Thalyx como máquina para Claude Code: los costes de la traza, atacados — 2026-08-30 +## La máquina agente lleva su propio Rust — 2026-08-31 **Éste es el estado actual.** Los bloques de abajo son cómo se llegó. +La corrida compacta perdió por una causa física, y ya está identificada exacta: +dentro de la VM, Claude eligió la primitiva correcta —`context` y luego +`rename`— y Thalyx contestó `source: index`, `analyzer_starts: 0` y +**`there is no cargo on this machine`**. La máquina prometía semántica Rust y no +llevaba compilador. + +Lo cerrado en esta sesión está entero en [[Runtime-Rust-Agente]]. En corto: + +- **`dev/build-rust-runtime.sh`** construye un artefacto propio de Thalyx desde + tarballs oficiales con digest fijado: las herramientas musl de Rust + (`cargo`, `rustc`, `rust-analyzer`, servidor de proc-macros, `rust-std`, + `rust-src`) más un `libc.so` de musl **compilado aquí** desde el release de + musl, y un `libgcc_s.so.1` enlazado del propio `libunwind.a` de Rust. 644 MB. + Ningún byte sale de la máquina que lo construye. +- **Vive en el store**, en `/opt/thalyx/toolchains/rust//`, nunca en + el initramfs — con una prueba que arma el archivo y lo cuenta. +- **PID 1** hace que `/lib/ld-musl-x86_64.so.1` apunte al cargador del artefacto + después de montar el store, que es lo que el kernel lee de `PT_INTERP`. +- **El descubrimiento** pone el runtime de Thalyx en segundo lugar, sólo detrás + de una variable que nombre un archivo. Cuando Thalyx lleva compilador, ése es + el compilador. +- **El preflight ya no puede decir READY sobre esto**: `--needs-rust` le + pregunta a la máquina el verbo `toolchain`, que corre `cargo --version` y + `rust-analyzer --version` **dentro** y lee los manifiestos. El banco lo pide + solo cuando el proyecto tiene `Cargo.toml`. + +### Qué está probado y qué falta + +Probado en el contenedor, físicamente: el artefacto es **cerrado** —cada +biblioteca que sus programas nombran está dentro—, y dentro de un chroot que no +tiene más que el artefacto, un `/proc` y tres nodos de dispositivo —sin shell, +sin `/usr`, sin `/lib64`— `cargo`, `rustc` y `rust-analyzer` arrancan, +`cargo metadata` describe un workspace sintético, y una sesión LSP completa +resolvió una definición y devolvió **cuatro ediciones de rename reales**. + +**Falta la prueba en la VM real**, que es de César: los comandos están al final +de [[Runtime-Rust-Agente]] y en el mensaje de esta sesión. Hasta que esa corra, +lo que hay es un artefacto que funciona en un chroot, no una máquina Thalyx +arrancada resolviendo nombres. + +**No se corrió el banco pagado.** + +## Thalyx como máquina para Claude Code: los costes de la traza, atacados — 2026-08-30 + La corrida A/B compacta (`/var/tmp/thalyx-bench-compact-1/`) quedó físicamente cerrada con los dos brazos VÁLIDOS. Demostró **la propiedad buena**: Claude sí delegó trabajo compuesto y Thalyx hizo 36 operaciones internas en 3 programas. @@ -94,8 +138,6 @@ Ver `vault/07-Adopcion-y-Fases/Agentes-Externos.md`. ## El vertical físico, cerrado: la ventana de denegación es de quien la necesita — 2026-08-30 -**Éste es el estado actual.** Los bloques de abajo son cómo se llegó. - La transacción programable estaba construida. La corrida real en Fedora sobre `8ff60fa` la ejerció completa y encontró defectos de **integración y arnés**, no de diseño — y todos colapsaban en una sola contradicción. diff --git a/vault/06-Pendientes/Tareas-Pendientes.md b/vault/06-Pendientes/Tareas-Pendientes.md index 30dd6f4..a823171 100644 --- a/vault/06-Pendientes/Tareas-Pendientes.md +++ b/vault/06-Pendientes/Tareas-Pendientes.md @@ -881,3 +881,22 @@ no, están en la evidencia de la máquina y nada en el anfitrión las tiene. **Lo que sigue abierto y sigue siendo de Cesar**: correr la validación de este sprint en Fedora antes de gastar otra corrida A/B pagada. + +## El runtime Rust del agente — 2026-08-31 + +Cerrado: la máquina agente lleva su propio Rust. Ver [[Runtime-Rust-Agente]]. + +Lo que queda abierto, y **cada cosa es su propia causa conocida**: + +- **Compilar dentro de la máquina.** El artefacto resuelve y renombra; no + enlaza. Hace falta un enlazador y un `libgcc_s` con los builtins enteros de + libgcc, no sólo el unwinder. +- **Cache de Cargo administrado por Thalyx.** El proveedor semántico no tiene + red por construcción. Un workspace con dependencias de registro lo va a + necesitar. **No se hizo porque todavía no falló por eso.** +- **Subir musl de 1.2.4 a 1.2.5.** Una línea en el script y la misma prueba + física otra vez. 1.2.4 es la que se corrió de punta a punta. +- **`toolchain` como verbo que el modelo local pueda proponer.** Hoy sólo se + escribe o se pide por la superficie externa; «¿puedes renombrar símbolos + aquí?» es una frase que César podría decir en voz alta. Registrado en + `NOT_A_SENTENCE`, en `catalogue.rs`, donde se decidiría. diff --git a/vault/09-Notas-Tecnicas/Estado-de-Implementacion.md b/vault/09-Notas-Tecnicas/Estado-de-Implementacion.md index b5c93e8..711111d 100644 --- a/vault/09-Notas-Tecnicas/Estado-de-Implementacion.md +++ b/vault/09-Notas-Tecnicas/Estado-de-Implementacion.md @@ -306,6 +306,20 @@ cosas ningún verbo puede usar. | `contexto` y `renombrar-simbolo` | `crates/thalyx-cli/src/semantic.rs` | La cara caliente. `contexto` contesta qué es un nombre en unos cientos de bytes con un asa, `contexto expandir=` trae exactamente las líneas de esa declaración, `presupuesto=N` acota y dice qué no cupo, y `usos=N` pide los lugares donde se usa — el número siempre viene, la lista sólo si se pide, porque sobre un nombre común la lista es todo el presupuesto. Cada respuesta dice `source` —`rust-analyzer` resolvió, `index` coincidió— y `fresh`. `renombrar-simbolo` escribe en cada lugar que de veras usa el nombre, no donde el texto coincide, y por la frontera de la sesión. **Desde el 2026-08-30 contesta también `edits_by_file`** —cuántos lugares tocó en cada archivo— contado del `WorkspaceEdit` que rust-analyzer ya entregó, mientras se aplica, nunca re-escaneando el árbol: un segundo pase sería un conteo textual de una cadena, que es justo lo que este camino existe para superar. Y `definition` aparece **sólo cuando el lugar se alcanzó a través de la declaración del símbolo**; dado `archivo:línea:columna` el llamador apuntó a algún lado y el campo se omite en vez de inventarse. Ver [[Contexto-Progresivo]] | | Importar un proyecto a la máquina | `image/Makefile` (`agent`, `run-agent`, `agent-export`) | Una copia descartable, como subvolumen propio para que `intento` tenga qué fotografiar. El checkout del anfitrión no se toca y no es alcanzable desde adentro | +### El runtime Rust del agente, que es de Thalyx y no del anfitrión — 2026-08-31 + +`dev/build-rust-runtime.sh` construye un artefacto de 644 MB desde tarballs +oficiales con digest fijado —las herramientas musl de Rust, más un `libc.so` de +musl compilado aquí y un `libgcc_s.so.1` enlazado del `libunwind.a` del propio +Rust— y `make -C image agent PROJECT=…` lo pone en el store, jamás en la imagen. +`thalyx-rust::runtime` lo encuentra, `thalyx dev rust-runtime` comprueba que se +lleva consigo todo lo que sus programas nombran, y PID 1 hace que +`/lib/ld-musl-x86_64.so.1` apunte a su cargador. El verbo `toolchain` y +`thalyx-mcp --preflight --needs-rust` hacen que un banco no pueda volver a pagar +por una máquina sin compilador. Entero en [[Runtime-Rust-Agente]]. + +**Lo que no hace:** compilar. No lleva enlazador, a propósito y explicado ahí. + ## No construido todavía | Pieza | Bloqueante para | diff --git a/vault/09-Notas-Tecnicas/Estrategia-de-Pruebas.md b/vault/09-Notas-Tecnicas/Estrategia-de-Pruebas.md index 01c0740..fb67e0e 100644 --- a/vault/09-Notas-Tecnicas/Estrategia-de-Pruebas.md +++ b/vault/09-Notas-Tecnicas/Estrategia-de-Pruebas.md @@ -6025,3 +6025,49 @@ controla, se dice de quién es la afirmación y quién la comprobó. Un presupue que sólo es correcto en el cliente que alguien revisó una vez no es un presupuesto. + +## Regla derivada: una comprobación que le pregunta al anfitrión contesta sobre el anfitrión — 2026-08-31 + +`ldd` sobre el toolchain que va al store dice **lo que resolvería esta Fedora**, +porque arranca el cargador real contra el `/lib` real. La pregunta que tenía el +artefacto era la contraria: si se lleva consigo todo lo que sus propios programas +nombran, en una máquina —Thalyx— cuyo `/lib` está vacío. + +Un artefacto al que le falte una biblioteca que casualmente vive en el `/usr/lib` +del anfitrión pasa `ldd` **perfecto** en el anfitrión, y es un directorio de ELF +muertos dentro de la máquina. La forma del fallo tampoco ayuda: `execve` contesta +`ENOENT` y lo que la persona lee es *«no hay cargo en esta máquina»* sobre un +cargo que está ahí. + +Así que `thalyx dev rust-runtime` lee las cabeceras: `PT_INTERP` y cada +`DT_NEEDED`, y pregunta si están **dentro del artefacto**. No necesita máquina +para contestar, que es también lo que lo hace comprobable en un contenedor. + +La regla general: cuando la propiedad es *«esto funciona en otra máquina»*, toda +herramienta que consulte a ésta es el instrumento equivocado — regla 5 apuntada +al ambiente en vez de al arnés. Se lee la descripción del artefacto, no la +respuesta del sistema que lo construyó. + +Y su reverso, el mismo día: para *ejecutar* lo que se envía sin contaminarlo, el +cargador de musl se invoca directo —`/lib/libc.so /bin/cargo`— +con el entorno vaciado. Corre el binario que se envía, en cualquier anfitrión, sin +que nadie escriba en `/lib`, que sería la regla 11: una prueba que escribe algo +global cambió la máquina que estaba midiendo. + +## Regla derivada: `$ORIGIN` del programa principal, bajo musl, se lee de `/proc` — 2026-08-31 + +Mismo binario, mismo artefacto, mismo `RPATH: [$ORIGIN/../lib]`: con `/proc` +montado arranca, sin `/proc` muere con + +``` +Error loading shared library librustc_driver-.so: No such file or directory +``` + +que se lee como una biblioteca que falta y no lo es. `fixup_rpath` de musl +resuelve el `$ORIGIN` del ejecutable mapeado por el kernel leyendo +`/proc/self/exe`; sin `/proc` no hay origen y el `RPATH` no expande. + +La regla: cuando un `RPATH` relativo falla, la pregunta antes de tocar el +artefacto es **qué sabe el proceso de sí mismo**. Y como no se debe depender de +que quien arranque el proceso se acordara de montar `/proc`, el directorio se +nombra además en `LD_LIBRARY_PATH`, que no depende de nada. diff --git a/vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md b/vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md new file mode 100644 index 0000000..def284b --- /dev/null +++ b/vault/09-Notas-Tecnicas/Runtime-Rust-Agente.md @@ -0,0 +1,228 @@ +--- +tipo: especificacion +estado: decretado +fecha-decreto: 2026-08-31 +tags: [rust, runtime, agente, store, artefacto, semantica, no-negociable] +--- + +# El runtime Rust del agente + +> Esta nota existe por una corrida pagada. El 2026-08-30, dentro de la máquina, +> Claude eligió **exactamente la primitiva correcta** —preguntar qué es un +> símbolo y luego renombrarlo— y Thalyx le contestó que no había compilador. +> Todo lo demás de esa traza es consecuencia de eso. + +## Lo que pasó + +En `/var/tmp/thalyx-bench-compact-1/armB.ndjson`, la primera estrategia del +modelo fue la buena: + +```js +const def = thalyx.context('…'); +const r1 = thalyx.rename('…', '…'); +``` + +y la máquina respondió: + +``` +context: source: index resolution: matched analyzer_starts: 0 +rename: ok: false error: unresolved + message: there is no `cargo` on this machine +``` + +El preflight había dicho `READY`. Y tenía razón en lo que preguntaba: la máquina +estaba viva y sostenía el árbol correcto. Ninguna de las dos cosas es la +capacidad que la tarea necesitaba, y no había nada entre el dinero y enterarse. + +## El decreto + +**El anfitrión no aporta ni un archivo del entorno de programación mientras el +agente trabaja.** Ni `cargo`, ni `rustc`, ni `rust-analyzer`, ni el servidor de +proc-macros, ni la biblioteca estándar, ni sus fuentes, ni `librustc_driver`, ni +LLVM, ni el cargador dinámico. + +El primer intento de arreglo fue copiar el `~/.rustup` de Fedora al store. César +lo detuvo, y tenía razón: [[Filosofia-Fundacional]] dice que Thalyx **es** el +sistema, así que una cara de programación que sólo funciona porque el anfitrión +tiene rustup instalado es una cara que le pertenece al anfitrión. Apagar Fedora, +mover el disco a otra x86_64 y arrancar no puede hacer desaparecer el proveedor +semántico. + +Criterio, en una frase: **el artefacto se lleva consigo todo lo que sus propios +programas nombran.** + +## Dónde vive + +`/opt/thalyx/toolchains/rust/rust--/`, o sea el subvolumen +`system` del store. + +**Nunca el initramfs.** [[Construccion-del-ISO]] dice que la imagen es el kernel +de Linux y un programa, y lo dice de forma contable a propósito. Seiscientos +megabytes de compilador son *software instalado sobre Thalyx*, que es justo la +distinción entre lo que Thalyx **es** y lo que le han puesto encima — la misma +razón por la que el motor y los pesos viven en el store y no en la imagen. + +Hay una prueba que lo sostiene, no una promesa: `the_rust_runtime_is_not_in_the_image` +arma el archivo y cuenta lo que hay dentro. + +## De qué está hecho, y por qué de eso + +### Se eligió el toolchain musl, y fue una medición + +Los dos caminos, medidos el 2026-08-31 antes de escribir una línea: + +| | lo que le falta al artefacto | +|---|---| +| `x86_64-unknown-linux-gnu` | cargador de glibc, `libc`, `libm`, `libdl`, `librt`, `libpthread`, `libgcc_s`, `libz` — y además un `libLLVM.so` aparte de 191 MB | +| `x86_64-unknown-linux-musl` | **dos archivos**: el cargador de musl y `libgcc_s.so.1`. LLVM va dentro de `librustc_driver`, así que no hay `libLLVM` | + +Rust publica herramientas de anfitrión para `x86_64-unknown-linux-musl` de forma +oficial, con un sha256 por archivo en su propio manifiesto de canal. Dos +archivos que faltan es un problema que una persona cierra; media distribución no +lo es. Ésa es toda la razón de la elección. + +Es además la convención que este repositorio ya tenía: `dev/build-engine.sh` se +niega a terminar si el motor no es un ELF sin `INTERP` y sin `NEEDED`, porque +**dentro de la máquina no hay libc**. El toolchain es el primer programa que no +puede ser estático, así que se trae su propio mundo. + +### El cargador se compila, no se copia + +`libc.so` de musl, construido desde el tarball de release de musl con su digest +fijado. El mismo arreglo que ya tiene el kernel de Linux en `image/Makefile`: un +tarball fijado, comprobado antes de creerle, compilado por nosotros. Es un +megabyte y tarda segundos. + +Bajo musl, `libc.so` **es** también el enlazador dinámico, así que ese único +archivo cubre el `PT_INTERP` y el `DT_NEEDED libc.so` de todos los binarios. + +### `libgcc_s.so.1` sale del propio Rust + +Se enlaza desde el `libunwind.a` que Rust envía dentro de `rust-std`, en +`self-contained/`. No entra ninguna fuente nueva: viene del mismo artefacto +verificado que el compilador. + +Que alcance es una medición, no una esperanza. De los 883 símbolos indefinidos +de `cargo`, `rustc`, `rust-analyzer`, el servidor de proc-macros y +`librustc_driver`, todo resuelve contra musl y `librustc_driver` salvo 29 — y de +esos 29 los únicos que **no** son débiles son los quince `_Unwind_*`, que +`libunwind.a` define todos. El resto son los stubs de memoria transaccional, +`__register_frame_info`, y los dos `pidfd_*` que la std de Rust enlaza débilmente +para musl más nuevos. + +Después se corrió en vez de argumentarse: `rustc` desenrolló un error fatal y +salió con 1 limpiamente, que es exactamente el `catch_unwind` que rustc usa para +su propio `FatalError`. + +## Lo que deliberadamente **no** lleva + +| fuera | por qué | +|---|---| +| `share/`, páginas de manual, documentación | no se ejecuta nunca, y son 13 MB sólo del componente `rustc` | +| `lib/rustlib//bin/` | 174 MB de **enlazadores** — `rust-lld`, `wasm-component-ld`, `rust-objcopy`. El proveedor semántico nunca enlaza: ni `cargo metadata` ni el análisis de rust-analyzer lo hacen | +| cualquier otro target | un anfitrión, un objetivo | +| `rustdoc`, `rustfmt`, `clippy` | no es para lo que sirve un proveedor semántico | + +Dejar fuera los enlazadores es además lo honesto: `rust-lld` pide los builtins +enteros de libgcc (`__popcountdi2` y compañía), que el `libgcc_s` de sólo +unwinder no tiene. **Enviar un enlazador que no arranca es peor que no enviar +ninguno.** + +Así que este artefacto **resuelve y renombra; no compila**. El día que algo +dentro de Thalyx necesite compilar una proc-macro o un build script, ésa es la +siguiente causa conocida y le toca su propio cambio con su propia evidencia. Una +causa a la vez. + +## `rust-src` es obligatorio, y se supo por medición + +Sin `lib/rustlib/src`, rust-analyzer escribe `can't load standard library, try +installing rust-src` y se muere a media primera pasada de análisis. Está en la +lista de archivos requeridos por eso, no por prolijidad. + +## El detalle de musl que costó una tarde + +Los binarios llevan `RPATH: [$ORIGIN/../lib]`, y **musl resuelve `$ORIGIN` del +programa principal leyendo `/proc/self/exe`**. El mismo binario, mismo artefacto: +con `/proc` montado arranca; sin `/proc` muere con + +``` +Error loading shared library librustc_driver-.so: No such file or directory +``` + +Que es un `execve` que parece un archivo que falta y no lo es. + +Hay dos formas de cerrarlo y Thalyx usa las dos: el sandbox monta `/proc` tras el +pivot, y `toolchain::environment` nombra el directorio en `LD_LIBRARY_PATH`. La +segunda es la que no depende de que quien arranque el proceso se acordara de la +primera. + +## Cómo se descubre, y en qué orden + +`thalyx-rust::toolchain`: + +1. una variable que nombra un archivo (`THALYX_CARGO`, `THALYX_RUST_ANALYZER`); +2. **el runtime de Thalyx en el store**; +3. `RUSTUP_HOME`; +4. el home de quien invocó (`SUDO_USER`), luego `HOME`; +5. `/usr/local/bin`, `/usr/bin`. + +El segundo lugar es el decreto: cuando Thalyx lleva compilador, **ése es el +compilador**, y uno instalado en el anfitrión es el respaldo y no al revés. Sólo +una persona nombrando un archivo lo supera. + +Y sigue valiendo la regla que ese archivo tiene desde que existe: **un candidato +es la herramienta después de contestar `--version`**, nunca antes. Un runtime a +medio copiar es un directorio de ELF perfectos que no arrancan; la búsqueda cae +al siguiente lugar y lo dice, en vez de entregar algo que muere después. + +En una máquina sin store —cualquier portátil, este contenedor, `dev/verify.sh`— +el paso 2 no existe y nada cambió. + +## Qué comprueba Thalyx del artefacto, y por qué `ldd` no sirve + +`thalyx dev rust-runtime ` lee las cabeceras ELF de los programas +staged y pregunta si **cada** biblioteca que nombran está dentro del artefacto, +y si el cargador que le piden al kernel viaja con él. + +`ldd` no puede contestar eso: `ldd` arranca el cargador real contra el `/lib` +real, así que dice lo que resolvería *esta* Fedora. Un artefacto al que le falte +una biblioteca que casualmente está en `/usr/lib` del anfitrión se ve **perfecto** +en el anfitrión y es un directorio de ELF muertos dentro de Thalyx. + +## El preflight ya no puede decir READY sobre esto + +`thalyx-mcp --preflight --needs-rust` le pregunta a la máquina el verbo +`toolchain`, que corre `cargo --version` y `rust-analyzer --version` **dentro** y +lee los manifiestos del workspace. No escribe nada: la lección del 2026-08-29 fue +una sonda que cambió el estado inicial de la corrida que estaba despejando, y +`cargo metadata --no-deps` no resuelve nada, así que no escribe `Cargo.lock`. + +`dev/bench-external-agent.sh` lo pide solo cuando el proyecto tiene `Cargo.toml` +en la raíz — derivado del árbol y no de una bandera, porque una bandera que +alguien tiene que acordarse de pasar es la bandera que faltó en la corrida que la +necesitaba, que es 2026-08-30 exactamente. + +## Camino de uso + +```sh +make -C image rust-runtime # una vez: ~170 MB, y compila musl +make -C image agent PROJECT=/ruta/proyecto # RUST=auto: se activa si hay Cargo.toml +``` + +`RUST=1` lo fuerza, `RUST=0` lo apaga, `RUSTRUNTIME=` reusa un artefacto ya +construido. El stage imprime de dónde salió, cuánto copió y dónde queda dentro de +Thalyx, y **se niega** —borrando lo copiado— si el artefacto no pasa la +comprobación de cierre. + +## Lo que queda abierto + +- **Compilar dentro de la máquina.** Hace falta un enlazador y un `libgcc_s` + completo. Es la siguiente causa conocida, no ésta. +- **Cache de Cargo.** El proveedor semántico no tiene red por construcción. Un + workspace con dependencias de registro va a necesitar un `CARGO_HOME` + aprovisionado y administrado por Thalyx. **No se implementó porque todavía no + falló por eso**: el workspace sintético de la prueba física no tiene + dependencias. Una causa conocida cada vez. +- **musl 1.2.4 y no 1.2.5**, porque 1.2.4 es la que se construyó y corrió de + punta a punta. Regla 12 de [[Estrategia-de-Pruebas]]: lo que se verifica tiene + que ser lo que se envía. Subirla es una línea y la misma prueba física otra vez.