FRE-14b — read a 0.1.x rows-mode tree into a store, and record that it was read

CHANGELOG.md

@@ -314,7 +314,44 @@ only the control plane, not the segment frames.
   order. It rides X-ERR's existing mapping — `503 incompatible_store`, CLI exit 65 — so there is no
   new status code. The refusal writes nothing but `LOCK` and `LOCK.holder`: the test asserts the
   file set is otherwise unchanged, that no marker is added, and that a second open refuses the same
-  way. Turning those rows into frames is the second half of FRE-14 and is still open.
+  way. Turning those rows into frames is the second half of FRE-14, below.
+- **A WeftDB 0.1.x data directory can be read into a store** (1.0 plan FRE-14, second half).
+  `weftdb::rows_mode` reads the rows-mode tree the refusal above names. `discover` reports the
+  series under `<subject>/<aspect>/measurements.db` — row count, span, how many rows share an
+  instant — and lists, with a reason, everything it will not carry: the `metadata.db` catalog,
+  the databases 0.1.x kept beside the rows, and the `dictionaries/` directories. An aspect's name
+  comes from its **directory**, not from the database: `measurements.db` holds only
+  `(id, dataset_id, timestamp, value)` — 0.1.x's own DDL (`weftdb/src/types/aspect.rs` in the
+  `v0.1.0` tag) — so the directory is the only record of what a series is called. The value column
+  has to be the `value` that DDL created: a database that names it after the aspect is refused, not
+  read, because an aspect called `dataset_id` would otherwise be read out of the column holding its
+  identifiers. `import_into` reads
+  one subject into the store scoped to that `(database, subject)` pair, declares each aspect under
+  the narrowest encoding that holds its values exactly (tolerance zero, `recommend_encoding` — the
+  reading trades no precision across the version boundary), seals the rows in batches
+  (`rows_per_segment`, default `DEFAULT_ROWS_PER_SEGMENT`) without ever splitting a group of rows that
+  share one instant across two segments — the store folds two segments whose windows intersect and
+  keeps the newer row at an instant they share, so a group cut in two would lose rows in a compaction
+  the reading does not control; a segment can therefore come out a few rows longer than asked. The
+  reading then writes an `ImportRecord` into the
+  store's own index under `rows_mode_import:<database>:<subject>`; `read_import_record` reads it
+  back after a reopen. That record is what stops the same tree being read twice
+  (`RowsModeError::AlreadyImported`), because after the fact a migrated row and a row the store
+  sealed itself are indistinguishable — so an aspect that already holds segments refuses first
+  (`AspectNotEmpty`), as does a tree whose database directory is not the store's database
+  (`ScopeMismatch`: 0.1.x puts its database only in the path, so the path is the only thing that
+  can say where a tree belongs). Timestamps are `Millis` and cross unchanged: 0.1.x stamped
+  `timestamp_millis()` into an `INTEGER`, and the `resolution` its catalog records is the sampling
+  grid, not a unit, so there is nothing to infer and nothing is inferred. A value cell that is not
+  the text of a decimal stops the read and quotes the row (`ValueNotDecimal`) rather than putting
+  a number nobody can vouch for into the store. The tree keeps its bytes: each `measurements.db`
+  is opened bare, not through the control plane's `connect`, which switches journal mode and so
+  writes; the test compares every file it found, byte for byte, after the reading. The aspect list is
+  written one aspect at a time and its record last, so a reading that stops partway leaves the aspects
+  that already crossed in the store with nothing recorded — the next attempt refuses on `AspectNotEmpty`
+  rather than finishing the job, and saying what an operator does then is the remaining part of FRE-14:
+  the route an operator runs from the command line or the server, the numbered migrations that register
+  `metadata.db`'s schema, and a reading of a tree too big to hold one aspect in memory.
 
 ### Changed
 

ROADMAP.md

@@ -1160,7 +1160,7 @@ any write, never purge an unproven frame), with its unfinished work on `durabili
   - [x] → **ROB-2** Storage-engine panic fail-stop: a turso-origin panic aborts — *#92: the write scope was already here — `weftdb::exec::control_plane_write`, and every `IndexTxn` operation runs inside it, which weftdb's own `every_op_runs_inside_the_control_plane_write_scope` asserts — so C12's "hook and knob" was the half that was missing, and it lives in the binary: `weft_server::panic_hook::install()` chains the hook it replaces, and when `in_control_plane_write()` says the panicking task held the scope it names the panic and calls `std::process::abort()`: SIGABRT for a supervisor, not the poison path's exit 70, and never an HTTP response (FRE-6a). Startup calls it immediately after `init_tracing`, because `set_hook` hands back the hook it replaced. It deliberately does not write the poison marker: that region reads before it writes (`txn.plans()`, `SegmentIndex::open`), and the marker survives the panic until an operator deletes a file. Nothing is caught, so ROB-3 stays the only catch layer and read-path panics still answer 500; the legacy `weftdb/src/types/aspect.rs` lane is ROB-5's. Proved by `weft-server/tests/control_plane_panic.rs`, which re-executes the test binary the way `poison_exit.rs` does.*
   - [x] **PAC-3** Library crate hygiene: no global allocator or wgpu features imposed on library users — *#93: two decisions were being made for every program that links these crates. `turso` 0.8.1 puts a `#[global_allocator]` in its own crate root behind its `mimalloc` feature, so the workspace's `features = ["mimalloc"]` on a **published library** (`weftdb/Cargo.toml`) chose the allocator of every embedder's process; and `splimes`' default features pull `wgpu`, so `wgpu` sat in the normal graph of `weftdb`, `weft-line-protocol`, `weft-reduce` and `weft-orchestration` — 52 packages of Vulkan, Metal and DX12 code for two crates that never call a GPU API. The allocator moved to the place that owns the process: `weft-server/src/main.rs` and `weft-tui/src/main.rs` each declare `static ALLOCATOR: mimalloc::MiMalloc` through a direct `mimalloc` dependency, so the same allocator on the same version still runs everything we benchmark and ship, and `weftdb` now carries no allocator feature at all — the plan's own lane row names this slice at `main.rs` (`docs/release/1.0-plan.md:126`). GPU is opt-in the same way: the workspace asks splimes for `default-features = false, features = ["serde"]`, `weft-server` and `weft-bench` add `gpu` themselves, and `weftdb`'s interpolation benches take it as a dev-dependency. Distinct packages in the normal graph: `weft-line-protocol` and `weft-reduce` 77 → 25, `weft-orchestration` 267 → 231, `weftdb` 266 → 227; `cargo tree -p weftdb -i wgpu -e normal` and `-i mimalloc` now return nothing. Two keeps are deliberate: splimes' `serde` stays on because `weftdb::BatchedMeasurement` holds a `splimes::Resolution` and derives `Serialize`/`Deserialize`, and turso's other default, `fts`, stays off exactly as it already was — an attempt to re-add it pulled `tantivy`, `zstd`, `datasketches` and friends, +435 lines of `Cargo.lock`, for full-text search nothing in this workspace uses. `tracing-subscriber` became a dev-dependency of `weftdb` (only test code installs a subscriber). One consequence for our own numbers: `cargo bench -p weftdb` used to inherit mimalloc and now runs on the system allocator, so those nine criterion benches are not comparable across this commit; the nightly regression track (EVI-5) uses `weft-bench`, which never linked `weftdb` and never had mimalloc, so no baseline moves there.*
   - [x] L-FMT: **X-FRAME** Frame registry, v7 envelope, `read_frame*`/`describe_frame`, `FrameFault` classes (absorbs FRE-4, TAG-2) — *#94: the three frame versions are named in one place, `weft_physical_type::frame_registry` — 5 single-block, 6 paged, 7 envelope, `layout_of`, `FrameExtension`, `FrameInfo`, `FrameFault { Integrity, Undecodable { newer_format } }` and `fault_class()`, with `KNOWN_EXTENSIONS` deliberately **empty** because an extension id is a permanent namespace. The envelope (`wrap_frame_envelope` in `weft-physical-type/src/weftseg.rs`) stores the older frame verbatim, magic, version and CRC included, for 16 bytes of overhead, so every byte-for-byte pin in this repo still holds and a legacy reader still answers `UnsupportedVersion { found: 7 }`; `frame_parts` verifies that CRC before it trusts a single header field, and `EnvelopeBodyMismatch` catches a header that disagrees with a body which is already under the checksum. Reading is now routed by **what the file says**: `weftdb/src/types/frame_read.rs` probes the header (`frame_layout_of`) instead of trusting `descriptor.format_version`, and its `FrameFaultError` reproduces the historical `decoding paged segment {path}: {sentence}` byte-for-byte, at all six call sites. Nothing opts into v7 yet — writers keep emitting 5 and 6, because `weftdb/src/types/segment_store.rs:7022` promises an unconfigured deployment bytes that are historical. `error::Code::FrameUndecodable` finally has a producer: a frame from a newer version is 503 `incompatible_store` and everything undecodable-but-current is 500, asserted in `weft-server/tests/error_contract.rs` by corrupting a real `.weftseg` (one byte, and a re-stamped-CRC version bump), which also shows the store surviving a damaged frame and the other aspect still reading. 306 `weft-physical-type` and 400 `weftdb` lib tests pass.*
-  - [ ] Legacy lane: **FRE-14** Legacy data-directory marker, numbered migrations and measurement upgrade test
+  - [ ] Legacy lane: **FRE-14** Legacy data-directory marker, numbered migrations and measurement upgrade test — *two of its three parts are in: 14a makes the build **refuse** a 0.1.x rows-mode directory instead of opening it as an empty store (`store_format`'s probe and the refusal at open), and 14b gives that tree a reader — `weftdb::rows_mode` discovers the series under `<subject>/<aspect>/measurements.db`, reads one subject into a store scoped to its `(database, subject)`, and records the reading in that store's own index under `rows_mode_import:<database>:<subject>`, which is what makes a second reading a refusal rather than a doubling of rows. `the_v010_rows_mode_tree_reads_into_a_store_of_its_own_database` takes the tree the release itself wrote and carries all 371 of its rows into a store of that tree's own database — 237 `price`, 54 `temp`, 80 `flow`, the non-null subset of `rows.csv`, each arriving at the instant its writer stamped, a thousand times the number the grid spells — and afterwards every file of the tree is still byte for byte what the release left. What remains is 14c: the route an operator runs, the numbered migrations that register `metadata.db`'s schema, and a reading of a tree too big to hold one aspect in RAM. One trap is pinned where the two trees meet: the same release stamped its rows-mode instants in **millis** (`v0.1.0:weftdb/src/types/database/inputs.rs:142`) and its Storage-v2 instants in **seconds** from the same value grid, so neither leg may read the unit off the numbers it is holding.*
   - [ ] **PAC-2** Portable Linux binaries on a glibc 2.28 baseline
   - [ ] **EVI-5** Nightly benchmark-regression workflow and `weft-bench compare`
   - [ ] **EVI-2** Claims registry and markdown claim/link lint

weftdb/src/types/mod.rs

@@ -26,6 +26,7 @@ pub use partial_sidecar::{PartialSidecar, PartialSidecarPolicy, DEFAULT_PARTIAL_
 pub use pattern::{Pattern, PatternID};
 pub use pipeline::{DetectorMetadata, DetectorType, PipelineConfig, PipelineState};
 pub use relative::Relative;
+pub use rows_mode::{discover, import_into, read_import_record, record_key, ImportRecord, ImportedAspect, LegacyAspect, LegacyTree, NotCarried, RowsModeError, DEFAULT_ROWS_PER_SEGMENT, IMPORT_RECORD_KEY, MEASUREMENTS_FILE, TIMESTAMP_COLUMN, TIMESTAMP_TIME_UNIT, TIMESTAMP_UNIT};
 pub use segment_index::SegmentIndexStore;
 #[cfg(feature = "fault-injection")]
 #[doc(hidden)]
@@ -70,6 +71,7 @@ pub mod pattern;
 pub mod pipeline;
 pub(crate) mod reaper;
 pub mod relative;
+pub mod rows_mode;
 pub mod segment_index;
 pub mod segment_store;
 pub mod signal;