Castle

CASTLE — Commit via Atomic Swap, Table-scoped, Leaderless Engine.

Castle is a Rust library, not a service: it gives a consumer crate a leaderless commit loop that converges independent writers on one agreed history of a keyspace, using only the backing object store's own conditional-write primitive. A consumer implements four small extension traits (ports) and calls one function, commit(), to get a commit attempt that is designed to be correct against contention, partial failure, and retry.

Quickstart

castle-core is not published to crates.io yet (publish = false, see crates/castle-core/Cargo.toml) — depend on it by git or path. The quickstart below also needs the testing feature (an in-memory StorageBackend double the crate ships for exactly this purpose — see below):

[dependencies]
castle-core = { git = "https://github.com/allodops/castle", features = ["testing"] }

Or, if you're vendoring/building inside this workspace:

[dependencies]
castle-core = { path = "../castle/crates/castle-core", features = ["testing"] }

(A real consumer that isn't just trying the quickstart drops features = ["testing"] — it gates a test-only double, not something a production build should carry.)

A consumer implements the four ports — Fragment (what a committed unit of work is), MergeStrategy (how existing fragments combine or retire), KeyspacePolicy (how the address space is scoped), and RetentionPolicy (how long a retired fragment must wait before physical deletion) — and calls commit(), which takes a StorageBackend, the three ports the commit loop itself needs, and a build closure that proposes each attempt's additions and merge inputs fresh from the state commit() just read:

This example compiles and runs as written, using castle_core::testing::fault_backend::FaultInjectingBackend so it needs no real object store. A real deployment implements StorageBackend against its own backend instead; see examples/lumen for a worked example of that plus all four ports together.

use castle_core::testing::fault_backend::FaultInjectingBackend;
use castle_core::{
    commit, BuildOutput, Checksum, ConservationValue, Fragment, FragmentId,
    IdentityClass, KeyspacePath, KeyspacePolicy, KeyspaceSegment, MergeOutcome,
    MergeStrategy, ObjectKeyPrefix, ReadState, RetentionPolicy,
    RetentionTimingBounds, Timestamp,
};
use std::time::Duration;

// 1. Fragment Port: what a committed unit of work is.
#[derive(Clone)]
struct MyFragment { id: FragmentId, bytes: Vec<u8> }

#[derive(Clone, PartialEq)]
struct Count(u64);
impl ConservationValue for Count {
    fn identity() -> Self { Count(0) }
    fn combine(&self, other: &Self) -> Self { Count(self.0.wrapping_add(other.0)) }
}

impl Fragment for MyFragment {
    type Contribution = Count;
    fn fragment_id(&self) -> FragmentId { self.id.clone() }
    fn byte_len(&self) -> u64 { self.bytes.len() as u64 }
    fn checksum(&self) -> Checksum { Checksum::from_bytes(self.bytes.clone()) }
    fn contribution(&self) -> Self::Contribution { Count(1) }
}

// 2. Merge Port: how fragments already committed combine or retire.
struct NoMerge;
impl MergeStrategy<MyFragment> for NoMerge {
    fn identity_class(&self) -> IdentityClass { IdentityClass::Preserving }
    fn merge(&self, _inputs: &[MyFragment]) -> MergeOutcome<MyFragment> {
        MergeOutcome { produced: vec![], retired: vec![] }
    }
}

// 3. Keyspace Port: how the address space is scoped. `partition_prefix`
// must be prefix-free (Keep Rule 5) — terminating *every* segment with
// the delimiter, including the last, is what makes ["a", "bc"] and
// ["a", "bcd"] map to "a/bc/" and "a/bcd/" rather than the colliding
// "a/bc" / "a/bcd" a plain join() would produce.
struct FlatKeyspace;
impl KeyspacePolicy for FlatKeyspace {
    fn partition_prefix(&self, path: &KeyspacePath) -> ObjectKeyPrefix {
        let mut joined = String::new();
        for segment in path.segments() {
            joined.push_str(segment.as_str());
            joined.push('/');
        }
        ObjectKeyPrefix::from_string(joined)
    }
}

// 4. Retention Port: how long a retired fragment waits before deletion
// (consumed by the reaper, not by commit() itself).
struct FixedGrace;
impl RetentionPolicy for FixedGrace {
    fn grace_window(&self, _bounds: &RetentionTimingBounds) -> Duration {
        Duration::from_secs(30)
    }
    fn eligible_for_deletion(
        &self, retired_at: Timestamp, now: Timestamp, bounds: &RetentionTimingBounds,
    ) -> bool {
        now >= retired_at.checked_add(self.grace_window(bounds)).expect("no overflow")
    }
}

fn main() {
    let backend = FaultInjectingBackend::new();
    let policy = FlatKeyspace;
    let merge_strategy = NoMerge;
    let path = KeyspacePath::new(vec![
        KeyspaceSegment::parse("my-tenant").unwrap(),
        KeyspaceSegment::parse("my-table").unwrap(),
    ]);
    let fragment = MyFragment {
        id: FragmentId::from_bytes(b"frag-1".to_vec()),
        bytes: b"hello castle".to_vec(),
    };

    let result = commit(
        &backend, &policy, &path, &merge_strategy, Duration::from_secs(5),
        |_read_state: &ReadState<'_>| BuildOutput {
            additions: vec![fragment.clone()],
            merge_inputs: vec![],
            created_at: Timestamp::from_epoch(Duration::from_secs(0)),
        },
    );

    match result {
        Ok(committed) => println!("committed at version {}", committed.version),
        Err(err) => eprintln!("commit failed: {err}"),
    }
}

Build and test:

cargo build --workspace --all-features
cargo test --workspace --all-features

MSRV: none is declared. No rust-version is set in any Cargo.toml in this workspace, so no minimum supported Rust version is guaranteed yet — build against a current stable toolchain.

Full API docs: not published to docs.rs yet (publish = false). Generate them locally instead:

cargo doc --open -p castle-core

What Castle is

Castle is a generic, extensible framework for coordinator-free, content-addressed compare-and-swap commit protocols over object storage. It specifies and implements exactly one thing — a leaderless commit loop in which independent writers converge on one agreed history of a keyspace using nothing but the object store's own conditional-write primitive — and stays deliberately blind to everything else: what a committed unit of work contains, how units get merged or compacted over time, and how the address space they live in is partitioned. Those three questions, plus how long a retired unit must safely wait before its bytes are physically deleted, are Castle's four extension points, called ports; a concrete consumer implements all four and gets a commit loop that is designed to be correct against contention, partial failure, and retry, provable in the sense CONSTITUTION.md §6 describes.

The generic commit loop, its three-tier formal-verification stack, the generic reaper, and a first reference extender (Lumen) are built and have each passed their own Adversarial Critic Pass Review (CONSTITUTION.md §9). A second, structurally different extender and port freeze are tracked next — see open issues and milestones for what's currently planned; nothing here is described as "planned" in prose.

Start here:

• CONSTITUTION.md — governance. What Castle is and is not, the Keep Rule / Yard Rule split that decides what can change casually versus what needs a human sign-off, the four extension ports at a governance level, the testing and formal-verification discipline, and the Adversarial Critic Pass Review (ACPR) practice every design decision and every "verified" claim in this repository must pass before it earns that word.
• docs/domain-model.md — mechanism. The actual commit-loop protocol, the four port contracts as Rust traits with their normative doc comments, the formal-verification story tying generic core proofs to per-extender proof obligations, a fully worked hypothetical extender exercising every contract by hand, and an honest accounting of where the design is known to be incomplete.
• docs/ledger.md — working notes on decisions still active enough to need them; settled ones fold into a Keep Rule or a tracked GitHub Issue and come out (CONSTITUTION.md §10).
• crates/castle-core — the framework itself. examples/lumen — the first reference extender. verification/commit-loop — the TLA+ model and independent Rust reimplementation.
• CONTRIBUTING.md — how to propose a change, including when a change needs an RFC first and what ACPR requires before something can be called "verified."

License: Apache-2.0. See LICENSE. Every dependency taken on by castle-core (CONSTITUTION.md §11) must be license-compatible with this choice — permissive or weak-copyleft terms that impose no obligation on Castle's own licensing, checked at the time it's added, not assumed.