[coderdan]

Summary

EQL (Encrypt Query Language) is the SQL bundle that stores and queries encrypted values in PostgreSQL; EQL v4 is the version for values written by Stack Encrypt, the Rust encryption engine in packages/stack-encrypt. Its json_tree type (unreleased) stores an encrypted JSON document as one entry per JSON node: a small ciphertext c holding that node alone, plus keyed tags that let PostgreSQL search and walk the document without the decryption key. Two of those tags matter here. A selector s is a keyed hash of a node's path. A token is a secret 16-byte value per node that the client hands the server in a query: from it the server can find that node's children, through each child's parent tag p, and so walk the subtree under the node.

This PR shrinks what each entry stores from nine fields to four (s, c, p, op) and changes how the server walks a subtree, so a walk no longer costs the document's widest node for every node it visits. Stacked on #1325. The stored format changes with it, so it has to land before the family ships.

• Three fields had no reader once the containment projection q was stored (perf(eql, stack-encrypt): store the json_tree projection, walk by parent map, budget the subtree #1325). q is the document's selector pairs grouped by array element, so PostgreSQL's own jsonb @> over it decides containment. Those three fields were the per-entry value selector vs and the two group tags n and g. The engine now reads q back as the projection instead of rebuilding it from the entries.
• Two more fields existed only so the server could continue the token chain: the wrapped token w and the ordinal o. A child's token is now H("t" ‖ t(parent) ‖ s ‖ LE32(o)), a hash of its parent's token, its own selector s and its position among its siblings, which the server derives as it walks. The positions under a node run 0, 1, 2, … with no gaps, so the walk probes them in turn and stops at the first miss. A subtree of N nodes costs about 2N probes (a SHA-256 and a lookup each) plus N token hashes, where it cost every position up to the document's widest node at every node visited.
• The client can no longer derive an object member's token: its position is a rank among sibling keys the client does not know. So a query sends the deepest token it can derive (a top-level field's, then any array positions directly after it) plus one gate per further path step: that step's selector, or #n:<selector> for an array position, with the array elements' selector. The server descends through the gates and returns the node they reach at depth d = 0, with its descendants beneath it.
• The leaf ordering term op is opt-in per path (commit ba62e3fc5). op is an order-preserving ciphertext of a number or string leaf, about 80 bytes each, which ORDER BY and range queries at that path use through eql_v4.ord_term. It was derived for every number and string leaf, which made it the largest field of a nested document (24.8 KB of 85 KB) and the one field that reveals something across rows at rest (the order of that leaf's value among every row that holds one), on paths nobody orders. A JSON index now names the paths it orders, with [*] covering every element of an array (a position such as [2] is refused, since -> reads it as [*]); nothing is ordered by default; and an ordering query at a path the index does not order is refused before any key loads, instead of silently matching no row.

Changes

• SQL (packages/eql/src/v3/json_tree/, rendered into EQL v4 only):
  • The entry validator accepts s, c, p and op only, so a document written by the earlier build of this unreleased type is refused.
  • eql_v4.json_tree_subtree_at(doc, token, gates text[], budget DEFAULT 64) replaces the earlier (doc, token, selector, budget) form. It walks the gates and returns the reached node at d = 0 with its descendants. json_tree_subtree(doc, token, budget) is now that function with no gates, and still returns the token's node's descendants from d = 1.
  • json_tree_children and json_tree_child_count take the same optional gates.
  • The budget is unchanged: when the document's root is sealed (its ciphertext holds the whole document) and a walk would return more than budget entries, the walk returns the root entry alone at j = 0, d = 0, and the client opens it and selects the path itself.
  • eql_v4_internal.json_tree_child_token is added; json_tree_unwrap, json_tree_xor16 and json_tree_fanout are removed.
• stack-encrypt (src/json/):
  • JsonEntry is {s, c, p, op}, and JsonDocument carries its projection, which the wire form requires as q.
  • JsonSubtreeKey holds a token and gates(); a position gate is #n:<selector>.
  • JsonDocument::children() probes positions in turn.
  • Rows from a gated walk open to the reached node's value.
  • JsonOptions gains ordered: BTreeSet<JsonPath> (so it is Clone, no longer Copy), JsonOptions::ordering([..]) parses paths into it, and JsonPath::covers is the match rule the walk and the query share. The data-plan wire is {"json": {"sealed_root": true, "ordered": ["$.age", "$.orders[*].total"]}}; the header does not record the set, since a stored leaf shows whether it has a term. JsonHeader::options() is removed and json_subtree_key takes &JsonOptions.
  • TermError::JsonPathNotOrdered (code stack_encrypt::json_path_not_ordered) refuses Lt, Lte, Gt, Gte and Between at an unordered path when the query is awaited.
• stack-encrypt (src/plan/, src/target/): a typed plan had no place for index options, since encrypt_into::<JsonDocument>() reads the type's static EncryptFrom. target::Layout<S> is a target's layout chosen by value, with the index it answers queries through; Json::new(options) implements it, and Plan::value().encrypt_as(layout) builds a ValuePlan<_, Laid<X, T>> that seals under the options and resolves every query through the plan's own index, so a query does not name the options again. Error gains From<JsonPathError>.
• eql-bindings (src/v4/json_tree.rs): JsonTreeEntry drops vs, o, w, n and g, and the typed document hands q to the engine. JsonTree::new(options) is the json_tree layout under options (a Layout<Value> whose target is JsonTreeDocument); the type's own EncryptFrom is that layout under the defaults.
• Tests in all three places are updated for the new fields and the gated walk.
• Design doc (docs/plans/2026-10-17-json-tree-index.md): the Phase 3b implementation note carries the measured outcome below, and a Phase 3c note records the opt-in and its measured sizes. The earlier sections, which still describe the wrapped token, group tags and stored ordinal, are kept as the record, with dated notes where they would otherwise mislead.
• Doc comments that still described the old format are corrected (the header's SQL reader, the projection's source, the walk's cost).
• Changeset: @cipherstash/eql minor.
• Byte encodings (commit 37e02ab42): every field the SQL compares, derives or decodes is padded standard base64 (s, the s/vs pairs of q, p, op, tokens and gates), which PostgreSQL's encode/decode handle natively and which decode refuses unpadded; c is RFC 1924 base85 through the base85-simd 0.1.0 crate (no SQL function decodes c); h stays base64 because the SQL reads the iv and the sealed-root flag from it. p is the first eight bytes of its hash: a lookup key within one document, and the AAD binds s and j, not p. eql_v4_internal.json_tree_ope_cllw is added because the shared ope_cllw(jsonb) of the scalar types reads hex. eql-bindings gets JsonTreeSelector, JsonTreeCiphertext and JsonTreeOrderingTerm newtypes so the v4 wire is documented apart from the v3 hex types. Design doc: a Phase 3d note.

Verification

• cargo nextest run -p stack-encrypt --all-features: 733 passed.

• cargo test -p eql-bindings --all-features json_tree: passed.

• mise run test:encryption:postgres against a fresh database with the v4 bundle built from this tree: all suites passed, including the json_tree oracle, which compares containment over the encrypted column with jsonb @> over the plaintexts (postgres_containment_over_the_encrypted_column_is_jsonb_containment, postgres_extracts_walks_and_orders_by_path).

• After the docs follow-up commit (3860ecc3c): cargo clippy --all-features --all-targets -D warnings and cargo fmt --check clean for stack-encrypt and eql-bindings, mise run test:schema passed, and mise run docs:validate's coverage and required-tags checks passed. Its documented-sql step needs the Docker database on port 7432 and could not connect locally; CI runs it.

• For the opt-in commit (ba62e3fc5): cargo nextest run -p stack-encrypt --all-features 737 passed (new: an_ordering_term_sits_only_at_an_ordered_path, a_laid_out_plan_opens_what_it_wrote, the refusal cases in an_accessor_asks_at_a_path and an_accessor_is_refused_before_its_keyset_loads, the ordered wire round trip); the 46 doctests, including the new encrypt_as example; rustdoc with -D warnings; clippy and fmt clean; the trybuild snapshot op_matches_on_integer.stderr regenerated, its only change being the new Laid impl in rustc's help list. mise run test:encryption (eql-bindings and the encryption test crate, no database) passed. The json_tree PostgreSQL oracle (postgres_extracts_walks_and_orders_by_path, postgres_containment_over_the_encrypted_column_is_jsonb_containment) passed against a database carrying this branch's v4 bundle, with the corpus sealed under $.hello and $.number ordered; the SQL did not change in that commit. Docker was not running locally, so the full test:encryption:postgres task was not re-run. cargo-mutants over the commit's diff found five survivors, all on the new Laid plan shape, pinned by 2e405c462 (ValuePlan::specs added for the test).

• Benchmarks, cipherstash/benches@2d9d5f2 (branch feat/json-tree-benches, report/JSON_TREE_COMPARISON.md, "Third measurement"). They ran at 10,000 and 100,000 rows on one machine (Apple M1 Max, PostgreSQL 17.10), over the same documents before and after, so every scenario returned the same rows. Query-only criterion medians in ms, before → after, on the nested corpus (about 213 entries a document):

  scenario	10,000 rows	100,000 rows
  subtree/orders, default budget	9.48 → 5.71	8.77 → 5.46
  subtree_at/items ($.orders[*].items), default budget	12.95 → 8.21	10.92 → 7.18
  subtree/orders, walked out (budget 0)	22.97 → 10.92	24.57 → 11.35
  subtree_at/items, walked out (budget 0)	23.12 → 10.93	24.83 → 11.27
  subtree/user	3.90 → 2.38	3.99 → 2.45
  path_query/wildcard	2.24 → 1.64	1.80 → 1.41
  field_order/nested	4.81 → 3.81	5.23 → 4.12
  children/orders	1.63 → 1.54	1.64 → 1.49
  child_count/orders	1.62 → 1.53	1.66 → 1.52

  children and child_count moved only 6 to 9%. The report's explanation, not yet tested, is that a per-call fixed cost (detoasting the document and building the parent-tag map) dominates them, and this change does not touch it.

  Containment and field queries are unchanged within noise. The flat corpus read +7% to +13% on byte-identical operands, plans and indexes, and the native jsonb controls over unchanged plaintext drifted +5% to +19% in the same run, so this reads as machine noise rather than the format.

• Sizes:

  • Nested document: 113,458 → 85,157 characters (−25%); sv 94,324 → 66,023 (−30%).
  • Nested pg_column_size: 81,264 → 62,143 bytes.
  • Nested heap: 810 MB → 621 MB at 10,000 rows, 8,093 MB → 6,201 MB at 100,000.
  • GIN index unchanged (40 MB / 358 MB). GIN build 5 s at 10,000 rows; 58 → 65 s at 100,000, one sample.
  • Flat document: 2,346 → 2,136 characters; pg_column_size 2,452 → 2,250 bytes.

• Sizes with the ordering term opt-in (cipherstash/benches@cd9643d, "Fourth measurement"), each corpus re-sealed with the one path its scenarios order ($.age flat, $.company.salary nested), correctness 462 of 462 at both tiers:

  • Nested pg_column_size: 62,143 → 40,578 bytes (−35%); heap 621 → 408 MiB at 10,000 rows, 6,201 → 4,069 MiB at 100,000; GIN unchanged.
  • Flat pg_column_size: 2,250 → 1,596 bytes (−29%); heap 27.1 → 17.0 MiB at 10,000 rows. A flat document now sits under the TOAST threshold, so the flat heap moves into the main relation and the flat GIN grows 12% at 100,000 rows from the wider row spread. Query timings were not re-run.

• Correctness: a new benches task, check:v4:json_tree <N>, ran at both tiers. It decrypted every walk form for 25 random rows and compared each with the plaintext, and it compared the encrypted id sets with native jsonb for 10 needles per predicate. 462 passed, 0 failed, at both tiers.

• For the encodings commit: cargo nextest run -p stack-encrypt --all-features 751 passed; doctests 46 passed; clippy and rustdoc (-D warnings) clean; mise run wasm:wasi-check passed (base85-simd compiles for wasm32-wasip1 on its scalar path); mise run test:encryption:postgres passed against the v4 bundle built from the tree, including the json_tree oracle. Size (1,000 stored rows of each corpus from the Phase 3c tables, re-encoded field by field as the code now writes them, reloaded as jsonb, PostgreSQL 17.10, pg_column_size averaged after VACUUM FULL): nested 40,808 → 31,894 bytes (−22%), flat 1,596 → 1,398 bytes (−12%); wire characters nested 58,111 → 47,038, flat 1,419 → 1,224. The corpora were not re-ingested and no query re-timed for this commit.

Related

• Refs perf(eql, stack-encrypt): store the json_tree projection, walk by parent map, budget the subtree #1325, the base of this stack.
• Refs json_tree: the smaller performance findings from the first benchmarks #1324, whose item 4 (a nested document's size) this partly addresses: the stored document is 25% smaller. The encoding lever listed there is taken in 37e02ab42 (a further 22% off a nested document); the nonce lever is rejected (a caller-chosen nonce would weaken vitaminc's misuse resistance for a 7% saving; design doc Phase 3d), and -> (item 1) is 15 to 19% faster only because sv is shorter.

Review notes

• Start with the walk comment block in functions.sql and json_tree_subtree_at, then subtree_key_for in stack-encrypt/src/json/derive.rs, which decides what the client derives and what it gates.
• The one semantic subtlety is in DecryptInto<Value> for JsonSubtree: a d = 0 row means "the node the gates reached", so its value is returned, rather than a container assembled from the rows. A sealed-root fallback also comes back at d = 0, at j = 0, and is told apart by that.
• Trust is unchanged: only c is authenticated, through its AAD (the associated data the AEAD binds), and q was already what containment ran over. The w mask's binding to c is gone. A swapped ciphertext no longer breaks the walk on the server, but it still fails when the client opens the entry, because c is bound to s and its position j in the document.
• At rest the format leaks less: there are no stored positions to count children from.
• feat(stack-encrypt, eql): the JSON index and the json_tree EQL v4 domain family #1288's changeset still describes the format this PR replaces; it will sit beside this one in the same release's changelog, and this one says what superseded it. perf(eql, stack-encrypt): store the json_tree projection, walk by parent map, budget the subtree #1325's changeset is edited here, to carry the opt-in ordering paragraph.

What a query looks like to a user

Users never see selectors, tokens or gates. The SDK derives them from a path and a plaintext needle, and the only encrypted thing in the SQL text is one typed operand.

SQL, as the v4 bundle ships it, for a column profile of type public.eql_v4_json_tree:

-- containment, served by the GIN index; $1 is the encrypted needle
SELECT id FROM users WHERE profile @> $1::eql_v4.query_json_tree;

-- range at a path, served by a btree on the functional index; $sel is the path selector
SELECT id FROM users WHERE (profile -> $sel) >= $2::eql_v4.query_json_tree_leaf;
SELECT id FROM users ORDER BY eql_v4.ord_term(profile -> $sel);

-- one node, opens on the client
SELECT profile -> $sel FROM users WHERE id = 7;

-- a subtree: the token and the gates this PR introduces; rows open on the client into the value
SELECT e FROM users, eql_v4.json_tree_subtree_at(profile, $tok, $gates::text[]) e WHERE id = 7;

Rust, as stack-encrypt has it on this branch:

// The plan names the paths the index orders; nothing is ordered by default.
let options = JsonOptions::default().ordering(["$.age", "$.orders[*].total"])?;
let plan = Plan::value::<Value>().context("users/profile").encrypt_as(Json::new(options)).build()?;
let q = cipher.query(op::Contains, &json!({"orders": [{"id": 2}]})).on(&plan).await?;
let b = cipher.query(op::AtPath(op::Gte), &("$.age", 18)).on(&plan).await?;   // ordered: derives
// cipher.query(op::AtPath(op::Gte), &("$.name", "m")).on(&plan): refused, `$.name` is not ordered
let k = keyset.json_subtree_key(&"$.orders[*].items".parse()?, &options, column).await?;
// k.token() and k.gates() are the arguments of json_tree_subtree_at
let items: Vec<Value> = cipher.decrypt_as(JsonTreeSubtree::from_key(&k, rows), column).await?;

TypeScript through Drizzle, as the EQL v3 surface reads today; a v4 column keeps it and gains the walk, which has no adapter shape yet:

db.select().from(users).where(await ops.contains(users.profile, { orders: [{ id: 2 }] }))
db.select().from(users).where(await ops.selector(users.profile, "$.age").gte(18))
db.select().from(users).orderBy(ops.asc(ops.selector(users.profile, "$.age")))

Review fixes (2026-10-23)

The review is the comment above. The fixes are on this branch:

• 67b1f543e: the chain step binds the child's selector, H("t" ‖ t(parent) ‖ s ‖ LE32(o)), and a position gate is #n:<selector>. Without the selector, a position could reach an object member in rows whose node at the path is an object. eql_v4_internal.json_tree_child_token now takes (token, s, o).
• d72d9fe15: a path that names an array position is refused in JsonOptions::ordering, in the data plan's ordered, in an ordering query, and in a seal (TermError::JsonOrderingAtPosition).
• d267735af: a json_wire_decode fuzz target, and the fuzz crate's refreshed Cargo.lock.
• d3d7f764b, 151fd5d7c, 6d9ece2e7: the docs in types.sql, both CHANGELOGs, and the design doc's eight-byte p bound.

Two decisions from the owner followed the review, as separate commits:

• f1f27f218: this is option (c) for the walk budget. JsonSubtreeKey carries its path and builds the rows' wrapper: key.open(rows) in stack-encrypt, JsonTreeSubtree::from_key(&key, rows) in eql-bindings. From its gates the key decides whether the rows open below the path's node or at it, so a path-less open no longer compiles.
• 63bec8f71: the json module's plaintext is held under vitaminc's Protected. Node plaintexts are encoded straight into Protected<Vec<u8>>. Opened keys, scalars and documents stay in Protected<PlainJson> until they are handed to the caller. The sealed-root fallback's document is wiped once the path is selected from it.

Verification counts above predate these fixes. With them: cargo nextest run -p stack-encrypt --all-features 764 passed, and mise run test:encryption:postgres passed against PostgreSQL 17.10, including the new postgres_a_position_reaches_array_elements_only.

https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI; Enforced overrides: Organization
• Review profile: CHILL
• Plan: Advanced
• Run ID: 76d1085d-2a72-4d94-91f2-7360c11886d7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Mutation testing (cargo-mutants, --in-diff, stack-auth + stack-encrypt)

caught	missed	unviable	timeout
135	0	81	0

Every mutant in the changed lines was caught by a test.

[changeset-bot]

🦋 Changeset detected

Latest commit: 62f2132

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages

Name	Type
@cipherstash/eql	Minor
stash	Patch
@cipherstash/stack-prisma	Patch
@cipherstash/basic-example	Patch
@cipherstash/e2e	Patch
@cipherstash/prisma-example	Patch
@cipherstash/stack-drizzle	Patch
@cipherstash/stack-supabase	Patch
@cipherstash/stack	Patch
@cipherstash/wizard	Patch
@cipherstash/bench	Patch
@cipherstash/test-kit	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[coderdan]

Review of #1329 (the ordinal chain, the opt-in ordering term and the byte encodings)

Scope: what this PR adds over #1325, git diff origin/perf/json-tree-walk-and-projection...7bf1d7533. Reviewed against the design document's Phase 3b, 3c and 3d notes and the "Known non-findings" in AGENTS.md. Fixes for every finding below follow on this branch; each inline comment gets a reply with its commit.

Some background for a reader new to EQL. A json_tree column stores one encrypted entry per JSON node. Each entry has a selector s (a keyed hash of the node's path, with every array position folded into [*]), a ciphertext c, and below the top-level fields a parent tag p. To read a subtree, the client hands PostgreSQL a token, a secret per path. The server finds the node's children by recomputing their parent tags from the token and each child's ordinal (its position among its siblings), and derives each child's own token the same way. Where the client cannot derive a token itself, it sends gates: a selector, or an array position, that each step down must match.

Findings

#	Severity	Finding
1	High	A position ($.a[0]) can return an object member's data. The ordinal chain dropped the key/index kind byte, so in rows where $.a is an object, the token for $.a[0] is the token of whichever member ranks first.
2	Medium	Ordering with an array position in the path silently compares the wrong element or matches no row. -> reads a position as [*] and returns the first element, so $.scores[2] can never be reached.
3	Low	No fuzz target covers the JSON wire decoders. Since this PR, those decoders run SIMD base85 over stored, unauthenticated text. Separately, the fuzz crate's Cargo.lock was left stale by the new base85-simd dependency.
4	Low	types.sql still calls c base64 and gives p as the whole hash.
5	Low	The CHANGELOGs miss this PR's public API: target::Layout, encrypt_as, Laid, ValuePlan::specs, From<JsonPathError>, the gates, the wire encodings, v4::JsonTree and the eql-bindings newtypes.
6	Nit	The eight-byte p argument counts collisions among stored tags only, not the walk's misses. The conclusion holds.
7	Nit	The PR body is stale in places (below).

Finding 1, in a user's terms. Say a column holds {"a": [{"x": 2}]} in one row and {"a": {"k": {"x": 1}}} in another. The application reads $.a[0] through json_tree_subtree. It gets {"x": 2} from the first row, which is right. From the second row it gets {"x": 1}, the value of $.a.k, presented as $.a[0]. PostgreSQL's jsonb_path_query_first(doc, 'strict $.a[0]') returns NULL there. A gated position ($.h.a[0], sent as the gate #0) has the same fault. The answer also changes with document size: past the walk budget the client opens the sealed root and selects the path over the plaintext, and that answer is correct, so the same row answers differently below and above 64 entries. This is a regression from #1325. That chain bound 0x01 ‖ key or 0x02 ‖ index into each step, which kept elements and members apart.

Finding 2, in a user's terms. Say the index orders $.scores[1]. A query "rows where $.scores[1] >= 5" is accepted. It then matches no row, because -> returns scores[0], which carries no ordering term. That is exactly the silent empty result the new JsonPathNotOrdered refusal exists to prevent. Now say the index orders $.scores[*]. The same query is accepted and compares scores[0], so it returns rows chosen by the wrong element.

Checked and found sound

• The walk ends on a hostile stored document (1325's question). Each recursion row yields at most two probes: down to ordinal 0 or a gated position, and across to the next ordinal. Each probe is one hash lookup. A hit is a distinct (parent token, ordinal) position, and a child's token is a SHA-256 of its parent's, so the chain cannot return to a token it has already visited. The walk therefore returns at most one row per stored tag, unless an attacker holds a token and finds eight-byte tag collisions; each extra row then costs about 2^32 hashes and adds one row, not a loop. The recursive CTE runs iteratively, so depth cannot exhaust a stack. With the budget on, the LIMIT on found stops the recursion early. The 300-entry chain test holds this.
• The rank is keyed (json/rank/v1 under the index key) and only the client computes it. Someone holding a token learns which selector sits at each ordinal. The member order is a keyed pseudo-random permutation, the same in every row for one parent token, so it reveals nothing about key names that the selectors do not already.
• Gates. A selector gate reveals nothing a selector does not already reveal. A gate cannot be forged into anything new or replayed between rows to effect, because the server only uses it to filter hits under a token it was given.
• Encodings. Every field the SQL decodes is padded base64, and the SQL never decodes c. RFC 1924's alphabet contains neither " nor \, and a test pins that. The header's iv sits at byte 22 (substr(…, 23, 16)) and the sealed-root flag at byte 1, which matches JsonHeader::to_bytes. base64ct decodes strictly. base85-simd 0.1.0 is CipherStash's own crate: its non-test code denies unwrap, panic and indexing, and it refuses overflowing blocks and invalid lengths.
• The eight-byte p is a lookup key only. The AAD binds s and j, so truncating p weakens no authentication.
• The newtypes. TryFrom and the node() and document() conversions are total: every failure maps to Error. The only v3 type v4 still uses is KeyHeader.
• No skill mentions json_tree or v4 JSON behaviour.

Outside this diff, for the owner

• At rest, the order of sv ranks key names. The walk visits object members in serde_json's map order. In this workspace that is sorted, since preserve_order is off. So in every row, sibling selectors appear in sv in lexicographic order of their key names, and across many rows the database can sort a column's key selectors alphabetically. The keyed rank hides member order from someone holding a token, but the stored order gives the same information away to anyone who can read the table. This dates from #1288's walk order, not this PR. A fix (seal members in rank order, and heads in a keyed order) changes the format, so I have left it for a decision. The leakage page in #1336 should say either way.
• A convenience for attaching paths (raised in the #1325 review). Whether a walk's rows need below(path) or at(path) is now fully determined by the key: no gates means json_tree_subtree, so below; any gates means at. A helper that takes the JsonSubtreeKey and the path would remove the misuse the default budget exposes. It is new API rather than a defect, so I have not added it here.
• These stay open by decision, and I have not touched them: (a) rows open under the selector the server's row carries; (b) the budget is on by default; (c) q is trusted as written.

PR body

Several references to "the last commit" and "the docs follow-up commit" no longer locate anything after the merges. The verification counts predate #1288's and #1325's fixes. The Rust example opens JsonTreeSubtree::new(rows) without .at(path), which fails once a document passes the budget. And "The changesets from #1288 and #1325 still describe the format" is no longer true: this PR edits #1325's changeset. I will update the body.

Checks run on the fixed branch

cargo nextest run -p stack-encrypt --all-features (759 passed), doctests (46 passed), mise run doc:stack-encrypt, clippy with -D warnings both with all features and with none, cargo fmt --check (root and EQL workspaces), mise run test:encryption, mise run test:encryption:postgres against PostgreSQL 17.10 (every suite passed, including the json_tree oracle and the new positional test), the eight EQL static checks, pnpm run test:scripts, the new fuzz target for two minutes, and cargo mutants --in-diff over the fix (17 mutants: 13 caught, 4 unviable, none missed).

[coderdan]

[High] A position can return an object member's data.

The step is H("t" ‖ t(parent) ‖ LE32(o)). The ordinal o is an array element's position, but for an object member it is the member's rank among its siblings, and nothing in the step says which kind of child it is. Rows of one column need not agree on a node's kind. So in a row whose $.a is an object, the token the client derives for $.a[0] here (in subtree_key_for, by child_token(&token, o)) is the token of the member that ranks first. json_tree_subtree then returns that member's subtree, and the client opens it as $.a[0].

For example, {"a": {"k": {"x": 1}}} opens to {"x": 1} at $.a[0], where jsonb_path_query_first(doc, 'strict $.a[0]') is NULL. Below a key step, the #n gate admits the same member: the SQL passes any hit at ordinal n (OR COALESCE(gates[probe.depth], '') LIKE '#%'). Past the 64-entry budget, the sealed-root fallback selects the path over the plaintext and answers correctly. So the same row answers differently depending on the document's size.

#1325's chain could not do this, because its step bound 0x01 ‖ key or 0x02 ‖ index. Fix: bind the child's own selector into the step, H("t" ‖ t(parent) ‖ s(child) ‖ LE32(o)). The server has every hit's s. The client knows an element's selector from the path: it is the wildcard's, and a member's selector names its key, so the two never collide. Also make a position gate carry the element selector, #n:<selector>, and admit the hit only if it carries that selector.

[coderdan]

Fixed in 67b1f54. The chain step now binds the child's selector: H("t" ‖ t(parent) ‖ s(child) ‖ LE32(o)), in Rust and in eql_v4_internal.json_tree_child_token(token, s, o). A position gate is #n:<selector> and admits the hit only if it carries that selector; a bare #n admits nothing. New tests: a_position_reaches_no_object_member (engine) and postgres_a_position_reaches_array_elements_only (both forms against jsonb_path_query_first(plain, 'strict …')). The design doc's Phase 3b note and the changeset record the change.

[coderdan]

[Medium] An ordering at a position compares the wrong element, or matches no row.

An ordering query compares the leaf that col -> sel extracts, and sel is the path's selector. A selector reads every position as [*] (json_tree_path_query_first returns the first entry that carries it). So -> reaches the first element at the path and never the one a position names. That leaves two failure modes:

• The index orders $.scores[1], and the query at $.scores[1] is accepted. -> returns scores[0], which has no op, so ord_term is NULL and the comparison matches no row. That is the silent empty result this function's refusal exists to prevent.
• The index orders $.scores[*], and a query at $.scores[2] is accepted (covers lets a wildcard cover a position). It compares scores[0] and returns rows selected by the wrong element.

The test an_accessor_asks_at_a_path asserts that the first case is accepted. Fix: refuse a path that names a position, here and in JsonOptions::ordering and the data plan's ordered, with its own error that says why. Making positions work would need an extractor that can reach a position, which -> by selector cannot.

[coderdan]

Fixed in d72d9fe. The new TermError::JsonOrderingAtPosition (code stack_encrypt::json_ordering_at_position) refuses a path that names a position in four places: JsonOptions::ordering (which now returns TermError), the data plan's ordered, an ordering query (before any keyset loads), and a seal under options whose ordered field was given one directly. The design doc's Phase 3c note and #1325's changeset paragraph now say a position is refused.

[coderdan]

[Low] The JSON wire decoders have no fuzz target, and the fuzz crate's lock is stale.

A stored document, and every entry that -> or a walk returns, comes back from the database through JsonDocument's and JsonNode's wire decoders before anything is authenticated. That path now runs base85_simd::decode, which is NEON and AVX2 unsafe code, over every c. It also runs base64ct over every tag and JsonHeader::from_bytes over h. docs/fuzzing.md says the repo fuzzes "bytes read back from storage", and none of these decoders is covered.

Separately, packages/stack-encrypt/fuzz is a detached workspace with its own Cargo.lock, and this dependency was not added to it: cargo metadata --locked there exits 101. cargo fuzz run regenerates the lock in memory, so CI stays green. That is the same silent drift AGENTS.md describes for --locked.

Fix: add a json_wire_decode target. It should require that an accepted document or node round-trips through its wire form, ship seeds, and be wired into both fuzz.yml jobs. Also commit the refreshed lock.

[coderdan]

Fixed in d267735. The new json_wire_decode fuzz target covers JsonHeader::from_bytes and the JSON wire forms of JsonDocument, JsonNode, JsonProjection and JsonLeafQuery, and requires an accepted document or node to round-trip. It has five seeds, a fuzz:json-wire task, entries in both fuzz.yml jobs and a row in docs/fuzzing.md. It ran two minutes locally (about 8.9M executions) with no crash. The commit also refreshes the fuzz crate's Cargo.lock, which cargo metadata --locked now accepts.

[coderdan]

[Low] Stale field docs. Since the encodings commit, c is RFC 1924 base85, which no SQL function decodes, not base64. A few lines down, p is the first eight bytes of the hash in base64, not the whole hash. This comment block is the first description of the stored fields a reader of the SQL sees.

[coderdan]

Fixed in d3d7f76.

[coderdan]

[Low] The CHANGELOGs miss this PR's public API. This entry describes the JSON index, but nothing in [Unreleased] covers what the PR adds beside it:

• target::Layout
• Plan::value().encrypt_as and plan::Laid
• ValuePlan::specs and ValuePlan::layout
• From<JsonPathError> for Error
• JsonSubtreeKey::gates(), which replaces selector()
• JsonOptions::ordering and JsonPath::covers
• the base64 and base85 wire encodings

eql-bindings' CHANGELOG likewise misses v4::JsonTree (the layout under options) and the JsonTreeSelector, JsonTreeCiphertext and JsonTreeOrderingTerm newtypes.

[coderdan]

Fixed in 151fd5d (both CHANGELOGs).

[coderdan]

[Nit] The bound leaves out the walk's misses. "Distinct among the entries of one document" is half the requirement. Every miss probe (one per visited node, at the end of its children) must also miss every stored tag. Otherwise an unrelated entry comes back as a child. That entry is authentic and opens, and the rebuild's checks of child counts and element positions catch it in most shapes but not all. The probability is the same order, about N² / 2^64 per walk and row, so the conclusion stands, but the note should say so.

[coderdan]

Fixed in 6d9ece2.

[coderdan]

The CI mutants gate failed on 63bec8f. Two mutants survived in the position arm of JsonPath::covers (path.rs:191). Once d72d9fe made ordered paths and ordering queries refuse positions, no test reached that arm any more, although covers is public and still matches positions. 1a86672 adds a_position_covers_its_own_position_only. Run locally over covers, cargo-mutants reports 9 mutants, 9 caught. #1336 is merged up to the new head (0d441a2).

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-23T13:34:29.940907Z	62f2132	Draft marked ready
🔒 Security Review	✅ Completed 2026-10-23T13:35:20.894313Z	62f2132	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 62f213233b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Reject gate positions outside PostgreSQL integer range

For a gated path such as $.items[2147483648], this conversion succeeds because the position fits u32, and the key serializes it as #2147483648:...; both json_tree_subtree_at and json_tree_children then cast that gate ordinal with ::integer in functions.sql, causing PostgreSQL error 22003 instead of returning no rows for the nonexistent position. Limit gated positions to i32::MAX or widen the SQL ordinal type consistently.

AGENTS.md reference: AGENTS.md:L684-L695

Useful? React with 👍 / 👎.