Skip to content

docs(eql): update the reference to the EQL 3.0.0 shapes#67

Open
coderdan wants to merge 2 commits into
v2from
docs/eql-3.0.0-shapes
Open

docs(eql): update the reference to the EQL 3.0.0 shapes#67
coderdan wants to merge 2 commits into
v2from
docs/eql-3.0.0-shapes

Conversation

@coderdan

@coderdan coderdan commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Updates the EQL reference to the type and term shapes landing in EQL 3.0.0, from encrypt-query-language #391 (version-prefixed public domains) and #389 (OPE as the default ordering mechanism).

Verified against the generated schema in crates/eql-bindings/schema/v3 and the domain SQL in src/v3, not against the PR descriptions. That matters here: #389's own body says text_search keeps [hm, ob, bf], but the schema on its branch says [hm, op, bf].

What changed

Version-prefixed domains. public.text_ord is now public.eql_v3_text_ord. 174 names across 15 pages. Query-operand domains already live in the versioned eql_v3 schema, so they stay unprefixed.

Ordering defaults to CLLW OPE.

Variant Term Extractor
_ord op eql_v3.ord_term
_ord_ope op eql_v3.ord_term (byte-identical twin of _ord)
_ord_ore ob eql_v3.ord_term_ore
text_search hm, op, bf eq_term / ord_term / match_term
text_search_ore hm, ob, bf eq_term / ord_term_ore / match_term

eql_v3.ord_ope_term is gone; eql_v3.ord_term took its name.

ORE needs superuser. An OPE term is a bytea domain, so an index on eql_v3.ord_term(col) binds the default bytea_ops. Block-ORE needs a hand-written operator class, and CREATE OPERATOR CLASS needs superuser. Where the installer runs as a non-superuser (cloud Supabase, most managed Postgres) it cannot create the class, so it disables every ORE-backed domain: the types exist, but a CHECK rejects the first value written with feature_not_supported. The pages previously said the domains were "not installed at all", which was wrong. Every page offering the choice now carries this callout.

An equality claim that was wrong in both directions. The old pages said equality on any _ord variant compares the ORE term, so _ord payloads carry no hm. Half right:

  • eq(bigint_ord) is ord_term(a) = ord_term(b). No hm. Ordering over the numeric scalars is equality-lossless.
  • eq(text_ord) is eq_term(a) = eq_term(b). Text carries hm on every orderable variant, because OPE over text is not equality-lossless.

Corrected in core-concepts, filtering, joins, grouping-and-aggregates, indexes, and the three type pages.

Also: payload examples now show op as a single hex string rather than a block array; the <T> placeholder meant bigint on one page and eql_v3_bigint on another, now consistently the former; json.mdx no longer date-stamps the oc to op rename against an alpha.

Blocked on the EQL 3.0.0 release

bun run build runs the drift check, which validates every schema-qualified symbol against the pinned manifest. The pin is eql-3.0.0-alpha.4, which predates both PRs: it still has unprefixed domains (public.bigint) and ord_ope_term. So the check reports 53 unknown symbols and the build fails.

All 53 were cross-checked against the SQL on the release branch and every one exists:

comm -23 <(sort -u unknown.txt) <(grep -rhoE '\b(public|eql_v3_internal|eql_v3)\.[a-z0-9_]+' src/v3 | sort -u)
# (empty)

To unblock: once EQL 3.0.0 publishes, bump EQL_RELEASE_TAG in scripts/generate-eql-docs.ts from eql-3.0.0-alpha.4 to eql-3.0.0. That is the only change needed; CI should then go green.

Checks

  • validate-content: pass
  • validate-links: pass
  • validate-mermaid: pass
  • lint: fails on pre-existing noNonNullAssertion in scripts/generate-cli-docs.ts, unrelated and present on the base commit
  • generate-docs:eql-api: fails as described above

One drive-by fix to the drift checker: it was extracting public.eql_v3_ as a symbol from the public.eql_v3_<T> prose placeholder. It now skips a match followed by <.

Rewrites the EQL reference against the type and term shapes landing in
EQL 3.0.0 (encrypt-query-language #391 and #389), verified against the
generated schema and the domain SQL rather than against the PR bodies.

Public domain types are now version-prefixed: `public.text_ord` becomes
`public.eql_v3_text_ord`. 174 names across 15 pages.

Ordering now defaults to CLLW OPE:

- `_ord` carries `op` and extracts with `eql_v3.ord_term`
- `_ord_ope` is its byte-identical, explicitly pinned twin
- `_ord_ore` carries `ob` and extracts with `eql_v3.ord_term_ore`
- `text_search` is `[hm, op, bf]`; the new `text_search_ore` is `[hm, ob, bf]`
- `eql_v3.ord_ope_term` is gone; `eql_v3.ord_term` took its name

OPE terms sort under `bytea_ops`, so ordering no longer needs a custom
operator class. Where the installer runs as a non-superuser it cannot
create one, and it disables the ORE-backed domains rather than let them
install half-working. Documented on every page that offers the choice.

Also corrects an equality claim the old pages had wrong in both
directions. Equality on the non-text `_ord` variants compares the
ordering term (those payloads carry no `hm`), but every text variant
carries `hm` and compares that, because ordering over text is not
equality-lossless. Confirmed in the domain SQL: `eq(bigint_ord)` calls
`ord_term`, `eq(text_ord)` calls `eq_term`.

The drift check stays red until EQL 3.0.0 publishes, because the pinned
manifest is alpha.4, which predates both PRs. Every symbol this commit
introduces was cross-checked against the SQL on the release branch.
@vercel

vercel Bot commented Jul 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
public-docs Ready Ready Preview, Comment Jul 9, 2026 3:17pm

Request Review

Bumps EQL_RELEASE_TAG from eql-3.0.0-alpha.4. The drift check now passes
against the real manifest, which confirms every domain and function this
branch renamed.

Two follow-ons the bump exposed:

- The 40 scalar query-operand domains (`eql_v3.query_text_eq` and its
  siblings) are created inside a `DO ... EXECUTE` block, which the
  manifest's catalog generator does not walk, so the drift check rejected
  a correct reference. `eql_v3.query_jsonb`, created at the top level, is
  present. Added a narrow, documented allowlist rather than weakening the
  check; verified against the release SQL.
- core-concepts named `public.text` / `public.json` / `public.integer`
  as the Postgres built-ins the `eql_v3_` prefix avoids shadowing. The
  drift check reads any `public.<name>` as an EQL domain reference. Say
  the same thing without schema-qualifying the built-ins.
@coderdan

coderdan commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

EQL 3.0.0 is released, so the blocker is gone. Bumped EQL_RELEASE_TAG to eql-3.0.0 and the drift check now passes against the real manifest, which independently confirms all 174 renamed domains and functions on this branch.

Two things the bump exposed, both fixed here:

  • The 40 scalar query-operand domains (eql_v3.query_text_eq and siblings) are created inside a DO ... EXECUTE block, which the manifest's catalog generator doesn't walk. eql_v3.query_jsonb, created at the top level, is in the manifest. So the drift check rejected a correct reference. Rather than weaken the check I added a narrow allowlist with the verification command inline (grep -c "CREATE DOMAIN eql_v3.query_" cipherstash-encrypt.sql → 40). Worth fixing in the EQL catalog generator eventually; the allowlist is there to be deleted.
  • core-concepts cited public.text / public.json / public.integer as the built-ins the eql_v3_ prefix avoids shadowing. The drift check reads any public.<name> as an EQL domain, so it flagged them. Reworded to name the built-ins without schema-qualifying them, which reads better anyway.

bun run build passes end to end. Ready to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant