Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/eql-v4-date-timestamp-numeric.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@cipherstash/eql': minor
---

**Stack Encrypt produces the EQL v4 date, timestamp and numeric types `Date`, `DateEq`, `Timestamp`, `TimestampEq`, `Numeric` and `NumericEq`** (`eql-bindings`, `stack-encrypt` feature). Their Rust plaintexts are the crate's own `v4::plaintext::Date`, `Timestamp` and `Decimal`, which hold the integers the engine stores (days since 1970-01-01; Unix seconds and nanoseconds; a mantissa and a scale), are range checked when built and are wiped when dropped. The `chrono` and `rust_decimal` features convert to and from those crates' types. `=` on `NumericEq` normalises the scale (`1.5 = 1.50`) and on `TimestampEq` compares to the microsecond, truncating (never rounding) the nanoseconds below it; each value decrypts as it was stored. The ordered types of these families stay unproducible until the engine derives their order terms. The Go SDK's `encrypt/eql` table marks the six producible, from `encrypt.Date`, `time.Time` and `encrypt.Decimal`.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
status: accepted
date: 2026-10-08
revised: 2026-10-12, amended 2026-10-18
revised: 2026-10-12, amended 2026-10-18 and 2026-10-19
relates-to: ADR-0001; stack-encrypt ADR-0003, ADR-0007
---

Expand Down Expand Up @@ -30,6 +30,10 @@ relates-to: ADR-0001; stack-encrypt ADR-0003, ADR-0007
> design (see "Exhaustive handling of kinds").
>
> The bullets below that this changes say so where they stand.
>
> **Amended 2026-10-19.** A timestamp's microsecond resolution truncates
> rather than rounding half to even, as vitaminc does for both the equality
> and the order term. The reason, and the rule, are under "Lossy transforms".

Every value Stack Encrypt handles is encoded three times: as a **ciphertext**
(reversible and self-describing), as an **equality term** (a keyed hash that
Expand Down Expand Up @@ -194,13 +198,22 @@ domain: timestamp resolution, text folding and prefix truncation. Each
transform and its version is named in the domain label, for example
`…/timestamp/day/v1`. The ones defined now:

- **Timestamp resolution**: exact, microsecond or day. Microsecond rounds to
the nearest microsecond, ties to even, on the signed microsecond count
- **Timestamp resolution**: exact, microsecond or day. ~~Microsecond rounds
to the nearest microsecond, ties to even, on the signed microsecond count
since the epoch. That is what Postgres does when it parses a timestamp with
more digits than it keeps (`.1234565` becomes `.123456` and `.1234575`
becomes `.123458`, before and after 1970; checked on Postgres 17.5). A
value that already has microsecond precision is unchanged. Day takes the
UTC calendar day.
becomes `.123458`, before and after 1970; checked on Postgres 17.5).~~
*Amended 2026-10-19:* microsecond truncates: the nanoseconds below the
microsecond are dropped, never rounded, which is floor on the signed count
since the epoch (`.1234567` becomes `.123456`, and before 1970 an instant
moves earlier). vitaminc applies this rule to the equality term and to the
order term alike (vitaminc-ore's timestamp domain is
`vitaminc/ore/value/timestamp-micros/v1`, and both terms normalise through
`vitaminc_normalize::timestamp`), so `=` and the order agree on which
Comment thread
coderdan marked this conversation as resolved.
instants are one. Matching PostgreSQL's parse-time rounding is not a goal:
timestamps are stored in many places besides PostgreSQL, and a term must
mean the same thing whichever store holds it. A value that already has
microsecond precision is unchanged. Day takes the UTC calendar day.
- **Text fold, v1**: NFD, then full Unicode case folding (`CaseFolding.txt`
statuses C and F, so `ß` becomes `ss`), then NFD again, then removal of
every accent, then NFC. An accent is a mark (General_Category `Mn`, `Mc`
Expand Down
9 changes: 9 additions & 0 deletions languages/golang/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,15 @@ listed under **Breaking**. Releases are tagged `languages/golang/vX.Y.Z`.

### Added

- **The date, timestamp and numeric EQL v4 types are producible** (#1130):
`Date` and `DateEq` from an `encrypt.Date`, `Timestamp` and `TimestampEq`
from a `time.Time`, and `Numeric` and `NumericEq` from an
`encrypt.Decimal`, with `encrypt_into=`. The `Eq` types answer `Eq`:
`NumericEq` normalises the scale (`1.5 = 1.50`) and `TimestampEq` compares
to the microsecond (truncated, never rounded), while each value opens as it was sealed. A
`time.Time` seals its instant and opens in UTC. Their `Ord`, `OrdOpe` and
`OrdOre` types stay unproducible until the engine derives order terms
for these kinds (#1236).
- **Every integer width, 128-bit integers, dates, timestamps and decimals
as sealed fields** (#1144). Each integer type seals at its own width
(`int8` as an int8, `uint16` as a uint16), so `record.Kind` and
Expand Down
6 changes: 5 additions & 1 deletion languages/golang/cmd/stashgen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,8 +165,12 @@ The engine produces these EQL types today, each from one Go field type:
| `Boolean` | `bool` |
| `Real`, `RealEq`, `RealOrd`, `RealOrdOpe` | `float32` |
| `Double`, `DoubleEq`, `DoubleOrd`, `DoubleOrdOpe` | `float64` |
| `Date`, `DateEq` | `encrypt.Date` |
| `Timestamp`, `TimestampEq` | `time.Time` |
| `Numeric`, `NumericEq` | `encrypt.Decimal` |

The `Eq` types answer `Eq`; the `Ord` and `OrdOpe` types answer `Lt`, `Lte`, `Gt` and `Gte`, and on text `Eq` too; `Text`, `Smallint`, `Integer`, `Bigint`, `Boolean`, `Real` and `Double` are stored and read only, with no query method.
The `Eq` types answer `Eq`; the `Ord` and `OrdOpe` types answer `Lt`, `Lte`, `Gt` and `Gte`, and on text `Eq` too; `Text`, `Smallint`, `Integer`, `Bigint`, `Boolean`, `Real`, `Double`, `Date`, `Timestamp` and `Numeric` are stored and read only, with no query method.
The ordered date, timestamp and numeric types are refused: the engine derives no order term for these kinds yet.
An ordered type has no `Between` method yet, because a `BETWEEN` operand needs EQL v4's range query domains.

Any built-in Go integer field takes an integer type (ADR-0002) and seals at the type's width: an `int16` field always fits an `Integer`, and an `int64` field is range checked value by value, `Encrypt` refusing a value that does not fit with `encrypt.ErrEncoding`. `encrypt.Int128` and `encrypt.Uint128` are not converted. A float's width must match: a `float64` field is not a `Real`.
Expand Down
22 changes: 19 additions & 3 deletions languages/golang/encrypt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ index: with 256 values, an order term would rank every row among 256, which
a few known rows resolve to the values, as a `bool`'s would say which rows
are true. Equality follows vitaminc's normalisation: an
integer equals itself at every width, decimals compare at any scale
(`1.50 = 1.5`), and a timestamp compares to the microsecond. An order index on
(`1.50 = 1.5`), and a timestamp compares to the microsecond, truncated, never rounded. An order index on
a decimal, date or timestamp is refused for now: the engine derives no
order term for those kinds yet.

Expand Down Expand Up @@ -493,6 +493,12 @@ The EQL types the engine produces, and the Go field types each takes:
| `Double` | `float64` | none |
| `DoubleEq` | `float64` | `=`, `<>` |
| `DoubleOrd`, `DoubleOrdOpe` | `float64` | `=`, `<>`, `<`, `<=`, `>`, `>=`, `ORDER BY` |
| `Date` | `encrypt.Date` | none |
| `DateEq` | `encrypt.Date` | `=`, `<>` |
| `Timestamp` | `time.Time` | none |
| `TimestampEq` | `time.Time` | `=`, `<>` (to the microsecond, truncated) |
| `Numeric` | `encrypt.Decimal` | none |
| `NumericEq` | `encrypt.Decimal` | `=`, `<>` (`1.5 = 1.50`) |

Any built-in Go integer field takes an integer EQL type (ADR-0002): the field
is sealed at the type's width, and opens back into the field's type. A type
Expand All @@ -508,7 +514,17 @@ A `TextMatch` value must be at least three characters long: shorter text has
no 3-grams to index and is refused. The number equality types compare as
PostgreSQL compares the plain types: an integer equals itself whatever its
width, and for floats `-0.0 = 0.0` and `NaN = NaN`; the stored value still
decrypts to exactly what was written.
decrypts to exactly what was written. The same holds for the date,
timestamp and numeric types: `NumericEq` normalises the scale, so `1.5` and
`1.50` are equal, and `TimestampEq` compares to the microsecond, truncating
(never rounding) the nanoseconds below it, so two instants in one
microsecond are equal; each still opens as it was written, a
decimal with its scale and a `time.Time` with its nanoseconds, in UTC. A
`time.Time` is sealed as its instant, whatever its location. A zero
`encrypt.Date`, a date or instant outside years -262143 to 262142 and a
decimal with more than 28 digits after the point are refused with
`ErrEncoding`. The ordered date, timestamp and numeric types are not
produced yet: the engine derives no order term for these kinds.

The `Ord` and `OrdOpe` types store an OPE term, which EQL compares as bytes,
and order by ADR-0002's order key: integers by value, floats as PostgreSQL's
Expand All @@ -526,7 +542,7 @@ answers `Eq` through an equality term only. A text order type also stores
neither `<` nor `>` each other and not `=` either. Text holding a code point
Unicode 16 does not assign has no order term and is refused. There is no
`Between` method yet: a `BETWEEN` needs EQL v4's range query domains. The rest of
the catalog (ORE ordering, dates, decimals, JSON) is listed in
the catalog (ORE ordering, ordered dates, timestamps and decimals, JSON) is listed in
`eql.Types` with the reason it is not produced yet, and `stashgen` refuses it
with that reason. The guest builds the value: the generated code
imports `encrypt/eql`, which embeds the build of the engine that holds the
Expand Down
12 changes: 8 additions & 4 deletions languages/golang/encrypt/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,19 @@
// encrypt_into names the EQL type the field is stored as. The Go field type
// must match it: TextEq, TextMatch and Text take a string, Smallint and
// SmallintEq an int16, Integer and IntegerEq an int32, Bigint and BigintEq
// an int64, Boolean a bool, Real and RealEq a float32, and Double and
// DoubleEq a float64. An integer EQL type
// an int64, Boolean a bool, Real and RealEq a float32, Double and DoubleEq a
// float64, Date and DateEq a [Date], Timestamp and TimestampEq a time.Time,
// and Numeric and NumericEq a [Decimal]. An integer EQL type
// also takes any other built-in Go integer type, sealed at the type's width:
// one every value of which fits it (an int16 into Integer) always seals, and
// a wider one (an int64 into Integer) is range checked, a value that does not
// fit refused with [ErrEncoding]. The *Eq types answer Eq; the *Ord and
// *OrdOpe types (IntegerOrd, SmallintOrd, BigintOrd, RealOrd, DoubleOrd,
// TextOrd and their OrdOpe twins) answer Lt, Lte, Gt and Gte, and Eq too on text; Smallint,
// Integer, Bigint, Boolean, Real, Double and Text are stored and read only.
// TextOrd and their OrdOpe twins) answer Lt, Lte, Gt and Gte, and Eq too on
// text; Smallint, Integer, Bigint, Boolean, Real, Double, Text, Date,
// Timestamp and Numeric are stored and read only. NumericEq compares
// decimals with their scale normalised (1.5 = 1.50), and TimestampEq
// compares instants to the microsecond, truncated, never rounded.
//
// 3. Run the generator. It writes user_stash.go beside the struct:
//
Expand Down
Loading
Loading