Skip to content

fix(localized): read an unset field of a request without a locale as null - #111

Merged
Upd4ting merged 6 commits into
mainfrom
fix/localized-unset-null
Nov 10, 2026
Merged

Upd4ting merged 6 commits into
mainfrom
fix/localized-unset-null

Conversation

@Upd4ting

@Upd4ting Upd4ting commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Why

A DMS create request that leaves out a @Localized() field, and sends no x-content-language header, used to answer that field as null (0.2.1). Since 0.2.2 the field is missing from the answer, while every other unset field still answers null. DMS's integration test files/route-answers fails on POST /api/native-attachments/new with 0.2.2 and 0.3.0, and that blocks dms#852. After this change the field answers null again, and the key undefined still stores nothing, as #100 made it.

What changed

  • interface-data-api locks and unlocks a localized field with the request's locale as the key, which is undefined when the header is missing.
  • In 0.2.1 lock stored the unsent value under the literal key "undefined". MongoDB wrote that undefined as null, and unlock(…, undefined) read the null back. fix(localization): refuse a non-record value under the "*" key #100 stopped lock from storing anything under no key, so unlock found nothing and answered undefined, which JSON leaves out.
  • LocalizationModifier.unlock (src/modifiers/localization.ts) now answers the fallbackLocale value for no key, else null. That matches unlockrequest, which already answers null for no key in queries. A defined key reads as before. The base ContainerModifier.unlock still answers undefined for no key.
  • The null broke a field with @Localized() over @Encrypted(): EncryptionModifier.unlock (src/modifiers/encryption.ts) destructured it and threw. It now reads null as no value and returns null. The shared modifier chain is unchanged, so custom modifiers read as before.
  • docs/3.table-modifiers.md and the shipped skill say a no-key read of a localized field answers null when it has no fallback value.
flowchart TD
  A["read a @Localized field with key undefined"] --> B{"fallbackLocale has a value?"}
  B -- yes --> C["that value"]
  B -- no --> D["null"]
  C --> E["@Encrypted decrypts it"]
  D --> F["@Encrypted passes null through"]
Loading

Proof

src/tests/localized_no_key.test.ts creates a row without a locale, as interface-data-api does for a request without the header: assign, lock(…, undefined), insert into MongoDB, read back, unlock(…, undefined), JSON.stringify. It runs with pnpm exec ajs module test . -f dist/tests/localized_no_key.test.js.

Before, with src/ from origin/main (1d74d32):

  Localized fields - a request without a locale
    1) answers null for a field the request did not send

  0 passing (65ms)
  1 failing

  1) Localized fields - a request without a locale
       answers null for a field the request did not send:
     AssertionError: expected { _id: 'page-1', _instance: null } to have deep property 'title'

With only the LocalizationModifier.unlock change (628bb9e), the encrypted localized field threw:

  1) Localized fields - a request without a locale
       answers null for a field the request did not send:
     TypeError: object null is not iterable (cannot read property Symbol(Symbol.iterator))
      at EncryptionModifier.unlock (dist/modifiers/encryption.js:98:11)

After, on this branch, with the suites that pin the #100 no-key behaviour:

  Localized fields - a request without a locale
    ✔ answers null for a field the request did not send
  ...
    ✔ answers the fallback locale for no key
    ✔ answers the fallback locale when no locale is given
    ✔ stores nothing and finds nothing under no key
  41 passing (81ms)

pnpm run checks: Checks: 11 passed, 0 failed.

Attention

A value a request sends with no locale is still not stored, as since #100. In 0.2.1 it was stored under the "undefined" key and read back by the next request without a locale. This PR does not bring that back.

@Upd4ting
Upd4ting merged commit 34d8ab3 into main Nov 10, 2026
8 checks passed
@Upd4ting
Upd4ting deleted the fix/localized-unset-null branch November 10, 2026 23:09
@Upd4ting

Copy link
Copy Markdown
Member Author

Known follow-ups (accepted for this patch)

Round 3 of the review left two findings, accepted as-is for 0.3.1 and tracked as a follow-up:

  1. A two-way modifier stacked after @Localized() now receives null on a no-key read. Before, the value was undefined and the modifier was skipped. A custom modifier, or a subclass of EncryptionModifier that overrides unlock, can throw there, and so can toJSON for the row. The built-in @Encrypted handles it. No AntelopeJS repo defines its own two-way modifier today.
  2. EncryptionModifier.unlock's published parameter widens to LockedType | null, so a type derived as Parameters<EncryptionModifier["unlock"]>[0] now includes null. An override with the old type still compiles.

Both come from producing null inside the modifier chain instead of at the end of it. The follow-up is to make null the end-of-read default in ModifiersDynamicMetadata.get, in a minor release, because a missing locale with no fallback would then read null instead of undefined.

Upd4ting added a commit that referenced this pull request Nov 11, 2026
…112)

## Summary

`PlainData<T>` says that `fromPlainData` can read a localized field as
one value per locale. The runtime did not do that:

- `fromPlainData({ displayName: { en, fr } }).localize("en")` stored `{
en: { en, fr } }`, so the record ended up nested under one locale.
- With no `localize()` call, the record stayed floating, and `insert`
stored no text.

`LocalizationModifier` now has a `fromPlainData` event handler. When the
plain data gives a plain object for a localized field, the handler
unlocks that field under `"*"`, so the object is locked as the field's
texts by locale. `attachModifier` registers the method as an event.
`insert`, `update` and `toDatabase` call `fromPlainData`, so they get
the same behaviour.

The handler acts only when Localization is the field's last modifier.
With `@Localized() @Encrypted()`, encryption runs inside the locales.
Locking under `"*"` would then store the cipher triple as locales `{0,
1, 2}`, and that row could never be read back. Such a field keeps its
old behaviour.

## Release

This is a patch. It makes the runtime match the published `PlainData`
type, and it changes no signature.

These callers can see a difference:

- A caller that passes a plain object for a localized field through
`fromPlainData`, `insert`, `update` or `toDatabase`. Before, the object
was nested under a locale or dropped. Now every locale is stored. On
`update`, the given record replaces the stored texts.
- A localized field whose own value is an object, such as a localized
`ImageValue`, given as a single value. Before, `fromPlainData({ image: {
key, alt } }).localize("en")` stored `{ en: { key, alt } }`. Now `key`
and `alt` are read as locales. `PlainData` already allows both readings,
and this PR picks the one the type documents. The DMS data API builds
its write instance itself and does not call `fromPlainData`, so it is
not affected. The one DMS `fromPlainData` call on a localized table
(`attachments/table-view.ts`) runs `unlock(…, "*")` straight after, so
its result does not change.

Rows that are already stored do not change. Only new writes that carry a
plain object for a localized field store a different shape.

## Scope

- Out of scope: ANT-1971 (null at the end of a read).
- Known gap, already on `main`: for `@Localized() @Encrypted()`,
`fromPlainData(…).localize("en")` on a new row throws in
`ModifiersDynamicMetadata.set`, for a single text too. That is a
separate bug.

## Related issue

ANT-1973, found by ANT-1912 (dms-saas).

## Verification

The new MongoDB suite `src/tests/localized_plain_data.test.ts` was run
through `ajs module test`.

Before, on `main` with only the test commit:

```
  Localized fields - plain data with values by locale
    1) stores every locale of a record read under one locale
    2) stores every locale of a record read with no locale
    ✔ stores a plain value under the locale it is read with
    ✔ stores a null value under the locale it is read with

  1) AssertionError: expected { Object (en) } to deeply equal { en: 'Members', fr: 'Membres' }
       -  "en": { "en": "Members", "fr": "Membres" }
  2) AssertionError: expected undefined to deeply equal { en: 'Members', fr: 'Membres' }
```

After:

```
  Localized fields - plain data with values by locale
    ✔ stores every locale of a record read under one locale
    ✔ stores every locale of a record read with no locale
    ✔ stores a plain value under the locale it is read with
    ✔ stores a null value under the locale it is read with
    ✔ stores no record for a field encrypted inside its locales

  5 passing
```

- The plain-value and null cases pass on `main` and after the fix. They
guard against a handler that also locks a string or `null` under `"*"`,
which would throw a `TypeError` or clear the field.
- The encrypted case fails with the guard removed (`expected { …(3) } to
equal undefined`) and passes with it.
- The #100–#103 and #111 suites still pass: `row_forms`,
`localized_no_key`, `localized_query_keys`, `modifiers/localization` and
`partial_update`. That includes the `undefined`-key behaviour.
- `pnpm run checks`: `Checks: 11 passed, 0 failed.`

```mermaid
flowchart LR
  P["plain data { displayName: { en, fr } }"] --> A[assign fields]
  A --> E{fromPlainData event}
  E -->|"plain object, Localization last"| S["unlock field under '*'"] --> L["lock { en, fr }"]
  E -->|"text, null, or inner modifier"| F[value floats until localize]
```

## Checklist

- [x] The pull request title follows Conventional Commits.
- [x] I added or updated tests when behavior changed.
- [x] I updated documentation when users or contributors are affected.
- [x] I documented breaking changes and migration steps.
- [x] I did not include credentials or unrelated changes.
Upd4ting added a commit that referenced this pull request Nov 11, 2026
…of the read (#113)

## Why

#111 made a `@Localized()` field read with no locale answer `null`,
which DMS's `files/route-answers` test needs. It produced that `null`
inside the modifier chain, and two problems were accepted for 0.3.1
([listed on
#111](#111 (comment))):

1. A two-way modifier stacked after `@Localized()` received `null` in
`unlock`. A custom `unlock`, or `toJSON` for the row, could throw.
2. `EncryptionModifier.unlock`'s published parameter widened to
`LockedType | null`.

This PR produces the `null` at the end of the read instead, so no
modifier ever receives it.

## Summary

```diff
 ModifiersDynamicMetadata.get(field)
   stored = row._internal.data[field]
   value = stored
   for each two-way modifier
-    if value !== undefined
+    if value is not undefined or null
       value = modifier.unlock(value, ...)
-  return value
+  return value === undefined && stored !== undefined ? null : value

 ModifiersDynamicMetadata.set / testValue
-  unlock the previous or stored value, whatever it is
+  unlock it only when it is not undefined or null

 LocalizationModifier.unlock(record, key)
-  no key and no fallback value -> null
+  value under key, else the fallback value (as before #111)

 LocalizationModifier.lock(record, value, key)
-  drops "" under a locale
+  drops "" and null under a locale

 EncryptionModifier.unlock
-  (locked: LockedType | null)
+  ([ciphertext, iv, authTag]: LockedType)   (as before #111)
```

- A field the row stores reads `null` when its modifiers find no value,
or when it stores `null`.
- A field the row does not store, or cannot read yet, still reads
`undefined`.
- No modifier's `unlock` receives `undefined` or `null`. That covers the
read, `set`'s previous-value unlock and `testValue`. As a side effect,
the first assignment to a stacked field such as `@Localized()
@Encrypted()` on a row that stores nothing no longer throws. It threw on
`main` too.
- `null` under a locale is stored as no text, as `""` already was. A row
read with no text answers `null`, so saving that answer again (GET, then
PUT, or the `lock()` helper) stores nothing, and the `fallbackLocale`
still answers. The round-1 review found that this write-back blocked the
fallback once reads answered `null`.
- `docs/3.table-modifiers.md` and the shipped skill say all of this.

### Behaviour change, minor release (0.4.0)

| Read | 0.3.2 | This PR |
| --- | --- | --- |
| `localize("de")` on a field with no `de` text and no `fallbackLocale`
| `undefined` (left out of JSON) | `null` |
| Base64 docs example, stored value not a string | `undefined` | `null`
|
| `@Encrypted()` field stored as `null` | `null` | `null` (no longer
decrypted) |
| No-key read of an unset `@Localized()` field (#111) | `null` | `null`
|
| Field the row does not store | `undefined` | `undefined` |
| `null` assigned under a locale | stored `{ de: null }` | stored as no
text |

## Evidence

Each new test fails on `main` (3b3ff72, or on 74a193b for the write-back
tests) and passes on this branch.

Finding 2. The test types its override from
`Parameters<EncryptionModifier["unlock"]>[0]` and reads `stored[1]`:

```
before: src/tests/modifiers/encryption.test.ts(162,28): error TS18047: 'stored' is possibly 'null'.
after:  ✔ unlocks a stored value typed from unlock's parameter, which excludes null
```

Finding 1, and the `localize("de")` change
(`src/tests/end_of_read_null.test.ts`):

```
before:
  Modified fields - a read that finds no value
    1) never passes null to a two-way modifier stacked after @Localized()
       TypeError: object null is not iterable
         at WrapModifier.unlock
         at ModifiersDynamicMetadata.get
    2) reads null for a locale with no value and no fallback
       AssertionError: expected undefined to equal null
    ✔ reads undefined for a field the row does not store
after:
    ✔ never passes null to a two-way modifier stacked after @Localized()
    ✔ reads null for a locale with no value and no fallback
    ✔ reads undefined for a field the row does not store
```

Write-back of a `null` read (round-1 review finding). Before is 74a193b,
the first commit of this branch:

```
before:
    written back
      1) keeps the fallback after a read answer is saved again
         AssertionError: expected null to equal 'Salut'
      2) stores no text when lock writes back a read with no text
         AssertionError: expected { fr: 'Salut', en: null } to deeply equal { fr: 'Salut' }
  Localized fields - plain data with values by locale
    3) stores a null value as no text under the locale it is read with
       AssertionError: expected { en: null } to deeply equal {}
after:
      ✔ keeps the fallback after a read answer is saved again
      ✔ stores no text when lock writes back a read with no text
    ✔ stores a null value as no text under the locale it is read with
```

Stored `null` on an `@Encrypted()` field (round-1 review finding; 0.3.2
handled it inside `unlock`):

```
before: 1) reads a stored null as null, without decrypting it
           TypeError: object null is not iterable (cannot read property Symbol(Symbol.iterator))
after:  ✔ reads a stored null as null, without decrypting it
```

Stacked field and `testValue` (round-2 review findings; the assignment
also throws on `main`):

```
before: 1) takes a new value on a stacked field that stores nothing or null
           TypeError: Cannot convert undefined or null to object
        2) reads a stored null as null, without decrypting it   (testValue)
           TypeError: object null is not iterable (cannot read property Symbol(Symbol.iterator))
after:  ✔ takes a new value on a stacked field that stores nothing or null
        ✔ reads a stored null as null, without decrypting it
```

`#111`'s own test still passes: `✔ answers null for a field the request
did not send`.

DMS `files/route-answers` against the packed head (ca69b81), linked into
a fresh dms `main` clone (ffcee020) through a `pnpm.overrides` `file:`
tarball. A load tracer confirmed the patched `dist/modifiers/common.js`
was the only copy loaded:

```
DMS_PREBUILT=1 DMS_TEST_DIR=dist/test/integration pnpm exec ajs module test . \
  -f dist/test/integration/setup.test.js -f dist/test/integration/files/route-answers.test.js

    ✔ answers the saved row from new and from every edit
    ✔ answers the deleted-row count from delete, and 404 from an edit of a missing row
    ✔ answers the refusal of a forbidden key
  3 passing
```

Other checks:

- `pnpm run checks` passes all 11 checks.
- The docs check (tooling-configs v0.0.9) reports 0 failures.
- `antelopejs-verify journey --plan` passes all 73 journeys.

## Merge Danger

**Door:** two-way. No stored schema changes. Rows written by 0.3.x read
the same, except where the table above says otherwise.

**Blast Radius:** consumers.

- A consumer that tests a localized read with `=== undefined` to detect
a missing locale now gets `null`. Hence the minor release.
- A JSON answer that used to leave such a field out now carries
`"field": null`.

## Attention

- **Still open.** On a `@Localized() @Encrypted()` field, a `null` read
posted back under a locale is encrypted before the localization modifier
sees it. It is stored as encrypted `null` under that locale, and it
still blocks the `fallbackLocale`. No AntelopeJS repo stacks the two
outside tests and verify playgrounds. Fixing it would need a write-side
rule for `null` in every two-way modifier, so it is left for a
follow-up.
- **No version bump.** The release commit sets 0.4.0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant