Skip to content

3.table-modifiers.md

Latest commit

 

History

History
488 lines (375 loc) · 22.6 KB

3.table-modifiers.md

File metadata and controls

488 lines (375 loc) · 22.6 KB
Raw
title Table modifiers
description Reference for the encryption, hashing, localization, and auto-date modifiers, the scrypt hash format, and the classes and events for writing a custom modifier.
mode reference

A table modifier transforms a field's value on its way into a row and on its way out. The built-in modifiers encrypt, hash, or localize a field, or set a timestamp. Add a modifier's mixin to a table class with Table.with(), then mark each field with the modifier's decorator.

Built-in modifiers

EncryptionModifier

The EncryptionModifier encrypts a field's value when the value is assigned and decrypts it when the field is read.

The default aes-256-gcm algorithm needs a key of exactly 32 bytes. secretKey goes to Node's cipher as a UTF-8 string, without hex or base64 decoding. Supply a high-entropy, 32-byte UTF-8 value from a secret manager, keep it stable while ciphertext exists, and fail startup if it is missing or invalid. A 64-character hex encoding is 64 bytes for this API, not a 32-byte key.

import {
  Table,
  Field,
  EncryptionModifier,
  Encrypted,
} from "@antelopejs/interface-database-decorators";

const secretKey = process.env.ENCRYPTION_KEY;
if (!secretKey || Buffer.byteLength(secretKey, "utf8") !== 32) {
  throw new Error("ENCRYPTION_KEY must contain exactly 32 UTF-8 bytes");
}

class UserCredentials extends Table.with(EncryptionModifier) {
  @Encrypted({
    secretKey,
    algorithm: "aes-256-gcm",
    ivSize: 16,
  })
  @Field("string")
  declare creditCardNumber: string;

  @Encrypted({ secretKey })
  @Field("any")
  declare personalData: {
    ssn: string;
    birthDate: string;
  };
}

How it works

  • An assignment encodes the value as JSON and encrypts it with the configured algorithm. The row stores [ciphertext, iv, authTag], each in base64.
  • Each write generates a random initialization vector (IV).
  • An authenticated algorithm, such as aes-256-gcm, also stores its auth tag. Other algorithms store no tag.
  • A read decrypts the value.
  • A Date comes back as a Date instance. The encoder writes it with toUTCString(), which drops milliseconds, so an encrypted date keeps whole-second precision only.

Options

Option Type Default Description
secretKey string (required) The encryption key.
algorithm string "aes-256-gcm" Encryption algorithm.
ivSize number 16 Initialization vector size in bytes.

HashModifier

The HashModifier stores a salted one-way hash of a field's value. A hash cannot be reversed, so the original value cannot be read back. By default the field stores a scrypt hash, a memory-hard function made for passwords.

import {
  Table,
  Field,
  HashModifier,
  Hashed,
} from "@antelopejs/interface-database-decorators";

class User extends Table.with(HashModifier) {
  @Field("string")
  declare email: string;

  @Hashed()
  @Field("string")
  declare password: string;

  verifyPassword(plainPassword: string): Promise<boolean> {
    return this.testHashAsync("password", plainPassword);
  }
}

How it works

  • The field hashes the value encoded as JSON, so "secret" is hashed as the text "\"secret\"".
  • Each write draws a random 16-byte salt and stores $scrypt$ln=15,r=8,p=3$<salt>$<key>. The salt and the 32-byte key are base64 without padding. ln is the base-2 logarithm of the scrypt cost N, so N is 32768. The string holds the parameters, the salt, and the key, so the cost can change later.
  • An assignment is synchronous, because a property setter cannot wait, and so are the conversions behind a model's insert and update of plain data. Each assignment blocks the event loop for one scrypt run, about 150 ms, which uses 32 MiB (128 × N × r bytes).
  • setHashAsync runs that scrypt run on the thread pool, then assigns the result. It stores the same format as an assignment, so rows written either way verify the same way.
  • Assigning the stored hash back to the field, or calling lock on a loaded row, keeps the hash as it is. Assigning undefined stores nothing.
  • testHash and testHashAsync read ln, r, p, the salt, and the key from the stored string. They derive a key of the stored key's length from the candidate and compare the two with crypto.timingSafeEqual, in constant time.
  • A stored scrypt string verifies only within these limits: ln, r, and p from 1 to 99, at most 128 MiB of memory, p at most 16, ln below 16 × r, and a salt and a key of 16 to 64 bytes each. A string outside them refuses every candidate.
  • On a field without an algorithm, a test spends one scrypt run whatever the row holds, a hash in another format or nothing included. The time of a refusal does not tell whether the row holds a current hash.

Options

Option Type Default Description
algorithm string none Digest to store instead of scrypt, any name createHash accepts. A digest is fast to guess, so use it only for non-secret fingerprints, never for credentials. See the next list.

A field with an algorithm stores the base64 digest of the row's salt followed by the JSON-encoded value. The salt is a random string that the first such write creates. The row keeps it in _internal.meta.HashModifier.salt and reuses it for every later write to its digest fields. A test of a digest field that holds a digest spends no scrypt run. The field still verifies a scrypt string it holds, with one scrypt run, and isHashOutdated never flags it, so a scrypt hash is never flagged for replacement by a weaker digest.

Mixin methods

The HashModifier mixin adds these methods to table instances:

testHash<This extends {}, F extends keyof This>(this: This, field: F, value: This[F]): boolean
testHashAsync<This extends {}, F extends keyof This>(this: This, field: F, value: This[F]): Promise<boolean>
setHashAsync<This extends object, F extends keyof This>(this: This, field: F, value: This[F]): Promise<void>
isHashOutdated<This extends {}>(this: This, field: keyof This): boolean
  • setHashAsync hashes value on the thread pool and resolves once the field holds the hash, as row[field] = value would leave it. Use it on a server for every write of a scrypt field, such as a signup or a password change, then save the row instance with the model. It rejects with Field <name> is not hashed for a field without @Hashed. On a field with an algorithm, it assigns at once, because a digest is fast. On a field with another modifier beside @Hashed, it also assigns at once, on the event loop. Calls on one row that overlap settle in the order their scrypt runs end, so await each one before the next.
const user = new User();
user.email = email;
await user.setHashAsync("password", password);
await userModel.insert(user);

userModel.insert({ email, password }) and userModel.update(id, { password }) still hash on the event loop, because they convert plain data through the setter.

  • testHashAsync resolves whether value matches the stored hash. It runs scrypt on the thread pool, so use it on a server. It rejects with Field <name> is not hashed for a field without @Hashed.
  • testHash gives the same result and blocks the event loop while scrypt runs. It throws Field <name> is not hashed for a field without @Hashed.
  • isHashOutdated returns whether the stored hash differs from what the field writes today, which is any stored string that does not start with $scrypt$ln=15,r=8,p=3$. It returns false when nothing is stored, and always on a field with an algorithm. It throws Field <name> is not hashed for a field without @Hashed.

Upgrade stored hashes

A row whose hash has another format still verifies. On a field without an algorithm, a string that is not a scrypt hash is tested as a salted SHA-256 digest, made with the salt in _internal.meta.HashModifier.salt, and isHashOutdated returns true for it. A scrypt hash of another cost verifies with its own parameters and is flagged too. To rewrite a flagged hash, set the value after a successful test, because only then is the plaintext known:

if (await user.testHashAsync("password", password)) {
  if (user.isHashOutdated("password")) {
    await user.setHashAsync("password", password);
    await userModel.update(user);
  }
}

userModel.update(user._id, { password }) rewrites the hash too. An update that leaves the hashed field out, by ID or with the whole row, keeps the stored hash.

To move a credential off @Hashed({ algorithm: "sha256" }), remove the option. Its rows keep verifying, and isHashOutdated flags them. A row written with another digest verifies only while the field declares that digest.

LocalizationModifier

The LocalizationModifier stores one value of a field per locale. To read or write a locale, call localize() on the table instance.

import {
  Table,
  Field,
  LocalizationModifier,
  Localized,
} from "@antelopejs/interface-database-decorators";

class Product extends Table.with(LocalizationModifier) {
  @Field("number")
  declare price: number;

  @Localized({ fallbackLocale: "en" })
  @Field("string")
  declare name: string;

  @Localized({ fallbackLocale: "en" })
  @Field("string")
  declare description: string;
}

LocalizationModifier.fallbackLocaleOf(TableClass, field) returns a field's fallbackLocale, or undefined when the field is not localized or declares none. Generated forms use it to require a value in that locale.

Usage

const product = new Product();
product._id = "prod-123";
product.price = 99.99;

// Set localized content
product.localize("en").name = "Premium Headphones";
product.localize("fr").name = "Casque Premium";
product.localize("es").name = "Auriculares Premium";

// Read localized content
const spanish = product.localize("es");
console.log(spanish.name); // "Auriculares Premium"

// Fall back to the default locale when a translation is missing
const german = product.localize("de");
console.log(german.name); // "Premium Headphones" (fallback)

A field without a fallbackLocale reads null in a locale that has no text. A field the row does not store at all reads undefined. An empty text "" or null assigned in a locale is stored as no text, so saving a row read with no text stores nothing and the fallback still answers.

Wildcard key

The wildcard key "*" reads or writes all translations at once. List the localized fields with it, and localize("*", fields) types each listed field as its values by locale, a LocalizedRecords<T, K>: a field declared as string reads as a Record<string, string>. Calling localize() on that view with one locale gives back the row type. The view is not the row type, so pass the row itself to insert() or update(). Write all translations with Object.assign, which calls the field's setter. undefined or null clears every translation. Any other value that is not a record, such as a single text, throws a TypeError and leaves the stored translations as they were:

// Get all translations
const allNames: Record<string, string> = product.localize("*", ["name"]).name;
// { en: "Premium Headphones", fr: "Casque Premium", es: "Auriculares Premium" }

// Set all translations
Object.assign(product.localize("*"), {
  name: {
    en: "Premium Headphones",
    fr: "Casque Premium",
    es: "Auriculares Premium",
    de: "Premium-Kopfhoerer",
  },
});

Options

Option Type Description
fallbackLocale string Locale to use when the requested locale has no value or an empty one.

Mixin method

The LocalizationModifier mixin adds a localize method:

localize<This extends { constructor: any }, K extends keyof This>(this: This, locale: "*", fields: readonly K[]): LocalizedRecords<This, K>
localize<This extends { constructor: any }>(this: This, locale: string, fields?: ReadonlyArray<keyof This>): This

Without fields, localize("*") unlocks every localized field but keeps the class's field types. A listed method stays callable.

localize() changes the active locale and returns the same object, not a copy for that locale.

  • References share state. spanish, german, and product above refer to one object. A later localize() call changes what all three references read.
  • Fields can be restricted. When you supply fields, only those fields switch to the selected locale.
  • Copy values before switching. Save the values you need before you change locales, and do not share one instance between concurrent operations on different locales.

AutoDateModifier

The AutoDateModifier sets timestamp fields on insert and update. Mark the fields with the CreationTime and UpdateTime decorators.

import {
  Table,
  CreationTime,
  UpdateTime,
} from "@antelopejs/interface-database-decorators";

class Article extends Table {
  declare title: string;

  @CreationTime()
  declare creationDate: Date;

  @UpdateTime()
  declare updateDate: Date;
}

How it works

  • On insert, both CreationTime and UpdateTime fields get the current date.
  • On update, UpdateTime fields get the current date, and CreationTime fields are removed from the payload, so the stored creation date stays.
  • The modifier only handles events. It does not transform stored values and has no mixin, so the table class needs no Table.with().

Combine multiple modifiers

Pass several modifier classes to Table.with():

import {
  Table,
  Field,
  EncryptionModifier,
  HashModifier,
  LocalizationModifier,
  Encrypted,
  Hashed,
  Localized,
} from "@antelopejs/interface-database-decorators";

const secretKey = process.env.ENCRYPTION_KEY;
if (!secretKey || Buffer.byteLength(secretKey, "utf8") !== 32) {
  throw new Error("ENCRYPTION_KEY must contain exactly 32 UTF-8 bytes");
}

class UserProfile extends Table.with(
  EncryptionModifier,
  HashModifier,
  LocalizationModifier,
) {
  @Encrypted({ secretKey })
  @Field("any")
  declare privateInfo: {
    address: string;
    phoneNumber: string;
  };

  @Hashed()
  @Field("string")
  declare password: string;

  @Localized({ fallbackLocale: "en" })
  @Field("string")
  declare bio: string;
}

Create custom modifiers

OneWayModifier

A OneWayModifier transforms data in one direction, when a value is written. Use it for a transformation that cannot be reversed, such as hashing.

import {
  OneWayModifier,
  MixinSymbol,
  attachModifier,
} from "@antelopejs/interface-database-decorators";
import { MakePropertyDecorator } from "@antelopejs/interface-core/decorators";
import { createHash } from "crypto";

class PrefixHashModifier extends OneWayModifier<
  string | undefined,
  [],
  {},
  { algorithm?: string }
> {
  public readonly autolock = true;

  public override lock(_locked: string | undefined, value: unknown) {
    if (value === undefined) return undefined;
    return createHash(this.options.algorithm || "sha256")
      .update(JSON.stringify(value))
      .digest("hex");
  }

  [MixinSymbol] = class {};
}

const PrefixHashed = MakePropertyDecorator(
  (target, propertyKey, options?: { algorithm?: string }) => {
    attachModifier(
      target.constructor,
      PrefixHashModifier,
      propertyKey,
      options || {},
    );
  },
);

The generic parameters of OneWayModifier<LockedType, Args, Meta, Options> are:

Parameter Description
LockedType The type stored in the database after the transformation.
Args Additional arguments for lock and test. Defaults to [].
Meta The modifier's state for a row, kept in _internal.meta.<modifier class name> and shared by its fields.
Options The configuration that the decorator passes to attachModifier.

TwoWayModifier

A TwoWayModifier extends OneWayModifier with an unlock method, for a transformation that can be reversed, such as encryption. A read never passes undefined or null to unlock: a field that stores null reads null, and a field the row stores reads null when its modifiers find no value in it:

import {
  TwoWayModifier,
  MixinSymbol,
  attachModifier,
} from "@antelopejs/interface-database-decorators";
import { MakePropertyDecorator } from "@antelopejs/interface-core/decorators";
import type { ValueProxy } from "@antelopejs/interface-database";

class Base64Modifier extends TwoWayModifier<
  unknown,
  [],
  {},
  { prefix?: string }
> {
  public readonly autolock = true;
  public readonly autounlock = true;

  public override lock(_locked: unknown, value: unknown) {
    if (value === undefined) return undefined;
    const prefix = this.options.prefix || "b64-";
    return `${prefix}${Buffer.from(JSON.stringify(value)).toString("base64")}`;
  }

  public override unlock(locked: unknown): unknown {
    if (typeof locked !== "string") return undefined;
    const prefix = this.options.prefix || "b64-";
    const encoded = locked.substring(prefix.length);
    return JSON.parse(Buffer.from(encoded, "base64").toString("utf8"));
  }

  public override unlockrequest(
    data: ValueProxy<unknown>,
  ): ValueProxy<unknown> {
    return data;
  }

  [MixinSymbol] = class {};
}

const Base64Encoded = MakePropertyDecorator(
  (target, propertyKey, options?: { prefix?: string }) => {
    attachModifier(
      target.constructor,
      Base64Modifier,
      propertyKey,
      options || {},
    );
  },
);

Base64 is an encoding, not encryption. The example declares its LockedType as unknown, so unlockrequest() can return the stored value's proxy as it is. Database expressions therefore see the encoded value, and only instance reads decode it. unlock reads a stored value that is not a string as undefined, so the field reads null: a field the row stores reads null when its modifiers find no value.

ContainerModifier

A modifier that stores several values under keys, as LocalizationModifier does, extends ContainerModifier. Its lock, unlock, and unlockrequest take the key as their argument, a string or undefined. lock and unlock treat the wildcard key "*" as the whole record, and lock throws a TypeError for a value under "*" that is not a record. undefined is no key: lock stores nothing under it, and unlock and unlockrequest find no value, so a LocalizationModifier answers its fallbackLocale, and the field reads null when that locale has no value either. The modifier chain passes no null to a modifier stacked after it:

import { ContainerModifier } from "@antelopejs/interface-database-decorators";

class MyContainerModifier extends ContainerModifier<{}, { fallback?: string }> {
  // Override unlock() to add fallback behavior
  // Override unlockrequest() for query-time data access
}

Attach a modifier

A decorator attaches its modifier with attachModifier(target.constructor, Modifier, propertyKey, options), as the examples above do. TypeScript checks options against the modifier's options type. It does not check that the field names a member of the class.

A modifier with lock replaces the field with an accessor that routes reads and writes through the field's modifiers. attachModifier throws an Error when such a modifier goes on constructor or __proto__, or on a field that already has a one-way modifier. A modifier without lock only handles events, and the field keeps its plain value.

Read a modified field in a query

unlockrequest(table, row, field, modifiers) reads a field unlocked inside a database query. It reads table only for its modifier metadata. Each two-way modifier on the field passes the stored value through its own unlockrequest(), with the arguments that modifiers gives for it by class name. A field with only one-way modifiers yields its stored value as it is. unlockrequest throws a TypeError when modifiers has no entry for a two-way modifier on the field. TypeScript checks that field is a key of the row, not each entry's arguments.

Modifier events

A modifier hooks into a row's lifecycle through methods named after events. The engine calls each one with the modifier as this, then the table instance, the field name, and the event's arguments:

Event Triggered when
fromPlainData After plain data is converted to an instance.
toPlainData After an instance is converted to plain data.
fromDatabase After a stored record is converted to an instance, as reads do.
toDatabase When an instance is converted to the record to store, before a write.
insert Before a model or a fixture inserts a new record.
update Before a model updates an existing record. 
import {
  Modifier,
  MixinSymbol,
} from "@antelopejs/interface-database-decorators";

class TimestampModifier extends Modifier {
  public insert<F extends string>(object: Partial<Record<F, Date>>, field: F) {
    if (field === "createdAt" && !object[field]) {
      object[field] = new Date();
    }
  }

  public update<F extends string>(object: Partial<Record<F, Date>>, field: F) {
    if (field === "updatedAt") {
      object[field] = new Date();
    }
  }

  [MixinSymbol] = class {};
}

A modifier without lock and unlock only handles events. attachModifier then defines no accessor on the field, which keeps its plain value. Such a modifier fits side effects such as logging, validation, or filling in fields.