diff --git a/.changeset/immutable-query-major.md b/.changeset/immutable-query-major.md new file mode 100644 index 00000000..631bb9fe --- /dev/null +++ b/.changeset/immutable-query-major.md @@ -0,0 +1,31 @@ +--- +"@cleverbrush/async": major +"@cleverbrush/auth": major +"@cleverbrush/client": major +"@cleverbrush/deep": major +"@cleverbrush/di": major +"@cleverbrush/env": major +"@cleverbrush/knex-clickhouse": major +"@cleverbrush/knex-schema": major +"@cleverbrush/log": major +"@cleverbrush/mapper": major +"@cleverbrush/orm-cli": major +"@cleverbrush/orm": major +"@cleverbrush/otel": major +"@cleverbrush/react-form": major +"@cleverbrush/scheduler": major +"@cleverbrush/schema-json": major +"@cleverbrush/schema": major +"@cleverbrush/server-openapi": major +"@cleverbrush/server": major +--- + +Make Framework query builders immutable and infer row schemas automatically. + +Retain returned query builders, return synchronous builders from scopes and grouped predicates, and supply an explicit Framework object output schema for opaque raw SELECTs. Ordinary, aliased, polymorphic and ORM queries expose their row schemas directly. Projections replace scalar selections, and projected/aggregate/raw queries cannot perform entity writes. Reads and write-returning rows consistently preserve exact decimal/bigint strings, Date objects and SQL nulls. + +All published Framework packages advance together to the next major version. Tracked entity objects remain mutable. + +### Migrating from v4.x to v5 + +Remove `withRowSchema()` calls and retain each configured query instead of relying on mutation. Replace raw base-query overloads with explicit output contracts. See `libs/knex-schema/MIGRATION-v5.md` for the complete migration guide. diff --git a/.changeset/schema-aware-read-predicates.md b/.changeset/schema-aware-read-predicates.md new file mode 100644 index 00000000..c105dbec --- /dev/null +++ b/.changeset/schema-aware-read-predicates.md @@ -0,0 +1,10 @@ +--- +"@cleverbrush/knex-schema": minor +--- + +Add shape-preserving grouped AND/OR predicates, captured IN/EXISTS subqueries, +bound raw predicates and ordering, and typed SQL column references to ordinary +and aliased schema-aware readers. These capabilities also work in ordinary ORM +reads and nested relation customizers while retaining immutable query plans and +stable row-schema identity. Group callbacks are synchronous and predicate-only; +unrestricted raw query mutation remains unavailable. diff --git a/.changeset/schema-boundaries.md b/.changeset/schema-boundaries.md index c4c0d255..199781d1 100644 --- a/.changeset/schema-boundaries.md +++ b/.changeset/schema-boundaries.md @@ -10,4 +10,4 @@ Shape, validation-rule, default, fallback and extension changes clear inherited names; apply schemaName after those edits to establish a new named definition. Preserve one canonical definition in JSON Schema, OpenAPI and AsyncAPI with strict name collision checks. Keep existing -type inference and legacy optional null acceptance unchanged. +type inference and optional null acceptance unchanged. diff --git a/AGENTS.md b/AGENTS.md index 4b8d890f..d93978be 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -135,6 +135,17 @@ The `demos/` directory is linted separately (see `demos/todo-backend/biome.json` - Target `ES2022`; use modern syntax freely - Type assertions with `as` are acceptable (the linter won't block them) +## Documentation and Project Boundaries + +- Framework is an independent, application-agnostic project. Use generic domain + examples in source, documentation, changesets and PR descriptions. Consumer + application references belong only in website showcase links. +- Describe the current supported API in READMEs, guides and JSDoc. Keep historical + API comparisons and upgrade instructions in explicitly labeled v4.x-to-v5 + migration documentation, and link to it from current guides where useful. +- Preserve accurate API contracts and deprecation annotations; do not change + runtime behavior just to simplify documentation. + --- ## Testing Conventions diff --git a/demos/e2e/README.md b/demos/e2e/README.md index 94ed7f84..aab01e1f 100644 --- a/demos/e2e/README.md +++ b/demos/e2e/README.md @@ -27,7 +27,7 @@ npx vitest --run src/api/todos.api.test.ts # single file ### One-time browser install -The UI project requires Chromium. After `npm install`, run: +The UI project requires Chromium. After `npm ci`, run: ```bash cd demos/e2e && npx playwright install chromium @@ -41,6 +41,7 @@ cd demos/e2e && npx playwright install chromium | `RESET` | `0` | If `1`, run `docker compose down -v` before bringing the stack up (wipes Postgres). | | `CI` | unset | Setting `CI=true` flips `KEEP_STACK` default to `0` and forces full teardown. | | `HEADED` | `0` | If `1`, launch Chromium headed so you can watch UI tests run. | +| `E2E_BROWSER_EXECUTABLE_PATH` | unset | Optional path to an existing Chromium executable; otherwise uses Playwright's managed browser. | | `SLOWMO` | `0` | Slow-motion delay (ms) for Playwright actions — useful with `HEADED=1`. | | `E2E_API_URL` | `http://localhost:3000` | Backend HTTP base URL. | | `E2E_WS_URL` | `ws://localhost:3000` | Backend WebSocket base URL. | @@ -115,7 +116,7 @@ src/ - **Todos** — full CRUD, list pagination, `getWithAuthor`, polymorphic events (`assigned` / `commented` / `completed`), optimistic concurrency on `complete` (200 / 409 with `If-Match`), cross-user 403, attachment - download, `legacyReplace` redirect. + download, PUT redirect. - **Import / Export** — 207 small batch, 202 large batch, idempotency header (contract-level), CSV export with quoting + content headers. - **Users (admin)** — list (admin only), delete user, self-delete blocked, diff --git a/demos/e2e/src/api/telemetry.smoke.test.ts b/demos/e2e/src/api/telemetry.smoke.test.ts index 0fb18998..55c78690 100644 --- a/demos/e2e/src/api/telemetry.smoke.test.ts +++ b/demos/e2e/src/api/telemetry.smoke.test.ts @@ -41,8 +41,6 @@ describe('Telemetry smoke — ClickHouse logs & traces correlation', () => { TraceId: string; }>( `SELECT body AS Body, trace_id AS TraceId FROM signoz_logs.distributed_logs_v2 WHERE trace_id = '${traceId}' FORMAT JSON`, -ace_id = '${traceId}' FORMAT JSON`, - 45_000, 1_000 ); @@ -55,10 +53,8 @@ ace_id = '${traceId}' FORMAT JSON`, const spans = await waitForRows<{ SpanName: string; ServiceName: string; - }>(name AS SpanName, resources_string['service.name'] AS ServiceName FROM signoz_traces.distributed_signoz_index_v3 WHERE trace_id = '${traceId}' FORMAT JSON`, - + }>( `SELECT name AS SpanName, resources_string['service.name'] AS ServiceName FROM signoz_traces.distributed_signoz_index_v3 WHERE trace_id = '${traceId}' FORMAT JSON`, - 45_000, 1_000 ); @@ -67,7 +63,7 @@ ace_id = '${traceId}' FORMAT JSON`, expect(services.has('todo-backend')).toBe(true); }); - it('ClickHouse is reachable and reports recent losignoz_logs.distributed_logs_v2 WHERE toDateTime(intDiv(timestamp, 1000000000)) + it('ClickHouse is reachable and reports recent logs', async () => { const { rows } = await clickhouseQuery<{ recent: string }>( `SELECT toString(count()) AS recent FROM signoz_logs.distributed_logs_v2 WHERE toDateTime(intDiv(timestamp, 1000000000)) >= now() - INTERVAL 1 HOUR FORMAT JSON` ); diff --git a/demos/e2e/src/api/todos.api.test.ts b/demos/e2e/src/api/todos.api.test.ts index fdf28824..edb420cb 100644 --- a/demos/e2e/src/api/todos.api.test.ts +++ b/demos/e2e/src/api/todos.api.test.ts @@ -228,10 +228,17 @@ describe('Todos — attachment & legacyReplace', () => { body: { title: uniqueTitle('attach') } }) ); + const boundary = 'framework-attachment-fixture'; + const uploaded = await r('POST', `/api/todos/${created.id}/attachment`, { + raw: true, + headers: { 'content-type': `multipart/form-data; boundary=${boundary}` }, + body: `--${boundary}\r\nContent-Disposition: form-data; name="attachment"; filename="example.txt"\r\nContent-Type: text/plain\r\n\r\nImmutable query demo\r\n--${boundary}--\r\n` + }); + expect(uploaded.status).toBe(201); const res = await r('GET', `/api/todos/${created.id}/attachment`); expect(res.status).toBe(200); expect(res.headers['content-type']).toMatch(/text\/plain/); - expect(res.body.length).toBeGreaterThan(0); + expect(res.body).toBe('Immutable query demo'); }); it('legacyReplace (PUT) returns a redirect', async () => { diff --git a/demos/e2e/src/support/playwright.ts b/demos/e2e/src/support/playwright.ts index c01df835..bf5239d5 100644 --- a/demos/e2e/src/support/playwright.ts +++ b/demos/e2e/src/support/playwright.ts @@ -11,6 +11,7 @@ let browser: Browser | null = null; async function getBrowser(): Promise { if (!browser) { browser = await chromium.launch({ + executablePath: process.env.E2E_BROWSER_EXECUTABLE_PATH || undefined, headless: !config.headed, slowMo: config.slowMo }); diff --git a/demos/e2e/src/ui/todo-crud.ui.test.ts b/demos/e2e/src/ui/todo-crud.ui.test.ts index 4a93b440..8c252195 100644 --- a/demos/e2e/src/ui/todo-crud.ui.test.ts +++ b/demos/e2e/src/ui/todo-crud.ui.test.ts @@ -62,6 +62,12 @@ describe('UI — todo CRUD', () => { // Detail page after creation await page.waitForURL(/\/todos\/\d+$/, { timeout: 10_000 }); + // Stay on the detail page until its requests settle: an admin-only + // picker lookup must not sign out an ordinary user after creation. + await page.getByRole('button', { name: 'Save Changes', exact: true }).waitFor(); + await page.waitForLoadState('networkidle'); + expect(page.url()).toMatch(/\/todos\/\d+$/); + expect(await page.locator('input').first().inputValue()).toBe(title); const url = page.url(); const todoId = Number(url.match(/\/todos\/(\d+)$/)![1]); diff --git a/demos/todo-backend/src/api/handlers/todos.ts b/demos/todo-backend/src/api/handlers/todos.ts index 0c60aea1..04bc0eef 100644 --- a/demos/todo-backend/src/api/handlers/todos.ts +++ b/demos/todo-backend/src/api/handlers/todos.ts @@ -452,13 +452,13 @@ export const uploadAttachmentHandler: Handler< return ActionResult.created({ id: updated.id, title: updated.title, - description: updated.description, + description: updated.description ?? undefined, completed: updated.completed, userId: updated.userId, createdAt: updated.createdAt, updatedAt: updated.updatedAt, - attachmentName: updated.attachmentName, - attachmentMimeType: updated.attachmentMimeType, + attachmentName: updated.attachmentName ?? undefined, + attachmentMimeType: updated.attachmentMimeType ?? undefined, attachmentSize: file.size }); }; diff --git a/demos/todo-backend/src/api/mappers.ts b/demos/todo-backend/src/api/mappers.ts index eaba2470..74a74b47 100644 --- a/demos/todo-backend/src/api/mappers.ts +++ b/demos/todo-backend/src/api/mappers.ts @@ -7,7 +7,6 @@ import type { TodoActivityResponse } from './schemas.js'; const UserRowSchema = object({ id: number(), email: string(), - passwordHash: string().optional(), role: string(), authProvider: string(), createdAt: date() @@ -16,13 +15,13 @@ const UserRowSchema = object({ const TodoRowSchema = object({ id: number(), title: string(), - description: string().optional(), + description: string().nullable(), completed: boolean(), userId: number(), createdAt: date(), updatedAt: date(), - attachmentName: string().optional(), - attachmentMimeType: string().optional() + attachmentName: string().nullable(), + attachmentMimeType: string().nullable() }); export const mappingRegistry = mapper() @@ -33,6 +32,10 @@ export const mappingRegistry = mapper() m .for(t => t.description) .compute(f => f.description ?? undefined) + .for(t => t.attachmentName) + .compute(f => f.attachmentName ?? undefined) + .for(t => t.attachmentMimeType) + .compute(f => f.attachmentMimeType ?? undefined) .for(t => t.attachmentSize) .ignore() ); @@ -40,7 +43,7 @@ export const mappingRegistry = mapper() const _mapUserFn = mappingRegistry.getMapper(UserRowSchema, UserResponseSchema); const _mapTodoFn = mappingRegistry.getMapper(TodoRowSchema, TodoResponseSchema); -export const mapUser = (row: UserDb) => _mapUserFn(row); +export const mapUser = (row: Omit) => _mapUserFn(row); export const mapTodo = (row: TodoDb) => _mapTodoFn(row); export function mapTodoActivity(row: ActivityDb & Record): TodoActivityResponse { diff --git a/demos/todo-backend/src/db/schemas.ts b/demos/todo-backend/src/db/schemas.ts index 277907e2..6333b074 100644 --- a/demos/todo-backend/src/db/schemas.ts +++ b/demos/todo-backend/src/db/schemas.ts @@ -1,4 +1,5 @@ import { + type EntityResult, array, boolean, date, @@ -144,12 +145,7 @@ const TodoSchema = object({ 'attachmentMimeType' ) .projection('ownership', 'id', 'userId') - .scope( - 'recentFirst', - (q: { - orderBy: (column: string, direction: 'asc' | 'desc') => unknown; - }) => q.orderBy('created_at', 'desc') - ); + .scope('recentFirst', q => q.orderBy('createdAt', 'desc')); export const TodoEntity = defineEntity(TodoSchema) .belongsTo( @@ -187,32 +183,6 @@ export const entityMap: AppEntityMap = { // ── Plain row types (used by mappers) ─────────────────────────────────────── -export type ActivityDb = { - id: number; - todoId: number; - type: string; - actorUserId?: number; - completedAt?: Date | null; - createdAt: Date; -}; - -export type UserDb = { - id: number; - email: string; - passwordHash?: string; - role: string; - authProvider: string; - createdAt: Date; -}; - -export type TodoDb = { - id: number; - title: string; - description?: string; - completed: boolean; - userId: number; - createdAt: Date; - updatedAt: Date; - attachmentName?: string; - attachmentMimeType?: string; -}; +export type ActivityDb = EntityResult; +export type UserDb = EntityResult; +export type TodoDb = EntityResult; diff --git a/demos/todo-frontend/src/features/todos/TodoDetailPage.tsx b/demos/todo-frontend/src/features/todos/TodoDetailPage.tsx index e095545c..c1e58291 100644 --- a/demos/todo-frontend/src/features/todos/TodoDetailPage.tsx +++ b/demos/todo-frontend/src/features/todos/TodoDetailPage.tsx @@ -18,6 +18,7 @@ import { Field, useSchemaForm } from '@cleverbrush/react-form'; import { UpdateTodoBodySchema } from '@cleverbrush/todo-backend/contract'; import { ApiError, isTimeoutError, isNetworkError } from '@cleverbrush/client'; import { client } from '../../api/client'; +import { useAuth } from '../../lib/auth-context'; type TodoEvent = Parameters[0]['body']; import { ConfirmDialog } from '../../components/ConfirmDialog'; @@ -25,6 +26,7 @@ import { ConfirmDialog } from '../../components/ConfirmDialog'; type TodoWithAuthor = Awaited>; export function TodoDetailPage() { + const { isAdmin } = useAuth(); const { id } = useParams<{ id: string }>(); const navigate = useNavigate(); @@ -79,11 +81,13 @@ export function TodoDetailPage() { // Load user list once for the "assigned" picker useEffect(() => { + // The user directory is admin-only. Regular users retain the ID input. + if (!isAdmin) return; client.users .list({ query: { page: 1, limit: 100 } }) .then(rows => setUsers(rows.map(u => ({ id: u.id, email: u.email })))) .catch(() => {/* ignore — picker will fall back to text input */}); - }, []); + }, [isAdmin]); useEffect(() => { load(); }, [load]); diff --git a/docs/assets/immutable-query-demo.png b/docs/assets/immutable-query-demo.png new file mode 100644 index 00000000..e73fa294 Binary files /dev/null and b/docs/assets/immutable-query-demo.png differ diff --git a/docs/assets/immutable-query-docs.png b/docs/assets/immutable-query-docs.png new file mode 100644 index 00000000..fb6c4d84 Binary files /dev/null and b/docs/assets/immutable-query-docs.png differ diff --git a/docs/assets/schema-aware-read-predicates.png b/docs/assets/schema-aware-read-predicates.png new file mode 100644 index 00000000..c762f48c Binary files /dev/null and b/docs/assets/schema-aware-read-predicates.png differ diff --git a/docs/cache-form-migration.md b/docs/cache-form-migration.md index be9e262d..5c1fee99 100644 --- a/docs/cache-form-migration.md +++ b/docs/cache-form-migration.md @@ -1,13 +1,13 @@ -# Cache keys and schema-form lifecycle +# Migrating from Framework v4.x to v5: cache keys and schema-form lifecycle These changes are application-agnostic. Libraries do not choose an application's auth scope, UI kit, notifications, navigation behavior, or cache backend. -This batch is scheduled as a coordinated **minor release** of the fixed package -group. This release classification does not remove the compatibility changes -below: external-cache users must coordinate the key-format migration, and -consumers relying on previous `deepEqual` results must review those assumptions. -The existing public form APIs remain available. +This guide covers cache and form contracts to review when upgrading from v4.x +to the coordinated v5 release. External-cache users must coordinate the key-format +migration, and consumers relying on previous `deepEqual` results must review those +assumptions. If a v4.x prerelease already provided these contracts, retain that +integration. The public form APIs described below remain available. ## Cache key migration (breaking) diff --git a/libs/client/CHANGELOG.md b/libs/client/CHANGELOG.md index 717637da..a2f204e8 100644 --- a/libs/client/CHANGELOG.md +++ b/libs/client/CHANGELOG.md @@ -26,7 +26,7 @@ ### Minor Changes -- c75bff4: Add framework affordances discovered while reviewing Xpenser: +- c75bff4: Add reusable environment, HTTP, cache, database and migration capabilities: - `@cleverbrush/env`: add `envBoolean()` for environment-style boolean values such as `1`, `0`, `yes`, `no`, `on`, and `off`. diff --git a/libs/deep/README.md b/libs/deep/README.md index 7c70dc8e..a3d2205a 100644 --- a/libs/deep/README.md +++ b/libs/deep/README.md @@ -98,12 +98,9 @@ deepEqual(new Map(), new Map()); // => false (opaque objects compare by identity) ``` -**Breaking comparison corrections:** previous versions could throw for object/null -pairs, consider Dates equal to unrelated objects, compare distinct opaque objects -by enumerable shape, and reject equivalent cycles or repeated references. Signed -zero, invalid Dates, symbol keys and sparse arrays now follow the rules above. -Audit consumers relying on those outcomes. To compare Maps/Sets/custom instances -by content, explicitly project their relevant state into plain data first. +To compare Maps/Sets/custom instances by content, explicitly project their +relevant state into plain data first. For upgrade considerations, see the +[v4.x-to-v5 migration guide](../../docs/cache-form-migration.md#shared-data-snapshots-and-equality-breaking). ### `deepExtend(...objects)` diff --git a/libs/env/CHANGELOG.md b/libs/env/CHANGELOG.md index 0fb62f33..2b34e186 100644 --- a/libs/env/CHANGELOG.md +++ b/libs/env/CHANGELOG.md @@ -24,7 +24,7 @@ ### Minor Changes -- c75bff4: Add framework affordances discovered while reviewing Xpenser: +- c75bff4: Add reusable environment, HTTP, cache, database and migration capabilities: - `@cleverbrush/env`: add `envBoolean()` for environment-style boolean values such as `1`, `0`, `yes`, `no`, `on`, and `off`. diff --git a/libs/knex-schema/CHANGELOG.md b/libs/knex-schema/CHANGELOG.md index c7f59edc..68cc4bf1 100644 --- a/libs/knex-schema/CHANGELOG.md +++ b/libs/knex-schema/CHANGELOG.md @@ -22,7 +22,7 @@ ### Minor Changes -- c75bff4: Add framework affordances discovered while reviewing Xpenser: +- c75bff4: Add reusable environment, HTTP, cache, database and migration capabilities: - `@cleverbrush/env`: add `envBoolean()` for environment-style boolean values such as `1`, `0`, `yes`, `no`, `on`, and `off`. diff --git a/libs/knex-schema/MIGRATION-v5.md b/libs/knex-schema/MIGRATION-v5.md new file mode 100644 index 00000000..1f188b49 --- /dev/null +++ b/libs/knex-schema/MIGRATION-v5.md @@ -0,0 +1,212 @@ +# Migrating from Framework v4.x to v5: immutable queries + +This is a coordinated **major release of all 19 published Framework packages**. +Upgrade them together. Prerelease snapshots still use beta versions; the stable +changeset release is major. There is no mutable compatibility mode. + +Schema builders were already immutable. This release makes **query builders** +immutable too, so their inferred TypeScript result and runtime `rowSchema` cannot +drift when a shared query is configured elsewhere. SQL remains lazy: configuration +and metadata access do not execute it; each await/execute runs a new statement. + +## 1. Keep every configured query + +Before: + +```ts +const users = db.users.query(); +if (search) users.where('name', 'ilike', `%${search}%`); +users.orderBy('id'); +return users; +``` + +After: + +```ts +let users = db.users.query(); +if (search) users = users.where('name', 'ilike', `%${search}%`); +return users.orderBy('id'); +``` + +Independent branches can safely share a base: + +```ts +const enabled = query(knex, User).where('enabled', true); +const firstPage = enabled.orderBy('id').limit(20); +const matching = enabled.where('name', 'Alice'); +// Neither branch changes enabled or its sibling. +``` + +Audit helpers, loops, conditional filters, transactions, relation customizers, +scopes and retained query variables. Ignoring a returned builder now does nothing. +`toKnexQuery()` returns an independently mutable snapshot; executing that native +snapshot bypasses Framework decoding and ORM tracking. + +## 2. Return synchronous Framework configuration results + +Before: + +```ts +query(knex, User).where(group => { + group.where('name', 'Alice'); + group.orWhere('name', 'Bob'); +}); +``` + +After: + +```ts +query(knex, User).where(group => + group.where('name', 'Alice').orWhere('name', 'Bob')); + +db.projects.include(r => r.tasks, tasks => + tasks.where('done', false).orderBy('id').limit(3)); +``` + +Grouped predicates expose predicates only. Scopes expose filters, ordering and +paging, not selection, writes or execution. All Framework customizers must return +a configured builder derived from the supplied one; void, async and unrelated +results are rejected. Configuration callbacks run once when attached. An empty +group must return its input unchanged. Later branches from a retained group cannot +modify the attached query. Transaction work callbacks remain asynchronous. + +Default scopes run once when the source is created. Cloning, inspecting metadata, +rendering SQL and repeated execution do not rerun them. `.unscoped()` removes only +the default scope; explicit predicates remain. It **does not** disable soft deletion: +use `.withDeleted()` or `.onlyDeleted()` explicitly. A default scope and caller OR +groups are combined independently, so OR cannot bypass the default scope. + +## 3. Remove `.withRowSchema()` and use the actual result schema + +Before: + +```ts +const userRead = query(knex, User).withRowSchema() + .select(u => ({ id: u.id, name: u.name })); +``` + +After: + +```ts +export const userRead = query(knex, User) + .select(u => ({ id: u.id, name: u.name })); +export const UserRow = userRead.rowSchema; +``` + +`userRead` is a prepared query, not a schema or a row. `UserRow` describes exactly +the decoded result. Export both from a query module and import the schema into +separate mapper modules; no monolithic query/mapper expression is needed. The +[README](./README.md#definitions-can-live-in-separate-files) has a compiled +four-file example. Filters, ordering, paging and transaction clones retain schema +identity, allowing reuse of prepared mapper registrations. + +`select()` and `projected()` replace the scalar selection on a new query; existing +queries are unchanged. Included relation fields remain independent. Aliased flat +joins need an explicit non-empty selection before reading metadata or executing. +Polymorphic roots expose a union `rowSchema` and per-variant `variantRowSchemas`. +Use `forVariant()` for typed branch projections and retain the discriminator. +Use `selectVariants()` for union narrowing, and common root filters/order/pagination +for the combined result. Table-only operations are not advertised on ORM unions. + +## 4. Adopt one storage representation for reads and writes + +| Storage | Framework row value | +| --- | --- | +| Integer / floating-point column | finite `number` | +| Declared decimal / numeric / bigint | exact `string` | +| Optional / nullable SQL column | present property with a value or `null` | +| SQL date / timestamp | `Date` | +| Missing optional JSON property | absent property | +| Optional object relation / missing collection | `null` / `[]` | + +These rules apply to ordinary reads, nested graphs, ORM reloads, inserts, upserts +and other returning writes. SQL text casts protect exact numbers before driver +parsing. Timezone-less timestamps are interpreted as UTC; JavaScript dates still +have millisecond precision. Input defaults, coercers and preprocessors are not +replayed on persisted rows. Separate request/input schemas from storage schemas; +optional storage fields with input defaults are ambiguous and rejected. + +Before, application code often relied on an omitted optional property: + +```ts +// Previous entity output type: { amount?: number; deletedAt?: Date } +``` + +After, derive the row type from `rowSchema` (or `EntityResult` / `InferDatabaseRow`): + +```ts +// Decimal optional storage: { amount: string | null; deletedAt: Date | null } +// Map SQL null to undefined explicitly when an HTTP DTO omits optional fields. +``` + +Do not blindly convert exact strings to numbers. Use explicit domain conversion +only where rounding is acceptable. Insert/update payloads accept exact storage +values, and bigint optimistic-concurrency versions increment without rounding. + +## 5. Declare complete raw output contracts + +Implicit raw base-query overloads and untyped shape-changing escapes are removed. +Before: + +```ts +const report = query(knex, User, knex('users').count({ total: '*' })); +``` + +After: + +```ts +const Report = object({ total: number().coerce() }); +const report = query(knex, User).apply( + sql => sql.clearSelect().count({ total: '*' }), + { output: Report } +); +// report.rowSchema === Report +const rows = await report; + +const rawRows = await rawQuery(knex, Report, + 'select count(*)::text as total from users where enabled = ?', [true]); +``` + +`selectRaw(sql, bindings, { output })` uses the same contract. The output must be +a synchronous, introspectable Framework object schema. It parses each raw row +**once**, with no entity decoding pass or implicit column-to-property remapping. +Alias SQL columns to output property names and cast exact numeric expressions to +text yourself. SQL and bindings are captured; modifying a retained native builder +after configuration does not affect the query. Async raw callbacks are rejected. +Native Knex configuration is the explicit mutable boundary; return that builder +or `undefined`. Raw queries are read-only and detached. Raw SQL remains trusted +application code, not a sandbox or an authorization mechanism. + +## 6. Keep entity writes and projection reads separate + +Full ORM entity reads still use the identity map when tracking is enabled. Their +objects remain mutable, including the familiar `saveChanges()`, `reload()` and +concurrency flow. Repeated queries can return the same tracked object even though +the query builders are independent. Generated IDs/versions are applied to tracked +objects only after the write transaction commits. + +Selected, grouped, distinct and raw rows are detached. Selecting every scalar +field explicitly is still a projection; it is not implicitly promoted to an entity. +Page containers and scalar results are not tracked. A partial row cannot overwrite +a full tracked entity. Writes through projections, relation-loaded queries or +aggregate queries are rejected both statically and at runtime. Start writes from +an unprojected table query; use ORM `ofVariant()` for polymorphic writes. Paginated +writes require a declared primary key. + +Replace `joinOne` / `joinMany` `foreignQuery`, `mappers` and `orderBy` spec properties +with typed child customizers. Use application mappers after decoding for DTO changes. +Child default scopes and soft deletion apply automatically; disable them explicitly +on the child when that is the intended policy. + +## Upgrade checklist + +- Upgrade the fixed Framework package group together; remove `.withRowSchema()` calls. +- Retain returned queries and return synchronous configuration results. +- Replace raw base queries and shape-changing SQL with explicit output contracts. +- Audit API DTOs for exact numbers, dates, SQL nulls and storage/input separation. +- Migrate relation customizers and polymorphic branch projections. +- Verify identity tracking, writes, transaction rollback and concurrency in the app. +- Run TypeScript, unit and real PostgreSQL tests; test representative endpoint flows. + +This release does not change native Knex itself, make entity objects immutable, +introduce automatic DTO mapping, or guarantee equivalent graph SQL on other dialects. diff --git a/libs/knex-schema/README.md b/libs/knex-schema/README.md index 0811875a..26fa1779 100644 --- a/libs/knex-schema/README.md +++ b/libs/knex-schema/README.md @@ -2,6 +2,9 @@ Type-safe, schema-driven query builder for [Knex](https://knexjs.org/). Use `@cleverbrush/schema` object builders to describe your PostgreSQL tables — column name mapping, eager loading, and full CRUD are handled automatically with complete TypeScript inference. +Every Framework query is immutable and exposes `.rowSchema` automatically. +Upgrading? See [Migrating from v4.x to v5](./MIGRATION-v5.md). + ## Installation ```bash @@ -32,7 +35,7 @@ const db = knex({ client: 'pg', connection: process.env.DB_URL }); const adults = await query(db, UserSchema) .where(t => t.age, '>', 18) .orderBy(t => t.lastName); -// → typed as Array<{ id: number; firstName: string; lastName: string; age?: number; createdAt: Date }> +// → typed as Array<{ id: number; firstName: string; lastName: string; age: number | null; createdAt: Date }> ``` ## Schema Definition @@ -217,7 +220,7 @@ query(db, UserSchema) ## Eager Loading (No N+1) -Related rows are loaded in a **single query** using PostgreSQL CTEs and `jsonb_agg`. +Related rows are loaded in a **single query** using correlated PostgreSQL subqueries and `jsonb_agg`. ### `joinOne` — one-to-one / many-to-one @@ -248,27 +251,32 @@ const users = await query(db, UserSchema) foreignColumn: t => t.authorId, as: 'posts', limit: 5, - orderBy: { column: t => t.id, direction: 'desc' }, - }); + }, posts => posts.orderBy(t => t.id, 'desc')); // users[0].posts — typed as Array<{ id: number; title: string; authorId: number }> ✓ ``` -The `joinMany` spec supports: -- `limit` / `offset` — per-parent pagination using `row_number()` window functions -- `orderBy` — `{ column, direction }` for the sub-collection -- `foreignQuery` — pre-filtered `Knex.QueryBuilder` (e.g. for soft-delete scopes) -- `required` (`joinOne` only) — `true` = inner join, `false` = left join (nullable result) +The `joinMany` spec accepts per-parent `limit` / `offset`. Return the configured +child from its second argument for filtering, ordering and projection. Child +default scopes and soft-delete filters apply automatically. `joinOne` additionally +accepts `required: false` for nullable related objects; required relations filter +parents without a matching child. Raw `foreignQuery`, `mappers` and `orderBy` spec +properties are replaced by the typed child customizer. --- ## Escape Hatch -When you need a Knex feature not exposed by this API, use `.apply()`: +When Framework cannot infer a SQL shape, declare a complete object output schema. +The Knex callback runs once on an isolated builder. The output parser receives +each raw row once; it does not get an additional entity-decoding pass: ```typescript -const rows = await query(db, UserSchema) - .apply(qb => qb.forUpdate().noWait()) - .where(t => t.id, id); +const Totals = object({ count: number().coerce() }); +const totals = query(db, UserSchema) + .where(t => t.age, '>', 18) + .apply(qb => qb.clearSelect().count({ count: '*' }), { output: Totals }); +const rows = await totals; +// totals.rowSchema === Totals ``` --- @@ -295,7 +303,7 @@ const posts = await query(db, PostSchema) .scoped('published') .scoped('recent'); -// Bypass default scope (also skips soft-delete filter if present) +// Bypass only the default scope; use .withDeleted() separately for soft deletes const all = await query(db, PostSchema).unscoped(); ``` @@ -340,10 +348,12 @@ The accessor receives the schema's property-descriptor tree; each element resolv property name at runtime. This form is more refactor-safe but does not provide the compile-time `Pick<>` narrowing that the tuple form offers. -### Conflict rules +### Projection replacement -`.projected()` cannot be combined with `.select()`, `.distinct()`, or any aggregate -(`.count()`, `.min()`, etc.) on the same query. Attempting to do so throws at runtime. +Each `.select()`, `.projected()`, or aggregate projection returns a new query and +replaces its scalar selection. Previously prepared queries keep their shape. +Included relations remain independent of scalar projections. Projected, grouped, +distinct and joined queries are read-only; begin writes from an unprojected table query. ### Column-name mapping @@ -382,7 +392,7 @@ export const PostEntity = defineEntity(PostSchema) .belongsTo(t => t.author, l => l.authorId, r => r.id); ``` -The returned `Entity` carries the relation map in its type, so downstream `query(db, entity)` +The returned `Entity` carries the relation map in its type, so downstream `query(db, entity.schema)` calls (and `@cleverbrush/orm`'s `DbSet.include()`) get full inference. For many-to-many replacement flows, use the link table directly: delete the @@ -520,16 +530,16 @@ identifier quoting. Existing table/column metadata APIs are not restricted by it The read-only aliased builder requires an explicit, non-empty projection and supports `where`, `whereIn`, `whereNull`, `whereNotNull`, `orderBy`, `orderByRaw`, `groupBy`, `having`, `limit`, `offset`, `first`, `execute`, `transacting`, and -awaiting the query. `apply`/`toKnexQuery` remain raw escape hatches whose effects -on result shape/cardinality are the caller's responsibility. Values remain bound +awaiting the query. `apply` requires `{ output }`; `toKnexQuery()` returns an independent native SQL +snapshot whose execution bypasses Framework decoding. Values remain bound and identifiers quoted. Flat collection joins can repeat parents; they do not deduplicate or fetch one related row at a time. Choose ORM eager loading for nested related objects instead. For reusable connection/transaction handling, ordinary and aliased schemas retain the same inference through `createQuery(knex)`, `withTransaction(trx)`, and -`transaction(callback)`. Only ordinary-schema calls accept a custom Knex base -query. These APIs and their JSDoc are also available through `@cleverbrush/orm`. +`transaction(callback)`. Use `rawQuery(knex, Output, sql)` +or `.apply(configure, { output })` for an explicit raw output contract. These APIs and their JSDoc are also available through `@cleverbrush/orm`. ### Aggregates with optional output schemas @@ -571,9 +581,10 @@ use native SQL values, not decoded/text-formatted values. Counts reject malformed results and integers outside JavaScript's safe range. Sum/average preserve PostgreSQL's result as text without changing global driver parsers. This does not make floating-point source columns exact. Numeric/decimal/ -bigint extrema retain strings; because existing SQL overrides are not fully -represented in schema types, numeric extrema are conservatively typed as -`number | string | null`. Date extrema return `Date | null`. +bigint extrema retain strings; known SQL-type metadata controls their inferred +representation. Ordinary number extrema are `number | null`, exact numeric extrema +are `string | null`, and Date extrema are `Date | null`. Dynamic widened SQL hints +may conservatively infer `number | string | null`. An optional **output schema replaces the default decoder**. Its synchronous `parse` receives the raw driver value, including null, before default conversion. @@ -587,8 +598,8 @@ Scalar helpers clone the source, ignore its limit/offset/order, and retain filters, semantic joins, scopes, and transactions. They reject grouped, HAVING, distinct, and already aggregated queries; use aggregate projections for those. An empty scalar count is zero; other empty/all-null aggregates are null. An -empty grouped query returns no rows. Legacy `.count()`, `.sum()`, etc. keep their -existing behavior and signatures. +empty grouped query returns no rows. `.count()`, `.sum()`, etc. now produce typed +single-field projections with automatically updated row schemas. ### Eager-loading order @@ -653,9 +664,11 @@ URL is missing. CI runs PostgreSQL 16 integration tests alongside unit/type test ## Projection-aware reads -`withRowSchema()` is an **opt-in PostgreSQL read API**. Schema-aware queries are immutable, -detached read plans: capture the returned value when adding filters or includes. -They do not track entities, save changes, or run SQL when inspecting metadata. +Every Framework query is immutable and exposes its decoded `.rowSchema` +automatically. +Capture returned queries when adding filters, projections or includes. Inspecting +metadata never runs SQL. Ordinary table queries also support writes; projected +results are detached, while full ORM entities can still use identity tracking. ### Definitions can live in separate files @@ -666,11 +679,12 @@ import { knex } from './database.js'; const UserTable = object({ id: number().primaryKey(), name: string(), + tenantId: number().hasColumnName('tenant_id'), lastSeen: date().optional().hasColumnName('last_seen'), secret: string() }).hasTableName('users'); -export const userRead = createQuery(knex)(UserTable).withRowSchema() +export const userRead = createQuery(knex)(UserTable) .select(u => ({ id: u.id, name: u.name, lastSeen: u.lastSeen })); export const UserRow = userRead.rowSchema; ``` @@ -724,10 +738,10 @@ are rejected. A widened dynamic numeric SQL type has a conservative `number | string` read type. Driver custom parsers must still honor the declared representation, or decoding fails. -Timezone-less SQL dates/timestamps are interpreted as UTC in this opt-in mode; +Timezone-less SQL dates/timestamps are interpreted as UTC; explicit offsets preserve their instant. Native date fields are projected as text before decoding, so root and nested values do not depend on a driver's local-time -date parser. This is a deliberate opt-in convention, not a change to legacy reads. +date parser. This applies to ordinary reads, nested graphs and write-returning rows. Input defaults, preprocessors and input-only validators are not replayed on stored rows. Read schemas are structural output schemas. Optional schemas with input @@ -742,7 +756,7 @@ in private text columns, independently of the public date values. ### Projections, aliases and aggregates ```ts -const read = query(knex, alias(UserTable, 'user')).withRowSchema() +const read = query(knex, alias(UserTable, 'user')) .leftJoin(alias(ProfileTable, 'profile'), t => eq(t.user.id, t.profile.userId)) .select(t => ({ id: t.user.id, displayName: t.profile.displayName })); // displayName is nullable even if ProfileTable declares it required. @@ -755,14 +769,94 @@ Aliased reads require an explicit selection before execution or accessing readers. Typed aggregates work in object selections: count returns a safe number, sum/average preserve exact text, and empty extrema/sums remain nullable. An explicit Framework output schema replaces aggregate decoding and is parsed once; opaque -parser objects without schema introspection are rejected in this mode. +parser objects without schema introspection cannot supply projection metadata. + +### Filtering and ordering without changing the result schema + +Ordinary and aliased readers provide the following shape-preserving operations. +All return an independent reader with the **same `rowSchema` object**; retain the +returned reader when adding conditional filters. Filtering a nullable field does +not implicitly narrow its declared result type. + +| Operation | API | +| --- | --- | +| Comparisons and parenthesized groups | `where`, `andWhere`, `orWhere` | +| SQL null checks | `whereNull`, `whereNotNull`, `orWhereNull`, `orWhereNotNull` | +| Value-list or SELECT-subquery membership | `whereIn`, `whereNotIn`, `orWhereIn`, `orWhereNotIn` | +| SELECT-subquery existence | `whereExists`, `whereNotExists`, `orWhereExists`, `orWhereNotExists` | +| Bound custom predicates | `whereRaw(sql, bindings)`, `orWhereRaw(sql, bindings)` | +| Bound custom ordering | `orderByRaw(sql, bindings)` | +| Quoted mapped column reference | `ref(columnSelector)` | + +Reuse the prepared read from `user-read.ts` in a separate query-composition file: + +```ts +// user-search.ts +import { number, object, query, string } from '@cleverbrush/knex-schema'; +import { knex } from './database.js'; +import { userRead } from './user-read.js'; + +const UserLabel = object({ + userId: number().hasColumnName('user_id'), label: string() +}).hasTableName('user_labels'); + +export function searchUsers(tenantId: number, term: string, priorityUserId: number) { + const labeledUsers = query(knex, UserLabel) + .where(l => l.userId, userRead.ref(u => u.id)) + .where(l => l.label, term) + .select(l => l.userId).toKnexQuery(); + + return userRead.where(u => u.tenantId, tenantId) + .andWhere(group => group + .where(u => u.name, 'ilike', `%${term}%`) + .orWhereExists(labeledUsers)) + .orderByRaw('case when ?? = ? then 0 else 1 end', [ + userRead.ref(u => u.id), priorityUserId + ]) + .orderBy(u => u.name) + .orderBy(u => u.id); +} +``` + +The outer tenant filter applies to the **entire** search group. Group callbacks +are synchronous and run once when the predicate is attached, not during SQL +execution. Their predicate-only builder is immutable too: it has no selection, +join, ordering, write or execution methods. Return the configured group; a void +return, async result or unrelated builder is rejected. An empty group returned +unchanged adds no condition. Retaining a group and deriving another branch later +cannot change the already-attached predicates. + +`ref()` resolves mapped columns and generated aliases for ordinary readers, +explicit aliases for joined readers, and the correct child alias inside relation +customizers. Pass references as `??` identifier bindings or as Knex comparison +values when correlating a subquery. Values use `?` bindings. Raw SQL fragments +must be application-authored, not interpolated user input; this API is **not a SQL +sandbox**. Search escaping and application authorization remain caller policies. + +For membership, pass a value array or a single-column Knex SELECT subquery, for +example `read.whereIn(u => u.id, labelPage.toKnexQuery())`. Put ordering and limits +on that subquery to restrict IDs before joining/aggregating. EXISTS accepts a +Knex SELECT subquery; construct it separately instead of passing a Knex callback. +Subquery SQL and bindings are captured on attachment without database execution, +including nested subquery callbacks. Later changes to those builders do not +change the prepared reader. Empty IN lists match no rows; empty NOT IN lists +match all rows, with ordinary SQL null semantics for non-empty lists/subqueries. + +These operations also work in ordinary ORM reads and nested relation customizers. +Polymorphic roots also offer common-property predicates, including bound raw +filters. Use `forVariant()` for branch-specific predicates, projections and raw +ordering; root `orderBy()` orders the union globally. Numbered pagination retains raw ordering while +its count drops ordering/limits/offsets. `paginateAfter()` uses its explicit +complete `orderBy` specification, replacing prior ordering, including raw order. +Opaque SQL changes require `.apply(configure, { output })` or +`.selectRaw(sql, bindings, { output })`; the declared schema owns raw row parsing. ### Nested graphs Declare relations on entities as usual, and return the child query from customizers: ```ts -const read = db.projects.withRowSchema() +const read = db.projects .select(p => ({ id: p.id, name: p.name })) .include(r => r.tasks, tasks => tasks .select(t => ({ title: t.title, createdAt: t.createdAt })) @@ -793,7 +887,7 @@ default. Use `selectVariants(['photo'])` to narrow the returned union, or `forVariant()` to project a branch or load its relations: ```ts -const read = db.assets.withRowSchema() +const read = db.assets .forVariant('photo', q => q.include(r => r.tags)) .forVariant('text', q => q.select(a => ({ id: a.id, kind: a.kind, body: a.body }))); @@ -816,20 +910,22 @@ branch separately. Use `forVariant()` for branch-specific filtering/includes. ### API boundaries -- Enter read mode **before** legacy select/include/join, ordering/pagination, - raw callbacks or variant-specific operations. Continue configuration on the - immutable reader. Default scopes may filter rows but must not preselect, - order or paginate them. -- The opt-in surface is deliberately read-only. It is not a replacement for - entity writes or tracked queries, and has no raw SQL shape escape hatch. -- Reads are PostgreSQL-oriented; this does not promise equivalent JSON/CTI SQL - behavior on other Knex dialects. -- Declared graph metadata must match storage. Unsupported opaque shapes throw - rather than pretending an entity schema describes their output. Explicit - TypeScript assertions such as `hasType()` remain the caller's responsibility. -- Ordinary queries are mutable and retain their driver-value behavior. - Read mode is opt-in; it does not change application nulls or numeric values - globally. +- Query configuration is immutable across tables, aliases, polymorphic roots and + ORM entry points. Native Knex remains mutable only inside explicit raw callbacks + or separately obtained snapshots. +- Scopes are synchronous, shape-preserving callbacks. Return their configured + query. They can filter, order and paginate; they cannot select, load relations, + execute or write. Defaults are captured once when creating the query, not on + every render/execution. `unscoped()` preserves explicit filters and soft deletion. +- Writes require an unprojected, ungrouped, non-distinct table query with no loaded + relations. A primary key is required to target a limited/offset write safely. + Polymorphic writes use ORM `ofVariant()` instead of a union query. +- Reads and JSON/CTI SQL are PostgreSQL-oriented; equivalent behavior is not + promised on other Knex dialects. Schema metadata must match actual storage. +- Raw outputs require synchronous, introspectable Framework object schemas. They + are detached and read-only. Explicit assertions such as `hasType()` remain the + caller's responsibility. Use SQL aliases matching output property names and + text casts for exact numbers before driver parsing. - `getSyncMapper()` accepts only complete mappings with synchronous final steps and nested mappings. It never probes callbacks. Known async mappings are rejected; a disguised thenable throws when invoked. Keep `getMapper()` for diff --git a/libs/knex-schema/integration/immutable-queries.test.ts b/libs/knex-schema/integration/immutable-queries.test.ts new file mode 100644 index 00000000..c519f797 --- /dev/null +++ b/libs/knex-schema/integration/immutable-queries.test.ts @@ -0,0 +1,375 @@ +import { randomUUID } from 'node:crypto'; +import { + alias, + boolean, + ConcurrencyError, + createDb, + date, + defineEntity, + number, + object, + query, + rawQuery, + string +} from '@cleverbrush/orm'; +import Knex from 'knex'; +import { + afterAll, + beforeAll, + beforeEach, + describe, + expect, + it, + vi +} from 'vitest'; + +const connection = process.env.QUERY_TEST_DATABASE_URL; +if (!connection) throw new Error('QUERY_TEST_DATABASE_URL is required'); +const knex = Knex({ client: 'pg', connection, pool: { min: 0, max: 4 } }); +const table = `cb_immutable_${randomUUID().replaceAll('-', '')}`; +const firstId = '9007199254740993'; +const secondId = '9007199254740994'; +const otherId = '9007199254740995'; +const scope = vi.fn(q => q.where('tenantId', 1).where('enabled', true)); +const Account = object({ + id: number().bigint().primaryKey(), + tenantId: number().hasColumnName('tenant_id'), + enabled: boolean(), + name: string(), + balance: number().decimal(30, 6), + version: number().bigint().rowVersion(), + createdAt: date().hasColumnName('created_at'), + deletedAt: date().optional().hasColumnName('deleted_at') +}) + .hasTableName(table) + .softDelete() + .defaultScope(scope); +const Entity = defineEntity(Account); + +beforeAll(async () => { + await knex.schema.createTable(table, t => { + t.bigInteger('id').primary(); + t.integer('tenant_id').notNullable(); + t.boolean('enabled').notNullable(); + t.text('name').notNullable(); + t.decimal('balance', 30, 6).notNullable(); + t.bigInteger('version').notNullable().defaultTo('9007199254740993'); + t.timestamp('created_at', { useTz: true }) + .notNullable() + .defaultTo(knex.fn.now()); + t.timestamp('deleted_at', { useTz: true }); + }); +}); +beforeEach(async () => { + await knex(table).delete(); + await knex(table).insert([ + { + id: firstId, + tenant_id: 1, + enabled: true, + name: 'first', + balance: '12345678901234567890.012345' + }, + { + id: secondId, + tenant_id: 1, + enabled: true, + name: 'second', + balance: '0.000001' + }, + { + id: otherId, + tenant_id: 2, + enabled: false, + name: 'other', + balance: '9.000001' + } + ]); + scope.mockClear(); +}); +afterAll(async () => { + await knex.schema.dropTableIfExists(table); + await knex.destroy(); +}); + +describe('immutable public query semantics against PostgreSQL', () => { + it('inserts exact CTI keys and filters mapped polymorphic columns without losing the discriminator', async () => { + const baseName = `${table}_poly`; + const detailsName = `${table}_details`; + await knex.transaction(async trx => { + await trx.schema.createTable(baseName, t => { + t.bigInteger('asset_id').primary(); + t.text('asset_kind').notNullable(); + t.text('display_name').notNullable(); + }); + await trx.schema.createTable(detailsName, t => { + t.bigInteger('asset_id').primary(); + t.bigInteger('size').notNullable(); + }); + const Asset = defineEntity( + object({ + id: number() + .bigint() + .primaryKey() + .hasColumnName('asset_id'), + kind: string().hasColumnName('asset_kind'), + name: string().hasColumnName('display_name') + }).hasTableName(baseName) + ) + .discriminator('kind') + .ctiVariant( + 'photo', + defineEntity( + object({ + assetId: number() + .bigint() + .hasColumnName('asset_id'), + size: number().bigint() + }).hasTableName(detailsName) + ), + p => p.assetId + ); + const db = createDb(trx, { assets: Asset }, { tracking: true }); + const inserted = await db.assets + .ofVariant('photo') + .insert({ id: firstId, name: 'photo', size: secondId }); + expect(inserted).toMatchObject({ + id: firstId, + kind: 'photo', + name: 'photo', + size: secondId + }); + expect( + await db.assets + .where('name', 'photo') + .where('kind', 'photo') + .first() + ).toBe(inserted); + const page = await db.assets + .orderBy('id') + .paginate({ page: 1, pageSize: 1 }); + expect(page.total).toBe(1); + expect(page.data[0]).toBe(inserted); + expect(() => db.entry(page as any)).toThrow(/not tracked/i); + const variantPage = await db.assets + .ofVariant('photo') + .paginate({ page: 1, pageSize: 1 }); + expect(variantPage.data[0]).toBe(inserted); + expect(() => db.entry(variantPage as any)).toThrow(/not tracked/i); + const bindings = ['id']; + const rawOrdered = db.assets.orderByRaw( + '(__read_poly ->> ?)::numeric desc', + bindings + ); + bindings[0] = 'missing'; + expect((await rawOrdered)[0]).toBe(inserted); + expect(db.entry(inserted).isModified()).toBe(false); + inserted.size = otherId; + expect(db.entry(inserted).isModified()).toBe(true); + expect((await db.saveChanges()).updated).toBe(1); + expect((await trx(detailsName).first()).size).toBe(otherId); + expect(db.entry(inserted).isModified()).toBe(false); + await trx.schema.dropTable(detailsName); + await trx.schema.dropTable(baseName); + }); + }); + + it('captures scopes once, isolates branches and does not let OR bypass visibility', async () => { + const root = query(knex, Account); + const first = root.where('id', firstId); + const withAlternative = first.orWhere('id', otherId); + expect((await withAlternative).map(row => row.id)).toEqual([firstId]); + expect((await root).map(row => row.id)).toEqual([firstId, secondId]); + expect((await withAlternative.unscoped()).map(row => row.id)).toEqual([ + firstId, + otherId + ]); + expect(scope).toHaveBeenCalledTimes(1); + expect(first.rowSchema).toBe(root.rowSchema); + }); + + it('keeps scoped writes independent and applies scope/predicates to bulk updates', async () => { + const root = query(knex, Account); + const first = root.where('id', firstId); + expect( + await first.bulkUpdate([ + { where: { id: firstId }, set: { name: 'changed' } }, + { where: { id: secondId }, set: { name: 'wrong sibling' } }, + { where: { id: otherId }, set: { name: 'wrong tenant' } } + ]) + ).toBe(1); + expect((await root.orderBy('id')).map(row => row.name)).toEqual([ + 'changed', + 'second' + ]); + expect((await root.unscoped().where('id', otherId).first())?.name).toBe( + 'other' + ); + expect( + (await first.update({ balance: '12345678901234567890.999999' }))[0] + .balance + ).toBe('12345678901234567890.999999'); + expect(await root.where('id', secondId).delete()).toBe(1); + expect((await root).map(row => row.id)).toEqual([firstId]); + const restored = await root.onlyDeleted().restore(); + expect(restored[0].id).toBe(secondId); + expect(restored[0].deletedAt).toBeNull(); + }); + + it('decodes insert, conflict and bulk returning rows identically to SELECT', async () => { + const root = query(knex, Account); + const inserted = await root.insert({ + id: '9007199254740996', + tenantId: 1, + enabled: true, + name: 'inserted', + balance: '4.123456' + }); + expect(inserted).toMatchObject({ + id: '9007199254740996', + balance: '4.123456', + version: firstId, + deletedAt: null + }); + expect(inserted.createdAt).toBeInstanceOf(Date); + expect(await root.where('id', inserted.id).first()).toEqual(inserted); + const merged = await root.onConflict('id').merge({ + id: inserted.id, + tenantId: 1, + enabled: true, + name: 'merged', + balance: '5.123456' + }); + expect(merged).toMatchObject({ + id: inserted.id, + balance: '5.123456', + deletedAt: null + }); + const rows = await root.bulkInsert([ + { + id: '9007199254740997', + tenantId: 1, + enabled: true, + name: 'bulk', + balance: '6.123456' + } + ]); + expect(rows[0].createdAt).toBeInstanceOf(Date); + expect(rows[0].version).toBe(firstId); + expect(await root.insertMany([])).toEqual([]); + }); + + it('maintains tracked identity through immutable branches and page data only', async () => { + const tracked = createDb( + knex, + { accounts: Entity }, + { tracking: true } + ); + const root = tracked.accounts.query(); + const one = root.where('id', firstId); + const a = (await one)[0]; + expect((await one)[0]).toBe(a); + expect(await root.find(firstId)).toBe(a); + const page = await root + .orderBy('id') + .paginate({ page: 1, pageSize: 1 }); + expect(page.data[0]).toBe(a); + expect(() => tracked.entry(page as any)).toThrow(/not tracked/i); + const projected = await one + .select(t => ({ id: t.id, name: t.name })) + .first(); + expect(projected).not.toBe(a); + expect(() => tracked.entry(projected!)).toThrow(/not tracked/i); + const raw = await one + .selectRaw('id::text as id', [], { + output: object({ id: string() }) + }) + .first(); + expect(() => tracked.entry(raw!)).toThrow(/not tracked/i); + expect(tracked.entry(a).state).toBe('Unchanged'); + }); + + it('increments exact bigint versions and reloads using the same decoded shape', async () => { + const tracked = createDb( + knex, + { accounts: Entity }, + { tracking: true } + ); + const a = await tracked.accounts.findOrFail(firstId); + a.name = 'saved'; + await tracked.saveChanges(); + expect(a.version).toBe(secondId); + expect(tracked.entry(a).state).toBe('Unchanged'); + await knex(table).where('id', firstId).update({ + balance: '12345678901234567890.999998', + deleted_at: knex.fn.now() + }); + await tracked.reload(a); + expect(a.balance).toBe('12345678901234567890.999998'); + expect(a.createdAt).toBeInstanceOf(Date); + expect(a.deletedAt).toBeInstanceOf(Date); + }); + + it('rolls back writes and generated in-memory versions on a later concurrency failure', async () => { + const tracked = createDb( + knex, + { accounts: Entity }, + { tracking: true } + ); + const a = await tracked.accounts.findOrFail(firstId); + const b = await tracked.accounts.findOrFail(secondId); + a.name = 'pending first'; + b.name = 'pending second'; + await knex(table).where('id', secondId).update({ version: secondId }); + await expect(tracked.saveChanges()).rejects.toBeInstanceOf( + ConcurrencyError + ); + expect(a.version).toBe(firstId); + expect(b.version).toBe(firstId); + expect( + (await query(knex, Account).where('id', firstId).first())?.name + ).toBe('first'); + expect(tracked.entry(a).isModified()).toBe(true); + await tracked.reload(b); + await tracked.saveChanges(); + expect(a.version).toBe(secondId); + }); + + it('captures raw aliases and bindings while applying a parser exactly once', async () => { + const output = object({ count: number().coerce() }); + const parse = vi.spyOn(output, 'parse'); + const root = query(knex, alias(Account, 'account')).where( + t => t.account.id, + firstId + ); + const raw = root.selectRaw('count(*)::text as count', [], { output }); + expect(await raw).toEqual([{ count: 1 }]); + expect(parse).toHaveBeenCalledTimes(1); + expect( + await rawQuery( + knex, + object({ id: string() }), + knex(table) + .select(knex.raw('id::text as id')) + .where('id', firstId) + ) + ).toEqual([{ id: firstId }]); + }); + + it('allows filtered writes without a declared primary key when not paginated', async () => { + const NoKey = object({ name: string() }).hasTableName(table); + expect( + await query(knex, NoKey) + .where('name', 'other') + .update({ name: 'updated without key' }) + ).toEqual([{ name: 'updated without key' }]); + expect( + await query(knex, NoKey) + .where('name', 'updated without key') + .delete() + ).toBe(1); + await expect( + query(knex, NoKey).limit(1).update({ name: 'unsafe' }) + ).rejects.toThrow(/primary key/); + }); +}); diff --git a/libs/knex-schema/integration/queries.test.ts b/libs/knex-schema/integration/queries.test.ts index 0609dd02..c033de8b 100644 --- a/libs/knex-schema/integration/queries.test.ts +++ b/libs/knex-schema/integration/queries.test.ts @@ -74,7 +74,7 @@ const taskEntity = defineEntity(Task) const db = createDb(knex, { tasks: taskEntity }); it('transaction clones retain row schemas and raw-query safety guards', async () => { - const read = db.tasks.withRowSchema().select(t => ({ title: t.title })); + const read = db.tasks.select(t => ({ title: t.title })); await knex.transaction(async trx => { const inTransaction = read.transacting(trx); expect(inTransaction.rowSchema).toBe(read.rowSchema); @@ -82,13 +82,12 @@ it('transaction clones retain row schemas and raw-query safety guards', async () await read.where(t => t.id, 104).first() ); expect(() => - query(knex, Task) + query(knex, taskEntity.schema) .apply(q => { q.whereRaw('true'); }) .transacting(trx) - .withRowSchema() - ).toThrow(/before raw/); + ).toThrow(/output/); }); }); @@ -98,9 +97,7 @@ it('schema-aware reads stay detached in a tracked context', async () => { const listener = (sql: unknown) => calls.push(sql); knex.on('query', listener); try { - const read = tracked.tasks - .withRowSchema() - .select(t => ({ id: t.id, title: t.title })); + const read = tracked.tasks.select(t => ({ id: t.id, title: t.title })); expect(calls).toHaveLength(0); const row = await read.where(t => t.id, 102).first(); expect(calls).toHaveLength(1); @@ -112,8 +109,8 @@ it('schema-aware reads stay detached in a tracked context', async () => { }); it('schema-aware explicit joins preserve nullable objects and custom collections', async () => { - const read = query(knex, Task) - .withRowSchema() + const read = query(knex, taskEntity.schema) + .select(t => ({ id: t.id })) .joinOne( { @@ -152,7 +149,7 @@ it('schema-aware optional belongs-to joins keep unmatched parents', async () => { optional: true } ); const read = query(knex, optional.schema) - .withRowSchema() + .select(t => ({ id: t.id })) .include( r => r.owner, @@ -166,7 +163,7 @@ it('schema-aware optional belongs-to joins keep unmatched parents', async () => it('schema-aware flat joins preserve precision and scoped outer join nulls', async () => { const read = query(knex, alias(Task, 'task')) - .withRowSchema() + .leftJoin(alias(User, 'owner'), t => eq(t.task.ownerId, t.owner.id)) .select(t => ({ id: t.task.id, @@ -190,7 +187,7 @@ it('schema-aware flat joins preserve precision and scoped outer join nulls', asy it('schema-aware cursor pages keep microsecond ordering private', async () => { const read = db.tasks - .withRowSchema() + .select(t => ({ title: t.title })) .include(r => r.notes) .where(t => t.projectId, 1); @@ -321,7 +318,7 @@ afterAll(async () => { describe('flat joins', () => { it('reads exact values and selected relation schemas without hidden queries', async () => { const read = db.tasks - .withRowSchema() + .select(t => ({ id: t.id, amount: t.amount, @@ -353,14 +350,12 @@ describe('flat joins', () => { ]); expect(read.rowSchema.validate(rows[0]).valid).toBe(true); const exact = await db.tasks - .withRowSchema() + .where(t => t.id, 102) .select(t => ({ amount: t.amount })) .first(); expect(exact).toEqual({ amount: '9007199254740993.000001' }); - const AmountRow = db.tasks - .withRowSchema() - .select(t => ({ amount: t.amount })); + const AmountRow = db.tasks.select(t => ({ amount: t.amount })); const PublicAmount = object({ amount: string().optional() }); const toPublic = mapper() .configure(AmountRow.rowSchema, PublicAmount, m => @@ -377,7 +372,7 @@ describe('flat joins', () => { it('decodes typed aggregate outputs once in schema-aware reads', async () => { const read = db.tasks - .withRowSchema() + .where(t => t.projectId, 1) .select(t => ({ count: aggregate.count(), @@ -476,7 +471,10 @@ describe('eager ordering', () => { .orderBy(t => t.title) .orderBy(t => t.id, 'desc') .include(t => t.notes) - .include(t => t.owner) + .include( + t => t.owner, + owner => owner.unscoped().withDeleted() + ) .limit(2); expect(rows.map(row => row.id)).toEqual([104, 103]); expect(rows[0].notes).toHaveLength(2); @@ -488,7 +486,7 @@ describe('eager ordering', () => { it('handles projected-away sort/FK fields, mapped aliases, raw bindings, offset and transactions', async () => { await knex.transaction(async trx => { - const rows = await query(knex, Task) + const rows = await query(knex, taskEntity.schema) .where(t => t.projectId, 1) .orderByRaw('case when ?? = ? then 0 else 1 end, ?? desc', [ 'id', @@ -513,7 +511,7 @@ describe('eager ordering', () => { it('retains distinct/grouped parent cardinality', async () => { for (const distinct of [true, false]) { - let q = query(knex, Task) + let q = query(knex, taskEntity.schema) .where(t => t.projectId, 1) .select(t => ({ owner_id: t.ownerId })) .orderBy(t => t.ownerId); @@ -533,7 +531,7 @@ describe('eager ordering', () => { '"taskId" desc', '1 desc' ])('retains ordering by a projected alias or position: %s', async order => { - const rows = await query(knex, Task) + const rows = await query(knex, taskEntity.schema) .where(t => t.projectId, 1) .select(t => ({ taskId: t.id })) .orderByRaw(order) @@ -563,11 +561,10 @@ describe('aggregate results', () => { .include(t => t.notes) .countValue() ).toBe(4); - const visibleOwner = query(knex, Task) + const visibleOwner = query(knex, taskEntity.schema) .where(t => t.projectId, 1) .joinOne({ foreignSchema: User, - foreignQuery: query(knex, User), localColumn: t => t.ownerId, foreignColumn: t => t.id, as: 'owner', @@ -580,20 +577,22 @@ describe('aggregate results', () => { }); it('retains default-scope filters while ignoring default-scope pagination', async () => { - const Scoped = object({ id: number().primaryKey() }) + const Scoped = object({ + id: number().primaryKey(), + projectId: number().hasColumnName('project_id'), + deletedAt: date().optional().hasColumnName('deleted_at') + }) .hasTableName(tables.tasks) .defaultScope((q: any) => q - .where('project_id', 1) - .whereNull('deleted_at') + .where('projectId', 1) + .whereNull('deletedAt') .limit(1) .offset(1) ); expect(await query(knex, Scoped).countValue()).toBe(4); const Grouped = Scoped.defaultScope((q: any) => q.groupBy('id')); - await expect(query(knex, Grouped).countValue()).rejects.toThrow( - 'ungrouped' - ); + expect(() => query(knex, Grouped)).toThrow(/Scopes/); }); it('preserves sum/average/decimal extrema and returns date/string extrema', async () => { @@ -615,7 +614,7 @@ describe('aggregate results', () => { }); it('handles empty/all-null inputs and caller-supplied parsers', async () => { - const empty = () => query(knex, Task).where(t => t.id, -1); + const empty = () => query(knex, taskEntity.schema).where(t => t.id, -1); expect(await empty().countValue()).toBe(0); for (const method of [ 'sumValue', @@ -626,12 +625,12 @@ describe('aggregate results', () => { expect(await empty()[method]('amount')).toBe(null); } expect( - await query(knex, Task) + await query(knex, taskEntity.schema) .where(t => t.id, 103) .sumValue(t => t.amount) ).toBe(null); expect( - await query(knex, Task) + await query(knex, taskEntity.schema) .where(t => t.id, 105) .sumValue(t => t.amount, { output: number().isFloat().coerce() @@ -643,7 +642,7 @@ describe('aggregate results', () => { }); it('decodes every grouped aggregate without treating rows as entities', async () => { - const rows = await query(knex, Task) + const rows = await query(knex, taskEntity.schema) .where(t => t.projectId, 1) .groupBy(t => t.ownerId) .orderBy(t => t.ownerId) @@ -671,14 +670,14 @@ describe('aggregate results', () => { max: 'A' }); await expect( - query(knex, Task) + query(knex, taskEntity.schema) .groupBy(t => t.ownerId) .countValue() ).rejects.toThrow('ungrouped'); }); it('retains source state and transaction visibility', async () => { - const base = query(knex, Task) + const base = query(knex, taskEntity.schema) .where(t => t.projectId, 1) .select(t => ({ id: t.id })) .limit(1); @@ -745,11 +744,10 @@ describe('aggregate results', () => { describe('composite cursor pages', () => { it('applies required relation filters before testing whether another page exists', async () => { const base = () => - query(knex, Task) + query(knex, taskEntity.schema) .where(t => t.projectId, 1) .joinOne({ foreignSchema: User, - foreignQuery: query(knex, User), localColumn: t => t.ownerId, foreignColumn: t => t.id, as: 'owner', @@ -769,7 +767,7 @@ describe('composite cursor pages', () => { it('does not skip tied timestamps and retains microsecond precision with projections and includes', async () => { const base = () => - query(knex, Task) + query(knex, taskEntity.schema) .where(t => t.projectId, 1) .select(t => ({ taskId: t.id })) .joinMany({ @@ -813,7 +811,7 @@ describe('composite cursor pages', () => { .where(t => t.projectId, -1) .paginateAfter({ limit: 1, orderBy }); expect(empty).toEqual({ data: [], hasMore: false, nextCursor: null }); - const projected = await query(knex, Task) + const projected = await query(knex, taskEntity.schema) .where(t => t.projectId, 1) .select(t => t.id) .paginateAfter({ limit: 2, orderBy }); @@ -847,7 +845,7 @@ describe('composite cursor pages', () => { it('groups an existing OR filter before applying continuation predicates', async () => { const base = () => - query(knex, Task) + query(knex, taskEntity.schema) .where(t => t.id, 105) .orWhere(t => t.id, 104); const first = await base().paginateAfter({ limit: 1, orderBy }); diff --git a/libs/knex-schema/integration/read-graphs.test.ts b/libs/knex-schema/integration/read-graphs.test.ts index 7d6b4917..2cf4763e 100644 --- a/libs/knex-schema/integration/read-graphs.test.ts +++ b/libs/knex-schema/integration/read-graphs.test.ts @@ -136,9 +136,26 @@ afterAll(async () => { }); describe('schema-aware polymorphic graphs', () => { + it('allows schema-preserving predicates inside an explicit variant branch', async () => { + const read = db.assets + + .selectVariants(['photo']) + .forVariant('photo', photo => + photo + .where(p => p.where(a => a.id, 1).orWhere(a => a.id, 999)) + .whereRaw('?? > ?', [ + photo.ref(a => a.size), + '9007199254740992' + ]) + .orderByRaw('?? desc', [photo.ref(a => a.id)]) + ); + const rows = await read; + expect(rows.map(row => row.id)).toEqual([1]); + expect(read.rowSchema.validate(rows[0]).valid).toBe(true); + }); it('orders using native values even when branch projections omit sort fields', async () => { const read = db.assets - .withRowSchema() + .forVariant('photo', q => q.select(a => ({ id: a.id, kind: a.kind })) ) @@ -154,14 +171,12 @@ describe('schema-aware polymorphic graphs', () => { { id: 1, kind: 'photo' } ]); expect(() => - db.assets - .withRowSchema() - .forVariant('text', q => q.select(a => ({ id: a.id }))) + db.assets.forVariant('text', q => q.select(a => ({ id: a.id }))) ).toThrow(/retain.*discriminator/); }); it('exposes exact branch schemas, dates and numeric ordering in one statement', async () => { const read = db.assets - .withRowSchema() + .forVariant('photo', q => q.include( r => r.labels, @@ -204,7 +219,7 @@ describe('schema-aware polymorphic graphs', () => { it('decodes nested polymorphic relations and empty arrays without hidden fetches', async () => { const read = db.albums - .withRowSchema() + .include( r => r.assets, assets => @@ -242,25 +257,20 @@ describe('schema-aware polymorphic graphs', () => { ]); try { await expect( - db.assets - .withRowSchema() - .where(a => a.id, 3) - .execute() + db.assets.where(a => a.id, 3).execute() ).rejects.toThrow('unknown polymorphic discriminator'); await expect( - db.assets - .withRowSchema() - .where(a => a.id, 4) - .execute() + db.assets.where(a => a.id, 4).execute() ).rejects.toThrow('missing CTI variant body'); const tolerant = defineEntity(Asset.schema) .discriminator(a => a.kind) .ctiVariant('photo', Photo, p => p.assetId, { allowOrphan: true }); - const reader = createDb(knex, { assets: tolerant }) - .assets.withRowSchema() - .where(a => a.id, 4); + const reader = createDb(knex, { assets: tolerant }).assets.where( + a => a.id, + 4 + ); const orphan = await reader.first(); expect(orphan).toMatchObject({ id: 4, diff --git a/libs/knex-schema/integration/read-predicates.test.ts b/libs/knex-schema/integration/read-predicates.test.ts new file mode 100644 index 00000000..8e7af343 --- /dev/null +++ b/libs/knex-schema/integration/read-predicates.test.ts @@ -0,0 +1,373 @@ +import { randomUUID } from 'node:crypto'; +import { mapper } from '@cleverbrush/mapper'; +import { + aggregate, + alias, + array, + createDb, + date, + defineEntity, + eq, + number, + object, + query, + string +} from '@cleverbrush/orm'; +import Knex from 'knex'; +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; + +const connection = process.env.QUERY_TEST_DATABASE_URL; +if (!connection) throw new Error('QUERY_TEST_DATABASE_URL is required'); +const knex = Knex({ client: 'pg', connection }); +const prefix = `cb_pred_${randomUUID().replaceAll('-', '')}`; +const names = { + projects: `${prefix}_projects`, + tasks: `${prefix}_tasks`, + labels: `${prefix}_labels`, + links: `${prefix}_links` +}; +const Task = object({ + id: number().primaryKey(), + projectId: number().hasColumnName('project_id'), + title: string(), + kind: string(), + amount: number().decimal(24, 6), + done: date().optional() +}).hasTableName(names.tasks); +const Project = object({ + id: number().primaryKey(), + ownerId: number().hasColumnName('owner_id'), + name: string() +}).hasTableName(names.projects); +const Label = object({ + id: number().primaryKey(), + projectId: number().hasColumnName('project_id'), + name: string() +}).hasTableName(names.labels); +const Link = object({ + taskId: number().hasColumnName('task_id'), + labelId: number().hasColumnName('label_id') +}).hasTableName(names.links); +const ProjectEntity = defineEntity( + Project.addProp('tasks', array(Task).optional()) +).hasMany( + p => p.tasks, + p => p.id, + t => t.projectId +); +const timestamp = '2026-09-30T12:00:00.123456Z'; + +beforeAll(async () => { + await knex.schema.createTable(names.projects, t => { + t.integer('id').primary(); + t.integer('owner_id'); + t.text('name'); + }); + await knex.schema.createTable(names.tasks, t => { + t.integer('id').primary(); + t.integer('project_id'); + t.text('title'); + t.text('kind'); + t.decimal('amount', 24, 6); + t.timestamp('done', { useTz: true }); + }); + await knex.schema.createTable(names.labels, t => { + t.integer('id').primary(); + t.integer('project_id'); + t.text('name'); + }); + await knex.schema.createTable(names.links, t => { + t.integer('task_id'); + t.integer('label_id'); + }); + await knex(names.projects).insert([ + { id: 1, owner_id: 1, name: 'Alpha' }, + { id: 2, owner_id: 1, name: 'Beta' }, + { id: 3, owner_id: 2, name: 'Private' } + ]); + await knex(names.tasks).insert([ + { + id: 101, + project_id: 1, + title: 'literal 50%_!', + kind: 'task', + amount: '9007199254740993.000001', + done: timestamp + }, + { + id: 102, + project_id: 1, + title: 'label match', + kind: 'task', + amount: '12.340000', + done: null + }, + { + id: 103, + project_id: 2, + title: 'other project', + kind: 'task', + amount: '1.000000', + done: null + }, + { + id: 104, + project_id: 1, + title: 'milestone', + kind: 'milestone', + amount: '2.000000', + done: null + }, + { + id: 201, + project_id: 3, + title: 'literal 50%_!', + kind: 'task', + amount: '3.000000', + done: null + } + ]); + await knex(names.labels).insert([ + { id: 1, project_id: 1, name: 'Alpha' }, + { id: 2, project_id: 1, name: 'Beta' }, + { id: 3, project_id: 1, name: 'Unused' }, + { id: 4, project_id: 3, name: 'Private' } + ]); + await knex(names.links).insert([ + { task_id: 101, label_id: 1 }, + { task_id: 102, label_id: 1 }, + { task_id: 102, label_id: 2 }, + { task_id: 201, label_id: 2 } + ]); +}); +afterAll(async () => { + for (const name of [names.links, names.labels, names.tasks, names.projects]) + await knex.schema.dropTableIfExists(name); + await knex.destroy(); +}); + +describe('schema-aware read predicates against PostgreSQL', () => { + it('keeps outer access filters around grouped raw search and correlated EXISTS', async () => { + const base = query(knex, alias(Task, 'task')).join( + alias(Project, 'project'), + t => eq(t.task.projectId, t.project.id) + ); + const linked = query(knex, Link) + .where(l => l.labelId, 2) + .where( + l => l.taskId, + base.ref(t => t.task.id) + ) + .select(l => l.taskId) + .toKnexQuery(); + const filtered = base + .where(t => t.project.ownerId, 1) + .whereRaw('case when ?? = ? then ? else ? end = ?', [ + base.ref(t => t.task.kind), + 'milestone', + 'planned', + 'active', + 'active' + ]) + .andWhere(p => + p + .whereRaw("?? ilike ? escape '!'", [ + p.ref(t => t.task.title), + '%50!%!_!!%' + ]) + .orWhereExists(linked) + ); + const rowsQuery = filtered + .select(t => ({ + id: t.task.id, + amount: t.task.amount, + done: t.task.done + })) + .orderBy(t => t.task.id); + const countQuery = filtered.select(() => ({ + total: aggregate.count() + })); + const [rows, count] = await Promise.all([ + rowsQuery, + countQuery.first() + ]); + expect(rows.map(row => row.id)).toEqual([101, 102]); + expect(count?.total).toBe(2); + expect(rows[0].amount).toBe('9007199254740993.000001'); + expect(rows[0].done).toEqual(new Date(timestamp)); + expect(rows[1].done).toBeNull(); + for (const row of rows) + expect(rowsQuery.rowSchema.validate(row).valid).toBe(true); + expect(await rowsQuery.limit(1).offset(1)).toEqual([rows[1]]); + expect((await rowsQuery).length).toBe(2); + expect((await countQuery.first())?.total).toBe(2); + }); + + it('limits label IDs in a subquery before aggregating matching links', async () => { + const page = query(knex, Label) + .where(l => l.projectId, 1) + .orderBy(l => l.name) + .limit(1) + .offset(1) + .select(l => l.id) + .toKnexQuery(); + const read = query(knex, alias(Label, 'label')) + + .leftJoin(alias(Link, 'link'), t => eq(t.label.id, t.link.labelId)) + .whereIn(t => t.label.id, page) + .groupBy( + t => t.label.id, + t => t.label.name + ) + .select(t => ({ + id: t.label.id, + name: t.label.name, + total: aggregate.count(t.link.taskId) + })); + page.clear('limit').where('id', 99); + expect(await read).toEqual([{ id: 2, name: 'Beta', total: 2 }]); + expect(read.rowSchema.validate((await read)[0]).valid).toBe(true); + }); + + it('uses raw conditional ordering with independent ordinary numbered and cursor pages', async () => { + const source = query(knex, Project) + + .where(p => p.ownerId, 1) + .select(p => ({ id: p.id, name: p.name })); + const priority = source + .orderByRaw('case when ?? = ? then 0 else 1 end', [ + source.ref(p => p.id), + 2 + ]) + .orderBy(p => p.name); + const first = await priority.paginate({ page: 1, pageSize: 1 }); + const second = await priority.paginate({ page: 2, pageSize: 1 }); + expect(first.data.map(p => p.id)).toEqual([2]); + expect(second.data.map(p => p.id)).toEqual([1]); + expect(first.total).toBe(2); + expect(second.total).toBe(2); + expect(priority.rowSchema).toBe(source.rowSchema); + expect(source.toQuery()).not.toContain('order by'); + const cursor = await priority.paginateAfter({ + limit: 1, + orderBy: [{ column: p => p.id, direction: 'asc' }] + }); + expect(cursor.data.map(p => p.id)).toEqual([1]); + const next = await priority.paginateAfter({ + limit: 1, + cursor: cursor.nextCursor, + orderBy: [{ column: p => p.id, direction: 'asc' }] + }); + expect(next.data.map(p => p.id)).toEqual([2]); + }); + + it('orders aliased reads with bound CASE expressions and retains nullable left joins', async () => { + const source = query(knex, alias(Project, 'project')) + + .leftJoin(alias(Task, 'task'), t => + eq(t.project.id, t.task.projectId) + ) + .where(t => t.project.ownerId, 1) + .select(t => ({ id: t.project.id, taskId: t.task.id })); + const result = source + .orderByRaw('case when ?? = ? then 0 else 1 end', [ + source.ref(t => t.project.id), + 2 + ]) + .orderBy(t => t.task.id); + expect((await result)[0]).toEqual({ id: 2, taskId: 103 }); + expect(result.rowSchema).toBe(source.rowSchema); + const empty = query(knex, alias(Project, 'project')) + + .leftJoin(alias(Label, 'label'), t => + eq(t.project.id, t.label.projectId) + ) + .where(t => t.project.id, 2) + .select(t => ({ id: t.project.id, label: t.label.name })); + expect(await empty).toEqual([{ id: 2, label: null }]); + expect(empty.rowSchema.validate({ id: 2, label: null }).valid).toBe( + true + ); + }); + + it('filters nested ORM reads, stays detached, and reuses the prepared mapper', async () => { + const db = createDb( + knex, + { projects: ProjectEntity }, + { tracking: true } + ); + const source = db.projects + + .select(p => ({ id: p.id })) + .include( + p => p.tasks, + tasks => + tasks + .where(p => + p + .where(t => t.kind, 'task') + .orWhere(t => t.kind, 'note') + ) + .orderByRaw('?? desc', [tasks.ref(t => t.id)]) + .limit(1) + .select(t => ({ + title: t.title, + amount: t.amount, + done: t.done + })) + ); + const Target = object({ + id: number(), + tasks: array( + object({ + title: string(), + amount: string(), + done: date().nullable() + }) + ) + }); + const map = mapper() + .configure(source.rowSchema, Target, m => m) + .getSyncMapper(source.rowSchema, Target); + const read = source.where(p => p.id, 1); + expect(read.rowSchema).toBe(source.rowSchema); + const [row] = await read; + expect(map(row)).toEqual({ + id: 1, + tasks: [{ title: 'label match', amount: '12.340000', done: null }] + }); + row.tasks[0].title = 'detached mutation'; + await db.saveChanges(); + expect((await knex(names.tasks).where('id', 102).first()).title).toBe( + 'label match' + ); + }); + + it('preserves captured predicates in transaction clones without touching the source', async () => { + const read = query(knex, Task).where(t => t.projectId, 1); + const noLabels = knex(names.links) + .select('task_id') + .where( + 'task_id', + read.ref(t => t.id) + ); + const filtered = read + .whereNotExists(noLabels) + .select(t => ({ id: t.id })); + await knex.transaction(async trx => { + await trx(names.tasks).insert({ + id: 105, + project_id: 1, + title: 'transaction', + kind: 'task', + amount: '1', + done: null + }); + const transactional = filtered.transacting(trx).orderBy(t => t.id); + expect(transactional.rowSchema).toBe(filtered.rowSchema); + expect((await transactional).map(t => t.id)).toEqual([104, 105]); + await trx.rollback(); + }); + expect((await filtered).map(t => t.id)).toEqual([104]); + }); +}); diff --git a/libs/knex-schema/src/AliasedReadQuery.ts b/libs/knex-schema/src/AliasedQueryBuilder.ts similarity index 65% rename from libs/knex-schema/src/AliasedReadQuery.ts rename to libs/knex-schema/src/AliasedQueryBuilder.ts index 8b34cb1f..d7cebc8c 100644 --- a/libs/knex-schema/src/AliasedReadQuery.ts +++ b/libs/knex-schema/src/AliasedQueryBuilder.ts @@ -1,7 +1,7 @@ import { type InferType, object } from '@cleverbrush/schema'; import type { Knex } from 'knex'; import type { - AliasedQueryBuilder, + AliasedQuerySource, AliasTables, JoinPredicate, TableAlias @@ -11,6 +11,14 @@ import { type AliasedColumn, COLUMN } from './expressions.js'; +import { OpaqueQuery, type QueryOutput } from './OpaqueQuery.js'; +import { + captureReadRaw, + captureValue, + type ReadPredicate, + type ReadPredicateContext, + ReadPredicates +} from './read-predicates.js'; import { compileReadProjection, type ReadField } from './read-projection.js'; import { compileReadSchema, @@ -20,7 +28,7 @@ import { type ReadValue, type SchemaForValue } from './read-schema.js'; -import type { ReadColumn, ReadProjection } from './SchemaReadQuery.js'; +import type { ReadColumn, ReadProjection } from './SchemaQueryBuilder.js'; /** Schema-backed aliases whose exact numeric and outer-join values match decoded rows. */ export type ReadAliasTables = { @@ -36,11 +44,17 @@ type Selection = Record | AggregateExpression>; type Selector = (tables: ReadAliasTables) => AliasedColumn; /** Immutable flat joined read. Supply select() before accessing rowSchema or executing. */ -export class AliasedReadQuery { +export class AliasedQueryBuilder< + T, + Row extends ReadObject = never +> extends ReadPredicates> { private fields?: Record; private schema?: Row; - /** @internal Enter through an aliased query's withRowSchema() method. */ - constructor(private planner: AliasedQueryBuilder) {} + private predicates: readonly ReadPredicate[] = []; + /** @internal Create through query(knex, alias(schema, name)). */ + constructor(private planner: AliasedQuerySource) { + super(); + } /** Exact structural schema, stable across operations that do not change selection. */ get rowSchema(): Row { @@ -59,7 +73,7 @@ export class AliasedReadQuery { join( table: N extends keyof T ? never : TableAlias, on: (tables: ReadAliasTables>) => JoinPredicate - ): AliasedReadQuery, Row> { + ): AliasedQueryBuilder, Row> { const copy = this.copy(); copy.planner = copy.planner.join(table, on as any) as any; return copy as any; @@ -70,7 +84,7 @@ export class AliasedReadQuery { on: ( tables: ReadAliasTables> ) => JoinPredicate - ): AliasedReadQuery, Row> { + ): AliasedQueryBuilder, Row> { const copy = this.copy(); copy.planner = copy.planner.leftJoin(table, on as any) as any; return copy as any; @@ -78,11 +92,7 @@ export class AliasedReadQuery { /** Select exact columns and aggregates; opaque raw expressions are deliberately unsupported. */ select

( select: (tables: ReadAliasTables) => P - ): AliasedReadQuery> { - if (this.fields) - throw new ReadSchemaError( - 'Only one projection is allowed per read query' - ); + ): AliasedQueryBuilder> { const { knex, columns } = this.planner.readContext(); const entries = Object.values( columns as Record>> @@ -113,33 +123,31 @@ export class AliasedReadQuery { ) as unknown as Row; return copy as any; } - /** Add a bound equality predicate. */ - where(column: Selector, value: unknown): this; - /** Add a bound predicate with an explicit supported operator. */ - where(column: Selector, operator: string, value: unknown): this; - /** Return a filtered clone without changing metadata identity. */ - where(column: Selector, ...args: [unknown] | [string, unknown]): this { - const copy = this.copy(); - if (args.length === 1) copy.planner.where(column as any, args[0]); - else copy.planner.where(column as any, args[0], args[1]); - return copy; - } - /** Match SQL null, including absent outer-joined rows. */ - whereNull(column: Selector): this { - const copy = this.copy(); - copy.planner.whereNull(column as any); - return copy; - } - /** Exclude SQL null without silently narrowing result types. */ - whereNotNull(column: Selector): this { - const copy = this.copy(); - copy.planner.whereNotNull(column as any); - return copy; + protected readPredicateContext(): ReadPredicateContext> { + const { knex, columns } = this.planner.readContext(); + const entries = Object.values( + columns as Record>> + ).flatMap(table => Object.values(table)); + return { + knex, + column: selector => { + if (typeof selector !== 'function') + throw new ReadSchemaError( + 'Aliased predicates require a column selector' + ); + const column = selector(columns as ReadAliasTables); + if (!entries.includes(column)) + throw new ReadSchemaError( + 'Predicate column does not belong to this query' + ); + const info = column[COLUMN]; + return `${info.alias}.${info.column}`; + } + }; } - /** Match a bound value list. */ - whereIn(column: Selector, values: readonly unknown[]): this { + protected addReadPredicate(predicate: ReadPredicate): this { const copy = this.copy(); - copy.planner.whereIn(column as any, values); + copy.predicates = [...this.predicates, predicate]; return copy; } /** Order by native database values before decoding. */ @@ -148,6 +156,14 @@ export class AliasedReadQuery { copy.planner.orderBy(column as any, direction); return copy; } + /** Append trusted raw ordering with captured bindings; ref() quotes mapped aliased columns. */ + orderByRaw(sql: string, bindings: readonly Knex.RawBinding[] = []): this { + const { knex } = this.planner.readContext(); + const captured = captureReadRaw(knex, sql, bindings)().toSQL(); + const copy = this.copy(); + copy.planner.orderByRaw(captured.sql, captured.bindings); + return copy; + } /** Group native columns for an aggregate projection. */ groupBy(...columns: Selector[]): this { const copy = this.copy(); @@ -163,7 +179,11 @@ export class AliasedReadQuery { right: unknown ): this { const copy = this.copy(); - copy.planner.having(value as any, operator, right); + copy.planner.having( + value as any, + operator, + captureValue(this.planner.readContext().knex, right)() + ); return copy; } /** Limit the flat row count, including repeated parents produced by joins. */ @@ -192,6 +212,7 @@ export class AliasedReadQuery { compile(): Knex.QueryBuilder { void this.rowSchema; const { sql, knex } = this.planner.readContext(); + for (const predicate of this.predicates) predicate(sql); return sql .clearSelect() .select( @@ -207,6 +228,41 @@ export class AliasedReadQuery { toQuery(): string { return this.compile().toQuery(); } + /** Return an independent mutable Knex snapshot. */ + toKnexQuery(): Knex.QueryBuilder { + return this.compile(); + } + /** Configure raw SQL once and declare its complete output contract. */ + apply( + configure: (query: Knex.QueryBuilder) => Knex.QueryBuilder | undefined, + options: QueryOutput + ): OpaqueQuery { + const { knex, sql: source } = this.planner.readContext(); + const sql = this.fields ? this.compile() : source; + if (!this.fields) + for (const predicate of this.predicates) predicate(sql); + const result = configure(sql); + if (result !== undefined && result !== sql) { + if (result instanceof Promise) void result.catch(() => {}); + throw new ReadSchemaError( + 'Raw configuration must synchronously configure the supplied Knex builder' + ); + } + return OpaqueQuery.capture(knex, sql, options); + } + /** Replace the projection with trusted SQL and an explicit output contract. */ + selectRaw( + sql: string, + bindings: readonly Knex.RawBinding[], + options: QueryOutput + ): OpaqueQuery { + const { knex } = this.planner.readContext(); + const captured = captureReadRaw(knex, sql, bindings); + return this.apply( + query => query.clearSelect().select(captured()), + options + ); + } /** Execute one statement and validate/decode its detached results. */ async execute(): Promise[]> { const nodes = Object.fromEntries( diff --git a/libs/knex-schema/src/OpaqueQuery.ts b/libs/knex-schema/src/OpaqueQuery.ts new file mode 100644 index 00000000..72b1a842 --- /dev/null +++ b/libs/knex-schema/src/OpaqueQuery.ts @@ -0,0 +1,130 @@ +import type { InferType } from '@cleverbrush/schema'; +import type { Knex } from 'knex'; +import { captureReadRaw } from './read-predicates.js'; +import { type ReadObject, ReadSchemaError } from './read-schema.js'; + +/** Explicit output contract required when Framework cannot infer the SQL row shape. */ +export interface QueryOutput { + /** Synchronous, introspectable Framework object schema; parses each raw row once. */ + output: S; +} + +/** + * Immutable raw SELECT with an explicit output contract. Knex remains mutable only + * inside apply(); its compiled SQL and bindings are captured before this object is returned. + */ +export class OpaqueQuery { + /** The supplied output schema, without a second decoding or input-parser pass. */ + readonly rowSchema: S; + /** @internal Use a query's apply() or selectRaw() method. */ + constructor( + private readonly knex: Knex, + private readonly sql: Knex.QueryBuilder, + options: QueryOutput + ) { + if ( + !options?.output || + typeof options.output.introspect !== 'function' || + options.output.introspect().type !== 'object' + ) + throw new ReadSchemaError( + 'Raw query output requires an introspectable Framework object schema' + ); + this.rowSchema = options.output; + } + /** @internal Capture the completed SELECT; external builders and callbacks are not retained. */ + static capture( + knex: Knex, + sql: Knex.QueryBuilder, + options: QueryOutput + ): OpaqueQuery { + const compiled = sql.toSQL(); + if (Array.isArray(compiled) || compiled.method !== 'select') + throw new ReadSchemaError( + 'Raw query configuration must produce a SELECT' + ); + const raw = captureReadRaw( + knex, + compiled.sql, + compiled.bindings as Knex.RawBinding[] + ); + return new OpaqueQuery( + knex, + knex + .queryBuilder() + .from(raw().wrap('(', ') as __opaque')) + .select('*'), + options + ); + } + /** Configure isolated Knex SQL once; every opaque change must declare its resulting output. */ + apply( + configure: (query: Knex.QueryBuilder) => Knex.QueryBuilder | undefined, + options: QueryOutput + ): OpaqueQuery { + const sql = this.toKnexQuery(); + const result = configure(sql); + if (result !== undefined && result !== sql) { + if (result instanceof Promise) void result.catch(() => {}); + throw new ReadSchemaError( + 'Raw configuration must synchronously configure the supplied Knex builder' + ); + } + return OpaqueQuery.capture(this.knex, sql, options); + } + /** Return an independent mutable SQL snapshot. */ + toKnexQuery(): Knex.QueryBuilder { + return this.sql.clone(); + } + /** Render debug SQL without execution. */ + toQuery(): string { + return this.sql.toQuery(); + } + /** Limit an independent query while keeping schema identity. */ + limit(count: number): OpaqueQuery { + if (!Number.isInteger(count) || count < 0) + throw new ReadSchemaError('Limit must be a non-negative integer'); + return new OpaqueQuery(this.knex, this.sql.clone().limit(count), { + output: this.rowSchema + }); + } + /** Offset an independent query while keeping schema identity. */ + offset(count: number): OpaqueQuery { + if (!Number.isInteger(count) || count < 0) + throw new ReadSchemaError('Offset must be a non-negative integer'); + return new OpaqueQuery(this.knex, this.sql.clone().offset(count), { + output: this.rowSchema + }); + } + /** Bind an independent query to an existing transaction. */ + transacting(trx: Knex.Transaction): OpaqueQuery { + return new OpaqueQuery(trx, this.sql.clone().transacting(trx), { + output: this.rowSchema + }); + } + /** Execute again on every call and synchronously parse each raw row exactly once. */ + async execute(): Promise[]> { + return (await this.sql.clone()).map((row: unknown) => { + const result = this.rowSchema.parse(row); + if (result && typeof (result as any).then === 'function') { + if (result instanceof Promise) void result.catch(() => {}); + throw new ReadSchemaError( + 'Raw output schemas must parse synchronously' + ); + } + return result; + }); + } + /** Fetch the first row, or undefined. */ + async first(): Promise | undefined> { + return (await this.limit(1).execute())[0]; + } + /** Awaiting executes this query; results are not cached. */ + // biome-ignore lint/suspicious/noThenProperty: query builders intentionally support await + then[], E = never>( + resolve?: ((rows: InferType[]) => R | PromiseLike) | null, + reject?: ((error: any) => E | PromiseLike) | null + ): Promise { + return this.execute().then(resolve, reject); + } +} diff --git a/libs/knex-schema/src/PolymorphicReadQuery.ts b/libs/knex-schema/src/PolymorphicQueryBuilder.ts similarity index 53% rename from libs/knex-schema/src/PolymorphicReadQuery.ts rename to libs/knex-schema/src/PolymorphicQueryBuilder.ts index e813f368..b412ec7a 100644 --- a/libs/knex-schema/src/PolymorphicReadQuery.ts +++ b/libs/knex-schema/src/PolymorphicQueryBuilder.ts @@ -10,18 +10,21 @@ import { import type { Knex } from 'knex'; import { buildColumnMap, getPrimaryKeyColumns } from './columns.js'; import type { SchemaProps } from './entity.js'; -import { COLUMN } from './expressions.js'; -import { getVariants } from './extension.js'; -import { - getEffectiveBaseQuery, - getSchemaQueryBuilderCtor -} from './operations/helpers.js'; +import { type AliasedColumn, COLUMN } from './expressions.js'; +import { getTableName, getVariants } from './extension.js'; +import { OpaqueQuery, type QueryOutput } from './OpaqueQuery.js'; import { privateColumn } from './operations/ordering.js'; import type { EntityReadSchema, ReadRelations, ReadVariantMetadata } from './read-entity.js'; +import { + captureReadRaw, + type ReadPredicate, + type ReadPredicateContext, + ReadPredicates +} from './read-predicates.js'; import { type ObjectReadSchema, type ReadObject, @@ -34,8 +37,12 @@ import { type ReadColumns, type ReadCorrelation, type ReadQueryShape, - SchemaReadQuery -} from './SchemaReadQuery.js'; + type Related, + type RelationField, + type SchemaAwareQuery, + SchemaQueryBuilder +} from './SchemaQueryBuilder.js'; +import type { PaginationResult } from './types.js'; type VariantMap = ReadVariantMetadata extends { variants: infer V } ? V : {}; @@ -109,7 +116,7 @@ type BranchQueries< S extends ReadObject, B extends Record > = { - [K in keyof B & keyof VariantMap & string]: SchemaReadQuery< + [K in keyof B & keyof VariantMap & string]: SchemaQueryBuilder< BranchSource, B[K], ReadRelations & ReadRelations> @@ -118,35 +125,69 @@ type BranchQueries< type Selector = ( columns: ReadColumns> ) => { readonly [COLUMN]: { column: string } }; +let polymorphicAliasSequence = 0; +type PolymorphicOrder = + | { key: string; direction: 'asc' | 'desc' } + | { raw: () => Knex.Raw }; /** * Immutable polymorphic read graph. Branches are combined in one PostgreSQL statement; * rowSchema is a real discriminated union and variantRowSchemas supplies object schemas * for separately configured mappers. Framework does not choose application DTO mappings. */ -export class PolymorphicReadQuery< +export class PolymorphicQueryBuilder< S extends ReadObject, B extends Record = VariantReadSchemas -> { +> extends ReadPredicates>> { /** @internal Nominal identity for typed child-query customizers. */ declare readonly [READ_QUERY]: true; /** Runtime union matching decoded results, including selected variant bodies. */ readonly rowSchema: PolymorphicRowSchema; /** Stable object schemas keyed by discriminator, suitable for mapper.configure(). */ readonly variantRowSchemas: Readonly; - private branches: Record>; - private fallback: SchemaReadQuery; - private orders: Array<{ key: string; direction: 'asc' | 'desc' }> = []; + private branches: Record< + string, + SchemaQueryBuilder + >; + private fallback: SchemaQueryBuilder; + private orders: PolymorphicOrder[] = []; private rowLimit?: number; private rowOffset?: number; private includeUnknown = true; + private predicates: readonly ReadPredicate[] = []; + private defaults?: { + predicates: readonly ReadPredicate[]; + orders: PolymorphicOrder[]; + limit?: number; + offset?: number; + }; + private skipDefaults = false; + private deleted: 'exclude' | 'include' | 'only' = 'exclude'; + private readonly predicateAlias = + `__polymorphic_${polymorphicAliasSequence++}`; + private readonly columns: Record>; - /** @internal Use withRowSchema() instead of constructing polymorphic readers. */ + /** @internal Create through query() or an ORM DbSet. */ constructor( private readonly knex: Knex, private readonly source: S, private readonly base: Knex.QueryBuilder ) { + super(); + this.columns = Object.fromEntries( + Object.entries(source.introspect().properties).map( + ([key, schema]) => [ + key, + { + [COLUMN]: { + alias: this.predicateAlias, + column: key, + schema + } + } + ] + ) + ); const config = getVariants(source); if (!config) throw new ReadSchemaError('No polymorphic variants are declared'); @@ -158,16 +199,27 @@ export class PolymorphicReadQuery< buildColumnMap(source).propToCol.get(config.discriminatorKey) ?? config.discriminatorKey; // Invert only the discriminator guard, not caller/default-scope filters. - this.fallback = new SchemaReadQuery( + this.fallback = new SchemaQueryBuilder( knex, common, knex .from(base.clone().as('__read_unknown')) + .select( + Object.fromEntries( + Object.keys(common.introspect().properties).map(key => [ + key, + knex.ref( + `__read_unknown.${buildColumnMap(source).propToCol.get(key) ?? key}` + ) + ]) + ) + ) .where(q => q .whereNotIn(discriminator, Object.keys(config.variants)) .orWhereNull(discriminator) - ) + ), + this.predicateAlias ); const schemas = Object.fromEntries( Object.entries(this.branches).map(([key, q]) => [key, q.rowSchema]) @@ -176,11 +228,47 @@ export class PolymorphicReadQuery< this.rowSchema = this.unionSchema( schemas ) as unknown as PolymorphicRowSchema; + const scope = source.introspect().extensions?.defaultScope; + if (typeof scope === 'function') { + const configured = scope(this.copy()); + if ( + !this.sameSource(configured) || + configured.rowSchema !== this.rowSchema || + configured.deleted !== this.deleted || + configured.skipDefaults !== this.skipDefaults || + configured.knex !== this.knex + ) { + if (configured instanceof Promise) + void configured.catch(() => {}); + throw new ReadSchemaError( + 'Scopes must synchronously return a shape-preserving query' + ); + } + this.defaults = { + predicates: configured.predicates, + orders: configured.orders, + limit: configured.rowLimit, + offset: configured.rowOffset + }; + } } private commonSource(): ReadObject { const info = this.source.introspect(); - return (object(info.properties) as any) + const relationNames = new Set( + ((info.extensions?.relations ?? []) as { name: string }[]).map( + relation => relation.name + ) + ); + const properties = Object.fromEntries( + Object.entries(info.properties) + .filter(([key]) => !relationNames.has(key)) + .map(([key, schema]) => [ + key, + (schema as ReadSchema).withExtension('columnName', key) + ]) + ); + return (object(properties) as any) .withExtension('tableName', info.extensions?.tableName) .withExtension('relations', info.extensions?.relations ?? []); } @@ -197,7 +285,10 @@ export class PolymorphicReadQuery< ); } - private branch(key: string, body: boolean): SchemaReadQuery { + private branch( + key: string, + body: boolean + ): SchemaQueryBuilder { const config = getVariants(this.source)!; const variant = config.variants[key]; const baseInfo = this.source.introspect(); @@ -237,10 +328,11 @@ export class PolymorphicReadQuery< throw new ReadSchemaError( 'CTI read graphs require a single-column primary key' ); - const Constructor = getSchemaQueryBuilderCtor(); - const bodyQuery = getEffectiveBaseQuery( - new Constructor(this.knex, variant.schema) - ).clone(); + const bodyQuery = new SchemaQueryBuilder( + this.knex, + variant.schema, + this.knex(getTableName(variant.schema)) + ).storageQuery(); query.leftJoin( bodyQuery.as(bodyAlias), `${bodyAlias}.${variant.foreignKey}`, @@ -310,10 +402,11 @@ export class PolymorphicReadQuery< '__read_cti_present' ); } - return new SchemaReadQuery( + return new SchemaQueryBuilder( this.knex, schema, - query.select(columns) + query.select(columns), + this.predicateAlias ); } @@ -327,7 +420,7 @@ export class PolymorphicReadQuery< /** @internal Check the identity of the original read source, retained by clones. */ sameSource(other: unknown): boolean { return ( - other instanceof PolymorphicReadQuery && + other instanceof PolymorphicQueryBuilder && this.base === other.base && this.source === other.source ); @@ -343,37 +436,189 @@ export class PolymorphicReadQuery< }); } - /** Add a comparison to every branch without changing its declared result schema. */ - where(selector: Selector, value: unknown): this; - /** Add an explicit supported comparison operator to every branch. */ - where(selector: Selector, operator: string, value: unknown): this; - /** Add a bound comparison; existing reader instances remain unchanged. */ - where(selector: Selector, ...args: unknown[]): this { + protected readPredicateContext(): ReadPredicateContext< + ReadColumns> + > { + return { + knex: this.knex, + column: selector => { + const column = + typeof selector === 'string' + ? this.columns[selector] + : selector(this.columns as any); + if (!column || !Object.values(this.columns).includes(column)) + throw new ReadSchemaError( + 'Column does not belong to this polymorphic query' + ); + return `${this.predicateAlias}.${column[COLUMN].column}`; + } + }; + } + protected addReadPredicate(predicate: ReadPredicate): this { + const copy = this.copy(); + copy.predicates = [...this.predicates, predicate]; + return copy; + } + /** Remove the default scope while preserving explicit predicates. */ + unscoped(): this { + const copy = this.copy(); + copy.skipDefaults = true; + return copy; + } + /** Include soft-deleted entities in every branch. */ + withDeleted(): this { + const copy = this.copy(); + copy.deleted = 'include'; + return copy; + } + /** Match only soft-deleted entities in every branch. */ + onlyDeleted(): this { const copy = this.copy(); - for (const [key, branch] of Object.entries(copy.branches)) - copy.branches[key] = (branch.where as Function)(selector, ...args); - copy.fallback = (copy.fallback.where as Function)(selector, ...args); + copy.deleted = 'only'; return copy; } + /** Apply a named immutable scope once. */ + scoped(name: string): this { + const scope = ( + this.source.introspect().extensions?.scopes as + | Record + | undefined + )?.[name]; + if (!scope) throw new ReadSchemaError(`Unknown scope: ${name}`); + const configured = scope(this.copy()); + if ( + !this.sameSource(configured) || + configured.rowSchema !== this.rowSchema || + configured.deleted !== this.deleted || + configured.skipDefaults !== this.skipDefaults || + configured.knex !== this.knex + ) { + if (configured instanceof Promise) void configured.catch(() => {}); + throw new ReadSchemaError( + 'Scopes must synchronously return a shape-preserving query' + ); + } + return configured; + } + /** True when all branches retain complete entity rows. */ + get returnsEntityRows(): boolean { + return Object.values(this.branches).every( + branch => branch.returnsEntityRows + ); + } + /** Customize a relation on one discriminator branch. */ + includeVariant( + key: keyof B & keyof VariantMap & string, + relation: string, + customize?: (query: SchemaQueryBuilder) => ReadQueryShape + ): this { + return this.forVariant(key, query => + query.include(() => relation as any, customize as any) + ) as unknown as this; + } + /** Load a common relation on every branch, configuring the child exactly once. */ + include< + K extends keyof ReadRelations & string, + Child extends ReadQueryShape = SchemaAwareQuery< + Related[K]> + > + >( + selector: K | ((relations: { [P in keyof ReadRelations]: P }) => K), + customize?: ( + query: SchemaAwareQuery[K]>> + ) => Child + ): PolymorphicQueryBuilder< + S, + { + [P in keyof B]: ObjectSchemaBuilder< + SchemaProps & + Record< + K, + RelationField[K], Child['rowSchema']> + > + >; + } + > { + const relations = (this.source.introspect().extensions?.relations ?? + []) as { name: string }[]; + const name = + typeof selector === 'string' + ? selector + : selector( + Object.fromEntries( + relations.map(relation => [ + relation.name, + relation.name + ]) + ) as any + ); + if (!relations.some(relation => relation.name === name)) { + const variants = getVariants(this.source)!.variants; + const candidates = Object.entries(variants).filter(([, variant]) => + variant.relations.some(relation => relation.name === name) + ); + if (candidates.length > 1) + throw new ReadSchemaError( + `Ambiguous relation: ${name}; use includeVariant` + ); + if (candidates.length === 1) + return this.includeVariant( + candidates[0][0] as any, + name, + customize as any + ) as any; + throw new ReadSchemaError(`Unknown relation: ${name}`); + } + const copy = this.copy(); + const entries = Object.entries(copy.branches); + const [firstKey, first] = entries[0]; + const prepared = first.include(name, customize as any); + copy.branches[firstKey] = prepared; + for (const [key, branch] of entries.slice(1)) + copy.branches[key] = branch.includeFrom(name, prepared); + copy.fallback = copy.fallback.includeFrom(name, prepared); + copy.refresh(); + return copy as any; + } + /** Filter one branch using schema property names; other variants remain unaffected. */ + whereVariant( + key: keyof B & keyof VariantMap & string, + selector: string | ((columns: any) => any), + operator: string, + value: unknown + ): this { + return this.forVariant(key, query => + query.where(selector as any, operator, value) + ) as unknown as this; + } /** Order all variants together, not independently within each branch. */ - orderBy(selector: Selector, direction: 'asc' | 'desc' = 'asc'): this { + orderBy( + selector: + | Selector + | (keyof ReadColumns> & string), + direction: 'asc' | 'desc' = 'asc' + ): this { if (direction !== 'asc' && direction !== 'desc') throw new ReadSchemaError('Invalid ordering direction'); - const properties = this.source.introspect().properties; - const key = selector( - Object.fromEntries( - Object.keys(properties).map(k => [ - k, - { [COLUMN]: { column: k } } - ]) - ) as any - )[COLUMN].column; - if (!Object.hasOwn(properties, key)) - throw new ReadSchemaError('Unknown polymorphic ordering column'); + const column = + typeof selector === 'string' + ? this.columns[selector] + : selector(this.columns as any); + if (!column || !Object.values(this.columns).includes(column as any)) + throw new ReadSchemaError( + 'Column does not belong to this polymorphic query' + ); + const key = column[COLUMN].column; const copy = this.copy(); copy.orders.push({ key, direction }); return copy; } + /** Order the combined JSON-envelope SQL using trusted SQL and captured bindings. */ + orderByRaw(sql: string, bindings: readonly Knex.RawBinding[] = []): this { + const copy = this.copy(); + copy.orders.push({ raw: captureReadRaw(this.knex, sql, bindings) }); + return copy; + } /** Limit the combined result across all variants. */ limit(count: number): this { if (!Number.isInteger(count) || count < 0) @@ -385,7 +630,7 @@ export class PolymorphicReadQuery< /** Restrict returned discriminator branches and narrow both runtime and inferred schemas. */ selectVariants( keys: K - ): PolymorphicReadQuery> { + ): PolymorphicQueryBuilder> { if ( !keys.length || new Set(keys).size !== keys.length || @@ -421,14 +666,16 @@ export class PolymorphicReadQuery< >( key: K, configure: (query: BranchQueries[K]) => Q - ): PolymorphicReadQuery & Record> { + ): PolymorphicQueryBuilder & Record> { const current = this.branches[key]; if (!current) throw new ReadSchemaError(`Unknown variant: ${key}`); const configured = configure(current as any); - if (!current.sameSource(configured)) + if (!current.sameSource(configured)) { + if (configured instanceof Promise) void configured.catch(() => {}); throw new ReadSchemaError( 'Variant customizer must return its configured read query' ); + } const discriminator = getVariants(this.source)!.discriminatorKey; if ( configured.rowSchema @@ -439,7 +686,7 @@ export class PolymorphicReadQuery< 'Variant projections must retain the original discriminator' ); const copy = this.copy(); - copy.branches[key] = configured as unknown as SchemaReadQuery< + copy.branches[key] = configured as unknown as SchemaQueryBuilder< any, any, any @@ -450,36 +697,71 @@ export class PolymorphicReadQuery< /** @internal Compile one UNION ALL statement; JSON preserves distinct branch shapes. */ compile(correlate?: ReadCorrelation): Knex.QueryBuilder { + const defaults = this.skipDefaults ? undefined : this.defaults; const reserved = Object.values(this.branches).flatMap(branch => Object.keys(branch.rowSchema.introspect().properties) ); - const order = this.orders.map(item => { - const hidden = privateColumn(reserved, 'read_order'); - reserved.push(hidden); - return { ...item, hidden }; - }); + const order = [...(defaults?.orders ?? []), ...this.orders].map( + item => { + if ('raw' in item) return item; + const hidden = privateColumn(reserved, 'read_order'); + reserved.push(hidden); + return { ...item, hidden }; + } + ); const queries = [ ...Object.values(this.branches), ...(this.includeUnknown ? [this.fallback] : []) - ].map(branch => - this.knex + ].map(original => { + let branch = original; + for (const predicates of [ + defaults?.predicates ?? [], + this.predicates + ]) { + if (predicates.length) + branch = branch.withPredicate(query => { + query.where(nested => { + for (const predicate of predicates) + predicate(nested); + }); + }); + } + const softDelete = this.source.introspect().extensions + ?.softDelete as { column: string } | undefined; + if (softDelete && this.deleted !== 'include') { + const key = + buildColumnMap(this.source).colToProp.get( + softDelete.column + ) ?? softDelete.column; + branch = branch.withPredicate(query => { + query[ + this.deleted === 'only' ? 'whereNotNull' : 'whereNull' + ](key); + }); + } + return this.knex .from( branch .compile((sql, alias, source) => { correlate?.(sql, alias, source); const columns = buildColumnMap(source).propToCol; - for (const { key, hidden } of order) + for (const item of order) { + if ('raw' in item) continue; + const { key, hidden } = item; sql.select({ [hidden]: this.knex.raw( 'cast(?? as text)', [`${alias}.${columns.get(key) ?? key}`] ) }); + } }) .as('__read_branch') ) - .select(this.knex.raw('to_jsonb(__read_branch) as __read_poly')) - ); + .select( + this.knex.raw('to_jsonb(__read_branch) as __read_poly') + ); + }); const query = this.knex .from( this.knex @@ -488,7 +770,12 @@ export class PolymorphicReadQuery< .as('__read_variants') ) .select('__read_poly'); - for (const { key, direction, hidden } of order) { + for (const item of order) { + if ('raw' in item) { + query.orderByRaw(item.raw()); + continue; + } + const { key, direction, hidden } = item; const info = this.source.introspect().properties[key].introspect(); const type = info.type === 'number' @@ -509,8 +796,10 @@ export class PolymorphicReadQuery< [hidden] ); } - if (this.rowLimit !== undefined) query.limit(this.rowLimit); - if (this.rowOffset !== undefined) query.offset(this.rowOffset); + const limit = this.rowLimit ?? defaults?.limit; + const offset = this.rowOffset ?? defaults?.offset; + if (limit !== undefined) query.limit(limit); + if (offset !== undefined) query.offset(offset); return query; } /** @internal Decode using exactly the selected branch's schema and codecs. */ @@ -528,6 +817,90 @@ export class PolymorphicReadQuery< toQuery(): string { return this.compile().toQuery(); } + /** Return a separately mutable Knex snapshot of the union statement. */ + toKnexQuery(): Knex.QueryBuilder { + return this.compile(); + } + /** Configure a captured union SELECT and declare its complete raw output shape. */ + apply( + configure: (query: Knex.QueryBuilder) => Knex.QueryBuilder | undefined, + options: QueryOutput + ): OpaqueQuery { + const sql = this.compile(); + const result = configure(sql); + if (result !== undefined && result !== sql) { + if (result instanceof Promise) void result.catch(() => {}); + throw new ReadSchemaError( + 'Raw configuration must synchronously configure the supplied Knex builder' + ); + } + return OpaqueQuery.capture(this.knex, sql, options); + } + /** Select trusted SQL from the union envelope with an explicit output contract. */ + selectRaw( + sql: string, + bindings: readonly Knex.RawBinding[], + options: QueryOutput + ): OpaqueQuery { + const captured = captureReadRaw(this.knex, sql, bindings); + return this.apply( + query => query.clearSelect().select(captured()), + options + ); + } + /** Count matching rows across variants, excluding global pagination and ordering. */ + async countValue(): Promise { + const source = this.compile() + .clearOrder() + .clear('limit') + .clear('offset'); + const row = await this.knex + .from(source.as('__variant_count')) + .count({ count: '*' }) + .first(); + const count = Number(row?.count ?? 0); + if (!Number.isSafeInteger(count)) + throw new ReadSchemaError('Count exceeds the safe integer range'); + return count; + } + /** Globally paginate the discriminated union, leaving the source and its metadata unchanged. */ + async paginate({ + page, + pageSize + }: { + page: number; + pageSize: number; + }): Promise>>> { + if ( + !Number.isInteger(page) || + page < 1 || + !Number.isInteger(pageSize) || + pageSize < 1 + ) + throw new ReadSchemaError( + 'Page and pageSize must be positive integers' + ); + const total = await this.countValue(); + const data = await this.offset((page - 1) * pageSize) + .limit(pageSize) + .execute(); + const totalPages = Math.ceil(total / pageSize); + return { + data, + total, + page, + pageSize, + totalPages, + hasNextPage: page < totalPages, + hasPreviousPage: page > 1 + }; + } + /** Read a property common to the selected variants; returned scalar values are detached. */ + async pluck>>( + key: K + ): Promise>[K][]> { + return (await this.execute()).map(row => (row as any)[key]); + } /** Execute and decode one statement containing every requested branch. */ async execute(): Promise>[]> { return (await this.compile()).map((row: any) => this.decode(row)); diff --git a/libs/knex-schema/src/QuerySource.ts b/libs/knex-schema/src/QuerySource.ts new file mode 100644 index 00000000..ff5788aa --- /dev/null +++ b/libs/knex-schema/src/QuerySource.ts @@ -0,0 +1,1389 @@ +// @cleverbrush/knex-schema — QuerySource + +import type { InferType } from '@cleverbrush/schema'; +import { + EXTRA_TYPE_BRAND, + METHOD_LITERAL_BRAND, + type ObjectSchemaBuilder +} from '@cleverbrush/schema'; +import type { Knex } from 'knex'; +import { buildColumnMap } from './columns.js'; +import type { + AggregateOptions, + AggregateResult, + ExtremumResult, + ExtremumValue, + OutputSchema +} from './expressions.js'; +import { getTableName } from './extension.js'; +import { scalarAggregate } from './operations/aggregate.js'; +import { + type CompositeCursorOptions, + compositeCursor +} from './operations/composite-cursor.js'; +// Operations +import { + avgImpl, + countDistinctImpl, + countImpl, + distinctImpl, + maxImpl, + minImpl, + projectedImpl, + scopedImpl, + selectImpl, + selectRawImpl, + sumImpl, + unscopedImpl +} from './operations/select.js'; +import type { + ColumnRef, + CursorPaginationResult, + InsertType, + JoinManySpec, + JoinOneSpec, + PaginationResult, + SelectProjection, + SelectSelector +} from './types.js'; + +export { + OnConflictBuilder, + type OnConflictMergeHelpers, + type OnConflictMergeOptions, + type OnConflictUpdateData, + type OnConflictUpdateValue +} from './operations/insert.js'; + +import { + deleteImpl, + hardDeleteImpl, + onlyDeletedImpl, + restoreImpl, + withDeletedImpl +} from './operations/delete.js'; +import { + ALLOWED_OPS, + buildQuery, + cleanAndMapRow, + getQuery, + getVariantConfig, + invalidateCache, + registerQuerySource, + resolveColumn +} from './operations/helpers.js'; +import { + bulkInsertImpl, + bulkUpsertImpl, + insertImpl, + insertManyImpl, + onConflictImpl, + upsertImpl +} from './operations/insert.js'; +import { + includeImpl, + includeVariantImpl, + joinManyImpl, + joinOneImpl +} from './operations/join.js'; +import { + executeImpl, + limitImpl, + offsetImpl, + paginateAfterImpl, + paginateImpl +} from './operations/pagination.js'; +import { getState, setState } from './operations/state.js'; +import { bulkUpdateImpl, updateImpl } from './operations/update.js'; +import { + andWhereImpl, + groupByImpl, + groupByRawImpl, + havingImpl, + havingRawImpl, + orderByImpl, + orderByRawImpl, + orWhereImpl, + orWhereInImpl, + orWhereNotInImpl, + orWhereNotNullImpl, + orWhereNullImpl, + whereBetweenImpl, + whereExistsImpl, + whereILikeImpl, + whereImpl, + whereInImpl, + whereJsonPathImpl, + whereLikeImpl, + whereNotBetweenImpl, + whereNotExistsImpl, + whereNotImpl, + whereNotInImpl, + whereNotNullImpl, + whereNullImpl, + whereRawImpl +} from './operations/where.js'; + +// --------------------------------------------------------------------------- +// Type-level helpers +// --------------------------------------------------------------------------- + +type ScopesOf = S extends { + readonly [METHOD_LITERAL_BRAND]?: infer N; +} + ? Extract + : never; + +type ProjectionsOf = S extends { + readonly [EXTRA_TYPE_BRAND]?: infer P; +} + ? P extends Record + ? P + : Record + : Record; + +type ProjectionKeysOf< + S, + K extends keyof ProjectionsOf & string +> = ProjectionsOf[K] extends readonly (infer T extends string)[] + ? T + : string; + +// --------------------------------------------------------------------------- +// QuerySource +// --------------------------------------------------------------------------- + +/** + * Build schema-aware SQL with mapped columns, projections and eager relations. + * Fluent configuration methods mutate this builder; create a fresh query for each + * independent operation. Await the builder or call execute() to obtain mapped rows. + * @internal Private native SQL/write planner; public consumers use immutable query(). + */ +export class QuerySource< + TLocalSchema extends ObjectSchemaBuilder, + TResult +> { + /** + * Create a query over the schema's configured table. + * @param knex - Database connection or transaction used to execute the query. + * @param localSchema - Schema containing property/column and relation metadata. + * @param baseQuery - Optional existing Knex query to configure; it is not cloned. + */ + constructor( + knex: Knex, + localSchema: TLocalSchema, + baseQuery?: Knex.QueryBuilder + ) { + const tableName = getTableName(localSchema); + setState(this, { + knex, + baseQuery: baseQuery ?? knex(tableName), + localSchema, + specs: [], + tableName, + explicitSelects: null, + selectionMode: null, + appliedProjection: null, + projectionColumns: null, + projectionDecoders: {}, + hiddenColumns: new Set(), + includeDeleted: false, + onlyDeleted: false, + skipDefaultScope: false, + variantConfig: undefined, + enabledVariants: null, + variantWhereFilters: [], + variantRelationIncludes: [], + cachedBuiltQuery: null + }); + } + + // ======================================================================= + // SELECT / DISTINCT / AGGREGATES + // ======================================================================= + + /** + * Choose columns, or use an object selector to infer a flat result shape. + * Object values can be schema descriptors or aggregate expressions. Column-list + * selection retains the existing result type; raw SQL cannot infer a new shape. + * @returns This builder, narrowed to the object projection when one is supplied. + */ + select(...columns: (ColumnRef | Knex.Raw)[]): this; + /** + * Choose columns, or use an object selector to infer a flat result shape. + * Object values can be schema descriptors or aggregate expressions. Column-list + * selection retains the existing result type; raw SQL cannot infer a new shape. + * @returns This builder, narrowed to the object projection when one is supplied. + */ + select>( + selector: TSel + ): QuerySource>>; + /** + * Choose columns, or use an object selector to infer a flat result shape. + * Object values can be schema descriptors or aggregate expressions. Column-list + * selection retains the existing result type; raw SQL cannot infer a new shape. + * @returns This builder, narrowed to the object projection when one is supplied. + */ + select(...args: unknown[]): any { + return selectImpl(this as any, ...args); + } + + /** + * Apply SQL DISTINCT to the selected columns, optionally adding columns. + * Property selectors are mapped to database names; SQL decides row equality. + */ + distinct(...columns: (ColumnRef | Knex.Raw)[]): this { + return (distinctImpl as any)(this, ...columns); + } + + /** + * Append a internal SQL COUNT selection without executing the query. + * The driver controls the result shape/value type. Prefer countValue() for a + * checked scalar number, or aggregate.count() in a typed object projection. + */ + count(column?: ColumnRef | Knex.Raw): this { + return (countImpl as any)(this, column); + } + + /** + * Append a internal COUNT(DISTINCT column) selection. + * Prefer countDistinctValue() for a checked scalar or aggregate.countDistinct() + * for an inferred grouped result; this planner method retains the builder type. + */ + countDistinct(column?: ColumnRef | Knex.Raw): this { + return (countDistinctImpl as any)(this, column); + } + + /** + * Append a internal MIN selection without changing the result type. + * Use minValue() for a scalar with explicit decoding, or aggregate.min() in a + * typed projection. SQL returns null for an empty/all-null input. + */ + min(column: ColumnRef | Knex.Raw): this { + return (minImpl as any)(this, column); + } + + /** + * Append a internal MAX selection without changing the result type. + * Use maxValue() for a scalar with explicit decoding, or aggregate.max() in a + * typed projection. SQL returns null for an empty/all-null input. + */ + max(column: ColumnRef | Knex.Raw): this { + return (maxImpl as any)(this, column); + } + + /** + * Append a internal SUM selection, leaving numeric conversion to the driver. + * Prefer sumValue() or aggregate.sum() to preserve exact numeric text by default. + */ + sum(column: ColumnRef | Knex.Raw): this { + return (sumImpl as any)(this, column); + } + + /** + * Append a internal AVG selection, leaving numeric conversion to the driver. + * Prefer avgValue() or aggregate.avg() for a typed, precision-preserving result. + */ + avg(column: ColumnRef | Knex.Raw): this { + return (avgImpl as any)(this, column); + } + + /** + * Count matching rows, or non-null column values, without mutating this query. + * Ignores source ordering/limits/offsets while retaining filters and transactions. + * @param options - Optional output parser; receives the raw driver value. + * @returns A safe integer by default, including zero for an empty source. + * @throws If the default count overflows, the source is grouped/distinct, or parsing fails. + */ + countValue | undefined = undefined>( + options?: AggregateOptions + ): Promise>; + /** + * Count non-null column values without mutating the source query. + * Source paging/order is ignored; filters, scopes and transactions remain. + * @param column - Mapped schema property to count. + * @param options - Optional parser replacing safe-integer decoding. + * @throws If the default result overflows or the source is grouped/distinct. + */ + countValue | undefined = undefined>( + column: ColumnRef, + options?: AggregateOptions + ): Promise>; + /** + * Count matching rows, or non-null column values, without mutating this query. + * Ignores source ordering/limits/offsets while retaining filters and transactions. + * @param options - Optional output parser; receives the raw driver value. + * @returns A safe integer by default, including zero for an empty source. + * @throws If the default count overflows, the source is grouped/distinct, or parsing fails. + */ + countValue( + columnOrOptions?: ColumnRef | AggregateOptions, + options?: AggregateOptions + ): Promise { + const hasColumn = + typeof columnOrOptions === 'string' || + typeof columnOrOptions === 'function'; + return scalarAggregate( + this, + 'count', + hasColumn ? columnOrOptions : undefined, + hasColumn ? options : columnOrOptions + ); + } + + /** + * Count distinct non-null values in an unpaginated clone of this query. + * @param column - Schema property to count; SQL nulls do not contribute. + * @param options - Optional parser replacing default safe-integer conversion. + * @returns A safe integer, or the parser's inferred output type. + * @throws If the count is unsafe, the source is grouped/distinct, or parsing fails. + */ + countDistinctValue | undefined = undefined>( + column: ColumnRef, + options?: AggregateOptions + ): Promise> { + return scalarAggregate(this, 'countDistinct', column, options); + } + + /** + * Sum non-null values in an unpaginated clone, preserving numeric precision. + * @param column - Numeric schema property to sum. + * @param options - Optional parser receiving the raw driver value, including null. + * @returns Database numeric text, or null for empty/all-null input by default. + * An output parser replaces default decoding and controls the result type. + */ + sumValue | undefined = undefined>( + column: ColumnRef, + options?: AggregateOptions + ): Promise> { + return scalarAggregate(this, 'sum', column, options); + } + + /** + * Average non-null values in an unpaginated clone of this query. + * @param column - Numeric schema property to aggregate. + * @param options - Optional parser receiving the raw driver result, including null. + * @returns Exact database numeric text or null by default; a parser overrides this. + * @remarks Text preserves database precision, not precision already lost in floating-point storage. + */ + avgValue | undefined = undefined>( + column: ColumnRef, + options?: AggregateOptions + ): Promise> { + return scalarAggregate(this, 'avg', column, options); + } + + /** + * Find the smallest non-null column value without retaining source paging. + * @param column - Schema property to aggregate. + * @param options - Optional parser replacing default decoding, including null handling. + * @returns Null for empty/all-null input; otherwise the column representation. + * Dates return Date; numeric SQL overrides may return exact strings. + */ + minValue< + C extends ColumnRef, + S extends OutputSchema | undefined = undefined + >( + column: C, + options?: AggregateOptions + ): Promise< + AggregateResult< + S, + C extends (...args: any[]) => infer D + ? ExtremumValue + : C extends keyof InferType + ? ExtremumResult[C]> + : unknown + > + > { + return scalarAggregate(this, 'min', column, options); + } + + /** + * Find the largest non-null column value without retaining source paging. + * @param column - Schema property to aggregate. + * @param options - Optional parser replacing default decoding, including null handling. + * @returns Null for empty/all-null input; otherwise the column representation. + * Dates return Date; numeric SQL overrides may return exact strings. + */ + maxValue< + C extends ColumnRef, + S extends OutputSchema | undefined = undefined + >( + column: C, + options?: AggregateOptions + ): Promise< + AggregateResult< + S, + C extends (...args: any[]) => infer D + ? ExtremumValue + : C extends keyof InferType + ? ExtremumResult[C]> + : unknown + > + > { + return scalarAggregate(this, 'max', column, options); + } + + /** + * Append a raw SELECT expression with optional Knex value/identifier bindings. + * The caller owns its SQL and result shape; this does not infer a new result type. + */ + selectRaw(sql: string, bindings?: any[]): this { + return selectRawImpl(this as any, sql, bindings); + } + + /** + * Apply a named schema projection and narrow the selected property type. + * @param name - Projection registered with the schema's projection() extension. + * @throws If the projection is unknown or conflicts with a prior selection. + */ + projected & string>( + name: K + ): QuerySource< + TLocalSchema, + Pick & keyof TResult> + > { + return projectedImpl(this as any, name); + } + + /** + * Apply a named schema scope to this query. + * @param name - Scope registered with the schema's scope() extension. + * @throws If the requested scope is not registered. + */ + scoped>(name: K): this { + return scopedImpl(this as any, name as string); + } + + /** + * Disable the default read scope and include soft-deleted rows. + * Explicit filters already added to this builder remain in place. + */ + unscoped(): this { + return unscopedImpl(this as any); + } + + // ======================================================================= + // WHERE + // ======================================================================= + + /** + * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. + * Selector/key and record forms map property names to database columns. Grouped + * callbacks receive a Knex builder and therefore use database column names. + * Values are bound; use the operator form for comparisons other than equality. + */ + where(column: ColumnRef, operator: string, value: any): this; + /** + * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. + * Selector/key and record forms map property names to database columns. Grouped + * callbacks receive a Knex builder and therefore use database column names. + * Values are bound; use the operator form for comparisons other than equality. + */ + where(column: ColumnRef, value: any): this; + /** + * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. + * Selector/key and record forms map property names to database columns. Grouped + * callbacks receive a Knex builder and therefore use database column names. + * Values are bound; use the operator form for comparisons other than equality. + */ + where(raw: Knex.Raw, operator: string, value: any): this; + /** + * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. + * Selector/key and record forms map property names to database columns. Grouped + * callbacks receive a Knex builder and therefore use database column names. + * Values are bound; use the operator form for comparisons other than equality. + */ + where(callback: (builder: Knex.QueryBuilder) => void): this; + /** + * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. + * Selector/key and record forms map property names to database columns. Grouped + * callbacks receive a Knex builder and therefore use database column names. + * Values are bound; use the operator form for comparisons other than equality. + */ + where(record: Record): this; + /** + * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. + * Selector/key and record forms map property names to database columns. Grouped + * callbacks receive a Knex builder and therefore use database column names. + * Values are bound; use the operator form for comparisons other than equality. + */ + where(raw: Knex.Raw): this; + /** + * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. + * Selector/key and record forms map property names to database columns. Grouped + * callbacks receive a Knex builder and therefore use database column names. + * Values are bound; use the operator form for comparisons other than equality. + */ + where(columnOrRaw: any, ...args: any[]): this { + return whereImpl(this as any, columnOrRaw, ...args); + } + + /** + * Add an AND condition; an explicit synonym for where(). + * Property references and record keys are mapped; raw callbacks use Knex columns. + */ + andWhere( + column: ColumnRef, + operator: string, + value: any + ): this; + /** + * Add an AND condition; an explicit synonym for where(). + * Property references and record keys are mapped; raw callbacks use Knex columns. + */ + andWhere(column: ColumnRef, value: any): this; + /** + * Add an AND condition; an explicit synonym for where(). + * Property references and record keys are mapped; raw callbacks use Knex columns. + */ + andWhere(record: Record): this; + /** + * Add an AND condition; an explicit synonym for where(). + * Property references and record keys are mapped; raw callbacks use Knex columns. + */ + andWhere(callback: (builder: Knex.QueryBuilder) => void): this; + /** + * Add an AND condition; an explicit synonym for where(). + * Property references and record keys are mapped; raw callbacks use Knex columns. + */ + andWhere(raw: Knex.Raw): this; + /** + * Add an AND condition; an explicit synonym for where(). + * Property references and record keys are mapped; raw callbacks use Knex columns. + */ + andWhere(columnOrRaw: any, ...args: any[]): this { + return andWhereImpl(this as any, columnOrRaw, ...args); + } + + /** + * Add an OR condition using a mapped property, record, raw SQL or Knex group. + * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. + */ + orWhere( + column: ColumnRef, + operator: string, + value: any + ): this; + /** + * Add an OR condition using a mapped property, record, raw SQL or Knex group. + * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. + */ + orWhere(column: ColumnRef, value: any): this; + /** + * Add an OR condition using a mapped property, record, raw SQL or Knex group. + * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. + */ + orWhere(record: Record): this; + /** + * Add an OR condition using a mapped property, record, raw SQL or Knex group. + * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. + */ + orWhere(callback: (builder: Knex.QueryBuilder) => void): this; + /** + * Add an OR condition using a mapped property, record, raw SQL or Knex group. + * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. + */ + orWhere(raw: Knex.Raw): this; + /** + * Add an OR condition using a mapped property, record, raw SQL or Knex group. + * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. + */ + orWhere(columnOrRaw: any, ...args: any[]): this { + return orWhereImpl(this as any, columnOrRaw, ...args); + } + + /** + * Add a negated condition using a mapped property, record or Knex group. + * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. + */ + whereNot( + column: ColumnRef, + operator: string, + value: any + ): this; + /** + * Add a negated condition using a mapped property, record or Knex group. + * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. + */ + whereNot(column: ColumnRef, value: any): this; + /** + * Add a negated condition using a mapped property, record or Knex group. + * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. + */ + whereNot(record: Record): this; + /** + * Add a negated condition using a mapped property, record or Knex group. + * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. + */ + whereNot(callback: (builder: Knex.QueryBuilder) => void): this; + /** + * Add a negated condition using a mapped property, record or Knex group. + * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. + */ + whereNot(raw: Knex.Raw): this; + /** + * Add a negated condition using a mapped property, record or Knex group. + * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. + */ + whereNot(columnOrRaw: any, ...args: any[]): this { + return whereNotImpl(this as any, columnOrRaw, ...args); + } + + /** + * Require the mapped column to match a value list or a single-column subquery. + * An empty list matches no rows. Subqueries use Knex's database column names. + */ + whereIn( + column: ColumnRef, + values: readonly any[] | Knex.QueryBuilder + ): this { + return (whereInImpl as any)(this, column, values); + } + + /** + * Exclude values returned by a list or single-column subquery. + * SQL null semantics apply; a null in the set is not equivalent to a missing value. + */ + whereNotIn( + column: ColumnRef, + values: readonly any[] | Knex.QueryBuilder + ): this { + return (whereNotInImpl as any)(this, column, values); + } + + /** + * Add an OR membership condition against a list or single-column subquery. + */ + orWhereIn( + column: ColumnRef, + values: readonly any[] | Knex.QueryBuilder + ): this { + return (orWhereInImpl as any)(this, column, values); + } + + /** + * Add an OR non-membership condition; SQL NOT IN null semantics apply. + */ + orWhereNotIn( + column: ColumnRef, + values: readonly any[] | Knex.QueryBuilder + ): this { + return (orWhereNotInImpl as any)(this, column, values); + } + + /** + * Add an AND IS NULL condition for a mapped schema property. + */ + whereNull(column: ColumnRef): this { + return (whereNullImpl as any)(this, column); + } + + /** + * Add an AND IS NOT NULL condition for a mapped schema property. + */ + whereNotNull(column: ColumnRef): this { + return (whereNotNullImpl as any)(this, column); + } + + /** + * Add an OR IS NULL condition for a mapped schema property. + */ + orWhereNull(column: ColumnRef): this { + return (orWhereNullImpl as any)(this, column); + } + + /** + * Add an OR IS NOT NULL condition for a mapped schema property. + */ + orWhereNotNull(column: ColumnRef): this { + return (orWhereNotNullImpl as any)(this, column); + } + + /** + * Require the mapped column to lie within an inclusive [lower, upper] range. + */ + whereBetween( + column: ColumnRef, + range: readonly [any, any] + ): this { + return (whereBetweenImpl as any)(this, column, range); + } + + /** + * Exclude the inclusive [lower, upper] range from a mapped column. + */ + whereNotBetween( + column: ColumnRef, + range: readonly [any, any] + ): this { + return (whereNotBetweenImpl as any)(this, column, range); + } + + /** + * Match a mapped column against a SQL LIKE pattern. + * Percent and underscore remain wildcards; values are bound, not wildcard-escaped. + */ + whereLike(column: ColumnRef, value: string): this { + return (whereLikeImpl as any)(this, column, value); + } + + /** + * Match a mapped column using PostgreSQL's case-insensitive ILIKE operator. + * Percent and underscore remain pattern wildcards. + */ + whereILike(column: ColumnRef, value: string): this { + return (whereILikeImpl as any)(this, column, value); + } + + /** + * Append raw WHERE SQL with Knex bindings. Never interpolate untrusted values. + * Raw SQL uses database names and is outside schema-level result/type checking. + */ + whereRaw(sql: string, ...bindings: any[]): this { + return (whereRawImpl as any)(this, sql, ...bindings); + } + + /** + * Add an EXISTS filter from a Knex subquery or query-building callback. + * Use qualified database columns to correlate it with the parent query. + */ + whereExists(callback: Knex.QueryCallback | Knex.QueryBuilder): this { + return (whereExistsImpl as any)(this, callback); + } + + /** + * Add a NOT EXISTS filter from a Knex subquery or query-building callback. + */ + whereNotExists(callback: Knex.QueryCallback | Knex.QueryBuilder): this { + return (whereNotExistsImpl as any)(this, callback); + } + + /** + * Compare a JSON-path value inside a mapped JSON column. + * @param path - JSON path understood by the Knex database dialect. + * @param operator - SQL comparison operator forwarded to Knex. + * @param value - Bound comparison value. + */ + whereJsonPath( + column: ColumnRef, + path: string, + operator?: string, + value?: any + ): this { + return (whereJsonPathImpl as any)(this, column, path, operator, value); + } + + // ======================================================================= + // ORDER BY + // ======================================================================= + + /** + * Append ordering by a mapped property or raw expression (ascending by default). + * Add a unique tie-breaker for stable pages. Eager loading retains parent order. + */ + orderBy( + column: ColumnRef | Knex.Raw, + direction?: 'asc' | 'desc' + ): this { + return (orderByImpl as any)(this, column, direction); + } + + /** + * Append a raw ORDER BY expression with Knex bindings. + * Use database column names; parent ordering is retained during eager loading. + */ + orderByRaw(sql: string, ...bindings: any[]): this { + return (orderByRawImpl as any)(this, sql, ...bindings); + } + + // ======================================================================= + // GROUP BY / HAVING + // ======================================================================= + + /** + * Group rows by mapped schema properties or raw expressions. + * Combine with aggregate expressions in select() to infer grouped DTO results. + */ + groupBy(...columns: (ColumnRef | Knex.Raw)[]): this { + return (groupByImpl as any)(this, ...columns); + } + + /** + * Append raw GROUP BY SQL with optional Knex bindings. + */ + groupByRaw(sql: string, ...bindings: any[]): this { + return (groupByRawImpl as any)(this, sql, ...bindings); + } + + /** + * Filter SQL groups by a mapped column/raw expression, operator and bound value. + */ + having( + column: ColumnRef | Knex.Raw, + operator: string, + value: any + ): this { + return (havingImpl as any)(this, column, operator, value); + } + + /** + * Append raw HAVING SQL with Knex bindings, for example aggregate comparisons. + */ + havingRaw(sql: string, ...bindings: any[]): this { + return havingRawImpl(this as any, sql, ...bindings); + } + + // ======================================================================= + // PAGINATION + // ======================================================================= + + /** + * Set the maximum number of parent rows to select; mutates this query. + * Included collections do not consume the parent limit. + */ + limit(n: number): this { + return limitImpl(this as any, n); + } + + /** + * Skip this many parent rows before applying the limit; mutates this query. + * Use deterministic ordering when navigating offset-based pages. + */ + offset(n: number): this { + return offsetImpl(this as any, n); + } + + /** + * Execute a one-based offset page and a separate matching-source count query. + * Mutates this builder's limit/offset and returns mapped rows plus page metadata. + * The count and page are separate reads, not a snapshot unless your transaction provides one. + */ + async paginate(opts: { + /** One-based page number. */ + page: number; + /** Maximum parent rows in a page. */ + pageSize: number; + }): Promise> { + return paginateImpl(this as any, opts) as Promise< + PaginationResult + >; + } + + /** + * Read a cursor page without running a total-count query. + * The orderBy form clones the source and preserves exact composite sort values; + * its non-null sort must contain a declared unique key. The single-column form + * mutates this builder and defaults to id descending. Reapply access filters on + * every request: cursors are positions, not authorization or snapshots. + * @returns Mapped data, hasMore, and a nextCursor that is null on the last page. + * @throws For invalid composite cursors or unsupported composite query shapes. + */ + paginateAfter( + opts: CompositeCursorOptions + ): Promise>; + /** + * Read a cursor page without running a total-count query. + * The orderBy form clones the source and preserves exact composite sort values; + * its non-null sort must contain a declared unique key. The single-column form + * mutates this builder and defaults to id descending. Reapply access filters on + * every request: cursors are positions, not authorization or snapshots. + * @returns Mapped data, hasMore, and a nextCursor that is null on the last page. + * @throws For invalid composite cursors or unsupported composite query shapes. + */ + paginateAfter(opts: { + /** Previous raw single-column position; omit for the first page. */ + cursor?: any; + /** Maximum parent rows to return; one extra row determines hasMore. */ + limit: number; + /** Unique sort property; defaults to id for single-column paging. */ + column?: ColumnRef; + /** Sort/continuation direction; defaults to descending. */ + direction?: 'asc' | 'desc'; + }): Promise>; + /** + * Read a cursor page without running a total-count query. + * The orderBy form clones the source and preserves exact composite sort values; + * its non-null sort must contain a declared unique key. The single-column form + * mutates this builder and defaults to id descending. Reapply access filters on + * every request: cursors are positions, not authorization or snapshots. + * @returns Mapped data, hasMore, and a nextCursor that is null on the last page. + * @throws For invalid composite cursors or unsupported composite query shapes. + */ + async paginateAfter(opts: any): Promise> { + if ('orderBy' in opts) return compositeCursor(this as any, opts); + return (paginateAfterImpl as any)(this, opts) as Promise< + CursorPaginationResult + >; + } + + // ======================================================================= + // WRITE OPERATIONS + // ======================================================================= + + /** + * Insert one schema-shaped row and return its mapped database representation. + * Applies configured insert hooks, column mappings and timestamp defaults. + */ + async insert(data: InsertType): Promise { + return insertImpl(this as any, data) as Promise; + } + + /** + * Insert an array of schema-shaped rows and return their mapped representations. + * Returns an empty array for empty input; use bulkInsert() to control chunking. + */ + async insertMany(data: InsertType[]): Promise { + return insertManyImpl(this as any, data) as Promise; + } + + /** + * Configure an upsert conflict target using mapped properties. + * Call merge() or ignore() on the returned builder to insert the row. + */ + onConflict( + ...conflictColumns: ColumnRef[] + ): import('./operations/insert.js').OnConflictBuilder< + TLocalSchema, + TResult + > { + return (onConflictImpl as any)(this, ...conflictColumns); + } + + /** + * Insert one row or update it when the chosen conflict target already exists. + * @param opts - Conflict properties and optional subset of properties to update. + * @returns The inserted or updated row mapped to schema property names. + */ + async upsert( + data: InsertType, + opts: { + /** Properties identifying an existing row on conflict. */ + conflictColumns: ColumnRef[]; + /** Properties to update on conflict; omit to merge insert values. */ + updateColumns?: ColumnRef[]; + } + ): Promise { + return (upsertImpl as any)(this, data, opts); + } + + /** + * Insert rows in chunks, optionally ignoring or merging conflicts. + * @param opts - Chunk size (default 500), conflict policy and conflict properties. + * @returns Mapped rows returned by PostgreSQL; ignored conflicts produce no row. + */ + async bulkInsert( + rows: InsertType[], + opts?: { + /** Requested rows per statement, capped by parameter limits; default 500. */ + chunkSize?: number; + /** Optional PostgreSQL conflict policy applied to each chunk. */ + onConflict?: 'ignore' | 'merge'; + /** Conflict target properties when a conflict policy is supplied. */ + conflictColumns?: ColumnRef[]; + } + ): Promise { + return (bulkInsertImpl as any)(this, rows, opts); + } + + /** + * Insert/update rows in chunks using the specified conflict properties. + * @param opts - Required conflict target and optional chunk size (default 500). + * @returns The database rows produced by each chunk, mapped to schema properties. + */ + async bulkUpsert( + rows: InsertType[], + opts: { + /** Properties identifying an existing row on conflict. */ + conflictColumns: ColumnRef[]; + /** Requested rows per statement, capped by parameter limits; default 500. */ + chunkSize?: number; + } + ): Promise { + return (bulkUpsertImpl as any)(this, rows, opts); + } + + // ======================================================================= + // UPDATE + // ======================================================================= + + /** + * Update rows matching this query's explicit filters and return mapped rows. + * Applies update hooks and timestamp metadata. Add a WHERE clause to avoid a + * table-wide update; this method does not track entity identity. + */ + async update(data: Partial>): Promise { + return updateImpl(this as any, data) as Promise; + } + + /** + * Apply per-row where/set pairs and return the total affected-row count. + * Maps both filter and update property names and applies configured update hooks. + */ + async bulkUpdate( + updates: ReadonlyArray<{ + /** Equality filters identifying the rows for this update. */ + where: Partial>; + /** Schema properties to assign to those rows. */ + set: Partial>; + }> + ): Promise { + return bulkUpdateImpl(this as any, updates as any); + } + + // ======================================================================= + // DELETE / SOFT DELETE + // ======================================================================= + + /** + * Delete rows matching explicit filters and return the affected-row count. + * Runs beforeDelete hooks; with soft-delete metadata it sets the deletion timestamp + * instead of removing rows. Add filters to avoid a table-wide write. + */ + async delete(): Promise { + return deleteImpl(this as any); + } + + /** + * Include soft-deleted rows in read results without removing explicit filters. + */ + withDeleted(): this { + return withDeletedImpl(this as any); + } + + /** + * Restrict reads to rows whose configured soft-delete column is non-null. + */ + onlyDeleted(): this { + return onlyDeletedImpl(this as any); + } + + /** + * Permanently delete rows matching explicit filters, even on a soft-delete schema. + * Runs beforeDelete hooks and returns the affected count. This cannot be undone + * without a transaction rollback or backup. + */ + async hardDelete(): Promise { + return hardDeleteImpl(this as any); + } + + /** + * Clear the deletion timestamp on rows matching explicit filters and return them. + * @throws If the schema has no soft-delete configuration. + */ + async restore(): Promise { + return restoreImpl(this as any) as Promise; + } + + // ======================================================================= + // EAGER LOADING (JOIN) + // ======================================================================= + + /** + * Eager-load a related object under spec.as using mapped join columns. + * Required joins remove unmatched parents; optional joins return null. Unlike a + * flat join, related fields remain nested and are mapped with the foreign schema. + */ + joinOne< + TForeignSchema extends ObjectSchemaBuilder< + any, + any, + any, + any, + any, + any, + any + >, + TFieldName extends string, + TRequired extends boolean = true + >( + spec: JoinOneSpec + ): QuerySource< + TLocalSchema, + import('./types.js').WithJoinedOne< + TResult, + TFieldName, + TForeignSchema, + TRequired + > + > { + return joinOneImpl(this as any, spec); + } + + /** + * Eager-load a nested array without multiplying parent rows. + * The relation's own order/limit/offset controls children separately from parent paging. + */ + joinMany< + TForeignSchema extends ObjectSchemaBuilder< + any, + any, + any, + any, + any, + any, + any + >, + TFieldName extends string + >( + spec: JoinManySpec + ): QuerySource< + TLocalSchema, + import('./types.js').WithJoinedMany + > { + return joinManyImpl(this as any, spec); + } + + /** + * Eager-load a relation registered on the schema by name. + * @param customize - Optional callback configuring the related query. + * Use an ORM DbSet when you need typed relation names and customization fields. + */ + include( + relationName: string, + customize?: (q: QuerySource) => void + ): this { + return includeImpl(this as any, relationName, customize); + } + + /** + * Eager-load a relation declared for one polymorphic discriminator value. + * Other variants are not populated with this relation. Use ORM entity declarations + * to retain known relation customization types. + */ + includeVariant( + variantKey: string, + relationName: string, + customize?: (q: QuerySource) => void + ): this { + return includeVariantImpl( + this as any, + variantKey, + relationName, + customize + ); + } + + // ======================================================================= + // POLYMORPHIC VARIANTS + // ======================================================================= + + /** + * Add a filter applying only to the named polymorphic branch. + * Other discriminator values remain eligible. Maps the variant property to its + * CTI/STI storage column; throws for unknown variants or unsupported operators. + */ + whereVariant( + key: string, + column: string, + operator: string, + value: any + ): this { + const state = getState(this); + const variantConfig = getVariantConfig(this); + if (!variantConfig) { + throw new Error( + 'whereVariant() can only be used on a polymorphic schema (created with .withVariants())' + ); + } + + const spec = variantConfig.variants[key]; + if (!spec) { + throw new Error( + `whereVariant: unknown variant key "${key}". ` + + `Valid keys: ${Object.keys(variantConfig.variants).join(', ')}` + ); + } + + const op = operator.toLowerCase(); + if (!ALLOWED_OPS.has(op)) { + throw new Error( + `whereVariant: operator "${operator}" is not allowed. ` + + `Allowed operators: ${[...ALLOWED_OPS].join(', ')}` + ); + } + + const { propToCol } = buildColumnMap(spec.schema); + const colName = propToCol.get(column) ?? column; + + let qualifiedColumn: string; + if (spec.storage === 'cti') { + qualifiedColumn = `__v_${key}.${colName}`; + } else { + qualifiedColumn = `${state.tableName}.${colName}`; + } + + state.variantWhereFilters.push({ key, qualifiedColumn, op, value }); + invalidateCache(this); + return this; + } + + /** + * Choose which polymorphic variant bodies are loaded. + * This controls variant joins/selection, not a discriminator filter on base rows. + * @throws If the schema is not polymorphic. + */ + selectVariants(keys: string[]): this { + const state = getState(this); + if (!getVariantConfig(this)) { + throw new Error( + 'selectVariants() can only be used on a polymorphic schema (created with .withVariants())' + ); + } + state.enabledVariants = new Set(keys); + invalidateCache(this); + return this; + } + + // ======================================================================= + // ESCAPE HATCH + // ======================================================================= + + /** + * Configure the underlying mutable Knex query as an escape hatch. + * Raw changes do not infer a new result type; the caller owns column names, + * result shape and cardinality introduced by the callback. + */ + apply(fn: (builder: Knex.QueryBuilder) => void): this { + const state = getState(this); + state.opaqueReadShape = true; + invalidateCache(this); + fn(state.baseQuery); + return this; + } + + // ======================================================================= + // TRANSACTION + // ======================================================================= + + /** + * Clone this query and its eager-relation queries onto an existing transaction. + * The source builder is unchanged; transaction commit/rollback stays with the caller. + */ + transacting(trx: Knex.Transaction): QuerySource { + const state = getState(this); + const builder = new QuerySource( + trx as unknown as Knex, + state.localSchema as TLocalSchema, + state.baseQuery.clone().transacting(trx) + ); + const builderState = getState(builder); + for (const spec of state.specs) { + builderState.specs.push({ + ...spec, + foreignQuery: spec.foreignQuery.clone().transacting(trx) + }); + } + builderState.explicitSelects = state.explicitSelects + ? [...state.explicitSelects] + : null; + builderState.selectionMode = state.selectionMode; + builderState.appliedProjection = state.appliedProjection; + builderState.projectionColumns = state.projectionColumns + ? { ...state.projectionColumns } + : null; + builderState.projectionDecoders = { ...state.projectionDecoders }; + builderState.hiddenColumns = new Set(state.hiddenColumns); + builderState.includeDeleted = state.includeDeleted; + builderState.onlyDeleted = state.onlyDeleted; + builderState.skipDefaultScope = state.skipDefaultScope; + builderState.opaqueReadShape = state.opaqueReadShape; + builderState.variantConfig = state.variantConfig; + builderState.enabledVariants = + state.enabledVariants !== null + ? new Set(state.enabledVariants) + : null; + builderState.variantWhereFilters = [...state.variantWhereFilters]; + builderState.variantRelationIncludes = [ + ...state.variantRelationIncludes + ]; + return builder; + } + + // ======================================================================= + // EXECUTION + // ======================================================================= + + /** + * Render SQL for debugging without executing it. + * Bindings may appear as literal values; avoid logging sensitive inputs. + */ + toQuery(): string { + return getQuery(this).toQuery(); + } + + /** @internal ORM tracking must never attach aggregate or DTO rows as entities. */ + get returnsEntityRows(): boolean { + return getState(this).selectionMode === null; + } + + /** + * Expose the built Knex query without executing Framework row mapping. + * Direct execution returns raw database rows, potentially including internal fields. + * Treat mutations as an escape hatch rather than typed Framework configuration. + */ + toKnexQuery(): Knex.QueryBuilder { + return getQuery(this); + } + + /** + * Render the query as a debugging SQL string; equivalent to toQuery(). + */ + toString(): string { + return getQuery(this).toString(); + } + + /** + * Execute SQL and map all rows, including eager relations and aggregate decoders. + * @returns An empty array when no rows match. + * @throws Database errors and output-parser validation failures. + */ + async execute(): Promise { + return executeImpl(this) as Promise; + } + + /** + * Execute this query for its first mapped row, or undefined if none matches. + * Set ordering when the choice of first row matters. + */ + async first(): Promise { + const query = getQuery(this).first(); + const row = await query; + + if (!row) return undefined; + return cleanAndMapRow(this, row) as TResult; + } + + /** + * Execute a selection and collect one mapped column's values into an array. + * @param column - Schema property whose database column should be read. + */ + async pluck( + column: ColumnRef + ): Promise { + const _state = getState(this); + const col = resolveColumn(this, column, 'pluck') as string; + const rows = await buildQuery(this).select(col); + return rows.map( + (row: any) => row[col] ?? row[column as string] + ) as TResult[K][]; + } + + /** + * Promise-compatible execution hook enabling await query(...). + * Runs execute() and forwards fulfillment/rejection; repeated awaits may execute again. + */ + // biome-ignore lint/suspicious/noThenProperty: intentional thenable + then( + onfulfilled?: + | ((value: TResult[]) => TReturn1 | PromiseLike) + | null, + onrejected?: ((reason: any) => TReturn2 | PromiseLike) | null + ): Promise { + return this.execute().then(onfulfilled, onrejected); + } +} + +// Register the constructor for circular-dependency-safe access +registerQuerySource(QuerySource); diff --git a/libs/knex-schema/src/SchemaQueryBuilder.ts b/libs/knex-schema/src/SchemaQueryBuilder.ts index 104a8724..197c057a 100644 --- a/libs/knex-schema/src/SchemaQueryBuilder.ts +++ b/libs/knex-schema/src/SchemaQueryBuilder.ts @@ -1,47 +1,67 @@ -// @cleverbrush/knex-schema — SchemaQueryBuilder - -import type { InferType } from '@cleverbrush/schema'; import { + type ArraySchemaBuilder, + array, EXTRA_TYPE_BRAND, - METHOD_LITERAL_BRAND, - type ObjectSchemaBuilder + type InferType, + type ObjectSchemaBuilder, + object, + SYMBOL_SCHEMA_PROPERTY_DESCRIPTOR } from '@cleverbrush/schema'; import type { Knex } from 'knex'; import { - AliasedQueryBuilder, - type AliasTables, - isTableAlias, - type TableAlias -} from './aliased-query.js'; -import { buildColumnMap } from './columns.js'; -import type { - AggregateOptions, - AggregateResult, - ExtremumResult, - ExtremumValue, - OutputSchema + buildColumnMap, + getPrimaryKeyColumns, + resolvePropertyKey +} from './columns.js'; +import type { RelationInfo, SchemaProps } from './entity.js'; +import { + type AggregateExpression, + type AggregateKind, + type AggregateOptions, + type AggregateResult, + type AliasedColumn, + COLUMN, + compileAggregate, + createAggregate, + isAggregate, + type OutputSchema } from './expressions.js'; -import { getTableName, POLYMORPHIC_TYPE_BRAND } from './extension.js'; -import { scalarAggregate } from './operations/aggregate.js'; +import { getProjections, getTableName, getVariants } from './extension.js'; +import { OpaqueQuery, type QueryOutput } from './OpaqueQuery.js'; import { type CompositeCursorOptions, compositeCursor } from './operations/composite-cursor.js'; -// Operations +import type { ScopesOf } from './operations/helpers.js'; +import { getQuerySourceCtor } from './operations/helpers.js'; +import { getState } from './operations/state.js'; +import { PolymorphicQueryBuilder } from './PolymorphicQueryBuilder.js'; +import { QuerySource } from './QuerySource.js'; +import type { ReadRelations, ReadVariantMetadata } from './read-entity.js'; +import { + captureReadRaw, + captureValue, + type ReadPredicate, + type ReadPredicateContext, + type ReadPredicateSelector, + ReadPredicates +} from './read-predicates.js'; +import { compileReadProjection, type ReadField } from './read-projection.js'; import { - avgImpl, - countDistinctImpl, - countImpl, - distinctImpl, - maxImpl, - minImpl, - projectedImpl, - scopedImpl, - selectImpl, - selectRawImpl, - sumImpl, - unscopedImpl -} from './operations/select.js'; + type ColumnReadSchema, + compileReadSchema, + decodeObject, + type ObjectReadSchema, + type ReadNode, + type ReadObject, + type ReadSchema, + ReadSchemaError, + readExpression, + type SchemaForValue +} from './read-schema.js'; + +export { type BoundQuery, createQuery, query } from './query.js'; + import type { ColumnRef, CursorPaginationResult, @@ -49,264 +69,429 @@ import type { JoinManySpec, JoinOneSpec, PaginationResult, - SelectProjection, - SelectSelector + RelationSpec } from './types.js'; -export { - OnConflictBuilder, - type OnConflictMergeHelpers, - type OnConflictMergeOptions, - type OnConflictUpdateData, - type OnConflictUpdateValue -} from './operations/insert.js'; - -import { - deleteImpl, - hardDeleteImpl, - onlyDeletedImpl, - restoreImpl, - withDeletedImpl -} from './operations/delete.js'; -import { - ALLOWED_OPS, - buildQuery, - cleanAndMapRow, - getQuery, - getVariantConfig, - invalidateCache, - registerSchemaQueryBuilder, - resolveColumn -} from './operations/helpers.js'; -import { - bulkInsertImpl, - bulkUpsertImpl, - insertImpl, - insertManyImpl, - onConflictImpl, - upsertImpl -} from './operations/insert.js'; -import { - includeImpl, - includeVariantImpl, - joinManyImpl, - joinOneImpl -} from './operations/join.js'; -import { - executeImpl, - limitImpl, - offsetImpl, - paginateAfterImpl, - paginateImpl -} from './operations/pagination.js'; -import { getState, setState } from './operations/state.js'; -import { bulkUpdateImpl, updateImpl } from './operations/update.js'; -import { - andWhereImpl, - groupByImpl, - groupByRawImpl, - havingImpl, - havingRawImpl, - orderByImpl, - orderByRawImpl, - orWhereImpl, - orWhereInImpl, - orWhereNotInImpl, - orWhereNotNullImpl, - orWhereNullImpl, - whereBetweenImpl, - whereExistsImpl, - whereILikeImpl, - whereImpl, - whereInImpl, - whereJsonPathImpl, - whereLikeImpl, - whereNotBetweenImpl, - whereNotExistsImpl, - whereNotImpl, - whereNotInImpl, - whereNotNullImpl, - whereNullImpl, - whereRawImpl -} from './operations/where.js'; -import { type SchemaAwareQuery, schemaReadQuery } from './SchemaReadQuery.js'; - -// --------------------------------------------------------------------------- -// Type-level helpers -// --------------------------------------------------------------------------- - -type ScopesOf = S extends { - readonly [METHOD_LITERAL_BRAND]?: infer N; +const READ_COLUMN = Symbol('schema-read-column'); +const JSON_PATH = Symbol('schema-json-path'); +/** @internal Nominal read-query identity used without recursively comparing fluent APIs. */ +export const READ_QUERY = Symbol('schema-read-query'); +/** Minimal type surface for a customizer's strongly inferred result schema. */ +export interface ReadQueryShape { + /** @internal Nominal marker; constructing a plain schema object is not a query. */ + readonly [READ_QUERY]: true; + /** Decoded result schema inferred from the returned query. */ + readonly rowSchema: R; } - ? Extract - : never; - -type ProjectionsOf = S extends { - readonly [EXTRA_TYPE_BRAND]?: infer P; +let readAliasSequence = 0; +/** A typed SQL column; selectors receive descriptions, not row values. */ +export interface ReadColumn + extends AliasedColumn> { + /** @internal Decoding and projection metadata shared by the query compiler. */ + readonly [READ_COLUMN]: ReadNode; + /** @internal Captured JSON path beneath a storage column. */ + readonly [JSON_PATH]?: readonly string[]; + /** Type-only property identity for column-list projections. */ + readonly __property?: K; } - ? P extends Record - ? P - : Record - : Record; - -type ProjectionKeysOf< +/** A JSON object column also exposes its known nested properties. */ +export type NestedReadColumn = ReadColumn< + ColumnReadSchema, + Key +> & + (S extends ReadObject + ? { + [K in keyof SchemaProps & string]: NestedReadColumn< + SchemaProps[K], + `${Key}.${K}` + >; + } + : {}); +/** Columns available for typed projections and filters. */ +export type ReadColumns< + S extends ReadObject, + Relations extends PropertyKey = never +> = { + [K in Exclude, Relations> & string]: NestedReadColumn< + SchemaProps[K], + K + >; +}; +type SelectedValue = + | (Selector extends (...args: any[]) => AliasedColumn + ? Value + : Selector extends keyof Columns + ? Columns[Selector] extends AliasedColumn + ? Value + : never + : never) + | null; +type Selection = Record | AggregateExpression>; +type MergeProps = { + [K in keyof A | keyof B]: K extends keyof B + ? B[K] + : K extends keyof A + ? A[K] + : never; +}; +type NamedProjections = S extends { readonly [EXTRA_TYPE_BRAND]?: infer P } + ? P + : {}; +type NamedKeys< S, - K extends keyof ProjectionsOf & string -> = ProjectionsOf[K] extends readonly (infer T extends string)[] - ? T - : string; - -type QueryResultType = TLocalSchema extends { - readonly [POLYMORPHIC_TYPE_BRAND]?: infer U; -} - ? NonNullable - : InferType; - -// --------------------------------------------------------------------------- -// SchemaQueryBuilder -// --------------------------------------------------------------------------- + K extends keyof NamedProjections +> = NamedProjections[K] extends readonly (infer Key extends string)[] + ? Key + : never; +/** Structural schema inferred from a typed projection. */ +export type ReadProjection = ObjectSchemaBuilder<{ + [K in keyof S & string]: S[K] extends ReadColumn + ? R + : S[K] extends AggregateExpression + ? SchemaForValue + : never; +}>; +type AddField< + S extends ReadObject, + K extends string, + F extends ReadSchema +> = ObjectSchemaBuilder, K> & Record>; +/** @internal Foreign schema retained by a declared relation. */ +export type Related = R extends RelationInfo ? S : never; +/** @internal Output cardinality and nullability of a loaded relation. */ +export type RelationField = + R extends RelationInfo<'hasMany' | 'belongsToMany', any> + ? ArraySchemaBuilder + : R extends RelationInfo<'belongsTo', any> + ? R extends { optional: infer O } + ? true extends O + ? SchemaForValue | null> + : S + : S + : SchemaForValue | null>; +type AnyReadQuery = + | SchemaQueryBuilder + | PolymorphicQueryBuilder; +type Loaded = { + name: string; + query: AnyReadQuery; + relation: RelationSpec; + required: boolean; +}; +/** Query factory result: a table query or a declared polymorphic union. */ +export type SchemaAwareQuery = + ReadVariantMetadata extends { + discriminator: string; + variants: Record; + } + ? PolymorphicQueryBuilder + : SchemaQueryBuilder; +/** @internal Apply parent correlation before child selection, ordering and pagination. */ +export type ReadCorrelation = ( + query: Knex.QueryBuilder, + alias: string, + source: ReadObject +) => void; /** - * Build schema-aware SQL with mapped columns, projections and eager relations. - * Fluent configuration methods mutate this builder; create a fresh query for each - * independent operation. Await the builder or call execute() to obtain mapped rows. - * Use query() to infer both the schema and result types automatically. + * Immutable table query whose row schema follows its decoded selection and relations. + * Configuration returns independent lazy builders; metadata access never executes SQL. + * Only unprojected table queries can write. Full ORM rows may participate in tracking. */ export class SchemaQueryBuilder< - TLocalSchema extends ObjectSchemaBuilder, - TResult -> { - /** - * Enter immutable, detached schema-aware read mode before selecting/loading fields. - * Row metadata never runs SQL; existing query behavior remains unchanged. - * @throws ReadSchemaError if result-shaping operations were already applied. - */ - withRowSchema(): SchemaAwareQuery { - return schemaReadQuery(this); - } - /** - * Create a query over the schema's configured table. - * @param knex - Database connection or transaction used to execute the query. - * @param localSchema - Schema containing property/column and relation metadata. - * @param baseQuery - Optional existing Knex query to configure; it is not cloned. - */ + S extends ReadObject, + Row extends ReadObject = ObjectReadSchema>, + Relations extends Record = ReadRelations, + Writable extends boolean = true +> extends ReadPredicates> { + /** @internal Nominal identity for typed child-query customizers. */ + declare readonly [READ_QUERY]: true; + private declare readonly writable: Writable; + private readonly alias = `__schema_read_${readAliasSequence++}`; + private fields: Record; + private loaded: Loaded[] = []; + private base: Knex.QueryBuilder; + private selected = false; + private grouped = false; + private distinctRows = false; + private defaults?: Knex.QueryBuilder; + private skipDefaults = false; + private deleted: 'exclude' | 'include' | 'only' = 'exclude'; + private readonly columns: Record>; + private readonly columnSet = new Set>(); + /** Runtime structural schema; stable across filters, pagination and transaction clones. */ + readonly rowSchema: Row; + + /** @internal Use query() or createQuery() instead of constructing directly. */ constructor( - knex: Knex, - localSchema: TLocalSchema, - baseQuery?: Knex.QueryBuilder + private readonly knex: Knex, + private readonly source: S, + base: Knex.QueryBuilder, + alias?: string ) { - const tableName = getTableName(localSchema); - setState(this, { - knex, - baseQuery: baseQuery ?? knex(tableName), - localSchema, - specs: [], - tableName, - explicitSelects: null, - selectionMode: null, - appliedProjection: null, - projectionColumns: null, - projectionDecoders: {}, - hiddenColumns: new Set(), - includeDeleted: false, - onlyDeleted: false, - skipDefaultScope: false, - variantConfig: undefined, - enabledVariants: null, - variantWhereFilters: [], - variantRelationIncludes: [], - cachedBuiltQuery: null + super(); + if (alias !== undefined) this.alias = alias; + this.base = knex.queryBuilder().from(base.clone().as(this.alias)); + const relations = (source.introspect().extensions?.relations ?? + []) as RelationSpec[]; + const excluded = new Set(relations.map(r => r.name)); + const { propToCol } = buildColumnMap(source); + this.fields = Object.create(null); + this.columns = Object.create(null); + for (const [key, schema] of Object.entries( + source.introspect().properties ?? {} + )) { + if (excluded.has(key)) continue; + const node = compileReadSchema(schema as ReadSchema); + const column = `${this.alias}.${propToCol.get(key) ?? key}`; + this.columns[key] = this.describeColumn( + schema as ReadSchema, + propToCol.get(key) ?? key + ); + this.fields[key] = { + node, + expression: knex => readExpression(knex, node, column) + }; + } + this.rowSchema = this.schema() as Row; + const defaultScope = source.introspect().extensions?.defaultScope; + if (typeof defaultScope === 'function') { + const scope = this.copy(); + scope.base = knex.queryBuilder(); + this.defaults = this.checkScope( + scope, + defaultScope(scope) + ).base.clone(); + } + } + + private describeColumn( + schema: ReadSchema, + column: string, + path: readonly string[] = [] + ): ReadColumn { + const described: ReadColumn = { + [COLUMN]: { + alias: this.alias, + column, + schema + }, + [READ_COLUMN]: compileReadSchema(schema), + [JSON_PATH]: path + }; + this.columnSet.add(described); + const info = schema.introspect(); + if (info.type === 'object') { + for (const [key, child] of Object.entries((info as any).properties)) + Object.defineProperty(described, key, { + value: this.describeColumn(child as ReadSchema, column, [ + ...path, + key + ]), + enumerable: true + }); + } + return described; + } + + private schema(): ReadObject { + return object( + Object.fromEntries( + Object.entries(this.fields).map(([key, field]) => [ + key, + field.node.schema + ]) + ) + ); + } + + private copy(): this { + const copy = Object.create(Object.getPrototypeOf(this)) as this; + Object.assign(copy, this, { + base: this.base.clone(), + fields: { ...this.fields }, + loaded: [...this.loaded] }); + return copy; } - // ======================================================================= - // SELECT / DISTINCT / AGGREGATES - // ======================================================================= + /** @internal Prevent customizers from substituting an unrelated query source. */ + sameSource(other: unknown): boolean { + return ( + other instanceof SchemaQueryBuilder && + this.columns === other.columns + ); + } - /** - * Choose columns, or use an object selector to infer a flat result shape. - * Object values can be schema descriptors or aggregate expressions. Column-list - * selection retains the existing result type; raw SQL cannot infer a new shape. - * @returns This builder, narrowed to the object projection when one is supplied. - */ - select(...columns: (ColumnRef | Knex.Raw)[]): this; - /** - * Choose columns, or use an object selector to infer a flat result shape. - * Object values can be schema descriptors or aggregate expressions. Column-list - * selection retains the existing result type; raw SQL cannot infer a new shape. - * @returns This builder, narrowed to the object projection when one is supplied. - */ - select>( - selector: TSel - ): SchemaQueryBuilder>>; - /** - * Choose columns, or use an object selector to infer a flat result shape. - * Object values can be schema descriptors or aggregate expressions. Column-list - * selection retains the existing result type; raw SQL cannot infer a new shape. - * @returns This builder, narrowed to the object projection when one is supplied. - */ - select(...args: unknown[]): any { - return selectImpl(this as any, ...args); + private column( + selector: ReadPredicateSelector> + ): ReadColumn { + const column = + typeof selector === 'string' + ? selector + .split('.') + .reduce((node: any, key) => node?.[key], this.columns) + : selector(this.columns as ReadColumns); + if (!column || !this.columnSet.has(column as ReadColumn)) + throw new ReadSchemaError( + 'Column does not belong to this read query' + ); + return column as ReadColumn; + } + + private name( + column: ReadColumn, + alias = column[COLUMN].alias + ): string | Knex.Raw { + const name = `${alias}.${column[COLUMN].column}`; + const path = column[JSON_PATH]; + if (!path?.length) return name; + const type = column[READ_COLUMN].schema.introspect().type; + const json = type === 'array' || type === 'object'; + const extracted = this.knex.raw(`?? ${json ? '#>' : '#>>'} ?::text[]`, [ + name, + [...path] + ]); + if (type === 'number') + return this.knex.raw('cast(? as numeric)', [extracted]); + if (type === 'boolean') + return this.knex.raw('cast(? as boolean)', [extracted]); + return extracted; + } + + private checkScope(input: this, result: unknown): this { + if ( + !(result instanceof SchemaQueryBuilder) || + !input.sameSource(result) || + result.rowSchema !== input.rowSchema || + result.grouped !== input.grouped || + result.selected !== input.selected || + result.distinctRows !== input.distinctRows || + result.deleted !== input.deleted || + result.skipDefaults !== input.skipDefaults || + result.knex !== input.knex + ) { + if (result instanceof Promise) void result.catch(() => {}); + throw new ReadSchemaError( + 'Scopes must synchronously return the supplied query with filters, ordering or pagination only' + ); + } + return result as this; } - /** - * Apply SQL DISTINCT to the selected columns, optionally adding columns. - * Property selectors are mapped to database names; SQL decides row equality. - */ - distinct(...columns: (ColumnRef | Knex.Raw)[]): this { - return (distinctImpl as any)(this, ...columns); + /** Apply a named, synchronous shape-preserving scope once to an independent query. */ + scoped(name: ScopesOf): this { + const scope = ( + this.source.introspect().extensions?.scopes as + | Record + | undefined + )?.[name]; + if (!scope) throw new ReadSchemaError(`Unknown scope: ${name}`); + const copy = this.copy(); + return this.checkScope(copy, scope(copy)); } - /** - * Append a legacy SQL COUNT selection without executing the query. - * The driver controls the result shape/value type. Prefer countValue() for a - * checked scalar number, or aggregate.count() in a typed object projection. - */ - count(column?: ColumnRef | Knex.Raw): this { - return (countImpl as any)(this, column); + /** Exclude only the default scope; keep explicitly configured predicates. */ + unscoped(): this { + const copy = this.copy(); + copy.skipDefaults = true; + return copy; } - /** - * Append a legacy COUNT(DISTINCT column) selection. - * Prefer countDistinctValue() for a checked scalar or aggregate.countDistinct() - * for an inferred grouped result; this legacy method retains the builder type. - */ - countDistinct(column?: ColumnRef | Knex.Raw): this { - return (countDistinctImpl as any)(this, column); + /** Include soft-deleted rows without changing the source query. */ + withDeleted(): this { + const copy = this.copy(); + copy.deleted = 'include'; + return copy; } - /** - * Append a legacy MIN selection without changing the result type. - * Use minValue() for a scalar with explicit decoding, or aggregate.min() in a - * typed projection. SQL returns null for an empty/all-null input. - */ - min(column: ColumnRef | Knex.Raw): this { - return (minImpl as any)(this, column); + /** Match only soft-deleted rows. */ + onlyDeleted(): this { + const copy = this.copy(); + copy.deleted = 'only'; + return copy; } - /** - * Append a legacy MAX selection without changing the result type. - * Use maxValue() for a scalar with explicit decoding, or aggregate.max() in a - * typed projection. SQL returns null for an empty/all-null input. - */ - max(column: ColumnRef | Knex.Raw): this { - return (maxImpl as any)(this, column); + /** True only when rows retain their complete entity shape. */ + get returnsEntityRows(): boolean { + return !this.selected && !this.grouped && !this.distinctRows; } - /** - * Append a legacy SUM selection, leaving numeric conversion to the driver. - * Prefer sumValue() or aggregate.sum() to preserve exact numeric text by default. - */ - sum(column: ColumnRef | Knex.Raw): this { - return (sumImpl as any)(this, column); + /** @internal Build an independent statement containing defaults and explicit filters. */ + private filtered(): Knex.QueryBuilder { + const query = this.base.clone(); + const explicitWhere = (query as any)._statements.filter( + (statement: any) => statement.grouping === 'where' + ); + (query as any)._statements = (query as any)._statements.filter( + (statement: any) => statement.grouping !== 'where' + ); + if (this.defaults && !this.skipDefaults) { + const defaults = this.defaults.clone() as any; + const where = defaults._statements.filter( + (statement: any) => statement.grouping === 'where' + ); + (query as any)._statements = [ + ...defaults._statements.filter( + (statement: any) => statement.grouping !== 'where' + ), + ...(query as any)._statements + ]; + (query as any)._single = { + ...defaults._single, + ...(query as any)._single + }; + if (where.length) + query.where(nested => { + (nested as any)._statements = [...where]; + }); + } + if (explicitWhere.length) + query.where(nested => { + (nested as any)._statements = [...explicitWhere]; + }); + const softDelete = this.source.introspect().extensions?.softDelete as + | { column: string } + | undefined; + if (softDelete && this.deleted !== 'include') { + query[this.deleted === 'only' ? 'whereNotNull' : 'whereNull']( + `${this.alias}.${softDelete.column}` + ); + } + return query; } - /** - * Append a legacy AVG selection, leaving numeric conversion to the driver. - * Prefer avgValue() or aggregate.avg() for a typed, precision-preserving result. - */ - avg(column: ColumnRef | Knex.Raw): this { - return (avgImpl as any)(this, column); + private async scalar( + kind: AggregateKind, + selector?: ReadPredicateSelector>, + options?: AggregateOptions + ): Promise { + if ( + this.grouped || + this.distinctRows || + Object.values(this.fields).some(field => field.aggregate) + ) + throw new ReadSchemaError( + 'Scalar aggregates require an ungrouped source' + ); + const column = + selector === undefined ? undefined : this.column(selector); + const compiled = compileAggregate( + this.knex, + createAggregate(kind, column, options), + () => this.name(column!, '__aggregate') + ); + const source = this.compile() + .clearSelect() + .clearOrder() + .clear('limit') + .clear('offset') + .select(this.knex.raw('??.*', [this.alias])); + const row = await this.knex + .from(source.as('__aggregate')) + .select({ value: compiled.sql }) + .first(); + return compiled.decode(row?.value); } /** @@ -316,9 +501,9 @@ export class SchemaQueryBuilder< * @returns A safe integer by default, including zero for an empty source. * @throws If the default count overflows, the source is grouped/distinct, or parsing fails. */ - countValue | undefined = undefined>( - options?: AggregateOptions - ): Promise>; + countValue | undefined = undefined>( + options?: AggregateOptions + ): Promise>; /** * Count non-null column values without mutating the source query. * Source paging/order is ignored; filters, scopes and transactions remain. @@ -326,10 +511,10 @@ export class SchemaQueryBuilder< * @param options - Optional parser replacing safe-integer decoding. * @throws If the default result overflows or the source is grouped/distinct. */ - countValue | undefined = undefined>( - column: ColumnRef, - options?: AggregateOptions - ): Promise>; + countValue | undefined = undefined>( + column: ReadPredicateSelector>, + options?: AggregateOptions + ): Promise>; /** * Count matching rows, or non-null column values, without mutating this query. * Ignores source ordering/limits/offsets while retaining filters and transactions. @@ -338,14 +523,15 @@ export class SchemaQueryBuilder< * @throws If the default count overflows, the source is grouped/distinct, or parsing fails. */ countValue( - columnOrOptions?: ColumnRef | AggregateOptions, + columnOrOptions?: + | ReadPredicateSelector> + | AggregateOptions, options?: AggregateOptions ): Promise { const hasColumn = typeof columnOrOptions === 'string' || typeof columnOrOptions === 'function'; - return scalarAggregate( - this, + return this.scalar( 'count', hasColumn ? columnOrOptions : undefined, hasColumn ? options : columnOrOptions @@ -359,11 +545,11 @@ export class SchemaQueryBuilder< * @returns A safe integer, or the parser's inferred output type. * @throws If the count is unsafe, the source is grouped/distinct, or parsing fails. */ - countDistinctValue | undefined = undefined>( - column: ColumnRef, - options?: AggregateOptions - ): Promise> { - return scalarAggregate(this, 'countDistinct', column, options); + countDistinctValue | undefined = undefined>( + column: ReadPredicateSelector>, + options?: AggregateOptions + ): Promise> { + return this.scalar('countDistinct', column, options); } /** @@ -373,11 +559,11 @@ export class SchemaQueryBuilder< * @returns Database numeric text, or null for empty/all-null input by default. * An output parser replaces default decoding and controls the result type. */ - sumValue | undefined = undefined>( - column: ColumnRef, - options?: AggregateOptions - ): Promise> { - return scalarAggregate(this, 'sum', column, options); + sumValue | undefined = undefined>( + column: ReadPredicateSelector>, + options?: AggregateOptions + ): Promise> { + return this.scalar('sum', column, options); } /** @@ -387,11 +573,11 @@ export class SchemaQueryBuilder< * @returns Exact database numeric text or null by default; a parser overrides this. * @remarks Text preserves database precision, not precision already lost in floating-point storage. */ - avgValue | undefined = undefined>( - column: ColumnRef, - options?: AggregateOptions - ): Promise> { - return scalarAggregate(this, 'avg', column, options); + avgValue | undefined = undefined>( + column: ReadPredicateSelector>, + options?: AggregateOptions + ): Promise> { + return this.scalar('avg', column, options); } /** @@ -402,22 +588,15 @@ export class SchemaQueryBuilder< * Dates return Date; numeric SQL overrides may return exact strings. */ minValue< - C extends ColumnRef, - S extends OutputSchema | undefined = undefined + C extends ReadPredicateSelector>, + O extends OutputSchema | undefined = undefined >( column: C, - options?: AggregateOptions + options?: AggregateOptions ): Promise< - AggregateResult< - S, - C extends (...args: any[]) => infer D - ? ExtremumValue - : C extends keyof InferType - ? ExtremumResult[C]> - : unknown - > + AggregateResult, C>> > { - return scalarAggregate(this, 'min', column, options); + return this.scalar('min', column, options); } /** @@ -428,1177 +607,1038 @@ export class SchemaQueryBuilder< * Dates return Date; numeric SQL overrides may return exact strings. */ maxValue< - C extends ColumnRef, - S extends OutputSchema | undefined = undefined + C extends ReadPredicateSelector>, + O extends OutputSchema | undefined = undefined >( column: C, - options?: AggregateOptions + options?: AggregateOptions ): Promise< - AggregateResult< - S, - C extends (...args: any[]) => infer D - ? ExtremumValue - : C extends keyof InferType - ? ExtremumResult[C]> - : unknown - > + AggregateResult, C>> > { - return scalarAggregate(this, 'max', column, options); + return this.scalar('max', column, options); } - /** - * Append a raw SELECT expression with optional Knex value/identifier bindings. - * The caller owns its SQL and result shape; this does not infer a new result type. - */ - selectRaw(sql: string, bindings?: any[]): this { - return selectRawImpl(this as any, sql, bindings); + /** Select an exact flat row shape, retaining per-field runtime schemas. */ + select & string>( + ...columns: Array< + | K + | ((columns: ReadColumns) => ReadColumn) + > + ): SchemaQueryBuilder< + S, + ObjectSchemaBuilder>, K>>, + Relations, + false + >; + /** Select named output fields and aggregates, replacing the previous scalar projection. */ + select

( + selector: (columns: ReadColumns) => P + ): SchemaQueryBuilder< + S, + ObjectSchemaBuilder< + MergeProps< + SchemaProps>, + Pick< + SchemaProps, + Extract, keyof Relations> + > + > + >, + Relations, + false + >; + select(...selectors: any[]): any { + const selections = selectors.map(selector => + typeof selector === 'function' + ? selector(this.columns) + : this.columns[selector] + ); + const selection = + selections.length === 1 && + selections[0] && + !(COLUMN in selections[0]) + ? selections[0] + : Object.fromEntries( + selections.map(column => { + const key = Object.keys(this.columns).find( + key => this.columns[key] === column + ); + if (!key) + throw new ReadSchemaError( + 'Selection must reference columns from this query' + ); + return [key, column]; + }) + ); + if (this.loaded.length && Object.values(selection).some(isAggregate)) + throw new ReadSchemaError( + 'Aggregate projections cannot contain relations' + ); + const copy = this.copy(); + copy.fields = compileReadProjection( + this.knex, + selection, + expression => { + if (!this.columnSet.has(expression as ReadColumn)) + throw new ReadSchemaError( + 'Projection column does not belong to this query' + ); + const column = expression as ReadColumn; + return { node: column[READ_COLUMN], name: this.name(column) }; + } + ); + copy.selected = true; + // Included relations are independent of the scalar projection. + for (const relation of copy.loaded) + copy.fields[relation.name] = this.fields[relation.name]; + Object.assign(copy, { rowSchema: copy.schema() }); + return copy as any; } - /** - * Apply a named schema projection and narrow the selected property type. - * @param name - Projection registered with the schema's projection() extension. - * @throws If the projection is unknown or conflicts with a prior selection. - */ - projected & string>( + /** Select a typed COUNT result on a new query. */ + count( + column?: ReadPredicateSelector> + ): SchemaQueryBuilder< + S, + ReadProjection<{ count: AggregateExpression }>, + Relations, + false + > { + return this.select(() => ({ + count: createAggregate( + 'count', + column === undefined ? undefined : this.column(column) + ) + })) as any; + } + /** Select a typed COUNTDISTINCT result on a new query. */ + countDistinct( + column: ReadPredicateSelector> + ): SchemaQueryBuilder< + S, + ReadProjection<{ countDistinct: AggregateExpression }>, + Relations, + false + > { + return this.select(() => ({ + countDistinct: createAggregate( + 'countDistinct', + column === undefined ? undefined : this.column(column) + ) + })) as any; + } + /** Select a typed SUM result on a new query. */ + sum( + column: ReadPredicateSelector> + ): SchemaQueryBuilder< + S, + ReadProjection<{ sum: AggregateExpression }>, + Relations, + false + > { + return this.select(() => ({ + sum: createAggregate( + 'sum', + column === undefined ? undefined : this.column(column) + ) + })) as any; + } + /** Select a typed AVG result on a new query. */ + avg( + column: ReadPredicateSelector> + ): SchemaQueryBuilder< + S, + ReadProjection<{ avg: AggregateExpression }>, + Relations, + false + > { + return this.select(() => ({ + avg: createAggregate( + 'avg', + column === undefined ? undefined : this.column(column) + ) + })) as any; + } + /** Select a typed MIN result on a new query. */ + min>>( + column: C + ): SchemaQueryBuilder< + S, + ReadProjection<{ + min: AggregateExpression< + SelectedValue, C> + >; + }>, + Relations, + false + > { + return this.select(() => ({ + min: createAggregate< + SelectedValue, C> + >('min', this.column(column)) + })) as any; + } + /** Select a typed MAX result on a new query. */ + max>>( + column: C + ): SchemaQueryBuilder< + S, + ReadProjection<{ + max: AggregateExpression< + SelectedValue, C> + >; + }>, + Relations, + false + > { + return this.select(() => ({ + max: createAggregate< + SelectedValue, C> + >('max', this.column(column)) + })) as any; + } + + /** Keep only distinct selected rows, preserving the row schema. */ + distinct(): SchemaQueryBuilder; + /** Select a property subset and eliminate duplicate rows on that immutable projection. */ + distinct & string>( + ...columns: Array< + | K + | ((columns: ReadColumns) => ReadColumn) + > + ): SchemaQueryBuilder< + S, + ObjectSchemaBuilder>, K>>, + Relations, + false + >; + distinct(...columns: any[]): any { + const copy = columns.length ? this.select(...columns) : this.copy(); + copy.distinctRows = true; + return copy as any; + } + /** Filter grouped results using trusted SQL and captured bindings. */ + havingRaw( + sql: string, + bindings: readonly Knex.RawBinding[] = [] + ): SchemaQueryBuilder { + const copy = this.copy(); + copy.base.havingRaw(captureReadRaw(this.knex, sql, bindings)()); + copy.grouped = true; + return copy as any; + } + /** Filter groups by a mapped column comparison. */ + having( + column: ReadPredicateSelector>, + operator: string, + value: unknown + ): SchemaQueryBuilder { + const copy = this.copy(); + copy.base.having( + this.name(this.column(column)), + operator, + captureValue(this.knex, value)() + ); + copy.grouped = true; + return copy as any; + } + /** Group using trusted SQL without changing the explicit projection. */ + groupByRaw( + sql: string, + bindings: readonly Knex.RawBinding[] = [] + ): SchemaQueryBuilder { + const copy = this.copy(); + copy.base.groupByRaw(captureReadRaw(this.knex, sql, bindings)()); + copy.grouped = true; + return copy as any; + } + + /** Apply a named schema projection with the same exact row-schema guarantees. */ + projected & string>( name: K ): SchemaQueryBuilder< - TLocalSchema, - Pick & keyof TResult> + S, + ObjectSchemaBuilder< + Pick< + SchemaProps>, + Extract, keyof SchemaProps>> + > & + Pick< + SchemaProps, + Extract, keyof Relations> + > + >, + Relations, + false > { - return projectedImpl(this as any, name); - } - - /** - * Apply a named schema scope to this query. - * @param name - Scope registered with the schema's scope() extension. - * @throws If the requested scope is not registered. - */ - scoped>(name: K): this { - return scopedImpl(this as any, name as string); - } - - /** - * Disable the default read scope and include soft-deleted rows. - * Explicit filters already added to this builder remain in place. - */ - unscoped(): this { - return unscopedImpl(this as any); - } - - // ======================================================================= - // WHERE - // ======================================================================= - - /** - * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. - * Selector/key and record forms map property names to database columns. Grouped - * callbacks receive a Knex builder and therefore use database column names. - * Values are bound; use the operator form for comparisons other than equality. - */ - where(column: ColumnRef, operator: string, value: any): this; - /** - * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. - * Selector/key and record forms map property names to database columns. Grouped - * callbacks receive a Knex builder and therefore use database column names. - * Values are bound; use the operator form for comparisons other than equality. - */ - where(column: ColumnRef, value: any): this; - /** - * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. - * Selector/key and record forms map property names to database columns. Grouped - * callbacks receive a Knex builder and therefore use database column names. - * Values are bound; use the operator form for comparisons other than equality. - */ - where(raw: Knex.Raw, operator: string, value: any): this; - /** - * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. - * Selector/key and record forms map property names to database columns. Grouped - * callbacks receive a Knex builder and therefore use database column names. - * Values are bound; use the operator form for comparisons other than equality. - */ - where(callback: (builder: Knex.QueryBuilder) => void): this; - /** - * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. - * Selector/key and record forms map property names to database columns. Grouped - * callbacks receive a Knex builder and therefore use database column names. - * Values are bound; use the operator form for comparisons other than equality. - */ - where(record: Record): this; - /** - * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. - * Selector/key and record forms map property names to database columns. Grouped - * callbacks receive a Knex builder and therefore use database column names. - * Values are bound; use the operator form for comparisons other than equality. - */ - where(raw: Knex.Raw): this; - /** - * Add an AND filter using mapped schema columns, a record, or raw Knex SQL. - * Selector/key and record forms map property names to database columns. Grouped - * callbacks receive a Knex builder and therefore use database column names. - * Values are bound; use the operator form for comparisons other than equality. - */ - where(columnOrRaw: any, ...args: any[]): this { - return whereImpl(this as any, columnOrRaw, ...args); + const definition = getProjections(this.source)[name]; + if (!definition) + throw new ReadSchemaError(`Unknown projection: ${name}`); + return this.select(() => + Object.fromEntries( + definition.keys.map(key => [key, this.columns[key]]) + ) + ) as any; + } + + protected readPredicateContext(): ReadPredicateContext< + ReadColumns + > { + return { + knex: this.knex, + column: selector => this.name(this.column(selector)) + }; } - - /** - * Add an AND condition; an explicit synonym for where(). - * Property references and record keys are mapped; raw callbacks use Knex columns. - */ - andWhere( - column: ColumnRef, - operator: string, - value: any - ): this; - /** - * Add an AND condition; an explicit synonym for where(). - * Property references and record keys are mapped; raw callbacks use Knex columns. - */ - andWhere(column: ColumnRef, value: any): this; - /** - * Add an AND condition; an explicit synonym for where(). - * Property references and record keys are mapped; raw callbacks use Knex columns. - */ - andWhere(record: Record): this; - /** - * Add an AND condition; an explicit synonym for where(). - * Property references and record keys are mapped; raw callbacks use Knex columns. - */ - andWhere(callback: (builder: Knex.QueryBuilder) => void): this; - /** - * Add an AND condition; an explicit synonym for where(). - * Property references and record keys are mapped; raw callbacks use Knex columns. - */ - andWhere(raw: Knex.Raw): this; - /** - * Add an AND condition; an explicit synonym for where(). - * Property references and record keys are mapped; raw callbacks use Knex columns. - */ - andWhere(columnOrRaw: any, ...args: any[]): this { - return andWhereImpl(this as any, columnOrRaw, ...args); + protected addReadPredicate(predicate: ReadPredicate): this { + const copy = this.copy(); + predicate(copy.base); + return copy; } - - /** - * Add an OR condition using a mapped property, record, raw SQL or Knex group. - * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. - */ - orWhere( - column: ColumnRef, - operator: string, - value: any - ): this; - /** - * Add an OR condition using a mapped property, record, raw SQL or Knex group. - * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. - */ - orWhere(column: ColumnRef, value: any): this; - /** - * Add an OR condition using a mapped property, record, raw SQL or Knex group. - * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. - */ - orWhere(record: Record): this; - /** - * Add an OR condition using a mapped property, record, raw SQL or Knex group. - * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. - */ - orWhere(callback: (builder: Knex.QueryBuilder) => void): this; - /** - * Add an OR condition using a mapped property, record, raw SQL or Knex group. - * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. - */ - orWhere(raw: Knex.Raw): this; - /** - * Add an OR condition using a mapped property, record, raw SQL or Knex group. - * Group mixed AND/OR conditions explicitly when SQL precedence would change intent. - */ - orWhere(columnOrRaw: any, ...args: any[]): this { - return orWhereImpl(this as any, columnOrRaw, ...args); + /** @internal Apply an already captured Framework predicate to an independent query. */ + withPredicate(predicate: ReadPredicate): this { + return this.addReadPredicate(predicate); } - - /** - * Add a negated condition using a mapped property, record or Knex group. - * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. - */ - whereNot( - column: ColumnRef, - operator: string, - value: any - ): this; - /** - * Add a negated condition using a mapped property, record or Knex group. - * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. - */ - whereNot(column: ColumnRef, value: any): this; - /** - * Add a negated condition using a mapped property, record or Knex group. - * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. - */ - whereNot(record: Record): this; - /** - * Add a negated condition using a mapped property, record or Knex group. - * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. - */ - whereNot(callback: (builder: Knex.QueryBuilder) => void): this; - /** - * Add a negated condition using a mapped property, record or Knex group. - * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. - */ - whereNot(raw: Knex.Raw): this; - /** - * Add a negated condition using a mapped property, record or Knex group. - * Use whereNotIn()/whereNotBetween() for their dedicated SQL operators. - */ - whereNot(columnOrRaw: any, ...args: any[]): this { - return whereNotImpl(this as any, columnOrRaw, ...args); + /** @internal Native storage columns for composing CTI table sources. */ + storageQuery(): Knex.QueryBuilder { + return this.filtered().select(this.knex.raw('??.*', [this.alias])); } - - /** - * Require the mapped column to match a value list or a single-column subquery. - * An empty list matches no rows. Subqueries use Knex's database column names. - */ - whereIn( - column: ColumnRef, - values: readonly any[] | Knex.QueryBuilder + /** Order parent rows independently of any child relation's ordering. */ + orderBy( + column: ReadPredicateSelector>, + direction: 'asc' | 'desc' = 'asc' ): this { - return (whereInImpl as any)(this, column, values); + const copy = this.copy(); + copy.base.orderBy(this.name(this.column(column)), direction); + return copy; + } + /** Append trusted raw ordering with captured positional bindings; ref() supplies quoted columns. */ + orderByRaw(sql: string, bindings: readonly Knex.RawBinding[] = []): this { + const captured = captureReadRaw(this.knex, sql, bindings); + const copy = this.copy(); + copy.base.orderByRaw(captured()); + return copy; + } + /** Group rows before typed aggregate projection. */ + groupBy( + ...columns: ReadPredicateSelector>[] + ): SchemaQueryBuilder { + const copy = this.copy(); + copy.base.groupBy(columns.map(c => this.name(this.column(c)))); + copy.grouped = true; + return copy as any; + } + /** Limit parent rows; relation limits apply independently within each parent. */ + limit(count: number): this { + if (!Number.isInteger(count) || count < 0) + throw new ReadSchemaError('Limit must be a non-negative integer'); + const copy = this.copy(); + copy.base.limit(count); + return copy; + } + /** Skip parent rows; use a deterministic order for pagination. */ + offset(count: number): this { + if (!Number.isInteger(count) || count < 0) + throw new ReadSchemaError('Offset must be a non-negative integer'); + const copy = this.copy(); + copy.base.offset(count); + return copy; + } + + /** + * Load a declared relation in the same SQL statement. Return the customized child + * query so its selected fields and nested includes remain strongly typed. + */ + include< + K extends keyof Relations & string, + Child extends ReadQueryShape = SchemaAwareQuery> + >( + selector: K | ((relations: { [P in keyof Relations]: P }) => K), + customize?: (query: SchemaAwareQuery>) => Child + ): SchemaQueryBuilder< + S, + AddField>, + Relations, + false + > { + if (Object.values(this.fields).some(f => f.aggregate) || this.grouped) + throw new ReadSchemaError( + 'Grouped/aggregate reads cannot load entity relations' + ); + const definitions = (this.source.introspect().extensions?.relations ?? + []) as RelationSpec[]; + const key = + typeof selector === 'string' + ? selector + : selector( + Object.fromEntries( + definitions.map(r => [r.name, r.name]) + ) as any + ); + const relation = definitions.find(r => r.name === key); + if (!relation) throw new ReadSchemaError(`Unknown relation: ${key}`); + if (this.loaded.some(r => r.name === key)) + throw new ReadSchemaError(`Duplicate relation: ${key}`); + const foreign = + typeof relation.schema === 'function' + ? relation.schema() + : relation.schema; + return this.load( + relation, + this.child(foreign, customize as any), + relation.type === 'belongsTo' && !relation.optional + ) as any; + } + + private child( + foreign: ReadObject, + customize?: (query: any) => ReadQueryShape + ): AnyReadQuery { + let child: AnyReadQuery = createReadQuery( + this.knex, + foreign, + this.knex(getTableName(foreign)) + ); + if (customize) { + const customized = customize(child as any); + if (!child.sameSource(customized)) { + if (customized instanceof Promise) + void customized.catch(() => {}); + throw new ReadSchemaError( + 'Relation customizer must return its configured read query' + ); + } + child = customized as any; + } + return child; } - /** - * Exclude values returned by a list or single-column subquery. - * SQL null semantics apply; a null in the set is not equivalent to a missing value. - */ - whereNotIn( - column: ColumnRef, - values: readonly any[] | Knex.QueryBuilder + private load( + relation: RelationSpec, + child: AnyReadQuery, + required: boolean ): this { - return (whereNotInImpl as any)(this, column, values); - } - - /** - * Add an OR membership condition against a list or single-column subquery. - */ - orWhereIn( - column: ColumnRef, - values: readonly any[] | Knex.QueryBuilder + const key = relation.name; + if (!key || typeof key !== 'string') + throw new ReadSchemaError('A non-empty relation alias is required'); + if (Object.values(this.fields).some(f => f.aggregate)) + throw new ReadSchemaError( + 'Grouped/aggregate reads cannot load entity relations' + ); + if ( + this.loaded.some(r => r.name === key) || + Object.hasOwn(this.fields, key) + ) + throw new ReadSchemaError(`Duplicate result field: ${key}`); + const many = + relation.type === 'hasMany' || relation.type === 'belongsToMany'; + const schema = many + ? array(child.rowSchema) + : required + ? child.rowSchema + : child.rowSchema.nullable(); + const node: ReadNode = { + schema, + exact: false, + decode: (value, path) => { + if (value === null && !required && !many) return null; + if (many) { + if (!Array.isArray(value)) + throw new ReadSchemaError( + `${path}: expected a relation array` + ); + return value.map((row, i) => + child.decode(row, `${path}[${i}]`) + ); + } + return child.decode(value, path); + } + }; + const copy = this.copy(); + copy.loaded.push({ name: key, query: child, relation, required }); + copy.fields[key] = { + node, + expression: () => { + throw new ReadSchemaError( + 'Relation expression must be compiled in context' + ); + } + }; + Object.assign(copy, { rowSchema: copy.schema() }); + return copy as any; + } + + /** @internal Reuse a captured child query across polymorphic parent branches. */ + includeFrom( + name: string, + prepared: SchemaQueryBuilder ): this { - return (orWhereInImpl as any)(this, column, values); + const loaded = prepared.loaded.find(relation => relation.name === name); + if (!loaded) + throw new ReadSchemaError(`Unknown prepared relation: ${name}`); + return this.load(loaded.relation, loaded.query, loaded.required); } - /** - * Add an OR non-membership condition; SQL NOT IN null semantics apply. - */ - orWhereNotIn( - column: ColumnRef, - values: readonly any[] | Knex.QueryBuilder - ): this { - return (orWhereNotInImpl as any)(this, column, values); + /** Join one typed nested object with explicit property keys, including nullable joins. */ + joinOne< + F extends ReadObject, + K extends string, + Required extends boolean = true, + Child extends ReadQueryShape = SchemaAwareQuery + >( + spec: Omit, 'foreignQuery' | 'mappers'>, + customize?: (query: SchemaAwareQuery) => Child + ): SchemaQueryBuilder< + S, + AddField< + Row, + K, + Required extends true + ? Child['rowSchema'] + : SchemaForValue | null> + >, + Relations & Record>, + false + > { + if ('foreignQuery' in spec || 'mappers' in spec) + throw new ReadSchemaError( + 'Use the typed child customizer instead of foreignQuery/mappers in schema-aware mode' + ); + return this.load( + { + name: spec.as, + type: 'hasOne', + schema: spec.foreignSchema, + localKey: resolvePropertyKey( + spec.localColumn as any, + this.source, + 'joinOne' + ), + remoteKey: resolvePropertyKey( + spec.foreignColumn as any, + spec.foreignSchema, + 'joinOne' + ) + }, + this.child(spec.foreignSchema, customize as any), + spec.required !== false + ) as any; + } + + /** Join a typed collection, applying child projection and pagination independently per parent. */ + joinMany< + F extends ReadObject, + K extends string, + Child extends ReadQueryShape = SchemaAwareQuery + >( + spec: Omit< + JoinManySpec, + 'orderBy' | 'foreignQuery' | 'mappers' + >, + customize?: (query: SchemaAwareQuery) => Child + ): SchemaQueryBuilder< + S, + AddField>, + Relations & Record>, + false + > { + if ('foreignQuery' in spec || 'mappers' in spec || 'orderBy' in spec) + throw new ReadSchemaError( + 'Use the typed child customizer for ordering instead of raw foreignQuery/mappers' + ); + let child = this.child(spec.foreignSchema, customize as any); + if (spec.limit !== undefined) child = child.limit(spec.limit); + if (spec.offset !== undefined) child = child.offset(spec.offset); + return this.load( + { + name: spec.as, + type: 'hasMany', + schema: spec.foreignSchema, + localKey: resolvePropertyKey( + spec.localColumn as any, + this.source, + 'joinMany' + ), + remoteKey: resolvePropertyKey( + spec.foreignColumn as any, + spec.foreignSchema, + 'joinMany' + ) + }, + child, + false + ) as any; + } + + /** @internal Decode a SQL/JSON row using the same metadata exposed to consumers. */ + decode(row: any, path = 'row'): any { + if ( + this.source.introspect().extensions?.readOrphanColumn && + row.__read_cti_present == null + ) + throw new ReadSchemaError(`${path}: missing CTI variant body`); + return decodeObject( + Object.fromEntries( + Object.entries(this.fields).map(([key, field]) => [ + key, + field.node + ]) + ), + row, + path + ); } - /** - * Add an AND IS NULL condition for a mapped schema property. - */ - whereNull(column: ColumnRef): this { - return (whereNullImpl as any)(this, column); + /** @internal Compile a bound SQL statement; callers never receive the mutable builder. */ + compile(correlate?: ReadCorrelation): Knex.QueryBuilder { + const query = this.filtered().clearSelect(); + correlate?.(query, this.alias, this.source); + const expressions: Record = Object.create(null); + for (const [key, field] of Object.entries(this.fields)) { + if (!this.loaded.some(r => r.name === key)) + expressions[key] = field.expression(this.knex); + } + const orphanColumn = + this.source.introspect().extensions?.readOrphanColumn; + if (orphanColumn) + expressions.__read_cti_present = this.knex.raw('??', [ + `${this.alias}.${orphanColumn}` + ]); + for (const loaded of this.loaded) { + const { relation, query: child } = loaded; + const parentTable = this.alias; + const parentPk = getPrimaryKeyColumns(this.source).columnNames; + const foreignKey = relation.foreignKey; + const resolveKey = (schema: ReadObject, key: any) => { + if (typeof key === 'function') { + const descriptor = key( + ObjectSchemaBuilderValue.getPropertiesFor(schema) + ); + key = + descriptor[SYMBOL_SCHEMA_PROPERTY_DESCRIPTOR] + .propertyName; + } + return buildColumnMap(schema).propToCol.get(key) ?? key; + }; + const childSql = child.compile((sql, childTable, childSource) => { + const childPk = getPrimaryKeyColumns(childSource).columnNames; + if (relation.localKey && relation.remoteKey) { + sql.where( + `${childTable}.${resolveKey(childSource, relation.remoteKey)}`, + this.knex.ref( + `${parentTable}.${resolveKey(this.source, relation.localKey)}` + ) + ); + return; + } + if (parentPk.length !== 1) + throw new ReadSchemaError( + 'Automatic relation reads require single-column primary keys' + ); + if (childPk.length !== 1) + throw new ReadSchemaError( + 'Automatic relation reads require single-column primary keys' + ); + if (relation.type === 'belongsTo') + sql.where( + `${childTable}.${childPk[0]}`, + this.knex.ref( + `${parentTable}.${resolveKey(this.source, foreignKey)}` + ) + ); + else if (relation.type === 'belongsToMany') { + const through = relation.through!; + sql.join( + through.table, + `${through.table}.${through.foreignKey}`, + `${childTable}.${childPk[0]}` + ).where( + `${through.table}.${through.localKey}`, + this.knex.ref(`${parentTable}.${parentPk[0]}`) + ); + } else + sql.where( + `${childTable}.${resolveKey(childSource, foreignKey)}`, + this.knex.ref(`${parentTable}.${parentPk[0]}`) + ); + }); + const many = + relation.type === 'hasMany' || + relation.type === 'belongsToMany'; + if (!many) childSql.limit(1); + const wrapped = this.knex + .queryBuilder() + .from(childSql.clone().as('__read_relation')); + expressions[loaded.name] = many + ? this.knex.raw( + "(select coalesce(jsonb_agg(to_jsonb(__read_relation)), '[]'::jsonb) from (?) as __read_relation)", + [childSql] + ) + : this.knex.raw( + '(select to_jsonb(__read_relation) from (?) as __read_relation)', + [childSql] + ); + if (loaded.required) + query.whereExists(wrapped.clone().select(this.knex.raw('1'))); + } + return (this.distinctRows ? query.distinct() : query).select( + expressions + ); } - /** - * Add an AND IS NOT NULL condition for a mapped schema property. - */ - whereNotNull(column: ColumnRef): this { - return (whereNotNullImpl as any)(this, column); + /** Render debugging SQL without execution; bound values may be sensitive. */ + toQuery(): string { + return this.compile().toQuery(); } - - /** - * Add an OR IS NULL condition for a mapped schema property. - */ - orWhereNull(column: ColumnRef): this { - return (orWhereNullImpl as any)(this, column); + /** Return an independent mutable Knex snapshot, never the query's owned state. */ + toKnexQuery(): Knex.QueryBuilder { + return this.compile(); + } + + /** Configure an isolated Knex SELECT once, declaring the complete raw row output. */ + apply( + configure: (query: Knex.QueryBuilder) => Knex.QueryBuilder | undefined, + options: QueryOutput + ): OpaqueQuery { + const sql = this.compile(); + const result = configure(sql); + if (result !== undefined && result !== sql) { + if (result instanceof Promise) void result.catch(() => {}); + throw new ReadSchemaError( + 'Raw configuration must synchronously configure the supplied Knex builder' + ); + } + return OpaqueQuery.capture(this.knex, sql, options); + } + /** Replace the projection with trusted raw SQL and an explicit output schema. */ + selectRaw( + sql: string, + bindings: readonly Knex.RawBinding[], + options: QueryOutput + ): OpaqueQuery { + const captured = captureReadRaw(this.knex, sql, bindings); + return this.apply( + query => query.clearSelect().select(captured()), + options + ); } - /** - * Add an OR IS NOT NULL condition for a mapped schema property. - */ - orWhereNotNull(column: ColumnRef): this { - return (orWhereNotNullImpl as any)(this, column); + private writer(filtered = true): QuerySource> { + if (!this.returnsEntityRows || this.loaded.length) + throw new ReadSchemaError( + 'Writes require an unprojected table query without relations or aggregation' + ); + let base = this.knex(getTableName(this.source)); + if (filtered) { + const keys = getPrimaryKeyColumns(this.source).columnNames; + if (keys.length) { + base.whereIn( + keys, + this.filtered() + .clearSelect() + .select(keys.map(key => `${this.alias}.${key}`)) + ); + } else { + const captured = this.filtered(); + if ( + (captured as any)._single.limit !== undefined || + (captured as any)._single.offset !== undefined + ) + throw new ReadSchemaError( + 'Paginated writes require a primary key' + ); + base = captured + .from({ [this.alias]: getTableName(this.source) }) + .clearSelect() + .clearOrder(); + } + } + const writer = new QuerySource>( + this.knex, + this.source, + base + ); + const state = getState(writer); + state.skipDefaultScope = true; + state.includeDeleted = true; + state.decodeRow = row => this.decode(row); + state.hookQuery = this; + return writer; } - /** - * Require the mapped column to lie within an inclusive [lower, upper] range. - */ - whereBetween( - column: ColumnRef, - range: readonly [any, any] - ): this { - return (whereBetweenImpl as any)(this, column, range); + /** Configure immutable conflict handling for one-row inserts. */ + onConflict( + this: Writable extends true ? this : never, + ...columns: ColumnRef[] + ): Pick< + ReturnType>['onConflict']>, + 'merge' | 'ignore' + > { + const conflict = this.writer(false).onConflict(...columns); + return { + merge: (async (...args: any[]) => { + const row = await (conflict.merge as Function)(...args); + return row; + }) as typeof conflict.merge, + ignore: async data => { + const row = await conflict.ignore(data); + return row; + } + }; + } + + /** Insert one row, returning the same decoded storage representation as reads. */ + async insert( + this: Writable extends true ? this : never, + data: InsertType + ): Promise> { + return this.writer(false).insert(data); + } + /** Insert multiple rows and decode each returned row. */ + async insertMany( + this: Writable extends true ? this : never, + data: InsertType[] + ): Promise[]> { + const writer = this.writer(false); + if (data.length === 0) return []; + return writer.insertMany(data); + } + /** Update matching entities; projections and relations cannot be written. */ + async update( + this: Writable extends true ? this : never, + data: Partial | InferType> + ): Promise[]> { + return this.writer().update(data as any); + } + /** Delete matching rows, respecting configured soft deletion. */ + delete(this: Writable extends true ? this : never): Promise { + return this.writer().delete(); + } + /** Permanently delete matching rows. */ + hardDelete(this: Writable extends true ? this : never): Promise { + return this.writer().hardDelete(); + } + /** Restore matching soft-deleted rows. */ + async restore( + this: Writable extends true ? this : never + ): Promise[]> { + return this.writer().restore(); + } + /** Insert in parameter-safe chunks, retaining schema hooks and conflict handling. */ + async bulkInsert( + this: Writable extends true ? this : never, + ...args: Parameters>['bulkInsert']> + ): Promise[]> { + return this.writer(false).bulkInsert(...args); } - - /** - * Exclude the inclusive [lower, upper] range from a mapped column. - */ - whereNotBetween( - column: ColumnRef, - range: readonly [any, any] - ): this { - return (whereNotBetweenImpl as any)(this, column, range); + /** Upsert a row using mapped conflict keys. */ + async upsert( + this: Writable extends true ? this : never, + ...args: Parameters>['upsert']> + ): Promise> { + return this.writer(false).upsert(...args); } - - /** - * Match a mapped column against a SQL LIKE pattern. - * Percent and underscore remain wildcards; values are bound, not wildcard-escaped. + /** Upsert rows in parameter-safe chunks. */ + async bulkUpsert( + this: Writable extends true ? this : never, + ...args: Parameters>['bulkUpsert']> + ): Promise[]> { + return this.writer(false).bulkUpsert(...args); + } + /** Update rows by their declared keys in parameter-safe chunks. */ + bulkUpdate( + this: Writable extends true ? this : never, + updates: ReadonlyArray<{ + where: Partial>; + set: Partial | InferType>; + }> + ): Promise { + return this.writer().bulkUpdate(updates as any); + } + + /** Read values of one selected property without altering this query. */ + async pluck & string>( + key: K + ): Promise[K]>[]> { + return (await this.execute()).map(row => (row as any)[key]); + } + /** Execute one statement and decode its selected row graph. */ + async execute(): Promise[]> { + return (await this.compile()).map((row: unknown) => this.decode(row)); + } + /** Execute a limited copy, returning undefined when no row matches. */ + async first(): Promise | undefined> { + return (await this.limit(1).execute())[0]; + } + /** Awaiting executes the query; repeated awaits deliberately execute again. */ + // biome-ignore lint/suspicious/noThenProperty: query readers intentionally support await + then[], E = never>( + resolve?: ((rows: InferType[]) => T | PromiseLike) | null, + reject?: ((error: any) => E | PromiseLike) | null + ): Promise { + return this.execute().then(resolve, reject); + } + /** Bind an independent query graph to a caller-owned transaction. */ + transacting(trx: Knex.Transaction): this { + const copy = this.copy(); + copy.base.transacting(trx); + Object.assign(copy, { knex: trx }); + copy.loaded = this.loaded.map(r => ({ + ...r, + query: r.query.transacting(trx) + })); + return copy; + } + /** + * Read a lossless composite cursor page using native SQL ordering. Cursor sort + * values remain private text columns, independent of projections and Date decoding. + * Requires non-null scalar order columns containing a declared unique key. */ - whereLike(column: ColumnRef, value: string): this { - return (whereLikeImpl as any)(this, column, value); + paginateAfter( + options: CompositeCursorOptions + ): Promise>>; + /** Read a page using a single raw cursor value; choose a unique, non-null sort column. */ + paginateAfter(options: { + cursor?: unknown; + limit: number; + column?: ColumnRef; + direction?: 'asc' | 'desc'; + }): Promise>>; + /** Execute a cursor page without changing this query's filters, ordering or schema. */ + async paginateAfter( + options: + | CompositeCursorOptions + | { + cursor?: unknown; + limit: number; + column?: ColumnRef; + direction?: 'asc' | 'desc'; + } + ): Promise>> { + if (this.grouped || Object.values(this.fields).some(f => f.aggregate)) + throw new ReadSchemaError( + 'Cursor pagination cannot be used for grouped or aggregate reads' + ); + if (!('orderBy' in options)) { + if (!Number.isInteger(options.limit) || options.limit < 1) + throw new ReadSchemaError( + 'Cursor page limit must be a positive integer' + ); + const key = + options.column === undefined + ? 'id' + : resolvePropertyKey( + options.column as any, + this.source, + 'cursor' + ); + const column = this.columns[key]; + if (!column) + throw new ReadSchemaError(`Unknown cursor column: ${key}`); + const direction = options.direction ?? 'desc'; + if (direction !== 'asc' && direction !== 'desc') + throw new ReadSchemaError('Invalid cursor direction'); + const name = this.name(column); + let hidden = '__cursor_value'; + while (Object.hasOwn(this.fields, hidden)) hidden += '_'; + const sql = this.compile() + .clearOrder() + .clear('offset') + .limit(options.limit + 1) + .orderBy(name, direction) + .select({ + [hidden]: readExpression( + this.knex, + column[READ_COLUMN], + name + ) + }); + if (options.cursor != null) + sql.where( + name as any, + direction === 'desc' ? '<' : '>', + options.cursor as any + ); + const rows = await sql; + const hasMore = rows.length > options.limit; + const page = rows.slice(0, options.limit); + return { + data: page.map((row: any) => this.decode(row)), + hasMore, + nextCursor: hasMore + ? String(page[page.length - 1][hidden]) + : null + }; + } + const Constructor = getQuerySourceCtor(); + const source = this.source.withExtension('tableName', this.alias); + const legacy = new Constructor(this.knex, source, this.compile()); + const state = getState(legacy); + state.skipDefaultScope = true; + state.includeDeleted = true; + // The query is already projected; reserve its aliases against cursor fields. + state.hiddenColumns = new Set(Object.keys(this.fields)); + return compositeCursor( + legacy, + options, + row => this.decode(row), + this.source.introspect().extensions?.tableName as string + ); } - - /** - * Match a mapped column using PostgreSQL's case-insensitive ILIKE operator. - * Percent and underscore remain pattern wildcards. - */ - whereILike(column: ColumnRef, value: string): this { - return (whereILikeImpl as any)(this, column, value); - } - - /** - * Append raw WHERE SQL with Knex bindings. Never interpolate untrusted values. - * Raw SQL uses database names and is outside schema-level result/type checking. - */ - whereRaw(sql: string, ...bindings: any[]): this { - return (whereRawImpl as any)(this, sql, ...bindings); - } - - /** - * Add an EXISTS filter from a Knex subquery or query-building callback. - * Use qualified database columns to correlate it with the parent query. - */ - whereExists(callback: Knex.QueryCallback | Knex.QueryBuilder): this { - return (whereExistsImpl as any)(this, callback); - } - - /** - * Add a NOT EXISTS filter from a Knex subquery or query-building callback. - */ - whereNotExists(callback: Knex.QueryCallback | Knex.QueryBuilder): this { - return (whereNotExistsImpl as any)(this, callback); - } - - /** - * Compare a JSON-path value inside a mapped JSON column. - * @param path - JSON path understood by the Knex database dialect. - * @param operator - SQL comparison operator forwarded to Knex. - * @param value - Bound comparison value. - */ - whereJsonPath( - column: ColumnRef, - path: string, - operator?: string, - value?: any - ): this { - return (whereJsonPathImpl as any)(this, column, path, operator, value); - } - - // ======================================================================= - // ORDER BY - // ======================================================================= - - /** - * Append ordering by a mapped property or raw expression (ascending by default). - * Add a unique tie-breaker for stable pages. Eager loading retains parent order. - */ - orderBy( - column: ColumnRef | Knex.Raw, - direction?: 'asc' | 'desc' - ): this { - return (orderByImpl as any)(this, column, direction); - } - - /** - * Append a raw ORDER BY expression with Knex bindings. - * Use database column names; parent ordering is retained during eager loading. - */ - orderByRaw(sql: string, ...bindings: any[]): this { - return (orderByRawImpl as any)(this, sql, ...bindings); - } - - // ======================================================================= - // GROUP BY / HAVING - // ======================================================================= - - /** - * Group rows by mapped schema properties or raw expressions. - * Combine with aggregate expressions in select() to infer grouped DTO results. - */ - groupBy(...columns: (ColumnRef | Knex.Raw)[]): this { - return (groupByImpl as any)(this, ...columns); - } - - /** - * Append raw GROUP BY SQL with optional Knex bindings. - */ - groupByRaw(sql: string, ...bindings: any[]): this { - return (groupByRawImpl as any)(this, sql, ...bindings); - } - - /** - * Filter SQL groups by a mapped column/raw expression, operator and bound value. - */ - having( - column: ColumnRef | Knex.Raw, - operator: string, - value: any - ): this { - return (havingImpl as any)(this, column, operator, value); - } - - /** - * Append raw HAVING SQL with Knex bindings, for example aggregate comparisons. - */ - havingRaw(sql: string, ...bindings: any[]): this { - return havingRawImpl(this as any, sql, ...bindings); - } - - // ======================================================================= - // PAGINATION - // ======================================================================= - - /** - * Set the maximum number of parent rows to select; mutates this query. - * Included collections do not consume the parent limit. - */ - limit(n: number): this { - return limitImpl(this as any, n); - } - - /** - * Skip this many parent rows before applying the limit; mutates this query. - * Use deterministic ordering when navigating offset-based pages. - */ - offset(n: number): this { - return offsetImpl(this as any, n); - } - - /** - * Execute a one-based offset page and a separate matching-source count query. - * Mutates this builder's limit/offset and returns mapped rows plus page metadata. - * The count and page are separate reads, not a snapshot unless your transaction provides one. - */ - async paginate(opts: { - /** One-based page number. */ + /** Fetch a numbered page and a count without mutating the source query. */ + async paginate(options: { page: number; - /** Maximum parent rows in a page. */ pageSize: number; - }): Promise> { - return paginateImpl(this as any, opts) as Promise< - PaginationResult - >; - } - - /** - * Read a cursor page without running a total-count query. - * The orderBy form clones the source and preserves exact composite sort values; - * its non-null sort must contain a declared unique key. The legacy column form - * mutates this builder and defaults to id descending. Reapply access filters on - * every request: cursors are positions, not authorization or snapshots. - * @returns Mapped data, hasMore, and a nextCursor that is null on the last page. - * @throws For invalid composite cursors or unsupported composite query shapes. - */ - paginateAfter( - opts: CompositeCursorOptions - ): Promise>; - /** - * Read a cursor page without running a total-count query. - * The orderBy form clones the source and preserves exact composite sort values; - * its non-null sort must contain a declared unique key. The legacy column form - * mutates this builder and defaults to id descending. Reapply access filters on - * every request: cursors are positions, not authorization or snapshots. - * @returns Mapped data, hasMore, and a nextCursor that is null on the last page. - * @throws For invalid composite cursors or unsupported composite query shapes. - */ - paginateAfter(opts: { - /** Previous raw single-column position; omit for the first page. */ - cursor?: any; - /** Maximum parent rows to return; one extra row determines hasMore. */ - limit: number; - /** Unique sort property; defaults to id in the legacy API. */ - column?: ColumnRef; - /** Sort/continuation direction; defaults to descending. */ - direction?: 'asc' | 'desc'; - }): Promise>; - /** - * Read a cursor page without running a total-count query. - * The orderBy form clones the source and preserves exact composite sort values; - * its non-null sort must contain a declared unique key. The legacy column form - * mutates this builder and defaults to id descending. Reapply access filters on - * every request: cursors are positions, not authorization or snapshots. - * @returns Mapped data, hasMore, and a nextCursor that is null on the last page. - * @throws For invalid composite cursors or unsupported composite query shapes. - */ - async paginateAfter(opts: any): Promise> { - if ('orderBy' in opts) return compositeCursor(this as any, opts); - return (paginateAfterImpl as any)(this, opts) as Promise< - CursorPaginationResult - >; - } - - // ======================================================================= - // WRITE OPERATIONS - // ======================================================================= - - /** - * Insert one schema-shaped row and return its mapped database representation. - * Applies configured insert hooks, column mappings and timestamp defaults. - */ - async insert(data: InsertType): Promise { - return insertImpl(this as any, data) as Promise; - } - - /** - * Insert an array of schema-shaped rows and return their mapped representations. - * Returns an empty array for empty input; use bulkInsert() to control chunking. - */ - async insertMany(data: InsertType[]): Promise { - return insertManyImpl(this as any, data) as Promise; - } - - /** - * Configure an upsert conflict target using mapped properties. - * Call merge() or ignore() on the returned builder to insert the row. - */ - onConflict( - ...conflictColumns: ColumnRef[] - ): import('./operations/insert.js').OnConflictBuilder< - TLocalSchema, - TResult - > { - return (onConflictImpl as any)(this, ...conflictColumns); - } - - /** - * Insert one row or update it when the chosen conflict target already exists. - * @param opts - Conflict properties and optional subset of properties to update. - * @returns The inserted or updated row mapped to schema property names. - */ - async upsert( - data: InsertType, - opts: { - /** Properties identifying an existing row on conflict. */ - conflictColumns: ColumnRef[]; - /** Properties to update on conflict; omit to merge insert values. */ - updateColumns?: ColumnRef[]; - } - ): Promise { - return (upsertImpl as any)(this, data, opts); - } - - /** - * Insert rows in chunks, optionally ignoring or merging conflicts. - * @param opts - Chunk size (default 500), conflict policy and conflict properties. - * @returns Mapped rows returned by PostgreSQL; ignored conflicts produce no row. - */ - async bulkInsert( - rows: InsertType[], - opts?: { - /** Requested rows per statement, capped by parameter limits; default 500. */ - chunkSize?: number; - /** Optional PostgreSQL conflict policy applied to each chunk. */ - onConflict?: 'ignore' | 'merge'; - /** Conflict target properties when a conflict policy is supplied. */ - conflictColumns?: ColumnRef[]; - } - ): Promise { - return (bulkInsertImpl as any)(this, rows, opts); - } - - /** - * Insert/update rows in chunks using the specified conflict properties. - * @param opts - Required conflict target and optional chunk size (default 500). - * @returns The database rows produced by each chunk, mapped to schema properties. - */ - async bulkUpsert( - rows: InsertType[], - opts: { - /** Properties identifying an existing row on conflict. */ - conflictColumns: ColumnRef[]; - /** Requested rows per statement, capped by parameter limits; default 500. */ - chunkSize?: number; - } - ): Promise { - return (bulkUpsertImpl as any)(this, rows, opts); - } - - // ======================================================================= - // UPDATE - // ======================================================================= - - /** - * Update rows matching this query's explicit filters and return mapped rows. - * Applies update hooks and timestamp metadata. Add a WHERE clause to avoid a - * table-wide update; this method does not track entity identity. - */ - async update(data: Partial>): Promise { - return updateImpl(this as any, data) as Promise; - } - - /** - * Apply per-row where/set pairs and return the total affected-row count. - * Maps both filter and update property names and applies configured update hooks. - */ - async bulkUpdate( - updates: ReadonlyArray<{ - /** Equality filters identifying the rows for this update. */ - where: Partial>; - /** Schema properties to assign to those rows. */ - set: Partial>; - }> - ): Promise { - return bulkUpdateImpl(this as any, updates as any); - } - - // ======================================================================= - // DELETE / SOFT DELETE - // ======================================================================= - - /** - * Delete rows matching explicit filters and return the affected-row count. - * Runs beforeDelete hooks; with soft-delete metadata it sets the deletion timestamp - * instead of removing rows. Add filters to avoid a table-wide write. - */ - async delete(): Promise { - return deleteImpl(this as any); - } - - /** - * Include soft-deleted rows in read results without removing explicit filters. - */ - withDeleted(): this { - return withDeletedImpl(this as any); - } - - /** - * Restrict reads to rows whose configured soft-delete column is non-null. - */ - onlyDeleted(): this { - return onlyDeletedImpl(this as any); - } - - /** - * Permanently delete rows matching explicit filters, even on a soft-delete schema. - * Runs beforeDelete hooks and returns the affected count. This cannot be undone - * without a transaction rollback or backup. - */ - async hardDelete(): Promise { - return hardDeleteImpl(this as any); - } - - /** - * Clear the deletion timestamp on rows matching explicit filters and return them. - * @throws If the schema has no soft-delete configuration. - */ - async restore(): Promise { - return restoreImpl(this as any) as Promise; - } - - // ======================================================================= - // EAGER LOADING (JOIN) - // ======================================================================= - - /** - * Eager-load a related object under spec.as using mapped join columns. - * Required joins remove unmatched parents; optional joins return null. Unlike a - * flat join, related fields remain nested and are mapped with the foreign schema. - */ - joinOne< - TForeignSchema extends ObjectSchemaBuilder< - any, - any, - any, - any, - any, - any, - any - >, - TFieldName extends string, - TRequired extends boolean = true - >( - spec: JoinOneSpec - ): SchemaQueryBuilder< - TLocalSchema, - import('./types.js').WithJoinedOne< - TResult, - TFieldName, - TForeignSchema, - TRequired - > - > { - return joinOneImpl(this as any, spec); - } - - /** - * Eager-load a nested array without multiplying parent rows. - * The relation's own order/limit/offset controls children separately from parent paging. - */ - joinMany< - TForeignSchema extends ObjectSchemaBuilder< - any, - any, - any, - any, - any, - any, - any - >, - TFieldName extends string - >( - spec: JoinManySpec - ): SchemaQueryBuilder< - TLocalSchema, - import('./types.js').WithJoinedMany - > { - return joinManyImpl(this as any, spec); - } - - /** - * Eager-load a relation registered on the schema by name. - * @param customize - Optional callback configuring the related query. - * Use an ORM DbSet when you need typed relation names and customization fields. - */ - include( - relationName: string, - customize?: (q: SchemaQueryBuilder) => void - ): this { - return includeImpl(this as any, relationName, customize); - } - - /** - * Eager-load a relation declared for one polymorphic discriminator value. - * Other variants are not populated with this relation. Use ORM entity declarations - * to retain known relation customization types. - */ - includeVariant( - variantKey: string, - relationName: string, - customize?: (q: SchemaQueryBuilder) => void - ): this { - return includeVariantImpl( - this as any, - variantKey, - relationName, - customize - ); - } - - // ======================================================================= - // POLYMORPHIC VARIANTS - // ======================================================================= - - /** - * Add a filter applying only to the named polymorphic branch. - * Other discriminator values remain eligible. Maps the variant property to its - * CTI/STI storage column; throws for unknown variants or unsupported operators. - */ - whereVariant( - key: string, - column: string, - operator: string, - value: any - ): this { - const state = getState(this); - const variantConfig = getVariantConfig(this); - if (!variantConfig) { - throw new Error( - 'whereVariant() can only be used on a polymorphic schema (created with .withVariants())' + }): Promise>> { + const { page, pageSize } = options; + if ( + !Number.isInteger(page) || + page < 1 || + !Number.isInteger(pageSize) || + pageSize < 1 + ) + throw new ReadSchemaError( + 'Page and pageSize must be positive integers' ); - } - - const spec = variantConfig.variants[key]; - if (!spec) { - throw new Error( - `whereVariant: unknown variant key "${key}". ` + - `Valid keys: ${Object.keys(variantConfig.variants).join(', ')}` - ); - } - - const op = operator.toLowerCase(); - if (!ALLOWED_OPS.has(op)) { - throw new Error( - `whereVariant: operator "${operator}" is not allowed. ` + - `Allowed operators: ${[...ALLOWED_OPS].join(', ')}` - ); - } - - const { propToCol } = buildColumnMap(spec.schema); - const colName = propToCol.get(column) ?? column; - - let qualifiedColumn: string; - if (spec.storage === 'cti') { - qualifiedColumn = `__v_${key}.${colName}`; - } else { - qualifiedColumn = `${state.tableName}.${colName}`; - } - - state.variantWhereFilters.push({ key, qualifiedColumn, op, value }); - invalidateCache(this); - return this; - } - - /** - * Choose which polymorphic variant bodies are loaded. - * This controls variant joins/selection, not a discriminator filter on base rows. - * @throws If the schema is not polymorphic. - */ - selectVariants(keys: string[]): this { - const state = getState(this); - if (!getVariantConfig(this)) { - throw new Error( - 'selectVariants() can only be used on a polymorphic schema (created with .withVariants())' + const countQuery = this.compile() + .clearOrder() + .clear('limit') + .clear('offset'); + const countRow = await this.knex + .from(countQuery.as('__read_count')) + .count({ count: '*' }) + .first(); + const total = Number(countRow?.count ?? 0); + if (!Number.isSafeInteger(total)) + throw new ReadSchemaError( + 'Pagination count exceeds the safe integer range' ); - } - state.enabledVariants = new Set(keys); - invalidateCache(this); - return this; - } - - // ======================================================================= - // ESCAPE HATCH - // ======================================================================= - - /** - * Configure the underlying mutable Knex query as an escape hatch. - * Raw changes do not infer a new result type; the caller owns column names, - * result shape and cardinality introduced by the callback. - */ - apply(fn: (builder: Knex.QueryBuilder) => void): this { - const state = getState(this); - state.opaqueReadShape = true; - invalidateCache(this); - fn(state.baseQuery); - return this; - } - - // ======================================================================= - // TRANSACTION - // ======================================================================= - - /** - * Clone this query and its eager-relation queries onto an existing transaction. - * The source builder is unchanged; transaction commit/rollback stays with the caller. - */ - transacting( - trx: Knex.Transaction - ): SchemaQueryBuilder { - const state = getState(this); - const builder = new SchemaQueryBuilder( - trx as unknown as Knex, - state.localSchema as TLocalSchema, - state.baseQuery.clone().transacting(trx) - ); - const builderState = getState(builder); - for (const spec of state.specs) { - builderState.specs.push({ - ...spec, - foreignQuery: spec.foreignQuery.clone().transacting(trx) - }); - } - builderState.explicitSelects = state.explicitSelects - ? [...state.explicitSelects] - : null; - builderState.selectionMode = state.selectionMode; - builderState.appliedProjection = state.appliedProjection; - builderState.projectionColumns = state.projectionColumns - ? { ...state.projectionColumns } - : null; - builderState.projectionDecoders = { ...state.projectionDecoders }; - builderState.hiddenColumns = new Set(state.hiddenColumns); - builderState.includeDeleted = state.includeDeleted; - builderState.onlyDeleted = state.onlyDeleted; - builderState.skipDefaultScope = state.skipDefaultScope; - builderState.opaqueReadShape = state.opaqueReadShape; - builderState.variantConfig = state.variantConfig; - builderState.enabledVariants = - state.enabledVariants !== null - ? new Set(state.enabledVariants) - : null; - builderState.variantWhereFilters = [...state.variantWhereFilters]; - builderState.variantRelationIncludes = [ - ...state.variantRelationIncludes - ]; - return builder; - } - - // ======================================================================= - // EXECUTION - // ======================================================================= - - /** - * Render SQL for debugging without executing it. - * Bindings may appear as literal values; avoid logging sensitive inputs. - */ - toQuery(): string { - return getQuery(this).toQuery(); - } - - /** @internal ORM tracking must never attach aggregate or DTO rows as entities. */ - get returnsEntityRows(): boolean { - return getState(this).selectionMode === null; - } - - /** - * Expose the built Knex query without executing Framework row mapping. - * Direct execution returns raw database rows, potentially including internal fields. - * Treat mutations as an escape hatch rather than typed Framework configuration. - */ - toKnexQuery(): Knex.QueryBuilder { - return getQuery(this); - } - - /** - * Render the query as a debugging SQL string; equivalent to toQuery(). - */ - toString(): string { - return getQuery(this).toString(); - } - - /** - * Execute SQL and map all rows, including eager relations and aggregate decoders. - * @returns An empty array when no rows match. - * @throws Database errors and output-parser validation failures. - */ - async execute(): Promise { - return executeImpl(this) as Promise; - } - - /** - * Execute this query for its first mapped row, or undefined if none matches. - * Set ordering when the choice of first row matters. - */ - async first(): Promise { - const query = getQuery(this).first(); - const row = await query; - - if (!row) return undefined; - return cleanAndMapRow(this, row) as TResult; - } - - /** - * Execute a selection and collect one mapped column's values into an array. - * @param column - Schema property whose database column should be read. - */ - async pluck( - column: ColumnRef - ): Promise { - const _state = getState(this); - const col = resolveColumn(this, column, 'pluck') as string; - const rows = await buildQuery(this).select(col); - return rows.map( - (row: any) => row[col] ?? row[column as string] - ) as TResult[K][]; - } - - /** - * Promise-compatible execution hook enabling await query(...). - * Runs execute() and forwards fulfillment/rejection; repeated awaits may execute again. - */ - // biome-ignore lint/suspicious/noThenProperty: intentional thenable - then( - onfulfilled?: - | ((value: TResult[]) => TReturn1 | PromiseLike) - | null, - onrejected?: ((reason: any) => TReturn2 | PromiseLike) | null - ): Promise { - return this.execute().then(onfulfilled, onrejected); + const data = await this.offset((page - 1) * pageSize) + .limit(pageSize) + .execute(); + const totalPages = Math.ceil(total / pageSize); + return { + data, + total, + page, + pageSize, + totalPages, + hasNextPage: page < totalPages, + hasPreviousPage: page > 1 + }; } } -// Register the constructor for circular-dependency-safe access -registerSchemaQueryBuilder(SchemaQueryBuilder); - -// --------------------------------------------------------------------------- -// query() — main entry point -// --------------------------------------------------------------------------- - -/** - * Create a schema-aware query while preserving its schema and inferred row type. - * Pass a table alias for a flat multi-table query requiring an explicit projection; - * pass an ordinary schema for nested eager loading and schema-aware writes. - * @param knex - Knex connection or transaction. - * @param schema - Table schema or immutable alias(schema, name). - * @returns The appropriately typed, unexecuted query builder. - */ -export function query< - S extends ObjectSchemaBuilder, - N extends string ->(knex: Knex, schema: TableAlias): AliasedQueryBuilder>; -/** - * Create a schema-aware query while preserving its schema and inferred row type. - * Pass a table alias for a flat multi-table query requiring an explicit projection; - * pass an ordinary schema for nested eager loading and schema-aware writes. - * @param knex - Knex connection or transaction. - * @param schema - Table schema or immutable alias(schema, name). - * @returns The appropriately typed, unexecuted query builder. - */ -export function query< - TLocalSchema extends ObjectSchemaBuilder ->( - knex: Knex, - schema: TLocalSchema -): SchemaQueryBuilder>; - -/** - * Create a schema-aware query while preserving its schema and inferred row type. - * Pass a table alias for a flat multi-table query requiring an explicit projection; - * pass an ordinary schema for nested eager loading and schema-aware writes. - * @param knex - Knex connection or transaction. - * @param schema - Table schema or immutable alias(schema, name). - * @returns The appropriately typed, unexecuted query builder. - */ -export function query< - TLocalSchema extends ObjectSchemaBuilder ->( - knex: Knex, - schema: TLocalSchema, - baseQuery: Knex.QueryBuilder -): SchemaQueryBuilder>; +import { ObjectSchemaBuilder as ObjectSchemaBuilderValue } from '@cleverbrush/schema'; -/** - * Create a schema-aware query while preserving its schema and inferred row type. - * Pass a table alias for a flat multi-table query requiring an explicit projection; - * pass an ordinary schema for nested eager loading and schema-aware writes. - * @param knex - Knex connection or transaction. - * @param schema - Table schema or immutable alias(schema, name). - * @returns The appropriately typed, unexecuted query builder. - */ -export function query< - S extends ObjectSchemaBuilder, - N extends string ->( +/** @internal Shared reader factory for roots and nested relations. */ +export function createReadQuery( knex: Knex, - schema: S | TableAlias, - baseQuery?: Knex.QueryBuilder -): - | SchemaQueryBuilder> - | AliasedQueryBuilder> { - if (isTableAlias(schema)) - return new AliasedQueryBuilder>(knex, schema); - return new SchemaQueryBuilder>( - knex, - schema, - baseQuery - ); -} - -// --------------------------------------------------------------------------- -// createQuery() — knex-bound factory -// --------------------------------------------------------------------------- - -/** - * A query factory bound to a connection or transaction. - * Ordinary schemas retain schema/result inference; aliases retain the table context. - * Use withTransaction() to reuse a transaction or transaction() to create one. - */ -export interface BoundQuery { - /** - * Start a typed query on the bound connection using a schema or table alias. - * Only ordinary schema calls accept an existing Knex base query. - */ - < - S extends ObjectSchemaBuilder, - N extends string - >( - schema: TableAlias - ): AliasedQueryBuilder>; - /** - * Start a typed query on the bound connection using a schema or table alias. - * Only ordinary schema calls accept an existing Knex base query. - */ - < - TLocalSchema extends ObjectSchemaBuilder< - any, - any, - any, - any, - any, - any, - any - > - >( - schema: TLocalSchema - ): SchemaQueryBuilder>; - /** - * Start a typed query on the bound connection using a schema or table alias. - * Only ordinary schema calls accept an existing Knex base query. - */ - < - TLocalSchema extends ObjectSchemaBuilder< - any, - any, - any, - any, - any, - any, - any - > - >( - schema: TLocalSchema, - baseQuery: Knex.QueryBuilder - ): SchemaQueryBuilder>; - /** - * Create a factory bound to an existing transaction without committing it. - */ - withTransaction(trx: Knex.Transaction): BoundQuery; - /** - * Run a callback with a transaction-bound factory. - * Resolves to the callback result on commit; rejects and rolls back on failure. - */ - transaction(callback: (db: BoundQuery) => Promise): Promise; -} - -/** - * Bind query() to a connection, retaining all schema/alias overloads. - * @param knexInstance - Connection or existing transaction to bind. - * @returns A callable factory with transaction helpers. - * @example - * const db = createQuery(knex); - * const rows = await db(TaskSchema).select(t => ({ id: t.id })); - */ -export function createQuery(knexInstance: Knex): BoundQuery { - function boundQuery< - S extends ObjectSchemaBuilder, - N extends string - >(schema: TableAlias): AliasedQueryBuilder>; - function boundQuery< - TLocalSchema extends ObjectSchemaBuilder< - any, - any, - any, - any, - any, - any, - any - > - >( - schema: TLocalSchema, - baseQuery?: Knex.QueryBuilder - ): SchemaQueryBuilder>; - function boundQuery< - S extends ObjectSchemaBuilder, - N extends string - >( - schema: S | TableAlias, - baseQuery?: Knex.QueryBuilder - ): - | SchemaQueryBuilder> - | AliasedQueryBuilder> { - if (isTableAlias(schema)) return query(knexInstance, schema); - return baseQuery - ? query(knexInstance, schema, baseQuery) - : query(knexInstance, schema); - } - - return Object.assign(boundQuery, { - withTransaction(trx: Knex.Transaction): BoundQuery { - return createQuery(trx); - }, - transaction(callback: (db: BoundQuery) => Promise): Promise { - return knexInstance.transaction(trx => callback(createQuery(trx))); - } - }) satisfies BoundQuery; + schema: S, + base: Knex.QueryBuilder +): SchemaAwareQuery { + return ( + getVariants(schema) + ? new PolymorphicQueryBuilder(knex, schema, base) + : new SchemaQueryBuilder(knex, schema, base) + ) as SchemaAwareQuery; } diff --git a/libs/knex-schema/src/SchemaReadQuery.ts b/libs/knex-schema/src/SchemaReadQuery.ts deleted file mode 100644 index c980565b..00000000 --- a/libs/knex-schema/src/SchemaReadQuery.ts +++ /dev/null @@ -1,885 +0,0 @@ -import { - type ArraySchemaBuilder, - array, - EXTRA_TYPE_BRAND, - type InferType, - type ObjectSchemaBuilder, - object, - SYMBOL_SCHEMA_PROPERTY_DESCRIPTOR -} from '@cleverbrush/schema'; -import type { Knex } from 'knex'; -import { - buildColumnMap, - getPrimaryKeyColumns, - resolvePropertyKey -} from './columns.js'; -import type { RelationInfo, SchemaProps } from './entity.js'; -import { - type AggregateExpression, - type AliasedColumn, - COLUMN, - isAggregate -} from './expressions.js'; -import { getProjections, getVariants } from './extension.js'; -import { - type CompositeCursorOptions, - compositeCursor -} from './operations/composite-cursor.js'; -import { - ALLOWED_OPS, - getEffectiveBaseQuery, - getSchemaQueryBuilderCtor -} from './operations/helpers.js'; -import { getState } from './operations/state.js'; -import { PolymorphicReadQuery } from './PolymorphicReadQuery.js'; -import type { ReadRelations, ReadVariantMetadata } from './read-entity.js'; -import { compileReadProjection, type ReadField } from './read-projection.js'; -import { - type ColumnReadSchema, - compileReadSchema, - decodeObject, - type ObjectReadSchema, - type ReadNode, - type ReadObject, - type ReadSchema, - ReadSchemaError, - readExpression, - type SchemaForValue -} from './read-schema.js'; -import type { SchemaQueryBuilder } from './SchemaQueryBuilder.js'; -import type { - CursorPaginationResult, - JoinManySpec, - JoinOneSpec, - PaginationResult, - RelationSpec -} from './types.js'; - -const READ_COLUMN = Symbol('schema-read-column'); -/** @internal Nominal read-query identity used without recursively comparing fluent APIs. */ -export const READ_QUERY = Symbol('schema-read-query'); -/** Minimal type surface for a customizer's strongly inferred result schema. */ -export interface ReadQueryShape { - /** @internal Nominal marker; constructing a plain schema object is not a query. */ - readonly [READ_QUERY]: true; - /** Decoded result schema inferred from the returned query. */ - readonly rowSchema: R; -} -let readAliasSequence = 0; -/** A typed SQL column; selectors receive descriptions, not row values. */ -export interface ReadColumn - extends AliasedColumn> { - /** @internal Decoding and projection metadata shared by the query compiler. */ - readonly [READ_COLUMN]: ReadNode; -} -/** Columns available for typed projections and filters. */ -export type ReadColumns< - S extends ReadObject, - Relations extends PropertyKey = never -> = { - [K in Exclude, Relations> & string]: ReadColumn< - ColumnReadSchema[K]> - >; -}; -type Selection = Record | AggregateExpression>; -type MergeProps = { - [K in keyof A | keyof B]: K extends keyof B - ? B[K] - : K extends keyof A - ? A[K] - : never; -}; -type NamedProjections = S extends { readonly [EXTRA_TYPE_BRAND]?: infer P } - ? P - : {}; -type NamedKeys< - S, - K extends keyof NamedProjections -> = NamedProjections[K] extends readonly (infer Key extends string)[] - ? Key - : never; -/** Structural schema inferred from a typed projection. */ -export type ReadProjection = ObjectSchemaBuilder<{ - [K in keyof S & string]: S[K] extends ReadColumn - ? R - : S[K] extends AggregateExpression - ? SchemaForValue - : never; -}>; -type AddField< - S extends ReadObject, - K extends string, - F extends ReadSchema -> = ObjectSchemaBuilder, K> & Record>; -type Related = R extends RelationInfo ? S : never; -type RelationField = - R extends RelationInfo<'hasMany' | 'belongsToMany', any> - ? ArraySchemaBuilder - : R extends RelationInfo<'belongsTo', any> - ? R extends { optional: infer O } - ? true extends O - ? SchemaForValue | null> - : S - : S - : SchemaForValue | null>; -type Selector = (columns: C) => ReadColumn; -type AnyReadQuery = - | SchemaReadQuery - | PolymorphicReadQuery; -type Loaded = { - name: string; - query: AnyReadQuery; - relation: RelationSpec; - required: boolean; -}; -/** Result of entering opt-in read mode: an object reader or a declared variant union. */ -export type SchemaAwareQuery = - ReadVariantMetadata extends { - discriminator: string; - variants: Record; - } - ? PolymorphicReadQuery - : SchemaReadQuery; -/** @internal Apply parent correlation before child selection, ordering and pagination. */ -export type ReadCorrelation = ( - query: Knex.QueryBuilder, - alias: string, - source: ReadObject -) => void; - -/** - * Immutable, detached read query whose row schema matches its decoded SQL result. - * Enter through withRowSchema() before projections/includes. Ordinary legacy queries - * remain mutable and keep their existing values. Reading metadata never executes SQL. - */ -export class SchemaReadQuery< - S extends ReadObject, - Row extends ReadObject = ObjectReadSchema>, - Relations extends Record = ReadRelations -> { - /** @internal Nominal identity for typed child-query customizers. */ - declare readonly [READ_QUERY]: true; - private readonly alias = `__schema_read_${readAliasSequence++}`; - private fields: Record; - private loaded: Loaded[] = []; - private base: Knex.QueryBuilder; - private selected = false; - private grouped = false; - private readonly columns: Record>; - /** Runtime structural schema; stable across filters, pagination and transaction clones. */ - readonly rowSchema: Row; - - /** @internal Use withRowSchema() on a query/DbSet instead of constructing directly. */ - constructor( - private readonly knex: Knex, - private readonly source: S, - base: Knex.QueryBuilder - ) { - this.base = knex.queryBuilder().from(base.clone().as(this.alias)); - const relations = (source.introspect().extensions?.relations ?? - []) as RelationSpec[]; - const excluded = new Set(relations.map(r => r.name)); - const { propToCol } = buildColumnMap(source); - this.fields = Object.create(null); - this.columns = Object.create(null); - for (const [key, schema] of Object.entries( - source.introspect().properties ?? {} - )) { - if (excluded.has(key)) continue; - const node = compileReadSchema(schema as ReadSchema); - const column = `${this.alias}.${propToCol.get(key) ?? key}`; - this.columns[key] = { - [COLUMN]: { - alias: this.alias, - column: propToCol.get(key) ?? key, - schema - }, - [READ_COLUMN]: node - }; - this.fields[key] = { - node, - expression: knex => readExpression(knex, node, column) - }; - } - this.rowSchema = this.schema() as Row; - } - - private schema(): ReadObject { - return object( - Object.fromEntries( - Object.entries(this.fields).map(([key, field]) => [ - key, - field.node.schema - ]) - ) - ); - } - - private copy(): this { - const copy = Object.create(Object.getPrototypeOf(this)) as this; - Object.assign(copy, this, { - base: this.base.clone(), - fields: { ...this.fields }, - loaded: [...this.loaded] - }); - return copy; - } - - /** @internal Prevent customizers from substituting an unrelated query source. */ - sameSource(other: unknown): boolean { - return ( - other instanceof SchemaReadQuery && this.columns === other.columns - ); - } - - private column( - selector: Selector> - ): ReadColumn { - const column = selector( - this.columns as ReadColumns - ); - if (!column || !Object.values(this.columns).includes(column)) - throw new ReadSchemaError( - 'Column does not belong to this read query' - ); - return column; - } - - private name(column: ReadColumn): string { - return `${column[COLUMN].alias}.${column[COLUMN].column}`; - } - - /** Select an exact flat row shape, retaining per-field runtime schemas. */ - select

( - selector: (columns: ReadColumns) => P - ): SchemaReadQuery< - S, - ObjectSchemaBuilder< - MergeProps< - SchemaProps>, - Pick< - SchemaProps, - Extract, keyof Relations> - > - > - >, - Relations - > { - if (this.selected) - throw new ReadSchemaError( - 'Only one projection is allowed per read query' - ); - const selection = selector( - this.columns as ReadColumns - ); - if (this.loaded.length && Object.values(selection).some(isAggregate)) - throw new ReadSchemaError( - 'Aggregate projections cannot contain relations' - ); - const copy = this.copy(); - copy.fields = compileReadProjection( - this.knex, - selection, - expression => { - if ( - !Object.values(this.columns).includes( - expression as ReadColumn - ) - ) - throw new ReadSchemaError( - 'Projection column does not belong to this query' - ); - const column = expression as ReadColumn; - return { node: column[READ_COLUMN], name: this.name(column) }; - } - ); - copy.selected = true; - // Included relations are independent of the scalar projection. - for (const relation of copy.loaded) - copy.fields[relation.name] = this.fields[relation.name]; - Object.assign(copy, { rowSchema: copy.schema() }); - return copy as any; - } - - /** Apply a named schema projection with the same exact row-schema guarantees. */ - projected & string>( - name: K - ): SchemaReadQuery< - S, - ObjectSchemaBuilder< - Pick< - SchemaProps>, - Extract, keyof SchemaProps>> - > & - Pick< - SchemaProps, - Extract, keyof Relations> - > - >, - Relations - > { - const definition = getProjections(this.source)[name]; - if (!definition) - throw new ReadSchemaError(`Unknown projection: ${name}`); - return this.select(() => - Object.fromEntries( - definition.keys.map(key => [key, this.columns[key]]) - ) - ) as any; - } - - /** Add a bound comparison, returning an independent query with the same row schema. */ - where( - column: Selector>, - value: unknown - ): this; - /** Add a bound comparison using a supported SQL operator. */ - where( - column: Selector>, - operator: string, - value: unknown - ): this; - /** Add a bound comparison using a supported SQL operator. */ - where( - column: Selector>, - ...args: [unknown] | [string, unknown] - ): this { - const operator = args.length === 1 ? '=' : args[0].toLowerCase(); - if (!ALLOWED_OPS.has(operator)) - throw new ReadSchemaError( - `Unsupported comparison operator: ${operator}` - ); - const copy = this.copy(); - copy.base.where( - this.name(this.column(column)), - operator, - (args.length === 1 ? args[0] : args[1]) as any - ); - return copy; - } - - /** Filter SQL null values without changing the declared read shape. */ - whereNull(column: Selector>): this { - const copy = this.copy(); - copy.base.whereNull(this.name(this.column(column))); - return copy; - } - /** Exclude SQL null values without implicitly narrowing schema nullability. */ - whereNotNull(column: Selector>): this { - const copy = this.copy(); - copy.base.whereNotNull(this.name(this.column(column))); - return copy; - } - /** Filter against a bound list of values. An empty list produces no rows. */ - whereIn( - column: Selector>, - values: readonly unknown[] - ): this { - const copy = this.copy(); - copy.base.whereIn(this.name(this.column(column)), [...values] as any[]); - return copy; - } - /** Order parent rows independently of any child relation's ordering. */ - orderBy( - column: Selector>, - direction: 'asc' | 'desc' = 'asc' - ): this { - const copy = this.copy(); - copy.base.orderBy(this.name(this.column(column)), direction); - return copy; - } - /** Group rows before typed aggregate projection. */ - groupBy(...columns: Selector>[]): this { - const copy = this.copy(); - copy.base.groupBy(columns.map(c => this.name(this.column(c)))); - copy.grouped = true; - return copy; - } - /** Limit parent rows; relation limits apply independently within each parent. */ - limit(count: number): this { - if (!Number.isInteger(count) || count < 0) - throw new ReadSchemaError('Limit must be a non-negative integer'); - const copy = this.copy(); - copy.base.limit(count); - return copy; - } - /** Skip parent rows; use a deterministic order for pagination. */ - offset(count: number): this { - if (!Number.isInteger(count) || count < 0) - throw new ReadSchemaError('Offset must be a non-negative integer'); - const copy = this.copy(); - copy.base.offset(count); - return copy; - } - - /** - * Load a declared relation in the same SQL statement. Return the customized child - * query so its selected fields and nested includes remain strongly typed. - */ - include< - K extends keyof Relations & string, - Child extends ReadQueryShape = SchemaAwareQuery> - >( - selector: (relations: { [P in keyof Relations]: P }) => K, - customize?: (query: SchemaAwareQuery>) => Child - ): SchemaReadQuery< - S, - AddField>, - Relations - > { - if (Object.values(this.fields).some(f => f.aggregate) || this.grouped) - throw new ReadSchemaError( - 'Grouped/aggregate reads cannot load entity relations' - ); - const definitions = (this.source.introspect().extensions?.relations ?? - []) as RelationSpec[]; - const key = selector( - Object.fromEntries(definitions.map(r => [r.name, r.name])) as any - ); - const relation = definitions.find(r => r.name === key); - if (!relation) throw new ReadSchemaError(`Unknown relation: ${key}`); - if (this.loaded.some(r => r.name === key)) - throw new ReadSchemaError(`Duplicate relation: ${key}`); - const foreign = - typeof relation.schema === 'function' - ? relation.schema() - : relation.schema; - return this.load( - relation, - this.child(foreign, customize as any), - relation.type === 'belongsTo' && !relation.optional - ) as any; - } - - private child( - foreign: ReadObject, - customize?: (query: any) => ReadQueryShape - ): AnyReadQuery { - const Constructor = getSchemaQueryBuilderCtor(); - let child: AnyReadQuery = createReadQuery( - this.knex, - foreign, - getEffectiveBaseQuery(new Constructor(this.knex, foreign)).clone() - ); - if (customize) { - const customized = customize(child as any); - if (!child.sameSource(customized)) - throw new ReadSchemaError( - 'Relation customizer must return its configured read query' - ); - child = customized as any; - } - return child; - } - - private load( - relation: RelationSpec, - child: AnyReadQuery, - required: boolean - ): this { - const key = relation.name; - if (this.grouped || Object.values(this.fields).some(f => f.aggregate)) - throw new ReadSchemaError( - 'Grouped/aggregate reads cannot load entity relations' - ); - if ( - this.loaded.some(r => r.name === key) || - Object.hasOwn(this.fields, key) - ) - throw new ReadSchemaError(`Duplicate result field: ${key}`); - const many = - relation.type === 'hasMany' || relation.type === 'belongsToMany'; - const schema = many - ? array(child.rowSchema) - : required - ? child.rowSchema - : child.rowSchema.nullable(); - const node: ReadNode = { - schema, - exact: false, - decode: (value, path) => { - if (value === null && !required && !many) return null; - if (many) { - if (!Array.isArray(value)) - throw new ReadSchemaError( - `${path}: expected a relation array` - ); - return value.map((row, i) => - child.decode(row, `${path}[${i}]`) - ); - } - return child.decode(value, path); - } - }; - const copy = this.copy(); - copy.loaded.push({ name: key, query: child, relation, required }); - copy.fields[key] = { - node, - expression: () => { - throw new ReadSchemaError( - 'Relation expression must be compiled in context' - ); - } - }; - Object.assign(copy, { rowSchema: copy.schema() }); - return copy as any; - } - - /** Join one typed nested object with explicit property keys, including nullable joins. */ - joinOne< - F extends ReadObject, - K extends string, - Required extends boolean = true, - Child extends ReadQueryShape = SchemaAwareQuery - >( - spec: JoinOneSpec, - customize?: (query: SchemaAwareQuery) => Child - ): SchemaReadQuery< - S, - AddField< - Row, - K, - Required extends true - ? Child['rowSchema'] - : SchemaForValue | null> - >, - Relations & Record> - > { - if (spec.foreignQuery || spec.mappers) - throw new ReadSchemaError( - 'Use the typed child customizer instead of foreignQuery/mappers in schema-aware mode' - ); - return this.load( - { - name: spec.as, - type: 'hasOne', - schema: spec.foreignSchema, - localKey: resolvePropertyKey( - spec.localColumn as any, - this.source, - 'joinOne' - ), - remoteKey: resolvePropertyKey( - spec.foreignColumn as any, - spec.foreignSchema, - 'joinOne' - ) - }, - this.child(spec.foreignSchema, customize as any), - spec.required !== false - ) as any; - } - - /** Join a typed collection, applying child projection and pagination independently per parent. */ - joinMany< - F extends ReadObject, - K extends string, - Child extends ReadQueryShape = SchemaAwareQuery - >( - spec: Omit, 'orderBy'>, - customize?: (query: SchemaAwareQuery) => Child - ): SchemaReadQuery< - S, - AddField>, - Relations & Record> - > { - if (spec.foreignQuery || spec.mappers || 'orderBy' in spec) - throw new ReadSchemaError( - 'Use the typed child customizer for ordering instead of raw foreignQuery/mappers' - ); - let child = this.child(spec.foreignSchema, customize as any); - if (spec.limit !== undefined) child = child.limit(spec.limit); - if (spec.offset !== undefined) child = child.offset(spec.offset); - return this.load( - { - name: spec.as, - type: 'hasMany', - schema: spec.foreignSchema, - localKey: resolvePropertyKey( - spec.localColumn as any, - this.source, - 'joinMany' - ), - remoteKey: resolvePropertyKey( - spec.foreignColumn as any, - spec.foreignSchema, - 'joinMany' - ) - }, - child, - false - ) as any; - } - - /** @internal Decode a SQL/JSON row using the same metadata exposed to consumers. */ - decode(row: any, path = 'row'): any { - if ( - this.source.introspect().extensions?.readOrphanColumn && - row.__read_cti_present == null - ) - throw new ReadSchemaError(`${path}: missing CTI variant body`); - return decodeObject( - Object.fromEntries( - Object.entries(this.fields).map(([key, field]) => [ - key, - field.node - ]) - ), - row, - path - ); - } - - /** @internal Compile a bound SQL statement; callers never receive the mutable builder. */ - compile(correlate?: ReadCorrelation): Knex.QueryBuilder { - const query = this.base.clone().clearSelect(); - correlate?.(query, this.alias, this.source); - const expressions: Record = Object.create(null); - for (const [key, field] of Object.entries(this.fields)) { - if (!this.loaded.some(r => r.name === key)) - expressions[key] = field.expression(this.knex); - } - const orphanColumn = - this.source.introspect().extensions?.readOrphanColumn; - if (orphanColumn) - expressions.__read_cti_present = this.knex.raw('??', [ - `${this.alias}.${orphanColumn}` - ]); - for (const loaded of this.loaded) { - const { relation, query: child } = loaded; - const parentTable = this.alias; - const parentPk = getPrimaryKeyColumns(this.source).columnNames; - const foreignKey = relation.foreignKey; - const resolveKey = (schema: ReadObject, key: any) => { - if (typeof key === 'function') { - const descriptor = key( - ObjectSchemaBuilderValue.getPropertiesFor(schema) - ); - key = - descriptor[SYMBOL_SCHEMA_PROPERTY_DESCRIPTOR] - .propertyName; - } - return buildColumnMap(schema).propToCol.get(key) ?? key; - }; - const childSql = child.compile((sql, childTable, childSource) => { - const childPk = getPrimaryKeyColumns(childSource).columnNames; - if (relation.localKey && relation.remoteKey) { - sql.where( - `${childTable}.${resolveKey(childSource, relation.remoteKey)}`, - this.knex.ref( - `${parentTable}.${resolveKey(this.source, relation.localKey)}` - ) - ); - return; - } - if (parentPk.length !== 1) - throw new ReadSchemaError( - 'Automatic relation reads require single-column primary keys' - ); - if (childPk.length !== 1) - throw new ReadSchemaError( - 'Automatic relation reads require single-column primary keys' - ); - if (relation.type === 'belongsTo') - sql.where( - `${childTable}.${childPk[0]}`, - this.knex.ref( - `${parentTable}.${resolveKey(this.source, foreignKey)}` - ) - ); - else if (relation.type === 'belongsToMany') { - const through = relation.through!; - sql.join( - through.table, - `${through.table}.${through.foreignKey}`, - `${childTable}.${childPk[0]}` - ).where( - `${through.table}.${through.localKey}`, - this.knex.ref(`${parentTable}.${parentPk[0]}`) - ); - } else - sql.where( - `${childTable}.${resolveKey(childSource, foreignKey)}`, - this.knex.ref(`${parentTable}.${parentPk[0]}`) - ); - }); - const many = - relation.type === 'hasMany' || - relation.type === 'belongsToMany'; - if (!many) childSql.limit(1); - const wrapped = this.knex - .queryBuilder() - .from(childSql.clone().as('__read_relation')); - expressions[loaded.name] = many - ? this.knex.raw( - "(select coalesce(jsonb_agg(to_jsonb(__read_relation)), '[]'::jsonb) from (?) as __read_relation)", - [childSql] - ) - : this.knex.raw( - '(select to_jsonb(__read_relation) from (?) as __read_relation)', - [childSql] - ); - if (loaded.required) - query.whereExists(wrapped.clone().select(this.knex.raw('1'))); - } - return query.select(expressions); - } - - /** Render debugging SQL without execution; bound values may be sensitive. */ - toQuery(): string { - return this.compile().toQuery(); - } - /** Execute one statement and decode its selected row graph. */ - async execute(): Promise[]> { - return (await this.compile()).map((row: unknown) => this.decode(row)); - } - /** Execute a limited copy, returning undefined when no row matches. */ - async first(): Promise | undefined> { - return (await this.limit(1).execute())[0]; - } - /** Awaiting executes the query; repeated awaits deliberately execute again. */ - // biome-ignore lint/suspicious/noThenProperty: query readers intentionally support await - then[], E = never>( - resolve?: ((rows: InferType[]) => T | PromiseLike) | null, - reject?: ((error: any) => E | PromiseLike) | null - ): Promise { - return this.execute().then(resolve, reject); - } - /** Bind an independent query graph to a caller-owned transaction. */ - transacting(trx: Knex.Transaction): this { - const copy = this.copy(); - copy.base.transacting(trx); - Object.assign(copy, { knex: trx }); - copy.loaded = this.loaded.map(r => ({ - ...r, - query: r.query.transacting(trx) - })); - return copy; - } - /** - * Read a lossless composite cursor page using native SQL ordering. Cursor sort - * values remain private text columns, independent of projections and Date decoding. - * Requires non-null scalar order columns containing a declared unique key. - */ - async paginateAfter( - options: CompositeCursorOptions - ): Promise>> { - if (this.grouped || Object.values(this.fields).some(f => f.aggregate)) - throw new ReadSchemaError( - 'Cursor pagination cannot be used for aggregate reads' - ); - const Constructor = getSchemaQueryBuilderCtor(); - const source = this.source.withExtension('tableName', this.alias); - const legacy = new Constructor(this.knex, source, this.compile()); - const state = getState(legacy); - state.skipDefaultScope = true; - state.includeDeleted = true; - // The query is already projected; reserve its aliases against cursor fields. - state.hiddenColumns = new Set(Object.keys(this.fields)); - return compositeCursor( - legacy, - options, - row => this.decode(row), - this.source.introspect().extensions?.tableName as string - ); - } - /** Fetch a numbered page and a count without mutating the source query. */ - async paginate(options: { - page: number; - pageSize: number; - }): Promise>> { - const { page, pageSize } = options; - if ( - !Number.isInteger(page) || - page < 1 || - !Number.isInteger(pageSize) || - pageSize < 1 - ) - throw new ReadSchemaError( - 'Page and pageSize must be positive integers' - ); - const countQuery = this.compile() - .clearOrder() - .clear('limit') - .clear('offset'); - const countRow = await this.knex - .from(countQuery.as('__read_count')) - .count({ count: '*' }) - .first(); - const total = Number(countRow?.count ?? 0); - if (!Number.isSafeInteger(total)) - throw new ReadSchemaError( - 'Pagination count exceeds the safe integer range' - ); - const data = await this.offset((page - 1) * pageSize) - .limit(pageSize) - .execute(); - const totalPages = Math.ceil(total / pageSize); - return { - data, - total, - page, - pageSize, - totalPages, - hasNextPage: page < totalPages, - hasPreviousPage: page > 1 - }; - } -} - -import { ObjectSchemaBuilder as ObjectSchemaBuilderValue } from '@cleverbrush/schema'; - -/** @internal Enter schema-aware mode only before result-shaping legacy operations. */ -export function schemaReadQuery( - builder: SchemaQueryBuilder -): SchemaAwareQuery { - const state = getState(builder); - if ( - state.selectionMode !== null || - state.specs.length || - state.variantRelationIncludes.length - ) - throw new ReadSchemaError( - 'Call withRowSchema() before select/include/join operations' - ); - if ( - state.opaqueReadShape || - state.enabledVariants || - state.variantWhereFilters.length - ) - throw new ReadSchemaError( - 'Call withRowSchema() before raw or variant-specific operations' - ); - const base = getEffectiveBaseQuery(builder).clone(); - const statements = (base as any)._statements as Array<{ grouping: string }>; - if ( - statements.some(s => s.grouping === 'order') || - (base as any)._single.limit !== undefined || - (base as any)._single.offset !== undefined - ) - throw new ReadSchemaError( - 'Call withRowSchema() before ordering or pagination' - ); - if ( - statements.some(s => - ['columns', 'join', 'group', 'having', 'union'].includes(s.grouping) - ) - ) - throw new ReadSchemaError( - 'Existing raw projections, joins and aggregates cannot declare a read schema' - ); - return createReadQuery(state.knex, state.localSchema as S, base); -} - -/** @internal Shared reader factory for roots and nested relations. */ -export function createReadQuery( - knex: Knex, - schema: S, - base: Knex.QueryBuilder -): SchemaAwareQuery { - return ( - getVariants(schema) - ? new PolymorphicReadQuery(knex, schema, base) - : new SchemaReadQuery(knex, schema, base) - ) as SchemaAwareQuery; -} diff --git a/libs/knex-schema/src/aliased-query.ts b/libs/knex-schema/src/aliased-query.ts index 455b16dd..1b98e8c9 100644 --- a/libs/knex-schema/src/aliased-query.ts +++ b/libs/knex-schema/src/aliased-query.ts @@ -1,6 +1,6 @@ import type { InferType, ObjectSchemaBuilder } from '@cleverbrush/schema'; import type { Knex } from 'knex'; -import { AliasedReadQuery } from './AliasedReadQuery.js'; + import { buildColumnMap } from './columns.js'; import type { SchemaProps } from './entity.js'; import { @@ -10,11 +10,9 @@ import { compileAggregate, isAggregate } from './expressions.js'; -import { - ALLOWED_OPS, - getEffectiveBaseQuery, - getSchemaQueryBuilderCtor -} from './operations/helpers.js'; +import { getTableName } from './extension.js'; +import { ALLOWED_OPS } from './operations/helpers.js'; +import { SchemaQueryBuilder } from './SchemaQueryBuilder.js'; import { isSqlIdentifier } from './sql-identifiers.js'; type TableSchema = ObjectSchemaBuilder; @@ -135,8 +133,8 @@ export function or(...items: JoinPredicate[]): JoinPredicate { return { [PREDICATE]: { op: 'or', items } }; } -/** Read-only flat query builder. Explicit projections avoid ambiguous SELECT *. */ -export class AliasedQueryBuilder { +/** @internal Mutable native SQL planner, never returned by a public factory. */ +export class AliasedQuerySource { private sql: Knex.QueryBuilder; private tables = new Map(); private selected = false; @@ -144,17 +142,8 @@ export class AliasedQueryBuilder { private nullableTables = new Set(); private opaqueReadShape = false; - /** Enter immutable read mode before select/raw changes; an explicit projection is required. */ - withRowSchema(): AliasedReadQuery { - if (this.selected || this.opaqueReadShape) - throw new Error( - 'Call withRowSchema() before select/apply operations' - ); - return new AliasedReadQuery(this.cloneReadSource()); - } - /** @internal Snapshot the SQL planner without sharing mutable query state. */ - cloneReadSource(): AliasedQueryBuilder { + cloneReadSource(): AliasedQuerySource { const copy = Object.assign( Object.create(Object.getPrototypeOf(this)), this @@ -166,7 +155,7 @@ export class AliasedQueryBuilder { return copy; } - /** @internal Schema-backed state used by the immutable opt-in adapter. */ + /** @internal Schema-backed state used by the immutable public builder. */ readContext(): { knex: Knex; sql: Knex.QueryBuilder; columns: TTables } { return { knex: this.knex, sql: this.sql.clone(), columns: this.tree() }; } @@ -185,9 +174,12 @@ export class AliasedQueryBuilder { } private source(table: TableAlias): Knex.QueryBuilder { - const Constructor = getSchemaQueryBuilderCtor(); - return getEffectiveBaseQuery(new Constructor(this.knex, table.schema)) - .clone() + return new SchemaQueryBuilder( + this.knex, + table.schema, + this.knex(getTableName(table.schema)) + ) + .storageQuery() .as(table.name); } @@ -257,7 +249,7 @@ export class AliasedQueryBuilder { join( table: N extends keyof TTables ? never : TableAlias, on: (tables: TTables & AliasTables) => JoinPredicate - ): AliasedQueryBuilder, TResult> { + ): AliasedQuerySource, TResult> { this.addJoin(table, on as any, false); return this as any; } @@ -270,7 +262,7 @@ export class AliasedQueryBuilder { leftJoin( table: N extends keyof TTables ? never : TableAlias, on: (tables: TTables & AliasTables) => JoinPredicate - ): AliasedQueryBuilder, TResult> { + ): AliasedQuerySource, TResult> { this.addJoin(table, on as any, true); return this as any; } @@ -421,7 +413,7 @@ export class AliasedQueryBuilder { */ select( selector: (tables: TTables) => S - ): AliasedQueryBuilder> { + ): AliasedQuerySource> { if (this.selected) throw new Error('Only one object projection per query'); const columns: Record = {}; @@ -467,12 +459,9 @@ export class AliasedQueryBuilder { * Clone this builder onto an existing transaction, preserving its tables/projection. * Does not modify the source or commit/roll back the transaction. */ - transacting(trx: Knex.Transaction): AliasedQueryBuilder { - const [name, schema] = this.tables.entries().next().value!; - const copy = new AliasedQueryBuilder( - trx as unknown as Knex, - alias(schema, name) - ); + transacting(trx: Knex.Transaction): AliasedQuerySource { + const copy = this.cloneReadSource(); + Object.assign(copy, { knex: trx }); copy.sql = this.sql.clone().transacting(trx); copy.tables = new Map(this.tables); copy.selected = this.selected; diff --git a/libs/knex-schema/src/composable-query.test-d.ts b/libs/knex-schema/src/composable-query.test-d.ts index a03fa6f4..77dc4b84 100644 --- a/libs/knex-schema/src/composable-query.test-d.ts +++ b/libs/knex-schema/src/composable-query.test-d.ts @@ -35,8 +35,12 @@ test('factories retain schema, alias and result inference through every call sha expectTypeOf(await plain.select(t => ({ id: t.id }))).toEqualTypeOf< { id: number }[] >(); + // @ts-expect-error implicit raw base queries cannot declare an output contract + query(db, Task, db('tasks')); expectTypeOf( - await query(db, Task, db('tasks')).select(t => ({ amount: t.amount })) + await query(db, Task).selectRaw('1 as amount', [], { + output: object({ amount: number() }) + }) ).toEqualTypeOf<{ amount: number }[]>(); for (const factory of [bound, transactional]) { const ordinary = factory(Task); @@ -44,8 +48,12 @@ test('factories retain schema, alias and result inference through every call sha expectTypeOf( await ordinary.select(t => ({ createdAt: t.createdAt })) ).toEqualTypeOf<{ createdAt: Date }[]>(); + // @ts-expect-error bound factories cannot accept an opaque raw base query + factory(Task, db('tasks')); expectTypeOf( - await factory(Task, db('tasks')).select(t => ({ id: t.id })) + await factory(Task).selectRaw('1 as id', [], { + output: object({ id: number() }) + }) ).toEqualTypeOf<{ id: number }[]>(); const aliased = factory(alias(Task, 'task')); expectTypeOf(aliased).not.toBeAny(); @@ -113,10 +121,10 @@ test('aggregate defaults and supplied output schemas infer accurately', async () await query(db, Task).maxValue('createdAt') ).toEqualTypeOf(); expectTypeOf(await query(db, Task).minValue('amount')).toEqualTypeOf< - number | string | null + number | null >(); expectTypeOf(await query(db, Task).minValue(t => t.amount)).toEqualTypeOf< - number | string | null + number | null >(); expectTypeOf( await query(db, Task).countValue({ output: string() }) diff --git a/libs/knex-schema/src/composable-query.test.ts b/libs/knex-schema/src/composable-query.test.ts index b3a3dbed..9a9e2f6e 100644 --- a/libs/knex-schema/src/composable-query.test.ts +++ b/libs/knex-schema/src/composable-query.test.ts @@ -118,8 +118,8 @@ describe('typed aliases and aggregate SQL', () => { count: aggregate.countDistinct(t.id) })) .toQuery(); - expect(sql).toContain('count(distinct "id")'); - expect(sql).toContain('group by "owner_id"'); + expect(sql).toMatch(/count\(distinct "__schema_read_\d+"\."id"\)/); + expect(sql).toMatch(/group by "__schema_read_\d+"\."owner_id"/); }); }); diff --git a/libs/knex-schema/src/entity.ts b/libs/knex-schema/src/entity.ts index fe6ea98e..d226d506 100644 --- a/libs/knex-schema/src/entity.ts +++ b/libs/knex-schema/src/entity.ts @@ -910,7 +910,7 @@ export type EntityVariantUnion = /** * Type-level helper: union of relation key names declared on an entity. - * Used by `SchemaQueryBuilder.insert()/update()/upsert()` to omit relation + * Used by `QuerySource.insert()/update()/upsert()` to omit relation * navigation properties from accepted input. * @public */ diff --git a/libs/knex-schema/src/expressions.ts b/libs/knex-schema/src/expressions.ts index 1b198372..7b19cf45 100644 --- a/libs/knex-schema/src/expressions.ts +++ b/libs/knex-schema/src/expressions.ts @@ -46,7 +46,7 @@ export interface AliasedColumn { * Type-only marker carrying column nullability and value type; not a runtime row value. */ readonly __value?: T; - /** Type-only original schema used by opt-in projection metadata. */ + /** Type-only source schema used by projection metadata. */ readonly __readSource?: S; /** Type-only join nullability, independent of the stored column schema. */ readonly __leftJoined?: Nullable; diff --git a/libs/knex-schema/src/extension.ts b/libs/knex-schema/src/extension.ts index e5875aac..5f75d00d 100644 --- a/libs/knex-schema/src/extension.ts +++ b/libs/knex-schema/src/extension.ts @@ -1,4 +1,5 @@ // @cleverbrush/knex-schema — Schema extension: hasColumnName / hasTableName + import type { AnySchemaBuilder, ArraySchemaBuilder, @@ -28,6 +29,7 @@ import { stringExtensions, withExtensions } from '@cleverbrush/schema'; +import type { QueryScope } from './query-scope.js'; import type { ResolvedVariantConfig, ResolvedVariantRelationSpec, @@ -731,12 +733,15 @@ export const ddlExtension = defineExtension({ }, /** Register a named query scope. * @param name - Scope name to use with `.scoped(name)`. - * @param fn - Function that receives a `SchemaQueryBuilder` and applies filters. + * @param fn - Synchronous callback returning its configured immutable query (filters/order/paging only). */ - scope( - this: ObjectSchemaBuilder, + scope< + N extends string, + S extends ObjectSchemaBuilder + >( + this: S, name: N, - fn: Function + fn: (query: QueryScope) => QueryScope ): typeof this & { readonly [METHOD_LITERAL_BRAND]?: N } { const existing = (this.getExtension('scopes') as Record) ?? {}; @@ -886,12 +891,11 @@ export const ddlExtension = defineExtension({ }; }, /** Set a default scope applied to all queries unless `.unscoped()` is called. - * @param fn - Function that receives a `SchemaQueryBuilder` and applies filters. + * @param fn - Synchronous function that returns its configured immutable query scope. */ - defaultScope( - this: ObjectSchemaBuilder, - fn: Function - ) { + defaultScope< + S extends ObjectSchemaBuilder + >(this: S, fn: (query: QueryScope) => QueryScope) { return this.withExtension('defaultScope', fn); }, /** Register a before-insert lifecycle hook. @@ -928,7 +932,7 @@ export const ddlExtension = defineExtension({ return this.withExtension('beforeUpdate', [...existing, fn]); }, /** Register a before-delete lifecycle hook. - * @param fn - Async function `(query)` called before deleting. + * @param fn - Observational async function `(query)` called before deleting; query configuration is immutable. Apply delete filters before calling delete(). */ beforeDelete( this: ObjectSchemaBuilder, @@ -939,72 +943,8 @@ export const ddlExtension = defineExtension({ return this.withExtension('beforeDelete', [...existing, fn]); } - /** - * Declare polymorphic variants for this schema. - * - * Turns a base schema into a **polymorphic schema** where a discriminator - * column determines which variant each row belongs to. Variants can store - * their extra fields either in a separate table (CTI — Class Table - * Inheritance) or as nullable columns on the base table (STI — Single - * Table Inheritance). - * - * The return type carries a phantom brand - * (`[POLYMORPHIC_TYPE_BRAND]`) so that `query(db, schema)` automatically - * infers the full discriminated-union result type. - * - * @param config.discriminator - Property key (or accessor) of the - * discriminator column on the base table (e.g. `'type'` or `t => t.type`). - * @param config.variants - Map from discriminator value to - * `{ schema, storage, foreignKey?, allowOrphan?, enforceCheck? }`. - * - `storage: 'cti'` — variant fields are in a separate table; - * `foreignKey` (the FK column on the variant table) is required. - * - `storage: 'sti'` — variant fields are nullable columns on the base table. - * - * @example - * ```ts - * const FileBase = object({ id: number().primaryKey(), name: string(), type: string() }) - * .hasTableName('files'); - * - * const ImageExtras = object({ width: number(), height: number(), format: string() }) - * .hasTableName('image_file'); - * - * const DocumentExtras = object({ size: number(), issueDate: date() }) - * .hasTableName('document_file'); - * - * const ImageExtras = object({ - * fileId: number().hasColumnName('file_id'), - * type: string('image'), - * width: number(), height: number(), format: string() - * }).hasTableName('image_file'); - * - * const DocumentExtras = object({ - * fileId: number().hasColumnName('file_id'), - * type: string('document'), - * size: number(), issueDate: date() - * }).hasTableName('document_file'); - * - * const FileSchema = FileBase.withVariants({ - * discriminator: t => t.type, - * variants: { - * image: { schema: ImageExtras, storage: 'cti', foreignKey: t => t.fileId }, - * document: { schema: DocumentExtras, storage: 'cti', foreignKey: t => t.fileId }, - * }, - * }); - * - * // query(db, FileSchema) returns: - * // Array< - * // | { id: number; name: string; type: 'image'; width: number; height: number; format: string } - * // | { id: number; name: string; type: 'document'; size: number; issueDate: Date } - * // > - * ``` - */ - // NOTE: the public `.withVariants()` schema-level method has been - // removed. Variants are now declared on the {@link Entity} chain via - // `defineEntity(...).discriminator(...).ctiVariant(...).stiVariant(...)`. - // The internal worker {@link applyVariantsToSchema} (below this - // `defineExtension` block) is invoked by the Entity layer and stores - // the same `'variants'` / `'polymorphicVariants'` extensions that - // `SchemaQueryBuilder` reads at runtime. + // Entity declarations use applyVariantsToSchema to store the variant + // metadata consumed by polymorphic queries. } }); @@ -1035,7 +975,7 @@ export interface VariantInputForResolver { /** * @internal Validate + apply a fully-resolved variant config to a base * schema. Stores the `'variants'` and `'polymorphicVariants'` extensions - * read by {@link SchemaQueryBuilder}. + * read by {@link QuerySource}. * * Called by the {@link Entity} chain (`.discriminator().ctiVariant().stiVariant()`). * Replaces the previous schema-level `.withVariants()` method. @@ -1333,7 +1273,7 @@ export function getProjections( * Retrieve the resolved variant configuration stored by `.withVariants()`. * Returns `null` when the schema is not polymorphic. * - * @internal — used by {@link SchemaQueryBuilder}. + * @internal — used by {@link QuerySource}. */ export function getVariants( schema: ObjectSchemaBuilder diff --git a/libs/knex-schema/src/immutable-query.test-d.ts b/libs/knex-schema/src/immutable-query.test-d.ts new file mode 100644 index 00000000..014aba57 --- /dev/null +++ b/libs/knex-schema/src/immutable-query.test-d.ts @@ -0,0 +1,60 @@ +import type { InferType } from '@cleverbrush/schema'; +import Knex from 'knex'; +import { expectTypeOf, test } from 'vitest'; +import { number, object, string } from './extension.js'; +import { query } from './query.js'; + +const db = Knex({ client: 'pg' }); +const User = object({ + id: number().bigint().primaryKey(), + name: string(), + settings: object({ enabled: number() }) +}).hasTableName('users'); + +test('automatic projections preserve exact structural row types', async () => { + const users = query(db, User); + const selected = users.select(u => ({ userId: u.id, displayName: u.name })); + type Row = InferType; + expectTypeOf().toEqualTypeOf<{ + userId: string; + displayName: string; + }>(); + expectTypeOf(await selected.first()).toEqualTypeOf(); + const ids = users.select(u => u.id); + expectTypeOf>().toEqualTypeOf<{ + id: string; + }>(); + const nested = users.select(u => ({ enabled: u.settings.enabled })); + expectTypeOf>().toEqualTypeOf<{ + enabled: number; + }>(); + // @ts-expect-error projected queries cannot update entities + selected.update({ name: 'unsafe' }); + // @ts-expect-error projected queries cannot insert entities + selected.insert({ name: 'unsafe' }); + // @ts-expect-error grouped predicate callbacks must return their configured builder + users.where(group => { + group.where('name', 'discarded'); + }); + // @ts-expect-error raw SQL cannot infer its output shape + users.apply(sql => sql.select('*')); + // @ts-expect-error removed opt-in method has no compatibility alias + users.withRowSchema(); + // Lossless storage representations are valid update values. + users.update({ id: '9007199254740993' }); + // @ts-expect-error grouped sources cannot write entities + users.groupBy(u => u.name).update({ name: 'unsafe' }); + // @ts-expect-error distinct sources cannot write entities + users.distinct().delete(); + // @ts-expect-error HAVING queries are grouped read-only sources + users.havingRaw('count(*) > 0').delete(); + User.scope('named', scope => { + // @ts-expect-error scopes cannot change projections + scope.where('name', 'x').select('id'); + return scope.where('name', 'x'); + }); + // @ts-expect-error scope callbacks cannot discard their immutable result + User.defaultScope(scope => { + scope.where('name', 'x'); + }); +}); diff --git a/libs/knex-schema/src/immutable-query.test.ts b/libs/knex-schema/src/immutable-query.test.ts new file mode 100644 index 00000000..bb2cee50 --- /dev/null +++ b/libs/knex-schema/src/immutable-query.test.ts @@ -0,0 +1,163 @@ +import { + number as outputNumber, + object as outputObject +} from '@cleverbrush/schema'; +import Knex from 'knex'; +import { describe, expect, it, vi } from 'vitest'; +import { defineEntity } from './entity.js'; +import { number, object, string } from './extension.js'; +import { createQuery, query } from './query.js'; + +const knex = Knex({ client: 'pg' }); +const Account = object({ + id: number().primaryKey(), + name: string(), + active: number() +}).hasTableName('accounts'); + +describe('immutable public queries', () => { + it('owns polymorphic ordering columns and rejects visibility-changing scopes', () => { + const Asset = defineEntity( + object({ id: number().primaryKey(), kind: string() }).hasTableName( + 'assets' + ) + ) + .discriminator('kind') + .stiVariant('note', object({ text: string() })); + const one = query(knex, Asset.schema); + const two = query(knex, Asset.schema); + let foreign: any; + one.orderBy(c => { + foreign = c.id; + return c.id; + }); + expect(() => two.orderBy(() => foreign)).toThrow(/does not belong/); + const invalid = Asset.schema.defaultScope(((q: any) => + q.withDeleted()) as any); + expect(() => query(knex, invalid)).toThrow(/shape-preserving/); + }); + it('rejects removed raw-source overloads instead of silently dropping SQL', () => { + expect(() => (query as any)(knex, Account, knex('accounts'))).toThrow( + /output/ + ); + expect(() => + (createQuery(knex) as any)(Account, knex('accounts')) + ).toThrow(/output/); + }); + it('derives metadata automatically and preserves it across filters and paging', () => { + const base = query(knex, Account); + const one = base.where('id', 1).limit(1); + const two = base.where('id', 2).offset(3); + expect(one).not.toBe(base); + expect(one.rowSchema).toBe(base.rowSchema); + expect(two.rowSchema).toBe(base.rowSchema); + expect(base.toQuery()).not.toContain('where'); + expect(one.toQuery()).toContain('= 1'); + expect(two.toQuery()).toContain('= 2'); + expect('withRowSchema' in base).toBe(false); + }); + + it('replaces projections without mutating either source', () => { + const base = query(knex, Account); + const named = base.select(t => ({ name: t.name })); + const ids = named.select(t => ({ key: t.id })); + expect(Object.keys(base.rowSchema.introspect().properties)).toEqual([ + 'id', + 'name', + 'active' + ]); + expect(Object.keys(named.rowSchema.introspect().properties)).toEqual([ + 'name' + ]); + expect(Object.keys(ids.rowSchema.introspect().properties)).toEqual([ + 'key' + ]); + expect(ids.rowSchema).not.toBe(named.rowSchema); + }); + + it('runs groups once and ignores discarded immutable branches', () => { + let retained: any; + const callback = vi.fn(group => { + retained = group; + group.where('id', 99); + return group.where('id', 1).orWhere('id', 2); + }); + const filtered = query(knex, Account).where(callback); + retained.where('id', 3); + expect(filtered.toQuery()).toContain('= 1 or'); + expect(filtered.toQuery()).not.toContain('99'); + expect(filtered.toQuery()).not.toContain('= 3'); + expect(callback).toHaveBeenCalledTimes(1); + }); + + it('rejects void, async and unrelated group results without executing them', () => { + const base = query(knex, Account); + expect(() => base.where((() => undefined) as any)).toThrow( + /must return/ + ); + expect(() => base.where((async () => undefined) as any)).toThrow( + /synchronous/ + ); + expect(() => base.where((() => base) as any)).toThrow( + /synchronous|must return/ + ); + }); + + it('captures default scopes once and removes only their effects', () => { + const scope = vi.fn(q => q.where('active', 1)); + const schema = Account.defaultScope(scope); + const base = query(knex, schema); + const explicit = base.where('id', 7); + const unscoped = explicit.unscoped(); + expect(explicit.toQuery()).toMatch(/"active" = 1/); + expect(unscoped.toQuery()).not.toMatch(/"active" = 1/); + expect(unscoped.toQuery()).toMatch(/"id" = 7/); + expect(scope).toHaveBeenCalledTimes(1); + expect(unscoped.rowSchema).toBe(base.rowSchema); + }); + + it('isolates mutable Knex snapshots', () => { + const base = query(knex, Account).where('active', 1); + const snapshot = base.toKnexQuery(); + snapshot.where('id', 9).limit(1); + expect(base.toQuery()).not.toContain('= 9'); + expect(base.toQuery()).not.toContain('limit'); + }); + + it('captures opaque SQL once and requires an explicit object output', () => { + const output = outputObject({ total: outputNumber().coerce() }); + let retained: any; + const configure = vi.fn(sql => { + retained = sql; + return sql.clearSelect().count({ total: '*' }); + }); + const raw = query(knex, Account).apply(configure, { output }); + retained.where('id', 99); + expect(raw.rowSchema).toBe(output); + expect(raw.toQuery()).toContain('count(*)'); + expect(raw.toQuery()).not.toContain('99'); + raw.toQuery(); + expect(configure).toHaveBeenCalledTimes(1); + expect(() => + (query(knex, Account) as any).selectRaw('1 as total', []) + ).toThrow(/output/); + }); + + it('rejects writes through projected query instances at runtime too', async () => { + const selected = query(knex, Account).select(t => ({ name: t.name })); + await expect( + (selected as any).insert({ name: 'unsafe' }) + ).rejects.toThrow(/unprojected/); + await expect( + (selected as any).update({ name: 'unsafe' }) + ).rejects.toThrow(/unprojected/); + await expect((selected as any).insertMany([])).rejects.toThrow( + /unprojected/ + ); + await expect( + (query(knex, Account).havingRaw('count(*) > 0') as any).update({ + name: 'unsafe' + }) + ).rejects.toThrow(/unprojected/); + }); +}); diff --git a/libs/knex-schema/src/index.ts b/libs/knex-schema/src/index.ts index 90904f0b..e727ed58 100644 --- a/libs/knex-schema/src/index.ts +++ b/libs/knex-schema/src/index.ts @@ -1,7 +1,7 @@ // @cleverbrush/knex-schema — Type-safe schema-driven query builder for Knex -export type { ReadAliasTables } from './AliasedReadQuery.js'; -export { AliasedReadQuery } from './AliasedReadQuery.js'; +export type { ReadAliasTables } from './AliasedQueryBuilder.js'; +export { AliasedQueryBuilder } from './AliasedQueryBuilder.js'; export type { AliasTables, JoinedProjection, @@ -9,7 +9,7 @@ export type { TableAlias } from './aliased-query.js'; // Types -export { AliasedQueryBuilder, alias, and, eq, or } from './aliased-query.js'; +export { alias, and, eq, or } from './aliased-query.js'; export type { PrimaryKeyColumns, RowVersionColumn, @@ -92,13 +92,18 @@ export { tableExistsInDb, validateEntitiesAgainstDatabase } from './migration.js'; +export { OpaqueQuery, type QueryOutput } from './OpaqueQuery.js'; export type { CompositeCursorOptions } from './operations/composite-cursor.js'; export type { PolymorphicRowSchema, VariantReadSchema, VariantReadSchemas -} from './PolymorphicReadQuery.js'; -export { PolymorphicReadQuery } from './PolymorphicReadQuery.js'; +} from './PolymorphicQueryBuilder.js'; +export { PolymorphicQueryBuilder } from './PolymorphicQueryBuilder.js'; +export type { BoundQuery } from './query.js'; +// Main entry point +export { createQuery, query } from './query.js'; +export type { QueryScope } from './query-scope.js'; // Raw query execution export { rawQuery } from './raw.js'; export type { @@ -110,6 +115,12 @@ export type { WithReadVariant } from './read-entity.js'; export { READ_ENTITY } from './read-entity.js'; +export type { + ReadMembership, + ReadPredicateBuilder, + ReadPredicateGroup, + ReadPredicateSelector +} from './read-predicates.js'; export type { ColumnReadSchema, ObjectReadSchema, @@ -119,20 +130,14 @@ export type { SchemaForValue } from './read-schema.js'; export { ReadSchemaError } from './read-schema.js'; -export type { BoundQuery } from './SchemaQueryBuilder.js'; -// Main entry point -export { - createQuery, - query, - SchemaQueryBuilder -} from './SchemaQueryBuilder.js'; export type { ReadColumn, ReadColumns, ReadProjection, + ReadQueryShape, SchemaAwareQuery -} from './SchemaReadQuery.js'; -export { SchemaReadQuery } from './SchemaReadQuery.js'; +} from './SchemaQueryBuilder.js'; +export { SchemaQueryBuilder } from './SchemaQueryBuilder.js'; // Snapshot-based migration export { entitiesToSnapshot, diff --git a/libs/knex-schema/src/mappers.ts b/libs/knex-schema/src/mappers.ts index 050c3058..c59d3586 100644 --- a/libs/knex-schema/src/mappers.ts +++ b/libs/knex-schema/src/mappers.ts @@ -98,7 +98,7 @@ export function mapObject>( * the `mappers` defined on each spec to the nested data, and returns the * mutated row. * - * This is an internal helper used by {@link SchemaQueryBuilder}'s result + * This is an internal helper used by {@link QuerySource}'s result * mapping pipeline. Exported to allow custom post-processing if needed. * * @param row - The raw result row (mutated in place). diff --git a/libs/knex-schema/src/operations/aggregate.ts b/libs/knex-schema/src/operations/aggregate.ts index 063ee36c..964fa5fd 100644 --- a/libs/knex-schema/src/operations/aggregate.ts +++ b/libs/knex-schema/src/operations/aggregate.ts @@ -8,21 +8,21 @@ import { createAggregate, type SelectableColumn } from '../expressions.js'; -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; import type { ColumnRef } from '../types.js'; import { buildQuery, getEffectiveBaseQuery, - getSchemaQueryBuilderCtor + getQuerySourceCtor } from './helpers.js'; import { getState } from './state.js'; /** Clone Framework metadata as well as Knex state; terminal helpers never mutate the source. */ export function cloneQuery( - builder: SchemaQueryBuilder -): SchemaQueryBuilder { + builder: QuerySource +): QuerySource { const state = getState(builder); - const Constructor = getSchemaQueryBuilderCtor(); + const Constructor = getQuerySourceCtor(); const copy = new Constructor( state.knex, state.localSchema, @@ -78,7 +78,7 @@ export function assertScalarSource(query: Knex.QueryBuilder): void { export async function scalarAggregate< S extends ObjectSchemaBuilder >( - builder: SchemaQueryBuilder, + builder: QuerySource, kind: AggregateKind, column?: ColumnRef, options?: AggregateOptions diff --git a/libs/knex-schema/src/operations/composite-cursor.ts b/libs/knex-schema/src/operations/composite-cursor.ts index 342bda95..243d2ef9 100644 --- a/libs/knex-schema/src/operations/composite-cursor.ts +++ b/libs/knex-schema/src/operations/composite-cursor.ts @@ -5,7 +5,7 @@ import { getPrimaryKeyColumns, resolvePropertyKey } from '../columns.js'; -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; import type { ColumnRef, CursorPaginationResult } from '../types.js'; import { cloneQuery, statements } from './aggregate.js'; import { cleanAndMapRow, getEffectiveBaseQuery, getQuery } from './helpers.js'; @@ -30,7 +30,7 @@ export interface CompositeCursorOptions< } export async function compositeCursor( - builder: SchemaQueryBuilder, + builder: QuerySource, options: CompositeCursorOptions, decode?: (row: Record) => any, sourceIdentity?: string diff --git a/libs/knex-schema/src/operations/delete.ts b/libs/knex-schema/src/operations/delete.ts index 5f8a7962..506d275f 100644 --- a/libs/knex-schema/src/operations/delete.ts +++ b/libs/knex-schema/src/operations/delete.ts @@ -1,11 +1,12 @@ // @cleverbrush/knex-schema — DELETE / soft-delete / restore operations -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; +import { returningReadColumns } from '../read-schema.js'; import { getSoftDelete, invalidateCache, mapRow } from './helpers.js'; import { getState } from './state.js'; export async function deleteImpl( - builder: SchemaQueryBuilder + builder: QuerySource ): Promise { const state = getState(builder); @@ -14,7 +15,7 @@ export async function deleteImpl( | Function[] | undefined) ?? []; for (const hook of hooks) { - await hook(builder); + await hook(state.hookQuery ?? builder); } const softDelete = getSoftDelete(builder); @@ -26,14 +27,14 @@ export async function deleteImpl( return state.baseQuery.delete(); } -export function withDeletedImpl(builder: SchemaQueryBuilder): any { +export function withDeletedImpl(builder: QuerySource): any { const state = getState(builder); invalidateCache(builder); state.includeDeleted = true; return builder; } -export function onlyDeletedImpl(builder: SchemaQueryBuilder): any { +export function onlyDeletedImpl(builder: QuerySource): any { const state = getState(builder); invalidateCache(builder); state.onlyDeleted = true; @@ -42,7 +43,7 @@ export function onlyDeletedImpl(builder: SchemaQueryBuilder): any { } export async function hardDeleteImpl( - builder: SchemaQueryBuilder + builder: QuerySource ): Promise { const state = getState(builder); const hooks = @@ -50,13 +51,13 @@ export async function hardDeleteImpl( | Function[] | undefined) ?? []; for (const hook of hooks) { - await hook(builder); + await hook(state.hookQuery ?? builder); } return state.baseQuery.delete(); } export async function restoreImpl( - builder: SchemaQueryBuilder + builder: QuerySource ): Promise { const state = getState(builder); const softDelete = getSoftDelete(builder); @@ -67,6 +68,6 @@ export async function restoreImpl( } const rows = await state.baseQuery .update({ [softDelete.column]: null }) - .returning('*'); + .returning(returningReadColumns(state.knex, state.localSchema)); return rows.map((row: any) => mapRow(builder, row)); } diff --git a/libs/knex-schema/src/operations/helpers.ts b/libs/knex-schema/src/operations/helpers.ts index f47c5ad9..a39c1c01 100644 --- a/libs/knex-schema/src/operations/helpers.ts +++ b/libs/knex-schema/src/operations/helpers.ts @@ -1,4 +1,4 @@ -// @cleverbrush/knex-schema — Extracted helper functions from SchemaQueryBuilder +// @cleverbrush/knex-schema — Extracted helper functions from QuerySource import type { InferType } from '@cleverbrush/schema'; import { @@ -19,7 +19,7 @@ import { POLYMORPHIC_TYPE_BRAND } from '../extension.js'; import { clearRow } from '../mappers.js'; -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; import type { ColumnRef, ResolvedVariantConfig, @@ -65,7 +65,7 @@ export type QueryResultType = TLocalSchema extends { // --------------------------------------------------------------------------- export function resolveColumn( - builder: SchemaQueryBuilder, + builder: QuerySource, ref: any, label = 'column' ): string | Knex.Raw { @@ -78,12 +78,12 @@ export function resolveColumn( ); } -export function invalidateCache(builder: SchemaQueryBuilder): void { +export function invalidateCache(builder: QuerySource): void { getState(builder).cachedBuiltQuery = null; } export function getSoftDelete( - builder: SchemaQueryBuilder + builder: QuerySource ): { column: string } | null { const state = getState(builder); const ext = (state.localSchema as any).getExtension?.('softDelete'); @@ -91,7 +91,7 @@ export function getSoftDelete( } export function getDefaultScope( - builder: SchemaQueryBuilder + builder: QuerySource ): Function | null { const state = getState(builder); const fn = (state.localSchema as any).getExtension?.('defaultScope'); @@ -99,7 +99,7 @@ export function getDefaultScope( } export function getTimestamps( - builder: SchemaQueryBuilder + builder: QuerySource ): { createdAt: string; updatedAt: string } | null { const state = getState(builder); const ts = (state.localSchema as any).getExtension?.('timestamps'); @@ -127,7 +127,7 @@ const ALLOWED_OPS = new Set([ export { ALLOWED_OPS }; export function getVariantConfig( - builder: SchemaQueryBuilder + builder: QuerySource ): ResolvedVariantConfig | null { const state = getState(builder); if (state.variantConfig !== undefined) return state.variantConfig; @@ -149,7 +149,7 @@ export function getVariantConfig( } export function applyVariantJoins( - builder: SchemaQueryBuilder, + builder: QuerySource, base: Knex.QueryBuilder, variantConfig: ResolvedVariantConfig ): Knex.QueryBuilder { @@ -289,24 +289,22 @@ export function applyVariantJoins( return qb; } -// Circular-dependency-safe SchemaQueryBuilder constructor reference -// Set by SchemaQueryBuilder.ts after the class is defined. -let SchemaQueryBuilderCtor: new (...args: any[]) => any = null!; -export function registerSchemaQueryBuilder( - ctor: new (...args: any[]) => any -): void { - SchemaQueryBuilderCtor = ctor; +// Circular-dependency-safe QuerySource constructor reference +// Set by QuerySource.ts after the class is defined. +let QuerySourceCtor: new (...args: any[]) => any = null!; +export function registerQuerySource(ctor: new (...args: any[]) => any): void { + QuerySourceCtor = ctor; } -export function getSchemaQueryBuilderCtor(): new (...args: any[]) => any { - return SchemaQueryBuilderCtor; +export function getQuerySourceCtor(): new (...args: any[]) => any { + return QuerySourceCtor; } export function buildVariantRelationSelect( - builder: SchemaQueryBuilder, + builder: QuerySource, foreignSchema: ObjectSchemaBuilder, relAlias: string, foreignTableName: string, - customize?: (q: SchemaQueryBuilder) => void + customize?: (q: QuerySource) => void ): Knex.Raw[] { const state = getState(builder); const knex = state.knex; @@ -318,7 +316,7 @@ export function buildVariantRelationSelect( let columnsToSelect: string[]; if (customize) { - const probe = new (SchemaQueryBuilderCtor as any)( + const probe = new (QuerySourceCtor as any)( state.knex, foreignSchema, state.knex(foreignTableName) @@ -348,7 +346,7 @@ export function buildVariantRelationSelect( } export function mapPolymorphicRow( - builder: SchemaQueryBuilder, + builder: QuerySource, row: Record, variantConfig: ResolvedVariantConfig ): Record { @@ -458,14 +456,14 @@ export function mapPolymorphicRow( } export function resolveSchema( - _builder: SchemaQueryBuilder, + _builder: QuerySource, schema: any ): ObjectSchemaBuilder { return typeof schema === 'function' ? schema() : schema; } export function findPrimaryKeyColumn( - _builder: SchemaQueryBuilder, + _builder: QuerySource, schema: ObjectSchemaBuilder ): string { const pk = getPrimaryKeyColumns(schema); @@ -473,7 +471,7 @@ export function findPrimaryKeyColumn( return 'id'; } -export function resolvePkColumns(builder: SchemaQueryBuilder): { +export function resolvePkColumns(builder: QuerySource): { propertyKeys: readonly string[]; columnNames: readonly string[]; } { @@ -488,7 +486,7 @@ export function resolvePkColumns(builder: SchemaQueryBuilder): { } export function getEffectiveBaseQuery( - builder: SchemaQueryBuilder + builder: QuerySource ): Knex.QueryBuilder { const state = getState(builder); let effectiveBase = state.baseQuery; @@ -516,7 +514,7 @@ export function getEffectiveBaseQuery( effectiveBase = effectiveBase.clone(); cloned = true; } - const proxy = new (SchemaQueryBuilderCtor as any)( + const proxy = new (QuerySourceCtor as any)( state.knex, state.localSchema, effectiveBase @@ -531,7 +529,7 @@ export function getEffectiveBaseQuery( } export function buildJoinOne( - builder: SchemaQueryBuilder, + builder: QuerySource, resultQuery: Knex.QueryBuilder, spec: ValidatedSpec & { type: 'one' }, relationAlias: string @@ -592,7 +590,7 @@ export function buildJoinOne( } export function buildJoinMany( - builder: SchemaQueryBuilder, + builder: QuerySource, resultQuery: Knex.QueryBuilder, spec: ValidatedSpec & { type: 'many' }, relationAlias: string, @@ -738,9 +736,7 @@ export function buildJoinMany( }); } -export function buildQuery( - builder: SchemaQueryBuilder -): Knex.QueryBuilder { +export function buildQuery(builder: QuerySource): Knex.QueryBuilder { const state = getState(builder); const effectiveBase = getEffectiveBaseQuery(builder); @@ -853,9 +849,7 @@ export function buildQuery( return resultQuery; } -export function getQuery( - builder: SchemaQueryBuilder -): Knex.QueryBuilder { +export function getQuery(builder: QuerySource): Knex.QueryBuilder { const state = getState(builder); if (!state.cachedBuiltQuery) { state.cachedBuiltQuery = buildQuery(builder); @@ -864,7 +858,7 @@ export function getQuery( } export function mapRow( - builder: SchemaQueryBuilder, + builder: QuerySource, row: Record ): Record { if (!row) return row; @@ -887,11 +881,11 @@ export function mapRow( } } - return result; + return state.decodeRow ? state.decodeRow(result) : result; } export function cleanAndMapRow( - builder: SchemaQueryBuilder, + builder: QuerySource, row: Record ): Record { const state = getState(builder); @@ -911,7 +905,7 @@ export function cleanAndMapRow( } export function mapObjectToColumns( - builder: SchemaQueryBuilder, + builder: QuerySource, obj: Record ): Record { const state = getState(builder); @@ -931,14 +925,14 @@ export function mapObjectToColumns( } export function mapRecordToColumns( - builder: SchemaQueryBuilder, + builder: QuerySource, record: Record ): Record { return mapObjectToColumns(builder, record); } export function resolveColumnArg( - builder: SchemaQueryBuilder, + builder: QuerySource, col: any ): string | Knex.Raw { if (typeof col === 'string') { @@ -951,7 +945,7 @@ export function resolveColumnArg( } export function isColumnAccessor( - builder: SchemaQueryBuilder, + builder: QuerySource, fn: Function ): boolean { const state = getState(builder); @@ -972,7 +966,7 @@ export function isColumnAccessor( } export function assertNotProjection( - builder: SchemaQueryBuilder, + builder: QuerySource, method: string ): void { const state = getState(builder); @@ -986,7 +980,7 @@ export function assertNotProjection( } export function assertNotExplicitSelect( - builder: SchemaQueryBuilder, + builder: QuerySource, method: string ): void { const state = getState(builder); diff --git a/libs/knex-schema/src/operations/insert.ts b/libs/knex-schema/src/operations/insert.ts index 83b18033..8595f497 100644 --- a/libs/knex-schema/src/operations/insert.ts +++ b/libs/knex-schema/src/operations/insert.ts @@ -4,7 +4,8 @@ import type { InferType, ObjectSchemaBuilder } from '@cleverbrush/schema'; import type { Knex } from 'knex'; import { buildColumnMap, resolveColumnRef } from '../columns.js'; import { getTableName } from '../extension.js'; -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; +import { returningReadColumns } from '../read-schema.js'; import type { ColumnRef, InsertType } from '../types.js'; import { getTimestamps, @@ -82,12 +83,13 @@ export interface OnConflictMergeOptions { } /** - * Configure one-row conflict handling after SchemaQueryBuilder.onConflict(); merge()/ignore() execute the insert. + * Configure one-row conflict handling after QuerySource.onConflict(); merge()/ignore() execute the insert. */ export class OnConflictBuilder { readonly #knex: Knex; readonly #localSchema: TLocalSchema; readonly #conflictColumns: string[]; + readonly #decodeRow?: (row: Record) => Record; /** * Create conflict handling for a schema and resolved conflict columns. @@ -96,12 +98,13 @@ export class OnConflictBuilder { constructor( knex: Knex, localSchema: TLocalSchema, - _parent: SchemaQueryBuilder, + _parent: QuerySource, conflictColumns: string[] ) { this.#knex = knex; this.#localSchema = localSchema; this.#conflictColumns = conflictColumns; + this.#decodeRow = getState(_parent).decodeRow; } /** @@ -221,7 +224,9 @@ export class OnConflictBuilder { options?.where?.(qb as unknown as Knex.QueryBuilder, helpers); } - const rows = await (qb as any).returning('*'); + const rows = await (qb as any).returning( + returningReadColumns(this.#knex, this.#localSchema) + ); if (!rows || rows.length === 0) return undefined; const { colToProp } = buildColumnMap(this.#localSchema as any); @@ -229,7 +234,7 @@ export class OnConflictBuilder { for (const [col, val] of Object.entries(rows[0])) { result[colToProp.get(col) ?? col] = val; } - return result as TResult; + return (this.#decodeRow ? this.#decodeRow(result) : result) as TResult; } #createMergeHelpers(): OnConflictMergeHelpers { @@ -273,7 +278,7 @@ function isMergeOptions( // --------------------------------------------------------------------------- export async function insertImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, data: InsertType ): Promise { const state = getState(builder); @@ -299,7 +304,7 @@ export async function insertImpl( const [row] = await state .knex(state.tableName) .insert(mapped) - .returning('*'); + .returning(returningReadColumns(state.knex, state.localSchema)); const result = mapRow(builder, row); const afterHooks = @@ -314,7 +319,7 @@ export async function insertImpl( } export async function insertManyImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, data: InsertType[] ): Promise { const state = getState(builder); @@ -341,7 +346,7 @@ export async function insertManyImpl( const rows = await state .knex(state.tableName) .insert(mapped) - .returning('*'); + .returning(returningReadColumns(state.knex, state.localSchema)); const results = rows.map((row: any) => mapRow(builder, row)); const afterHooks = @@ -358,7 +363,7 @@ export async function insertManyImpl( } export function onConflictImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, ...conflictColumns: ColumnRef[] ): OnConflictBuilder { const state = getState(builder); @@ -369,7 +374,7 @@ export function onConflictImpl( } export async function upsertImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, data: InsertType, opts: { conflictColumns: ColumnRef[]; @@ -395,12 +400,14 @@ export async function upsertImpl( (qb as any).merge(); } - const [row] = await (qb as any).returning('*'); + const [row] = await (qb as any).returning( + returningReadColumns(state.knex, state.localSchema) + ); return mapRow(builder, row); } export async function bulkInsertImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, rows: InsertType[], opts?: { chunkSize?: number; @@ -485,7 +492,9 @@ export async function bulkInsertImpl( } } - const inserted: any[] = await qb.returning('*'); + const inserted: any[] = await qb.returning( + returningReadColumns(state.knex, state.localSchema) + ); for (const row of inserted) { const mappedRow = mapRow(builder, row); for (const hook of afterHooks) { @@ -499,7 +508,7 @@ export async function bulkInsertImpl( } export async function bulkUpsertImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, rows: InsertType[], opts: { conflictColumns: ColumnRef[]; diff --git a/libs/knex-schema/src/operations/join.ts b/libs/knex-schema/src/operations/join.ts index b1390aae..617107c7 100644 --- a/libs/knex-schema/src/operations/join.ts +++ b/libs/knex-schema/src/operations/join.ts @@ -3,7 +3,7 @@ import type { Knex } from 'knex'; import { resolveColumnRef } from '../columns.js'; import { getTableName } from '../extension.js'; -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; import type { JoinManySpec, JoinOneSpec, @@ -17,7 +17,7 @@ import { } from '../validate.js'; import { findPrimaryKeyColumn, - getSchemaQueryBuilderCtor, + getQuerySourceCtor, getVariantConfig, invalidateCache, resolveSchema @@ -25,7 +25,7 @@ import { import { getState } from './state.js'; export function joinOneImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, spec: JoinOneSpec ): any { const state = getState(builder); @@ -37,7 +37,7 @@ export function joinOneImpl( } export function joinManyImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, spec: JoinManySpec ): any { const state = getState(builder); @@ -49,9 +49,9 @@ export function joinManyImpl( } export function includeImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, relationName: string, - customize?: (q: SchemaQueryBuilder) => void + customize?: (q: QuerySource) => void ): any { const state = getState(builder); invalidateCache(builder); @@ -110,7 +110,7 @@ export function includeImpl( const foreignQuery1: Knex.QueryBuilder = state.knex(foreignTableName); if (customize) { - const SQB = getSchemaQueryBuilderCtor(); + const SQB = getQuerySourceCtor(); const proxy = new SQB(state.knex, foreignSchema, foreignQuery1); customize(proxy); } @@ -138,7 +138,7 @@ export function includeImpl( const foreignQuery2: Knex.QueryBuilder = state.knex(foreignTableName); if (customize) { - const SQB = getSchemaQueryBuilderCtor(); + const SQB = getQuerySourceCtor(); const proxy = new SQB(state.knex, foreignSchema, foreignQuery2); customize(proxy); } @@ -167,7 +167,7 @@ export function includeImpl( const foreignQuery3: Knex.QueryBuilder = state.knex(foreignTableName); if (customize) { - const SQB = getSchemaQueryBuilderCtor(); + const SQB = getQuerySourceCtor(); const proxy = new SQB(state.knex, foreignSchema, foreignQuery3); customize(proxy); } @@ -201,7 +201,7 @@ export function includeImpl( ); if (customize) { - const SQB = getSchemaQueryBuilderCtor(); + const SQB = getQuerySourceCtor(); const proxy = new SQB(state.knex, foreignSchema, foreignQuery); customize(proxy); } @@ -221,10 +221,10 @@ export function includeImpl( } export function includeVariantImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, variantKey: string, relationName: string, - customize?: (q: SchemaQueryBuilder) => void + customize?: (q: QuerySource) => void ): any { const state = getState(builder); invalidateCache(builder); diff --git a/libs/knex-schema/src/operations/pagination.ts b/libs/knex-schema/src/operations/pagination.ts index 5968b16b..b3ad95d6 100644 --- a/libs/knex-schema/src/operations/pagination.ts +++ b/libs/knex-schema/src/operations/pagination.ts @@ -1,7 +1,7 @@ // @cleverbrush/knex-schema — Pagination (offset & cursor-based) import { resolvePropertyKey } from '../columns.js'; -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; import type { ColumnRef, CursorPaginationResult, @@ -16,20 +16,14 @@ import { import { getState } from './state.js'; import { orderByImpl, whereImpl } from './where.js'; -export function limitImpl( - builder: SchemaQueryBuilder, - n: number -): any { +export function limitImpl(builder: QuerySource, n: number): any { const state = getState(builder); invalidateCache(builder); state.baseQuery.limit(n); return builder; } -export function offsetImpl( - builder: SchemaQueryBuilder, - n: number -): any { +export function offsetImpl(builder: QuerySource, n: number): any { const state = getState(builder); invalidateCache(builder); state.baseQuery.offset(n); @@ -37,7 +31,7 @@ export function offsetImpl( } export async function paginateImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, opts: { page: number; pageSize: number; @@ -74,7 +68,7 @@ export async function paginateImpl( } export async function paginateAfterImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, opts: { cursor?: any; limit: number; @@ -112,7 +106,7 @@ export async function paginateAfterImpl( } export async function executeImpl( - builder: SchemaQueryBuilder + builder: QuerySource ): Promise { const query = getQuery(builder); const rows = await query; diff --git a/libs/knex-schema/src/operations/select.ts b/libs/knex-schema/src/operations/select.ts index 223631bc..5c62e018 100644 --- a/libs/knex-schema/src/operations/select.ts +++ b/libs/knex-schema/src/operations/select.ts @@ -8,7 +8,7 @@ import type { Knex } from 'knex'; import { buildColumnMap } from '../columns.js'; import { compileAggregate, isAggregate } from '../expressions.js'; import { getProjections } from '../extension.js'; -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; import type { ColumnRef } from '../types.js'; import { assertNotExplicitSelect, @@ -20,7 +20,7 @@ import { import { getState } from './state.js'; export function selectImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, ...args: unknown[] ): any { const state = getState(builder); @@ -104,7 +104,7 @@ export function selectImpl( } export function distinctImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, ...columns: (ColumnRef | Knex.Raw)[] ): any { invalidateCache(builder); @@ -114,7 +114,7 @@ export function distinctImpl( } export function countImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column?: ColumnRef | Knex.Raw ): any { const state = getState(builder); @@ -130,7 +130,7 @@ export function countImpl( } export function countDistinctImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column?: ColumnRef | Knex.Raw ): any { const state = getState(builder); @@ -148,7 +148,7 @@ export function countDistinctImpl( } export function minImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef | Knex.Raw ): any { const state = getState(builder); @@ -160,7 +160,7 @@ export function minImpl( } export function maxImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef | Knex.Raw ): any { const state = getState(builder); @@ -172,7 +172,7 @@ export function maxImpl( } export function sumImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef | Knex.Raw ): any { const state = getState(builder); @@ -184,7 +184,7 @@ export function sumImpl( } export function avgImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef | Knex.Raw ): any { const state = getState(builder); @@ -196,7 +196,7 @@ export function avgImpl( } export function selectRawImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, sql: string, bindings?: any[] ): any { @@ -211,7 +211,7 @@ export function selectRawImpl( } export function projectedImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, name: string ): any { const state = getState(builder); @@ -245,10 +245,7 @@ export function projectedImpl( return builder; } -export function scopedImpl( - builder: SchemaQueryBuilder, - name: string -): any { +export function scopedImpl(builder: QuerySource, name: string): any { const state = getState(builder); invalidateCache(builder); const scopes = (state.localSchema as any).getExtension?.('scopes') as @@ -264,7 +261,7 @@ export function scopedImpl( return builder; } -export function unscopedImpl(builder: SchemaQueryBuilder): any { +export function unscopedImpl(builder: QuerySource): any { const state = getState(builder); invalidateCache(builder); state.skipDefaultScope = true; diff --git a/libs/knex-schema/src/operations/state.ts b/libs/knex-schema/src/operations/state.ts index c71645fa..42c8a211 100644 --- a/libs/knex-schema/src/operations/state.ts +++ b/libs/knex-schema/src/operations/state.ts @@ -1,8 +1,8 @@ -// @cleverbrush/knex-schema — Shared mutable state store for SchemaQueryBuilder +// @cleverbrush/knex-schema — Shared mutable state store for QuerySource import type { ObjectSchemaBuilder } from '@cleverbrush/schema'; import type { Knex } from 'knex'; -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; import type { ResolvedVariantConfig, ValidatedSpec, @@ -10,6 +10,8 @@ import type { } from '../types.js'; export interface QueryBuilderState { + /** Decode write-returning property rows before lifecycle hooks observe them. */ + decodeRow?: (row: Record) => Record; /** Raw callbacks have no statically declared result shape for opt-in reads. */ opaqueReadShape?: boolean; knex: Knex; @@ -31,6 +33,9 @@ export interface QueryBuilderState { projectionDecoders: Record unknown>; hiddenColumns: Set; + /** Immutable public query snapshot supplied to observational delete hooks. */ + hookQuery?: unknown; + /** When true, soft-delete filter is NOT applied. */ includeDeleted: boolean; @@ -53,29 +58,27 @@ export interface QueryBuilderState { variantRelationIncludes: Array<{ variantKey: string; relationName: string; - customize?: (q: SchemaQueryBuilder) => void; + customize?: (q: QuerySource) => void; }>; /** Memoized result of buildQuery(). null = needs rebuild. */ cachedBuiltQuery: Knex.QueryBuilder | null; } -const STATE = new WeakMap, QueryBuilderState>(); +const STATE = new WeakMap, QueryBuilderState>(); -export function getState( - builder: SchemaQueryBuilder -): QueryBuilderState { +export function getState(builder: QuerySource): QueryBuilderState { const s = STATE.get(builder); if (!s) { throw new Error( - 'SchemaQueryBuilder state not found — builder was not properly initialized' + 'QuerySource state not found — builder was not properly initialized' ); } return s; } export function setState( - builder: SchemaQueryBuilder, + builder: QuerySource, state: QueryBuilderState ): void { STATE.set(builder, state); diff --git a/libs/knex-schema/src/operations/update.ts b/libs/knex-schema/src/operations/update.ts index ba89037a..c65461de 100644 --- a/libs/knex-schema/src/operations/update.ts +++ b/libs/knex-schema/src/operations/update.ts @@ -3,7 +3,8 @@ import type { InferType } from '@cleverbrush/schema'; import type { Knex } from 'knex'; import { buildColumnMap } from '../columns.js'; -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; +import { returningReadColumns } from '../read-schema.js'; import { getTimestamps, mapObjectToColumns, @@ -13,7 +14,7 @@ import { import { getState } from './state.js'; export async function updateImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, data: Partial> ): Promise { const state = getState(builder); @@ -35,12 +36,14 @@ export async function updateImpl( mapped[timestamps.updatedAt] = state.knex.fn.now(); } - const rows = await state.baseQuery.update(mapped).returning('*'); + const rows = await state.baseQuery + .update(mapped) + .returning(returningReadColumns(state.knex, state.localSchema)); return rows.map((row: any) => mapRow(builder, row)); } export async function bulkUpdateImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, updates: ReadonlyArray<{ where: Partial>; set: Partial>; @@ -124,7 +127,8 @@ export async function bulkUpdateImpl( ] as any); } - let qb: any = knex(state.tableName).update(updateExpr); + // Preserve the immutable caller's captured visibility and predicates. + let qb: any = state.baseQuery.clone().update(updateExpr); if (pk.columnNames.length === 1) { qb = qb.whereIn( pk.columnNames[0], diff --git a/libs/knex-schema/src/operations/where.ts b/libs/knex-schema/src/operations/where.ts index 51c77f74..2163c113 100644 --- a/libs/knex-schema/src/operations/where.ts +++ b/libs/knex-schema/src/operations/where.ts @@ -1,7 +1,7 @@ // @cleverbrush/knex-schema — WHERE / ORDER BY / GROUP BY / HAVING operations import type { Knex } from 'knex'; -import type { SchemaQueryBuilder } from '../SchemaQueryBuilder.js'; +import type { QuerySource } from '../QuerySource.js'; import type { ColumnRef } from '../types.js'; import { invalidateCache, @@ -13,7 +13,7 @@ import { import { getState } from './state.js'; export function whereImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, columnOrRaw: any, ...args: any[] ): any { @@ -42,7 +42,7 @@ export function whereImpl( } export function andWhereImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, columnOrRaw: any, ...args: any[] ): any { @@ -71,7 +71,7 @@ export function andWhereImpl( } export function orWhereImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, columnOrRaw: any, ...args: any[] ): any { @@ -100,7 +100,7 @@ export function orWhereImpl( } export function whereNotImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, columnOrRaw: any, ...args: any[] ): any { @@ -129,7 +129,7 @@ export function whereNotImpl( } export function whereInImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef, values: readonly any[] | Knex.QueryBuilder ): any { @@ -143,7 +143,7 @@ export function whereInImpl( } export function whereNotInImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef, values: readonly any[] | Knex.QueryBuilder ): any { @@ -157,7 +157,7 @@ export function whereNotInImpl( } export function orWhereInImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef, values: readonly any[] | Knex.QueryBuilder ): any { @@ -171,7 +171,7 @@ export function orWhereInImpl( } export function orWhereNotInImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef, values: readonly any[] | Knex.QueryBuilder ): any { @@ -185,7 +185,7 @@ export function orWhereNotInImpl( } export function whereNullImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef ): any { const state = getState(builder); @@ -197,7 +197,7 @@ export function whereNullImpl( } export function whereNotNullImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef ): any { const state = getState(builder); @@ -209,7 +209,7 @@ export function whereNotNullImpl( } export function orWhereNullImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef ): any { const state = getState(builder); @@ -221,7 +221,7 @@ export function orWhereNullImpl( } export function orWhereNotNullImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef ): any { const state = getState(builder); @@ -233,7 +233,7 @@ export function orWhereNotNullImpl( } export function whereBetweenImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef, range: readonly [any, any] ): any { @@ -247,7 +247,7 @@ export function whereBetweenImpl( } export function whereNotBetweenImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef, range: readonly [any, any] ): any { @@ -261,7 +261,7 @@ export function whereNotBetweenImpl( } export function whereLikeImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef, value: string ): any { @@ -275,7 +275,7 @@ export function whereLikeImpl( } export function whereILikeImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef, value: string ): any { @@ -289,7 +289,7 @@ export function whereILikeImpl( } export function whereRawImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, sql: string, ...bindings: any[] ): any { @@ -300,7 +300,7 @@ export function whereRawImpl( } export function whereExistsImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, callback: Knex.QueryCallback | Knex.QueryBuilder ): any { const state = getState(builder); @@ -310,7 +310,7 @@ export function whereExistsImpl( } export function whereNotExistsImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, callback: Knex.QueryCallback | Knex.QueryBuilder ): any { const state = getState(builder); @@ -320,7 +320,7 @@ export function whereNotExistsImpl( } export function whereJsonPathImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef, path: string, operator?: string, @@ -356,7 +356,7 @@ export function whereJsonPathImpl( } export function orderByImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef | Knex.Raw, direction?: 'asc' | 'desc' ): any { @@ -368,7 +368,7 @@ export function orderByImpl( } export function orderByRawImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, sql: string, ...bindings: any[] ): any { @@ -379,7 +379,7 @@ export function orderByRawImpl( } export function groupByImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, ...columns: (ColumnRef | Knex.Raw)[] ): any { const state = getState(builder); @@ -390,7 +390,7 @@ export function groupByImpl( } export function groupByRawImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, sql: string, ...bindings: any[] ): any { @@ -401,7 +401,7 @@ export function groupByRawImpl( } export function havingImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, column: ColumnRef | Knex.Raw, operator: string, value: any @@ -414,7 +414,7 @@ export function havingImpl( } export function havingRawImpl( - builder: SchemaQueryBuilder, + builder: QuerySource, sql: string, ...bindings: any[] ): any { diff --git a/libs/knex-schema/src/orm-extensions.test.ts b/libs/knex-schema/src/orm-extensions.test.ts index bc7d113e..f946a764 100644 --- a/libs/knex-schema/src/orm-extensions.test.ts +++ b/libs/knex-schema/src/orm-extensions.test.ts @@ -76,12 +76,14 @@ describe('SchemaQueryBuilder.select(selector) — DTO projection', () => { query(knex, User).select(_t => ({ bogus: 'not-a-descriptor' as any })) - ).toThrow(/property descriptor/); + ).toThrow(/does not belong/); }); it('still supports the existing column-list overload', () => { const sql = query(knex, User).select('id', 'name').toQuery(); - expect(sql).toContain('select "id", "name"'); + expect(sql).toMatch( + /select "__schema_read_\d+"\."id" as "id", "__schema_read_\d+"\."name" as "name"/ + ); }); }); diff --git a/libs/knex-schema/src/orm.test.ts b/libs/knex-schema/src/orm.test.ts index 5d9e00b8..174df58a 100644 --- a/libs/knex-schema/src/orm.test.ts +++ b/libs/knex-schema/src/orm.test.ts @@ -3,6 +3,7 @@ import Knex from 'knex'; import { afterAll, describe, expect, expectTypeOf, it, vi } from 'vitest'; import { + array, boolean, date, defineEntity, @@ -317,7 +318,9 @@ describe('DDL extensions', () => { }); it('string.jsonb() shorthand stores columnType', () => { - const schema = object({ meta: string().jsonb() }).hasTableName('test'); + const schema = object({ + meta: object({ status: string() }).jsonb() + }).hasTableName('test'); const ext = (schema.introspect() as any).properties.meta.introspect() .extensions; expect(ext.columnType).toBe('jsonb'); @@ -484,7 +487,7 @@ describe('generateCreateTable', () => { id: number().primaryKey(), count: number().bigint(), externalId: string().asUuid(), - meta: string().jsonb(), + meta: object({ status: string() }).jsonb(), price: number().decimal(10, 2), createdAt: date().timestamptz(), birthDate: date().dateOnly() @@ -599,55 +602,64 @@ describe('relation metadata', () => { // ═══════════════════════════════════════════════════════════════════════════ describe('include()', () => { - const PostWithRelations = Post.belongsTo('author', { - schema: User, - foreignKey: (t: any) => t.authorId - }); + const PostWithRelations = defineEntity( + Post.addProp('author', User.optional()) + ).belongsTo( + t => t.author, + t => t.authorId, + u => u.id + ).schema; it('include belongsTo generates joinOne SQL', () => { const sql = query(knex, PostWithRelations).include('author').toQuery(); - expect(sql).toContain('originalQuery'); + expect(sql).toContain('to_jsonb'); expect(sql).toContain('"users"'); expect(sql).toContain('"author"'); }); it('include category belongsTo generates joinOne SQL', () => { - const PostWithCategory = Post.belongsTo('category', { - schema: Category, - foreignKey: (t: any) => t.categoryId - }); + const PostWithCategory = defineEntity( + Post.addProp('category', Category.optional()) + ).belongsTo( + t => t.category, + t => t.categoryId, + c => c.id + ).schema; const sql = query(knex, PostWithCategory).include('category').toQuery(); - expect(sql).toContain('originalQuery'); + expect(sql).toContain('to_jsonb'); expect(sql).toContain('"categories"'); expect(sql).toContain('"category"'); }); it('include throws for unknown relation', () => { expect(() => + // @ts-expect-error runtime guard for an undeclared relation query(knex, PostWithRelations).include('nonexistent') - ).toThrow('Unknown relation "nonexistent"'); + ).toThrow('Unknown relation: nonexistent'); }); it('hasMany include generates joinMany SQL', () => { - const UserWithPosts = User.hasMany('posts', { - schema: Post, - foreignKey: (t: any) => t.authorId - }); + const UserWithPosts = defineEntity( + User.addProp('posts', array(Post).optional()) + ).hasMany( + u => u.posts, + u => u.id, + p => p.authorId + ).schema; const sql = query(knex, UserWithPosts).include('posts').toQuery(); - expect(sql).toContain('originalQuery'); + expect(sql).toContain('to_jsonb'); expect(sql).toContain('"posts"'); }); it('belongsToMany include generates pivot join SQL', () => { - const PostWithTags = Post.belongsToMany('tags', { - schema: Tag, - through: { - table: 'post_tags', - localKey: 'post_id', - foreignKey: 'tag_id' - } - }); + const PostWithTags = defineEntity( + Post.addProp('tags', array(Tag).optional()) + ).belongsToMany(p => p.tags, { + table: 'post_tags', + localKey: 'post_id', + foreignKey: 'tag_id' + }).schema; const sql = query(knex, PostWithTags).include('tags').toQuery(); expect(sql).toContain('post_tags'); @@ -698,9 +710,9 @@ describe('soft delete', () => { expect(sql).toContain('"deleted_at" is not null'); }); - it('unscoped() removes soft delete filter', () => { + it('unscoped() preserves the independent soft delete filter', () => { const sql = query(knex, SoftPost).unscoped().toQuery(); - expect(sql).not.toContain('deleted_at'); + expect(sql).toContain('"deleted_at" is null'); }); it('delete() generates UPDATE for soft-delete schemas', () => { @@ -807,7 +819,7 @@ describe('scopes', () => { expect(() => // @ts-expect-error — 'nonexistent' is not a registered scope name query(knex, ScopedPost).scoped('nonexistent') - ).toThrow('Unknown scope "nonexistent"'); + ).toThrow('Unknown scope: nonexistent'); }); it('scoped() type: only registered scope names are accepted', () => { @@ -834,7 +846,7 @@ describe('scopes', () => { it('unscoped() bypasses default scope', () => { const sql = query(knex, ScopedPost).unscoped().toQuery(); - expect(sql).not.toContain('is_active'); + expect(sql).not.toContain('"is_active" = true'); }); }); @@ -912,7 +924,7 @@ describe('projections', () => { expect(() => // @ts-expect-error — 'bogus' is not a registered projection name query(knex, PostSchema).projected('bogus') - ).toThrow('Unknown projection "bogus"'); + ).toThrow('Unknown projection: bogus'); }); it('projected() type: only registered names are accepted', () => { @@ -927,40 +939,46 @@ describe('projections', () => { void _check3; }); - it('projected() after select() throws', () => { - expect(() => - query(knex, PostSchema) - .select(t => t.id) - .projected('summary') - ).toThrow(/projected.*select|select.*projected/i); - }); - - it('select() after projected() throws', () => { - expect(() => - query(knex, PostSchema) - .projected('summary') - .select(t => t.id) - ).toThrow(/select.*projected|projected.*select/i); - }); - - it('count() after projected() throws', () => { - expect(() => - query(knex, PostSchema).projected('summary').count() - ).toThrow(/count.*projected|projected.*count/i); - }); - - it('projected() after count() throws', () => { - expect(() => - query(knex, PostSchema).count().projected('summary') - ).toThrow(/projected.*aggregate|aggregate.*projected/i); - }); - - it('two projected() calls throw', () => { - expect(() => - query(knex, PostSchema).projected('summary').projected('withStatus') - ).toThrow( - /Cannot call .projected\(\).*projected\(|Only one projection/i - ); + it.each([ + [ + 'selected to named', + (q: any) => q.select('id').projected('summary'), + ['id', 'title'] + ], + [ + 'named to selected', + (q: any) => q.projected('summary').select('id'), + ['id'] + ], + [ + 'named to aggregate', + (q: any) => q.projected('summary').count(), + ['count'] + ], + [ + 'aggregate to named', + (q: any) => q.count().projected('summary'), + ['id', 'title'] + ], + [ + 'named to named', + (q: any) => q.projected('summary').projected('withStatus'), + ['id', 'status'] + ] + ])('replaces a projection: %s', (_name, configure, keys) => { + const root = query(knex, PostSchema); + const projected = configure(root); + expect( + Object.keys(projected.rowSchema.introspect().properties) + ).toEqual(keys); + expect(Object.keys(root.rowSchema.introspect().properties)).toEqual([ + 'id', + 'title', + 'body', + 'status', + 'isActive' + ]); + expect(projected).not.toBe(root); }); it('projection() throws on duplicate name', () => { @@ -1035,8 +1053,9 @@ describe('selectRaw', () => { it('selectRaw() adds raw SQL to select clause', () => { const sql = query(knex, Post) .selectRaw( - '*, ts_rank(search_vector, plainto_tsquery(?)) AS rank', - ['search term'] + 'ts_rank(search_vector, plainto_tsquery(?)) AS rank', + ['search term'], + { output: object({ rank: number() }) } ) .toQuery(); expect(sql).toContain('ts_rank'); @@ -1619,21 +1638,25 @@ describe('extension method chaining', () => { describe('Phase 2 query methods', () => { it('whereNotExists generates correct SQL', () => { - const sql = query(knex, User) + const root = query(knex, User); + const sql = root .whereNotExists( knex .queryBuilder() .from('posts') - .where('posts.author_id', knex.raw('users.id')) + .where( + 'posts.author_id', + root.ref(t => t.id) + ) ) .toQuery(); - expect(sql).toContain('where not exists'); + expect(sql).toContain('not exists'); }); it('whereJsonPath generates jsonb_path_query_first SQL', () => { const DataSchema = object({ id: number().primaryKey(), - meta: string().jsonb() + meta: object({ status: string() }).jsonb() }).hasTableName('data_items'); const sql = query(knex, DataSchema) @@ -1646,7 +1669,7 @@ describe('Phase 2 query methods', () => { it('whereJsonPath throws on non-pg client', () => { const DataSchema = object({ id: number().primaryKey(), - meta: string().jsonb() + meta: object({ status: string() }).jsonb() }).hasTableName('data_items'); // Create a separate pg knex instance and override the client config @@ -1669,7 +1692,7 @@ describe('Phase 2 query methods', () => { it('whereJsonPath with @? operator generates existence check SQL', () => { const DataSchema = object({ id: number().primaryKey(), - tags: string().jsonb() + tags: object({ tags: array(string()) }).jsonb() }).hasTableName('data_items'); const sql = query(knex, DataSchema) @@ -1751,16 +1774,16 @@ describe('Phase 2 query methods', () => { expect(typeof q.pluck).toBe('function'); }); - it('toQuery() result is memoized and invalidated by mutations', () => { + it('toQuery() is stable and configuration creates an independent branch', () => { const q = query(knex, User).where(t => t.name, 'Alice'); const sql1 = q.toQuery(); const sql2 = q.toQuery(); // Same instance, same result — should be identical strings expect(sql1).toBe(sql2); - // Mutating invalidates the cache — new SQL includes the extra condition - q.where(t => t.role, 'admin'); - const sql3 = q.toQuery(); + const branch = q.where(t => t.role, 'admin'); + const sql3 = branch.toQuery(); + expect(q.toQuery()).toBe(sql1); expect(sql3).not.toBe(sql1); expect(sql3).toContain('admin'); }); @@ -1832,19 +1855,19 @@ describe('nested object jsonb columns', () => { }).jsonb() }).hasTableName('people'); - it('accessor t => t.address.city generates ->? SQL', () => { + it('accessor t => t.address.city generates typed JSON path SQL', () => { const sql = query(knex, PersonSchema) .where(t => (t as any).address.city, '=', 'NYC') .toQuery(); - expect(sql).toContain('->'); + expect(sql).toContain('#>>'); expect(sql).toContain('city'); }); - it('dotted string path address.city generates ->? SQL', () => { + it('dotted string path address.city generates typed JSON path SQL', () => { const sql = query(knex, PersonSchema) .where('address.city' as any, '=', 'NYC') .toQuery(); - expect(sql).toContain('->'); + expect(sql).toContain('#>>'); expect(sql).toContain('city'); }); @@ -2091,9 +2114,10 @@ describe('withVariants (polymorphic schemas)', () => { it('documents the expected result shape via SQL analysis', () => { // Integration-style: verify the SELECT aliases are generated const sql = query(knex, FileSchema).toQuery(); - // Each CTI column should appear as __v_image__ - expect(sql).toContain('__v_image'); - expect(sql).toContain('__v_document'); + // Branch shapes stay separate inside the union's JSON envelope. + expect(sql).toContain('to_jsonb(__read_branch)'); + expect(sql).toContain('image_file'); + expect(sql).toContain('document_file'); }); }); @@ -2188,23 +2212,23 @@ describe('withVariants — per-variant relations', () => { // SQL generation — includeVariant // ------------------------------------------------------------------------- - it('includeVariant generates LEFT JOIN with namespaced alias', () => { + it('includeVariant generates a CTI body join and a correlated relation', () => { const sql = query(knex, AssetSchema) .includeVariant('licensed', 'owner') .toQuery(); expect(sql.toLowerCase()).toContain('left join'); expect(sql.toLowerCase()).toContain('owners'); - expect(sql).toContain('__v_licensed__rel_owner'); + expect(sql).toContain('as "owner"'); }); - it('includeVariant selects foreign columns with prefix', () => { + it('includeVariant aliases foreign properties inside a nested object', () => { const sql = query(knex, AssetSchema) .includeVariant('licensed', 'owner') .toQuery(); - expect(sql).toContain('__v_licensed__rel_owner__id'); - expect(sql).toContain('__v_licensed__rel_owner__email'); + expect(sql).toContain('"id" as "id"'); + expect(sql).toContain('"email" as "email"'); }); it('includeVariant with projection selects only projected columns', () => { @@ -2233,9 +2257,9 @@ describe('withVariants — per-variant relations', () => { ) .toQuery(); - expect(sql).toContain('__v_licensed__rel_owner__id'); - expect(sql).toContain('__v_licensed__rel_owner__email'); - expect(sql).not.toContain('__v_licensed__rel_owner__name'); + expect(sql).toContain('"id" as "id"'); + expect(sql).toContain('"email" as "email"'); + expect(sql).not.toContain('"name" as "name"'); }); // ------------------------------------------------------------------------- @@ -2243,13 +2267,15 @@ describe('withVariants — per-variant relations', () => { // ------------------------------------------------------------------------- it('include() auto-routes to includeVariant when relation is unambiguous', () => { - const sql = query(knex, AssetSchema).include('owner').toQuery(); - expect(sql).toContain('__v_licensed__rel_owner'); + const sql = (query(knex, AssetSchema) as any) + .include('owner') + .toQuery(); + expect(sql).toContain('as "owner"'); }); it('include() throws for truly unknown relations on polymorphic schema', () => { expect(() => { - query(knex, AssetSchema).include('nonexistent').toQuery(); + (query(knex, AssetSchema) as any).include('nonexistent').toQuery(); }).toThrow(/Unknown relation/); }); @@ -2280,7 +2306,7 @@ describe('withVariants — per-variant relations', () => { }).schema; expect(() => { - query(knex, AmbigSchema).include('owner').toQuery(); + (query(knex, AmbigSchema) as any).include('owner').toQuery(); }).toThrow(/[Aa]mbiguous/); }); @@ -2291,39 +2317,40 @@ describe('withVariants — per-variant relations', () => { it('includeVariant throws for non-polymorphic schema', () => { const Plain = object({ id: number().primaryKey() }).hasTableName('t'); expect(() => { + // @ts-expect-error non-polymorphic queries do not expose variant operations query(knex, Plain).includeVariant('x', 'y'); - }).toThrow(/not polymorphic/); + }).toThrow(/includeVariant is not a function/); }); it('includeVariant throws for unknown variant key', () => { expect(() => { + // @ts-expect-error runtime guard for an unknown variant query(knex, AssetSchema).includeVariant('unknown_variant', 'owner'); - }).toThrow(/unknown variant key/); + }).toThrow(/Unknown variant/); }); it('includeVariant throws for unknown relation name', () => { expect(() => { query(knex, AssetSchema).includeVariant('licensed', 'nonexistent'); - }).toThrow(/unknown relation/); + }).toThrow(/Unknown relation/); }); // ------------------------------------------------------------------------- // Row mapping — Pass 3 (observable via SQL + alias presence) // ------------------------------------------------------------------------- - it('includeVariant generates relation column aliases in SQL (Pass 3 input)', () => { - // Verify all 3 Owner columns are aliased into the query result so that - // Pass 3 of #mapPolymorphicRow can read them. + it('includeVariant includes every related property inside its JSON object', () => { + // All three Owner properties are selected inside the related JSON object. const sql = query(knex, AssetSchema) .includeVariant('licensed', 'owner') .toQuery(); - expect(sql).toContain('__v_licensed__rel_owner__id'); - expect(sql).toContain('__v_licensed__rel_owner__email'); - expect(sql).toContain('__v_licensed__rel_owner__name'); + expect(sql).toContain('"id" as "id"'); + expect(sql).toContain('"email" as "email"'); + expect(sql).toContain('"name" as "name"'); }); - it('STI variant with includeVariant generates correct ON discriminator gate', () => { + it('STI relation is confined to the matching discriminator branch', () => { // STI variant: no separate table alias, gate uses base table discriminator const StiSchema = defineEntity(AssetBase) .discriminator('kind') @@ -2344,7 +2371,7 @@ describe('withVariants — per-variant relations', () => { // Should join owners and gate ON discriminator = 'free' expect(sql.toLowerCase()).toContain('owners'); - expect(sql).toContain('__v_free__rel_owner'); + expect(sql).toContain('as "owner"'); expect(sql).toContain('free'); }); }); @@ -2358,7 +2385,7 @@ describe('joinOne / joinMany validation errors', () => { foreignColumn: (t: any) => t.authorId, as: '' as any }) - ).toThrow('as must be a non-empty string'); + ).toThrow(/non-empty/); }); it('joinMany throws when as is missing', () => { @@ -2369,7 +2396,7 @@ describe('joinOne / joinMany validation errors', () => { foreignColumn: (t: any) => t.authorId, as: '' as any }) - ).toThrow('as must be a non-empty string'); + ).toThrow(/non-empty/); }); it('joinOne throws when mappers is not an object', () => { @@ -2379,8 +2406,9 @@ describe('joinOne / joinMany validation errors', () => { localColumn: (t: any) => t.id, foreignColumn: (t: any) => t.authorId, as: 'posts', + // @ts-expect-error removed raw mapper escape cannot supply a row schema mappers: 'invalid' as any }) - ).toThrow('mappers must be an object'); + ).toThrow(/typed child customizer/); }); }); diff --git a/libs/knex-schema/src/public-api-docs.test.ts b/libs/knex-schema/src/public-api-docs.test.ts index 506bc058..bc3039e4 100644 --- a/libs/knex-schema/src/public-api-docs.test.ts +++ b/libs/knex-schema/src/public-api-docs.test.ts @@ -109,7 +109,7 @@ describe('published query/ORM API documentation', () => { it('documents aggregate expressions, bound factory overloads and cursor options', () => { for (const [path, name] of [ ['../dist/expressions.d.ts', 'aggregate'], - ['../dist/SchemaQueryBuilder.d.ts', 'BoundQuery'], + ['../dist/query.d.ts', 'BoundQuery'], [ '../dist/operations/composite-cursor.d.ts', 'CompositeCursorOptions' diff --git a/libs/knex-schema/src/query-scope.ts b/libs/knex-schema/src/query-scope.ts new file mode 100644 index 00000000..1e815379 --- /dev/null +++ b/libs/knex-schema/src/query-scope.ts @@ -0,0 +1,24 @@ +import type { Knex } from 'knex'; +import type { ReadRelations } from './read-entity.js'; +import type { + ReadPredicateSelector, + ReadPredicates +} from './read-predicates.js'; +import type { ReadObject } from './read-schema.js'; +import type { ReadColumns } from './SchemaQueryBuilder.js'; + +/** Shape-preserving immutable API supplied to named and default scopes. */ +export interface QueryScope + extends ReadPredicates>> { + /** Append native column ordering. Return the new query. */ + orderBy( + column: ReadPredicateSelector>>, + direction?: 'asc' | 'desc' + ): this; + /** Append trusted ordering with captured bindings. */ + orderByRaw(sql: string, bindings?: readonly Knex.RawBinding[]): this; + /** Limit a new query without changing its row shape. */ + limit(count: number): this; + /** Offset a new query without changing its row shape. */ + offset(count: number): this; +} diff --git a/libs/knex-schema/src/query.ts b/libs/knex-schema/src/query.ts new file mode 100644 index 00000000..7c5e517e --- /dev/null +++ b/libs/knex-schema/src/query.ts @@ -0,0 +1,74 @@ +import type { Knex } from 'knex'; +import { AliasedQueryBuilder } from './AliasedQueryBuilder.js'; +import { + AliasedQuerySource, + type AliasTables, + isTableAlias, + type TableAlias +} from './aliased-query.js'; +import { getTableName } from './extension.js'; +import { QuerySource } from './QuerySource.js'; +import type { ReadObject } from './read-schema.js'; +import { + createReadQuery, + type SchemaAwareQuery +} from './SchemaQueryBuilder.js'; + +// Register the private SQL/write planner before creating relation queries. +void QuerySource; + +/** Create an immutable, lazy query with an automatically inferred row schema. */ +export function query( + knex: Knex, + schema: TableAlias +): AliasedQueryBuilder>; +/** Create an immutable table or polymorphic query. Raw output requires apply(..., { output }). */ +export function query( + knex: Knex, + schema: S +): SchemaAwareQuery; +export function query( + knex: Knex, + schema: S | TableAlias, + ...unsupported: unknown[] +): SchemaAwareQuery | AliasedQueryBuilder> { + if (unsupported.length) + throw new TypeError( + 'Raw query sources are not supported; use apply(..., { output })' + ); + if (isTableAlias(schema)) + return new AliasedQueryBuilder(new AliasedQuerySource(knex, schema)); + return createReadQuery(knex, schema, knex(getTableName(schema))); +} + +/** Connection-bound query factory. Query configuration is lazy and immutable. */ +export interface BoundQuery { + /** Start a flat multi-table query. Select its output before execution. */ + ( + schema: TableAlias + ): AliasedQueryBuilder>; + /** Start a schema-backed table query. */ + (schema: S): SchemaAwareQuery; + /** Reuse an existing transaction without committing it. */ + withTransaction(trx: Knex.Transaction): BoundQuery; + /** Run work atomically with a transaction-bound factory. */ + transaction(callback: (db: BoundQuery) => Promise): Promise; +} + +/** Bind query() to a connection while preserving schema and alias inference. */ +export function createQuery(knex: Knex): BoundQuery { + return Object.assign( + (schema: any, ...unsupported: unknown[]) => { + if (unsupported.length) + throw new TypeError( + 'Raw query sources are not supported; use apply(..., { output })' + ); + return query(knex, schema); + }, + { + withTransaction: (trx: Knex.Transaction) => createQuery(trx), + transaction: (callback: (db: BoundQuery) => Promise) => + knex.transaction(trx => callback(createQuery(trx))) + } + ) as BoundQuery; +} diff --git a/libs/knex-schema/src/raw.test.ts b/libs/knex-schema/src/raw.test.ts index 087f0f09..10019a40 100644 --- a/libs/knex-schema/src/raw.test.ts +++ b/libs/knex-schema/src/raw.test.ts @@ -1,123 +1,115 @@ -// @cleverbrush/knex-schema — rawQuery() tests - -import Knex from 'knex'; -import { afterAll, describe, expect, it, vi } from 'vitest'; +import Knex, { type Knex as Connection } from 'knex'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { number, object, string } from './index.js'; import { rawQuery } from './raw.js'; -const Post = object({ - id: number(), - title: string(), - authorId: number().hasColumnName('author_id'), - createdAt: string().hasColumnName('created_at') -}).hasTableName('posts'); - -const knex = Knex({ client: 'pg' }); - -afterAll(async () => { - await knex.destroy(); -}); - -describe('rawQuery', () => { - it('runs a raw SQL string and maps column names back to property names', async () => { - const rawSpy = vi.spyOn(knex, 'raw').mockResolvedValueOnce({ - rows: [ - { id: 1, title: 'Hi', author_id: 7, created_at: '2024-01-01' } - ] - } as any); +// Stub only the driver boundary: raw() must still build real captured SQL. +describe('rawQuery explicit output contract', () => { + let knex: Connection; + let response: unknown[]; + let statements: Array<{ sql: string; bindings: unknown[] }>; + beforeEach(() => { + knex = Knex({ client: 'pg' }); + response = []; + statements = []; + const client = knex.client as any; + client.acquireConnection = async () => ({}); + client.releaseConnection = async () => {}; + client._query = async (_connection: unknown, statement: any) => { + statements.push(statement); + return { rows: response }; + }; + client.processResponse = (result: any) => result.rows; + }); + afterEach(async () => { + await knex.destroy(); + }); + it('parses SQL-aliased properties once, preserving the caller output contract', async () => { + const output = object({ + id: number(), + authorId: number(), + total: number().coerce() + }); + const parse = vi.spyOn(output, 'parse'); + response = [{ id: 1, authorId: 7, total: '12' }]; const rows = await rawQuery( knex, - Post, - 'SELECT * FROM posts WHERE id = ?', - [1] - ); - - expect(rawSpy).toHaveBeenCalledWith( - 'SELECT * FROM posts WHERE id = ?', + output, + 'select id, author_id as "authorId", total from posts where id = ?', [1] ); - expect(rows).toEqual([ - { id: 1, title: 'Hi', authorId: 7, createdAt: '2024-01-01' } - ]); - - rawSpy.mockRestore(); + expect(rows).toEqual([{ id: 1, authorId: 7, total: 12 }]); + expect(parse).toHaveBeenCalledExactlyOnceWith(response[0]); + expect(statements[0].bindings).toEqual([1]); }); - it('defaults bindings to [] when omitted', async () => { - const rawSpy = vi - .spyOn(knex, 'raw') - .mockResolvedValueOnce({ rows: [] } as any); - - await rawQuery(knex, Post, 'SELECT 1'); - - expect(rawSpy).toHaveBeenCalledWith('SELECT 1', []); - - rawSpy.mockRestore(); + it('does not silently remap raw output or accept a missing selected column', async () => { + response = [{ author_id: 7 }]; + await expect( + rawQuery( + knex, + object({ authorId: number() }), + 'select author_id from posts' + ) + ).rejects.toThrow(); }); - it('handles drivers that return an array directly (no .rows wrapper)', async () => { - const rawSpy = vi - .spyOn(knex, 'raw') - .mockResolvedValueOnce([ - { id: 2, title: 'X', author_id: 3, created_at: '2024-02-02' } - ] as any); - - const rows = await rawQuery(knex, Post, 'SELECT *'); - - expect(rows).toEqual([ - { id: 2, title: 'X', authorId: 3, createdAt: '2024-02-02' } - ]); - - rawSpy.mockRestore(); + it('returns an empty result without invoking the row parser', async () => { + const output = object({ id: number() }); + const parse = vi.spyOn(output, 'parse'); + expect(await rawQuery(knex, output, 'select id from posts')).toEqual( + [] + ); + expect(parse).not.toHaveBeenCalled(); + expect(statements[0].bindings).toEqual([]); }); - it('returns [] when rows is not an array', async () => { - const rawSpy = vi - .spyOn(knex, 'raw') - .mockResolvedValueOnce({ rows: { not: 'an array' } } as any); - - const rows = await rawQuery(knex, Post, 'SELECT *'); - expect(rows).toEqual([]); - - rawSpy.mockRestore(); + it('snapshots a caller-owned Knex SELECT without mutating or retaining it', async () => { + response = [{ id: 9 }]; + const source = knex('posts').select('id').where('id', 9); + const pending = rawQuery(knex, object({ id: number() }), source); + source.where('id', 100); + expect(await pending).toEqual([{ id: 9 }]); + expect(statements[0].bindings).toEqual([9]); }); - it('passes through extra columns not in the schema unchanged', async () => { - const rawSpy = vi.spyOn(knex, 'raw').mockResolvedValueOnce({ - rows: [ - { - id: 1, - title: 'Hi', - author_id: 7, - created_at: '2024-01-01', - extra_count: 42 - } - ] - } as any); - - const rows = await rawQuery(knex, Post, 'SELECT *'); - - expect(rows[0]).toMatchObject({ - id: 1, - authorId: 7, - createdAt: '2024-01-01', - extra_count: 42 // unmapped column passes through - }); - - rawSpy.mockRestore(); + it('keeps exact numeric text exact with an explicit text output', async () => { + response = [{ amount: '12345678901234567890.012345' }]; + expect( + await rawQuery( + knex, + object({ amount: string() }), + 'select amount::text as amount from invoices' + ) + ).toEqual(response); }); - it('awaits a Knex query builder directly', async () => { - const result = [ - { id: 9, title: 'Q', author_id: 1, created_at: '2024-03-03' } - ]; - const qb = Promise.resolve(result) as any; + it('rejects invalid raw rows', async () => { + response = [{ id: 'not a number' }]; + await expect( + rawQuery(knex, object({ id: number() }), 'select id from posts') + ).rejects.toThrow(); + }); - const rows = await rawQuery(knex, Post, qb); + it('rejects non-object output and non-SELECT builders before execution', () => { + expect(() => + rawQuery(knex, string() as any, 'select id from posts') + ).toThrow(/object schema/); + expect(() => + rawQuery(knex, object({ id: number() }), knex('posts').delete()) + ).toThrow(/SELECT/); + expect(statements).toEqual([]); + }); - expect(rows).toEqual([ - { id: 9, title: 'Q', authorId: 1, createdAt: '2024-03-03' } - ]); + it('rejects asynchronous output parsers instead of returning promises as rows', async () => { + response = [{ id: 1 }]; + const output = object({ id: number() }); + vi.spyOn(output, 'parse').mockImplementation((async () => ({ + id: 1 + })) as any); + await expect( + rawQuery(knex, output, 'select id from posts') + ).rejects.toThrow(/synchronously/); }); }); diff --git a/libs/knex-schema/src/raw.ts b/libs/knex-schema/src/raw.ts index d2a21dc8..d1550fe3 100644 --- a/libs/knex-schema/src/raw.ts +++ b/libs/knex-schema/src/raw.ts @@ -1,68 +1,43 @@ -// @cleverbrush/knex-schema — Raw query execution with schema result mapping - -import type { InferType, ObjectSchemaBuilder } from '@cleverbrush/schema'; +import type { InferType } from '@cleverbrush/schema'; import type { Knex } from 'knex'; -import { buildColumnMap } from './columns.js'; +import { OpaqueQuery } from './OpaqueQuery.js'; +import { captureReadRaw } from './read-predicates.js'; +import type { ReadObject } from './read-schema.js'; /** - * Execute a raw SQL query or Knex query builder and map the result rows - * through the schema's column→property name mapping. - * - * This is the escape hatch for complex queries that can't be expressed with - * the typed `SchemaQueryBuilder` API. The schema is used only for result - * mapping — column names in the result are converted back to property names. - * Extra columns (not in the schema) are passed through unchanged. - * - * @param knex - A configured Knex instance. - * @param schema - The `ObjectSchemaBuilder` for result mapping. - * @param queryOrSql - A raw SQL string or a `Knex.QueryBuilder`. - * @param bindings - Optional bindings for parameterised SQL queries. - * @returns Mapped result rows. + * Execute a captured raw SELECT with an explicit complete output schema. + * SQL must alias columns to the output property names. Each driver row is parsed + * exactly once; there is no implicit column mapping or entity decoding. Cast + * exact numeric values to text in SQL before a driver can lose their precision. * + * @param knex - Connection or transaction used for execution. + * @param output - Synchronous Framework object schema describing every returned row. + * @param queryOrSql - Trusted SELECT SQL or an independently captured Knex SELECT. + * @param bindings - Bound values for the trusted SQL string. + * @returns Parsed rows, never attached to an ORM identity map. * @example * ```ts - * // Raw SQL with schema result mapping - * const results = await rawQuery(knex, PostSchema, ` - * SELECT p.*, COUNT(c.id) AS comment_count - * FROM posts p - * LEFT JOIN comments c ON c.post_id = p.id - * GROUP BY p.id - * ORDER BY comment_count DESC - * LIMIT ? - * `, [10]); - * - * // Knex query builder as the source - * const subQuery = knex('posts').select('author_id', knex.raw('COUNT(*) as post_count')).groupBy('author_id'); - * const results = await rawQuery(knex, UserSchema, subQuery); + * const totals = await rawQuery(knex, object({ total: string() }), + * 'select sum(amount)::text as total from invoices where owner_id = ?', [ownerId]); * ``` */ -export async function rawQuery< - TSchema extends ObjectSchemaBuilder ->( +export function rawQuery( knex: Knex, - schema: TSchema, + output: S, queryOrSql: string | Knex.QueryBuilder, - bindings?: any[] -): Promise<(InferType & Record)[]> { - let rows: any[]; - - if (typeof queryOrSql === 'string') { - const result = await knex.raw(queryOrSql, bindings ?? []); - rows = result.rows ?? result; - } else { - rows = await queryOrSql; - } - - if (!rows || !Array.isArray(rows)) return []; - - const { colToProp } = buildColumnMap(schema); - - return rows.map(row => { - const mapped: Record = {}; - for (const [key, value] of Object.entries(row)) { - const propName = colToProp.get(key); - mapped[propName ?? key] = value; - } - return mapped; - }) as any; + bindings: readonly Knex.RawBinding[] = [] +): Promise[]> { + const sql = + typeof queryOrSql === 'string' + ? knex + .queryBuilder() + .from( + captureReadRaw(knex, queryOrSql, bindings)().wrap( + '(', + ') as __raw_output' + ) + ) + .select('*') + : queryOrSql; + return OpaqueQuery.capture(knex, sql, { output }).execute(); } diff --git a/libs/knex-schema/src/read-consumer.test.ts b/libs/knex-schema/src/read-consumer.test.ts index 5cfb3d9f..fc612086 100644 --- a/libs/knex-schema/src/read-consumer.test.ts +++ b/libs/knex-schema/src/read-consumer.test.ts @@ -42,7 +42,7 @@ test('documentation and multi-file metadata consumers compile against published ) ]; expect(namedBlocks(llms)).toHaveLength(4); - expect(namedBlocks(queryDocs)).toHaveLength(3); + expect(namedBlocks(queryDocs)).toHaveLength(4); for (const [, name, content] of [ ...namedBlocks(llms), ...namedBlocks(queryDocs) @@ -91,7 +91,7 @@ const value: InferType = undefined; const invalidUnit: InferExtensionMetadata['unit'] = 'seconds'; // @ts-expect-error importing database libraries cannot install global storage methods number().bigint(); -export const read = query(knex, Account).withRowSchema().select(a => ({ id: a.id, balance: a.balance })); +export const read = query(knex, Account).select(a => ({ id: a.id, balance: a.balance })); const PublicAccount = object({ id: string(), balance: string().nullable() }); export const convert = mapper().configure(read.rowSchema, PublicAccount, m => m) .getSyncMapper(read.rowSchema, PublicAccount); diff --git a/libs/knex-schema/src/read-graph.test-d.ts b/libs/knex-schema/src/read-graph.test-d.ts index 25ec2482..14de479c 100644 --- a/libs/knex-schema/src/read-graph.test-d.ts +++ b/libs/knex-schema/src/read-graph.test-d.ts @@ -46,14 +46,12 @@ const Album = defineEntity( const knex = Knex({ client: 'pg' }); test('polymorphic read branches retain discriminators and nested relation types', async () => { - const read = query(knex, Asset.schema) - .withRowSchema() - .forVariant('photo', q => - q.include( - r => r.labels, - labels => labels.select(l => ({ text: l.label })) - ) - ); + const read = query(knex, Asset.schema).forVariant('photo', q => + q.include( + r => r.labels, + labels => labels.select(l => ({ text: l.label })) + ) + ); type PhotoRow = InferType; type Expected = { id: number; @@ -80,12 +78,10 @@ test('polymorphic read branches retain discriminators and nested relation types' }); test('polymorphic reads compose inside an ordinary relation', async () => { - const read = query(knex, Album.schema) - .withRowSchema() - .include( - r => r.assets, - assets => assets.forVariant('photo', q => q.include(r => r.labels)) - ); + const read = query(knex, Album.schema).include( + r => r.assets, + assets => assets.forVariant('photo', q => q.include(r => r.labels)) + ); const rows = await read; const asset = rows[0].assets[0]; if (asset.kind === 'photo') diff --git a/libs/knex-schema/src/read-mapping.test.ts b/libs/knex-schema/src/read-mapping.test.ts index c138e1bc..e5dd4333 100644 --- a/libs/knex-schema/src/read-mapping.test.ts +++ b/libs/knex-schema/src/read-mapping.test.ts @@ -25,7 +25,7 @@ describe('query schemas as reusable mapping sources', () => { n => n.taskId ); const read = query(Knex({ client: 'pg' }), Task.schema) - .withRowSchema() + .select(t => ({ id: t.id, amount: t.amount, done: t.done })) .include( r => r.notes, diff --git a/libs/knex-schema/src/read-predicates.test-d.ts b/libs/knex-schema/src/read-predicates.test-d.ts new file mode 100644 index 00000000..bee3769a --- /dev/null +++ b/libs/knex-schema/src/read-predicates.test-d.ts @@ -0,0 +1,94 @@ +import type { InferType } from '@cleverbrush/schema'; +import Knex from 'knex'; +import { expectTypeOf, test } from 'vitest'; +import { alias, eq } from './aliased-query.js'; +import { number, object, string } from './extension.js'; +import type { ReadPredicateBuilder } from './read-predicates.js'; +import { query } from './SchemaQueryBuilder.js'; + +const knex = Knex({ client: 'pg' }); +const Item = object({ + id: number().primaryKey(), + name: string(), + amount: number().decimal(18, 2).optional() +}).hasTableName('items'); + +test('ordinary reader predicates retain projection types and contextual groups', async () => { + const read = query(knex, Item).select(t => ({ + name: t.name, + amount: t.amount + })); + const filtered = read + .where(p => p.where(t => t.id, 1).orWhere(t => t.name, 'two')) + .whereIn(t => t.id, knex('links').select('item_id')) + .whereExists( + knex('links') + .select('item_id') + .where( + 'item_id', + read.ref(t => t.id) + ) + ) + .whereRaw('?? = ?', [read.ref(t => t.name), 'name']) + .orderByRaw('?? asc', [read.ref(t => t.name)]); + expectTypeOf(filtered.rowSchema).toEqualTypeOf(read.rowSchema); + expectTypeOf(await filtered).toEqualTypeOf< + { name: string; amount: string | null }[] + >(); + // @ts-expect-error only projected fields are available on rows + const _id: number = (await filtered)[0].id; + // @ts-expect-error unknown selector properties are rejected + read.where(p => p.where(t => t.missing, 1)); + // @ts-expect-error refs use the same typed column context + read.ref(t => t.missing); + // @ts-expect-error grouped callbacks cannot be async + read.where(async p => { + p.where(t => t.id, 1); + }); + read.where(p => { + expectTypeOf(p).toExtend>(); + // @ts-expect-error groups cannot shape rows + p.select(t => t.id); + // @ts-expect-error groups cannot order rows + p.orderByRaw('id'); + // @ts-expect-error groups cannot execute SQL + p.execute(); + // @ts-expect-error group lifecycle is owned by the reader + p.finish(); + return p; + }); + // @ts-expect-error unrestricted mutation is still unavailable + read.apply(q => q.select('*')); +}); + +test('aliased selectors preserve joined nullability and exact storage types', async () => { + const read = query(knex, alias(Item, 'item')) + + .leftJoin(alias(Item, 'parent'), t => eq(t.item.id, t.parent.id)) + .select(t => ({ + id: t.item.id, + parentName: t.parent.name, + amount: t.parent.amount + })); + const filtered = read + .where(t => t.item.id, 1) + .andWhere(p => + p + .whereNull(t => t.parent.name) + .orWhere(n => n.where(t => t.parent.id, 2)) + ) + .orWhereNotExists(knex('links').select('item_id')) + .orderByRaw('?? asc', [read.ref(t => t.item.id)]); + expectTypeOf>().toEqualTypeOf<{ + id: number; + parentName: string | null; + amount: string | null; + }>(); + expectTypeOf(await filtered).toEqualTypeOf< + InferType[] + >(); + // @ts-expect-error unknown aliases are rejected inside groups + filtered.where(p => p.where(t => t.unknown.id, 1)); + // @ts-expect-error async group callbacks are rejected on aliased readers too + filtered.orWhere(async p => p.where(t => t.item.id, 1)); +}); diff --git a/libs/knex-schema/src/read-predicates.test.ts b/libs/knex-schema/src/read-predicates.test.ts new file mode 100644 index 00000000..c6371f3c --- /dev/null +++ b/libs/knex-schema/src/read-predicates.test.ts @@ -0,0 +1,294 @@ +import Knex, { type Knex as KnexTypes } from 'knex'; +import { describe, expect, it, vi } from 'vitest'; +import { alias, eq } from './aliased-query.js'; +import type { AliasedColumn } from './expressions.js'; +import { date, number, object, string } from './extension.js'; +import type { + ReadPredicateBuilder, + ReadPredicateGroup +} from './read-predicates.js'; +import { query } from './SchemaQueryBuilder.js'; + +const Task = object({ + id: number().primaryKey(), + projectId: number().hasColumnName('project_id'), + title: string(), + amount: number().decimal(24, 6).optional(), + completedAt: date().optional().hasColumnName('completed_at') +}).hasTableName('tasks'); +const knex = Knex({ client: 'pg' }); +const readTask = () => query(knex, Task); + +describe('shape-preserving read predicates', () => { + it('groups AND/OR conditions without mutating the source or its schema', () => { + const source = readTask().select(t => ({ id: t.id, amount: t.amount })); + const filtered = source + .where(t => t.projectId, 1) + .andWhere(p => + p + .where(t => t.title, 'one') + .orWhere(n => + n + .where(t => t.id, '>', 2) + .whereNotNull(t => t.completedAt) + ) + ); + expect(filtered.rowSchema).toBe(source.rowSchema); + expect(filtered.toQuery()).toMatch( + /"project_id" = 1 and \(.*"title" = 'one' or \(.*"id" > 2 and .*"completed_at" is not null\)\)/ + ); + expect(source.toQuery()).not.toContain(' where '); + expect(source.where(t => t.projectId, 2).toQuery()).not.toContain( + "'one'" + ); + expect(Object.keys(filtered.rowSchema.introspect().properties)).toEqual( + ['id', 'amount'] + ); + }); + + it('supports comparisons, null variants and empty membership lists', () => { + const read = readTask(); + expect(read.where(t => t.completedAt, null).toQuery()).toContain( + 'is null' + ); + expect( + read.where(t => t.completedAt, 'is not', null).toQuery() + ).toContain('is not null'); + expect(read.whereIn(t => t.id, []).toQuery()).toContain('1 = 0'); + expect(read.whereNotIn(t => t.id, []).toQuery()).toContain('1 = 1'); + const sql = read + .where(t => t.id, 1) + .orWhereIn(t => t.id, [2, 3]) + .orWhereNotIn(t => t.id, [4]) + .orWhereNull(t => t.completedAt) + .orWhereNotNull(t => t.amount) + .toQuery(); + expect(sql).toContain('in (2, 3)'); + expect(sql).toContain('not in (4)'); + expect(sql).toContain('or '); + expect(() => read.where(t => t.id, 'unsafe operator', 1)).toThrow( + /Unsupported comparison operator/ + ); + }); + + it('captures value lists, dates and raw bindings before external mutation', () => { + const read = readTask(); + const ids = [1, 2]; + const time = new Date('2026-01-01T00:00:00Z'); + const bindings: KnexTypes.RawBinding[] = [ + read.ref(t => t.title), + 'old' + ]; + const filtered = read + .whereIn(t => t.id, ids) + .where(t => t.completedAt, '>=', time) + .whereRaw('lower(??) = ?', bindings) + .orderByRaw('case when ?? = ? then 0 else 1 end', bindings); + const before = filtered.toQuery(); + ids.push(3); + time.setUTCFullYear(2030); + bindings[1] = 'new'; + expect(filtered.toQuery()).toBe(before); + expect(filtered.rowSchema).toBe(read.rowSchema); + const compiled = filtered.compile().toSQL(); + expect(compiled.sql).not.toContain("'old'"); + expect(compiled.bindings.filter(x => x === 'old')).toHaveLength(2); + }); + + it('quotes references for mapped columns and rejects foreign descriptors', () => { + const read = readTask(); + expect(read.ref(t => t.projectId).toSQL().sql).toMatch( + /^"__schema_read_\d+"\."project_id"$/ + ); + let other!: AliasedColumn; + readTask().ref(t => { + other = t.id; + return t.id; + }); + expect(() => read.ref(() => other)).toThrow(/does not belong/); + expect(() => read.ref(() => undefined as any)).toThrow( + /does not belong/ + ); + }); + + it('snapshots nested subquery callbacks once without executing SQL', () => { + const event = vi.fn(); + knex.on('query', event); + try { + const source = readTask(); + const inner = knex('links').select('task_id').where('label', 'old'); + const callback = vi.fn((q: KnexTypes.QueryBuilder) => { + q.whereIn('task_id', inner); + }); + const subquery = knex('labels').select('task_id').where(callback); + const filtered = source.whereIn(t => t.id, subquery); + expect(callback).toHaveBeenCalledTimes(1); + const before = filtered.toQuery(); + inner.where('label', 'changed'); + subquery.where('id', 99); + expect(filtered.toQuery()).toBe(before); + expect(callback).toHaveBeenCalledTimes(1); + expect(filtered.rowSchema).toBe(source.rowSchema); + expect(event).not.toHaveBeenCalled(); + } finally { + knex.removeListener('query', event); + } + }); + + it('supports correlated EXISTS and all OR/negative subquery forms', () => { + const read = readTask(); + const linked = knex('links') + .select('task_id') + .where( + 'task_id', + read.ref(t => t.id) + ); + const sql = read + .whereExists(linked) + .whereNotExists(linked) + .orWhereExists(linked) + .orWhereNotExists(linked) + .whereNotIn(t => t.id, linked) + .orWhereIn(t => t.id, linked) + .orWhereNotIn(t => t.id, linked) + .toQuery(); + expect(sql).toContain('exists (select'); + expect(sql).toContain('not exists (select'); + expect(sql).toContain('or exists (select'); + expect(sql).toContain('or not exists (select'); + expect(sql).toContain('not in (select'); + expect(sql).toContain('or '); + expect(sql).toMatch(/"task_id" = "__schema_read_\d+"\."id"/); + expect(() => read.whereExists(knex('links').delete())).toThrow( + /SELECT subquery/ + ); + expect(() => + read.whereIn(t => t.id, knex('links').update({ label: 'bad' })) + ).toThrow(/SELECT subquery/); + }); + + it('supports grouped raw predicates and escaped question-mark operators', () => { + const read = readTask(); + const value = "' OR 1=1 --"; + const filtered = read.where(p => + p + .whereRaw('?? = ?', [p.ref(t => t.title), value]) + .orWhereRaw('?::jsonb \\? ?', ['{"key":1}', 'key']) + ); + const compiled = filtered.compile().toSQL(); + expect(compiled.sql).not.toContain(value); + expect(compiled.bindings).toEqual([value, '{"key":1}', 'key']); + expect(compiled.toNative().sql).toContain('::jsonb ?'); + }); + + it('runs group callbacks once and isolates retained immutable builders', () => { + const read = readTask(); + let retained!: ReadPredicateBuilder; + const callback = vi.fn((p: ReadPredicateBuilder) => { + retained = p; + const configured = p.where(t => t.id, 1); + for (const method of [ + 'select', + 'join', + 'orderBy', + 'orderByRaw', + 'apply', + 'finish', + 'execute', + 'then' + ]) + expect(method in p).toBe(false); + return configured; + }); + const filtered = read.where(callback); + filtered.toQuery(); + filtered.toQuery(); + expect(callback).toHaveBeenCalledTimes(1); + expect(retained.where(t => t.id, 2)).not.toBe(retained); + expect(filtered.toQuery()).not.toContain('= 2'); + expect(filtered.toQuery()).toContain('= 1'); + expect(read.where(p => p).toQuery()).toBe(read.toQuery()); + }); + + it('rejects async or throwing groups without changing their parent', async () => { + const read = readTask(); + const asyncGroup = async (p: ReadPredicateBuilder) => { + await Promise.resolve(); + p.where(t => t.id, 1); + }; + expect(() => + read.where(asyncGroup as unknown as ReadPredicateGroup) + ).toThrow(/must be synchronous/); + expect(() => + read.where(p => { + p.where(t => t.id, 1); + throw new Error('stop'); + }) + ).toThrow('stop'); + await Promise.resolve(); + expect(read.toQuery()).not.toContain(' where '); + expect(() => read.where(p => (p as any).select('id'))).toThrow(); + }); + + it('rejects returned thenables without invoking them or executing a returned query', () => { + const read = readTask(); + const then = vi.fn(); + expect(() => read.where((() => ({ then })) as any)).toThrow( + /must be synchronous/ + ); + expect(then).not.toHaveBeenCalled(); + const foreign = knex('tasks').select('id'); + const execute = vi.spyOn(foreign, 'then'); + expect(() => read.where((() => foreign) as any)).toThrow( + /must be synchronous/ + ); + expect(execute).not.toHaveBeenCalled(); + }); + + it('applies the same predicates and binding snapshots to aliased joins', () => { + const base = query(knex, alias(Task, 'task')).leftJoin( + alias(Task, 'other'), + t => eq(t.task.id, t.other.projectId) + ); + const source = base.select(t => ({ + id: t.task.id, + otherId: t.other.id, + amount: t.other.amount + })); + const subquery = knex('links') + .select('task_id') + .where( + 'task_id', + source.ref(t => t.task.id) + ); + const bindings: KnexTypes.RawBinding[] = [ + source.ref(t => t.task.projectId), + 1 + ]; + const filtered = source + .where(t => t.task.projectId, 1) + .andWhere(p => + p.where(t => t.task.title, 'text').orWhereExists(subquery) + ) + .whereIn(t => t.task.id, subquery) + .orderByRaw('case when ?? = ? then 0 else 1 end', bindings); + const before = filtered.toQuery(); + bindings[1] = 2; + subquery.where('label', 'late'); + expect(filtered.toQuery()).toBe(before); + expect(before).toContain('"task"."project_id" = 1'); + expect(before).toContain('or exists (select'); + expect(filtered.rowSchema).toBe(source.rowSchema); + expect( + source.rowSchema.validate({ id: 1, otherId: null, amount: null }) + .valid + ).toBe(true); + expect(source.toQuery()).not.toContain(' where '); + expect('apply' in source).toBe(true); + expect(() => + query(knex, alias(Task, 'task')) + // @ts-expect-error opaque SQL requires its output contract + .apply(q => q.where('id', 1)) + ).toThrow(/output/); + }); +}); diff --git a/libs/knex-schema/src/read-predicates.ts b/libs/knex-schema/src/read-predicates.ts new file mode 100644 index 00000000..48f15f9f --- /dev/null +++ b/libs/knex-schema/src/read-predicates.ts @@ -0,0 +1,510 @@ +import type { Knex } from 'knex'; +import type { AliasedColumn } from './expressions.js'; +import { ALLOWED_OPS } from './operations/helpers.js'; +import { ReadSchemaError } from './read-schema.js'; + +const finishGroup = Symbol('finishReadPredicateGroup'); + +/** Select a column from the current reader's ordinary or aliased table context. */ +export type ReadPredicateSelector = + | ((columns: C) => AliasedColumn) + | (keyof C & string); + +/** A synchronous, parenthesized predicate group. The callback cannot shape or execute a query. */ +export type ReadPredicateGroup = ( + predicates: ReadPredicateBuilder +) => ReadPredicateBuilder; + +/** A bound value list or a caller-built SELECT subquery; captured without executing it. */ +export type ReadMembership = readonly unknown[] | Knex.QueryBuilder; + +/** @internal Predicate application contains only library-owned, already captured operations. */ +export type ReadPredicate = (query: Knex.QueryBuilder) => void; + +/** @internal Resolve references without exposing the parent's mutable SQL builder. */ +export interface ReadPredicateContext { + knex: Knex; + column: (selector: ReadPredicateSelector) => string | Knex.Raw; +} + +/** @internal Snapshot common mutable binding values independently of query builders. */ +function copyValue(value: any): any { + if (value instanceof Date) return new Date(value.getTime()); + if (Buffer.isBuffer(value)) return Buffer.from(value); + if (Array.isArray(value)) return value.map(copyValue); + if (value && typeof value === 'object') { + const prototype = Object.getPrototypeOf(value); + if (prototype === Object.prototype || prototype === null) { + return Object.fromEntries( + Object.entries(value).map(([key, item]) => [ + key, + copyValue(item) + ]) + ); + } + } + return value; +} + +/** @internal Compiled SQL is recreated per use, so externally owned builders are never retained. */ +function captureSql( + knex: Knex, + source: Knex.Raw | Knex.QueryBuilder +): () => Knex.Raw { + const compiled = source.toSQL(); + if (Array.isArray(compiled)) + throw new ReadSchemaError( + 'Read predicates require a single SQL expression' + ); + const sql = compiled.sql; + const bindings = compiled.bindings?.map(copyValue) ?? []; + return () => knex.raw(sql, bindings.map(copyValue)); +} + +/** @internal Capture trusted SQL and positional bindings now, including refs and nested raw expressions. */ +export function captureReadRaw( + knex: Knex, + sql: string, + bindings: readonly Knex.RawBinding[] = [] +): () => Knex.Raw { + return captureSql(knex, knex.raw(sql, [...bindings])); +} + +function captureSubquery(knex: Knex, query: Knex.QueryBuilder): () => Knex.Raw { + if ( + !query || + typeof query.toSQL !== 'function' || + typeof query.clone !== 'function' + ) + throw new ReadSchemaError('Expected a Knex SELECT subquery'); + const compiled = query.clone().toSQL(); + if ( + Array.isArray(compiled) || + !['select', 'first'].includes(compiled.method) + ) + throw new ReadSchemaError('Read predicates require a SELECT subquery'); + // Rewrap the compiled statement, not the original builder or its callbacks. + return captureSql( + knex, + knex.raw(compiled.sql, [...(compiled.bindings ?? [])]) + ); +} + +/** @internal Capture mutable bindings without retaining caller-owned values. */ +export function captureValue(knex: Knex, value: any): () => any { + if (value && typeof value.toSQL === 'function') { + return typeof value.clone === 'function' + ? captureSubquery(knex, value) + : captureSql(knex, value); + } + if (typeof value === 'function') + throw new ReadSchemaError('Predicate values cannot be callbacks'); + const captured = copyValue(value); + return () => copyValue(captured); +} + +/** + * Shared shape-preserving predicate methods for immutable readers and scoped groups. + * @internal Consumers obtain these methods through query factories, not inheritance. + */ +export abstract class ReadPredicates { + protected abstract readPredicateContext(): ReadPredicateContext; + protected abstract addReadPredicate(predicate: ReadPredicate): this; + + /** + * Quote a schema-backed column for raw bindings or correlated subqueries. + * The reference uses this reader's actual SQL alias and never executes SQL. + * @example read.whereRaw('lower(??) = ?', [read.ref(t => t.name), 'alice']) + */ + ref(selector: ReadPredicateSelector): Knex.Ref | Knex.Raw { + const { knex, column } = this.readPredicateContext(); + const resolved = column(selector); + return typeof resolved === 'string' ? knex.ref(resolved) : resolved; + } + + /** Add a parenthesized AND group using a synchronous predicate-only callback. */ + where(group: ReadPredicateGroup): this; + /** Match a record of property names and bound equality values. */ + where(values: Partial>): this; + /** Add a bound equality comparison. Null uses SQL IS NULL. */ + where(column: ReadPredicateSelector, value: unknown): this; + /** Add a bound comparison using a supported SQL operator. */ + where( + column: ReadPredicateSelector, + operator: string, + value: unknown + ): this; + where( + first: + | ReadPredicateSelector + | ReadPredicateGroup + | Partial>, + ...args: [] | [unknown] | [string, unknown] + ): this { + if (typeof first === 'object' && first !== null && !args.length) { + return Object.entries(first).reduce( + (query, [key, value]) => + query.where(key as keyof C & string, value), + this + ); + } + return this.comparison( + 'and', + first as ReadPredicateSelector | ReadPredicateGroup, + args + ); + } + + /** Explicit AND spelling of where(), including nested groups. */ + andWhere(group: ReadPredicateGroup): this; + /** Match all property/value pairs with AND semantics. */ + andWhere(values: Partial>): this; + /** Add a bound AND equality comparison. */ + andWhere(column: ReadPredicateSelector, value: unknown): this; + /** Add a bound AND comparison. */ + andWhere( + column: ReadPredicateSelector, + operator: string, + value: unknown + ): this; + andWhere( + first: + | ReadPredicateSelector + | ReadPredicateGroup + | Partial>, + ...args: [] | [unknown] | [string, unknown] + ): this { + if (typeof first === 'object' && first !== null && !args.length) + return this.where(first); + return this.comparison( + 'and', + first as ReadPredicateSelector | ReadPredicateGroup, + args + ); + } + + /** Add a parenthesized OR group. Use an enclosing AND group beside authorization filters. */ + orWhere(group: ReadPredicateGroup): this; + /** Match a parenthesized AND record as one alternative to the preceding predicates. */ + orWhere(values: Partial>): this; + /** Add a bound OR equality comparison. */ + orWhere(column: ReadPredicateSelector, value: unknown): this; + /** Add a bound OR comparison. */ + orWhere( + column: ReadPredicateSelector, + operator: string, + value: unknown + ): this; + orWhere( + first: + | ReadPredicateSelector + | ReadPredicateGroup + | Partial>, + ...args: [] | [unknown] | [string, unknown] + ): this { + if (typeof first === 'object' && first !== null && !args.length) + return this.orWhere(group => group.where(first)); + return this.comparison( + 'or', + first as ReadPredicateSelector | ReadPredicateGroup, + args + ); + } + + private comparison( + boolean: 'and' | 'or', + first: ReadPredicateSelector | ReadPredicateGroup, + args: [] | [unknown] | [string, unknown] + ): this { + const context = this.readPredicateContext(); + const method = boolean === 'and' ? 'where' : 'orWhere'; + if (!args.length) { + const group = new ReadPredicateBuilder(context); + let operations: readonly ReadPredicate[]; + { + const result: unknown = (first as ReadPredicateGroup)(group); + if ( + result && + typeof (result as PromiseLike).then === 'function' + ) { + // Consume native async rejection without assimilating foreign + // thenables (a Knex query's then() would execute SQL). + if (result instanceof Promise) void result.catch(() => {}); + throw new ReadSchemaError( + 'Read predicate groups must be synchronous' + ); + } + if ( + !(result instanceof ReadPredicateBuilder) || + !result.sameSource(group) + ) + throw new ReadSchemaError( + 'Predicate callbacks must return a builder from the supplied group' + ); + operations = result[finishGroup](); + } + return this.addReadPredicate(query => { + query[method](nested => { + for (const operation of operations) operation(nested); + }); + }); + } + const operator = args.length === 1 ? '=' : args[0]; + if ( + typeof operator !== 'string' || + !ALLOWED_OPS.has(operator.toLowerCase()) + ) + throw new ReadSchemaError( + `Unsupported comparison operator: ${operator}` + ); + const column = context.column(first as ReadPredicateSelector); + const value = captureValue( + context.knex, + args.length === 1 ? args[0] : args[1] + ); + return this.addReadPredicate(query => { + if (args.length === 1) query[method](column as any, value()); + else query[method](column as any, operator, value()); + }); + } + + private nullPredicate( + column: ReadPredicateSelector, + method: 'whereNull' | 'whereNotNull' | 'orWhereNull' | 'orWhereNotNull' + ): this { + const name = this.readPredicateContext().column(column); + return this.addReadPredicate(query => { + query[method](name as any); + }); + } + /** Match SQL null without changing the row schema. */ + whereNull(column: ReadPredicateSelector): this { + return this.nullPredicate(column, 'whereNull'); + } + /** Exclude SQL null without narrowing the declared row schema. */ + whereNotNull(column: ReadPredicateSelector): this { + return this.nullPredicate(column, 'whereNotNull'); + } + /** Add an OR SQL-null condition. */ + orWhereNull(column: ReadPredicateSelector): this { + return this.nullPredicate(column, 'orWhereNull'); + } + /** Add an OR SQL-not-null condition. */ + orWhereNotNull(column: ReadPredicateSelector): this { + return this.nullPredicate(column, 'orWhereNotNull'); + } + + private membership( + column: ReadPredicateSelector, + values: ReadMembership, + method: 'whereIn' | 'whereNotIn' | 'orWhereIn' | 'orWhereNotIn' + ): this { + const context = this.readPredicateContext(); + const name = context.column(column); + const captured = Array.isArray(values) + ? values.map(value => captureValue(context.knex, value)) + : captureSubquery(context.knex, values as Knex.QueryBuilder); + return this.addReadPredicate(query => { + if (typeof captured === 'function') { + const rawMethod = method.startsWith('or') + ? 'orWhereRaw' + : 'whereRaw'; + const operator = method.includes('Not') ? 'not in' : 'in'; + query[rawMethod](`?? ${operator} (?)`, [name, captured()]); + } else { + query[method]( + name as any, + captured.map(value => value()) + ); + } + }); + } + /** Match captured values or a SELECT subquery. An empty list matches no rows. */ + whereIn(column: ReadPredicateSelector, values: ReadMembership): this { + return this.membership(column, values, 'whereIn'); + } + /** Exclude captured values or a SELECT subquery. SQL NOT IN null semantics apply. */ + whereNotIn(column: ReadPredicateSelector, values: ReadMembership): this { + return this.membership(column, values, 'whereNotIn'); + } + /** Add an OR membership condition. */ + orWhereIn(column: ReadPredicateSelector, values: ReadMembership): this { + return this.membership(column, values, 'orWhereIn'); + } + /** Add an OR negative membership condition. */ + orWhereNotIn( + column: ReadPredicateSelector, + values: ReadMembership + ): this { + return this.membership(column, values, 'orWhereNotIn'); + } + + private exists( + subquery: Knex.QueryBuilder, + method: + | 'whereExists' + | 'whereNotExists' + | 'orWhereExists' + | 'orWhereNotExists' + ): this { + const captured = captureSubquery( + this.readPredicateContext().knex, + subquery + ); + return this.addReadPredicate(query => { + const rawMethod = method.startsWith('or') + ? 'orWhereRaw' + : 'whereRaw'; + const operator = method.includes('Not') ? 'not exists' : 'exists'; + query[rawMethod](`${operator} (?)`, [captured()]); + }); + } + /** Require a row in a captured SELECT subquery; use ref() to correlate it. */ + whereExists(subquery: Knex.QueryBuilder): this { + return this.exists(subquery, 'whereExists'); + } + /** Require no rows in a captured SELECT subquery. */ + whereNotExists(subquery: Knex.QueryBuilder): this { + return this.exists(subquery, 'whereNotExists'); + } + /** Add an OR EXISTS predicate. */ + orWhereExists(subquery: Knex.QueryBuilder): this { + return this.exists(subquery, 'orWhereExists'); + } + /** Add an OR NOT EXISTS predicate. */ + orWhereNotExists(subquery: Knex.QueryBuilder): this { + return this.exists(subquery, 'orWhereNotExists'); + } + + /** Trusted SQL predicate with positional value (?) and identifier (??) bindings; not a SQL sandbox. */ + whereRaw(sql: string, bindings: readonly Knex.RawBinding[] = []): this { + const captured = captureReadRaw( + this.readPredicateContext().knex, + sql, + bindings + ); + return this.addReadPredicate(query => { + query.whereRaw(captured()); + }); + } + /** Add an OR trusted SQL predicate with captured positional bindings. */ + orWhereRaw(sql: string, bindings: readonly Knex.RawBinding[] = []): this { + const captured = captureReadRaw( + this.readPredicateContext().knex, + sql, + bindings + ); + return this.addReadPredicate(query => { + query.orWhereRaw(captured()); + }); + } + + /** Negated bound equality (null uses SQL IS NOT NULL). */ + whereNot(column: ReadPredicateSelector, value: unknown): this { + const context = this.readPredicateContext(); + const name = context.column(column); + const captured = captureValue(context.knex, value); + return this.addReadPredicate(query => { + query.whereNot(name as any, captured()); + }); + } + /** Match an inclusive range of captured values. */ + whereBetween( + column: ReadPredicateSelector, + range: readonly [unknown, unknown] + ): this { + return this.range(column, range, false); + } + /** Exclude an inclusive range of captured values. */ + whereNotBetween( + column: ReadPredicateSelector, + range: readonly [unknown, unknown] + ): this { + return this.range(column, range, true); + } + private range( + column: ReadPredicateSelector, + range: readonly [unknown, unknown], + not: boolean + ): this { + const context = this.readPredicateContext(); + const name = context.column(column); + const values = range.map(value => captureValue(context.knex, value)); + return this.addReadPredicate(query => { + query[not ? 'whereNotBetween' : 'whereBetween'](name as any, [ + values[0](), + values[1]() + ]); + }); + } + /** Match a SQL LIKE pattern. Wildcards retain their SQL meaning. */ + whereLike(column: ReadPredicateSelector, value: string): this { + return this.where(column, 'like', value); + } + /** Match a case-insensitive PostgreSQL pattern. */ + whereILike(column: ReadPredicateSelector, value: string): this { + return this.where(column, 'ilike', value); + } + /** Compare a JSON path while preserving the declared output schema. */ + whereJsonPath( + column: ReadPredicateSelector, + path: string, + operator = '=', + value?: unknown + ): this { + const context = this.readPredicateContext(); + const name = context.column(column); + const client = context.knex.client.config.client; + if (!['pg', 'postgres', 'postgresql'].includes(client)) + throw new ReadSchemaError( + 'whereJsonPath() is only supported on PostgreSQL' + ); + if (operator === '@?' || operator === '@@') + return this.whereRaw(`?? ${operator === '@?' ? '@\\?' : '@@'} ?`, [ + name, + path + ]); + if (!ALLOWED_OPS.has(operator.toLowerCase())) + throw new ReadSchemaError('Unsupported JSON comparison operator'); + return this.whereRaw( + `jsonb_path_query_first(??, ?) ${operator} ?::jsonb`, + [ + name, + path.startsWith('$') ? path : `$.${path}`, + JSON.stringify(value) + ] + ); + } +} + +/** + * Predicate-only builder supplied to grouped where/andWhere/orWhere callbacks. + * Group methods return independent builders; attachment snapshots the returned group. + * No select, join, order, raw-query escape hatch, then, or execution method exists. + * Retaining this builder and mutating it after the callback throws. + */ +export class ReadPredicateBuilder extends ReadPredicates { + #context: ReadPredicateContext; + #operations: ReadPredicate[] = []; + /** @internal Created only for grouped predicates. */ + constructor(context: ReadPredicateContext) { + super(); + this.#context = context; + } + protected readPredicateContext(): ReadPredicateContext { + return this.#context; + } + protected addReadPredicate(predicate: ReadPredicate): this { + const copy = new ReadPredicateBuilder(this.#context); + copy.#operations = [...this.#operations, predicate]; + return copy as this; + } + /** @internal Verify that a returned builder belongs to the supplied group. */ + sameSource(other: ReadPredicateBuilder): boolean { + return this.#context === other.#context; + } + /** @internal Snapshot the immutable predicate list. */ + [finishGroup](): readonly ReadPredicate[] { + return [...this.#operations]; + } +} diff --git a/libs/knex-schema/src/read-projection.ts b/libs/knex-schema/src/read-projection.ts index 3d1251ff..923d5e2d 100644 --- a/libs/knex-schema/src/read-projection.ts +++ b/libs/knex-schema/src/read-projection.ts @@ -25,7 +25,10 @@ export type ReadField = { export function compileReadProjection( knex: Knex, selection: Record | AggregateExpression>, - resolve: (column: AliasedColumn) => { name: string; node: ReadNode } + resolve: (column: AliasedColumn) => { + name: string | Knex.Raw; + node: ReadNode; + } ): Record { if (!Object.keys(selection).length) throw new ReadSchemaError('A non-empty projection is required'); diff --git a/libs/knex-schema/src/read-schema.test-d.ts b/libs/knex-schema/src/read-schema.test-d.ts index 21da993e..5add4404 100644 --- a/libs/knex-schema/src/read-schema.test-d.ts +++ b/libs/knex-schema/src/read-schema.test-d.ts @@ -13,9 +13,11 @@ const Task = object({ done: date().optional() }).hasTableName('tasks'); test('projection row type equals its runtime schema inference', async () => { - const read = query(Knex({ client: 'pg' }), Task) - .withRowSchema() - .select(t => ({ id: t.id, amount: t.amount, done: t.done })); + const read = query(Knex({ client: 'pg' }), Task).select(t => ({ + id: t.id, + amount: t.amount, + done: t.done + })); type Row = { id: number; amount: string | null; done: Date | null }; expectTypeOf>().toEqualTypeOf(); expectTypeOf(await read).toEqualTypeOf(); @@ -25,7 +27,7 @@ test('projection row type equals its runtime schema inference', async () => { test('flat joins retain exact storage and left join nullability', async () => { const read = query(Knex({ client: 'pg' }), alias(Task, 'task')) - .withRowSchema() + .leftJoin(alias(Task, 'other'), t => eq(t.task.id, t.other.id)) .select(t => ({ amount: t.task.amount, @@ -56,18 +58,16 @@ test('optional declared joins and explicit relation customizers retain their sha u => u.id, { optional: true } ); - const read = query(Knex({ client: 'pg' }), entity.schema) - .withRowSchema() - .include( - r => r.owner, - q => q.select(u => ({ name: u.name })) - ); + const read = query(Knex({ client: 'pg' }), entity.schema).include( + r => r.owner, + q => q.select(u => ({ name: u.name })) + ); const row = (await read)[0]; expectTypeOf(row.owner).toExtend<{ name: string } | null>(); // @ts-expect-error the relation may be null const _required: { name: string } = row.owner; const explicit = query(Knex({ client: 'pg' }), Task) - .withRowSchema() + .select(t => ({ title: t.title })) .joinMany( { diff --git a/libs/knex-schema/src/read-schema.test.ts b/libs/knex-schema/src/read-schema.test.ts index 41db89f3..bb9a215d 100644 --- a/libs/knex-schema/src/read-schema.test.ts +++ b/libs/knex-schema/src/read-schema.test.ts @@ -16,13 +16,11 @@ const knex = Knex({ client: 'pg' }); describe('schema-aware read metadata', () => { it('describes only projected fields, SQL nulls and exact storage', () => { - const read = query(knex, Task) - .withRowSchema() - .select(t => ({ - title: t.title, - amount: t.amount, - done: t.completedAt - })); + const read = query(knex, Task).select(t => ({ + title: t.title, + amount: t.amount, + done: t.completedAt + })); const properties = read.rowSchema.introspect().properties; expect(Object.keys(properties)).toEqual(['title', 'amount', 'done']); expect( @@ -41,7 +39,7 @@ describe('schema-aware read metadata', () => { expect(read.toQuery()).not.toContain('"id" as'); }); it('preserves schema identity across immutable filters and transactions', () => { - const read = query(knex, Task).withRowSchema(); + const read = query(knex, Task); const one = read.where(t => t.id, 1).limit(1); const two = read.where(t => t.id, 2).offset(1); expect(one.rowSchema).toBe(read.rowSchema); @@ -88,58 +86,48 @@ describe('schema-aware read metadata', () => { ).toBe('2026-01-02T01:04:05.000Z'); }); it('retains aggregate output schemas and rejects opaque parsers', () => { - const read = query(knex, Task) - .withRowSchema() - .select(t => ({ - count: aggregate.count(), - total: aggregate.sum(t.amount) - })); + const read = query(knex, Task).select(t => ({ + count: aggregate.count(), + total: aggregate.sum(t.amount) + })); expect(read.rowSchema.validate({ count: 3, total: null }).valid).toBe( true ); expect(() => - query(knex, Task) - .withRowSchema() - .select(() => ({ - count: aggregate.count(undefined, { - output: { parse: () => 1 } - }) - })) + query(knex, Task).select(() => ({ + count: aggregate.count(undefined, { + output: { parse: () => 1 } + }) + })) ).toThrow(/introspectable/); }); - it('rejects entry after legacy shape changes', () => { - expect(() => + it('exposes metadata automatically and requires schemas for opaque SQL', () => { + expect( query(knex, Task) .orderBy(t => t.id) - .withRowSchema() - ).toThrow(/before ordering/); - expect(() => query(knex, Task).limit(1).withRowSchema()).toThrow( - /before ordering/ - ); - expect(() => query(knex, Task).offset(1).withRowSchema()).toThrow( - /before ordering/ - ); - expect(() => - query(knex, Task) - .select(t => ({ id: t.id })) - .withRowSchema() - ).toThrow(/before/); + .limit(1).rowSchema + ).toBeDefined(); + expect( + Object.keys( + query(knex, Task) + .select(t => ({ id: t.id })) + .rowSchema.introspect().properties + ) + ).toEqual(['id']); expect(() => - query(knex, Task).selectRaw('1 as other').withRowSchema() - ).toThrow(); + (query(knex, Task) as any).selectRaw('1 as other', []) + ).toThrow(/output/); expect(() => - query(knex, Task) - .apply(q => q.whereRaw('true')) - .withRowSchema() - ).toThrow(/before/); + (query(knex, Task) as any).apply((q: any) => q.whereRaw('true')) + ).toThrow(/output/); expect(() => - query(knex, alias(Task, 'task')) - .apply(q => q.select('id')) - .withRowSchema() - ).toThrow(/before/); + (query(knex, alias(Task, 'task')) as any).apply((q: any) => + q.select('id') + ) + ).toThrow(/output/); }); it('describes immutable flat left joins with SQL nulls', () => { - const base = query(knex, alias(Task, 'task')).withRowSchema(); + const base = query(knex, alias(Task, 'task')); expect(() => base.rowSchema).toThrow(/explicit select/); const joined = base .leftJoin(alias(Task, 'other'), t => eq(t.task.id, t.other.id)) diff --git a/libs/knex-schema/src/read-schema.ts b/libs/knex-schema/src/read-schema.ts index a075e2f2..16e8399f 100644 --- a/libs/knex-schema/src/read-schema.ts +++ b/libs/knex-schema/src/read-schema.ts @@ -13,6 +13,7 @@ import { union } from '@cleverbrush/schema'; import type { Knex } from 'knex'; +import { buildColumnMap } from './columns.js'; /** A schema builder accepted by the database-read schema compiler. */ export type ReadSchema = SchemaBuilder; @@ -298,9 +299,36 @@ export function decodeObject( export function readExpression( knex: Knex, node: ReadNode, - column: string + column: string | Knex.Raw ): Knex.Raw { return node.exact || node.schema.introspect().type === 'date' ? knex.raw('cast(?? as text)', [column]) : knex.raw('??', [column]); } + +/** @internal Preserve exact numerics and dates before driver parsing of write-returning rows. */ +export function returningReadColumns( + knex: Knex, + source: ReadObject +): Knex.Raw[] { + const info = source.introspect(); + const excluded = new Set( + ((info.extensions?.relations ?? []) as { name: string }[]).map( + relation => relation.name + ) + ); + const { propToCol } = buildColumnMap(source); + return Object.entries(info.properties) + .filter(([key]) => !excluded.has(key)) + .map(([key, schema]) => { + const column = propToCol.get(key) ?? key; + return knex.raw('? as ??', [ + readExpression( + knex, + compileReadSchema(schema as ReadSchema), + column + ), + column + ]); + }); +} diff --git a/libs/knex-schema/src/types.ts b/libs/knex-schema/src/types.ts index cd070d25..8eef5663 100644 --- a/libs/knex-schema/src/types.ts +++ b/libs/knex-schema/src/types.ts @@ -7,7 +7,10 @@ import type { PropertyDescriptorTree } from '@cleverbrush/schema'; import type { Knex } from 'knex'; +import type { SchemaProps } from './entity.js'; import type { AggregateExpression } from './expressions.js'; +import type { ReadRelations } from './read-entity.js'; +import type { ObjectReadSchema, ReadValue } from './read-schema.js'; // --------------------------------------------------------------------------- // Utility: extract string keys from an ObjectSchemaBuilder's inferred type @@ -258,18 +261,16 @@ export type ValidatedSpec = */ export type InsertType< T extends ObjectSchemaBuilder -> = InferType>; +> = { + [K in Exclude, keyof ReadRelations>]?: + | InferType[K]> + | ReadValue[K]>; +}; // --------------------------------------------------------------------------- // Database row helpers // --------------------------------------------------------------------------- -type OptionalKeys = { - [K in keyof T]-?: undefined extends T[K] ? K : never; -}[keyof T]; - -type RequiredKeys = Exclude>; - /** * Normalize a schema-inferred property type to the value shape commonly * returned by database rows. @@ -290,11 +291,7 @@ export type InferDatabaseValue = undefined extends T */ export type InferDatabaseRow< T extends ObjectSchemaBuilder -> = { - [K in RequiredKeys>]: InferDatabaseValue[K]>; -} & { - [K in OptionalKeys>]?: InferDatabaseValue[K]>; -}; +> = InferType>>; // --------------------------------------------------------------------------- // Primary-key type helpers (driven by PRIMARY_KEY_BRAND / COMPOSITE_PRIMARY_KEY_BRAND) @@ -379,7 +376,12 @@ export type PrimaryKeyValueOf< ? PkTupleValue> : never : PrimaryKeyOf extends string - ? InferType[PrimaryKeyOf & keyof InferType] + ? + | InferType[PrimaryKeyOf & keyof InferType] + | ReadValue< + SchemaPropsForPk[PrimaryKeyOf & + keyof SchemaPropsForPk] + > : never; /** @@ -392,7 +394,11 @@ type PkTupleValue< TKeys extends readonly string[] > = { [I in keyof TKeys]: TKeys[I] extends keyof InferType - ? InferType[TKeys[I]] + ? + | InferType[TKeys[I]] + | ReadValue< + SchemaPropsForPk[TKeys[I] & keyof SchemaPropsForPk] + > : unknown; }; @@ -536,7 +542,7 @@ export interface RelationSpec { name: string; schema: any; foreignKey?: any; - /** Explicit schema property names retained for opt-in read correlation. */ + /** Explicit schema property names used for read correlation. */ localKey?: string; remoteKey?: string; /** Nullable belongs-to relations do not filter out their parent rows. */ diff --git a/libs/knex-schema/src/unit.test.ts b/libs/knex-schema/src/unit.test.ts index 18a70a4e..8987ff3c 100644 --- a/libs/knex-schema/src/unit.test.ts +++ b/libs/knex-schema/src/unit.test.ts @@ -18,6 +18,19 @@ import { string } from './index.js'; +import { QuerySource } from './QuerySource.js'; +import type { ReadObject } from './read-schema.js'; + +// Preserve low-level SQL planner regressions independently of public immutable +// queries (covered by immutable-query, read-predicates and PostgreSQL suites). +function privateSource( + connection: KnexType, + schema: S, + base?: KnexType.QueryBuilder +) { + return new QuerySource(connection, schema, base); +} + // ═══════════════════════════════════════════════════════════════════════════ // Test schemas // ═══════════════════════════════════════════════════════════════════════════ @@ -180,43 +193,43 @@ describe('resolveColumnRef', () => { // Query builder — SQL snapshot tests // ═══════════════════════════════════════════════════════════════════════════ -describe('SchemaQueryBuilder', () => { +describe('private QuerySource planner', () => { describe('basic SELECT', () => { it('produces SELECT * FROM table', () => { - const sql = query(knex, User).toQuery(); + const sql = privateSource(knex, User).toQuery(); expect(sql).toBe('select * from "users"'); }); it('produces SELECT with SimpleTag (no column mapping needed)', () => { - const sql = query(knex, SimpleTag).toQuery(); + const sql = privateSource(knex, SimpleTag).toQuery(); expect(sql).toBe('select * from "tags"'); }); }); describe('WHERE', () => { it('.where with property descriptor', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .where(t => t.fullName, '=', 'John') .toQuery(); expect(sql).toContain('"full_name" = \'John\''); }); it('.where with string property key', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .where('fullName', '=', 'John') .toQuery(); expect(sql).toContain('"full_name" = \'John\''); }); it('.where with default column name', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .where('email', '=', 'test@test.com') .toQuery(); expect(sql).toContain('"email" = \'test@test.com\''); }); it('.where with record syntax', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .where({ fullName: 'John', role: 'admin' }) .toQuery(); expect(sql).toContain('"full_name" = \'John\''); @@ -224,7 +237,7 @@ describe('SchemaQueryBuilder', () => { }); it('.where with callback (knex sub-builder)', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .where((builder: KnexType.QueryBuilder) => { builder.where('role', 'admin'); }) @@ -233,42 +246,42 @@ describe('SchemaQueryBuilder', () => { }); it('.whereIn with property descriptor', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .whereIn(t => t.role, ['admin', 'user']) .toQuery(); expect(sql).toContain("\"role\" in ('admin', 'user')"); }); it('.whereNotIn', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .whereNotIn('role', ['banned']) .toQuery(); expect(sql).toContain('"role" not in (\'banned\')'); }); it('.whereNull with descriptor', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .whereNull(t => t.managerId) .toQuery(); expect(sql).toContain('"manager_id" is null'); }); it('.whereNotNull', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .whereNotNull(t => t.managerId) .toQuery(); expect(sql).toContain('"manager_id" is not null'); }); it('.whereBetween', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .whereBetween(t => t.departmentId, [1, 10]) .toQuery(); expect(sql).toContain('"department_id" between 1 and 10'); }); it('.andWhere / .orWhere chaining', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .where('role', '=', 'admin') .andWhere(t => t.departmentId, '>', 5) .orWhere('email', 'like', '%@co.com') @@ -279,7 +292,7 @@ describe('SchemaQueryBuilder', () => { }); it('.whereRaw passthrough', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .whereRaw('full_name ILIKE ?', '%smith%') .toQuery(); expect(sql).toContain("full_name ILIKE '%smith%'"); @@ -288,21 +301,24 @@ describe('SchemaQueryBuilder', () => { describe('ORDER BY', () => { it('.orderBy with descriptor', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .orderBy(t => t.createdAt, 'desc') .toQuery(); expect(sql).toContain('order by "created_at" desc'); }); it('.orderBy with string key', () => { - const sql = query(knex, User).orderBy('fullName').toQuery(); + const sql = privateSource(knex, User).orderBy('fullName').toQuery(); expect(sql).toContain('order by "full_name"'); }); }); describe('LIMIT / OFFSET', () => { it('.limit and .offset', () => { - const sql = query(knex, User).limit(10).offset(20).toQuery(); + const sql = privateSource(knex, User) + .limit(10) + .offset(20) + .toQuery(); expect(sql).toContain('limit 10'); expect(sql).toContain('offset 20'); }); @@ -310,14 +326,14 @@ describe('SchemaQueryBuilder', () => { describe('GROUP BY / HAVING', () => { it('.groupBy with descriptor', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .groupBy(t => t.role) .toQuery(); expect(sql).toContain('group by "role"'); }); it('.having', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .groupBy('role') .having('role', '!=', 'banned') .toQuery(); @@ -328,7 +344,7 @@ describe('SchemaQueryBuilder', () => { describe('SELECT / DISTINCT', () => { it('.select with descriptors', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .select( t => t.fullName, t => t.email @@ -339,7 +355,7 @@ describe('SchemaQueryBuilder', () => { }); it('.distinct with descriptor', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .distinct(t => t.role) .toQuery(); expect(sql).toContain('distinct "role"'); @@ -348,7 +364,7 @@ describe('SchemaQueryBuilder', () => { describe('escape hatch', () => { it('.apply passes through to knex builder', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .apply(qb => { qb.where('id', '>', 100); }) @@ -359,7 +375,7 @@ describe('SchemaQueryBuilder', () => { describe('chained queries', () => { it('complex chained query', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .where(t => t.role, '=', 'admin') .andWhere(t => t.departmentId, '>', 5) .whereNotNull(t => t.managerId) @@ -385,7 +401,7 @@ describe('SchemaQueryBuilder', () => { describe('eager loading', () => { it('joinOne produces CTE with jsonb_agg', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .joinOne({ localColumn: t => t.departmentId, foreignColumn: t => t.id, @@ -401,7 +417,7 @@ describe('eager loading', () => { }); it('joinMany produces CTE with jsonb_agg and coalesce', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .joinMany({ localColumn: t => t.id, foreignColumn: t => t.authorId, @@ -417,7 +433,7 @@ describe('eager loading', () => { }); it('joinOne with string column refs', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .joinOne({ localColumn: 'departmentId', foreignColumn: 'id', @@ -430,7 +446,7 @@ describe('eager loading', () => { }); it('joinMany with limit and orderBy', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .joinMany({ localColumn: t => t.id, foreignColumn: t => t.authorId, @@ -446,7 +462,7 @@ describe('eager loading', () => { }); it('chained joinOne + joinMany', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .joinOne({ localColumn: t => t.departmentId, foreignColumn: t => t.id, @@ -468,7 +484,7 @@ describe('eager loading', () => { }); it('joinOne with explicit foreignQuery as raw knex', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .joinOne({ localColumn: t => t.departmentId, foreignColumn: t => t.id, @@ -482,13 +498,13 @@ describe('eager loading', () => { }); it('joinOne with SchemaQueryBuilder as foreignQuery', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .joinOne({ localColumn: t => t.departmentId, foreignColumn: t => t.id, as: 'department', foreignSchema: Department, - foreignQuery: query(knex, Department).where( + foreignQuery: privateSource(knex, Department).where( t => t.budget, '>', 1000 @@ -500,7 +516,7 @@ describe('eager loading', () => { }); it('SchemaQueryBuilder foreignQuery produces same SQL as raw knex foreignQuery', () => { - const rawSql = query(knex, User) + const rawSql = privateSource(knex, User) .joinOne({ localColumn: t => t.departmentId, foreignColumn: t => t.id, @@ -510,13 +526,13 @@ describe('eager loading', () => { }) .toQuery(); - const schemaSql = query(knex, User) + const schemaSql = privateSource(knex, User) .joinOne({ localColumn: t => t.departmentId, foreignColumn: t => t.id, as: 'department', foreignSchema: Department, - foreignQuery: query(knex, Department).where( + foreignQuery: privateSource(knex, Department).where( t => t.budget, '>', 1000 @@ -528,13 +544,17 @@ describe('eager loading', () => { }); it('joinMany with SchemaQueryBuilder as foreignQuery', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .joinMany({ localColumn: t => t.id, foreignColumn: t => t.authorId, as: 'posts', foreignSchema: Post, - foreignQuery: query(knex, Post).where(t => t.categoryId, '=', 5) + foreignQuery: privateSource(knex, Post).where( + t => t.categoryId, + '=', + 5 + ) }) .toQuery(); @@ -542,7 +562,7 @@ describe('eager loading', () => { }); it('joinMany SchemaQueryBuilder foreignQuery matches raw knex', () => { - const rawSql = query(knex, User) + const rawSql = privateSource(knex, User) .joinMany({ localColumn: t => t.id, foreignColumn: t => t.authorId, @@ -552,13 +572,17 @@ describe('eager loading', () => { }) .toQuery(); - const schemaSql = query(knex, User) + const schemaSql = privateSource(knex, User) .joinMany({ localColumn: t => t.id, foreignColumn: t => t.authorId, as: 'posts', foreignSchema: Post, - foreignQuery: query(knex, Post).where(t => t.categoryId, '=', 5) + foreignQuery: privateSource(knex, Post).where( + t => t.categoryId, + '=', + 5 + ) }) .toQuery(); @@ -567,7 +591,7 @@ describe('eager loading', () => { it('throws on duplicate field names', () => { expect(() => { - query(knex, User) + privateSource(knex, User) .joinOne({ localColumn: t => t.departmentId, foreignColumn: t => t.id, @@ -586,7 +610,7 @@ describe('eager loading', () => { it('joinOne with .select() still includes localColumn in CTE', () => { // If the caller uses .select() and omits the join key (departmentId), // the generated SQL should still include it in the CTE so the join works. - const sql = query(knex, User) + const sql = privateSource(knex, User) .select(t => t.fullName) // intentionally omit departmentId .joinOne({ localColumn: t => t.departmentId, @@ -603,7 +627,7 @@ describe('eager loading', () => { }); it('joinMany with .select() still includes localColumn in CTE', () => { - const sql = query(knex, User) + const sql = privateSource(knex, User) .select(t => t.fullName) // intentionally omit id (localColumn for joinMany) .joinMany({ localColumn: t => t.id, @@ -683,7 +707,7 @@ describe('mappers', () => { describe('validateMappers via joinOne/joinMany', () => { it('accepts a function mapper in joinOne spec', () => { expect(() => - query(knex, User).joinOne({ + privateSource(knex, User).joinOne({ localColumn: t => t.departmentId, foreignColumn: t => t.id, as: 'department', @@ -695,7 +719,7 @@ describe('mappers', () => { it('accepts a built-in string mapper name in joinOne spec', () => { expect(() => - query(knex, Post).joinOne({ + privateSource(knex, Post).joinOne({ localColumn: t => t.authorId, foreignColumn: t => t.id, as: 'author', @@ -707,7 +731,7 @@ describe('mappers', () => { it('rejects an unknown built-in string mapper name in joinOne spec', () => { expect(() => - query(knex, User).joinOne({ + privateSource(knex, User).joinOne({ localColumn: t => t.departmentId, foreignColumn: t => t.id, as: 'department', @@ -719,7 +743,7 @@ describe('mappers', () => { it('accepts a built-in string mapper name in joinMany spec', () => { expect(() => - query(knex, User).joinMany({ + privateSource(knex, User).joinMany({ localColumn: t => t.id, foreignColumn: t => t.authorId, as: 'posts', @@ -731,7 +755,7 @@ describe('mappers', () => { it('rejects a non-function, non-string mapper value', () => { expect(() => - query(knex, User).joinOne({ + privateSource(knex, User).joinOne({ localColumn: t => t.departmentId, foreignColumn: t => t.id, as: 'department', @@ -750,16 +774,18 @@ describe('mappers', () => { // keys → column names; all other knex behaviour is preserved unchanged. // ═══════════════════════════════════════════════════════════════════════════ -describe('SQL parity with raw knex', () => { +describe('private planner SQL parity with raw knex', () => { // ── SELECT ──────────────────────────────────────────────────────────── it('SELECT *', () => { - expect(query(knex, User).toQuery()).toBe(knex('users').toQuery()); + expect(privateSource(knex, User).toQuery()).toBe( + knex('users').toQuery() + ); }); it('SELECT specific columns (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .select( t => t.fullName, t => t.email, @@ -772,38 +798,38 @@ describe('SQL parity with raw knex', () => { }); it('SELECT specific columns (string key)', () => { - expect(query(knex, User).select('fullName', 'email').toQuery()).toBe( - knex('users').select('full_name', 'email').toQuery() - ); + expect( + privateSource(knex, User).select('fullName', 'email').toQuery() + ).toBe(knex('users').select('full_name', 'email').toQuery()); }); it('SELECT — identity mapping (no hasColumnName)', () => { - expect(query(knex, SimpleTag).select('id', 'name').toQuery()).toBe( - knex('tags').select('id', 'name').toQuery() - ); + expect( + privateSource(knex, SimpleTag).select('id', 'name').toQuery() + ).toBe(knex('tags').select('id', 'name').toQuery()); }); // ── DISTINCT ────────────────────────────────────────────────────────── it('DISTINCT (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .distinct(t => t.role) .toQuery() ).toBe(knex('users').distinct('role').toQuery()); }); it('DISTINCT (string key)', () => { - expect(query(knex, User).distinct('departmentId').toQuery()).toBe( - knex('users').distinct('department_id').toQuery() - ); + expect( + privateSource(knex, User).distinct('departmentId').toQuery() + ).toBe(knex('users').distinct('department_id').toQuery()); }); // ── WHERE ───────────────────────────────────────────────────────────── it('.where (operator, descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .where(t => t.fullName, '=', 'Alice') .toQuery() ).toBe(knex('users').where('full_name', '=', 'Alice').toQuery()); @@ -811,19 +837,19 @@ describe('SQL parity with raw knex', () => { it('.where (operator, string key)', () => { expect( - query(knex, User).where('fullName', '=', 'Alice').toQuery() + privateSource(knex, User).where('fullName', '=', 'Alice').toQuery() ).toBe(knex('users').where('full_name', '=', 'Alice').toQuery()); }); it('.where (operator, identity column)', () => { - expect(query(knex, User).where('email', '=', 'a@b.com').toQuery()).toBe( - knex('users').where('email', '=', 'a@b.com').toQuery() - ); + expect( + privateSource(knex, User).where('email', '=', 'a@b.com').toQuery() + ).toBe(knex('users').where('email', '=', 'a@b.com').toQuery()); }); it('.where (record)', () => { expect( - query(knex, User) + privateSource(knex, User) .where({ fullName: 'Alice', role: 'admin' }) .toQuery() ).toBe( @@ -833,7 +859,7 @@ describe('SQL parity with raw knex', () => { it('.where (callback)', () => { expect( - query(knex, User) + privateSource(knex, User) .where((b: KnexType.QueryBuilder) => { b.where('role', 'admin'); }) @@ -849,7 +875,7 @@ describe('SQL parity with raw knex', () => { it('.andWhere (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .where('role', '=', 'admin') .andWhere(t => t.departmentId, '>', 5) .toQuery() @@ -863,7 +889,7 @@ describe('SQL parity with raw knex', () => { it('.orWhere (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .where('role', '=', 'admin') .orWhere(t => t.role, '=', 'editor') .toQuery() @@ -877,21 +903,21 @@ describe('SQL parity with raw knex', () => { it('.whereNot (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .whereNot(t => t.role, 'banned') .toQuery() ).toBe(knex('users').whereNot('role', 'banned').toQuery()); }); it('.whereNot (string key)', () => { - expect(query(knex, User).whereNot('departmentId', 99).toQuery()).toBe( - knex('users').whereNot('department_id', 99).toQuery() - ); + expect( + privateSource(knex, User).whereNot('departmentId', 99).toQuery() + ).toBe(knex('users').whereNot('department_id', 99).toQuery()); }); it('.whereIn (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .whereIn(t => t.role, ['admin', 'editor']) .toQuery() ).toBe(knex('users').whereIn('role', ['admin', 'editor']).toQuery()); @@ -899,13 +925,15 @@ describe('SQL parity with raw knex', () => { it('.whereIn (string key, mapped column)', () => { expect( - query(knex, User).whereIn('departmentId', [1, 2, 3]).toQuery() + privateSource(knex, User) + .whereIn('departmentId', [1, 2, 3]) + .toQuery() ).toBe(knex('users').whereIn('department_id', [1, 2, 3]).toQuery()); }); it('.whereNotIn (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .whereNotIn(t => t.role, ['banned']) .toQuery() ).toBe(knex('users').whereNotIn('role', ['banned']).toQuery()); @@ -913,7 +941,7 @@ describe('SQL parity with raw knex', () => { it('.whereNull (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .whereNull(t => t.managerId) .toQuery() ).toBe(knex('users').whereNull('manager_id').toQuery()); @@ -921,7 +949,7 @@ describe('SQL parity with raw knex', () => { it('.whereNotNull (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .whereNotNull(t => t.managerId) .toQuery() ).toBe(knex('users').whereNotNull('manager_id').toQuery()); @@ -929,7 +957,7 @@ describe('SQL parity with raw knex', () => { it('.orWhereNull (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .whereNull(t => t.managerId) .orWhereNull(t => t.departmentId) .toQuery() @@ -943,7 +971,7 @@ describe('SQL parity with raw knex', () => { it('.orWhereNotNull (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .whereNull(t => t.managerId) .orWhereNotNull(t => t.departmentId) .toQuery() @@ -957,7 +985,7 @@ describe('SQL parity with raw knex', () => { it('.whereBetween (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .whereBetween(t => t.departmentId, [1, 10]) .toQuery() ).toBe(knex('users').whereBetween('department_id', [1, 10]).toQuery()); @@ -965,7 +993,7 @@ describe('SQL parity with raw knex', () => { it('.whereNotBetween (string key)', () => { expect( - query(knex, User) + privateSource(knex, User) .whereNotBetween('departmentId', [20, 30]) .toQuery() ).toBe( @@ -975,7 +1003,9 @@ describe('SQL parity with raw knex', () => { it('.whereRaw passthrough', () => { expect( - query(knex, User).whereRaw('full_name ILIKE ?', '%smith%').toQuery() + privateSource(knex, User) + .whereRaw('full_name ILIKE ?', '%smith%') + .toQuery() ).toBe( knex('users').whereRaw('full_name ILIKE ?', '%smith%').toQuery() ); @@ -985,21 +1015,21 @@ describe('SQL parity with raw knex', () => { it('.orderBy asc (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .orderBy(t => t.fullName, 'asc') .toQuery() ).toBe(knex('users').orderBy('full_name', 'asc').toQuery()); }); it('.orderBy desc (string key)', () => { - expect(query(knex, User).orderBy('createdAt', 'desc').toQuery()).toBe( - knex('users').orderBy('created_at', 'desc').toQuery() - ); + expect( + privateSource(knex, User).orderBy('createdAt', 'desc').toQuery() + ).toBe(knex('users').orderBy('created_at', 'desc').toQuery()); }); it('.orderByRaw passthrough', () => { expect( - query(knex, User) + privateSource(knex, User) .orderByRaw('"created_at" DESC NULLS LAST') .toQuery() ).toBe( @@ -1010,19 +1040,19 @@ describe('SQL parity with raw knex', () => { // ── LIMIT / OFFSET ──────────────────────────────────────────────────── it('.limit', () => { - expect(query(knex, User).limit(25).toQuery()).toBe( + expect(privateSource(knex, User).limit(25).toQuery()).toBe( knex('users').limit(25).toQuery() ); }); it('.offset', () => { - expect(query(knex, User).offset(50).toQuery()).toBe( + expect(privateSource(knex, User).offset(50).toQuery()).toBe( knex('users').offset(50).toQuery() ); }); it('.limit + .offset', () => { - expect(query(knex, User).limit(10).offset(20).toQuery()).toBe( + expect(privateSource(knex, User).limit(10).offset(20).toQuery()).toBe( knex('users').limit(10).offset(20).toQuery() ); }); @@ -1031,33 +1061,33 @@ describe('SQL parity with raw knex', () => { it('.groupBy (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .groupBy(t => t.role) .toQuery() ).toBe(knex('users').groupBy('role').toQuery()); }); it('.groupBy (string key, mapped column)', () => { - expect(query(knex, User).groupBy('departmentId').toQuery()).toBe( - knex('users').groupBy('department_id').toQuery() - ); + expect( + privateSource(knex, User).groupBy('departmentId').toQuery() + ).toBe(knex('users').groupBy('department_id').toQuery()); }); it('.groupBy multiple columns', () => { expect( - query(knex, User).groupBy('role', 'departmentId').toQuery() + privateSource(knex, User).groupBy('role', 'departmentId').toQuery() ).toBe(knex('users').groupBy('role', 'department_id').toQuery()); }); it('.groupByRaw passthrough', () => { - expect(query(knex, User).groupByRaw('"role"').toQuery()).toBe( + expect(privateSource(knex, User).groupByRaw('"role"').toQuery()).toBe( knex('users').groupByRaw('"role"').toQuery() ); }); it('.having (string key)', () => { expect( - query(knex, User) + privateSource(knex, User) .groupBy('role') .having('role', '!=', 'banned') .toQuery() @@ -1071,7 +1101,7 @@ describe('SQL parity with raw knex', () => { it('.havingRaw passthrough', () => { expect( - query(knex, User) + privateSource(knex, User) .groupBy('role') .havingRaw('count(*) > 5') .toQuery() @@ -1083,14 +1113,14 @@ describe('SQL parity with raw knex', () => { // ── AGGREGATES ──────────────────────────────────────────────────────── it('.count()', () => { - expect(query(knex, User).count().toQuery()).toBe( + expect(privateSource(knex, User).count().toQuery()).toBe( knex('users').count().toQuery() ); }); it('.count(column, descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .count(t => t.id) .toQuery() ).toBe(knex('users').count('id').toQuery()); @@ -1098,7 +1128,7 @@ describe('SQL parity with raw knex', () => { it('.countDistinct(column, descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .countDistinct(t => t.departmentId) .toQuery() ).toBe(knex('users').countDistinct('department_id').toQuery()); @@ -1106,7 +1136,7 @@ describe('SQL parity with raw knex', () => { it('.min (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .min(t => t.createdAt) .toQuery() ).toBe(knex('users').min('created_at').toQuery()); @@ -1114,20 +1144,20 @@ describe('SQL parity with raw knex', () => { it('.max (descriptor)', () => { expect( - query(knex, User) + privateSource(knex, User) .max(t => t.createdAt) .toQuery() ).toBe(knex('users').max('created_at').toQuery()); }); it('.sum (string key)', () => { - expect(query(knex, User).sum('departmentId').toQuery()).toBe( + expect(privateSource(knex, User).sum('departmentId').toQuery()).toBe( knex('users').sum('department_id').toQuery() ); }); it('.avg (string key)', () => { - expect(query(knex, User).avg('departmentId').toQuery()).toBe( + expect(privateSource(knex, User).avg('departmentId').toQuery()).toBe( knex('users').avg('department_id').toQuery() ); }); @@ -1136,7 +1166,7 @@ describe('SQL parity with raw knex', () => { it('combined: SELECT + WHERE + ORDER BY + LIMIT + OFFSET', () => { expect( - query(knex, User) + privateSource(knex, User) .select( t => t.fullName, t => t.email, @@ -1162,7 +1192,7 @@ describe('SQL parity with raw knex', () => { it('combined: WHERE complex + GROUP BY + HAVING', () => { expect( - query(knex, User) + privateSource(knex, User) .select(t => t.role) .whereIn(t => t.role, ['admin', 'editor']) .groupBy(t => t.role) @@ -1182,7 +1212,7 @@ describe('SQL parity with raw knex', () => { describe('knex.raw() as column argument', () => { it('.where(knex.raw()) — raw as full WHERE expression', () => { expect( - query(knex, User) + privateSource(knex, User) .where(knex.raw('status = ?', ['active'])) .toQuery() ).toBe( @@ -1194,7 +1224,7 @@ describe('SQL parity with raw knex', () => { it('.where(knex.raw(), operator, value) — raw as LHS column', () => { expect( - query(knex, User) + privateSource(knex, User) .where(knex.raw('"full_name"'), '=', 'Alice') .toQuery() ).toBe( @@ -1206,7 +1236,7 @@ describe('SQL parity with raw knex', () => { it('.andWhere(knex.raw())', () => { expect( - query(knex, User) + privateSource(knex, User) .where('role', '=', 'admin') .andWhere(knex.raw('deleted_at IS NULL')) .toQuery() @@ -1220,7 +1250,7 @@ describe('SQL parity with raw knex', () => { it('.orWhere(knex.raw())', () => { expect( - query(knex, User) + privateSource(knex, User) .where('role', '=', 'admin') .orWhere(knex.raw('role = ?', ['superuser'])) .toQuery() @@ -1234,7 +1264,7 @@ describe('SQL parity with raw knex', () => { it('.whereNot(knex.raw())', () => { expect( - query(knex, User) + privateSource(knex, User) .whereNot(knex.raw('deleted_at IS NULL')) .toQuery() ).toBe( @@ -1244,7 +1274,7 @@ describe('SQL parity with raw knex', () => { it('.select(knex.raw()) — computed expression', () => { expect( - query(knex, User) + privateSource(knex, User) .select(knex.raw('count(*) as total')) .toQuery() ).toBe( @@ -1254,7 +1284,7 @@ describe('SQL parity with raw knex', () => { it('.select() mixing schema column and knex.raw()', () => { expect( - query(knex, User) + privateSource(knex, User) .select(t => t.role, knex.raw('count(*) as total')) .toQuery() ).toBe( @@ -1266,13 +1296,13 @@ describe('SQL parity with raw knex', () => { it('.distinct(knex.raw())', () => { expect( - query(knex, User).distinct(knex.raw('"role"')).toQuery() + privateSource(knex, User).distinct(knex.raw('"role"')).toQuery() ).toBe(knex('users').distinct(knex.raw('"role"')).toQuery()); }); it('.orderBy(knex.raw())', () => { expect( - query(knex, User) + privateSource(knex, User) .orderBy(knex.raw('"created_at" DESC NULLS LAST')) .toQuery() ).toBe( @@ -1284,7 +1314,7 @@ describe('SQL parity with raw knex', () => { it('.groupBy(knex.raw())', () => { expect( - query(knex, User) + privateSource(knex, User) .groupBy(knex.raw("date_trunc('day', created_at)")) .toQuery() ).toBe( @@ -1296,7 +1326,7 @@ describe('SQL parity with raw knex', () => { it('.groupBy() mixing schema column and knex.raw()', () => { expect( - query(knex, User) + privateSource(knex, User) .groupBy( t => t.role, knex.raw("date_trunc('day', created_at)") @@ -1311,7 +1341,7 @@ describe('SQL parity with raw knex', () => { it('.having(knex.raw(), operator, value)', () => { expect( - query(knex, User) + privateSource(knex, User) .groupBy(t => t.role) .having(knex.raw('count(*)'), '>', 5) .toQuery() @@ -1335,7 +1365,15 @@ describe('createQuery factory', () => { const q = createQuery(knex); it('produces same SQL as query(knex, schema) for SELECT *', () => { - expect(q(User).toQuery()).toBe(query(knex, User).toQuery()); + expect( + q(User) + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') + ).toBe( + query(knex, User) + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') + ); }); it('produces same SQL for WHERE via descriptor', () => { @@ -1343,10 +1381,12 @@ describe('createQuery factory', () => { q(User) .where(t => t.fullName, '=', 'Alice') .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') ).toBe( query(knex, User) .where(t => t.fullName, '=', 'Alice') .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') ); }); @@ -1357,27 +1397,22 @@ describe('createQuery factory', () => { .orderBy(t => t.createdAt, 'desc') .limit(10) .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') ).toBe( query(knex, User) .where(t => t.role, '=', 'admin') .orderBy(t => t.createdAt, 'desc') .limit(10) .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') ); }); - it('the baseQuery overload works', () => { - const base1 = knex('users').where('deleted_at', null); - const base2 = knex('users').where('deleted_at', null); - expect( - q(User, base1) - .where(t => t.role, '=', 'admin') - .toQuery() - ).toBe( - query(knex, User, base2) - .where(t => t.role, '=', 'admin') - .toQuery() - ); + it('raw SQL extensions declare their output explicitly', () => { + const schema = object({ role: string() }); + const configured = q(User).selectRaw('role', [], { output: schema }); + expect(configured.rowSchema).toBe(schema); + expect(configured.toQuery()).toContain('select role'); }); it('works with joinOne', () => { @@ -1390,6 +1425,7 @@ describe('createQuery factory', () => { foreignSchema: Department }) .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') ).toBe( query(knex, User) .joinOne({ @@ -1399,6 +1435,7 @@ describe('createQuery factory', () => { foreignSchema: Department }) .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') ); }); @@ -1412,6 +1449,7 @@ describe('createQuery factory', () => { foreignSchema: Post }) .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') ).toBe( query(knex, User) .joinMany({ @@ -1421,6 +1459,7 @@ describe('createQuery factory', () => { foreignSchema: Post }) .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') ); }); @@ -1430,7 +1469,15 @@ describe('createQuery factory', () => { // Both produce the same SQL — knex client config doesn't affect SQL // generation without a real connection, but they must be independent objects expect(q(User)).not.toBe(q2(User)); - expect(q(User).toQuery()).toBe(q2(User).toQuery()); + expect( + q(User) + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') + ).toBe( + q2(User) + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') + ); knex2.destroy(); }); }); @@ -1462,7 +1509,8 @@ describe('transaction support', () => { const sql = query(knex, User) .where(t => t.role, '=', 'admin') .transacting(trx) - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); expect(sql).toContain('"role" = \'admin\''); }); @@ -1470,8 +1518,9 @@ describe('transaction support', () => { const sql = query(knex, User) .orderBy(t => t.createdAt, 'desc') .transacting(trx) - .toQuery(); - expect(sql).toContain('order by "created_at" desc'); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); + expect(sql).toContain('order by "__schema_read"."created_at" desc'); }); it('preserves LIMIT / OFFSET after transacting()', () => { @@ -1479,12 +1528,13 @@ describe('transaction support', () => { .limit(10) .offset(5) .transacting(trx) - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); expect(sql).toContain('limit 10'); expect(sql).toContain('offset 5'); }); - it('transacting() after joinOne preserves CTE structure', () => { + it('transacting() after joinOne preserves correlated relation SQL', () => { const sql = query(knex, User) .joinOne({ localColumn: t => t.departmentId, @@ -1493,15 +1543,16 @@ describe('transaction support', () => { foreignSchema: Department }) .transacting(trx) - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); - expect(sql).toContain('with "originalQuery" as'); + expect(sql).toContain('to_jsonb'); expect(sql).toContain('"departments"'); - expect(sql).toContain('jsonb_agg'); + expect(sql).toContain('limit 1'); expect(sql).toContain('"department"'); }); - it('transacting() after joinMany preserves CTE structure', () => { + it('transacting() after joinMany preserves correlated relation SQL', () => { const sql = query(knex, User) .joinMany({ localColumn: t => t.id, @@ -1510,9 +1561,10 @@ describe('transaction support', () => { foreignSchema: Post }) .transacting(trx) - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); - expect(sql).toContain('with "originalQuery" as'); + expect(sql).toContain('to_jsonb'); expect(sql).toContain('"posts"'); expect(sql).toContain('jsonb_agg'); expect(sql).toContain('coalesce'); @@ -1522,13 +1574,15 @@ describe('transaction support', () => { const plain = query(knex, User) .where(t => t.departmentId, '>', 3) .orderBy(t => t.fullName) - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); const transacted = query(knex, User) .where(t => t.departmentId, '>', 3) .orderBy(t => t.fullName) .transacting(trx) - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); expect(transacted).toBe(plain); }); @@ -1538,7 +1592,8 @@ describe('transaction support', () => { .transacting(trx) .where(t => t.role, '=', 'editor') .limit(20) - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); expect(sql).toContain('"role" = \'editor\''); expect(sql).toContain('limit 20'); @@ -1556,7 +1611,15 @@ describe('transaction support', () => { it('withTransaction() factory produces same SQL as query(trx, schema)', () => { const dbTrx = db.withTransaction(trx); - expect(dbTrx(User).toQuery()).toBe(query(knex, User).toQuery()); + expect( + dbTrx(User) + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') + ).toBe( + query(knex, User) + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') + ); }); it('withTransaction() factory supports chaining', () => { @@ -1565,17 +1628,24 @@ describe('transaction support', () => { .where(t => t.role, '=', 'admin') .orderBy(t => t.createdAt, 'desc') .limit(5) - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); expect(sql).toContain('"role" = \'admin\''); - expect(sql).toContain('order by "created_at" desc'); + expect(sql).toContain('order by "__schema_read"."created_at" desc'); expect(sql).toContain('limit 5'); }); it('withTransaction() does not affect the original bound factory', () => { - const plainSql = db(User).toQuery(); + const plainSql = db(User) + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); db.withTransaction(trx); // should not mutate db - expect(db(User).toQuery()).toBe(plainSql); + expect( + db(User) + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read') + ).toBe(plainSql); }); it('withTransaction() supports joinOne', () => { @@ -1587,7 +1657,8 @@ describe('transaction support', () => { as: 'department', foreignSchema: Department }) - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); expect(sql).toContain('"departments"'); expect(sql).toContain('"department"'); @@ -1650,7 +1721,8 @@ describe('transaction support', () => { await db.transaction(async dbTrx => { sql = dbTrx(User) .where(t => t.role, '=', 'admin') - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); }); } finally { spy.mockRestore(); @@ -1672,7 +1744,8 @@ describe('default extensions', () => { }).hasTableName('test'); const sql = db(Schema) .where(t => t.id, '=', '1234') - .toQuery(); + .toQuery() + .replace(/__schema_read_\d+/g, '__schema_read'); expect(sql).toContain('"id" = \'1234\''); }); }); diff --git a/libs/knex-schema/src/validate.ts b/libs/knex-schema/src/validate.ts index d54a9bf2..5a46cb73 100644 --- a/libs/knex-schema/src/validate.ts +++ b/libs/knex-schema/src/validate.ts @@ -15,7 +15,7 @@ import type { /** * Resolve foreignQuery: use the provided one, or auto-derive from * the foreign schema's tableName extension. - * Also normalizes SchemaQueryBuilder instances to raw Knex.QueryBuilder + * Also normalizes QuerySource instances to raw Knex.QueryBuilder * by calling `.toKnexQuery()` if available. */ function resolveForeignQuery( diff --git a/libs/mapper/README.md b/libs/mapper/README.md index 4f0a7a5a..f2e54b7a 100644 --- a/libs/mapper/README.md +++ b/libs/mapper/README.md @@ -246,7 +246,7 @@ typed as synchronous that returns a promise/thenable throws when invoked. `configureSync()` API. Configure once and reuse the same source/target schema instances. Queries can -provide their projection schema through `.withRowSchema().rowSchema`; see the +provide their projection schema through `.rowSchema`; see the [projection-aware read guide](../knex-schema/README.md#projection-aware-reads), including separate definition/mapping/service files and explicit polymorphic dispatch. The mapper performs no database calls or application enrichment. diff --git a/libs/orm-cli/CHANGELOG.md b/libs/orm-cli/CHANGELOG.md index 7a77319d..d39807ef 100644 --- a/libs/orm-cli/CHANGELOG.md +++ b/libs/orm-cli/CHANGELOG.md @@ -22,7 +22,7 @@ ### Minor Changes -- c75bff4: Add framework affordances discovered while reviewing Xpenser: +- c75bff4: Add reusable environment, HTTP, cache, database and migration capabilities: - `@cleverbrush/env`: add `envBoolean()` for environment-style boolean values such as `1`, `0`, `yes`, `no`, `on`, and `off`. diff --git a/libs/orm/CHANGELOG.md b/libs/orm/CHANGELOG.md index b5a7f006..a06ebe63 100644 --- a/libs/orm/CHANGELOG.md +++ b/libs/orm/CHANGELOG.md @@ -25,7 +25,7 @@ ### Minor Changes -- c75bff4: Add framework affordances discovered while reviewing Xpenser: +- c75bff4: Add reusable environment, HTTP, cache, database and migration capabilities: - `@cleverbrush/env`: add `envBoolean()` for environment-style boolean values such as `1`, `0`, `yes`, `no`, `on`, and `off`. diff --git a/libs/orm/README.md b/libs/orm/README.md index 5d78ac10..aebbc37c 100644 --- a/libs/orm/README.md +++ b/libs/orm/README.md @@ -13,6 +13,8 @@ EF-Core-like typed ORM layer on top of [`@cleverbrush/knex-schema`](../knex-sche --- +Upgrading? See [Migrating from v4.x to v5](../knex-schema/MIGRATION-v5.md). + ## Installation ```sh @@ -80,7 +82,7 @@ const updated = await db.users.save({ id: 1, email: 'alice@example.com', name: ' | `.include(t => t.rel)` | Eager-loads a relation (chainable) | | `.save(graph)` | Insert or update a row graph (transactional) | | `.ofVariant(key)` | Return a typed `VariantDbSet` scoped to a polymorphic variant | -| `.withRowSchema()` | Creates a detached read-only query with a matching result schema | +| `.rowSchema` | Automatically describes the decoded result; reading metadata runs no SQL | | `.query()` | Returns the underlying `EntityQuery` for advanced querying | | `.withTransaction(trx)` | Returns a new `DbSet` bound to an existing transaction | @@ -211,15 +213,17 @@ async function updateUser(userId: number) { --- -## Detached reads with result schemas +## Immutable queries and projection schemas -`db.users.withRowSchema()` enters an immutable read-only API whose `rowSchema` -matches its decoded selection and includes. Results remain **detached even when -the context uses `{ tracking: true }`**. This avoids attaching partial projections -as incomplete tracked entities. Ordinary entity queries keep their old behavior. +Every DbSet/query chain is immutable and automatically exposes `rowSchema`. +Full entity reads still participate in the identity map when `{ tracking: true }`. +Selected, grouped, distinct and raw results are detached; partial rows never +replace tracked entities. Entity objects themselves remain mutable: edit a full +entity and call `saveChanges()` as before. Reads, reloads and write-returning rows +now consistently use exact bigint/decimal strings, `Date` objects and SQL `null`. ```ts -const read = db.users.withRowSchema() +const read = db.users .select(u => ({ id: u.id, name: u.name })); const Source = read.rowSchema; const toDto = mapper().configure(Source, UserDto, m => m) @@ -232,6 +236,15 @@ decoded in one SQL statement. STI/CTI readers expose `variantRowSchemas` for explicit application mapping. See the [read-schema consumer guide](../knex-schema/README.md#projection-aware-reads) for numeric/null/date rules, examples and compatibility boundaries. +Ordinary queries also support grouped `where`/`andWhere`/`orWhere`, +IN/EXISTS subqueries, bound `whereRaw`/`orderByRaw`, and `ref(selector)` for quoted +column references. These operations preserve the reader's `rowSchema` identity +and work in nested relation customizers; polymorphic branches use `forVariant()`. +Group callbacks are synchronous, immutable and predicate-only; always return +the configured group. Conditional filters must reassign the returned query. See +[filtering and ordering](../knex-schema/README.md#filtering-and-ordering-without-changing-the-result-schema) +for scoped search, subquery snapshots, pagination, and raw-SQL boundaries. + --- ## Polymorphic entities (STI / CTI) @@ -382,7 +395,7 @@ const tasks = await db.tasks .orderBy(t => t.createdAt, 'desc') .orderBy(t => t.id, 'desc') .include(t => t.owner, owners => { - owners.where(t => t.name, 'Alice'); // foreign schema, not any + return owners.where(t => t.name, 'Alice'); // foreign schema, not any }) .limit(20); ``` @@ -392,9 +405,9 @@ does not necessarily filter parents. Callback types infer the declared foreign schema, including variant queries when the relation schema is known. `paginateAfter({ limit, cursor, orderBy: [...] })` supports non-null scalar sorts -with a declared unique tie-breaker; single-column cursor calls are unchanged. +with a declared unique tie-breaker. Use `column` for a single-column cursor. See the [complete query guide](../knex-schema/README.md#composable-read-queries) for defaults, -precision policy, grouped aggregates, cursor restrictions, and migration examples. +precision policy, grouped aggregates, cursor restrictions, and examples. ## Related packages diff --git a/libs/orm/src/change-tracker.ts b/libs/orm/src/change-tracker.ts index 84726117..edbd7657 100644 --- a/libs/orm/src/change-tracker.ts +++ b/libs/orm/src/change-tracker.ts @@ -15,10 +15,13 @@ import { buildColumnMap, getPrimaryKeyColumns, - getRowVersionColumn + getRowVersionColumn, + getVariants, + query as schemaQuery } from '@cleverbrush/knex-schema'; import type { Knex } from 'knex'; import { ConcurrencyError, InvariantViolationError } from './errors.js'; +import { insertVariant } from './variant-write.js'; // --------------------------------------------------------------------------- // Types @@ -210,6 +213,12 @@ function snapshotEntity(entity: object, schema: any): Record { properties?: Record; }; const propKeys = new Set(Object.keys(introspected?.properties ?? {})); + const variants = getVariants(schema); + const variant = + variants?.variants[(entity as any)[variants.discriminatorKey]]; + if (variant) + for (const key of Object.keys(variant.schema.introspect().properties)) + propKeys.add(key); const snap: Record = {}; for (const [k, v] of Object.entries(entity as Record)) { if (propKeys.has(k)) snap[k] = v; @@ -498,16 +507,15 @@ export class ChangeTracker { const tableName = config.schema.getExtension?.('tableName') as string; if (!tableName || pkInfo.propertyKeys.length === 0) return; - const { propToCol, colToProp } = buildColumnMap(config.schema); - const qb = knex(tableName); + let qb = schemaQuery(knex, config.schema as any) + .unscoped() + .withDeleted(); const pkValues = extractPkValues( config.schema, entity as Record ); for (let i = 0; i < pkInfo.propertyKeys.length; i++) { - const colName = - propToCol.get(pkInfo.propertyKeys[i]) ?? pkInfo.propertyKeys[i]; - qb.andWhere(colName, pkValues[i] as any); + qb = qb.andWhere(pkInfo.propertyKeys[i], pkValues[i]); } const row = await qb.first(); if (!row) return; @@ -517,7 +525,7 @@ export class ChangeTracker { for (const [col, val] of Object.entries( row as Record )) { - mapped[colToProp.get(col) ?? col] = val; + mapped[col] = val; } // Refresh snapshot and rowVersion entry.originalSnapshot = snapshotEntity(entity, config.schema); @@ -667,6 +675,9 @@ export class ChangeTracker { } } + // Keep generated values local until every statement has committed. + // A later concurrency failure must not advance in-memory IDs/versions. + const committedValues = new Map>(); // 4. Execute all changes in a single transaction await knex.transaction(async (trx: Knex.Transaction) => { // Inserts (Added) @@ -678,27 +689,20 @@ export class ChangeTracker { ) as string; if (!tableName) continue; - const { propToCol } = buildColumnMap(config.schema); const current = entry.entity as Record; - const row: Record = {}; - for (const [propKey, val] of Object.entries(current)) { - const colName = propToCol.get(propKey) ?? propKey; - if (val !== undefined) row[colName] = val; - } - const result = await trx(tableName).insert(row).returning('*'); - const returned = - Array.isArray(result) && result.length > 0 - ? result[0] - : null; - if (returned && typeof returned === 'object') { - const { colToProp } = buildColumnMap(config.schema); - const mapped = entry.entity as Record; - for (const [col, val] of Object.entries( - returned as Record - )) { - mapped[colToProp.get(col) ?? col] = val; - } - } + const variants = getVariants(config.schema as any); + const returned = variants + ? await insertVariant( + trx, + config.schema, + String(current[variants.discriminatorKey]), + current, + trx + ) + : await schemaQuery(trx, config.schema as any).insert( + current as any + ); + if (returned) committedValues.set(current, returned); inserted++; } @@ -718,6 +722,23 @@ export class ChangeTracker { // Build the SET clause — only changed columns, excluding PK const pkPropSet = new Set(pkInfo.propertyKeys); const updateData: Record = {}; + const variant = getVariants(config.schema)?.variants[ + entry.variantKey ?? '' + ]; + const variantColumns = variant + ? buildColumnMap(variant.schema).propToCol + : new Map(); + const variantData: Record = {}; + const put = (key: string, value: unknown) => { + const variantColumn = + !propToCol.has(key) && variantColumns.get(key); + if (variantColumn && variant?.storage === 'cti') { + if (variantColumn !== variant.foreignKey) + variantData[variantColumn] = value; + } else + updateData[variantColumn || propToCol.get(key) || key] = + value; + }; for (const propKey of Object.keys(entry.originalSnapshot)) { if (pkPropSet.has(propKey)) continue; if ( @@ -726,8 +747,7 @@ export class ChangeTracker { current[propKey] ) ) { - updateData[propToCol.get(propKey) ?? propKey] = - current[propKey]; + put(propKey, current[propKey]); } } // Also pick up new keys not in snapshot @@ -737,7 +757,7 @@ export class ChangeTracker { !(propKey in entry.originalSnapshot) && val !== undefined ) { - updateData[propToCol.get(propKey) ?? propKey] = val; + put(propKey, val); } } @@ -747,18 +767,34 @@ export class ChangeTracker { const rvCol = propToCol.get(rv.propertyKey) ?? rv.propertyKey; if (rv.strategy === 'increment') { - const newVal = Number(rv.snapshotValue ?? 0) + 1; + const newVal = + typeof rv.snapshotValue === 'string' + ? (BigInt(rv.snapshotValue) + 1n).toString() + : Number(rv.snapshotValue ?? 0) + 1; + if ( + typeof newVal === 'number' && + !Number.isSafeInteger(newVal) + ) + throw new Error( + 'Row-version increment exceeds the safe integer range; use a bigint storage column' + ); updateData[rvCol] = newVal; - current[rv.propertyKey] = newVal; + committedValues.set(current, { + [rv.propertyKey]: newVal + }); } else if (rv.strategy === 'timestamp') { const now = new Date(); updateData[rvCol] = now; - current[rv.propertyKey] = now; + committedValues.set(current, { [rv.propertyKey]: now }); } // 'manual': caller already set the new value in current } - if (Object.keys(updateData).length === 0) continue; + if ( + Object.keys(updateData).length === 0 && + Object.keys(variantData).length === 0 + ) + continue; // Build WHERE clause with PK + optional rowVersion check let qb = trx(tableName); @@ -778,7 +814,11 @@ export class ChangeTracker { qb = qb.andWhere(rvCol, rv.snapshotValue as any) as any; } - const affected = await qb.update(updateData); + const affected = Object.keys(updateData).length + ? await qb.update(updateData) + : (await qb.forUpdate().first()) + ? 1 + : 0; if (affected === 0 && entry.rowVersion) { const tableName2 = config.schema.getExtension?.( 'tableName' @@ -789,6 +829,17 @@ export class ChangeTracker { entry.rowVersion.snapshotValue ); } + if ( + affected && + variant?.storage === 'cti' && + Object.keys(variantData).length + ) + await trx(variant.tableName!) + .where( + variant.foreignKey!, + current[pkInfo.propertyKeys[0]] as any + ) + .update(variantData); updated++; } @@ -836,6 +887,9 @@ export class ChangeTracker { } }); + for (const [entity, values] of committedValues) + Object.assign(entity, values); + // 5. Refresh snapshots for inserted and updated entries; detach deleted for (const entry of added) { entry.originalSnapshot = snapshotEntity( diff --git a/libs/orm/src/dbset.ts b/libs/orm/src/dbset.ts index 35110ac4..ff5c5787 100644 --- a/libs/orm/src/dbset.ts +++ b/libs/orm/src/dbset.ts @@ -8,15 +8,18 @@ // Calling any query method (`.where()`, `.include()`, `.first()`, `.insert()`, // etc.) on a `DbSet` allocates a fresh `SchemaQueryBuilder` and forwards the // call to it. Subsequent calls on the returned `EntityQuery` reuse that -// same underlying builder. +// independent immutable builders. import type { - ColumnRef, Entity, EntityRelations, EntitySchema, + PolymorphicQueryBuilder, PrimaryKeyValueOf, - SchemaAwareQuery + ReadQueryShape, + SchemaAwareQuery, + SchemaForValue, + VariantReadSchemas } from '@cleverbrush/knex-schema'; import { getPrimaryKeyColumns, @@ -24,7 +27,7 @@ import { type SchemaQueryBuilder, query as schemaQuery } from '@cleverbrush/knex-schema'; -import type { InferType } from '@cleverbrush/schema'; +import type { InferType, ObjectSchemaBuilder } from '@cleverbrush/schema'; import type { Knex } from 'knex'; import { EntityNotFoundError } from './errors.js'; @@ -37,7 +40,6 @@ import type { VariantInsertPayload, VariantResult, VariantUpdatePayload, - WithIncluded, WithVariantIncluded } from './result-types.js'; import { saveGraph } from './save-graph.js'; @@ -47,9 +49,45 @@ import { updateVariant as _updateVariant } from './variant-write.js'; +type EntityRowSchema = ObjectSchemaBuilder< + { [K in keyof T & string]-?: SchemaForValue }, + true, + false, + T +>; +type WriteMethod = + | 'insert' + | 'insertMany' + | 'update' + | 'delete' + | 'hardDelete' + | 'restore' + | 'bulkInsert' + | 'bulkUpdate' + | 'bulkUpsert' + | 'upsert' + | 'onConflict'; +type EntityWrites< + TEntity extends Entity, + TResult, + Writable extends boolean +> = { + [K in WriteMethod]: Writable extends true + ? OmitThisParameter< + SchemaQueryBuilder< + EntitySchema, + EntityRowSchema + >[K] + > + : never; +}; + // These methods return scalars or detached read plans, not tracked entity rows. const untrackedResultMethods = new Set([ - 'withRowSchema', + 'apply', + 'selectRaw', + 'toKnexQuery', + 'pluck', 'countValue', 'countDistinctValue', 'sumValue', @@ -73,30 +111,79 @@ const untrackedResultMethods = new Set([ * * @public */ -export interface EntityQuery, TResult> - extends Omit< - SchemaQueryBuilder, TResult>, - 'include' | 'includeVariant' | 'withRowSchema' - > { - /** - * Start an immutable detached read model with inferred runtime row schemas. - * Enable before select/include operations; results never attach to tracking. - */ - withRowSchema(): SchemaAwareQuery>; +export type EntityQuery< + TEntity extends Entity, + TResult, + Writable extends boolean = true +> = + SchemaAwareQuery> extends PolymorphicQueryBuilder< + any, + any + > + ? PolymorphicEntityQuery + : TableEntityQuery; + +/** A tracked-capable polymorphic read; projections use forVariant(). */ +export interface PolymorphicEntityQuery> + extends PolymorphicQueryBuilder>, + Pick< + TableEntityQuery>, + 'find' | 'findOrFail' | 'findMany' + > {} + +/** Table query with ORM identity tracking and primary-key lookup helpers. */ +export interface TableEntityQuery< + TEntity extends Entity, + TResult, + Writable extends boolean = true +> extends Omit< + SchemaQueryBuilder< + EntitySchema, + EntityRowSchema, + EntityRelations, + Writable + >, + 'include' | 'includeVariant' | WriteMethod + >, + EntityWrites { + /** Configure a discriminator branch with the canonical strongly typed query API. */ + forVariant: SchemaAwareQuery> extends { + forVariant: infer F; + } + ? F + : never; + /** Restrict a polymorphic query to declared variants and narrow its row schema. */ + selectVariants: SchemaAwareQuery> extends { + selectVariants: infer F; + } + ? F + : never; /** * Eager-load a relation declared on `TEntity` via `.hasOne()` / * `.hasMany()` / `.belongsTo()` / `.belongsToMany()`. Selector returns * the relation key as a string literal. */ - include & string>( + include< + K extends keyof EntityRelations & string, + Child extends ReadQueryShape = SchemaAwareQuery< + RelatedSchema + > + >( sel: (t: RelKeyTree) => K, customize?: ( - q: SchemaQueryBuilder< - RelatedSchema, - InferType> - > - ) => void - ): EntityQuery>; + query: SchemaAwareQuery> + ) => Child + ): EntityQuery< + TEntity, + TResult & { + [P in K]: EntityRelations[K] extends { + kind: 'hasMany' | 'belongsToMany'; + } + ? InferType[] + : InferType | null; + }, + false + >; /** * Eager-load a relation declared inside a polymorphic variant (CTI/STI). @@ -112,10 +199,10 @@ export interface EntityQuery, TResult> q: TRel extends keyof EntityRelations ? SchemaQueryBuilder< RelatedSchema, - InferType> + EntityRowSchema>> > : SchemaQueryBuilder - ) => void + ) => ReadQueryShape ): EntityQuery< TEntity, TRel extends keyof EntityRelations & string @@ -178,8 +265,14 @@ export interface EntityQuery, TResult> * * @public */ -export interface DbSet> - extends EntityQuery> { +export type DbSet> = EntityQuery< + TEntity, + EntityResult +> & + DbSetOperations; + +/** Entity registration, transactions and graph/variant write entry points. */ +export interface DbSetOperations> { /** The wrapped entity definition. */ readonly entity: TEntity; @@ -258,54 +351,13 @@ export interface DbSet> export interface VariantDbSet< TEntity extends Entity, K extends string -> extends Omit< - SchemaQueryBuilder, VariantResult>, - 'include' | 'includeVariant' | 'insert' | 'update' | 'delete' | 'where' +> extends PolymorphicQueryBuilder< + EntitySchema, + Pick< + VariantReadSchemas>, + Extract>> + > > { - // Re-declared so that `this` resolves to `VariantDbSet` - // rather than the raw `SchemaQueryBuilder` (Omit doesn't preserve `this`). - /** - * Add an AND filter on this polymorphic branch using a mapped property, record or Knex callback. - */ - where( - column: ColumnRef>, - operator: string, - value: any - ): this; - /** - * Add an AND filter on this polymorphic branch using a mapped property, record or Knex callback. - */ - where(column: ColumnRef>, value: any): this; - /** - * Add an AND filter on this polymorphic branch using a mapped property, record or Knex callback. - */ - where(raw: Knex.Raw, operator: string, value: any): this; - /** - * Add an AND filter on this polymorphic branch using a mapped property, record or Knex callback. - */ - where(callback: (builder: Knex.QueryBuilder) => void): this; - /** - * Add an AND filter on this polymorphic branch using a mapped property, record or Knex callback. - */ - where(record: Record): this; - /** - * Add an AND filter on this polymorphic branch using a mapped property, record or Knex callback. - */ - where(raw: Knex.Raw): this; - /** - * Eager-load a relation declared on `TEntity`. Identical to - * {@link EntityQuery.include}. - */ - include & string>( - sel: (t: RelKeyTree) => R, - customize?: ( - q: SchemaQueryBuilder< - RelatedSchema, - InferType> - > - ) => void - ): VariantDbSet; - /** Look up a single row by PK, typed to this variant. */ find( pk: PrimaryKeyValueOf> @@ -347,33 +399,6 @@ export interface VariantDbSet< // Runtime construction // --------------------------------------------------------------------------- -const ENTITY_KEY_TREE_CACHE = new WeakMap>(); - -function getEntityKeyTree( - entity: Entity -): Record { - const cached = ENTITY_KEY_TREE_CACHE.get(entity); - if (cached) return cached; - const props = - ( - entity.schema as { - introspect?: () => { properties?: Record }; - } - ).introspect?.()?.properties ?? {}; - const tree: Record = {}; - for (const k of Object.keys(props)) tree[k] = k; - ENTITY_KEY_TREE_CACHE.set(entity, tree); - return tree; -} - -/** - * Extract the Knex instance from a `SchemaQueryBuilder`. - * SchemaQueryBuilder stores it as `#knex` (private), but the `query()` - * factory function passes it as the first constructor argument. - * We retrieve it via the `.toQuery()` builder's knex client reference. - * @internal - */ - /** * Wrap a `SchemaQueryBuilder` in a Proxy that overlays typed * `.include()` / `.includeVariant()` methods and re-wraps `this`-returning @@ -382,7 +407,7 @@ function getEntityKeyTree( * @internal */ function wrapQuery, TResult>( - sqb: SchemaQueryBuilder, TResult>, + sqb: SchemaQueryBuilder, EntityRowSchema>, entity: TEntity, _knexInst: Knex, onResults?: (items: unknown[]) => unknown[] | undefined @@ -391,45 +416,6 @@ function wrapQuery, TResult>( sqb as unknown as object, { get(target, prop, receiver) { - if (prop === 'include') { - return ( - sel: (t: Record) => string, - customize?: (q: SchemaQueryBuilder) => void - ) => { - const name = sel(getEntityKeyTree(entity)); - ( - sqb as unknown as { - include: ( - n: string, - c?: ( - q: SchemaQueryBuilder - ) => void - ) => void; - } - ).include(name, customize); - return proxy; - }; - } - if (prop === 'includeVariant') { - return ( - variantKey: string, - relationName: string, - customize?: (q: SchemaQueryBuilder) => void - ) => { - ( - sqb as unknown as { - includeVariant: ( - v: string, - r: string, - c?: ( - q: SchemaQueryBuilder - ) => void - ) => void; - } - ).includeVariant(variantKey, relationName, customize); - return proxy; - }; - } if (prop === '_sqb') return sqb; if (prop === '_entity') return entity; @@ -466,9 +452,27 @@ function wrapQuery, TResult>( const value = Reflect.get(target, prop, receiver); if (typeof value !== 'function') return value; return function (this: unknown, ...args: unknown[]) { + if (prop === 'then') { + const [resolve, reject] = args; + return sqb + .execute() + .then(rows => { + if (!onResults || !sqb.returnsEntityRows) + return rows; + return onResults(rows) ?? rows; + }) + .then(resolve as any, reject as any); + } const result = (value as Function).apply(sqb, args); // Re-wrap `this`-returning methods so chains keep typing. if (result === sqb) return proxy; + if ( + result && + typeof result.execute === 'function' && + typeof result.sameSource === 'function' && + sqb.sameSource(result) + ) + return wrapQuery(result, entity, _knexInst, onResults); // Wrap Promise results to auto-attach tracked entities. if ( onResults != null && @@ -479,6 +483,21 @@ function wrapQuery, TResult>( ) { return (result as Promise).then( (resolved: unknown) => { + if ( + (prop === 'paginate' || + prop === 'paginateAfter') && + resolved && + typeof resolved === 'object' && + 'data' in resolved && + Array.isArray(resolved.data) + ) { + return { + ...resolved, + data: + onResults(resolved.data) ?? + resolved.data + }; + } if (Array.isArray(resolved)) { const entities = resolved.filter( r => @@ -544,9 +563,9 @@ function wrapQuery, TResult>( */ function makeFindMethod>( method: 'find' | 'findOrFail' | 'findMany', - sqb: SchemaQueryBuilder, + _sqb: SchemaQueryBuilder, entity: TEntity, - proxy: EntityQuery + proxy: any ): (...args: unknown[]) => Promise { return async (...args: unknown[]): Promise => { const pkInfo = getPrimaryKeyColumns( @@ -568,87 +587,51 @@ function makeFindMethod>( ).getExtension?.('tableName') ?? ''; const entityLabel = String(tableName); - const applyPkFilter = (pk: unknown): void => { + const tupleFor = (pk: unknown) => { const tuple = normalisePkTuple(pk, isComposite, method); - if (tuple.length !== propertyKeys.length) { + if (tuple.length !== propertyKeys.length) throw new Error( `${method}(): expected ${propertyKeys.length} primary-key value(s), got ${tuple.length}.` ); - } - for (let i = 0; i < propertyKeys.length; i++) { - ( - proxy as unknown as { - andWhere: ( - col: string, - op: string, - val: unknown - ) => void; - } - ).andWhere(propertyKeys[i], '=', tuple[i]); - } + return tuple; }; - if (method === 'findMany') { - const pks = (args[0] ?? []) as ReadonlyArray; - if (!Array.isArray(pks)) { + const pks = args[0] ?? []; + if (!Array.isArray(pks)) throw new Error( 'findMany(): expected an array of primary-key values.' ); - } - if (pks.length === 0) return []; - if (!isComposite) { - const propKey = propertyKeys[0]; - const tuples = pks.map(p => - normalisePkTuple(p, false, 'findMany') - ); - ( - proxy as unknown as { - whereIn: ( - col: string, - vals: readonly unknown[] - ) => void; - } - ).whereIn( - propKey, - tuples.map(t => t[0]) - ); - return await ( - proxy as unknown as { execute: () => Promise } - ).execute(); - } - // Composite PK — emit OR-grouped predicates. We must use - // COLUMN names (not property names) here because the inner - // knex `apply()` callback bypasses SchemaQueryBuilder's - // property-to-column translation. - const columnNames = pkInfo.columnNames; - ( - sqb as unknown as { - apply: (fn: (qb: Knex.QueryBuilder) => void) => void; - } - ).apply(qb => { - qb.andWhere(function (this: Knex.QueryBuilder) { - for (const pk of pks) { - const tuple = normalisePkTuple(pk, true, 'findMany'); - this.orWhere(function (this: Knex.QueryBuilder) { - for (let i = 0; i < columnNames.length; i++) { - this.andWhere(columnNames[i], tuple[i] as any); - } - }); - } - }); - }); - return await ( - proxy as unknown as { execute: () => Promise } - ).execute(); + if (!pks.length) return []; + const tuples = pks.map(tupleFor); + if (!isComposite) + return (proxy as any) + .whereIn( + propertyKeys[0], + tuples.map(tuple => tuple[0]) + ) + .execute(); + return (proxy as any) + .where((group: any) => + tuples.reduce( + (outer, tuple) => + outer.orWhere((inner: any) => + propertyKeys.reduce( + (query, key, i) => + query.where(key, tuple[i]), + inner + ) + ), + group + ) + ) + .execute(); } - - // find / findOrFail - applyPkFilter(args[0]); - const row = await ( - proxy as unknown as { - first: () => Promise; - } - ).first(); + const tuple = tupleFor(args[0]); + const filtered = propertyKeys.reduce( + (query, key, i) => query.andWhere(key, '=', tuple[i]), + proxy as any + ); + const row = await filtered.first(); if (row === undefined && method === 'findOrFail') { throw new EntityNotFoundError(entityLabel, args[0]); } @@ -700,9 +683,9 @@ export function makeDbSet>( entity.schema as EntitySchema ); return wrapQuery( - sqb as SchemaQueryBuilder< + sqb as unknown as SchemaQueryBuilder< EntitySchema, - EntityResult + EntityRowSchema> >, entity, knex, @@ -754,9 +737,9 @@ export function makeDbSet>( entity.schema as EntitySchema ); const query = wrapQuery( - fresh as SchemaQueryBuilder< + fresh as unknown as SchemaQueryBuilder< EntitySchema, - EntityResult + EntityRowSchema> >, entity, knex, @@ -789,7 +772,10 @@ function wrapVariantQuery< TEntity extends Entity, K extends string >( - sqb: SchemaQueryBuilder, VariantResult>, + sqb: SchemaQueryBuilder< + EntitySchema, + EntityRowSchema> + >, entity: TEntity, variantKey: K, knexInst: Knex, @@ -799,47 +785,6 @@ function wrapVariantQuery< sqb as unknown as object, { get(target, prop, receiver) { - // --- Typed include override (same as wrapQuery) --- - if (prop === 'include') { - return ( - sel: (t: Record) => string, - customize?: (q: SchemaQueryBuilder) => void - ) => { - const name = sel(getEntityKeyTree(entity)); - ( - sqb as unknown as { - include: ( - n: string, - c?: ( - q: SchemaQueryBuilder - ) => void - ) => void; - } - ).include(name, customize); - return proxy; - }; - } - if (prop === 'includeVariant') { - return ( - vk: string, - relationName: string, - customize?: (q: SchemaQueryBuilder) => void - ) => { - ( - sqb as unknown as { - includeVariant: ( - v: string, - r: string, - c?: ( - q: SchemaQueryBuilder - ) => void - ) => void; - } - ).includeVariant(vk, relationName, customize); - return proxy; - }; - } - if (prop === '_sqb') return sqb; if (prop === '_entity') return entity; @@ -963,8 +908,32 @@ function wrapVariantQuery< const value = Reflect.get(target, prop, receiver); if (typeof value !== 'function') return value; return function (this: unknown, ...args: unknown[]) { + if (prop === 'then') { + const [resolve, reject] = args; + return sqb + .execute() + .then(rows => { + if (!onResults || !sqb.returnsEntityRows) + return rows; + return onResults(rows) ?? rows; + }) + .then(resolve as any, reject as any); + } const result = (value as Function).apply(sqb, args); if (result === sqb) return proxy; + if ( + result && + typeof result.execute === 'function' && + typeof result.sameSource === 'function' && + sqb.sameSource(result) + ) + return wrapVariantQuery( + result, + entity, + variantKey, + knexInst, + onResults + ); if ( onResults != null && sqb.returnsEntityRows && @@ -974,6 +943,21 @@ function wrapVariantQuery< ) { return (result as Promise).then( (resolved: unknown) => { + if ( + (prop === 'paginate' || + prop === 'paginateAfter') && + resolved && + typeof resolved === 'object' && + 'data' in resolved && + Array.isArray(resolved.data) + ) { + return { + ...resolved, + data: + onResults(resolved.data) ?? + resolved.data + }; + } if (Array.isArray(resolved)) { const entities = resolved.filter( r => @@ -1055,19 +1039,13 @@ function makeVariantDbSet< } // Allocate a fresh SQB with the variant filter baked in, then // wrap it in the variant-aware query proxy. - const fresh = schemaQuery( - knex, - entity.schema as EntitySchema - ); - ( - fresh as unknown as { - selectVariants?: (keys: string[]) => void; - } - ).selectVariants?.([variantKey]); + const fresh = ( + schemaQuery(knex, entity.schema as any) as any + ).selectVariants([variantKey]); const query = wrapVariantQuery( - fresh as SchemaQueryBuilder< + fresh as unknown as SchemaQueryBuilder< EntitySchema, - VariantResult + EntityRowSchema> >, entity, variantKey, diff --git a/libs/orm/src/index.ts b/libs/orm/src/index.ts index e58fda81..ca290237 100644 --- a/libs/orm/src/index.ts +++ b/libs/orm/src/index.ts @@ -18,7 +18,14 @@ export type { TrackedDbContext } from './dbcontext.js'; export { createDb } from './dbcontext.js'; -export type { DbSet, EntityQuery, VariantDbSet } from './dbset.js'; +export type { + DbSet, + DbSetOperations, + EntityQuery, + PolymorphicEntityQuery, + TableEntityQuery, + VariantDbSet +} from './dbset.js'; export { ConcurrencyError, EntityNotFoundError, diff --git a/libs/orm/src/orm.test.ts b/libs/orm/src/orm.test.ts index 6e0149fe..ae5d3fe1 100644 --- a/libs/orm/src/orm.test.ts +++ b/libs/orm/src/orm.test.ts @@ -324,16 +324,16 @@ describe('EntityQuery proxy', () => { expect(typeof chain.toQuery).toBe('function'); }); - it('.include(selector) forwards the relation name and emits a JOIN', () => { + it('.include(selector) emits a correlated nested row', () => { const db = createDb(mock.knex, { todos: TodoEntity }); const sql = db.todos.include(t => t.author).toQuery(); - expect(sql.toLowerCase()).toContain('join'); + expect(sql.toLowerCase()).toContain('to_jsonb'); expect(sql).toContain('users'); }); it('.include(selector) accepts a customize callback', () => { const db = createDb(mock.knex, { todos: TodoEntity }); - const customize = vi.fn(); + const customize = vi.fn(q => q); db.todos.include(t => t.author, customize); expect(customize).toHaveBeenCalledOnce(); }); @@ -356,7 +356,7 @@ describe('EntityQuery proxy', () => { // than swallowing the call. const db = createDb(mock.knex, { todos: TodoEntity }); expect(() => db.todos.includeVariant('foo', 'author')).toThrow( - /not polymorphic/i + /includeVariant is not a function/i ); }); }); @@ -377,10 +377,12 @@ describe('DbSet.find / findOrFail / findMany', () => { // ---- single-PK happy paths --------------------------------------------- it('find(scalar) emits WHERE id = ? and returns the first row', async () => { - mock.responses.push([{ id: 42, email: 'a@b', name: 'A' }]); + mock.responses.push([ + { id: 42, email: 'a@b', name: 'A', createdAt: null } + ]); const db = createDb(mock.knex, { users: UserEntity }); const u = await db.users.find(42); - expect(u).toEqual({ id: 42, email: 'a@b', name: 'A' }); + expect(u).toEqual({ id: 42, email: 'a@b', name: 'A', createdAt: null }); expect(mock.captured).toHaveLength(1); expect(mock.captured[0].sql).toContain('"id" = $1'); expect(mock.captured[0].bindings).toContain(42); @@ -394,10 +396,12 @@ describe('DbSet.find / findOrFail / findMany', () => { }); it('findOrFail returns the row when present', async () => { - mock.responses.push([{ id: 1, email: 'x', name: 'Y' }]); + mock.responses.push([ + { id: 1, email: 'x', name: 'Y', createdAt: null } + ]); const db = createDb(mock.knex, { users: UserEntity }); const u = await db.users.findOrFail(1); - expect(u).toEqual({ id: 1, email: 'x', name: 'Y' }); + expect(u).toEqual({ id: 1, email: 'x', name: 'Y', createdAt: null }); }); it('findOrFail throws EntityNotFoundError when no row matches', async () => { @@ -427,8 +431,8 @@ describe('DbSet.find / findOrFail / findMany', () => { it('findMany on single-PK emits WHERE id IN (...)', async () => { mock.responses.push([ - { id: 1, email: 'a', name: 'A' }, - { id: 2, email: 'b', name: 'B' } + { id: 1, email: 'a', name: 'A', createdAt: null }, + { id: 2, email: 'b', name: 'B', createdAt: null } ]); const db = createDb(mock.knex, { users: UserEntity }); const rows = await db.users.findMany([1, 2]); @@ -440,10 +444,10 @@ describe('DbSet.find / findOrFail / findMany', () => { // ---- composite-PK paths ----------------------------------------------- it('find on composite-PK accepts a tuple', async () => { - mock.responses.push([{ postId: 1, tagId: 9 }]); + mock.responses.push([{ postId: 1, tagId: 9, addedAt: null }]); const db = createDb(mock.knex, { postTags: PostTagEntity }); const r = await db.postTags.find([1, 9]); - expect(r).toEqual({ postId: 1, tagId: 9 }); + expect(r).toEqual({ postId: 1, tagId: 9, addedAt: null }); expect(mock.captured[0].sql).toContain('"post_id"'); expect(mock.captured[0].sql).toContain('"tag_id"'); expect(mock.captured[0].bindings).toEqual( @@ -468,8 +472,8 @@ describe('DbSet.find / findOrFail / findMany', () => { it('findMany on composite-PK emits OR-grouped predicates', async () => { mock.responses.push([ - { postId: 1, tagId: 9 }, - { postId: 2, tagId: 9 } + { postId: 1, tagId: 9, addedAt: null }, + { postId: 2, tagId: 9, addedAt: null } ]); const db = createDb(mock.knex, { postTags: PostTagEntity }); const rows = await db.postTags.findMany([ @@ -636,7 +640,9 @@ describe('DbSet.save — graph persistence', () => { it('update path: PK present → emits UPDATE, no INSERT for root', async () => { stubTransaction(); // The update returns the updated row. - mock.responses.push([{ id: 5, name: 'X', email: 'x@y' }]); + mock.responses.push([ + { id: 5, name: 'X', email: 'x@y', createdAt: null } + ]); const db = createDb(mock.knex, { users: UserEntity }); const out = await db.users.save({ id: 5, @@ -657,7 +663,7 @@ describe('DbSet.save — graph persistence', () => { stubTransaction(); // Root todo insert. mock.responses.push([ - { id: 21, title: 'T', userId: null, completed: false } + { id: 21, title: 'T', userId: 0, completed: false } ]); // Pivot insert (todo_tags) returns []; we don't read it. mock.responses.push([]); @@ -687,7 +693,9 @@ describe('DbSet.save — graph persistence', () => { (mock.knex as any).isTransaction = true; const txSpy = vi.spyOn(mock.knex, 'transaction'); - mock.responses.push([{ id: 9, email: 'x', name: 'Y' }]); + mock.responses.push([ + { id: 9, email: 'x', name: 'Y', createdAt: null } + ]); const db = createDb(mock.knex, { users: UserEntity }); await db.users .withTransaction(mock.knex as unknown as KnexT.Transaction) @@ -748,7 +756,7 @@ describe('DbSet.save — graph persistence', () => { stubTransaction(); // Root todo insert. mock.responses.push([ - { id: 41, title: 'T', userId: null, completed: false } + { id: 41, title: 'T', userId: 0, completed: false } ]); // Tag insert (because PK was not supplied → create new row). mock.responses.push([{ id: 77, name: 'urgent' }]); @@ -885,8 +893,18 @@ describe('DbSet.ofVariant — insert', () => { stubTransaction(); // Base row insert returns generated PK. mock.responses.push([{ id: 5, type: 'assigned', todo_id: 42 }]); - // Variant row insert. + // Variant row insert, then read-back of the completed branch. mock.responses.push([]); + mock.responses.push([ + { + __read_poly: { + id: 5, + type: 'assigned', + todoId: 42, + assigneeId: 9 + } + } + ]); const db = createDb(mock.knex, { activities: ActivityEntityCTI }); const result = await db.activities.ofVariant('assigned').insert({ @@ -914,6 +932,9 @@ describe('DbSet.ofVariant — insert', () => { stubTransaction(); mock.responses.push([{ id: 7, type: 'commented', todo_id: 1 }]); mock.responses.push([]); + mock.responses.push([ + { __read_poly: { id: 7, type: 'commented', todoId: 1, body: 'hi' } } + ]); const db = createDb(mock.knex, { activities: ActivityEntityCTI }); await db.activities.ofVariant('commented').insert({ @@ -939,9 +960,9 @@ describe('DbSet.ofVariant — insert', () => { it('throws when the variant key is unknown', async () => { stubTransaction(); const db = createDb(mock.knex, { activities: ActivityEntitySTI }); - await expect( + expect(() => db.activities.ofVariant('nonexistent' as any).insert({}) - ).rejects.toThrow(/unknown/i); + ).toThrow(/declared variants/i); }); }); @@ -967,7 +988,15 @@ describe('VariantDbSet.update', () => { it('STI: emits an UPDATE on the base table filtered by discriminator', async () => { // First query: execute() to collect PKs → returns matching rows. mock.responses.push([ - { id: 3, type: 'assigned', todo_id: 10, user_id: 1, assignee_id: 4 } + { + __read_poly: { + id: 3, + type: 'assigned', + todoId: 10, + userId: 1, + assigneeId: 4 + } + } ]); // Second query: the UPDATE itself. mock.responses.push([]); @@ -987,7 +1016,16 @@ describe('VariantDbSet.update', () => { it('CTI: emits an UPDATE on the variant table', async () => { stubTransaction(); // execute() returns matched base-table rows. - mock.responses.push([{ id: 5, type: 'assigned', todo_id: 1 }]); + mock.responses.push([ + { + __read_poly: { + id: 5, + type: 'assigned', + todoId: 1, + assigneeId: 4 + } + } + ]); // UPDATE on the variant table. mock.responses.push([]); @@ -1040,7 +1078,17 @@ describe('VariantDbSet.delete', () => { it('STI: emits a DELETE on the base table with discriminator filter', async () => { stubTransaction(); // execute() to collect PKs. - mock.responses.push([{ id: 2, type: 'commented', todo_id: 1 }]); + mock.responses.push([ + { + __read_poly: { + id: 2, + type: 'commented', + todoId: 1, + userId: 7, + body: 'hi' + } + } + ]); // The DELETE. mock.responses.push([]); @@ -1058,7 +1106,16 @@ describe('VariantDbSet.delete', () => { it('CTI: deletes variant row first then base row', async () => { stubTransaction(); // execute() → matched base rows. - mock.responses.push([{ id: 7, type: 'assigned', todo_id: 3 }]); + mock.responses.push([ + { + __read_poly: { + id: 7, + type: 'assigned', + todoId: 3, + assigneeId: 4 + } + } + ]); // DELETE from variant table. mock.responses.push([]); // DELETE from base table. @@ -1107,7 +1164,15 @@ describe('VariantDbSet.find', () => { it('returns the matched row', async () => { mock.responses.push([ - { id: 3, type: 'assigned', todo_id: 5, user_id: 1, assignee_id: 2 } + { + __read_poly: { + id: 3, + type: 'assigned', + todoId: 5, + userId: 1, + assigneeId: 2 + } + } ]); const db = createDb(mock.knex, { activities: ActivityEntitySTI }); @@ -1184,8 +1249,12 @@ describe('Tracked DbContext', () => { // ------------------------------------------------------------------------- it('querying the same PK twice returns the same object reference', async () => { - mock.responses.push([{ id: 1, email: 'a@b', name: 'A' }]); - mock.responses.push([{ id: 1, email: 'a@b', name: 'A' }]); + mock.responses.push([ + { id: 1, email: 'a@b', name: 'A', createdAt: null } + ]); + mock.responses.push([ + { id: 1, email: 'a@b', name: 'A', createdAt: null } + ]); const db = createDb( mock.knex, @@ -1200,8 +1269,8 @@ describe('Tracked DbContext', () => { it('rows returned from all() are attached to the tracker', async () => { mock.responses.push([ - { id: 1, email: 'a@b', name: 'A' }, - { id: 2, email: 'c@d', name: 'B' } + { id: 1, email: 'a@b', name: 'A', createdAt: null }, + { id: 2, email: 'c@d', name: 'B', createdAt: null } ]); const db = createDb( @@ -1212,7 +1281,9 @@ describe('Tracked DbContext', () => { const rows = (await db.users.execute()) as any[]; // Querying one of the same PKs should return the existing object. - mock.responses.push([{ id: 1, email: 'a@b', name: 'A' }]); + mock.responses.push([ + { id: 1, email: 'a@b', name: 'A', createdAt: null } + ]); const reloaded = await db.users.find(1); expect(reloaded).toBe(rows[0]); }); @@ -1227,7 +1298,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 10, email: 'x@y', name: 'X' }; + const user = { id: 10, email: 'x@y', name: 'X', createdAt: null }; db.attach('users', user); const e = db.entry(user); expect(e.state).toBe('Unchanged'); @@ -1240,8 +1311,8 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const u1 = { id: 5, email: 'a@b', name: 'A' }; - const u2 = { id: 5, email: 'c@d', name: 'C' }; // same PK, different object + const u1 = { id: 5, email: 'a@b', name: 'A', createdAt: null }; + const u2 = { id: 5, email: 'c@d', name: 'C', createdAt: null }; // same PK, different object db.attach('users', u1); const returned = db.attach('users', u2); expect(returned).toBe(u1); // identity-map: existing wins @@ -1253,7 +1324,12 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b.com', name: 'Alice' }; + const user = { + id: 1, + email: 'a@b.com', + name: 'Alice', + createdAt: null + }; db.attach('users', user); const original = db.entry(user).originalValues; @@ -1267,7 +1343,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); expect(db.entry(user).isModified()).toBe(false); }); @@ -1278,7 +1354,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); user.name = 'Changed'; expect(db.entry(user).isModified()).toBe(true); @@ -1292,7 +1368,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); user.name = 'Changed'; db.entry(user).reset(); @@ -1306,7 +1382,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 99, email: 'x', name: 'X' }; + const user = { id: 99, email: 'x', name: 'X', createdAt: null }; expect(() => db.entry(user)).toThrow(/not tracked/i); }); @@ -1316,7 +1392,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); db.detach(user); expect(() => db.entry(user)).toThrow(/not tracked/i); @@ -1357,7 +1433,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); db.remove(user); expect(db.entry(user).state).toBe('Deleted'); @@ -1369,7 +1445,9 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - expect(() => db.remove({ id: 1, email: 'x', name: 'X' })).toThrow(); + expect(() => + db.remove({ id: 1, email: 'x', name: 'X', createdAt: null }) + ).toThrow(); }); // ------------------------------------------------------------------------- @@ -1378,14 +1456,16 @@ describe('Tracked DbContext', () => { it('saveChanges() detects silently mutated Unchanged entries and emits UPDATE', async () => { stubTransaction(); - mock.responses.push([{ id: 1, email_address: 'a@b', name: 'Updated' }]); + mock.responses.push([ + { id: 1, email_address: 'a@b', name: 'Updated', created_at: null } + ]); const db = createDb( mock.knex, { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); user.name = 'Updated'; // mutate without calling any set-state method @@ -1406,7 +1486,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); const result = await db.saveChanges(); @@ -1420,7 +1500,9 @@ describe('Tracked DbContext', () => { it('saveChanges() inserts Added entities', async () => { stubTransaction(); - mock.responses.push([{ id: 42, email_address: 'new@e', name: 'New' }]); + mock.responses.push([ + { id: 42, email_address: 'new@e', name: 'New', created_at: null } + ]); const db = createDb( mock.knex, @@ -1452,7 +1534,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 3, email: 'x@y', name: 'X' }; + const user = { id: 3, email: 'x@y', name: 'X', createdAt: null }; db.attach('users', user); db.remove(user); @@ -1474,7 +1556,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); user.name = 'B'; @@ -1494,7 +1576,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); (user as any).id = 99; // mutate PK @@ -1513,7 +1595,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); user.name = 'Changed'; @@ -1528,7 +1610,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); db.remove(user); expect(db.entry(user).state).toBe('Deleted'); @@ -1550,7 +1632,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); user.name = 'B'; @@ -1569,7 +1651,7 @@ describe('Tracked DbContext', () => { it('reload() refreshes entity values from DB', async () => { mock.responses.push([ - { id: 1, email_address: 'new@b', name: 'Refreshed' } + { id: 1, email: 'new@b', name: 'Refreshed', createdAt: null } ]); const db = createDb( @@ -1577,7 +1659,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); user.name = 'Dirty'; // simulate dirty state @@ -1616,7 +1698,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); // No mutations → dispose should not throw. await expect(db[Symbol.asyncDispose]()).resolves.toBeUndefined(); @@ -1628,7 +1710,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); user.name = 'Dirty'; @@ -1643,7 +1725,7 @@ describe('Tracked DbContext', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b', name: 'A' }; + const user = { id: 1, email: 'a@b', name: 'A', createdAt: null }; db.attach('users', user); user.name = 'Dirty'; @@ -2022,7 +2104,9 @@ describe('Tracked DbContext — first() single result onResults (line 481)', () }); it('first() in a tracked context attaches the resolved single object', async () => { - mock.responses.push([{ id: 2, email_address: 'b@b', name: 'Bob' }]); + mock.responses.push([ + { id: 2, email: 'b@b', name: 'Bob', createdAt: null } + ]); const db = createDb( mock.knex, @@ -2061,7 +2145,12 @@ describe('Tracked DbContext — reload() early returns', () => { { users: UserEntity }, { tracking: true } ); - const untracked = { id: 99, email: 'x@y.com', name: 'X' }; + const untracked = { + id: 99, + email: 'x@y.com', + name: 'X', + createdAt: null + }; // Should not throw and should not emit any SQL. await db.reload(untracked); expect(mock.captured).toHaveLength(0); @@ -2100,7 +2189,12 @@ describe('Tracked DbContext — attach() same-object snapshot refresh', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b.com', name: 'Alice' }; + const user = { + id: 1, + email: 'a@b.com', + name: 'Alice', + createdAt: null + }; db.attach('users', user); // Dirty the entity. @@ -2231,7 +2325,12 @@ describe('Tracked DbContext — pendingSummary() branch coverage', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 1, email: 'a@b.com', name: 'Alice' }; + const user = { + id: 1, + email: 'a@b.com', + name: 'Alice', + createdAt: null + }; db.attach('users', user); // Silently mutate (no explicit state change — isDirty triggers Modified count). @@ -2248,7 +2347,7 @@ describe('Tracked DbContext — pendingSummary() branch coverage', () => { { users: UserEntity }, { tracking: true } ); - const user = { id: 2, email: 'b@c.com', name: 'Bob' }; + const user = { id: 2, email: 'b@c.com', name: 'Bob', createdAt: null }; db.attach('users', user); db.remove(user); diff --git a/libs/orm/src/query-types.test-d.ts b/libs/orm/src/query-types.test-d.ts index a52747f4..d794583e 100644 --- a/libs/orm/src/query-types.test-d.ts +++ b/libs/orm/src/query-types.test-d.ts @@ -35,6 +35,7 @@ test('include customizers know their relation schema', () => { owners.where(t => t.name, 'Alice'); // @ts-expect-error field belongs to tasks, not users owners.where(t => t.ownerId, 1); + return owners.where(t => t.name, 'Alice'); } ); }); diff --git a/libs/orm/src/result-types.ts b/libs/orm/src/result-types.ts index 7029d7a3..94e67a4e 100644 --- a/libs/orm/src/result-types.ts +++ b/libs/orm/src/result-types.ts @@ -8,7 +8,8 @@ import type { Entity, EntityRelations, EntitySchema, - RelationInfo + RelationInfo, + SchemaAwareQuery } from '@cleverbrush/knex-schema'; import type { InferType, ObjectSchemaBuilder } from '@cleverbrush/schema'; @@ -28,12 +29,9 @@ import type { InferType, ObjectSchemaBuilder } from '@cleverbrush/schema'; * * @public */ -export type EntityResult> = - TEntity extends Entity - ? [U] extends [never] - ? InferType> - : U - : InferType>; +export type EntityResult> = InferType< + SchemaAwareQuery>['rowSchema'] +>; /** * Discriminated-union variant rows for a polymorphic entity. Resolves to @@ -45,7 +43,7 @@ export type EntityResultByVariant> = TEntity extends Entity ? [U] extends [never] ? never - : U + : EntityResult : never; /** diff --git a/libs/orm/src/save-graph.ts b/libs/orm/src/save-graph.ts index 7509a54d..146c2a5c 100644 --- a/libs/orm/src/save-graph.ts +++ b/libs/orm/src/save-graph.ts @@ -152,7 +152,7 @@ async function saveNode( // 2. Save self (insert vs update from PK presence). // --------------------------------------------------------------------- const isUpdate = hasFullPk(schema, ownFields); - const sqb = schemaQuery(trx, schema) as unknown as { + let sqb = schemaQuery(trx, schema) as unknown as { andWhere: (col: string, op: string, val: unknown) => unknown; update: (data: unknown) => Promise; insert: (data: unknown) => Promise; @@ -165,7 +165,7 @@ async function saveNode( const updateData: Record = { ...ownFields }; for (const k of pk.propertyKeys) delete updateData[k]; for (const k of pk.propertyKeys) { - sqb.andWhere(k, '=', ownFields[k]); + sqb = sqb.andWhere(k, '=', ownFields[k]) as typeof sqb; } // SchemaQueryBuilder.update returns the updated row(s). const result = (await sqb.update(updateData)) as unknown; diff --git a/libs/orm/src/variant-write.ts b/libs/orm/src/variant-write.ts index bf9dfd39..0f4eb8e6 100644 --- a/libs/orm/src/variant-write.ts +++ b/libs/orm/src/variant-write.ts @@ -9,14 +9,32 @@ import { buildColumnMap, getPrimaryKeyColumns, getVariants, + object, query as schemaQuery } from '@cleverbrush/knex-schema'; +import type { ObjectSchemaBuilder } from '@cleverbrush/schema'; import type { Knex } from 'knex'; // --------------------------------------------------------------------------- // Internal helpers // --------------------------------------------------------------------------- +/** Keep table metadata and hooks without constructing a polymorphic reader. */ +function storageSchema( + schema: any, + properties = schema.introspect().properties +): ObjectSchemaBuilder { + let stored: ObjectSchemaBuilder = + object(properties); + for (const [key, value] of Object.entries( + schema.introspect().extensions ?? {} + )) { + if (key !== 'variants' && key !== 'polymorphicVariants') + stored = stored.withExtension(key, value) as typeof stored; + } + return stored; +} + /** * Resolve the variant config for a schema, throwing if the schema is not * polymorphic or if the requested variant key is unknown. @@ -90,10 +108,13 @@ export async function insertVariant( if (spec.storage === 'sti') { // Single-table: insert into base table with discriminator column const row = { ...payload, [discKey]: variantKey }; - const sqb = schemaQuery(t, schema) as unknown as { - insert: (data: unknown) => Promise>; + const props = { + ...schema.introspect().properties, + ...spec.schema.introspect().properties }; - return (await sqb.insert(row)) ?? row; + const stored = storageSchema(schema, props); + const result = await schemaQuery(t, stored).insert(row as any); + return result as Record; } // CTI: two-table insert @@ -129,20 +150,11 @@ export async function insertVariant( // Keys that match neither schema are silently dropped } - // 1. Insert base row using raw knex (not SchemaQueryBuilder) to avoid - // polymorphic result-resolution running before the variant row exists. - const { propToCol: basePropToCol } = buildColumnMap(schema); - const baseRowForInsert: Record = {}; - for (const [propKey, val] of Object.entries(basePayload)) { - baseRowForInsert[basePropToCol.get(propKey) ?? propKey] = val; - } - const baseInsertResult = await (t as unknown as Knex)(baseTableName) - .insert(baseRowForInsert) - .returning('*'); - const baseRow: Record = - Array.isArray(baseInsertResult) && baseInsertResult.length > 0 - ? (baseInsertResult[0] as Record) - : baseRowForInsert; + // 1. Decode base RETURNING values before using the PK. In particular, + // bigint IDs must be text-cast in SQL before driver parsers can round them. + const baseRow = (await schemaQuery(t, storageSchema(schema)).insert( + basePayload as any + )) as Record; // Resolve the base PK value from the returned row const pkInfo = getPrimaryKeyColumns(schema); @@ -179,17 +191,17 @@ export async function insertVariant( } await (t as unknown as Knex)(variantTableName).insert(variantRow); - // 3. Merge and return - const { colToProp: baseColToProp } = buildColumnMap(schema); - const result: Record = {}; - for (const [col, val] of Object.entries(baseRow)) { - result[baseColToProp.get(col) ?? col] = val; - } - // Ensure discriminator and variant payload are in the result - result[discKey] = variantKey; - for (const [propKey, val] of Object.entries(variantPayload)) { - result[propKey] = val; - } + // Read the completed branch inside the same transaction for one consistent storage representation. + const result = await (schemaQuery(t, schema) as any) + .selectVariants([variantKey]) + .unscoped() + .withDeleted() + .where(pkPropKey, pkValue) + .first(); + if (!result) + throw new Error( + 'insertVariant: inserted row could not be read back' + ); return result; }; diff --git a/libs/schema/README.md b/libs/schema/README.md index 36bf1af8..8ae25fa6 100644 --- a/libs/schema/README.md +++ b/libs/schema/README.md @@ -122,10 +122,6 @@ if (result.valid) { const nameErrors = result.getErrorsFor((p) => p.name); console.log(nameErrors.isValid); // false console.log(nameErrors.errors); // ['Name must be at least 2 characters'] - - // result.errors on object schemas is deprecated — use getErrorsFor() instead - console.log('Errors:', result.errors); - // Array of { message: string } } ``` @@ -865,8 +861,7 @@ const result = UserSchema.validate(someObject); if (result.valid) { console.log(result.object); // typed as InferType } else { - // For object schemas, prefer getErrorsFor() for per-property error inspection (see below) - console.log(result.errors); // deprecated for object schemas — Array of { message: string } + console.log(result.getErrorsFor(t => t.name).errors); // field error strings } // Async validation (use when validators/preprocessors are async) @@ -883,12 +878,12 @@ const result = UserSchema.validate( { doNotStopOnFirstError: true } ); -console.log(result.errors); -// [ -// { message: 'Name must be at least 2 characters' }, -// { message: 'Please enter a valid email' }, -// { message: 'Age cannot be negative' } -// ] +console.log(result.getErrorsFor(t => t.name).errors); +// ['Name must be at least 2 characters'] +console.log(result.getErrorsFor(t => t.email).errors); +// ['Please enter a valid email'] +console.log(result.getErrorsFor(t => t.age).errors); +// ['Age cannot be negative'] ``` ### Custom Error Messages @@ -979,11 +974,11 @@ result.getErrorsFor((t) => t.password).errors; // → [] ``` -You can target multiple properties from a single validator by returning multiple errors with different `property` selectors. Errors without a `property` selector are attached to the root object as before. +You can target multiple properties from a single validator by returning multiple errors with different `property` selectors. Errors without a `property` selector are attached to the root object. ### Per-Property Errors with `getErrorsFor()` (Recommended) -`ObjectSchemaBuilder.validate()` returns an extended result with a `getErrorsFor()` method for inspecting errors on individual properties — perfect for showing inline form errors. **This is the recommended way to inspect validation errors on object schemas** and replaces the deprecated `errors` array on `ObjectSchemaValidationResult`: +`ObjectSchemaBuilder.validate()` returns a result with a `getErrorsFor()` method for inspecting errors on individual properties — useful for showing inline form errors: ```typescript const PersonSchema = object({ @@ -1167,7 +1162,7 @@ array(optionalText).parse(['ok', 42]); // ['ok', undefined] — no entries dropp Fallbacks are opt-in: a fallback on a property does not make a malformed required root object valid. A fallback factory runs only when validation fails. -**Null compatibility:** legacy optional schemas accept `null` at runtime even +**Null handling:** optional schemas accept `null` at runtime even though their inferred type does not include it. `.optional().catch(undefined)` therefore leaves `null` unchanged. Normalize it explicitly when needed: diff --git a/libs/schema/src/builders/ObjectSchemaBuilder.ts b/libs/schema/src/builders/ObjectSchemaBuilder.ts index 3be56a3c..5473c8d2 100644 --- a/libs/schema/src/builders/ObjectSchemaBuilder.ts +++ b/libs/schema/src/builders/ObjectSchemaBuilder.ts @@ -224,7 +224,7 @@ export type ObjectSchemaValidationResult< * This is the **recommended** way to inspect validation errors — it provides type-safe, * per-property error details including `isValid`, `errors`, and `seenValue`. * - * Prefer this over the deprecated `errors` array. + * Inspect root or property-specific validation errors with a selector. * * @param selector a callback function to select property from the schema. */ @@ -324,7 +324,7 @@ export type ObjectSchemaValidationResult< * }); * * // result.valid === false - * // result.errors is deprecated — use result.getErrorsFor() instead + * // Inspect property errors using result.getErrorsFor(). * // result.getErrorsFor((p) => p.age).errors // ["is expected to have property 'age'"] * ``` * diff --git a/libs/server/CHANGELOG.md b/libs/server/CHANGELOG.md index 387aa733..347477f2 100644 --- a/libs/server/CHANGELOG.md +++ b/libs/server/CHANGELOG.md @@ -30,7 +30,7 @@ ### Minor Changes -- c75bff4: Add framework affordances discovered while reviewing Xpenser: +- c75bff4: Add reusable environment, HTTP, cache, database and migration capabilities: - `@cleverbrush/env`: add `envBoolean()` for environment-style boolean values such as `1`, `0`, `yes`, `no`, `on`, and `off`. diff --git a/websites/docs/app/client/sections/cacheTags.tsx b/websites/docs/app/client/sections/cacheTags.tsx index 73993976..3fdb1b1f 100644 --- a/websites/docs/app/client/sections/cacheTags.tsx +++ b/websites/docs/app/client/sections/cacheTags.tsx @@ -130,7 +130,7 @@ externalCacheTags({ invalidateTag: revalidateTag });

-

Versioned keys: breaking migration

+

Deterministic cache keys

Server and client helpers, response caches and external invalidation share the deterministic ct2:{' '} @@ -141,11 +141,10 @@ externalCacheTags({ invalidateTag: revalidateTag }); values throw TypeError.

- Upgrade external cache writers and invalidators together and - flush or expire old entries. There is no legacy fallback. - Base invalidation labels remain literal names; TTLs are - unchanged. External invalidators send both base labels and - computed keys by default, including property-free tags. + External cache writers and invalidators must use the same + key format. Base invalidation labels are literal names. + External invalidators send both base labels and computed + keys by default, including property-free tags.

Response shape and auth/tenant isolation remain diff --git a/websites/docs/app/knex-schema/page.tsx b/websites/docs/app/knex-schema/page.tsx index 917f8b4a..715456f0 100644 --- a/websites/docs/app/knex-schema/page.tsx +++ b/websites/docs/app/knex-schema/page.tsx @@ -228,9 +228,10 @@ returning *`

.joinOne() and .joinMany(){' '} load related rows in a{' '} - single PostgreSQL query using CTEs and{' '} - jsonb_agg. The inferred TypeScript type is - updated automatically for each join you add. + single PostgreSQL query using + correlated subqueries and jsonb_agg. The + inferred TypeScript type is updated automatically for + each join you add.

                          t.id,
         foreignColumn: t => t.authorId,
         as:            'posts',
-        limit:         5,
-        orderBy:       { column: t => t.id, direction: 'desc' },
-    });
+    }, posts => posts.orderBy(t => t.id, 'desc').limit(5));
 // users[0].posts → Array<{ id: number; title: string; authorId: number }>
 
 // Many-to-one — attach the author to each post
@@ -298,22 +297,24 @@ const posts = await query(db, PostSchema)
                 

Escape Hatch

- Use .apply(fn) to call any Knex method not - exposed by this API — the raw{' '} - Knex.QueryBuilder is passed to your + Use .apply(fn, {'{ output }'}) with a + complete output schema for raw SQL. The independently + mutable Knex.QueryBuilder is passed to your callback:

                          t.id, id)
-    .apply(qb => qb.forUpdate().noWait());
+    .apply(qb => qb.clearSelect().select({ name: 'first_name' }), {
+        output: UserName,
+    });
 
-// Pre-scoped base query (e.g. soft-delete filter)
-const base = db('users').where('deleted_at', null);
-const activeUsers = await query(db, UserSchema, base)
-    .where(t => t.age, '>', 18);`)
+// Share an immutable Framework base query; retain each configured result.
+const base = query(db, UserSchema).whereNull(t => t.deletedAt);
+const activeUsers = await base.where(t => t.age, '>', 18);`)
                             }}
                         />
                     
@@ -373,7 +374,7 @@ const average = await query(knex, TaskSchema).avgValue(t => t.estimate, { }} />

- Existing APIs remain unchanged. Read the{' '} + Queries are immutable. Read the{' '} complete query guide {' '} @@ -382,18 +383,18 @@ const average = await query(knex, TaskSchema).avgValue(t => t.estimate, {

-

Opt-in projection-aware reads

+

Automatic projection-aware schemas

- Start with withRowSchema() before selecting or including - fields. The immutable PostgreSQL reader exposes the - actual decoded result schema: SQL null stays null, dates - are Date objects at every depth, and decimal/bigint - values are exact strings before JSON parsing. + Every query exposes rowSchema automatically. Immutable + PostgreSQL queries describe the actual decoded result + schema: SQL null stays null, dates are Date objects at + every depth, and decimal/bigint values are exact strings + before JSON parsing.

                          ({ title: t.title, amount: t.amount }));
 const Source = read.rowSchema;
 const rows = await read.where(t => t.id, taskId);`)
@@ -404,11 +405,49 @@ const rows = await read.where(t => t.id, taskId);`)
                         Typed aliases, aggregates, named projections and nested
                         graphs retain their selected shape. Polymorphic readers
                         expose a union rowSchema and per-variant object schemas.
-                        Raw shapes are rejected; existing query behavior is
-                        unchanged.
+                        Raw shapes require an explicit Framework output schema.
+                        Retain every returned builder when configuring a query.
                     

- Read representation, pagination and migration details + Read representation and pagination details + +
+ +
+

Filtering and ordering schema-aware reads

+

+ Group AND/OR search predicates without changing the + selected row schema. IN/EXISTS subqueries, null checks, + and bound raw predicates and ordering remain explicit. + Every outer operation returns an independent reader. +

+
+                         ({ id: t.id, title: t.title }));
+const read = base.where(t => t.projectId, projectId)
+    .andWhere(group => group
+        .where(t => t.title, 'ilike', pattern)
+        .orWhereExists(labelMatches))
+    .orderByRaw('case when ?? = ? then 0 else 1 end', [
+        base.ref(t => t.id), priorityTaskId
+    ]);
+// read.rowSchema === base.rowSchema`)
+                            }}
+                        />
+                    
+

+ Group callbacks run synchronously once and expose only + predicates. Subquery SQL and bindings are captured on + attachment, with no database execution. Use ref() for + quoted columns and generated aliases; keep values in + bindings and raw SQL fragments application-authored. Raw + apply() requires an explicit output schema. Return the + configured group from every predicate callback. +

+ + Multi-file examples and compatibility boundaries
@@ -534,7 +573,7 @@ const rows = await read.where(t => t.id, taskId);`) Escape hatch - .apply(fn),{' '} + .apply(fn, {'{ output }'}),{' '} .toQuery(),{' '} .toString() diff --git a/websites/docs/app/mapper/page.tsx b/websites/docs/app/mapper/page.tsx index fdf8aed4..9df84e73 100644 --- a/websites/docs/app/mapper/page.tsx +++ b/websites/docs/app/mapper/page.tsx @@ -229,7 +229,7 @@ const registry = mapper()
                          ({ id: u.id, name: u.name }));
 const Source = read.rowSchema; // no SQL
 const toDto = mapper().configure(Source, UserDto, m => m)
diff --git a/websites/docs/app/orm/page.tsx b/websites/docs/app/orm/page.tsx
index 610c276b..58be4c79 100644
--- a/websites/docs/app/orm/page.tsx
+++ b/websites/docs/app/orm/page.tsx
@@ -244,16 +244,16 @@ try {
                 

Detached reads with projection schemas

- db.users.withRowSchema() returns an immutable read-only - query with runtime metadata matching its projection and - relation graph. Results stay detached even in a tracking - context, so partial selections cannot silently become - incomplete tracked entities. + Every DbSet query is immutable, with metadata matching + its decoded projection and relations. Full entities + retain identity tracking when enabled. Projected, + grouped, distinct and raw results remain detached, so + incomplete selections cannot replace tracked entities.

                          ({ id: p.id, name: p.name }))
     .include(r => r.tasks, tasks => tasks
         .select(t => ({ title: t.title })));
@@ -265,7 +265,9 @@ const projects = await read; // one SQL statement`)
                     

Return child queries from customizers. STI/CTI reads expose variantRowSchemas for explicit mapper dispatch. - Ordinary entity reads and writes remain unchanged. + Entity objects remain mutable. Reads, reloads and + returning writes share exact numeric, date and null + representations.

Projection-aware consumer guide diff --git a/websites/docs/app/react-form/page.tsx b/websites/docs/app/react-form/page.tsx index 00f57c52..469b7519 100644 --- a/websites/docs/app/react-form/page.tsx +++ b/websites/docs/app/react-form/page.tsx @@ -307,8 +307,8 @@ function App() { createFormSystem. Its Field{' '} checks the selected value, variant and custom props. Extend it by spreading system.renderers. - Its optional Provider also configures - legacy fields. + Its optional Provider supplies renderers to + descendant fields.

                         
                                     Result of .validate(). Contains{' '}
                                     valid, errors, and{' '}
-                                    object. For object schemas,
-                                    also includes getErrorsFor() (
-                                    errors is{' '}
-                                    deprecated on object schema
-                                    results).
+                                    object. For object schemas, use{' '}
+                                    getErrorsFor() for per-property
+                                    errors.
                                 
                             
                             
diff --git a/websites/schema/app/docs/sections/schema-modifiers.tsx b/websites/schema/app/docs/sections/schema-modifiers.tsx
index 71c123dd..511d1668 100644
--- a/websites/schema/app/docs/sections/schema-modifiers.tsx
+++ b/websites/schema/app/docs/sections/schema-modifiers.tsx
@@ -140,8 +140,8 @@ array(text).parse(['ok', 42]); // ['ok', undefined] — no entries dropped`}
                 

- Legacy optional schemas accept null at runtime - even when their inferred type omits it. A fallback does not + Optional schemas accept null at runtime even + when their inferred type omits it. A fallback does not replace a value that passed validation, so normalize null explicitly when your application requires undefined:

diff --git a/websites/schema/app/docs/sections/validation.tsx b/websites/schema/app/docs/sections/validation.tsx index 7be966a3..bdc98d3f 100644 --- a/websites/schema/app/docs/sections/validation.tsx +++ b/websites/schema/app/docs/sections/validation.tsx @@ -19,9 +19,7 @@ export default function ValidationSection() { only when your schema includes async validators or preprocessors. For object schemas, the result also includes a{' '} getErrorsFor() method for per-property error - inspection — the flat errors array is{' '} - deprecated on object schema results and will be - removed in a future major version. + inspection.

                 {' '}
-                and replaces the deprecated errors array on object
-                schema validation results. It returns an object with{' '}
+                using property selectors. It returns an object with{' '}
                 isValid (boolean), errors (array of
                 error strings), and seenValue (the value that was
                 validated).
diff --git a/websites/schema/app/playground/schemaDeclarations.ts b/websites/schema/app/playground/schemaDeclarations.ts
index 11108581..aaff6960 100644
--- a/websites/schema/app/playground/schemaDeclarations.ts
+++ b/websites/schema/app/playground/schemaDeclarations.ts
@@ -2495,7 +2495,7 @@ export type ObjectSchemaValidationResult p.age).errors // ["is expected to have property 'age'"]
  * \`\`\`
  *