feat(framework)!: immutable queries and automatic row schemas (v5)

.changeset/immutable-query-major.md

@@ -0,0 +1,31 @@
+---
+"@cleverbrush/async": major
+"@cleverbrush/auth": major
+"@cleverbrush/client": major
+"@cleverbrush/deep": major
+"@cleverbrush/di": major
+"@cleverbrush/env": major
+"@cleverbrush/knex-clickhouse": major
+"@cleverbrush/knex-schema": major
+"@cleverbrush/log": major
+"@cleverbrush/mapper": major
+"@cleverbrush/orm-cli": major
+"@cleverbrush/orm": major
+"@cleverbrush/otel": major
+"@cleverbrush/react-form": major
+"@cleverbrush/scheduler": major
+"@cleverbrush/schema-json": major
+"@cleverbrush/schema": major
+"@cleverbrush/server-openapi": major
+"@cleverbrush/server": major
+---
+
+Make Framework query builders immutable and infer row schemas automatically.
+
+Retain returned query builders, return synchronous builders from scopes and grouped predicates, and supply an explicit Framework object output schema for opaque raw SELECTs. Ordinary, aliased, polymorphic and ORM queries expose their row schemas directly. Projections replace scalar selections, and projected/aggregate/raw queries cannot perform entity writes. Reads and write-returning rows consistently preserve exact decimal/bigint strings, Date objects and SQL nulls.
+
+All published Framework packages advance together to the next major version. Tracked entity objects remain mutable.
+
+### Migrating from v4.x to v5
+
+Remove `withRowSchema()` calls and retain each configured query instead of relying on mutation. Replace raw base-query overloads with explicit output contracts. See `libs/knex-schema/MIGRATION-v5.md` for the complete migration guide.

.changeset/schema-aware-read-predicates.md

@@ -0,0 +1,10 @@
+---
+"@cleverbrush/knex-schema": minor
+---
+
+Add shape-preserving grouped AND/OR predicates, captured IN/EXISTS subqueries,
+bound raw predicates and ordering, and typed SQL column references to ordinary
+and aliased schema-aware readers. These capabilities also work in ordinary ORM
+reads and nested relation customizers while retaining immutable query plans and
+stable row-schema identity. Group callbacks are synchronous and predicate-only;
+unrestricted raw query mutation remains unavailable.

.changeset/schema-boundaries.md

@@ -10,4 +10,4 @@ Shape, validation-rule, default, fallback and extension changes clear inherited
 names; apply schemaName after those edits to establish a new named definition.
 Preserve one canonical definition in JSON
 Schema, OpenAPI and AsyncAPI with strict name collision checks. Keep existing
-type inference and legacy optional null acceptance unchanged.
+type inference and optional null acceptance unchanged.

AGENTS.md

@@ -135,6 +135,17 @@ The `demos/` directory is linted separately (see `demos/todo-backend/biome.json`
 - Target `ES2022`; use modern syntax freely
 - Type assertions with `as` are acceptable (the linter won't block them)
 
+## Documentation and Project Boundaries
+
+- Framework is an independent, application-agnostic project. Use generic domain
+  examples in source, documentation, changesets and PR descriptions. Consumer
+  application references belong only in website showcase links.
+- Describe the current supported API in READMEs, guides and JSDoc. Keep historical
+  API comparisons and upgrade instructions in explicitly labeled v4.x-to-v5
+  migration documentation, and link to it from current guides where useful.
+- Preserve accurate API contracts and deprecation annotations; do not change
+  runtime behavior just to simplify documentation.
+
 ---
 
 ## Testing Conventions

demos/e2e/README.md

@@ -27,7 +27,7 @@ npx vitest --run src/api/todos.api.test.ts    # single file
 
 ### One-time browser install
 
-The UI project requires Chromium. After `npm install`, run:
+The UI project requires Chromium. After `npm ci`, run:
 
 ```bash
 cd demos/e2e && npx playwright install chromium
@@ -41,6 +41,7 @@ cd demos/e2e && npx playwright install chromium
 | `RESET`                   | `0`                      | If `1`, run `docker compose down -v` before bringing the stack up (wipes Postgres).    |
 | `CI`                      | unset                    | Setting `CI=true` flips `KEEP_STACK` default to `0` and forces full teardown.          |
 | `HEADED`                  | `0`                      | If `1`, launch Chromium headed so you can watch UI tests run.                          |
+| `E2E_BROWSER_EXECUTABLE_PATH` | unset                 | Optional path to an existing Chromium executable; otherwise uses Playwright's managed browser. |
 | `SLOWMO`                  | `0`                      | Slow-motion delay (ms) for Playwright actions — useful with `HEADED=1`.                |
 | `E2E_API_URL`             | `http://localhost:3000`  | Backend HTTP base URL.                                                                 |
 | `E2E_WS_URL`              | `ws://localhost:3000`    | Backend WebSocket base URL.                                                            |
@@ -115,7 +116,7 @@ src/
 - **Todos** — full CRUD, list pagination, `getWithAuthor`, polymorphic
   events (`assigned` / `commented` / `completed`), optimistic concurrency
   on `complete` (200 / 409 with `If-Match`), cross-user 403, attachment
-  download, `legacyReplace` redirect.
+  download, PUT redirect.
 - **Import / Export** — 207 small batch, 202 large batch, idempotency
   header (contract-level), CSV export with quoting + content headers.
 - **Users (admin)** — list (admin only), delete user, self-delete blocked,

demos/e2e/src/api/telemetry.smoke.test.ts

@@ -41,8 +41,6 @@ describe('Telemetry smoke — ClickHouse logs & traces correlation', () => {
             TraceId: string;
         }>(
             `SELECT body AS Body, trace_id AS TraceId FROM signoz_logs.distributed_logs_v2 WHERE trace_id = '${traceId}' FORMAT JSON`,
-ace_id = '${traceId}' FORMAT JSON`,
-
             45_000,
             1_000
         );
@@ -55,10 +53,8 @@ ace_id = '${traceId}' FORMAT JSON`,
         const spans = await waitForRows<{
             SpanName: string;
             ServiceName: string;
-        }>(name AS SpanName, resources_string['service.name'] AS ServiceName FROM signoz_traces.distributed_signoz_index_v3 WHERE trace_id = '${traceId}' FORMAT JSON`,
-
+        }>(
             `SELECT name AS SpanName, resources_string['service.name'] AS ServiceName FROM signoz_traces.distributed_signoz_index_v3 WHERE trace_id = '${traceId}' FORMAT JSON`,
-
             45_000,
             1_000
         );
@@ -67,7 +63,7 @@ ace_id = '${traceId}' FORMAT JSON`,
         expect(services.has('todo-backend')).toBe(true);
     });
 
-    it('ClickHouse is reachable and reports recent losignoz_logs.distributed_logs_v2 WHERE toDateTime(intDiv(timestamp, 1000000000))
+    it('ClickHouse is reachable and reports recent logs', async () => {
         const { rows } = await clickhouseQuery<{ recent: string }>(
             `SELECT toString(count()) AS recent FROM signoz_logs.distributed_logs_v2 WHERE toDateTime(intDiv(timestamp, 1000000000)) >= now() - INTERVAL 1 HOUR FORMAT JSON`
         );

demos/e2e/src/api/todos.api.test.ts

@@ -228,10 +228,17 @@ describe('Todos — attachment & legacyReplace', () => {
                 body: { title: uniqueTitle('attach') }
             })
         );
+        const boundary = 'framework-attachment-fixture';
+        const uploaded = await r('POST', `/api/todos/${created.id}/attachment`, {
+            raw: true,
+            headers: { 'content-type': `multipart/form-data; boundary=${boundary}` },
+            body: `--${boundary}\r\nContent-Disposition: form-data; name="attachment"; filename="example.txt"\r\nContent-Type: text/plain\r\n\r\nImmutable query demo\r\n--${boundary}--\r\n`
+        });
+        expect(uploaded.status).toBe(201);
         const res = await r('GET', `/api/todos/${created.id}/attachment`);
         expect(res.status).toBe(200);
         expect(res.headers['content-type']).toMatch(/text\/plain/);
-        expect(res.body.length).toBeGreaterThan(0);
+        expect(res.body).toBe('Immutable query demo');
     });
 
     it('legacyReplace (PUT) returns a redirect', async () => {

demos/e2e/src/support/playwright.ts

@@ -11,6 +11,7 @@ let browser: Browser | null = null;
 async function getBrowser(): Promise<Browser> {
     if (!browser) {
         browser = await chromium.launch({
+            executablePath: process.env.E2E_BROWSER_EXECUTABLE_PATH || undefined,
             headless: !config.headed,
             slowMo: config.slowMo
         });

demos/e2e/src/ui/todo-crud.ui.test.ts

@@ -62,6 +62,12 @@ describe('UI — todo CRUD', () => {
 
             // Detail page after creation
             await page.waitForURL(/\/todos\/\d+$/, { timeout: 10_000 });
+            // Stay on the detail page until its requests settle: an admin-only
+            // picker lookup must not sign out an ordinary user after creation.
+            await page.getByRole('button', { name: 'Save Changes', exact: true }).waitFor();
+            await page.waitForLoadState('networkidle');
+            expect(page.url()).toMatch(/\/todos\/\d+$/);
+            expect(await page.locator('input').first().inputValue()).toBe(title);
             const url = page.url();
             const todoId = Number(url.match(/\/todos\/(\d+)$/)![1]);
 

demos/todo-backend/src/api/handlers/todos.ts

@@ -452,13 +452,13 @@ export const uploadAttachmentHandler: Handler<
     return ActionResult.created({
         id: updated.id,
         title: updated.title,
-        description: updated.description,
+        description: updated.description ?? undefined,
         completed: updated.completed,
         userId: updated.userId,
         createdAt: updated.createdAt,
         updatedAt: updated.updatedAt,
-        attachmentName: updated.attachmentName,
-        attachmentMimeType: updated.attachmentMimeType,
+        attachmentName: updated.attachmentName ?? undefined,
+        attachmentMimeType: updated.attachmentMimeType ?? undefined,
         attachmentSize: file.size
     });
 };

demos/todo-backend/src/api/mappers.ts

@@ -7,7 +7,6 @@ import type { TodoActivityResponse } from './schemas.js';
 const UserRowSchema = object({
     id: number(),
     email: string(),
-    passwordHash: string().optional(),
     role: string(),
     authProvider: string(),
     createdAt: date()
@@ -16,13 +15,13 @@ const UserRowSchema = object({
 const TodoRowSchema = object({
     id: number(),
     title: string(),
-    description: string().optional(),
+    description: string().nullable(),
     completed: boolean(),
     userId: number(),
     createdAt: date(),
     updatedAt: date(),
-    attachmentName: string().optional(),
-    attachmentMimeType: string().optional()
+    attachmentName: string().nullable(),
+    attachmentMimeType: string().nullable()
 });
 
 export const mappingRegistry = mapper()
@@ -33,14 +32,18 @@ export const mappingRegistry = mapper()
         m
             .for(t => t.description)
             .compute(f => f.description ?? undefined)
+            .for(t => t.attachmentName)
+            .compute(f => f.attachmentName ?? undefined)
+            .for(t => t.attachmentMimeType)
+            .compute(f => f.attachmentMimeType ?? undefined)
             .for(t => t.attachmentSize)
             .ignore()
     );
 
 const _mapUserFn = mappingRegistry.getMapper(UserRowSchema, UserResponseSchema);
 const _mapTodoFn = mappingRegistry.getMapper(TodoRowSchema, TodoResponseSchema);
 
-export const mapUser = (row: UserDb) => _mapUserFn(row);
+export const mapUser = (row: Omit<UserDb, 'passwordHash'>) => _mapUserFn(row);
 export const mapTodo = (row: TodoDb) => _mapTodoFn(row);
 
 export function mapTodoActivity(row: ActivityDb & Record<string, unknown>): TodoActivityResponse {

demos/todo-backend/src/db/schemas.ts

@@ -1,4 +1,5 @@
 import {
+    type EntityResult,
     array,
     boolean,
     date,
@@ -144,12 +145,7 @@ const TodoSchema = object({
         'attachmentMimeType'
     )
     .projection('ownership', 'id', 'userId')
-    .scope(
-        'recentFirst',
-        (q: {
-            orderBy: (column: string, direction: 'asc' | 'desc') => unknown;
-        }) => q.orderBy('created_at', 'desc')
-    );
+    .scope('recentFirst', q => q.orderBy('createdAt', 'desc'));
 
 export const TodoEntity = defineEntity(TodoSchema)
     .belongsTo(
@@ -187,32 +183,6 @@ export const entityMap: AppEntityMap = {
 
 // ── Plain row types (used by mappers) ───────────────────────────────────────
 
-export type ActivityDb = {
-    id: number;
-    todoId: number;
-    type: string;
-    actorUserId?: number;
-    completedAt?: Date | null;
-    createdAt: Date;
-};
-
-export type UserDb = {
-    id: number;
-    email: string;
-    passwordHash?: string;
-    role: string;
-    authProvider: string;
-    createdAt: Date;
-};
-
-export type TodoDb = {
-    id: number;
-    title: string;
-    description?: string;
-    completed: boolean;
-    userId: number;
-    createdAt: Date;
-    updatedAt: Date;
-    attachmentName?: string;
-    attachmentMimeType?: string;
-};
+export type ActivityDb = EntityResult<typeof TodoActivityBaseEntity>;
+export type UserDb = EntityResult<typeof UserEntity>;
+export type TodoDb = EntityResult<typeof TodoEntity>;

demos/todo-frontend/src/features/todos/TodoDetailPage.tsx

@@ -18,13 +18,15 @@ import { Field, useSchemaForm } from '@cleverbrush/react-form';
 import { UpdateTodoBodySchema } from '@cleverbrush/todo-backend/contract';
 import { ApiError, isTimeoutError, isNetworkError } from '@cleverbrush/client';
 import { client } from '../../api/client';
+import { useAuth } from '../../lib/auth-context';
 
 type TodoEvent = Parameters<typeof client.todos.sendEvent>[0]['body'];
 import { ConfirmDialog } from '../../components/ConfirmDialog';
 
 type TodoWithAuthor = Awaited<ReturnType<typeof client.todos.getWithAuthor>>;
 
 export function TodoDetailPage() {
+    const { isAdmin } = useAuth();
     const { id } = useParams<{ id: string }>();
     const navigate = useNavigate();
 
@@ -79,11 +81,13 @@ export function TodoDetailPage() {
 
     // Load user list once for the "assigned" picker
     useEffect(() => {
+        // The user directory is admin-only. Regular users retain the ID input.
+        if (!isAdmin) return;
         client.users
             .list({ query: { page: 1, limit: 100 } })
             .then(rows => setUsers(rows.map(u => ({ id: u.id, email: u.email }))))
             .catch(() => {/* ignore — picker will fall back to text input */});
-    }, []);
+    }, [isAdmin]);
 
     useEffect(() => { load(); }, [load]);
 

libs/client/CHANGELOG.md

@@ -26,7 +26,7 @@
 
 ### Minor Changes
 
-- c75bff4: Add framework affordances discovered while reviewing Xpenser:
+- c75bff4: Add reusable environment, HTTP, cache, database and migration capabilities:
 
   - `@cleverbrush/env`: add `envBoolean()` for environment-style boolean values
     such as `1`, `0`, `yes`, `no`, `on`, and `off`.

libs/deep/README.md

@@ -98,12 +98,9 @@ deepEqual(new Map(), new Map());
 // => false (opaque objects compare by identity)
 ```
 
-**Breaking comparison corrections:** previous versions could throw for object/null
-pairs, consider Dates equal to unrelated objects, compare distinct opaque objects
-by enumerable shape, and reject equivalent cycles or repeated references. Signed
-zero, invalid Dates, symbol keys and sparse arrays now follow the rules above.
-Audit consumers relying on those outcomes. To compare Maps/Sets/custom instances
-by content, explicitly project their relevant state into plain data first.
+To compare Maps/Sets/custom instances by content, explicitly project their
+relevant state into plain data first. For upgrade considerations, see the
+[v4.x-to-v5 migration guide](../../docs/cache-form-migration.md#shared-data-snapshots-and-equality-breaking).
 
 ### `deepExtend(...objects)`
 

libs/env/CHANGELOG.md

@@ -24,7 +24,7 @@
 
 ### Minor Changes
 
-- c75bff4: Add framework affordances discovered while reviewing Xpenser:
+- c75bff4: Add reusable environment, HTTP, cache, database and migration capabilities:
 
   - `@cleverbrush/env`: add `envBoolean()` for environment-style boolean values
     such as `1`, `0`, `yes`, `no`, `on`, and `off`.

libs/knex-schema/CHANGELOG.md

@@ -22,7 +22,7 @@
 
 ### Minor Changes
 
-- c75bff4: Add framework affordances discovered while reviewing Xpenser:
+- c75bff4: Add reusable environment, HTTP, cache, database and migration capabilities:
 
   - `@cleverbrush/env`: add `envBoolean()` for environment-style boolean values
     such as `1`, `0`, `yes`, `no`, `on`, and `off`.