Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .changeset/bump-core-permissions-as-associations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
'@haverstack/eleventy': minor
---

Bump `@haverstack/core` to `^0.37.0` and `@haverstack/commons` to `^0.31.0`.

**Existing stacks must be recreated.** Core 0.35.0 stores record permissions as
associations and takes `parentId` off the version tier. The SQLite schema is
`CREATE TABLE IF NOT EXISTS` only and ships no migration, so a database written
by an older core opens and then fails on the first permission write: its
`associations` table keeps a `kind` constraint that refuses `permission` and
`anyone`, and has no `related_role` or `attachment_record_id` column.

`Permission` is gone. A record's ACL is now `AuthorityAssociation[]` — a
`permission` association whose bit is its label and whose `grantee` carries a
required `role`, or an `anyone` association spelling world-read affirmatively.
`[{ access: 'public' }]` becomes `[{ kind: 'anyone', label: 'read' }]`. The
`associations` key narrows to `DataAssociation[]`, since authority kinds now
live in the same table behind a separate key.

The plugin reads rather than writes both sets, so this is a dependency and
documentation change: no module needed a code change, and `StackRecord`'s two
projections keep an association loop from ever seeing an ACL entry. Attachment
candidates are now ordered by core's own total order (`createdAt`, then `id`)
so two uploads landing in the same millisecond stage the same way every build.

`docs/design.md § Rebuilding on change` is corrected, and the correction is the
part worth reading before building rebuild avoidance. Since 0.33–0.35 the
journal-tier ops — `associate`, `dissociate`, `permissions`, `reparent`,
`unlist`, `list` — leave `version` **and** `updatedAt` exactly where they stand.
The previously sketched `updatedAt: { after }` gate would therefore skip a
rebuild for precisely the edits this plugin cares most about: adding a record to
a site, scoping one with `for-site`, embedding an attachment, moving a page, or
publishing a draft. The gate now runs after load and fingerprints the index
(associations and permissions included) rather than asking the query layer a
question `updatedAt` can no longer answer. A watching `--serve` correspondingly
reads `RecordChange.ops`, never `version`.
88 changes: 59 additions & 29 deletions docs/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -326,33 +326,53 @@ scale that is sub-second and fine. Two ways to do better, in increasing order of

### A "did anything change?" gate (one-shot / CI)

`RecordFilter.updatedAt` takes a `{ after: Date }`, so a build can ask whether anything
relevant changed since last time and, if not, exit before emitting — the deploy step
then sees no diff. No server, works with `LocalAdapter`.

Sketch:

1. After a successful build, persist the **maximum `updatedAt` actually observed**
(not `Date.now()` — a write that landed mid-build must not be skipped next time),
alongside the set of permalinks emitted.
2. Next build, for each relevant type run
`query({ filter: { typeId, updatedAt: { after: last }, includeDeleted: true, includeUnlisted: true } })`,
looping the cursor as everywhere else.
3. Empty result → skip load/resolve/emit entirely.
4. Non-empty → full build, then rewrite the state file.

Things that make this subtle:

- **Query every type the build reads, not just members.** A `page-meta`, `menu`,
`site`, or `_attachment` edit moves _that_ record's `updatedAt`, not the described
record's.
- **Associations count** — `associate` / `dissociate` bump `version` / `updatedAt`, so a
new `site` membership or `embed` is caught.
- **Hard deletes (purges) are not caught** — nothing is left to query. Soft deletes show
with `includeDeleted: true`; for purges, diff the persisted permalink set against what
now resolves, or accept that a purge needs a manual full rebuild.
- A query result carries no count, so the gate query obeys the same
cursor-to-exhaustion rule as the load phase.
The obvious form of this — ask `RecordFilter.updatedAt` for `{ after: lastBuild }` and
exit before loading if nothing comes back — **is not sound for this plugin**, and the
reason is worth stating because the query still typechecks and still looks right.

Since core 0.33–0.35 the journal-tier ops (`associate`, `dissociate`, `permissions`,
`reparent`, `unlist`, `list`) leave `version` and `updatedAt` exactly where they stand.
Only `patch` and the whole-record verbs bump. Every one of those no-bump ops changes
what this plugin emits:

| No-bump op | What it moves in the output |
| ------------------------ | ---------------------------------------------------------- |
| `associate`/`dissociate` | `site` membership, `for-site` scoping, `embed` attachments |
| `reparent` | a page's ancestor slugs, so its permalink |
| `unlist`/`list` | whether a record is in listings, feeds and the sitemap |
| `permissions` | what a non-owner build can see at all |

So an `updatedAt` gate would skip the build for exactly the edits a site owner is most
likely to make between builds — adding an article to a site, publishing a draft, moving
a page. A gate that is wrong in the common case is worse than no gate.

The durable record of a no-bump write is the journal, and `getJournal()` is per-record:
it answers "what happened to this record", never "which records changed". There is no
cross-record journal read, so nothing short of the load sweep can see these changes
locally.

**The gate therefore moves after load, not before it.** Load is one bounded set of
queries and is not the slow part; resolve, asset staging and emit are. Sketch:

1. Run load as normal.
2. Fingerprint the index: for every record, its `id`, `version`, `updatedAt`,
`parentId`, `unlistedAt`, and its `associations` and `permissions` in a canonical
order — the no-bump aspects have to be in the hash precisely because `version`
does not cover them.
3. Compare against the fingerprint persisted by the last successful build. Equal →
skip resolve/emit/staging and leave the output tree alone, so the deploy step sees
no diff. Different → full build, then rewrite the state file.

Things that stay subtle:

- **Load every type the build reads, not just members.** A `page-meta`, `menu`, `site`,
or `_attachment` edit moves _that_ record, not the described record — the fingerprint
covers them because load already fetches them.
- **Hard deletes (purges) are caught by this shape**, unlike the `updatedAt` gate: a
purged record simply stops appearing in the sweep, which changes the fingerprint.
Soft deletes need `includeDeleted: true` only if the build wants to distinguish them.
- A query result carries no count, so the sweep obeys the same cursor-to-exhaustion
rule as the load phase.

### A watching `--serve` (needs a stack server)

Expand All @@ -376,8 +396,18 @@ the right call.
needed if the integration drives Eleventy programmatically instead.
- The subscription's `ChangeFilter` only narrows by `typeId` / `parentId` / `entityId` /
`kinds` — not content or associations — so expect changes the current site does not
care about and let the rebuild sort it out. Publishing a record (clearing
`unlistedAt`) arrives as an ordinary `changed` event, so no special case.
care about and let the rebuild sort it out.
- **Read `RecordChange.ops`, not `version`.** A change frame carries `ops: ChangeOp[]`,
and the journal-tier ops (`associate`, `dissociate`, `permissions`, `reparent`,
`unlist`, `list`) report the record's version and `updatedAt` unchanged. A watcher
that dedupes on `version` would drop precisely the membership, publish and move
events the site cares about. Publishing a record arrives as a `list` op under a
`changed` kind, so it needs no special case beyond not being filtered out by a
version comparison.
- `associate`/`dissociate` frames carry `associationsAdded`/`associationsRemoved`, so a
watcher can tell a `site` membership change from an unrelated tag without refetching
the record. Since the rebuild is full either way, this is only useful for deciding
whether to rebuild at all.

### Why not page-level incremental

Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@
"@11ty/eleventy": ">=3.0.0"
},
"dependencies": {
"@haverstack/commons": "^0.25.0",
"@haverstack/core": "^0.31.0",
"@haverstack/commons": "^0.31.0",
"@haverstack/core": "^0.37.0",
"markdown-it": "^15.0.1",
"sanitize-html": "^2.17.7"
},
Expand Down
20 changes: 10 additions & 10 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions pnpm-workspace.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@ allowBuilds:
# @haverstack/* publish frequently enough to land inside a minimumReleaseAge
# supply-chain policy window; excluded so a fresh release still installs.
minimumReleaseAgeExclude:
- '@haverstack/commons@0.25.0'
- '@haverstack/core@0.31.0'
- '@haverstack/commons@0.31.0'
- '@haverstack/core@0.37.0'
5 changes: 4 additions & 1 deletion src/assets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,10 @@ export function collectAssets(
if (!staged) continue;
// One fileId can have several _attachment@1 records (one per upload),
// each with its own filename. The body might reference any of them,
// so every known name points at the one staged path.
// so every known name points at the one staged path. This is why the
// association's `attachmentRecordId` is not consulted: content
// addressing gives a fileId exactly one staged path, so resolving
// which upload a reference came from would pick the same string.
for (const meta of index.attachmentsByFileId.get(fileId) ?? []) {
const filename = meta.content.filename;
if (typeof filename === 'string' && filename) embedMap.set(filename, staged.assetPath);
Expand Down
8 changes: 7 additions & 1 deletion src/load.ts
Original file line number Diff line number Diff line change
Expand Up @@ -312,8 +312,14 @@ export async function load(stack: StackClient, opts: LoadOptions = {}): Promise<
list.push(attachment);
attachmentsByFileId.set(fileId, list);
}
// Core's total order for "first recorded" (see firstRecordedAttachment in
// @haverstack/core/wire): createdAt, then id, so two uploads landing in the
// same millisecond still order the same way on every build.
for (const list of attachmentsByFileId.values()) {
list.sort((a, b) => a.createdAt.getTime() - b.createdAt.getTime());
list.sort(
(a, b) =>
a.createdAt.getTime() - b.createdAt.getTime() || (a.id < b.id ? -1 : a.id > b.id ? 1 : 0),
);
}

const byId = new Map<RecordId, StackRecord>();
Expand Down
4 changes: 2 additions & 2 deletions tests/assets.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { mkdtemp, readFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { beforeEach, describe, expect, test } from 'vitest';
import { Stack, type Permission } from '@haverstack/core';
import { Stack, type AuthorityAssociation } from '@haverstack/core';
import { MemoryAdapter } from '@haverstack/core/testing';
import { ARTICLE, PAGE, PHOTO, SITE } from '@haverstack/commons';
import {
Expand All @@ -16,7 +16,7 @@ import {
stageAssets,
} from '../src/index.js';

const PUBLIC: Permission[] = [{ access: 'public' }];
const PUBLIC: AuthorityAssociation[] = [{ kind: 'anyone', label: 'read' }];
const iso = (d: string) => new Date(d).toISOString();
const onSite = (id: string) => ({
kind: 'relationship' as const,
Expand Down
8 changes: 4 additions & 4 deletions tests/check.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { beforeEach, describe, expect, test } from 'vitest';
import { Stack, type Association, type Permission } from '@haverstack/core';
import { Stack, type AuthorityAssociation, type DataAssociation } from '@haverstack/core';
import { MemoryAdapter } from '@haverstack/core/testing';
import { ARTICLE, PAGE, SITE } from '@haverstack/commons';
import {
Expand All @@ -13,14 +13,14 @@ import {
SITE_MEMBERSHIP_LABEL,
} from '../src/index.js';

const PUBLIC: Permission[] = [{ access: 'public' }];
const PUBLIC: AuthorityAssociation[] = [{ kind: 'anyone', label: 'read' }];
const iso = (d: string) => new Date(d).toISOString();
const onSite = (id: string): Association => ({
const onSite = (id: string): DataAssociation => ({
kind: 'relationship',
label: SITE_MEMBERSHIP_LABEL,
target: { scope: 'record', recordId: id },
});
const forSite = (id: string): Association => ({
const forSite = (id: string): DataAssociation => ({
kind: 'relationship',
label: FOR_SITE_LABEL,
target: { scope: 'record', recordId: id },
Expand Down
4 changes: 2 additions & 2 deletions tests/emit.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { mkdtemp } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { beforeEach, describe, expect, test } from 'vitest';
import { Stack, type Permission } from '@haverstack/core';
import { Stack, type AuthorityAssociation } from '@haverstack/core';
import { MemoryAdapter } from '@haverstack/core/testing';
import { ARTICLE, PAGE, SITE } from '@haverstack/commons';
import {
Expand All @@ -16,7 +16,7 @@ import {
type EmitOptions,
} from '../src/index.js';

const PUBLIC: Permission[] = [{ access: 'public' }];
const PUBLIC: AuthorityAssociation[] = [{ kind: 'anyone', label: 'read' }];
const iso = (d: string) => new Date(d).toISOString();
const onSite = (id: string) => ({
kind: 'relationship' as const,
Expand Down
4 changes: 2 additions & 2 deletions tests/feeds.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { beforeEach, describe, expect, test } from 'vitest';
import { Stack, type Permission } from '@haverstack/core';
import { Stack, type AuthorityAssociation } from '@haverstack/core';
import { MemoryAdapter } from '@haverstack/core/testing';
import { ARTICLE, PAGE, SITE } from '@haverstack/commons';
import {
Expand All @@ -13,7 +13,7 @@ import {
type ResolvedSite,
} from '../src/index.js';

const PUBLIC: Permission[] = [{ access: 'public' }];
const PUBLIC: AuthorityAssociation[] = [{ kind: 'anyone', label: 'read' }];
const iso = (d: string) => new Date(d).toISOString();
const onSite = (id: string) => ({
kind: 'relationship' as const,
Expand Down
4 changes: 2 additions & 2 deletions tests/load.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { beforeEach, describe, expect, test } from 'vitest';
import { Stack, type Permission } from '@haverstack/core';
import { Stack, type AuthorityAssociation } from '@haverstack/core';
import { MemoryAdapter } from '@haverstack/core/testing';
import { ARTICLE, PAGE, SITE } from '@haverstack/commons';
import {
Expand All @@ -12,7 +12,7 @@ import {
SITE_MEMBERSHIP_LABEL,
} from '../src/index.js';

const PUBLIC: Permission[] = [{ access: 'public' }];
const PUBLIC: AuthorityAssociation[] = [{ kind: 'anyone', label: 'read' }];
const iso = (d: string) => new Date(d).toISOString();

const onSite = (siteId: string) => ({
Expand Down
6 changes: 3 additions & 3 deletions tests/publish.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { beforeEach, describe, expect, test } from 'vitest';
import { Stack, type Association, type Permission } from '@haverstack/core';
import { Stack, type AuthorityAssociation, type DataAssociation } from '@haverstack/core';
import { MemoryAdapter } from '@haverstack/core/testing';
import { ARTICLE, PAGE, PHOTO, POST, SITE } from '@haverstack/commons';
import {
Expand All @@ -9,9 +9,9 @@ import {
SITE_MEMBERSHIP_LABEL,
} from '../src/index.js';

const PUBLIC: Permission[] = [{ access: 'public' }];
const PUBLIC: AuthorityAssociation[] = [{ kind: 'anyone', label: 'read' }];
const iso = (d: string) => new Date(d).toISOString();
const onSite = (id: string): Association => ({
const onSite = (id: string): DataAssociation => ({
kind: 'relationship',
label: SITE_MEMBERSHIP_LABEL,
target: { scope: 'record', recordId: id },
Expand Down
6 changes: 3 additions & 3 deletions tests/resolve.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { beforeEach, describe, expect, test } from 'vitest';
import { Stack, type Association, type Permission } from '@haverstack/core';
import { Stack, type AuthorityAssociation, type DataAssociation } from '@haverstack/core';
import { MemoryAdapter } from '@haverstack/core/testing';
import { ARTICLE, PAGE, POST, SITE } from '@haverstack/commons';
import {
Expand All @@ -14,7 +14,7 @@ import {
type ResolveConfig,
} from '../src/index.js';

const PUBLIC: Permission[] = [{ access: 'public' }];
const PUBLIC: AuthorityAssociation[] = [{ kind: 'anyone', label: 'read' }];
const iso = (d: string) => new Date(d).toISOString();
const CONFIG: ResolveConfig = { slugStrategy: 'title' };

Expand Down Expand Up @@ -63,7 +63,7 @@ async function worked() {
const meta = (
parentId: string,
content: Record<string, unknown>,
associations: Association[] = [],
associations: DataAssociation[] = [],
) => stack.create(PAGE_META.id, content, { parentId, permissions: PUBLIC, associations });
const article = (
content: Record<string, unknown>,
Expand Down
Loading