Datum

Datum is an SEO content pipeline built on Payload CMS. It finds content-gap keywords, drafts articles from reusable templates, and checks each draft before publication.

Why Datum is different

Datum makes its editorial rules visible and editable. Editors set template rules in Payload. Maintainers keep shared rules in docs/style-guide.md. The generator and QA checks read both.

Before an editor can approve a draft, Datum checks its structure and readability without an LLM. Three model calls follow: one fact-checks claims against the open web, one reviews the writing against the template, the style guide and your brand voice, and one checks every claim the draft makes about you against an evidence bank you control. Datum records tokens and cost for responses that parse successfully. Responses that fail JSON parsing are not yet recorded.

Everything runs in your Payload and Postgres setup under the MIT license. If a rule is wrong, edit it. If a draft fails, the QA results say which check rejected it.

Workspaces

Path	Role
cms/	Payload CMS 3 app using Next.js 16 and Postgres. Stores templates and articles and provides the admin UI.
pipeline/	Node/TypeScript CLI that advances articles through research → generate → QA → scoring
docs/	Style guide and other content rules that the pipeline reads, plus docs/operations.md for queues, webhooks, caching, and limits

cms and pipeline share one Postgres database and the types generated in cms/src/payload-types.ts. The pipeline imports Payload in the same process. It does not call the CMS over HTTP.

Pipeline and data integration

External SEO data is isolated behind the AhrefsClient contract, which has three methods: contentGapKeywords and discoverKeywords find topics, and serpResearch reads a keyword's live results page. To add another provider, implement that interface and inject it through StageContext; topic discovery and SERP research consume the normalized results. Model calls are routed through one adapter of their own, picked per stage from the model id, so no stage has to choose one. The editable diagram source lives beside the SVG.

What a run reads and writes

A run resolves your workspace's own facts once, before the first stage, so activating an audience halfway through a batch cannot change what the second article was written against. Templates are the exception: each stage re-reads the one attached to the article it is working on. Three of these gate the run — without an active brand voice, a target domain and at least one active audience it refuses to start. As it goes it writes append-only rows for model costs, stage audits and information-gain scores, a corpus snapshot it may re-cluster later, and a review-queue row for every domain it cited that nobody has rated yet, updated in place when the same domain turns up again. The editable diagram source lives beside the SVG.

Article status flow

Research ends at brief_review: Datum writes a brief from the template, the research gaps and your brand voice, and waits. Approving the brief is what starts writing, checks and scoring, so the decision lands before the expensive half rather than after it. Research is not free — it makes a paid SERP request and extracts claims from what already ranks — but the brief is where you stop the far larger writing and review spend. The pipeline never picks up brief_review, needs_revision, needs_review or blocked on its own; those are yours. Publishing is yours too, unless you set a publishAt date on an approved article, in which case the scheduler publishes it through the same gates and audit trail. Its cache purge is the one difference: a manual publish purges in process, while a scheduled one rides the revalidate webhook, so point WEBHOOK_URL at <SITE_URL>/hooks/revalidate or the reader page can serve stale for up to 300 seconds. See docs/operations.md. The editable diagram source lives beside the SVG.

Setup and governance

The setup hub can suggest changes from recurring review corrections without model calls. See Setup suggestions for thresholds and decisions.

Everything that decides how Datum writes sits in the admin nav's Setup and Governance sections, and a run reads it when it starts. The template is the exception: each stage re-reads the one attached to the article it is working on, so editing a template mid-run can reach the later stages. (The nav has five sections in total; Content is covered in First run and making content below, and Records is the raw collections, folded by default, for when you need one directly.)

• Setup checklist (/admin/ops/setup) — seven steps, and /admin keeps showing it until the four required ones are done and the workspace holds at least one piece. Four are required, and a content run refuses to start without them: a workspace with a target domain (/admin/ops/setup/workspace), an active brand voice, at least one active audience (/admin/ops/setup/audiences), and at least one template (/admin/ops/templates). Three are recommended: your positioning (/admin/ops/setup/positioning) and an evidence bank (/admin/ops/setup/evidence) change what a draft may claim, and Models (below) picks which model runs each stage. The workspace, audience and positioning steps each have Draft with AI and Refine with AI, and the evidence bank offers the assistant on two of its three tabs; every reply drafts one section of one asset, so you read it before accepting it. They work from your own site: Fetch site pages reads your home page and up to seven marketing pages linked from it. Start with the demo workspace fills every required and recommended step but Templates and Models with the Kettle & Burr home-espresso demo in one click. See docs/tenant-context.md.
• Brand voice (/admin/ops/setup/brand-voice) — a workspace-wide voice every generated title, description, FAQ, and body follows, layered on top of docs/style-guide.md. Set it up with a nine-step onboarding stepper or by uploading an existing brand guide (.md/.txt/.pdf/.docx) for one-call extraction into a draft you review before activating. An active voice is required for content runs. Seed a demo voice with npm run seed -- --with-brand-voice.
• Models (/admin/globals/llm-settings) — eleven choices: one for each of the eight pipeline stages that call a model (generate, fact check, qualitative review, claim extraction, information-gain judge, evidence verification, evidence check, brief angles), plus brand-voice extraction, the setup assistant, and admin topic relevance. Pick a model here, or leave it blank to fall back to the matching PIPELINE_MODEL_* / BRAND_VOICE_EXTRACT_MODEL / SETUP_ASSIST_MODEL env var, or to claude-opus-5 if neither is set. Brief angles and topic relevance are the exception: unset, they use the small model of the generate model's provider (claude-haiku-4-5, or gpt-5.4-mini when generate is a gpt-* model). Each live model needs its provider's API key (ANTHROPIC_API_KEY for claude-*, OPENAI_API_KEY for gpt-*, o-numbered and chatgpt-* models) wherever that call runs; see the env var split below.
• Sources (/admin/collections/evidence-sources) and Source review (/admin/ops/governance/source-review) — Sources holds the domain rules you have already rated and is what scoring reads. Source review is the queue behind it: the domains the pipeline cited or saw ranking that nobody has rated yet. An unrated domain can't back a claim nobody else is making, so an article resting on one gets blocked; rate the ones you trust and the next run counts them. See docs/information-gain.md.
• Scoring policy (/admin/globals/information-gain-policy) — the thresholds that decide whether a scored draft passes, needs revision, needs your call, or is blocked. Every run stamps the policy version onto the score it writes. See docs/information-gain.md. Delivery settings sit apart from all of that, because no run reads them: Webhooks (/admin/globals/webhook-settings) sends a signed article.status_changed POST on every status transition, falling back to WEBHOOK_URL and WEBHOOK_SECRET. Nothing is sent until both a URL and a secret resolve. See docs/operations.md.

Prerequisites

• Node.js 22+
• npm with workspaces for all scripts and CI.
• Docker for local Postgres through Docker Compose

Quick start

# 1. Env files
cp .env.example .env
cp cms/.env.example cms/.env
# Generate a Payload secret:
#   openssl rand -hex 32
# Set PAYLOAD_SECRET in both .env files, or at least in cms/.env.

# 2. Database
docker compose up -d

# 3. Install & seed
npm install
npm run seed

# 4. Admin UI
npm run dev
# → http://localhost:3000

Seeded local admin

The seed creates this account for local development. Change its password before using Datum in a shared or deployed environment.

• Email: admin@datum.local
• Password: value of SEED_ADMIN_PASSWORD, or datum-dev-password if unset

Pipeline mock mode

Mock mode needs no API keys. Set MOCK_MODE=true, as shown in .env.example, and Datum uses fixtures instead of calling Ahrefs or Anthropic.

npm run pipeline:fetch -- --template Listicle --count 3
npm run pipeline:run
npm run pipeline:report -- --period week

First run and making content

/admin opens on the setup checklist and keeps showing it until the four required steps are done and the workspace holds at least one piece; the three recommended steps never block it. If you would rather look around first, Start with the demo workspace fills every required and recommended step but Templates and Models from the Kettle & Burr home-espresso fixtures in one click; it only ever fills what is still blank, so pressing it twice is safe. Templates are seeded. Missing live-provider keys show as a banner for whoever deploys — they never block an editor.

Then New content (/admin/ops/new) shows the same blockers as a notice if anything required is still missing; otherwise pick the kind of piece (a template card), say what it is about — suggested topics from Ahrefs, or a keyword you already know — and create it. Research starts on its own; the piece opens; when research is done a brief appears with the angle, audience and sections. Edit it, approve it, and Datum writes the draft, runs the checks and scores it. Content (/admin/ops/content) is the one list: every piece on a five-step stepper (Research → Brief → Writing → Review → Publish), who it is waiting on, and the one thing to do next. A stalled piece — waiting on a person rather than a run — carries a Run next stage button on its own page and a Run button on the board; select several and Run selected advances them together. Each of those queues the article as a pipeline-runs row, executed as a native Payload content-run task, so npm run jobs:run --workspace cms (or dev's in-process worker) is what actually processes it; a bar at the bottom of every admin page shows what the model is doing while a run is in flight. pipeline:run reaches the same stages directly from the CLI, outside that queue, and is no longer the only way to move a stalled piece forward. Live mode asks for a paid-provider confirmation before the expensive half starts.

Local development runs all three job queues in process: content and webhooks every two seconds, scheduled every ten. Production intentionally disables in-process autorun, so a durable worker or scheduler has to invoke each one:

npm run jobs:run --workspace cms                                              # one content run
npm run payload --workspace cms -- jobs:run --queue webhooks                  # webhook deliveries
npm run payload --workspace cms -- jobs:run --queue scheduled \
  --handle-schedules --limit 1                                                # publishes due articles

Each invocation of the first command processes one queued content run; schedule it repeatedly for continuous processing. Run the other two every minute or faster. docs/operations.md has the cadence, the delivery contract and the fixed limits.

For live API calls, set MOCK_MODE=false and provide the key for each chosen model's provider (ANTHROPIC_API_KEY or OPENAI_API_KEY). Ahrefs also needs AHREFS_API_KEY plus a target domain and competitors, which come from the Workspace setup page (/admin/ops/setup/workspace); TARGET_DOMAIN and COMPETITOR_DOMAINS are its fallback. See .env.example.

Environment variables

Copy .env.example and cms/.env.example — both. The pipeline CLI loads cms/.env first and the root .env second, but the admin UI (Next.js) only ever loads cms/.env, so any variable the admin needs at request time — API keys, MOCK_MODE, BRAND_VOICE_EXTRACT_MODEL — has to be set in cms/.env too, not just the root file. Environment variables set in the shell override both files.

Variable	Purpose
DATABASE_URL	Postgres connection string
PAYLOAD_SECRET	Payload token signing secret
SITE_URL	Public CMS origin used for canonical article URLs. Falls back to http://localhost:3000 outside production; left unset in production, canonical metadata is omitted.
ANTHROPIC_API_KEY	Claude models (claude-*). Not required in mock mode.
OPENAI_API_KEY	OpenAI models (gpt-*, o-numbered, chatgpt-*). Not required in mock mode.
AHREFS_API_KEY	Keyword and SERP research. Not required in mock mode.
AHREFS_COUNTRY	Country code for keyword and SERP queries (default us)
TOPIC_RELEVANCE_MODEL	CMS-only audience-fit labels for topic discovery; no CLI scoring or content-run key requirement.
TARGET_DOMAIN	Fallback for the Workspace global's target domain. The admin field wins; a mock run falls back again to a demo domain. The .env.example placeholder (example.com) is treated as unset rather than a real domain.
COMPETITOR_DOMAINS	Comma-separated fallback for the Workspace global's competitor list, same rule. The .env.example placeholders (competitor-a.com, competitor-b.com) are likewise treated as unset.
MOCK_MODE	true to use fixtures instead of paid APIs
SEED_ADMIN_PASSWORD	Password for the seeded admin user
PIPELINE_MODEL_GENERATE / _FACT_CHECK / _QUALITATIVE_REVIEW / _CLAIM_EXTRACTION / _INFORMATION_GAIN_JUDGE / _EVIDENCE_VERIFICATION / _EVIDENCE_CHECK / _BRIEF_ANGLE	Fallback model for each of the eight pipeline stages that call a model, when the Models admin field is blank
BRAND_VOICE_EXTRACT_MODEL	Fallback model for brand-guide upload extraction, same rule
SETUP_ASSIST_MODEL	Fallback model for the setup assistant. Falls back to BRAND_VOICE_EXTRACT_MODEL before the platform default.
INFORMATION_GAIN_*	Threshold overrides used when the Scoring policy global is blank. See docs/information-gain.md.
WEBHOOK_URL / WEBHOOK_SECRET	Fallback for the Webhooks global. Nothing is delivered until both resolve. See docs/operations.md.
PAYLOAD_AUTO_LOGIN (cms/.env only)	true to skip the admin login form in local dev; never honoured when NODE_ENV=production
PAYLOAD_AUTO_LOGIN_EMAIL (cms/.env only)	Which seeded user to auto-login as (default admin@datum.local)

Test-only variables

Read only by the test suites, never by the running app. Set these in your shell, not in a committed .env.

Variable	Purpose
DISABLE_DEV_PUSH	true to turn off Payload's dev schema push, which next dev otherwise runs on boot. Playwright's test:e2e script sets it automatically so its parallel workers don't race each other pushing the same schema; set it yourself in the shell that runs npm run dev when you point Playwright at that server via TEST_BASE_URL instead.
TEST_BASE_URL	Base URL Playwright drives instead of starting its own dev server — e.g. http://127.0.0.1:3000 to test against a server you already started with npm run dev. Unset, Playwright starts its own on TEST_PORT.
TEST_PORT	Port for the dev server Playwright starts when TEST_BASE_URL is unset (default 3101).

Root scripts

Script	Description
npm run dev	CMS Next.js dev server
npm run seed	Create or update templates, evidence-source rules, and the admin user. Add -- --with-brand-voice to also seed the Kettle & Burr home-espresso demo workspace: an active brand voice, the workspace profile, two audiences, a position, and an evidence bank.
npm run pipeline:fetch -- --template NAME_OR_ID	Create templated articles from content-gap keywords
npm run pipeline:run	Advance every eligible article one stage set: research (stops at the brief), then generate → QA → scoring for approved briefs
npm run pipeline:report	Print the QA and spending report
npm run jobs:run --workspace cms	Process one queued content run. The production content worker.
npm run typecheck / lint / test	Workspace checks described in CONTRIBUTING.md

Documentation

• CONTRIBUTING.md covers setup, tests, and pull request expectations.
• SECURITY.md explains how to report a vulnerability.
• CODE_OF_CONDUCT.md
• AGENTS.md documents the architecture and conventions for contributors and coding agents; CLAUDE.md points at it.
• RELEASING.md covers the release-please versioning rules.
• docs/style-guide.md contains the editorial rules and banned phrases checked by QA.
• docs/tenant-context.md explains the workspace profile, audiences, positioning and evidence bank, how they gate a run, and how they reach the prompts.
• docs/information-gain.md explains claim scoring, the corpus, and the source review queue.
• docs/operations.md documents the job queues, webhooks, scheduled publishing, caching, and the fixed limits.
• docs/open-source-checklist.md lists the maintainer steps for making the repository public.

Agent helpers live under .claude/ and vendor/claude-plugins/. Datum runs without them.

License

MIT © 2026