Research Force

Research Force is an MIT-licensed collaborative research workspace and public archive. ResearchForce.org is its first installation, beginning with the P1030680 Enigma project. Other teams can run independent installations from the same source.

The implementation follows the requirements, design, and task checklist. The checklist and acceptance inventory distinguish implemented behavior from remaining integrations and release evidence.

Software and research licensing

The software and its development documentation are covered by the MIT licence. Third-party dependencies and assets retain their own licences and notices.

Research records, attachments, imported journals, and other instance content are separate from the software licence. Each instance runner or owner chooses and configures the licence for that instance's research content, subject to the authors' agreements and third-party rights. ResearchForce.org's content licence is undecided. MIT is never a default content licence or an automatic grant over research stored in an installation.

Private research, uploads, backups, credentials, and real personal data stay outside the source repository. Examples and tests use synthetic data. The deployment work will make site identity, trusted origins, group policy text, sending address, Responsible Person, and content licence explicit installation configuration. The initial scope remains one installation per team using the Cloudflare architecture in the design.

Local checks

Use Node 22.22.2 (nvm use) and npm 11.21.1. The lockfile records the complete workspace toolchain. npm 10 can crash while resolving this toolchain's peer dependencies; use the pinned npm version when installing:

npx --yes npm@11.21.1 ci
npm run ci
npm run build
npm run check:workers

• lint: oxlint, stylelint design-token rules, and resolved import boundaries.
• typecheck: boundary checks and TypeScript 6 project-reference builds. Declaration output stays under ignored .cache/ directories, outside browser assets.
• test: Vitest in single-run mode, with Node and local workerd projects. Miniflare needs permission to open loopback ports. Storage uses local D1/FTS5, R2, and SQLite Durable Objects. AI is absent and operations is a local stub; remote bindings are disabled.
• test:nightly: the same single-run suite with FC_RUNS=1000. Defaults are 100 iterations. Lower or invalid values are refused. fast-check failures retain the failing seed and shrink path for replay. See packages/testing/src/fc.ts for the property tag format.
• build: the React 19/react-router 7 Workspace and hashed public shell assets, built with Vite 8. No watcher or development server starts.
• check:workers: Wrangler dry-run bundles for all four Workers, in both top-level and staging environments. Run build first. It never deploys or provisions resources.
• db:generate: drizzle-kit migrations for rf-core, rf-search, and rf-public from the Drizzle schemas. It writes files only and never applies them to a database.

Local development

npm run dev

This applies all three D1 migration sets locally, seeds synthetic data, starts rf-app through Wrangler/Miniflare at http://127.0.0.1:8787, and starts Vite at http://127.0.0.1:5173 with /api proxied to the Worker. Vite fails if its selected port is occupied so the configured Workspace origin cannot silently change. Stop the runner with Ctrl-C.

To enable local sign-in, put this line in the ignored repository-root .dev.vars file, then start or restart npm run dev:

RF_DEV_AUTH=1

Open http://127.0.0.1:5173/api/dev/sign-in and choose a synthetic Member. The Workspace also redirects unauthenticated local sessions there and shows controls to switch Members or sign out. Normal current Role, access, profile and session checks still apply; a Pending Member activates through the normal invitation path, and a Revoked Member cannot sign in. The seeded Profiles include a default specialty so the existing research screens are immediately usable. No licence agreement is generated.

The opt-in is read only from that file, not from inherited shell flags. Remove the line and restart to disable local sign-in; its endpoints then return 404. The runner generates a signing key with private file permissions under ignored .wrangler/dev/local-auth-key.pem, retaining it across normal restarts. Assertions stay in an HttpOnly, SameSite cookie and pass the existing JWT verifier; they are not displayed or written to logs. Signing out ends the stored session as well as clearing the cookie.

Production uses apps/app/src/index.ts and refuses a local-auth flag or key. Only the local runner selects apps/app/src/local.ts; it checks the actual launch config for local bindings and loopback origins. scripts/deploy.ts --check <config> refuses local-auth variables, local entrypoints and unsafe named-environment/vars-file settings, and performs no deployment. The Worker dry-run checks also run this guard. Full deployment orchestration remains task 48.4.

The controlled local entrypoint supplies OPS, AI and email stand-ins. Its generated config has a local SQLite LOCAL_SERVICES Durable Object and no remote AI/service binding, production credentials, public route or cron schedule. Production and deployment guards refuse the local service flag and namespace. Signed browser uploads and the download Worker are still outside this runner.

Service	Local behavior	Requires real staging verification
Cloudflare Access / OPS	Synthetic JWT sign-in; membership add/remove/session-revocation calls are recorded and a simulated Access list is maintained	Identity providers, Access policies, edge enforcement and actual remote session revocation
Workers AI	AI.run returns a deterministic, visibly synthetic text fixture for bounded non-streaming JSON and @cf/ model names; diagnostic history stores a digest rather than prompt bodies	Real model availability, output quality, streaming, token/Neuron accounting and provider latency
Microsoft 365	The real Mailer queue, batching, retry and rate-limit code writes each accepted message to the local capture store	Graph OAuth/RBAC, delivery, MIME conversion, retention and SPF/DKIM/DMARC

Sign in as the synthetic Administrator and follow Local email outbox (Administrator), or open http://127.0.0.1:5173/api/v1/dev/outbox. Invite a synthetic address through Members to exercise OPS and immediate invitation capture. The outbox retains 200 messages, pages 50 summaries at a time, and shows plain text and escaped HTML source with scripts and external resources disabled. A normal queue status of sent means accepted by the capture provider here; no delivery takes place. Members without the current Administrator Role cannot inspect other recipients’ captures.

Captures and call history survive server reloads/restarts in .wrangler/dev-state/; dev:reset clears them with the rest of the local state. Diagnostic history is also bounded. /api/v1/dev/services provides Administrator-only JSON diagnostics. From the signed-in Administrator’s Workspace page, the local AI probe can be exercised without implementing the future research drafting screens:

fetch('/api/v1/dev/ai', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json', 'X-RF-CSRF': '1' },
  body: JSON.stringify({ model: '@cf/meta/llama-3.1-8b-instruct', prompt: 'Synthetic demonstration' }),
}).then(response => response.json());

The probe is a local tool, not completion of the research AI-drafting feature in task 42. No prompts or messages are sent to Cloudflare AI or Microsoft 365 by these stand-ins. Current OPS simulation covers the membership RPCs; publishing, WAF and deployment behavior remains in its own tasks.

npm run dev:seed
npm run dev:reset

dev:seed applies pending local migrations and fills missing fixture identities without replacing edits. dev:reset requires the runner to be stopped, removes only .wrangler/dev-state, then migrates and seeds again. Neither command starts servers. All generated configs, fixture SQL/files and logs stay under ignored .wrangler/dev/; persistent D1, R2 and Durable Object data stay under .wrangler/dev-state/. A run lock prevents simultaneous reset/start commands. If a forcibly killed process leaves .wrangler/dev.lock, confirm the old runner has stopped before removing that lock.

Fixtures use example.test addresses: admin, researcher, colleague, pending and revoked. They include two Projects, the eight currently implemented Research_Item kinds, a large journal section, a Reply, Restricted correspondence/discussion, a CSV Attachment, private full-text search data and synthetic files in both local buckets. Research and Projects start Not Published; no public research views or licence agreements are invented. Research_Questions, Work_Items and Decision_Records will gain fixtures when their implementations arrive.

To change ports, set RF_DEV_WORKER_PORT and/or RF_DEV_WEB_PORT before running the command. Both listeners bind only to 127.0.0.1. The runner builds an allow-list local configuration rather than reusing production or staging configuration; generated files should not be edited by hand.

Structure and boundaries

packages/ contains domain, kernel, markdown, manifest, design, public-render, search-public, db, and testing. apps/ contains the app Worker and its web/ SPA, the public site shell, and submit, dl, and ops Workers. tools/ holds migration helpers as they are implemented.

Package modules use subpath imports such as @researchforce/domain/dates. Each package has a wildcard source export and its own composite TypeScript project. The direct references and oxlint restrictions follow the design's dependency table. The additional boundary checker resolves relative paths and checks static imports, re-exports, import types, dynamic imports, and require calls, including type-only kernel access from public-render and search-public. Tests may import shared testing helpers without adding production dependency cycles.

Visual values belong in packages/design/src/tokens.css. The stylelint plugin rejects raw colours, dimensions, spacing, fonts, and custom property definitions elsewhere. The initial tokens and construction notice are scaffolding; Task 8 completes the Design System.

Domain foundations

The domain package exposes types, text, tree, dates, languages, and richtext subpaths. It has no application or database dependencies.

• Internal IDs are canonical ULIDs; recorded timestamps are canonical ISO 8601 UTC strings with milliseconds. Stored text is never normalized. Character limits use Unicode code points, while span and edit offsets use UTF-16 code units on code point boundaries.
• Partial dates store { value, approx }, with value at year, month, or day precision in years 0001 to 9999. Human displays use c. for approximation. JSON preserves the flag; CSV helpers provide separate date and approximate fields so ISO text retains its original precision. Partial-date ordering uses the beginning of the entered period, and overlapping date ranges are allowed.
• Entered times retain their supplied offset and fractional precision. Recorded times use the Profile zone in the Workspace and Member emails, and marked UTC elsewhere. Inclusive local date filters return half-open UTC intervals (start <= instant < endExclusive), including short/long days and wholly skipped calendar dates. The clock is always supplied by the caller.
• Rich-text validation returns indexed issues for every faulty label. Insertions strictly inside a label join it; replacements include new text only when marked characters survive on both sides. Fully deleted labels disappear. Mentions are explicit picker-created atomic ranges: editing their contents removes the reference, and typed @name text creates no notification target.
• languages.ts pins the current ISO 639-1 codes and English names from the Library of Congress table; validation does not fetch external data.

Properties 37, 38, and 91 live in packages/domain/test/properties/, alongside example tests for Unicode, date precision, time zones, mentions, and immutable edits. The edit properties use a per-character reference model; the validity property uses independent character coverage. A workerd smoke test checks the same date functions in the deployment runtime.

Domain rules

• value-sets provides immutable add, rename, retire, and rights-publication operations. Results return either a new set or indexed field issues. Locked values have fixed IDs in LOCKED_VALUE_IDS; the database seeds must use these IDs so protection survives renaming. Retired values remain available for existing references, while new selections refuse them. Every configurable set except Specialties consists of Methods terms and requires Public Descriptions.
• identifiers generates opaque IDs through native crypto.getRandomValues. newItemIdentifier() returns { identifier, registryKey }; newSubmissionReference() returns a string. The registry still has to reserve each Item Identifier atomically and retry collisions in Task 17. File-kind detection is separate from MIME sniffing and preview safety.
• settings is the registry for defaults, units, validation and operational slice budgets. MB and GB settings use decimal bytes; explicitly named MiB transport constants use binary bytes. The direct-upload threshold defaults to zero because the design sends every file directly to R2. Byte limits without a specified maximum accept positive safe integers; Outside Submission limits accept non-negative safe integers, so zero can disable an allowance. Explicit requirement ranges remain fixed. defaultSettings(owner) requires an active campaign owner and replaces the Responsible Person bootstrap placeholder. Email credentials are secrets, not setting values.
• permissions seeds both built-in Roles from one map, evaluates current Member status and grants for every call, and enforces ownership and authorship limits. Custom Roles can draw only from existing grants. Project accents, Policy sections, the Responsible Person and backup delivery keep the explicit Administrator restrictions. Callers supply targets read from trusted current records, and use canAll before writes. The last-Administrator check also requires the atomic D1 guard introduced in Tasks 5 and 14.
• retention holds the periods, ending events and unlimited policies shared by sweepers and the Privacy table. The generated table preserves the distinct requested/scheduled export policies, response-link/hash lifetimes, decision-based file/email erasure, and backup deletion grace period. Retention is release configuration, not an Administrator-editable setting.
• search-text folds only derived search text using NFKD, combining-mark removal, lower case and sharp-S expansion. Paired double quotes form whole-word phrases; an unmatched quote is punctuation. Query validation counts code points and reports all date and query issues without changing entered text. Matching, ranking and FTS query construction come in later search tasks.

Properties 31, 44 and 97 cover operation sequences, the complete extension table, and generated retention rows. Example tests cover settings, cryptographic sources, scope restrictions, custom grants, review eligibility, status changes, and search syntax. Native crypto and Unicode folding are also exercised inside workerd.

Validation schemas

The schemas/people, schemas/items, and schemas/other subpaths expose Zod 4.6.6 validators for editable records and settings. Shared primitives and validate(schema, input) live in schemas/common; validation returns the existing RuleResult format with dotted field paths, including array indices. The validators reject unknown keys and report independent cross-field errors even when another field has a wrong JSON type. They never merge unvalidated input back into parsed output.

Factories take a trusted ValidationContext snapshot containing the current instant, viewer time zone, allowed reference records and value sets. Public endpoints must supply only reference records available to that visitor. Existing selections, previous Journal totals, fixed slugs, and edit/import/parser/restore modes come from trusted saved state or verified import/backup processing, never from an editable body. Contextual schemas should be rebuilt from current request data.

• Editable text stays in its original form. Long text uses { text, spans }; dates use the domain partial-date form. Validated output may add documented defaults, such as empty optional Profile fields, the digest hour, Public Profile Off, Open Finding/Hypothesis status, or Unknown rights. It never trims text or rounds Run counts, which remain strings of up to 30 digits.
• researchItemSchema(kind, context, options) retains the chosen kind in its output type. Submission mode requires a Log Summary except for Journal Entries and Topics. Edit mode protects existing summaries and uses the saved Journal section/character totals. Parser mode requires the actual source byte count within the journal limit; verified restore mode preserves existing sizes and supplied retired selections. An unchanged retired selection is accepted only for an existing record; a new Attachment cannot inherit that exception from its parent.
• BUILT_IN_VALUE_IDS now also defines the Finding status identities. Seeds must use them so renaming Withdrawn does not disable its reason requirement. Reserved slugs are exported from the Project schema for use beside the editor field.
• submissionReadiness validates complete uploads, metadata, file count and sizes and identifies unready files by index and original name. Upload ownership, checksums and storage facts still require the file service. Outside Submission validation requires explicit agreement to the configured current licence, with the Removal Request exemption.
• Validation performs no writes and grants no authority. The save pipeline must still run canAll, verify referenced and staged records, enforce evidence/superseding links, and repeat concurrency-sensitive rules in guarded D1 batches. These integration steps follow in the database and feature tasks.

Property 27 runs a matrix of record, field, setting and invalid-input cases against independent constraint expectations. Example tests cover large Journal boundaries, all independent error fields, exact counts and fractional times, retired selections, licence exemptions, and unknown ownership/disclosure fields. A workerd test exercises the same Zod validators in the deployment runtime.

Database

packages/db holds the Drizzle schemas for the three D1 databases (schema/core/*.ts, schema/search.ts, schema/public.ts) and their migrations in migrations/<database>/. Wrangler applies these folders in filename order.

• 0000_schema.sql files come from db:generate. SQL that Drizzle cannot express sits in hand-written migrations registered in the Drizzle journal: core/0001_raw.sql (the expression index for active Cross_Links and the triggers that keep Revisions, Moderation_Decisions, and Item_Identifiers permanent) and search/0001_fts.sql (the FTS5 table and its sync triggers).
• core/0002_seed.sql is generated by src/seed/seed.ts. The seed test compares the two; run npx vitest --run -u packages/db after changing a seed module. Seeds hold only what every installation shares: built-in Roles, value sets with fixed IDs for locked and built-in values, Methods_Term descriptions, default settings, Policy_Sections with neutral starting text, and the counters. The installation name, Responsible Person, first Administrator, first Project, and Content_Licence are bootstrap inputs.
• There are no foreign-key constraints; the save pipeline verifies references. CHECK constraints restate domain invariants that a single row can show.
• GuardedBatch (src/batch.ts) sends a save as one D1 batch. It starts with the backup-freeze guard and the change-counter increment, stamps updated_seq, actor_id, and rev, adds Revision and outbox rows, and expands bulk inserts with json_each. A guard inserts NULL into its _g_<code> table when its condition fails, which aborts the batch; the helper reports this as a GuardViolation with the rule's code and HTTP status. Batches above 40 statements are refused before sending. RequestAccountant sums statements, rows read, and rows written for a request.

Workerd tests in apps/app/test/db-schema.test.ts apply all three migration sets to local D1 and cover the seeds, the credential-column scan, immutability triggers, CHECK rules, stamping, guards, the freeze, bulk inserts, and FTS synchronisation. They use synthetic data only.

Cloudflare configuration

The Wrangler configurations have Workers development addresses and preview URLs disabled. Production and staging have separate resource names and cross-Worker bindings. rf-app owns the three D1 databases, two R2 buckets, Workers AI, four SQLite-backed Durable Object classes, the ops service binding, and the two cron schedules. rf-submit references the app's Budget and Mailer namespaces; rf-dl has only core data and private files. rf-ops has no fetch handler or public route.

Database UUIDs are deliberate placeholders for local validation. No resources, custom domains, secrets, Access policies, or public site deployment have been created. Task 48 provisions real resources and replaces these IDs. Secrets belong in Wrangler secrets or ignored .dev.vars files.

Files and protected downloads

Task 30 adds direct browser uploads, Attachment submissions and versions, and the rf-dl download Worker. New Items and Replies can include a files array; POST /api/v1/attachments also creates a file-only Submission on an existing Parent or adds a version with supersedesId. Metadata edits use PATCH /api/v1/attachments/{id} with the opened revision and an edit mark. Files and their flags, identifiers, notifications, activity, and outbox work commit together. Superseded files remain usable. All new files start Not Published; versions start with Rights Status Unknown.

Configure these installation values before using real R2 uploads:

• On rf-app: R2_ACCOUNT_ID, PRIVATE_BUCKET_NAME, and DOWNLOAD_ORIGIN; store the bucket-scoped R2_ACCESS_KEY_ID and R2_SECRET_ACCESS_KEY as secrets.
• Set the same independent DOWNLOAD_HMAC_KEY secret on rf-app and rf-dl. Set WORKSPACE_ORIGIN on both Workers. Production and staging must use separate credentials, buckets, secrets, and origins.
• Keep the private bucket private. Its CORS policy must allow PUT from the Workspace origin, allow content-length, x-amz-checksum-sha256, and if-none-match, and expose ETag. Add the public submission origin when task 34 needs it. Abort incomplete multipart uploads after one day with an R2 lifecycle rule.

POST /uploads reserves a file in a client-generated upload group and signs its length and SHA-256. Files up to 64 MiB use a write-once PUT; larger files use 16 MiB parts and a persisted SHA-256 per part. Completion checks size and R2's checksum, using S3 HEAD for the multipart composite checksum. File content never enters a Worker request body. The hourly sweep claims abandoned uploads before deleting them, while completed Submissions keep their files. Migration core/0009_files.sql preserves pre-existing upload rows and derives their states.

Download links expire after 15 minutes. Each GET or HEAD checks current membership and access to the Research Item; backup volumes additionally require an Administrator. The Worker supports byte ranges, safe download headers, and Workspace-only CORS; all refusals match its unknown-path 404. Previews use Member-bound links and browser memory. PDF.js renders canvases without scripting or annotation actions; PDF.js 6 has removed the earlier eval option and its remaining eval path. Raster previews require magic bytes; text is limited to the first decimal MB and CSV to 1,000 data rows through Papa Parse. Hashing runs incrementally in a Web Worker.

Tests use local D1/R2 for checksum rejection, completion mismatch, atomic saves, 100-file limits, migration preservation, sweeps, and concurrent versions, plus generated Version Chain, Download Link, and preview-boundary cases. S3 signing/control calls are simulated locally; staging still needs a real R2 multipart and CORS rehearsal when task 48 configures hosting.

Toolchain compatibility notes

• Cloudflare renamed @cloudflare/vitest-pool-workers to @cloudflare/vitest-plugin. The requested dependency name is retained as an npm alias to the maintained plugin, with the current cloudflareTest() API and a compatible Vitest 4.1 version. This avoids the retired package's outdated dependencies. See Cloudflare's configuration documentation.
• The legacy esbuild loader pulled in by drizzle-kit uses an esbuild override to 0.25.12 to remove its vulnerable older copy. Keep the override until drizzle-kit removes that legacy dependency, and validate Drizzle CLI operation when changing it.

All fixtures are synthetic. The HELUT journal and attachments stay outside this repository; later migration tasks read them only from explicitly supplied paths.

Resource profiles and capacity evidence

The versioned shared resource contract and installation templates are documented in infra/profiles. npm run profile:certify -- infra/profiles/Free_Qualified.example.json checks local encoded boundaries and produces synthetic growth forecasts. It leaves production certification pending and never activates a paid plan. The authenticated /api/v1/resource-profile endpoint and ApiClient.resourceProfile() expose the selected limits to browser consumers. Physical fragment storage, full staging integration and measured production capacity remain separately scheduled acceptance work.

Compatible releases and upgrade rehearsals

Release tooling defines Worker/schema/backup compatibility, immutable candidates, deployment preflight, bounded backfills and guarded contraction/recovery. Run npm run release:write after source or migration changes, then npm run release:check (also required by CI). npm run release:rehearse verifies disposable SQLite upgrades and reopened recovery copies. The catalog currently has a development baseline; real deployment, independent backup readers and production release certification remain gated work.

Immutable logical fields

Logical-field storage defines bounded immutable text/label pieces, scoped readers, retained history references, logical phrase search and public continuations. Existing inline data remains compatible during expansion. npm run fields:migrate -- --local <rehearsal/core-N.sqlite> exercises the exact resumable migration on a disposable recovery copy. Task 19.1 owns the full staged transport cutover; production deployment and recovery certification remain gated.