Skip to content

AGENTS.MD

Latest commit

 

History

History
266 lines (207 loc) · 17.9 KB

AGENTS.MD

File metadata and controls

266 lines (207 loc) · 17.9 KB
Raw

Agent Guide - Etherpad

Welcome to the Etherpad project. This guide provides essential context and instructions for AI agents and developers to effectively contribute to the codebase.

Project Overview

Etherpad is a real-time collaborative editor designed to be lightweight, scalable, and highly extensible via plugins.

Technical Stack

  • Runtime: Node.js >= 22.12.0
  • Package Manager: pnpm (>= 11.0.0)
  • Languages: TypeScript (primary for new code), JavaScript (legacy), CSS, HTML
  • Backend: Express.js 5, Socket.io 4
  • Frontend: Legacy core (src/static), Modern React UI (ui/), Admin UI (admin/)
  • Database: ueberdb2 abstraction (supports dirtyDB, MySQL, PostgreSQL, Redis)
  • Build Tools: Vite (for ui and admin), esbuild, tsx
  • Testing: Mocha (backend), Playwright (frontend E2E), Vitest (unit)
  • Auth: JWT (jose library), OIDC provider

Directory Structure

  • src/node/ - Backend logic, API handlers, database models, hooks
  • src/static/ - Core frontend logic (legacy jQuery-based editor)
  • src/static/js/pluginfw/ - Plugin framework (installer, hook system)
  • src/tests/ - Test suites (backend, frontend, container)
  • ui/ - Modern React OIDC login UI (Vite + TypeScript)
  • admin/ - Modern React admin panel (Vite + TypeScript + Radix UI)
  • bin/ - CLI utilities, build scripts, plugin management tools
  • bin/plugins/ - Plugin maintenance scripts (checkPlugin.ts, updateCorePlugins.sh)
  • doc/ - Documentation (VitePress + Markdown/AsciiDoc)
  • local_plugins/ - Directory for developing and testing plugins locally
  • var/ - Runtime data (logs, dirtyDB, etc. - ignored by git)

Quick Start

pnpm install                                    # Install all dependencies
pnpm run build:etherpad                         # Build admin UI and static assets
pnpm --filter ep_etherpad-lite run dev          # Start dev server (port 9001)
pnpm --filter ep_etherpad-lite run prod         # Start production server

Core Mandates & Conventions

Coding Style

  • Indentation: 2 spaces for all files (JS/TS/CSS/HTML). No tabs.
  • TypeScript: All new code should be TypeScript. Strict mode is enabled.
  • Comments: Provide clear comments for complex logic only.
  • Backward Compatibility: Always ensure compatibility with older versions of the database and configuration files.

Internationalisation (i18n) — Mandatory

  • Every user-facing string MUST go through i18n. That means JSX text, placeholder, title, aria-label, alt, toast/alert titles, <option> labels, error messages, breadcrumbs, empty-state copy — anything a user can see or hear via a screen reader.
  • React (admin SPA): use <Trans i18nKey="…"/> for JSX text and t('…') for attribute values. The t comes from useTranslation().
  • Pad UI (legacy): use data-l10n-id="…" (html10n) — never bind window._ directly to html10n.get and call it (it's unbound; returns undefined).
  • String keys live in src/locales/en.json. Other locales sync from translatewiki on its own cadence — never hand-edit non-EN locale files.
  • Reuse existing keys before inventing new ones. Check src/locales/en.json and the relevant plugin's static/locale/en.json (e.g. admin/public/ep_admin_pads/en.json) for an existing match. Duplicating a key like ep_admin_pads:ep_adminpads2_last-edited as a fresh admin_pads.col.last_edited fragments the translation surface for translatewiki.
  • Naming: dot-namespaced, kebab-or-underscore-cased: admin_plugins.subtitle, admin_pads.filter.all. Group by page/feature.
  • Pluralisation: use i18next's _one/_other suffix forms with t('key', {count: n}) — never n > 1 ? 'X items' : '1 item'.
  • Locale-aware formatters: pass a sanitised locale to Intl.* / toLocaleString. i18n.language is influenced by ?lng= (user-controlled) and a malformed tag throws RangeError. Use a sanitizeLocale() helper that normalises _→- and validates via Intl.DateTimeFormat.supportedLocalesOf(), falling back to 'en'.
  • defaultValue: in t() is for safety, not a substitute for adding the key to en.json. If you're tempted to inline English with defaultValue: and skip the key, you're shipping a future-broken translation.
  • Hard prohibition: literal German / French / Spanish / any non-English in JSX. The denylist test at src/tests/backend-new/specs/admin-i18n-source-lint.test.ts enforces this for the admin SPA — extend it when adding new admin files or new known-bad words.

Accessibility (a11y) — Mandatory

  • Icon-only buttons MUST have aria-label AND title (both — screen readers prefer aria-label; hover users get title). Lucide icons inside a button are not text content. Both labels must be t('…')-localised.
  • Sort controls are focusable. A <select> plus a paired direction toggle is fine; a clickable column header is fine; "click invisible part of the row to sort" is not. When restyling, never strip a direction toggle without adding back an equivalent.
  • Semantic HTML over <div> soup. Use <nav>, <main>, <button>, <a> (for navigation), <table> for tabular data. If a thing navigates externally, it is <a target="_blank" rel="noopener noreferrer">, not a click-handler on a <span>.
  • Don't drop existing affordances when restyling. A UI refresh that removes <a href="https://npmjs.com/{plugin}"> links, removes a sort-direction control, or replaces semantic elements with non-focusable divs is a regression even if it looks nicer. Audit the before/after for: focus order, keyboard reachability, external links, aria-labels, alt text.
  • Tests assert rendered strings + structural affordances. For UI changes, the Playwright spec must assert at least one rendered translated string (catches broken i18n loading) and one structural affordance you added/preserved (links, toggles, headings).

UI changes need a human click-through — Mandatory

Automated tests are not enough for UI. If an AI tool wrote or changed anything a user can see or interact with (a button, popup, form, setting, layout, wording, styling), a human must run it and use it before the PR is opened.

  • Agents: stop and ask before opening the PR. Start a local instance with the change, tell the human the URL and exactly what to click, and wait for them to say they tried it and are happy with how it looks and behaves. Do not open the PR until they do.
  • Never tick the click-through checkbox in the PR template yourself. Only the human who did the click-through ticks it. If they haven't, leave it unticked and say so in the PR description.
  • Fix what they find first. Layout, placement, wording and "this should be one click" feedback is the point of the click-through. Address it before the PR, not in review.
  • A Playwright spec is still required (see Accessibility above). It covers regressions, not whether the UI is any good.

Development Workflow

  • Branching: Work in feature branches. Issue PRs against the develop branch. Never PR directly to master.
  • Commits: Maintain a linear history (no merge commits). Use meaningful messages in the format: submodule: description.
  • Feature Flags: New features should be placed behind feature flags and disabled by default.
  • Deprecation: Never remove features abruptly; deprecate them first with a WARN log.
  • Forks: For etherpad-lite changes, commit to johnmclear/etherpad-lite fork on a new branch, then PR to ether/etherpad. For plugins (ep_* repos), committing directly is acceptable.

Testing & Validation

  • Requirement: Every bug fix MUST include a regression test in the same commit.
  • Always run tests locally before pushing to CI.
  • Linting: pnpm run lint
  • Type Check: pnpm --filter ep_etherpad-lite run ts-check
  • Build: pnpm run build:etherpad before production deployment

Running Backend Tests Locally

Backend tests use Mocha with tsx and run against a real server instance (started automatically by the test harness). No separate server process is needed.

# Run ALL backend tests (includes plugin tests)
pnpm --filter ep_etherpad-lite run test

# Run only utility tests (faster, ~5s timeout)
pnpm --filter ep_etherpad-lite run test-utils

# Run a single test file directly
cd src && cross-env NODE_ENV=production npx mocha --import=tsx --timeout 120000 tests/backend/specs/YOUR_TEST.ts

# Run unit tests (Vitest)
cd src && npx vitest
  • Tests run with NODE_ENV=production.
  • Default timeout is 120 seconds per test.
  • Test files live in src/tests/backend/specs/.
  • Plugin backend tests live in node_modules/ep_*/static/tests/backend/specs/ (at repo root) and are included automatically by the test script.

Running Frontend E2E Tests Locally

Frontend tests use Playwright. You must have a running Etherpad server before launching them — the Playwright config does not auto-start the server.

Before running frontend or admin tests, ensure Playwright browsers are installed. Check and install if needed:

# Check which browsers are installed
cd src && npx playwright install --dry-run

# Install all browsers and their system dependencies (must run from src/)
cd src && npx playwright install
cd src && sudo npx playwright install-deps

If sudo is unavailable, install system dependencies for webkit manually:

# Check which system libraries are missing for webkit
ldd ~/.cache/ms-playwright/webkit-*/minibrowser-wpe/MiniBrowser 2>&1 | grep "not found"

If browsers or system dependencies are missing, tests will fail silently or timeout — always verify browser installation before debugging test failures.

# 1. Start the dev server in a separate terminal
pnpm --filter ep_etherpad-lite run dev

# 2. Run frontend E2E tests
pnpm --filter ep_etherpad-lite run test-ui

# 3. Run with interactive Playwright UI (useful for debugging)
pnpm --filter ep_etherpad-lite run test-ui:ui

# Run a single test file
cd src && cross-env NODE_ENV=production npx playwright test tests/frontend-new/specs/YOUR_TEST.spec.ts
  • Tests expect the server at localhost:9001.
  • Test files live in src/tests/frontend-new/specs/.
  • Runs against chromium and firefox by default (webkit is disabled).
  • Playwright config is at src/playwright.config.ts.

Running Admin Panel Tests Locally

# Requires a running server and Playwright browsers installed (same as frontend tests)
pnpm --filter ep_etherpad-lite run test-admin

# Interactive UI mode
pnpm --filter ep_etherpad-lite run test-admin:ui
  • Admin tests run with --workers 1 (sequential) on chromium and firefox only.
  • Test files live in src/tests/frontend-new/admin-spec/.

Backend Test Auth

Tests use JWT authentication, not API keys. Pattern:

import * as common from 'ep_etherpad-lite/tests/backend/common';

const agent = await common.init();  // Starts server, returns supertest agent
const token = await common.generateJWTToken();
agent.get('/api/1/endpoint').set('authorization', token);

Do not use APIKEY.txt — it may not exist in the test environment.

Key Concepts

Easysync

The real-time synchronization engine. It is complex; refer to doc/public/easysync/ before modifying core synchronization logic.

Plugin Framework

Most functionality should be implemented as plugins (ep_*). Avoid modifying the core unless absolutely necessary.

Plugin structure:

ep_myplugin/
├── ep.json              # Hook declarations (server_hooks, client_hooks)
├── index.js             # Server-side hook implementations
├── package.json
├── static/
│   ├── js/              # Client-side code
│   ├── css/
│   └── tests/
│       ├── backend/specs/    # Backend tests (Mocha)
│       └── frontend-new/     # Frontend tests (Playwright)
├── templates/           # EJS templates
└── locales/             # i18n files

Plugin management:

pnpm run plugins i ep_plugin_name       # Install from npm
pnpm run plugins i --path ../plugin     # Install from local path
pnpm run plugins rm ep_plugin_name      # Remove
pnpm run plugins ls                     # List installed

Plugin installation internals: Plugins are installed to src/plugin_packages/ via live-plugin-manager, which stores them at src/plugin_packages/.versions/ep_name@version/. Symlinks are created: src/node_modules/ep_name → src/plugin_packages/ep_name → .versions/ep_name@ver/.

Plugin Repositories

  • Monorepo: ether/ether-plugins contains 80+ plugins with shared CI/publishing
  • Standalone repos: Individual ether/ep_* repos still exist for many plugins
  • Plugin CI templates: bin/plugins/lib/ contains workflow templates pushed to standalone plugin repos via checkPlugin.ts
  • Shared pipelines: ether/ether-pipelines contains reusable GitHub Actions workflows for plugin CI

Settings

Configured via settings.json. A template is available at settings.json.template. Environment variables can override any setting using "${ENV_VAR}" or "${ENV_VAR:default_value}".

Releasing

Releases are driven almost entirely by GitHub Actions. A maintainer dispatches one workflow; the version bump, tagging, GitHub Release, Docker images, and snap all cascade off the pushed tag. The npm publish is a separate manual dispatch.

Prerequisites (check these before dispatching)

  • A # X.Y.Z changelog section for the target version must already be at the top of CHANGELOG.md. bin/release.ts aborts with No changelog record for X.Y.Z, please create changelog record if it's missing. Write the section before dispatching.
  • All four package.json files must agree on the current version: root package.json, src/package.json, admin/package.json, bin/package.json. release.ts reads the current version from src/package.json and computes the next with semver.inc(current, type). If the files are out of sync (e.g. one was hand-edited), the computed target is wrong and the changelog guard fails. (This exact desync — src/bin left at 3.3.0 while root/admin were 3.2.0 — blocked the 3.3.0 release in June 2026.)

Cutting a release

  1. Actions → "Release etherpad" → Run workflow (workflow_dispatch), choose patch / minor / major. Cadence is monthly minors (3.0.0 → 3.1.0 → 3.2.0 → …); use major only for breaking changes.
  2. The Prepare release step runs bin/release.ts, which:
    • sanity-checks the tree (clean working dir, on develop, develop/master upstreams in sync, ../ether.github.com cloned & clean on master);
    • bumps the version in all four package.json files;
    • commits bump version, merges develop → master, creates both X.Y.Z and vX.Y.Z tags, merges master back to develop;
    • builds and stages the versioned docs into the website repo (see Documentation below).
  3. The Push after release step (bin/push-after-release.sh) pushes master, develop, the tag, --tags, and the ether.github.com docs commit.
  4. The pushed vX.Y.Z tag auto-triggers three workflows:
    • handleRelease.yml → builds Etherpad, extracts the matching changelog section via generateChangelog (bin/generateReleaseNotes.ts), and publishes the GitHub Release (make_latest: true);
    • docker.yml → builds & pushes the Docker images;
    • snap-publish.yml → publishes the snap.
  5. npm publish — PARKED, not part of the release. releaseEtherpad.yaml publishes the core as ep_etherpad to npm, but that package is not load-bearing: it has 0 dependents, nothing depends on it (plugins import ep_etherpad-lite from the local core install; plugin CI clones the repo), and Etherpad is run via clone/Docker/zip/snap — never npm install. The publish currently fails with E404 because ep_etherpad has no OIDC trusted publisher configured on npmjs.com, which is why npm sits at 2.5.0 while 3.x shipped fine without it. The workflow is gated behind a confirm: true input so it can't run by accident. Skip it for a normal release. To revive it, the npm owner of ep_etherpad (samtv12345) configures a trusted publisher (npmjs.com → ep_etherpad → Settings → Trusted Publisher → repo ether/etherpad, workflow releaseEtherpad.yml); otherwise the workflow can be removed. Decision pending.

Documentation

Two distinct things, both important:

1. Per-PR doc updates — your responsibility in every behaviour-change PR. The doc/ workspace holds the user/admin/API docs (Markdown + AsciiDoc). Whenever a PR changes API responses, CLI flags, settings keys, hooks, or error formats, update the relevant files in doc/ in the same PR — don't defer it to release time. The HTTP API reference is doc/api/http_api.{md,adoc} (keep both in sync). Preview locally with pnpm run makeDocs.

2. Release-time versioned docs publishing — automated, no manual step. During a release, release.ts runs pnpm run makeDocs (→ bin/make_docs.ts) to render doc/ into out/doc/, then copies it into the sibling website repo at ../ether.github.com/public/doc/vX.Y.Z, bumps that repo's version, and commits X.Y.Z docs; push-after-release.sh pushes it, publishing the versioned docs on etherpad.org. This requires ether.github.com checked out as a sibling directory — the Release etherpad workflow checks it out automatically. If you ever run release.ts by hand, clone it first: cd .. && git clone git@github.com:ether/ether.github.com.git.

Monorepo Structure

This project uses pnpm workspaces. The workspaces are:

  • src/ - Core Etherpad (package: ep_etherpad-lite)
  • bin/ - CLI tools and plugin scripts
  • ui/ - Login UI
  • admin/ - Admin panel
  • doc/ - Documentation

Root-level commands operate across all workspaces. Use pnpm --filter <package> to target specific workspaces.

AI-Specific Guidance

AI/Agent contributions are explicitly welcomed by the maintainers, provided they strictly adhere to the guidelines in CONTRIBUTING.md and this guide. Always prioritize stability, readability, and compatibility.