From a983a1cf646c4eb66b3105659fefd8017e0a406b Mon Sep 17 00:00:00 2001 From: NGHTBOY Date: Thu, 27 Aug 2026 16:27:07 +0200 Subject: [PATCH] feat(orchestrator): the memory write path and the landscape MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit v0.15.0 took the survey's taxonomy. This takes the rest, as two references under the spine: how an entry is made, changed and discarded, and what exists to build on and measure with. The formation table states each operation's cost rather than implying it — summarization is lossy by design, structured construction pays schema rigidity, latent is a black box, parametric cannot be precisely removed. Updating is separated from consolidation, and temporal annotation replaces deletion as the recommended shape because it keeps the decision reversible. The landscape file refuses to be a list that expires: frameworks are compared by the axes that separate them, and the build-or-adopt framing is that you own the judgement in both columns. Frontiers are marked as frontiers, with one acted on — memory operations as tool calls, because that is what makes the decision NOT to retrieve visible in a trace. Three mechanisms are named as deliberately absent rather than quietly added. The validator refused the first version: three files each repeated the pinned citation. The spine holds it now and the siblings name it. Co-Authored-By: Claude Opus 5 (1M context) --- .claude-plugin/marketplace.json | 2 +- CHANGELOG.md | 40 +++++ package.json | 2 +- .../agent-stack/.claude-plugin/plugin.json | 2 +- .../skills/agent-orchestrator/SKILL.md | 5 +- .../references/memory-architecture.md | 18 ++ .../references/memory-landscape.md | 128 +++++++++++++++ .../references/memory-lifecycle.md | 154 ++++++++++++++++++ 8 files changed, 347 insertions(+), 4 deletions(-) create mode 100644 plugins/agent-stack/skills/agent-orchestrator/references/memory-landscape.md create mode 100644 plugins/agent-stack/skills/agent-orchestrator/references/memory-lifecycle.md diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 8dd6efc..6d82cad 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -11,7 +11,7 @@ "displayName": "Agent Stack", "source": "./plugins/agent-stack", "description": "Two skills: agent-orchestrator — tool-calling loops, multi-stage pipelines with checkpoints, provider routing with fallback, four-layer memory, context engineering, plus the wallet side of reselling LLM access; and agent-evals — run/trace/thread evals, judges, and fixtures grown from production.", - "version": "0.15.0", + "version": "0.16.0", "author": { "name": "ssheleg", "url": "https://x.com/sshlg93" diff --git a/CHANGELOG.md b/CHANGELOG.md index 42214e9..5108c80 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,45 @@ # Changelog +## v0.16.0 — the whole survey, not just its taxonomy + +v0.15.0 took the taxonomy and the named failure modes. This takes the rest: the write +path and the landscape, as two references under the spine that already exists. + +- **`memory-lifecycle.md` — how an entry is made, changed and thrown away.** Five formation + operations with the cost of each stated rather than implied: semantic summarization is + *lossy by design* and wrong for evidence-critical tasks; structured construction buys + multi-hop and pays schema rigidity; latent is a black box; parametric cannot be precisely + removed. Summarization's two shapes fail differently — incremental drifts because each + summary is built from the last, partitioned loses cross-partition dependencies — and + **summarizing by fixed window is partitioned summarization with the worst partition rule**. +- **Updating is not consolidation.** One resolves conflict, the other abstracts, and a + system needs both. The field's own trajectory is worth copying rather than rediscovering: + early systems deleted the superseded entry and broke temporal continuity; the better + pattern is **temporal annotation** — mark a validity window instead — which also makes the + stability–plasticity decision reversible, and it has no general answer. +- **Consolidation has a cost this pack did not state**: it risks information smoothing, and + the outlier it smooths away is often the entry worth keeping. +- **`memory-landscape.md` — build or adopt, measure, and what is not practice yet.** Around + twenty-five open-source frameworks give you an index and leave you the judgement: what + becomes a memory, when to retrieve, what to abstain on, what to demote. **Adopt for the + index, not for the judgement.** Frameworks are compared by the axes that separate them + rather than by a list that expires. +- Benchmarks are split into memory-oriented and long-horizon-that-stresses-memory, and the + first question is neither: it is `agent-evals`' question — **what fails if memory is + silently disabled?** A memory never queried and a memory that is empty score identically + on every benchmark; only the retrieval log separates them. +- Frontiers are marked as frontiers, with one exception acted on: **expose memory operations + as tools the agent calls.** It makes every memory decision legible in the trace, including + the decision NOT to retrieve — which is the silent failure this whole subject is about, + and which a background memory module cannot show you. +- Three things the pack deliberately does NOT implement are named as absent rather than + quietly added: frequency-based forgetting, temporal annotation, dual-phase updating. Each + is a real change to a mechanism in production, and a reference's job is to say what the + options are, not to rewrite `patterns.md` from a survey. +- The citation now has **one home**. The member's own validator refused three files each + repeating the pinned source — "a fact with two homes disagrees with itself on the first + edit" — so the spine holds it and the siblings name it. + ## v0.15.0 — memory architecture, and the axes the layer table does not have - **`agent-orchestrator/references/memory-architecture.md`** — form, function and dynamics diff --git a/package.json b/package.json index ac6b380..f5c4c56 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@ssheleg/agent-stack", - "version": "0.15.0", + "version": "0.16.0", "scripts": { "test": "python3 test/validate.py && python3 test/plant_guard_test.py" }, diff --git a/plugins/agent-stack/.claude-plugin/plugin.json b/plugins/agent-stack/.claude-plugin/plugin.json index 93bb4a3..4f32453 100644 --- a/plugins/agent-stack/.claude-plugin/plugin.json +++ b/plugins/agent-stack/.claude-plugin/plugin.json @@ -2,7 +2,7 @@ "name": "agent-stack", "displayName": "Agent Stack", "description": "Two skills: agent-orchestrator — tool-calling loops, multi-stage pipelines with checkpoints, provider routing with fallback, four-layer memory, context engineering, plus the wallet side of reselling LLM access; and agent-evals — run/trace/thread evals, judges, and fixtures grown from production.", - "version": "0.15.0", + "version": "0.16.0", "author": { "name": "ssheleg", "url": "https://x.com/sshlg93" diff --git a/plugins/agent-stack/skills/agent-orchestrator/SKILL.md b/plugins/agent-stack/skills/agent-orchestrator/SKILL.md index 120f85b..400f901 100644 --- a/plugins/agent-stack/skills/agent-orchestrator/SKILL.md +++ b/plugins/agent-stack/skills/agent-orchestrator/SKILL.md @@ -244,7 +244,10 @@ answer built from nothing, in no error log. **Design a memory layer from [`references/memory-architecture.md`](references/memory-architecture.md)**, not from this -table. It also carries the context-budget trap, layer 0 carryover and workspace scale. +table — it also carries the context-budget trap, layer 0 carryover and workspace scale. +The write path is [`references/memory-lifecycle.md`](references/memory-lifecycle.md); +what to build on and measure with is +[`references/memory-landscape.md`](references/memory-landscape.md). ## 8. Self-Learning Feedback Loops Three cycles feed the memory layers, and they differ by what supplies the signal: a failed diff --git a/plugins/agent-stack/skills/agent-orchestrator/references/memory-architecture.md b/plugins/agent-stack/skills/agent-orchestrator/references/memory-architecture.md index d2f5c53..1e7abb5 100644 --- a/plugins/agent-stack/skills/agent-orchestrator/references/memory-architecture.md +++ b/plugins/agent-stack/skills/agent-orchestrator/references/memory-architecture.md @@ -30,6 +30,7 @@ the diversity and dynamics of contemporary agent memory systems."* - [8. Trustworthy memory](#8-trustworthy-memory) - [9. What this pack does NOT claim](#9-what-this-pack-does-not-claim) - [10. The checklist](#10-the-checklist) +- [11. The rest of it](#11-the-rest-of-it) ## The source, and what this file adds @@ -306,3 +307,20 @@ made by accident: 9. **Can a person see, correct and delete what is held about them?** (§8) 10. **What eval fails if memory is silently disabled?** If none, the memory layer is unmeasured and its value is a belief. (`agent-evals`) + +## 11. The rest of it + +This file is the decision spine and it deliberately stops at decisions. Two siblings carry +the depth, both pinned to the same survey and the same read date: + +- [`memory-lifecycle.md`](memory-lifecycle.md) — the **write path**. Five ways to turn + experience into an entry and how each one fails, consolidation at three granularities, + why temporal annotation beats deletion when a fact is superseded, dual-phase updating, + and the three forgetting policies. It ends with a table of what this pack already + implements and three things it deliberately does not. +- [`memory-landscape.md`](memory-landscape.md) — **build or adopt**, the axes that actually + separate the twenty-odd open-source memory frameworks, which benchmarks measure what, and + which frontier ideas are directions rather than practice. Its last section is the one + frontier worth acting on now: exposing memory operations as tools the agent calls, which + makes every memory decision visible in the trace — including the decision not to + retrieve. diff --git a/plugins/agent-stack/skills/agent-orchestrator/references/memory-landscape.md b/plugins/agent-stack/skills/agent-orchestrator/references/memory-landscape.md new file mode 100644 index 0000000..2c70fac --- /dev/null +++ b/plugins/agent-stack/skills/agent-orchestrator/references/memory-landscape.md @@ -0,0 +1,128 @@ +# Memory landscape — what exists to build on, what to measure with, where it is going + +**Load this when** deciding whether to build a memory layer or adopt one, choosing what to +evaluate it against, or judging whether a technique somebody is proposing is settled +practice or a research direction. + +**Source:** the survey pinned in [`memory-architecture.md`](memory-architecture.md) +— §6 resources, §7 positions and frontiers. That file is its one home; repeating the citation here would give it two. + +**Read the dates on everything here.** This is the fastest-moving part of the subject, and +a landscape file is stale the day after it is written. What does not go stale is the +*shape*: which axes the frameworks differ on, and which of the frontier ideas is a +direction rather than a practice. + +## Contents + +- [1. Build or adopt](#1-build-or-adopt) +- [2. Frameworks, by what actually separates them](#2-frameworks-by-what-actually-separates-them) +- [3. Evaluating a memory layer](#3-evaluating-a-memory-layer) +- [4. Frontiers — and which of them is not practice yet](#4-frontiers--and-which-of-them-is-not-practice-yet) +- [5. The one that changes how you build today](#5-the-one-that-changes-how-you-build-today) + +## 1. Build or adopt + +An ecosystem of open-source memory frameworks exists — the survey tabulates around +twenty-five, including MemGPT, Mem0, Memobase, MemoryOS, MemOS, Zep, LangMem, Cognee, +Memary, MIRIX, and vector stores used as memory (Pinecone, Chroma, Weaviate). + +**What most of them give you** is storage plus retrieval: a vector or graph database, an +API, a short/long-term split. **What most of them leave to you** is the part that decides +whether the memory works — *"they often leave agent behavior and evaluation protocols to +the application."* + +So the honest framing of the choice: + +| | Adopting a framework | Building on your own store | +|---|---|---| +| You get | schema, indexing, retrieval, a short/long split | nothing you did not write | +| You still own | formation policy, what enters at all, forgetting policy, retrieval timing, **evals** | the same list | +| Argues for it | not writing an index; graph or temporal support you would not build | your entries are few and structured, and a table plus a query is genuinely enough | + +**The decision is usually smaller than it looks.** The parts that go wrong — deciding what +becomes a memory, when to retrieve, what to abstain on, what to demote — are yours in both +columns. Adopt for the index, not for the judgement. + +## 2. Frameworks, by what actually separates them + +Rather than a list that expires, the axes that distinguish them — check a candidate on +these: + +- **Does it model experiential memory, or only factual?** Several store facts well and have + no notion of a strategy learned from a trajectory. If the agent is supposed to improve, + this is the axis that matters. +- **What is the structure?** Hierarchical short/long-term, graph, temporal knowledge graph, + flat vectors, profile-based. This decides which queries are cheap. (§3 of + `memory-architecture.md`.) +- **Is temporal validity first-class?** A store that can express *"this was true until + March"* supports soft updating; one that cannot will make you delete history to stay + correct. +- **Does it report results on memory benchmarks at all?** Most do not. Absence is not + failure, but a framework with published LoCoMo or LongMemEval numbers has at least been + measured by someone. +- **Multimodal?** Most are text-only. The survey is explicit that **no system provides + truly omnimodal support** yet. + +## 3. Evaluating a memory layer + +The survey groups benchmarks two ways, and the split is the useful part. + +**Memory-oriented** — built to test memory directly. `LoCoMo` and `LongMemEval` are the two +most frequently reported and the closest thing to a common yardstick. `PersonaMem`, +`PrefEval`, `MemoryBank`, `PerLTQA`, `MPR` stress user modelling and preference tracking. +`StreamBench`, `LifelongAgentBench`, `MemoryAgentBench`, `Evo-Memory` test lifelong and +self-evolving behaviour — new information arriving while old information becomes obsolete +or conflicting. `HaluMem` targets memory-induced hallucination specifically. + +**Long-horizon agent benchmarks that stress memory implicitly** — `SWE-Bench Verified`, +`GAIA`, `WebArena`, `ToolBench`, `ALFWorld`. Memory is not the measured target, but +performance depends on it. + +**Which to use is not the first question.** The first question is the one `agent-evals` +asks: **what fails if memory is silently disabled?** A public benchmark measures a system +against a distribution that is not yours. A fixture built from your own traces, where the +answer is obtainable *only* from memory, catches the failure mode this whole subject is +about — the agent that skips retrieval and answers confidently from nothing +(`memory-architecture.md` §5.1). Build that fixture first; reach for a public benchmark +when you need to compare against somebody else's system. + +**Instrument the negatives.** A memory never queried and a memory that is empty score +identically on every benchmark. Only the retrieval log separates them. + +## 4. Frontiers — and which of them is not practice yet + +Marked plainly, because the risk with a survey's frontier section is building on a +direction as though it were a technique. + +| Frontier | State | What it would change | +|---|---|---| +| **Memory generation** (over retrieval) | direction | memory synthesized on demand for the current task rather than fetched and concatenated. Two shapes: *retrieve-then-generate*, which is buildable now and is essentially §5.4 post-retrieval taken seriously; and *direct generation* with no retrieval step, which is research | +| **Automated memory management** | early | the agent reasons about its own memory through **explicit tool calls** — add/update/delete/retrieve as actions in its loop rather than a module beside it. See §5 | +| **RL-driven memory** | research | the progression is RL-free → RL for selected operations (reranking, the write policy) → fully learned. Most production systems are and will remain RL-free | +| **Multimodal memory** | partial | vision is furthest along, audio underexplored, and **no omnimodal system exists** | +| **Shared memory for multi-agent** | early | from isolated stores with message passing, through naive global stores, toward **role- and trust-aware** access. `memory-architecture.md` §7 | +| **Trustworthy memory** | **requirement, not frontier** | privacy, explainability, hallucination robustness. `memory-architecture.md` §8 | +| **Offline consolidation** ("sleep") | direction | a dedicated interval to reorganize, prune and replay, away from latency constraints. The buildable half of it is the **dual-phase update** in `memory-lifecycle.md` §4 | + +## 5. The one that changes how you build today + +Of everything above, **automated memory management via explicit tool calls** is the item +worth acting on now, and it costs little. + +Instead of a memory module that runs beside the agent — summarizing on a timer, evicting on +a policy, retrieving on every turn — expose memory operations as **tools the agent calls**: +`memory.search`, `memory.write`, `memory.update`, `memory.forget`. Three consequences, and +they are what make it worth doing rather than an aesthetic preference: + +1. **The decisions become legible.** Every memory operation is a tool call in the trace, + which means `agent-evals` can judge them and the retrieval log from + `memory-architecture.md` §5.1 exists for free — including the negatives, because a turn + with no `memory.search` call is visibly a turn that chose not to retrieve. +2. **The agent can reason about them** — retrieve, find it insufficient, decompose and + retrieve again, which is the fast–slow pattern that the timing stage needs anyway. +3. **They inherit everything the tool layer already has** — descriptions the model can act + on (`agent-harness`), error hierarchies, budgets and loop guards (`SKILL.md` §2). + +**The cost, so it is a choice:** more model calls, and a model that can decline to use +memory at all. That second one is the silent failure mode again, which is why point 1 +matters — with tool calls you can *see* it happen, and with a background module you cannot. diff --git a/plugins/agent-stack/skills/agent-orchestrator/references/memory-lifecycle.md b/plugins/agent-stack/skills/agent-orchestrator/references/memory-lifecycle.md new file mode 100644 index 0000000..255b43b --- /dev/null +++ b/plugins/agent-stack/skills/agent-orchestrator/references/memory-lifecycle.md @@ -0,0 +1,154 @@ +# Memory lifecycle — how an entry is made, changed and thrown away + +**Load this when** writing the code that decides what becomes a memory, how a new entry +meets the ones already there, and what leaves. `memory-architecture.md` decides *what kind* +of memory is being built and owns retrieval; this file is the **write path**. + +**Source:** the survey pinned in [`memory-architecture.md`](memory-architecture.md) +— §5.1 formation, §5.2 evolution. That file is its one home; repeating the citation here would give it two. + +The three operators run at **different frequencies**, and that is the design. Formation can +run per turn, evolution per task boundary or offline, retrieval per call. Short- and +long-term behaviour is a consequence of those frequencies, not of separate boxes. + +## Contents + +- [1. Formation — five ways to turn experience into an entry](#1-formation--five-ways-to-turn-experience-into-an-entry) +- [2. Choosing among them](#2-choosing-among-them) +- [3. Evolution — consolidation](#3-evolution--consolidation) +- [4. Evolution — updating, and the stability–plasticity dilemma](#4-evolution--updating-and-the-stabilityplasticity-dilemma) +- [5. Evolution — forgetting](#5-evolution--forgetting) +- [6. What this pack already implements](#6-what-this-pack-already-implements) + +## 1. Formation — five ways to turn experience into an entry + +*"Instead of passively logging all interaction history, the memory system selectively +identifies information with long-term utility."* Five operations, and they compose — one +system commonly runs several. + +| Operation | What it produces | Strength | The cost, stated | +|---|---|---|---| +| **Semantic summarization** | a compact narrative of a long stream | drastically shorter context; ideal for long dialogue | **lossy by design** — specific details and subtle cues get smoothed out, so it is wrong for evidence-critical tasks | +| **Knowledge distillation** | discrete reusable facts or strategies | fine-grained, function-specific | produces flat units with no relation between them | +| **Structured construction** | a graph or tree — entities, relations, hierarchy | explainability and multi-hop queries | **schema rigidity**; extraction and maintenance cost is high, and nuance that does not fit the schema is lost | +| **Latent representation** | vectors or KV states | high density, cross-modal, no decoding loss | **a black box** — cannot be inspected, edited or verified by a person | +| **Parametric internalization** | changed weights | zero retrieval cost, "instinctive" access | catastrophic forgetting, high update cost, and **cannot be precisely removed** | + +### Summarization has two shapes and they fail differently + +- **Incremental** — fuse each new chunk into the running summary. Supports streaming and + avoids reprocessing the whole history. Fails by **semantic drift**: errors compound + across iterations, because each summary is built from the last one. +- **Partitioned** — summarize segments independently, then aggregate. Finer-grained and + parallelizable. Fails by **losing cross-partition dependencies**, and by cutting at + arbitrary boundaries unless segments are chosen semantically rather than by length. + +If you summarize by fixed window size, you have chosen partitioned summarization with the +worst possible partition rule. + +### Distillation splits by what it is distilling + +- **Factual** — dialogue turns into stated facts, user intent, environment state. Watch for + *goal drift*: separate confirmed constraints from unresolved intents, or the agent starts + treating a floated idea as a decision. +- **Experiential** — strategies from trajectories. **From contrast, not from success.** The + survey's own split confirms what §8 of `SKILL.md` already requires: success-only + distillation learns the agent's habits, and the systems that work compare successful and + failed runs, or reflect against ground truth. + +## 2. Choosing among them + +The question is not which is best. It is **what the memory will be asked for later**: + +| If the later question is… | Form it now with | +|---|---| +| "what happened, roughly" | semantic summarization | +| "what is true about X" | knowledge distillation, factual | +| "how do I do this kind of task" | knowledge distillation, experiential | +| "what connects to what" / multi-hop | structured construction | +| "match this to anything similar" at volume | latent | +| "behave this way, always, with no lookup" | parametric — and read §2 of `memory-architecture.md` on why this is rarely the answer | + +**A schema chosen before this question has been asked is a guess.** The commonest and most +expensive version of that guess is a knowledge graph built because graphs sound thorough, +then maintained for multi-hop queries nobody runs. + +## 3. Evolution — consolidation + +Merging new entries with existing ones so learning is cumulative rather than a growing pile +of near-duplicates. Three granularities: + +- **Local** — a new entry retrieves its top-K nearest and a model decides whether to merge. + Cheapest, and the one to build first. +- **Cluster-level** — align a new cluster with similar existing clusters and fuse. Captures + regularities across instances that local merging cannot see. +- **Global integration** — periodically distil system-level insight from the whole store. + +**The cost of consolidation, which is the reason not to run it aggressively:** +*"it risks information smoothing, where outlier events or unique exceptions are lost during +the abstraction process."* The exception is often the entry worth keeping — see §5. + +## 4. Evolution — updating, and the stability–plasticity dilemma + +Updating resolves *conflict*; consolidation performs *abstraction*. They are different +operations and a system needs both. + +**The trajectory the field took, and it is worth copying rather than rediscovering:** +early systems detected a conflict and **replaced or deleted** the old entry — destructive, +and it erased historical context and broke temporal continuity. The better pattern is +**temporal annotation**: mark the superseded fact with a validity window instead of +deleting it. Soft, time-aware updating keeps both semantic consistency and history. + +**Dual-phase updating** is the shape that survives real load: a soft online update for +responsiveness, then an offline reflective pass that merges similar entries and resolves +conflicts properly. Eventual consistency, deliberately — because doing full reflective +consolidation inline puts a model call on the write path of every interaction. + +**The dilemma has no general answer:** *"determining when to overwrite existing knowledge +versus when to treat new information as noise. Incorrect updates can overwrite critical +information."* What a system can do is make the decision **reversible** — which is the +argument for annotation over deletion, again. + +## 5. Evolution — forgetting + +Three policies on three different signals — creation time, access frequency, judged +importance. They are orthogonal and most systems need more than one. + +| Policy | Signal | Watch for | +|---|---|---| +| **Time-based** | age | evicting on age alone drops stable facts that were simply written early | +| **Frequency-based** | reads | **the long-tail trap** — see below | +| **Importance-driven** | a composite, increasingly a model's judgement of salience | the judge becomes a dependency, and an unaudited judge silently sets policy | + +**The long-tail trap, stated because frequency-based eviction is the easy one to reach +for:** *"heuristic forgetting mechanisms like LRU may eliminate long-tail knowledge, which +is seldom accessed but essential for correct decision-making."* The entry read twice a year +is often the incident, the exception, the one customer whose setup differs — precisely the +entry that prevents an expensive mistake. + +**The rule:** where storage is not the binding constraint, **demote rather than delete**. +Move it out of the default retrieval path, keep it reachable by explicit query. The survey +reports this is what many systems do in practice: *"when storage cost is not a critical +constraint, many memory systems avoid directly deleting certain memories."* + +Deletion remains a **correctness and privacy** operation — a person asking to be forgotten +is not a capacity decision, and `memory-architecture.md` §8 covers it. + +## 6. What this pack already implements + +Stated so this file is read as an extension and not as a replacement: + +| Survey concept | Where it already lives here | +|---|---| +| Formation, experiential, from contrast | `SKILL.md` §8 · `patterns.md` → Learning Extraction Heuristics | +| Consolidation, local | `patterns.md` → Fuzzy Deduplication | +| Updating, conflict resolution | `patterns.md` → Conflict Resolution | +| Forgetting, time-based | `patterns.md` → Confidence Management | +| Global integration, cross-scope | `patterns.md` → Cross-Resource Learning Transfer | +| **Frequency-based forgetting** | **nowhere — and the long-tail trap above is why that is a deliberate omission rather than a gap to close carelessly** | +| **Temporal annotation instead of deletion** | **nowhere** — Conflict Resolution currently resolves rather than annotates | +| **Dual-phase updating** | **nowhere** — the pack updates inline | + +The last three are named as absent rather than quietly added: each is a real change to a +mechanism that is in production, and this file's job is to say what the options are, not to +change `patterns.md` from a survey.