diff --git a/docs/MERGE_IMPACT.md b/docs/MERGE_IMPACT.md new file mode 100644 index 0000000..16dd9fd --- /dev/null +++ b/docs/MERGE_IMPACT.md @@ -0,0 +1,136 @@ +# MergeField: change consequence calculus + +Status: experimental research prototype. + +`forge impact` currently answers a useful but narrower question: which graph dependents are +reachable from a symbol or file? MergeField asks a different question: + +> Given the exact kind of change we are making, which consequences and verification +> obligations can propagate before this change is merged? + +The distinction matters. A one-line public API or schema edit can be more consequential than +a 50-file formatter sweep. A Markdown change can be runtime-neutral but still invalidate a +published doc surface. A test is usually a verification obligation, not a runtime dependent. + +## Model + +A change is represented as a semantic vector over seven dimensions: + +`runtime, contract, verification, docs, config, delivery, merge` + +A formatting edit starts close to zero. A public API edit starts high in contract, +verification, runtime, and docs. Callers can override every dimension, so the built-in +profiles are priors rather than truth. + +A typed relationship is not assigned one scalar edge weight. It has a sparse transfer matrix. +For example: + +- `imports` mostly transports runtime and contract consequence; +- `verified_by` transforms runtime/contract/config consequence into a test obligation; +- `documented_by` transforms contract/runtime/config consequence into documentation drift; +- `generates` transforms generator behavior/config consequence into generated-doc and delivery + consequence; +- `workflow_uses` transports consequence into verification and delivery risk. + +This is why MergeField is not simply another code graph walk. A graph can store the +relationships, but the calculation is a typed consequence field over those relationships. + +## Mathematics + +For changed atom `i`, let its semantic signal be a vector: + +`S_i in [0,1]^7` + +For a typed relation `e: u -> v`, let `M_e` be its 7 x 7 transfer matrix, `q_e` its evidence +confidence, and `d` a decay term. + +For one changed atom, propagation keeps only the strongest supported path per artifact and +dimension: + +`P_i(v) = max_path(S_i * M_path * product(q_e * d))` + +This max-path rule deliberately prevents a cycle from counting the same original change over +and over. + +Different changed atoms are independent causes. Their contributions combine with noisy-OR: + +`P(v,k) = 1 - product_i(1 - P_i(v,k))` + +So two weak independent edits can create a meaningful combined consequence without pretending +two correlated paths from the same edit are independent evidence. + +The merge summary separates: + +- peak consequence; +- breadth of secondary impact; +- uncertainty from severe changes with weak or missing relationship coverage; +- verification gap for consequential artifacts that have no known test relation. + +The engine returns the full dimension vector for every impacted artifact rather than hiding +all semantics behind one number. + +## Real ForgeKit probes + +The first benchmark cases are tied to real ForgeKit history. + +### `98ce7ece` formatting-only test edit + +The commit only reformatted one `readFileSync` call in `test/cortex_mcp.test.js`. +MergeField classifies it as low risk and does not invent documentation or runtime impact. + +### `d0f11aa` docs renderer/check change + +The commit widened Markdown handling to MDX and changed Mermaid normalization. The real commit +also updated generated Mintlify surfaces and `test/docs_render.test.js`. + +With `src/docs_render.js` and `src/docs_check.js` as changed roots, typed `generates`, +`documented_by`, and `verified_by` relationships propagate into the MDX surfaces and the +regression test. This is a class of consequence a scalar import-only graph cannot express +cleanly. + +### One-line public API probe + +A synthetic one-line public API change on the real `src/atlas.js` surface propagates into +`src/substrate.js`, `src/imagine.js`, `test/atlas.test.js`, `docs/GUIDE.md`, and `README.md` +when those real relationships are supplied. Test impact is high on the verification dimension; +doc impact is high on the docs dimension and remains zero on runtime. + +## Automatic evidence adapter + +`src/merge_impact_adapter.js` turns repository evidence into MergeField inputs instead of asking +a caller to label everything by hand. + +For each diff file it extracts added/removed lines and classifies the semantic seed as formatting, +test, docs, CI, dependency, schema, public API, config, generated output, or executable logic. +Security-sensitive and deletion-heavy deltas add independent consequence signals rather than +replacing the primary class. Classification returns both confidence and human-readable reasons. + +Formatting detection is deliberately semantic enough to survive normal formatter behavior. The +real ForgeKit `98ce7ece` patch adds a legal trailing comma while reflowing a call, so the adapter +normalizes whitespace and trailing commas adjacent to closing delimiters before deciding that +the token sequence is unchanged. + +Atlas supplies topology, not meaning. Its dependency edges point from consumer to dependency, so +the adapter reverses them into consequence direction. The consuming artifact determines how the +relation is interpreted: test consumers become `verified_by`, documentation becomes +`documented_by`, workflows become `workflow_uses`, and ordinary source dependencies preserve +`imports`, `calls`, or `inherits`. Generated registries can add explicit `generates` relations. + +This separation is intentional: + +- the diff answers **what kind of change happened**; +- Atlas and other evidence answer **where that consequence can travel**; +- transfer matrices answer **how the consequence changes meaning while travelling**; +- MergeField answers **what must be inspected, tested, regenerated, or blocked before merge**. + +## What is not solved yet + +Automatic diff and Atlas adaptation now exists, but relation coverage is not yet universal. +Repository-specific generators, schemas, runtime reflection, external services, deployment +contracts, database semantics, and hidden operational dependencies may require additional evidence +providers. Missing coverage therefore increases uncertainty instead of being interpreted as safety. + +The next experiment is historical calibration across merged ForgeKit changes. We should measure +false-safe rate first, then impacted-file precision/recall, recall@k, test-obligation recall, +documentation-obligation recall, and risk calibration by bucket. Only measured performance should +decide whether MergeField replaces or augments the current `forge impact` path. diff --git a/src/merge_impact.js b/src/merge_impact.js new file mode 100644 index 0000000..2e76535 --- /dev/null +++ b/src/merge_impact.js @@ -0,0 +1,421 @@ +export const DIMENSIONS = Object.freeze([ + "runtime", + "contract", + "verification", + "docs", + "config", + "delivery", + "merge", +]); + +const ZERO = Object.freeze(Object.fromEntries(DIMENSIONS.map((dimension) => [dimension, 0]))); +const clamp01 = (value) => + Math.max(0, Math.min(1, Number.isFinite(value) ? Number(value) : 0)); +const vector = (value = {}) => + Object.fromEntries(DIMENSIONS.map((dimension) => [dimension, clamp01(value[dimension] ?? 0)])); +const maxDimension = (value) => + Math.max(...DIMENSIONS.map((dimension) => value[dimension] ?? 0)); + +export const CHANGE_PROFILES = Object.freeze({ + formatting: vector({ + runtime: 0.005, + contract: 0.005, + verification: 0.03, + docs: 0.01, + config: 0.005, + delivery: 0.02, + merge: 0.02, + }), + comment: vector({ + runtime: 0.005, + contract: 0.01, + verification: 0.02, + docs: 0.08, + config: 0.005, + delivery: 0.01, + merge: 0.01, + }), + docs: vector({ + docs: 0.82, + verification: 0.08, + delivery: 0.08, + merge: 0.03, + }), + test: vector({ + verification: 0.82, + delivery: 0.08, + merge: 0.04, + }), + logic: vector({ + runtime: 0.68, + contract: 0.28, + verification: 0.72, + docs: 0.12, + config: 0.1, + delivery: 0.18, + merge: 0.12, + }), + public_api: vector({ + runtime: 0.78, + contract: 0.96, + verification: 0.88, + docs: 0.78, + config: 0.18, + delivery: 0.28, + merge: 0.18, + }), + config: vector({ + runtime: 0.42, + contract: 0.38, + verification: 0.52, + docs: 0.28, + config: 0.92, + delivery: 0.68, + merge: 0.18, + }), + schema: vector({ + runtime: 0.78, + contract: 0.94, + verification: 0.86, + docs: 0.62, + config: 0.76, + delivery: 0.7, + merge: 0.2, + }), + dependency: vector({ + runtime: 0.72, + contract: 0.62, + verification: 0.68, + docs: 0.34, + config: 0.78, + delivery: 0.84, + merge: 0.16, + }), + generated: vector({ + runtime: 0.3, + contract: 0.32, + verification: 0.28, + docs: 0.78, + config: 0.4, + delivery: 0.52, + merge: 0.08, + }), + ci: vector({ + runtime: 0.16, + contract: 0.12, + verification: 0.58, + docs: 0.12, + config: 0.64, + delivery: 0.94, + merge: 0.12, + }), + security: vector({ + runtime: 0.76, + contract: 0.72, + verification: 0.94, + docs: 0.42, + config: 0.64, + delivery: 0.72, + merge: 0.2, + }), +}); + +// Sparse transfer matrices. Rows are target dimensions and columns are source dimensions. +// A relation says how one consequence kind transforms into another at its dependent. +export const RELATION_MATRICES = Object.freeze({ + imports: { + runtime: { runtime: 0.88, contract: 0.62, config: 0.35 }, + contract: { contract: 0.76 }, + verification: { runtime: 0.2, contract: 0.25, config: 0.2 }, + merge: { merge: 0.35 }, + }, + calls: { + runtime: { runtime: 0.94, contract: 0.52 }, + contract: { contract: 0.5 }, + verification: { runtime: 0.24, contract: 0.24 }, + merge: { merge: 0.3 }, + }, + inherits: { + runtime: { runtime: 0.92, contract: 0.72 }, + contract: { contract: 0.86 }, + verification: { runtime: 0.3, contract: 0.35 }, + docs: { contract: 0.14 }, + }, + verified_by: { + verification: { + runtime: 0.98, + contract: 0.98, + config: 0.9, + delivery: 0.55, + verification: 0.92, + }, + merge: { merge: 0.15 }, + }, + documented_by: { + docs: { contract: 0.96, runtime: 0.38, config: 0.55, docs: 0.92 }, + verification: { contract: 0.08, docs: 0.12 }, + }, + generates: { + docs: { runtime: 0.92, contract: 0.72, config: 0.9, docs: 0.96 }, + delivery: { runtime: 0.38, config: 0.72, delivery: 0.7 }, + verification: { runtime: 0.18, config: 0.2 }, + }, + configures: { + runtime: { config: 0.82, contract: 0.28 }, + contract: { config: 0.46, contract: 0.55 }, + verification: { config: 0.72 }, + config: { config: 0.94 }, + delivery: { config: 0.72, delivery: 0.5 }, + }, + publishes: { + contract: { contract: 0.94, runtime: 0.32, config: 0.35 }, + docs: { contract: 0.72, docs: 0.65 }, + delivery: { contract: 0.38, config: 0.5, delivery: 0.84 }, + verification: { contract: 0.35, delivery: 0.3 }, + }, + workflow_uses: { + verification: { runtime: 0.46, contract: 0.32, verification: 0.62, config: 0.62 }, + config: { config: 0.58 }, + delivery: { runtime: 0.32, contract: 0.22, config: 0.72, delivery: 0.94 }, + }, + registry_member: { + runtime: { runtime: 0.42, contract: 0.42 }, + contract: { contract: 0.78, config: 0.35 }, + docs: { contract: 0.62, docs: 0.35 }, + verification: { contract: 0.32, config: 0.22 }, + }, + historical_coupling: Object.fromEntries( + DIMENSIONS.map((dimension) => [dimension, { [dimension]: 0.28 }]), + ), +}); + +export function signalForChange(change = {}) { + const base = CHANGE_PROFILES[change.kind] || CHANGE_PROFILES.logic; + const lines = Math.max(1, Number(change.linesChanged ?? change.lines ?? 1) || 1); + const size = Math.min(1, Math.log2(1 + lines) / 8); + const scaled = Object.fromEntries( + DIMENSIONS.map((dimension) => [ + dimension, + clamp01(base[dimension] * (1 + 0.22 * size)), + ]), + ); + for (const [dimension, value] of Object.entries(change.signal || {})) { + if (DIMENSIONS.includes(dimension)) scaled[dimension] = clamp01(value); + } + return scaled; +} + +function applyMatrix(current, matrix, confidence, decay) { + const out = { ...ZERO }; + for (const target of DIMENSIONS) { + const row = matrix?.[target]; + if (!row) continue; + let best = 0; + for (const [source, weight] of Object.entries(row)) { + best = Math.max(best, (current[source] || 0) * clamp01(weight)); + } + out[target] = clamp01(best * confidence * decay); + } + return out; +} + +function improve(existing, candidate, epsilon) { + const next = { ...existing }; + let changed = false; + for (const dimension of DIMENSIONS) { + if ((candidate[dimension] || 0) > (existing[dimension] || 0) + epsilon) { + next[dimension] = candidate[dimension]; + changed = true; + } + } + return { next, changed }; +} + +function noisyOr(values) { + let remain = 1; + for (const value of values) remain *= 1 - clamp01(value); + return clamp01(1 - remain); +} + +function artifactOverall(signal, criticality = 0) { + const weights = { + runtime: 0.22, + contract: 0.2, + verification: 0.18, + docs: 0.1, + config: 0.12, + delivery: 0.12, + merge: 0.06, + }; + const peak = maxDimension(signal); + const mean = DIMENSIONS.reduce( + (sum, dimension) => sum + signal[dimension] * weights[dimension], + 0, + ); + const base = clamp01(0.7 * peak + 0.3 * mean); + return noisyOr([base, 0.28 * clamp01(criticality) * base]); +} + +function indexArtifacts(artifacts) { + return new Map((artifacts || []).map((artifact) => [artifact.id, artifact])); +} + +export function analyzeMergeImpact({ + artifacts = [], + changes = [], + relations = [], + decay = 0.9, + epsilon = 1e-6, +} = {}) { + const artifactsById = indexArtifacts(artifacts); + for (const change of changes) { + if (!artifactsById.has(change.artifact)) { + artifactsById.set(change.artifact, { id: change.artifact, kind: "unknown" }); + } + } + + const outgoing = new Map(); + for (const relation of relations) { + const current = outgoing.get(relation.from) || []; + current.push(relation); + outgoing.set(relation.from, current); + } + + /** @type {Array<{ change: { artifact: string }, signal: Record, best: Map> }>} */ + const perSeed = []; + let truncated = false; + for (const change of changes) { + const signal = signalForChange(change); + const best = new Map([[change.artifact, signal]]); + const queue = [change.artifact]; + let head = 0; + const maxRelaxations = Math.max( + 128, + (artifactsById.size + 1) * Math.max(1, relations.length) * 4, + ); + let relaxations = 0; + + while (head < queue.length && relaxations < maxRelaxations) { + const from = queue[head++]; + const current = best.get(from) || ZERO; + for (const relation of outgoing.get(from) || []) { + const matrix = relation.matrix || RELATION_MATRICES[relation.kind]; + if (!matrix) continue; + const candidate = applyMatrix( + current, + matrix, + clamp01(relation.confidence ?? 1), + clamp01(relation.decay ?? decay), + ); + const previous = best.get(relation.to) || ZERO; + const { next, changed } = improve(previous, candidate, epsilon); + if (changed) { + best.set(relation.to, next); + queue.push(relation.to); + } + relaxations++; + if (relaxations >= maxRelaxations) break; + } + } + if (head < queue.length) truncated = true; + perSeed.push({ change, signal, best }); + } + + const impacted = []; + for (const [id, artifact] of artifactsById.entries()) { + const combined = {}; + for (const dimension of DIMENSIONS) { + combined[dimension] = noisyOr( + perSeed.map((seed) => seed.best.get(id)?.[dimension] || 0), + ); + } + const overall = artifactOverall(combined, artifact.criticality || 0); + if (overall <= epsilon) continue; + impacted.push({ + id, + kind: artifact.kind || "unknown", + changed: changes.some((change) => change.artifact === id), + criticality: clamp01(artifact.criticality || 0), + dimensions: combined, + overall, + }); + } + impacted.sort((left, right) => right.overall - left.overall || left.id.localeCompare(right.id)); + + const changedSet = new Set(changes.map((change) => change.artifact)); + const secondary = impacted.filter((item) => !changedSet.has(item.id)); + const peak = impacted[0]?.overall || 0; + const secondaryMass = secondary.reduce((sum, item) => sum + item.overall, 0); + const breadth = clamp01(1 - Math.exp(-secondaryMass / 4)); + + const uncertaintyParts = []; + for (const seed of perSeed) { + const edges = outgoing.get(seed.change.artifact) || []; + const severity = maxDimension(seed.signal); + if (!edges.length) { + uncertaintyParts.push(0.65 * severity); + continue; + } + const totalConfidence = edges.reduce( + (sum, relation) => sum + clamp01(relation.confidence ?? 1), + 0, + ); + const averageConfidence = totalConfidence / edges.length; + uncertaintyParts.push(0.18 * severity * (1 - averageConfidence)); + } + if (truncated) uncertaintyParts.push(0.6); + const uncertainty = noisyOr(uncertaintyParts); + + const verificationGaps = []; + for (const item of impacted) { + if (["test", "documentation"].includes(item.kind)) continue; + const consequential = Math.max( + item.dimensions.runtime, + item.dimensions.contract, + item.dimensions.config, + item.dimensions.delivery, + ); + if (consequential < 0.35) continue; + const hasTest = (outgoing.get(item.id) || []).some( + (relation) => relation.kind === "verified_by", + ); + if (!hasTest) verificationGaps.push(consequential * 0.55); + } + const verificationGap = noisyOr(verificationGaps); + const risk = noisyOr([ + 0.58 * peak, + 0.38 * breadth, + 0.5 * uncertainty, + 0.52 * verificationGap, + ]); + const level = + risk >= 0.75 ? "critical" : risk >= 0.5 ? "high" : risk >= 0.25 ? "medium" : "low"; + + const obligations = { + tests: impacted + .filter((item) => item.kind === "test" && item.dimensions.verification >= 0.08) + .map((item) => item.id), + docs: impacted + .filter((item) => item.kind === "documentation" && item.dimensions.docs >= 0.08) + .map((item) => item.id), + config: impacted + .filter( + (item) => + ["config", "workflow", "manifest"].includes(item.kind) && + Math.max(item.dimensions.config, item.dimensions.delivery) >= 0.08, + ) + .map((item) => item.id), + }; + + return { + risk, + level, + peak, + breadth, + uncertainty, + verificationGap, + truncated, + impacted, + obligations, + }; +} diff --git a/src/merge_impact_adapter.js b/src/merge_impact_adapter.js new file mode 100644 index 0000000..9f79708 --- /dev/null +++ b/src/merge_impact_adapter.js @@ -0,0 +1,260 @@ +import { extname } from "node:path"; +import { analyzeMergeImpact } from "./merge_impact.js"; + +const DOC_EXTS = new Set([".md", ".mdx", ".rst", ".adoc"]); +const CONFIG_EXTS = new Set([".json", ".yaml", ".yml", ".toml", ".ini", ".cfg"]); +const TEST_RE = /(^|\/)(?:test|tests|spec|specs)(\/|$)|(?:^|\.)test\.[^.]+$|(?:^|\.)spec\.[^.]+$/i; +const WORKFLOW_RE = /(^|\/)\.github\/workflows\/|(?:^|\/)(?:Jenkinsfile|Dockerfile)$/i; +const DEPENDENCY_RE = + /(^|\/)(?:package(?:-lock)?\.json|pnpm-lock\.yaml|yarn\.lock|Cargo\.toml|Cargo\.lock|requirements(?:-[^/]+)?\.txt|pyproject\.toml|poetry\.lock|go\.mod|go\.sum|pom\.xml|build\.gradle(?:\.kts)?|Gemfile(?:\.lock)?|composer\.json|composer\.lock)$/i; +const SCHEMA_PATH_RE = /(?:^|\/)(?:schema|schemas|migrations?|openapi|swagger)(?:\/|\.|$)/i; +const GENERATED_PATH_RE = /(?:^|\/)(?:generated|dist|build|coverage)(?:\/|$)|\.generated\./i; +const PUBLIC_API_RE = + /\b(?:export\s+(?:default\s+)?(?:async\s+)?(?:function|class|const|let|var)|module\.exports|exports\.[A-Za-z_$]|pub\s+(?:fn|struct|enum|trait|mod|const|static|type)|public\s+(?:class|interface|record|enum)|__all__\s*=|@(?:Get|Post|Put|Patch|Delete|RequestMapping)\b)/; +const SCHEMA_TEXT_RE = + /\b(?:schema|openapi|swagger|migration|CREATE\s+TABLE|ALTER\s+TABLE|required\s*:|properties\s*:|type\s*:\s*["']?(?:object|array|string|number|integer|boolean))\b/i; +const SECURITY_RE = + /\b(?:auth(?:entication|orization)?|permission|role|scope|token|secret|password|credential|session|cookie|csrf|cors|crypto|encrypt|decrypt|sign(?:ature)?|verify|acl|rbac|oauth|jwt)\b/i; +const DEPENDENCY_TEXT_RE = + /["']?(?:dependencies|devDependencies|peerDependencies|optionalDependencies)["']?\s*:|\b(?:version|image)\s*:/i; + +const clamp01 = (value) => Math.max(0, Math.min(1, Number(value) || 0)); + +export function artifactKind(path = "") { + const normalized = String(path).replaceAll("\\", "/"); + const ext = extname(normalized).toLowerCase(); + if (TEST_RE.test(normalized)) return "test"; + if (DOC_EXTS.has(ext)) return "documentation"; + if (WORKFLOW_RE.test(normalized)) return "workflow"; + if (DEPENDENCY_RE.test(normalized)) return "manifest"; + if (SCHEMA_PATH_RE.test(normalized)) return "schema"; + if (CONFIG_EXTS.has(ext)) return "config"; + return "source"; +} + +function changedLines(patch = "") { + const added = []; + const removed = []; + for (const line of String(patch).split("\n")) { + if (line.startsWith("+++") || line.startsWith("---")) continue; + if (line.startsWith("+")) added.push(line.slice(1)); + else if (line.startsWith("-")) removed.push(line.slice(1)); + } + return { added, removed }; +} + +function compact(lines) { + return lines.join("").replace(/\s+/g, "").replace(/,([)\]}])/g, "$1"); +} + +function formattingOnly(added, removed) { + if (!added.length || !removed.length) return false; + const before = compact(removed); + const after = compact(added); + return Boolean(before) && before === after; +} + +function mergeSignal(base, extra) { + const out = { ...base }; + for (const [key, value] of Object.entries(extra)) out[key] = Math.max(out[key] || 0, value); + return out; +} + +export function classifyChangedFile(file = {}) { + const filename = String(file.filename || file.path || "").replaceAll("\\", "/"); + const patch = String(file.patch || ""); + const { added, removed } = changedLines(patch); + const text = [...added, ...removed].join("\n"); + const linesChanged = + Number(file.changes) || + Number(file.additions || 0) + Number(file.deletions || 0) || + added.length + removed.length || + 1; + const reasons = []; + let kind = "logic"; + let confidence = 0.62; + let signal = {}; + + if (formattingOnly(added, removed)) { + kind = "formatting"; + confidence = 0.98; + reasons.push("token sequence is unchanged after whitespace normalization"); + } else if (TEST_RE.test(filename)) { + kind = "test"; + confidence = 0.98; + reasons.push("test/spec path"); + } else if (DOC_EXTS.has(extname(filename).toLowerCase())) { + kind = "docs"; + confidence = 0.99; + reasons.push("documentation extension"); + } else if (WORKFLOW_RE.test(filename)) { + kind = "ci"; + confidence = 0.98; + reasons.push("delivery/workflow path"); + } else if (DEPENDENCY_RE.test(filename) || DEPENDENCY_TEXT_RE.test(text)) { + kind = "dependency"; + confidence = DEPENDENCY_RE.test(filename) ? 0.97 : 0.82; + reasons.push("dependency/manifest surface"); + } else if (SCHEMA_PATH_RE.test(filename) || SCHEMA_TEXT_RE.test(text)) { + kind = "schema"; + confidence = SCHEMA_PATH_RE.test(filename) ? 0.94 : 0.76; + reasons.push("schema or migration semantics"); + } else if (PUBLIC_API_RE.test(text)) { + kind = "public_api"; + confidence = 0.86; + reasons.push("exported/public contract changed"); + } else if (CONFIG_EXTS.has(extname(filename).toLowerCase())) { + kind = "config"; + confidence = 0.9; + reasons.push("configuration artifact"); + } else if (GENERATED_PATH_RE.test(filename)) { + kind = "generated"; + confidence = 0.88; + reasons.push("generated artifact path"); + } else { + reasons.push("executable/source delta without a stronger structural classifier"); + } + + if (SECURITY_RE.test(text)) { + signal = mergeSignal(signal, { + runtime: 0.82, + contract: 0.72, + verification: 0.96, + delivery: 0.62, + }); + confidence = Math.max(confidence, 0.78); + reasons.push("security-sensitive vocabulary changed"); + } + + const destructiveRatio = clamp01( + (Number(file.deletions) || removed.length) / Math.max(1, linesChanged), + ); + if (destructiveRatio > 0.65 && !["formatting", "docs", "test"].includes(kind)) { + signal = mergeSignal(signal, { contract: 0.68, verification: 0.78, merge: 0.22 }); + reasons.push("change is deletion-heavy"); + } + + return { + artifact: filename, + kind, + linesChanged, + signal, + confidence, + reasons, + }; +} + +export function changeAtomsFromDiff(files = []) { + return files.filter((file) => file?.filename || file?.path).map(classifyChangedFile); +} + +function relationKey(relation) { + return `${relation.from}\0${relation.to}\0${relation.kind}`; +} + +function atlasNodeMap(atlas) { + return new Map((atlas?.nodes || []).map((node) => [node.id, node])); +} + +export function atlasEvidence(atlas, { criticality = {}, generatedTargets = {} } = {}) { + const nodes = atlasNodeMap(atlas); + const artifactMap = new Map(); + const relations = new Map(); + + const addArtifact = (file, nodeKind) => { + if (!file) return; + const existing = artifactMap.get(file); + const inferred = nodeKind === "doc" ? "documentation" : artifactKind(file); + if (!existing) { + artifactMap.set(file, { + id: file, + kind: inferred, + criticality: clamp01(criticality[file] || 0), + }); + } + }; + + for (const node of nodes.values()) addArtifact(node.file, node.kind); + + for (const edge of atlas?.edges || []) { + if (edge.unresolved || edge.kind === "contains") continue; + const sourceNode = nodes.get(edge.source); + const targetNode = nodes.get(edge.target); + const sourceFile = sourceNode?.file; + const targetFile = targetNode?.file; + if (!sourceFile || !targetFile || sourceFile === targetFile) continue; + addArtifact(sourceFile, sourceNode.kind); + addArtifact(targetFile, targetNode.kind); + + const sourceKind = artifactMap.get(sourceFile)?.kind; + let kind; + if (sourceKind === "test") kind = "verified_by"; + else if (sourceKind === "documentation") kind = "documented_by"; + else if (sourceKind === "workflow") kind = "workflow_uses"; + else if (["config", "manifest", "schema"].includes(sourceKind)) kind = "configures"; + else if (["calls", "imports", "inherits"].includes(edge.kind)) kind = edge.kind; + else continue; + + // Atlas edges say source depends on target. Consequence flows the opposite way. + const relation = { + from: targetFile, + to: sourceFile, + kind, + confidence: clamp01(edge.confidence ?? 0.7), + }; + const key = relationKey(relation); + const prior = relations.get(key); + if (!prior || relation.confidence > prior.confidence) relations.set(key, relation); + } + + for (const [generator, targets] of Object.entries(generatedTargets || {})) { + addArtifact(generator, "source"); + for (const target of targets || []) { + addArtifact(target, "doc"); + const relation = { from: generator, to: target, kind: "generates", confidence: 0.98 }; + relations.set(relationKey(relation), relation); + } + } + + return { artifacts: [...artifactMap.values()], relations: [...relations.values()] }; +} + +export function analyzeDiffImpact({ + files = [], + atlas = null, + criticality = {}, + generatedTargets = {}, + extraRelations = [], +} = {}) { + const changes = changeAtomsFromDiff(files); + const evidence = atlasEvidence(atlas, { criticality, generatedTargets }); + const artifactMap = new Map(evidence.artifacts.map((artifact) => [artifact.id, artifact])); + for (const change of changes) { + if (!artifactMap.has(change.artifact)) { + artifactMap.set(change.artifact, { + id: change.artifact, + kind: artifactKind(change.artifact), + criticality: clamp01(criticality[change.artifact] || 0), + }); + } + } + + const result = analyzeMergeImpact({ + artifacts: [...artifactMap.values()], + changes, + relations: [...evidence.relations, ...extraRelations], + }); + + return { + ...result, + changes, + evidence: { + atlasRelations: evidence.relations.length, + extraRelations: extraRelations.length, + generatedRelations: Object.values(generatedTargets || {}).reduce( + (sum, targets) => sum + (targets?.length || 0), + 0, + ), + }, + }; +} diff --git a/test/merge_impact.test.js b/test/merge_impact.test.js new file mode 100644 index 0000000..cf9490f --- /dev/null +++ b/test/merge_impact.test.js @@ -0,0 +1,163 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { analyzeMergeImpact } from "../src/merge_impact.js"; + +const artifact = (id, kind = "source", criticality = 0) => ({ id, kind, criticality }); +const relation = (from, to, kind, confidence = 1) => ({ from, to, kind, confidence }); + +test("ForgeKit 98ce7ece formatting-only test edit stays low risk", () => { + const result = analyzeMergeImpact({ + artifacts: [artifact("test/cortex_mcp.test.js", "test")], + changes: [ + { + artifact: "test/cortex_mcp.test.js", + kind: "formatting", + linesChanged: 4, + }, + ], + }); + assert.equal(result.level, "low"); + assert.ok(result.risk < 0.12, `risk=${result.risk}`); + assert.deepEqual(result.obligations.docs, []); +}); + +test("single-line public API edit fans into consumers, tests, and docs", () => { + const artifacts = [ + artifact("src/atlas.js", "source", 0.9), + artifact("src/substrate.js", "source", 0.8), + artifact("src/imagine.js", "source", 0.5), + artifact("test/atlas.test.js", "test"), + artifact("docs/GUIDE.md", "documentation"), + artifact("README.md", "documentation"), + ]; + const relations = [ + relation("src/atlas.js", "src/substrate.js", "imports", 0.98), + relation("src/atlas.js", "src/imagine.js", "imports", 0.95), + relation("src/atlas.js", "test/atlas.test.js", "verified_by"), + relation("src/atlas.js", "docs/GUIDE.md", "documented_by", 0.95), + relation("src/atlas.js", "README.md", "documented_by", 0.8), + ]; + const result = analyzeMergeImpact({ + artifacts, + relations, + changes: [{ artifact: "src/atlas.js", kind: "public_api", linesChanged: 1 }], + }); + const ids = new Set(result.impacted.map((item) => item.id)); + for (const id of artifacts.map((item) => item.id)) assert.ok(ids.has(id), id); + assert.ok(result.obligations.tests.includes("test/atlas.test.js")); + assert.ok(result.obligations.docs.includes("docs/GUIDE.md")); + assert.ok( + result.impacted.find((item) => item.id === "test/atlas.test.js").dimensions.verification > 0.7, + ); + assert.ok(result.impacted.find((item) => item.id === "docs/GUIDE.md").dimensions.docs > 0.6); +}); + +test("ForgeKit d0f11aa docs renderer change predicts generated MDX surfaces and its test", () => { + const docs = [ + "ARCHITECTURE.md", + "mintlify/cli/overview.mdx", + "mintlify/concepts/config-compiler.mdx", + "mintlify/concepts/pre-action-gate.mdx", + "mintlify/concepts/proof-carrying-memory.mdx", + "mintlify/guides/team-memory.mdx", + "mintlify/guides/zero-config-onboarding.mdx", + ]; + const artifacts = [ + artifact("src/docs_render.js", "source", 0.65), + artifact("src/docs_check.js", "source", 0.6), + artifact("test/docs_render.test.js", "test"), + ...docs.map((id) => artifact(id, "documentation")), + ]; + const relations = [ + relation("src/docs_render.js", "test/docs_render.test.js", "verified_by"), + ...docs.map((id) => relation("src/docs_render.js", id, "generates", 0.96)), + ...docs.map((id) => relation("src/docs_check.js", id, "documented_by", 0.72)), + ]; + const result = analyzeMergeImpact({ + artifacts, + relations, + changes: [ + { + artifact: "src/docs_render.js", + kind: "logic", + linesChanged: 5, + signal: { docs: 0.65 }, + }, + { + artifact: "src/docs_check.js", + kind: "logic", + linesChanged: 6, + signal: { docs: 0.55 }, + }, + ], + }); + for (const id of docs) assert.ok(result.obligations.docs.includes(id), id); + assert.ok(result.obligations.tests.includes("test/docs_render.test.js")); + assert.ok(result.breadth > 0.4, `breadth=${result.breadth}`); +}); + +test("independent changed roots combine with noisy-OR instead of max-only propagation", () => { + const artifacts = [artifact("a.js"), artifact("b.js"), artifact("consumer.js")]; + const relations = [ + relation("a.js", "consumer.js", "historical_coupling"), + relation("b.js", "consumer.js", "historical_coupling"), + ]; + const one = analyzeMergeImpact({ + artifacts, + relations, + changes: [{ artifact: "a.js", kind: "logic" }], + }); + const two = analyzeMergeImpact({ + artifacts, + relations, + changes: [ + { artifact: "a.js", kind: "logic" }, + { artifact: "b.js", kind: "logic" }, + ], + }); + const oneProbability = one.impacted.find( + (item) => item.id === "consumer.js", + ).dimensions.runtime; + const twoProbability = two.impacted.find( + (item) => item.id === "consumer.js", + ).dimensions.runtime; + assert.ok(twoProbability > oneProbability, `${twoProbability} <= ${oneProbability}`); +}); + +test("cycle cannot self-amplify a single seed", () => { + const artifacts = [artifact("a.js"), artifact("b.js")]; + const relations = [ + relation("a.js", "b.js", "imports"), + relation("b.js", "a.js", "imports"), + ]; + const result = analyzeMergeImpact({ + artifacts, + relations, + changes: [{ artifact: "a.js", kind: "logic" }], + decay: 0.9, + }); + const a = result.impacted.find((item) => item.id === "a.js").dimensions.runtime; + const b = result.impacted.find((item) => item.id === "b.js").dimensions.runtime; + assert.ok(a < 0.9 && b < a, `a=${a} b=${b}`); + assert.equal(result.truncated, false); +}); + +test("high-severity orphan is uncertainty, never declared safe", () => { + const result = analyzeMergeImpact({ + artifacts: [artifact("unknown/schema.json", "schema")], + changes: [{ artifact: "unknown/schema.json", kind: "schema", linesChanged: 1 }], + }); + assert.ok(result.uncertainty > 0.5, `uncertainty=${result.uncertainty}`); + assert.notEqual(result.level, "low"); +}); + +test("documentation relation creates docs risk without pretending runtime execution", () => { + const result = analyzeMergeImpact({ + artifacts: [artifact("src/api.js"), artifact("README.md", "documentation")], + changes: [{ artifact: "src/api.js", kind: "public_api", linesChanged: 1 }], + relations: [relation("src/api.js", "README.md", "documented_by")], + }); + const doc = result.impacted.find((item) => item.id === "README.md"); + assert.ok(doc.dimensions.docs > 0.7); + assert.equal(doc.dimensions.runtime, 0); +}); diff --git a/test/merge_impact_adapter.test.js b/test/merge_impact_adapter.test.js new file mode 100644 index 0000000..8bb6048 --- /dev/null +++ b/test/merge_impact_adapter.test.js @@ -0,0 +1,194 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { + analyzeDiffImpact, + artifactKind, + atlasEvidence, + classifyChangedFile, +} from "../src/merge_impact_adapter.js"; + +const node = (id, file, kind = "module") => ({ id, file, kind, name: id }); + +test("real ForgeKit formatting commit is classified as formatting", () => { + const change = classifyChangedFile({ + filename: "test/cortex_mcp.test.js", + additions: 3, + deletions: 1, + patch: + '@@ -1 +1,3 @@\n-const body = readFileSync(file, "utf8");\n+const body = readFileSync(\n+ file,\n+ "utf8",\n+);', + }); + assert.equal(change.kind, "formatting"); + assert.ok(change.confidence > 0.95); +}); + +test("one exported-line delta becomes a public contract signal", () => { + const change = classifyChangedFile({ + filename: "src/atlas.js", + additions: 1, + deletions: 1, + patch: + "@@ -1 +1 @@\n-export function impact(atlas, target) {\n+export function impact(atlas, target, options = {}) {", + }); + assert.equal(change.kind, "public_api"); + assert.ok(change.confidence >= 0.85); +}); + +test("atlas dependency direction is inverted into consequence direction", () => { + const atlas = { + nodes: [ + node("module:src.atlas", "src/atlas.js"), + node("module:src.substrate", "src/substrate.js"), + node("module:test.atlas", "test/atlas.test.js"), + node("doc:README.md", "README.md", "doc"), + ], + edges: [ + { + source: "module:src.substrate", + target: "module:src.atlas", + kind: "imports", + confidence: 0.9, + }, + { + source: "module:test.atlas", + target: "module:src.atlas", + kind: "imports", + confidence: 0.95, + }, + { + source: "doc:README.md", + target: "module:src.atlas", + kind: "references", + confidence: 0.8, + }, + ], + }; + const evidence = atlasEvidence(atlas); + assert.ok( + evidence.relations.some( + (item) => + item.from === "src/atlas.js" && + item.to === "src/substrate.js" && + item.kind === "imports", + ), + ); + assert.ok( + evidence.relations.some( + (item) => + item.from === "src/atlas.js" && + item.to === "test/atlas.test.js" && + item.kind === "verified_by", + ), + ); + assert.ok( + evidence.relations.some( + (item) => + item.from === "src/atlas.js" && + item.to === "README.md" && + item.kind === "documented_by", + ), + ); +}); + +test("automatic diff plus atlas produces test and docs obligations", () => { + const atlas = { + nodes: [ + node("module:src.atlas", "src/atlas.js"), + node("module:src.substrate", "src/substrate.js"), + node("module:test.atlas", "test/atlas.test.js"), + node("doc:README.md", "README.md", "doc"), + ], + edges: [ + { + source: "module:src.substrate", + target: "module:src.atlas", + kind: "imports", + confidence: 0.98, + }, + { + source: "module:test.atlas", + target: "module:src.atlas", + kind: "imports", + confidence: 0.98, + }, + { + source: "doc:README.md", + target: "module:src.atlas", + kind: "references", + confidence: 0.9, + }, + ], + }; + const result = analyzeDiffImpact({ + atlas, + files: [ + { + filename: "src/atlas.js", + additions: 1, + deletions: 1, + patch: + "@@ -1 +1 @@\n-export function impact(atlas, target) {\n+export function impact(atlas, target, options = {}) {", + }, + ], + }); + assert.equal(result.changes[0].kind, "public_api"); + assert.ok(result.obligations.tests.includes("test/atlas.test.js")); + assert.ok(result.obligations.docs.includes("README.md")); + assert.ok(result.impacted.some((item) => item.id === "src/substrate.js")); +}); + +test("generator evidence carries registry changes into generated documentation", () => { + const atlas = { + nodes: [ + node("module:src.commands", "src/commands.js"), + node("module:src.docs_render", "src/docs_render.js"), + node("module:test.docs_render", "test/docs_render.test.js"), + ], + edges: [ + { + source: "module:src.docs_render", + target: "module:src.commands", + kind: "imports", + confidence: 0.98, + }, + { + source: "module:test.docs_render", + target: "module:src.docs_render", + kind: "imports", + confidence: 0.98, + }, + ], + }; + const result = analyzeDiffImpact({ + atlas, + generatedTargets: { + "src/docs_render.js": ["README.md", "docs/GUIDE.md"], + }, + files: [ + { + filename: "src/commands.js", + additions: 1, + deletions: 0, + patch: '@@ -1 +1,2 @@\n export const GROUPS = {\n+ Memory: ["impact"],', + }, + ], + }); + assert.ok(result.obligations.docs.includes("README.md")); + assert.ok(result.obligations.docs.includes("docs/GUIDE.md")); + assert.ok(result.obligations.tests.includes("test/docs_render.test.js")); +}); + +test("workflow and manifest paths get distinct change semantics", () => { + assert.equal(artifactKind(".github/workflows/ci.yml"), "workflow"); + assert.equal(artifactKind("package.json"), "manifest"); + assert.equal( + classifyChangedFile({ filename: ".github/workflows/ci.yml", patch: "+ run: npm test" }).kind, + "ci", + ); + assert.equal( + classifyChangedFile({ + filename: "package.json", + patch: '- "dependencies": {}\n+ "dependencies": {"x":"1.0.0"}', + }).kind, + "dependency", + ); +});