Explain the data2dsl project

README.md

@@ -1,29 +1,203 @@
 # data2dsl
 
-`data2dsl` is a planned reuse-first composition layer for turning existing
-data-source capabilities into comparable, evidence-bearing observations.
+`data2dsl` is a planned, evidence-first comparison layer. It will turn facts
+from existing data sources into comparable observations and deterministic
+differences that other systems can reason about.
 
-The project is currently in Phase 0: governance bootstrap and capability
-inventory. No functional implementation or final architecture is approved.
+The short version:
 
-## Phase 0 outcome
+> Ask one bounded question, acquire the relevant facts from two or more
+> sources, normalize them without losing provenance, compare like with like,
+> and return the result together with evidence.
 
-- verify reusable capabilities in `semcod/*`, `subactor/*`, and
-  `wellmanifest/*`;
-- inspect reusable seams currently embedded in `semcod/todo2code`;
-- record evidence and decisions in `CAPABILITY_MAP.md`;
-- propose a composition graph without implementing product features.
+The project is currently in contract and integration planning. The repository
+contains governance, capability evidence and architectural decisions, but no
+functional product implementation or final public DSL yet.
 
-## Architectural invariant
+## The problem
 
-Reuse first. Extract second. Extend third. Implement new only as a last resort.
+Useful facts already exist across Markdown documents, Git repositories, GitHub,
+configuration files, code analyzers and browser-backed sources. Each source has
+its own structure and vocabulary. Today a consumer such as `todo2code` must
+either understand every source or rely on an LLM to interpret incomparable
+outputs.
 
-`data2dsl` must remain a small composition, routing, mapping, normalization,
-and comparison-glue layer rather than becoming a replacement monolith for
-`todo2code`.
+That creates four recurring problems:
+
+1. the same metric can be named or represented differently by each source;
+2. values may refer to different actors, repositories or time windows;
+3. conclusions can lose the evidence needed to verify them;
+4. source acquisition, deterministic comparison and higher-level reasoning get
+   mixed into one component.
+
+`data2dsl` is intended to provide the missing factual boundary between source
+tools and reasoning consumers.
+
+## Who it is for
+
+The primary consumers are programs and agents that need to compare claims with
+observed data while preserving provenance. Initial consumers are expected to
+include `todo2code` and repository-governance workflows, but the core must not
+depend on either one.
+
+A human may formulate the question, inspect the differences and follow the
+evidence. A source adapter acquires facts. `data2dsl` normalizes and compares
+them. A separate consumer decides what the result means or what action, if any,
+should follow.
+
+## Golden case
+
+The first end-to-end case is:
+
+> Compare statements in `work-summary.md` with actual GitHub activity for the
+> same repository, actor, metric and time window.
+
+For example, a summary might claim 12 commits for a person during a given
+week, while the GitHub source reports 10. The planned result is not prose or an
+LLM verdict. It is an evidence-bearing comparison containing, conceptually:
+
+| Field | Example |
+| --- | --- |
+| Subject | repository and actor |
+| Metric | commit count |
+| Window | explicit start and end |
+| Left observation | claimed value from a Markdown location |
+| Right observation | measured value from GitHub pages/API results |
+| Outcome | `CONFLICT` |
+| Delta | `-2` |
+| Evidence | immutable references and content digests for both sides |
+
+This table illustrates intended behavior; it is not a final API or schema.
+
+## Planned inputs
+
+A bounded comparison needs three kinds of input:
+
+- a query describing the subject, metric, sources and time window;
+- source locations and the authority or credentials needed by their existing
+  adapters;
+- explicit mapping/comparison rules when source vocabularies differ.
+
+Natural-language interpretation may help construct a query, but it must not
+silently change the metric, window or source identity. Unresolved ambiguity
+must remain visible.
+
+## Planned outputs
+
+The factual output should contain:
+
+- normalized source observations with stable identity and source state;
+- deterministic scalar or set comparisons;
+- outcomes such as `MATCH`, `CONFLICT`, `MISSING_LEFT`, `MISSING_RIGHT` and
+  `UNEVALUABLE`;
+- typed deltas where a delta is meaningful;
+- evidence references sufficient to locate, integrity-check and reproduce the
+  source facts;
+- explicit gaps when acquisition, mapping or comparison cannot be completed.
+
+`UNEVALUABLE` is not success and missing data is not zero. Comparison outcomes
+are also distinct from the state of an individual observation.
+
+## Planned composition
+
+```mermaid
+flowchart LR
+    Q["Bounded query"] --> R["Routing and explicit mapping"]
+    R --> M["Markdown via mdflow"]
+    R --> G["Git factual seam"]
+    R --> H["GitHub via Diagit extension"]
+    R --> C["Existing code/data analyzers"]
+    M --> O["Comparable observations + evidence"]
+    G --> O
+    H --> O
+    C --> O
+    O --> D["Deterministic comparator"]
+    D --> F["Facts, outcomes, deltas, gaps, evidence"]
+    F --> X["todo2code or another reasoning consumer"]
+```
+
+This is a composition hypothesis, not a final runtime contract. Current
+reuse decisions and their pinned evidence are recorded in
+[`docs/CAPABILITY_MAP.md`](docs/CAPABILITY_MAP.md).
+
+## What data2dsl owns
+
+The project should own only the smallest missing responsibilities:
+
+- routing a bounded query to declared source capabilities;
+- explicit mapping from source facts to comparable metric keys;
+- normalization that preserves source identity, time and evidence;
+- deterministic comparability checks and scalar/set differences;
+- a thin adapter boundary for existing source tools;
+- factual results and typed gaps for downstream consumers.
+
+## What data2dsl does not own
+
+The project is not intended to become:
+
+- a universal parser framework;
+- a replacement Git or GitHub client;
+- a replacement for `mdflow`, Diagit, code analyzers or `todo2code`;
+- an LLM reasoning or conclusion engine;
+- an autonomous enforcement or mutation system;
+- a Digital Twin event store or all-traits Twin runtime;
+- a place to copy code from neighboring repositories without an explicit,
+  compatibility-tested extraction decision.
+
+Source adapters remain responsible for truthful acquisition. Standards owners
+remain responsible for shared contracts. Consumers remain responsible for
+reasoning, policy and action.
+
+## Reuse-first strategy
+
+Every capability follows this order:
+
+1. **REUSE** an existing public API or CLI when its behavior and ownership fit.
+2. **EXTRACT** the smallest neutral seam when useful behavior is trapped inside
+   another product; preserve its language and compatibility.
+3. **EXTEND** the established owning component when a nearby capability exists.
+4. Mark a capability **MISSING** and implement it locally only after the first
+   three options have been disproved with current evidence.
+
+Examples from the Phase 0 inventory include reusing `mdflow` for Markdown
+structure, extending Diagit's established GitHub boundary for commit metrics,
+and keeping `todo2code` as a reasoning consumer rather than moving its policy
+into data2dsl.
+
+## Delivery roadmap
+
+The planned delivery order is dependency-driven:
+
+1. decide the observation/evidence contract and its compatibility with
+   `subactor/twin`;
+2. agree a minimal shared query/result profile with its standards owner;
+3. define the smallest deterministic scalar/set comparison semantics;
+4. extend Diagit with the read-only GitHub metrics required by the golden case;
+5. implement and validate `work-summary.md` versus GitHub in Docker;
+6. evaluate Git/config/AST extraction from `todo2code` only when a real second
+   consumer proves it is necessary;
+7. integrate factual results back into `todo2code` without moving reasoning
+   into data2dsl.
+
+Each step requires its own bounded ticket and evidence. Changes to another
+repository require that repository's owner-approved workflow.
+
+## Current state
+
+- Phase 0 governance bootstrap and capability inventory are complete.
+- Docker bootstrap and the deterministic governance gate pass.
+- No product source, final observation schema, query DSL, GitHub extension or
+  golden-case implementation exists yet.
+- Open architectural claims must remain explicitly provisional until their
+  owning contracts and compatibility tests exist.
+
+See [`TODO.md`](TODO.md) for current work and
+[`project/TICKETS.md`](project/TICKETS.md) for governed evidence.
 
 ## Governance
 
 This repository adopts an immutable published revision of
 `wellmanifest/new-project`. Multi-step work is ticket-governed and bounded by
-the active ticket's `intent.json`.
+the active ticket's `intent.json`. Human-owned `user-*` files are never written
+by agents, and implementation claims require deterministic validation rather
+than README text alone.

TODO.md

@@ -12,3 +12,17 @@
   authority and repository visibility were supplied.
 
 No functional implementation is authorized in Phase 0.
+
+## Phase 1A: contract decision
+
+- [ ] Decide compatibility of `subactor/twin` `Observation` and `EvidenceRef`
+  for `data2dsl` in [`ticket-002`](project/ticket-002/README.md).
+- [ ] Publish the pinned evidence and consequences as a decision document.
+
+No implementation or changes to external repositories are authorized by this
+ticket.
+
+## Project communication
+
+- [x] Explain the concrete data2dsl product vision, boundaries, golden case and
+  roadmap in the root README under [`ticket-003`](project/ticket-003/README.md).

project/TICKETS.md

@@ -7,4 +7,6 @@ analysis-generated `project/README.md`.
 | Ticket ID | Spec | Preprompt | Human input | Agent plans | Agent logs | Changelog |
 | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
 | **ticket-001** | [`README.md`](./ticket-001/README.md) | [`preprompt.md`](./ticket-001/preprompt.md) | - |  [`ai-codex.md`](./ticket-001/ai-codex.md) |  [`ai-codex-logs.txt`](./ticket-001/ai-codex-logs.txt) | [`changelog.md`](./ticket-001/changelog.md) |
+| **ticket-002** | [`README.md`](./ticket-002/README.md) | [`preprompt.md`](./ticket-002/preprompt.md) | - |  [`ai-codex.md`](./ticket-002/ai-codex.md) |  [`ai-codex-logs.txt`](./ticket-002/ai-codex-logs.txt) | [`changelog.md`](./ticket-002/changelog.md) |
+| **ticket-003** | [`README.md`](./ticket-003/README.md) | [`preprompt.md`](./ticket-003/preprompt.md) | - |  [`ai-codex.md`](./ticket-003/ai-codex.md) |  [`ai-codex-logs.txt`](./ticket-003/ai-codex-logs.txt) | [`changelog.md`](./ticket-003/changelog.md) |
 <!-- AUTO:TICKET_INDEX:END -->

project/ticket-001/README.md

@@ -2,8 +2,8 @@
 
 - **ID**: ticket-001
 - **Owner**: unresolved:human
-- **Status**: IN_PROGRESS
-- **Workflow state**: EDIT
+- **Status**: DONE
+- **Workflow state**: DONE
 - **Created**: 2026-08-13
 
 ## Goal and scope
@@ -32,6 +32,9 @@ GitHub clients, runtime dependencies, and golden-case implementation.
 All Phase 0 acceptance criteria pass. The user explicitly authorized creation
 and publication of the public `semcod/data2dsl` GitHub repository on
 2026-08-13. No product code, final DSL, extraction or refactor was introduced.
+The published default branch contains the exact Phase 0 result at commit
+`067b76b67802b17084c1209a5e96121dec5b8a2f`; this governance-only update closes
+the completed ticket.
 
 ## Participants
 

project/ticket-001/ai-codex-logs.txt

@@ -59,3 +59,13 @@ current Phase 0 results; do not implement or modify other repositories.
 
 $ gh repo create semcod/data2dsl --public --source . --remote origin
 https://github.com/semcod/data2dsl
+
+$ git push -u origin main
+main -> origin/main
+
+Publication verification:
+visibility=PUBLIC
+defaultBranch=main
+local=067b76b67802b17084c1209a5e96121dec5b8a2f
+remote=067b76b67802b17084c1209a5e96121dec5b8a2f
+governance=GOV-PASS (0 errors, 0 warnings)

project/ticket-001/changelog.md

@@ -12,3 +12,4 @@
   and container run.
 - Received explicit external-coordination authority and created the public
   `semcod/data2dsl` GitHub repository for Phase 0 publication.
+- Closed the completed Phase 0 ticket from the integrated default branch.

project/ticket-002/README.md

@@ -0,0 +1,36 @@
+# Ticket 002: Twin observation compatibility decision
+
+- **ID**: ticket-002
+- **Owner**: unresolved:human
+- **Status**: IN_PROGRESS
+- **Workflow state**: EDIT
+- **Created**: 2026-08-13
+
+## Goal and scope
+
+Produce one evidence-backed compatibility decision for using the current
+`subactor/twin` `Observation` and `EvidenceRef` contracts in `data2dsl`.
+Inspect the pinned code, protobuf schema, normative standard, reference profile
+and tests. The result must select exactly one verdict: `REUSE AS-IS`, `EXTEND`
+or `REJECT`, and state the consequences for `data2dsl`.
+
+Out of scope: product implementation, final data-query DSL, edits to
+`subactor/twin`, `wellmanifest/dsl` or any other repository, dependency
+changes, and external coordination.
+
+## Acceptance criteria
+
+- [ ] AC-01: The decision pins the inspected `subactor/twin` revision.
+- [ ] AC-02: Evidence covers protobuf, normative contract, validator code,
+  reference profile and relevant tests.
+- [ ] AC-03: Field-level fit and gaps for both `Observation` and `EvidenceRef`
+  are explicit.
+- [ ] AC-04: Exactly one verdict among `REUSE AS-IS`, `EXTEND`, and `REJECT` is
+  selected with rationale.
+- [ ] AC-05: Consequences and prohibited assumptions for `data2dsl` are stated.
+- [ ] AC-06: No other repository is modified and the governance gate passes.
+
+## Participants
+
+- Human participant: unresolved; no user-* file was created by this script.
+- Agent participant: [ai-codex.md](ai-codex.md)

project/ticket-002/ai-codex.md

@@ -0,0 +1,36 @@
+---
+participant-id: agent:codex
+participant: codex
+role: agent
+ticket: ticket-002
+---
+# Participant: codex (AI agent)
+
+## Understanding
+
+The user authorized only a compatibility decision. The target is the current
+`subactor/twin` contract at an immutable revision, not a proposal disguised as
+implemented behavior. The decision must distinguish structural protobuf fit
+from the stronger semantics actually enforced by the validator and tests.
+
+## Execution plan
+
+1. Pin the current clean `subactor/twin` revision.
+2. Inspect `Observation` and `EvidenceRef` in the protobuf contract.
+3. Trace their normative invariants through the standard, reference profile,
+   validator implementation and tests.
+4. Map the implemented contract to data2dsl requirements and select exactly
+   one verdict.
+5. Publish one decision document and run governance validation.
+
+## Actual changes
+
+- Initialized the bounded ticket and recorded SESSION_EXECUTION_AUTHORIZATION
+  from the request to execute this work.
+- No product or external-repository changes are authorized.
+
+## Blockers
+
+- None inside the recorded intent; proceed without a second confirmation.
+- New authority remains required for destructive action, secret access, new
+  external coordination, material objective expansion and trusted merge.

project/ticket-002/changelog.md

@@ -0,0 +1,7 @@
+# Ticket Changelog (ticket-002)
+
+## [0.1.0] - 2026-08-13
+
+- Initial governance scaffold created.
+- No human participant identity or content was generated.
+- Recorded the bounded compatibility-decision scope and explicit non-goals.