Skip to content

US-14: Evidence complete needs an issue, a merged pull request with passing checks, and a release - #28

Merged
TahubCS merged 5 commits into
mainfrom
feature/us-14-evidence-policy
Nov 10, 2026
Merged

TahubCS merged 5 commits into
mainfrom
feature/us-14-evidence-policy

Conversation

@TahubCS

@TahubCS TahubCS commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

Problem

Jira SPEC-22 (US-14). The rule behind "Evidence complete" (ADR-053) only looked at linked code and its checks; issues and releases neither counted nor blocked. That rule was written during development and was never agreed by the team. The team's rule is the one in US-14:

  • the default policy requires a linked issue, a merged pull request, passing checks on its final commit, and a recorded release;
  • a requirement that is missing any part shows which part is missing;
  • "Evidence complete" is shown separately from "Accepted".

What changed

  • API (Requirements/EvidencePolicy.cs, Requirements/RequirementEndpoints.cs): EvidencePolicy.Missing returns the parts a requirement does not meet, in order: issue, merged_pull_request, passing_checks, release. The requirement list and every single-requirement response (get, create, edit, archive, restore) now carry evidenceComplete and evidenceMissing.
  • Rule details:
    • an issue counts in any state;
    • every merged pull request needs readable checks on its final commit, with at least one passed and none failed, running, or cancelled (skipped and neutral are fine);
    • every merged pull request must be in the history of a linked release, as GitHub confirmed (ADR-037), so an unrelated release does not count;
    • linked commits and unmerged pull requests neither count nor block.
  • Web:
    • the requirement list row shows "Evidence complete" or "Missing issue, release, …"; on phones the text wraps instead of being cut off;
    • the requirement page's Evidence section shows an "Evidence complete" / "Evidence incomplete" badge and lists each of the four parts as Met or Missing, apart from the Review section;
    • new component components/evidence-status.tsx; parsing in lib/projects.ts and lib/requirements.ts.
  • Docs: ADR-057 records the team's rule and replaces ADR-053's rule (the milestone part of ADR-053 stands), with notes on ADR-038 and ADR-053. PROJECT, API, ARCHITECTURE, TESTING, and HANDOFF are updated.

What did not change

  • No migration and no new environment variable; the status is computed on each read, not stored.
  • Acceptance is untouched: only a reviewer's decision makes a requirement Accepted.
  • The rule is still fixed. A per-project configurable policy would be a later decision.

How it was tested

All tests are Playwright Test, as the repo requires (ADR-005, docs/TESTING.md):

  • API over HTTP, against a real Postgres (tests/schema/milestones.spec.ts):
    • 22 cases of the rule (no evidence, issue only, open pull request, commits only, unreadable, empty, skipped-only, failed, running, cancelled, and neutral checks, no release, a release without the pull request or not yet compared, several merged pull requests);
    • each case is checked on the list and on the single requirement;
    • plus the create, edit, archive, and restore responses.
  • Web parsing (tests/api/project-data.spec.ts):
    • the documented shape and reordering into rule order;
    • refusing unknown, duplicate, or inconsistent values;
    • the previous API's response during a deploy.
  • Browser (tests/e2e/milestones.spec.ts):
    • the row text and the counts;
    • the Met/Missing list for a complete and an incomplete requirement;
    • no status shown when the previous API sends none;
    • 390px width on the list and the requirement page, with the missing parts not cut off.
  • Mutation check of every changed line: 32 deliberate breaks, one at a time — 17 API, 8 web parsing, 7 UI and layout. The first pass found three that no test caught: the edit response, an older API's malformed value, and cut-off text on phones. Tests were added for each, and all 32 now make a test fail.
  • Full checks on this PR's contents (after merging main with PR feat: suggest links from closing references and move links between re… #26 and ci: improve pipeline feedback and deploy health #27):
    • npm test: 409 passed;
    • npm run lint, npm run typecheck, dotnet format app/api --verify-no-changes --no-restore, npm run build:api (0 warnings), and git diff --check pass.
  • SDK note: checks were run locally with .NET SDK 10.0.114, the only one installable in that environment. global.json was pointed at it only while checking and is not changed in this PR, so CI uses the pinned 10.0.401.

Screenshots

The browser test saves milestone-filter.png (list with "Evidence complete" and "Missing issue, merged pull request, passing checks, release") and evidence-status.png (requirement page with the four parts as Missing) in its test output.

Migrations or environment changes

None.

Risks, limitations, follow-up

  • Counts fall after deploy: requirements that were Evidence complete under the old rule (commits only, or no issue or release) no longer are, and list counts fall until those links are added.
  • Merged later needs a refresh: a pull request merged after it was linked counts once the evidence is refreshed, because its state and merge commit are a snapshot.
  • During a deploy: the web may read the previous API; it then shows no evidence status rather than the old rule's claim.
  • Not exercised: the real GitHub.
  • Follow-up: after merge, move SPEC-22 to Done in Jira.

…d a release for Evidence complete

Replace the Evidence complete rule with the team's rule from US-14 (ADR-056).
A requirement is Evidence complete only when an issue is linked, a merged
pull request is linked, the checks on the final commit of every merged pull
request passed, and every merged pull request is included in a linked
release. The API returns the missing parts, and the requirement list and
page show which parts are met or missing, separately from Accepted.
@vercel

vercel Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
web Ready Ready Preview Nov 9, 2026 2:48pm UTC

@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-09T14:47:38.060658Z
We couldn’t complete this review. Comment @codex review to try again.
254cd22 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b20470ab-9568-4037-b42a-1315972cde87
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@TahubCS
TahubCS merged commit 72a67bf into main Nov 10, 2026
6 checks passed

This branch was successfully deployed

1 active deployment
Preview — 9efc0b7f Deployed Nov 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant