Skip to content

feat(cli): add deterministic minimality and coding-style patch reviews - #628

Open
tdullien1 wants to merge 13 commits into
mainfrom
codex/deterministic-core-patch-reviews
Open

feat(cli): add deterministic minimality and coding-style patch reviews#628
tdullien1 wants to merge 13 commits into
mainfrom
codex/deterministic-core-patch-reviews

Conversation

@tdullien1

@tdullien1 tdullien1 commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add opt-in, programmatically enforced review stages for generated security patches so minimality and local coding style are independently evaluated before a patch or draft pull request is accepted.

Problem description

There are two primary impacts from this:

  1. Adherence to the instructions to keep patches small, avoid refactoring, keep local style was often low. In testing, only 57% of runs actually followed all the prescribed steps, with noticeable impact on patch quality.
  2. Resulting patches were often too large, did spurious refactoring, or over-ambitious (re-designing code with significant externally visible effects, moving large amounts of code around for "testability" etc.).

Impact of PR

  • Add separate opt-in --review-minimality and --review-style flags to both patching entry points.
  • Either of these flags trigger deterministically enforced workflow where each stage is forcibly executed.
  • A further command line flag --max-review-revisions determines how often a stage should be retried.

Enabling these two steps reduces median patch/PR size by 50%+ in my experiments, and greatly reduces my own annoyance at overbroad "slop" patches that are difficult to review and test. The generated patches are more expensive (impact on token use still has to be quantified); the current PR keeps the original workflow intact, so the cost increase is optional.

A follow-on PR will add @soyeon-oai's risk analysis skill into this, including into the deterministic workflow.

Individual example

On a specific issue, CARCOD-8726, the initial patch author prompt creates a patch with 281 lines of production-code change (not counting tests etc.) due to an overly ambitious refactoring attempt and writing test harnesses to compensate for missing dependencies. Iterative reviews bring this down to 27 lines of production code changed. Manual review / human-expert curated change is 17 lines of code, so these changes reduce the fix size from approximately 10x human expert to approximately 2x human expert (on a single change).

-- End of human-authored PR description --

Changes

  • Add separate opt-in --review-minimality and --review-style flags to both patching entry points.
  • Run each selected reviewer in a separate read-only session, validate compact structured verdicts, and fail closed on blocked, malformed, or rejected reviews.
  • Add --max-review-revisions for bounded author revisions and restart earlier reviews after later-stage changes.
  • Share a behavior-preserving, minimal-change policy and prior reviewer decisions across authors, revisions, and reviewers.
  • Reconcile alternating reviewer disagreements once using a separate read-only decision stage.
  • Keep broader redesign suggestions in PR comments or patch summaries rather than generated code.
  • Cover reviewer ordering, revision limits, malformed verdicts, conflicting decisions, publication safety, and both patching entry points.
  • Risk assessment is intentionally excluded and proposed separately.

Testing

  • bun test --timeout 30000 tests-ts/cli-skills.test.ts tests-ts/cli-patch.test.ts — 66 tests passed.
  • node scripts/generate-models.cjs --check — passed.
  • node node_modules/typescript/bin/tsc --noEmit — passed.
  • node node_modules/prettier/bin/prettier.cjs --check --ignore-path .gitignore --ignore-path .prettierignore '**/*.{cjs,mjs,js,ts,tsx,json,md}' — passed.
  • A broader cli.test.ts run encounters existing bulk-scan fixture failures that reproduce unchanged on the original branch.

Risk and rollout

Both review stages are opt-in and disabled by default. Reviewers run read-only and cannot apply or merge patches. Explicit revision budgets bound corrective loops, and malformed, blocked, or inconsistent verdicts stop patch publication. No risk-assessment bundle, schema, or external artifact contract is included.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@tdullien1 tdullien1 changed the title Add deterministic minimality and coding-style patch reviews feat(cli): add deterministic minimality and coding-style patch reviews Aug 24, 2026
@github-actions github-actions Bot added the enhancement New feature or request label Aug 24, 2026
@tdullien1
tdullien1 marked this pull request as ready for review August 24, 2026 12:21
@tdullien1
tdullien1 requested a review from ianw-oai August 24, 2026 12:21
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review Completed 2026-08-24T12:26:08.558976Z a75ec06 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants