Ensure documentation regeneration does not introduce new uncommitted schema files as side effects of bug fixes #3100

What happened

PR #3096 fixed two bugs in src/utils/json_schema_updater.py and then regenerated documentation schemas. The regeneration produced a new file docs/models/common_responses.json (2111 lines) that does not exist on the main branch (directory listing has no common_responses.json). Combined with modifications to 6 existing schema files, the PR ballooned to 6020 additions / 3172 deletions — despite the actual code change being ~14 lines in one utility function plus ~96 lines of tests.

The PR was closed without merging by a human maintainer with no explanation. While the reason is unknown, the unexpectedly large diff from regenerated artifacts — particularly an entirely new file — may have contributed to the rejection.

What could go better

When a bug-fix PR regenerates documentation and the regeneration produces new files that didn't exist before, it signals that documentation generation has drifted from the committed state. The new common_responses.json file likely appeared because: (a) the documentation generation script was recently updated to produce this file, but regeneration hadn't been run since; or (b) the recursive_update() fix caused previously-dropped schema types to now appear, triggering a new output file.

Either way, the drift accumulated across multiple PRs and was then dumped into a single bug-fix PR, making it appear much larger and riskier than it actually was. This pattern makes agent PRs harder to review and more likely to be rejected.

Confidence: Medium. I can confirm the new file exists in the PR diff and is absent from main. I am speculating that this contributed to the PR's closure — the human left no explanation.

Proposed change

Add a CI check or Makefile target that verifies documentation schemas are up-to-date on every PR (not just PRs that modify models). This could be a lightweight step in CI that runs the schema generation command and fails if the output differs from the committed files — similar to how uv run make verify catches formatting drift.

For example, add a check-docs target to the Makefile:

check-docs:
	$(PYTHON) -m scripts.generate_documentation --check

Or add a CI step that runs uv run make generate-documentation and checks for uncommitted changes via git diff --exit-code docs/.

This ensures that documentation regeneration is never deferred across multiple PRs, preventing large accumulated diffs from appearing in unrelated bug-fix PRs.

Validation criteria

After implementing the CI check: (1) any PR that introduces model changes without regenerating documentation should fail CI, (2) the docs/models/common_responses.json file should be present on the main branch once the generation command is re-run, and (3) future bug-fix PRs that touch json_schema_updater.py or model files should only show incremental changes in documentation schemas, not accumulations of unrelated drift.

Related issues checked

Searched: documentation regeneration schema generate-documentation; documentation drift generated artifact bloat diff
No overlapping issues found.

Generated by retro agent from #3096