Repository navigation
Governance: require Autotest cases for integration tests - #8274
Open
Critsium-xy wants to merge 2 commits into
Open
Critsium-xy wants to merge 2 commits into
Critsium-xy wants to merge 2 commits into
Conversation
Integration tests are Autotest case directories with result.ref, listed in CASES_*.txt. Recent PRs instead register standalone Python drivers with add_test() that launch ABACUS, rewrite fixtures at run time, and assert on output themselves. Add AGENTS.md rule 22 and an "Integration Test Drivers" section that keep Autotest as the default, allow a standalone driver only for checks Autotest cannot express, and set placement, fixture, optional-interpreter, and MPI requirements for those drivers. Add a matching human-confirmation row to the rule grading matrix; no mechanical check is introduced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reminder
AGENTS.mdanddocs/developers_guide/agent_governance.md.source/changes.Linked Issue
No linked issue. This is a documentation-only governance change, prompted by a pattern seen in recent PRs (see below).
Unit Tests and/or Case Tests for my changes
git diff --cached --checkpython tools/03_code_analysis/agent_governance_check.py --stagedgit ls-files --eol).What's changed?
Integration tests in ABACUS are Autotest cases: a case directory with
INPUT/STRU/KPTandresult.ref, listed inCASES_*.txtand run byAutotest.sh. Several recent PRs (#8053, #8069, and the open #8235 and #8242) instead register standalone Python scripts withadd_test(). These scripts launch ABACUS, build or rewrite inputs at run time, and assert on the output themselves. Each PR has invented its own launcher, environment, and fixture handling. Examples:INPUTby exact-whitespace string replacement.find_package(Python3 ... REQUIRED), which makes Python a hard configure-time dependency for MPI test builds.MPIEXEC_PREFLAGS/POSTFLAGSand thePROCESSORSproperty.The written rules said nothing about this, so reviewers had no basis to push back.
This PR:
add_test()driver is allowed only for checks Autotest cannot express, such as an expected nonzero exit or per-rank stderr.CompareFile.py) are unaffected.agent_governance.mdwith the requirements for a justified driver:tests/integrate/validation_tools/and is namedcheck_*.py.INPUT.find_programplusif(), neverREQUIRED.PROCESSORS.OMP_NUM_THREADS=1, uses a temporary scratch directory, and sets a CTestTIMEOUT.Governance Notes
AGENTS.md,docs/developers_guide/agent_governance.md). No INPUT or user-facing behavior change.source/changes.🤖 Generated with Claude Code