Conversation
Add a "Test Categories" section to the write-tests skill explaining the test-developer-* vs test-independent-* file-naming convention: - test-independent-* validate outputs against an independent source of truth (a separate reference implementation, closed-form value, or identity). - test-developer-* exercise the package's own implementation behavior (defaults, S3 attributes, internal helpers, feature-survives-transform). This convention was previously implicit; documenting it so new tests land in the right file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jdblischak
left a comment
There was a problem hiding this comment.
@yihui thanks for opening this Issue. I agree that these testing conventions should be clarified and documented, since I am not 100% sure how this is supposed to work.
Here is a what I think I know. As part of qualifying analytical software to be used in regulatory submissions, there can be a "double programming" step where an independent developer writes validation tests. In other words, the independent tester writes their own code to implement a particular method, and then tests to confirm that the package software returns the same result. These tests are in the test-independent-*.R files.
Here is the part I don't know. Can we as the developers of the software ever write "independent" tests? What if we write a test that uses a mathematical identity? Should these go in "independent", or should by definition any test written by a developer go in test-developer-*.R (since we can't be independent of ourselves, and any misunderstanding we have when writing a function will be carried over into our testing of the method, even if we use an orthogonal computation technique).
|
@jdblischak Thanks for sharing your knowledge about these tests! I guess @LittleBeannie knows more than us and can help clarify. |
Adds a Test Categories section to the
write-testsskill documenting thetest-developer-*vstest-independent-*file-naming convention, which was previously implicit.test-independent-*— validate outputs against an independent source of truth: a separate reference implementation (e.g.gsDesign::gsSurv(),gsDesign::toInteger()), a closed-form/hand-computed value, or a mathematical identity. Should not rely on gsDesign2 internals.test-developer-*— exercise the package's own implementation behavior: default arguments, S3 class/attribute plumbing (e.g.uninteger_is_from), internal helpers (gsDesign2:::), consistency between exported and internal variants (e.g.gs_power_ahr()vsgs_power_ahr_()), and "does this feature survive this transformation?" checks.Motivated by the review on #673, where a
to_integer()harm-bound test was initially placed intest-independent-*but belonged intest-developer-*. Documenting the rule so future tests land in the right file.Docs-only; no code or test changes.
🤖 Generated with Claude Code