Repository navigation
Commit 0592ea7
authored
feat: add opt-in SQBRPYTHON rules for project Python (#1487)
* feat: list the SQBRPYTHON rules over Fensu's core Python rules
Depend on fensu-facts and fensu-native 0.30.0 without their Python
bindings and add the Python rule family to the catalogue: two SQLBuild
layout rules and 62 Fensu core rules in bands by Fensu family. The
family runs only when a selector names SQBRPYTHON or one of its codes,
so selecting SQBR does not turn on Python linting. Evaluation follows.
* feat: declare Fensu's thresholds as whole-number options on the Python rules
Add an integer built-in option kind with a minimum, shown by sqb rules
show and returned as RuleOption.integer by the Python catalogue, and
declare each Python rule's Fensu thresholds with Fensu's defaults.
* feat: put the project's Python sources into rules requests that select Python rules
When a selected rule reads Python, the rules request now carries every
supported project Python file with the digest of its bytes, plus the
interpreter version, so the memo, speculation and delta compiles see a
Python edit. Without a Python rule selected no Python file is read.
* build: record the rules crate's dependency on discovery in the lockfile
* feat: evaluate the SQBRPYTHON rules over project Python with Fensu's core rules
Each project Python file is placed the way Fensu places files: files under
python/ belong to the runtime package imported as python, with domains
directly below it; other supported files sit under their own folder and run
only the file-local rules. A unit's main.py is an entry and main module.
Fensu decides which file runs project-, scope- and package-wide rules, its
project questions are answered from one walk of the project, and node
functions and everything outside python/ count as loaded for reachability.
Findings keep Fensu's text, move to 1-based character columns, and rules for
the python/ package never report outside it. A file the parser rejects fails
the run. Python rules no longer make model rules read every model or change
the model cache key.
* feat: check what sits directly under python/ and how its packages are named
SQBRPYTHON001 reports modules directly under python/ and node-kind or role
folders at its root; SQBRPYTHON002 reports generic package names such as
common/ or shared/ anywhere under python/, except utils/ at its root. Each
folder is reported once, on its __init__.py or else its first file.
* feat: reuse each Python file's findings while its file, rules and project answers are unchanged
The Python rules keep their findings per file in python.json beside the
bulk rules cache. An entry holds while the file's bytes, its rules and
thresholds, the ruleset and the Python version are the same, and the
project questions its rules asked, such as whether an imported module
exists, still have the same answers. Rules that read the whole project
also key on every Python source. A speculative evaluation holds this
write with the bulk cache write until it is published.
* test: require a failure-corpus case for every discovery error code in unit tests
The compiler differential E2E only runs in the merge queue, so a discovery
code without a corpus case surfaced there first. This check reads the
declared cases without compiling them, so it fails on the pull request.
* test: require a golden for every failure-corpus case in unit tests
* perf: give the dead-code context only to the call that runs unreachable definitions
Each package file's context carried every loaded definition whenever a
project-wide Python rule ran, so evaluation grew with files times
definitions. A guard now bounds 4000 modules with 12000 findings.
* test: check the Python rules through compile and rules run on DuckShop fixtures
* docs: document the SQBRPYTHON rules and how SQBR leaves them out
* fix: select only named Python rules and key their cache on their own inputs
Naming one Python code next to SQBR no longer selects every default Python
rule. Layout rules that read folders now answer their cached queries from
project-relative paths, so adding a subdomain re-checks the package.
Python selectors and options no longer change the model-rule cache key,
and the Python cache key covers only the Python rules and their options.
* fix: treat audit factories and namespace packages as reachable in SQBRPYTHON306
Functions decorated with @audit_factory are loaded by SQLBuild, and
imports through python/ folders without __init__.py resolve, so neither
is reported as unreachable. The finding's remediation no longer points at
a setting SQLBuild does not have.
* fix: compare Python rule query paths with forward slashes so Windows caches see layout changes
Fensu spells planned query paths with /, while a Windows project directory
uses backslashes, so the paths stayed absolute and folder answers never
changed. A query for the project directory itself now uses the index's
root path.
* style: sort imports in the conforming Python rules fixture
* test: allow cases without an error code in the discovery-codes check
* refactor: name the iterator steps in the Python rule cache and namespace packages
* test: cover the dropped-stdout band with build help now that rules list outgrows it
rules list now prints more than CPython's 8 KiB stdout buffer, so it fails
its write instead of dropping the final flush. build --help stays between
4 and 8 KiB and does not grow with the rule catalogue.
* test: select the Python rules in the all-rules benchmark fixture
The dense and varied compile guards assert that their fixture selects
every built-in rule. The SQBRPYTHON rules are only selected by name, so
the fixture now names them, and the cold guard counts one rule-cache
miss per project Python file.
* test: guard unreachable-definition scaling by growth ratio instead of a near-limit budget
The 4000-module unreachable-definitions case ran at about 5.2 s against
its 5 s budget on the debug test profile, and at 2000 modules the code it
guards against fits well inside the budget. The case now times 500 and
2000 modules in five interleaved pairs and bounds the median ratio at
6.5: the current code measures 4.9 to 5.7 and the earlier per-file
context copy 7.4 to 8.7. The default-rules case keeps its absolute
budget, and both scaling tests take a shared lock so they never time
each other's work.
* test: check the growth guard's large run still reports every unreachable definition
Without a result check the ratio guard would pass if SQBRPYTHON306 stopped
walking this project, because both runs would get cheaper together. The
large run must report each unit's function and the shared helper.
* test: keep the delta compile project's rule selection without the Python rules
The dense fixture now selects the SQBRPYTHON rules by name. The delta
compile project reuses it, but its cases author minimal custom-rule and
factory modules that those rules reject, so the stored compile failed its
Rules and one edit no longer found the selection it rewrites. The delta
project now swaps back to its previous selection.
* test: drop the standalone comment from the delta series constants1 parent e38fc22 commit 0592ea7
103 files changed
Lines changed: 5042 additions & 90 deletions
File tree
- benchmarks/duckshop/edge_cases/python_rules
- conforming
- audits/generic
- hooks/python
- models
- python
- fulfillment
- shipment_export
- shipment_quality
- utils
- currency
- violations
- hooks/python
- python
- inventory
- common
- main
- restock
- crates
- sqlbuild-cli/src/native_commands/_helpers
- sqlbuild-discovery/src/python_paths
- main
- tests
- sqlbuild-pysource/src/definition_locations
- _helpers
- main
- tests
- sqlbuild-python/src/bindings/_helpers/rules
- sqlbuild-rules
- src
- engine
- _helpers
- tests
- python_rules
- _helpers
- main
- tests
- rules
- _helpers
- main
- scripts
- cold_compile_performance
- _helpers
- delta_compile_series
- main
- src/sqlbuild/.agents/skills/sqlbuild/references/docs
- concepts
- python-nodes
- rules
- tests
- e2e/src/sqlbuild/cli/commands/main/compile
- integration/src/sqlbuild/cli
- commands/main
- entry
- unit/scripts/compiler_differential/_helpers/corpus
- website/src
- content/docs/docs/concepts
- python-nodes
- rules
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
78 | 82 | | |
79 | 83 | | |
80 | 84 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
Whitespace-only changes.
Whitespace-only changes.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
0 commit comments