Skip to content

Commit 0592ea7

Browse files
authored
feat: add opt-in SQBRPYTHON rules for project Python (#1487)
* feat: list the SQBRPYTHON rules over Fensu's core Python rules Depend on fensu-facts and fensu-native 0.30.0 without their Python bindings and add the Python rule family to the catalogue: two SQLBuild layout rules and 62 Fensu core rules in bands by Fensu family. The family runs only when a selector names SQBRPYTHON or one of its codes, so selecting SQBR does not turn on Python linting. Evaluation follows. * feat: declare Fensu's thresholds as whole-number options on the Python rules Add an integer built-in option kind with a minimum, shown by sqb rules show and returned as RuleOption.integer by the Python catalogue, and declare each Python rule's Fensu thresholds with Fensu's defaults. * feat: put the project's Python sources into rules requests that select Python rules When a selected rule reads Python, the rules request now carries every supported project Python file with the digest of its bytes, plus the interpreter version, so the memo, speculation and delta compiles see a Python edit. Without a Python rule selected no Python file is read. * build: record the rules crate's dependency on discovery in the lockfile * feat: evaluate the SQBRPYTHON rules over project Python with Fensu's core rules Each project Python file is placed the way Fensu places files: files under python/ belong to the runtime package imported as python, with domains directly below it; other supported files sit under their own folder and run only the file-local rules. A unit's main.py is an entry and main module. Fensu decides which file runs project-, scope- and package-wide rules, its project questions are answered from one walk of the project, and node functions and everything outside python/ count as loaded for reachability. Findings keep Fensu's text, move to 1-based character columns, and rules for the python/ package never report outside it. A file the parser rejects fails the run. Python rules no longer make model rules read every model or change the model cache key. * feat: check what sits directly under python/ and how its packages are named SQBRPYTHON001 reports modules directly under python/ and node-kind or role folders at its root; SQBRPYTHON002 reports generic package names such as common/ or shared/ anywhere under python/, except utils/ at its root. Each folder is reported once, on its __init__.py or else its first file. * feat: reuse each Python file's findings while its file, rules and project answers are unchanged The Python rules keep their findings per file in python.json beside the bulk rules cache. An entry holds while the file's bytes, its rules and thresholds, the ruleset and the Python version are the same, and the project questions its rules asked, such as whether an imported module exists, still have the same answers. Rules that read the whole project also key on every Python source. A speculative evaluation holds this write with the bulk cache write until it is published. * test: require a failure-corpus case for every discovery error code in unit tests The compiler differential E2E only runs in the merge queue, so a discovery code without a corpus case surfaced there first. This check reads the declared cases without compiling them, so it fails on the pull request. * test: require a golden for every failure-corpus case in unit tests * perf: give the dead-code context only to the call that runs unreachable definitions Each package file's context carried every loaded definition whenever a project-wide Python rule ran, so evaluation grew with files times definitions. A guard now bounds 4000 modules with 12000 findings. * test: check the Python rules through compile and rules run on DuckShop fixtures * docs: document the SQBRPYTHON rules and how SQBR leaves them out * fix: select only named Python rules and key their cache on their own inputs Naming one Python code next to SQBR no longer selects every default Python rule. Layout rules that read folders now answer their cached queries from project-relative paths, so adding a subdomain re-checks the package. Python selectors and options no longer change the model-rule cache key, and the Python cache key covers only the Python rules and their options. * fix: treat audit factories and namespace packages as reachable in SQBRPYTHON306 Functions decorated with @audit_factory are loaded by SQLBuild, and imports through python/ folders without __init__.py resolve, so neither is reported as unreachable. The finding's remediation no longer points at a setting SQLBuild does not have. * fix: compare Python rule query paths with forward slashes so Windows caches see layout changes Fensu spells planned query paths with /, while a Windows project directory uses backslashes, so the paths stayed absolute and folder answers never changed. A query for the project directory itself now uses the index's root path. * style: sort imports in the conforming Python rules fixture * test: allow cases without an error code in the discovery-codes check * refactor: name the iterator steps in the Python rule cache and namespace packages * test: cover the dropped-stdout band with build help now that rules list outgrows it rules list now prints more than CPython's 8 KiB stdout buffer, so it fails its write instead of dropping the final flush. build --help stays between 4 and 8 KiB and does not grow with the rule catalogue. * test: select the Python rules in the all-rules benchmark fixture The dense and varied compile guards assert that their fixture selects every built-in rule. The SQBRPYTHON rules are only selected by name, so the fixture now names them, and the cold guard counts one rule-cache miss per project Python file. * test: guard unreachable-definition scaling by growth ratio instead of a near-limit budget The 4000-module unreachable-definitions case ran at about 5.2 s against its 5 s budget on the debug test profile, and at 2000 modules the code it guards against fits well inside the budget. The case now times 500 and 2000 modules in five interleaved pairs and bounds the median ratio at 6.5: the current code measures 4.9 to 5.7 and the earlier per-file context copy 7.4 to 8.7. The default-rules case keeps its absolute budget, and both scaling tests take a shared lock so they never time each other's work. * test: check the growth guard's large run still reports every unreachable definition Without a result check the ratio guard would pass if SQBRPYTHON306 stopped walking this project, because both runs would get cheaper together. The large run must report each unit's function and the shared helper. * test: keep the delta compile project's rule selection without the Python rules The dense fixture now selects the SQBRPYTHON rules by name. The delta compile project reuses it, but its cases author minimal custom-rule and factory modules that those rules reject, so the stored compile failed its Rules and one edit no longer found the selection it rewrites. The delta project now swaps back to its previous selection. * test: drop the standalone comment from the delta series constants
1 parent e38fc22 commit 0592ea7

103 files changed

Lines changed: 5042 additions & 90 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Cargo.lock‎

Lines changed: 47 additions & 5 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,10 @@ axum-server = { version = "0.8", default-features = false, features = ["tls-rust
7575
# Compresses the server's larger JSON reads and UI assets for browsers.
7676
tower-http = { version = "0.6", default-features = false, features = ["compression-gzip"] }
7777
fensu-policy = "=0.22.0"
78+
# Fensu's Python facts and core rules without their Python bindings: the default `python`
79+
# feature adds PyO3, rusqlite, flate2 and sha2.
80+
fensu-facts = { version = "=0.30.0", default-features = false }
81+
fensu-native = { version = "=0.30.0", default-features = false }
7882
futures-util = { version = "0.3", default-features = false, features = ["std"] }
7983
# `sqb` names compute log captures with random ids, as `uuid.uuid4().hex`.
8084
getrandom = "0.4"
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
AUDIT ();
2+
3+
SELECT *
4+
FROM @relation
5+
WHERE NOT (@expression)
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
"""Note each finished build."""
2+
3+
4+
def note_build(ctx: object) -> None:
5+
"""Note one finished build."""
6+
print(ctx)
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
MODEL (description "Shipped orders.", audit_factories [shipment_quality]);
2+
SELECT 1 AS shipment_id

‎benchmarks/duckshop/edge_cases/python_rules/conforming/python/__init__.py‎

Whitespace-only changes.

‎benchmarks/duckshop/edge_cases/python_rules/conforming/python/fulfillment/__init__.py‎

Whitespace-only changes.

‎benchmarks/duckshop/edge_cases/python_rules/conforming/python/fulfillment/shipment_export/__init__.py‎

Whitespace-only changes.
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
"""Write each shipped order as one carrier line."""
2+
3+
from python.utils.currency import amounts
4+
5+
6+
def export_shipments(ctx: object) -> None:
7+
"""Write each shipped order as one carrier line."""
8+
label: str = "Expédition"
9+
print(label, amounts.cents_text(1250), ctx)
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
"""Export shipped orders to the carrier."""
2+
3+
from python.fulfillment.shipment_export.main import export_shipments
4+
from sqlbuild.tasks import task
5+
6+
7+
@task
8+
def shipment_export(ctx: object) -> None:
9+
"""Export shipped orders to the carrier."""
10+
export_shipments(ctx)

0 commit comments

Comments
 (0)