Skip to content

Commit b6e4517

Browse files
authored
feat: add SQBRPYTHON node unit rules for Python nodes under python/ (#1490)
* feat: add SQBRPYTHON node unit rules for Python nodes under python/ SQBRPYTHON003 to SQBRPYTHON012 check that each node lives in its own unit folder under its domain: node.py declares one node or factory that only returns main(...) with keyword arguments, main.py holds one main() with a result annotation that fits the node, checks.py holds the unit's checks, imports stay within the unit's boundaries, factories loop over the unit's specs, nodes use description= instead of docstrings, and nodes declared outside a unit's node.py are reported. The rules are on by default within SQBRPYTHON, so projects that select SQBRPYTHON may get new layout findings; selecting SQBR alone never runs them. * fix: tighten SQBRPYTHON unit detection, import boundaries and factory loops Role folders and shallow node.py files no longer make units. Factory loops record the name they read through calls and attributes, import-only TYPE_CHECKING blocks count as imports and are boundary-checked, dotted imports of other units' nodes may be named in depends_on, and an unused import is no longer treated as depends_on-only. main bound to a module no longer counts as delegating. The unit project indexes files by path and parses them in parallel. A scaling guard covers a chain of linked units with dense findings. The Python rule helpers are grouped into engine/ and units/ buckets. * fix: accept SQBRPYTHON factory loops that wrap specs in a call A factory loop now records every name its iterable loads, so loops over enumerate(SPECS) or sorted(SPECS) read the spec instead of the builtin.
1 parent 3044653 commit b6e4517

53 files changed

Lines changed: 3159 additions & 59 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
"""Write each shipped order as one carrier line."""
22

33
from python.utils.currency import amounts
4+
from sqlbuild.executor.python_nodes.models import PythonNodeResult
5+
from sqlbuild.tasks import TaskContext
46

57

6-
def export_shipments(ctx: object) -> None:
8+
def main(ctx: TaskContext) -> PythonNodeResult:
79
"""Write each shipped order as one carrier line."""
810
label: str = "Expédition"
9-
print(label, amounts.cents_text(1250), ctx)
11+
return ctx.result(payload=f"{label} {amounts.cents_text(1250)}")
Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
"""Export shipped orders to the carrier."""
22

3-
from python.fulfillment.shipment_export.main import export_shipments
4-
from sqlbuild.tasks import task
3+
from python.fulfillment.shipment_export.main import main
4+
from sqlbuild.executor.python_nodes.models import PythonNodeResult
5+
from sqlbuild.tasks import TaskContext, task
56

67

7-
@task
8-
def shipment_export(ctx: object) -> None:
9-
"""Export shipped orders to the carrier."""
10-
export_shipments(ctx)
8+
@task(description="Export shipped orders to the carrier.")
9+
def shipment_export(ctx: TaskContext) -> PythonNodeResult:
10+
return main(ctx=ctx)

‎benchmarks/duckshop/edge_cases/python_rules/expected_findings.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,17 @@
77
["SQBRPYTHON306", "python/inventory/common/stock.py", 4, 1],
88
["SQBRPYTHON305", "python/inventory/main/publish.py", 1, 1],
99
["SQBRPYTHON306", "python/inventory/main/publish.py", 4, 1],
10+
["SQBRPYTHON614", "python/inventory/replenishment.py", 1, 1],
11+
["SQBRPYTHON012", "python/inventory/replenishment.py", 7, 5],
1012
["SQBRPYTHON617", "python/inventory/restock/main.py", 1, 1],
1113
["SQBRPYTHON306", "python/inventory/restock/main.py", 4, 1],
1214
["SQBRPYTHON306", "python/inventory/restock/main.py", 9, 1],
15+
["SQBRPYTHON003", "python/inventory/stock_count/node.py", 1, 1],
16+
["SQBRPYTHON004", "python/inventory/stock_count/node.py", 5, 1],
17+
["SQBRPYTHON605", "python/inventory/stock_count/node.py", 5, 1],
18+
["SQBRPYTHON005", "python/inventory/stock_count/node.py", 9, 5],
19+
["SQBRPYTHON007", "python/inventory/stock_count/node.py", 9, 5],
20+
["SQBRPYTHON011", "python/inventory/stock_count/node.py", 9, 5],
1321
["SQBRPYTHON001", "python/report.py", 1, 1],
1422
["SQBRPYTHON614", "python/report.py", 1, 1]
1523
]
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
"""Replenishment declared in a flat module."""
2+
3+
from sqlbuild.tasks import TaskContext, task
4+
5+
6+
@task(description="Reorder products that ran out.")
7+
def replenish(ctx: TaskContext) -> None:
8+
return None

‎benchmarks/duckshop/edge_cases/python_rules/violations/python/inventory/stock_count/__init__.py‎

Whitespace-only changes.
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
"""Count stock in every warehouse."""
2+
3+
from sqlbuild.tasks import TaskContext, task
4+
5+
WAREHOUSES: tuple[str, ...] = ("north", "south")
6+
7+
8+
@task
9+
def stock_count(context: TaskContext) -> int:
10+
"""Count stock in every warehouse."""
11+
return len(WAREHOUSES)

‎crates/sqlbuild-pysource/src/lib.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ pub mod import_rewrites;
88
pub mod literal_sql;
99
pub mod macro_modules;
1010
pub mod module_parse;
11+
pub mod node_units;
1112
pub mod orchestration_modules;
1213
pub mod rule_evidence;
1314
pub mod string_literals;
Lines changed: 215 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,215 @@
1+
//! A function's decorators, signature and body shape, and the decorated functions inside it.
2+
3+
use ruff_python_ast::visitor::{Visitor, walk_expr};
4+
use ruff_python_ast::{Decorator, Expr, Parameter, Stmt, StmtFunctionDef};
5+
use ruff_python_parser::{Mode, ParseOptions};
6+
use ruff_text_size::Ranged;
7+
8+
use crate::cpython_tree::main::walk_breadth_first::walk_breadth_first;
9+
use crate::cpython_tree::models::CpythonNode;
10+
use crate::node_units::_helpers::positions::{UnitText, unit_position};
11+
use crate::node_units::constants::{
12+
DEPENDS_ON_KEYWORD, MAIN_FUNCTION, NONE_NAME, OPTIONAL_NAME, UNION_NAMES,
13+
};
14+
use crate::node_units::models::{UnitDecorator, UnitFunction, UnitParameter};
15+
16+
/// The shape of `function` and of every decorated function inside it.
17+
pub(crate) fn unit_function(function: &StmtFunctionDef, source: &UnitText<'_>) -> UnitFunction {
18+
let parameters = &function.parameters;
19+
let body: Vec<CpythonNode<'_>> =
20+
walk_breadth_first(function.body.iter().map(CpythonNode::Stmt).collect());
21+
UnitFunction {
22+
name: function.name.to_string(),
23+
position: unit_position(source, function.name.range().start()),
24+
decorators: function
25+
.decorator_list
26+
.iter()
27+
.filter_map(decorator)
28+
.collect(),
29+
positional: parameters
30+
.posonlyargs
31+
.iter()
32+
.chain(&parameters.args)
33+
.map(|with_default| parameter(&with_default.parameter))
34+
.collect(),
35+
variadic: parameters.vararg.is_some(),
36+
keyword_only: parameters
37+
.kwonlyargs
38+
.iter()
39+
.map(|with_default| parameter(&with_default.parameter))
40+
.collect(),
41+
keywords_variadic: parameters.kwarg.is_some(),
42+
returns: function.returns.as_deref().map(union_names),
43+
docstring: leading_docstring(&function.body),
44+
returns_directly: matches!(without_docstring(&function.body), [Stmt::Return(_)]),
45+
delegation: delegation(&function.body),
46+
inner: body
47+
.iter()
48+
.filter_map(|node| match node {
49+
CpythonNode::Stmt(Stmt::FunctionDef(inner)) if !inner.decorator_list.is_empty() => {
50+
Some(unit_function(inner, source))
51+
}
52+
_ => None,
53+
})
54+
.collect(),
55+
loops: body
56+
.iter()
57+
.filter_map(|node| match node {
58+
CpythonNode::Stmt(Stmt::For(loop_statement)) => {
59+
Some(loaded_names(&loop_statement.iter))
60+
}
61+
_ => None,
62+
})
63+
.collect(),
64+
}
65+
}
66+
67+
/// Whether `body` opens with a string expression statement.
68+
pub(crate) fn leading_docstring(body: &[Stmt]) -> bool {
69+
matches!(body.first(), Some(Stmt::Expr(statement)) if statement.value.is_string_literal_expr())
70+
}
71+
72+
/// `body` without its leading docstring.
73+
fn without_docstring(body: &[Stmt]) -> &[Stmt] {
74+
if leading_docstring(body) {
75+
&body[1..]
76+
} else {
77+
body
78+
}
79+
}
80+
81+
/// For a body that is exactly `return main(...)` after any docstring, whether every argument is a named keyword.
82+
fn delegation(body: &[Stmt]) -> Option<bool> {
83+
let statements: &[Stmt] = if leading_docstring(body) {
84+
&body[1..]
85+
} else {
86+
body
87+
};
88+
let [Stmt::Return(returned)] = statements else {
89+
return None;
90+
};
91+
let Some(Expr::Call(call)) = returned.value.as_deref() else {
92+
return None;
93+
};
94+
let Expr::Name(function) = call.func.as_ref() else {
95+
return None;
96+
};
97+
(function.id.as_str() == MAIN_FUNCTION).then(|| {
98+
call.arguments.args.is_empty()
99+
&& call
100+
.arguments
101+
.keywords
102+
.iter()
103+
.all(|keyword| keyword.arg.is_some())
104+
})
105+
}
106+
107+
/// A decorator by its last name, with its keywords and the names its `depends_on` value reads.
108+
fn decorator(decorator: &Decorator) -> Option<UnitDecorator> {
109+
let (target, call) = match &decorator.expression {
110+
Expr::Call(call) => (call.func.as_ref(), Some(call)),
111+
other => (other, None),
112+
};
113+
let name: String = last_name(target)?;
114+
let keywords = call
115+
.map(|call| call.arguments.keywords.as_ref())
116+
.unwrap_or_default();
117+
Some(UnitDecorator {
118+
name,
119+
keywords: keywords
120+
.iter()
121+
.filter_map(|keyword| keyword.arg.as_ref().map(ToString::to_string))
122+
.collect(),
123+
depends_on: keywords
124+
.iter()
125+
.filter(|keyword| {
126+
keyword
127+
.arg
128+
.as_ref()
129+
.is_some_and(|arg| arg.as_str() == DEPENDS_ON_KEYWORD)
130+
})
131+
.flat_map(|keyword| walk_breadth_first(vec![CpythonNode::Expr(&keyword.value)]))
132+
.filter_map(|node| match node {
133+
CpythonNode::Expr(Expr::Name(name)) => Some(name.id.to_string()),
134+
_ => None,
135+
})
136+
.collect(),
137+
})
138+
}
139+
140+
fn parameter(parameter: &Parameter) -> UnitParameter {
141+
UnitParameter {
142+
name: parameter.name.to_string(),
143+
annotation: parameter.annotation.as_deref().and_then(last_name),
144+
}
145+
}
146+
147+
/// The last name of each member of an annotation written as `A | B`, `Optional[A]` or `Union[A, B]`.
148+
fn union_names(annotation: &Expr) -> Vec<String> {
149+
match annotation {
150+
Expr::BinOp(union) if union.op.is_bit_or() => {
151+
let mut names: Vec<String> = union_names(&union.left);
152+
names.extend(union_names(&union.right));
153+
names
154+
}
155+
Expr::NoneLiteral(_) => vec![NONE_NAME.to_owned()],
156+
Expr::StringLiteral(literal) => string_expression(literal.value.to_str())
157+
.map_or_else(Vec::new, |expression| union_names(&expression)),
158+
Expr::Subscript(subscript)
159+
if last_name(&subscript.value)
160+
.is_some_and(|name| UNION_NAMES.contains(&name.as_str())) =>
161+
{
162+
let mut names: Vec<String> = match subscript.slice.as_ref() {
163+
Expr::Tuple(members) => members.elts.iter().flat_map(union_names).collect(),
164+
member => union_names(member),
165+
};
166+
if last_name(&subscript.value).as_deref() == Some(OPTIONAL_NAME) {
167+
names.push(NONE_NAME.to_owned());
168+
}
169+
names
170+
}
171+
other => last_name(other).into_iter().collect(),
172+
}
173+
}
174+
175+
/// The last name of a name, an attribute chain, or the subscripted base of either.
176+
fn last_name(expression: &Expr) -> Option<String> {
177+
match expression {
178+
Expr::Name(name) => Some(name.id.to_string()),
179+
Expr::Attribute(attribute) => Some(attribute.attr.to_string()),
180+
Expr::Subscript(subscript) => last_name(&subscript.value),
181+
Expr::StringLiteral(literal) => last_name(&string_expression(literal.value.to_str())?),
182+
_ => None,
183+
}
184+
}
185+
186+
/// The expression a string annotation holds, if it parses.
187+
fn string_expression(text: &str) -> Option<Expr> {
188+
let parsed =
189+
ruff_python_parser::parse_unchecked(text.trim(), ParseOptions::from(Mode::Expression));
190+
if !parsed.has_no_syntax_errors() {
191+
return None;
192+
}
193+
Some(*parsed.try_into_expression()?.into_syntax().body)
194+
}
195+
196+
/// Every name `expression` loads, in source order: `enumerate` and `SPECS` for `enumerate(SPECS.items())`.
197+
fn loaded_names(expression: &Expr) -> Vec<String> {
198+
let mut names = LoadedNames::default();
199+
names.visit_expr(expression);
200+
names.names
201+
}
202+
203+
#[derive(Default)]
204+
struct LoadedNames {
205+
names: Vec<String>,
206+
}
207+
208+
impl<'a> Visitor<'a> for LoadedNames {
209+
fn visit_expr(&mut self, expression: &'a Expr) {
210+
if let Expr::Name(name) = expression {
211+
self.names.push(name.id.to_string());
212+
}
213+
walk_expr(self, expression);
214+
}
215+
}

0 commit comments

Comments
 (0)