Skip to content

[1b] Add typed where() with field-method conditions#84

Open
davegaeddert wants to merge 12 commits into
masterfrom
typed-where
Open

[1b] Add typed where() with field-method conditions#84
davegaeddert wants to merge 12 commits into
masterfrom
typed-where

Conversation

@davegaeddert

@davegaeddert davegaeddert commented Jul 23, 2026

Copy link
Copy Markdown
Member

What

Adds a typed query API alongside filter():

User.query.where(
    User.email.equals("foo@example.com"),
    ~User.role.equals("admin"),
    User.age.gte(18),
)

Each argument is a Q built from a field method, and positional conditions are ANDed together. Because every condition comes from a real field descriptor, a type checker rejects a misspelled field or a wrong value type at the call site — unlike string keyword lookups.

API

Field descriptors gain Q-returning condition methods:

  • All fields: equals, not_equal, gt, gte, lt, lte, is_null, is_in
  • Text fields: also contains, icontains, startswith, endswith

Conditions compose with |/& and negate with ~. is_in takes an iterable of the field's value type. QuerySet.where(*conditions: Q) ANDs the positional Qs through the existing _filter_or_exclude machinery.

Foreign-key traversal

Accessing a field through a relation builds the joined lookup:

Post.query.where(Post.author.email.equals("a@example.com"))  # Q(author__email="...")

Class-level access to a forward FK returns a RelatedFieldRef traversal proxy (see below); chained access builds nested proxies until a concrete field is reached, then a PrefixedFieldRef produces the multi-segment Q. At the type level, class access reads as type[T] (the related model) so the related model's field surface is visible for chaining.

Encrypted-field blocking

Encrypted fields store non-deterministic ciphertext, so value comparisons can never match. Their condition methods take a Never-typed parameter (rejected by the type checker) and raise TypeError at runtime — only is_null() is allowed. The block also fires through FK traversal, so it's a per-field guarantee rather than a per-call-site one. The legacy filter(field="x") kwarg path is blocked the same way, while filter(field=None) still rewrites to IS NULL.

Descriptor-shadowing fix

Previously, class-level FK access returned the descriptor itself and relied on __getattr__ (which only fires on failed lookup) to build the traversal proxy. A related model with a field named like a public descriptor attribute (field, is_cached, get_queryset, get_prefetch_queryset) would silently shadow it — Child.parent.field.equals(...) returned the descriptor's .field and built wrong SQL with no error.

Now ForwardForeignKeyDescriptor.__get__(instance=None) returns a fresh RelatedFieldRef whose namespace holds only traversal machinery, and attribute lookup reads the related model with inspect.getattr_static (which also resolves inherited fields without invoking their __get__). The one piece of framework code that needs the real descriptor via class access — get_prefetcher in the prefetch machinery — now reaches it with inspect.getattr_static too. Instance access (child.parent) is unchanged. A test fixture with fields named after every shadowed attribute asserts correct traversal end-to-end.

Tests / checks

  • ./scripts/test (all packages) and ./scripts/type-check plain-postgres pass. The # ty: ignore[...] markers in the typed-where tests are load-bearing regression detectors for the honest parameter typing.

davegaeddert added 12 commits May 23, 2026 13:10
@davegaeddert
Add typed where() with field-method conditions
ad3f25a 
First slice of the typed query API. Field classes are now generic over
their value type T; stubs return the parameterized descriptor instead of
the primitive. Combined with Field's overloaded __get__, this gives:

    class User(postgres.Model):
        email = types.EmailField()
        note = types.TextField(allow_null=True, required=False)

    User.email   # EmailField[str]         — typed reference, .equals()/.contains()/...
    user.email   # str                     — value type preserved
    User.note    # TextField[str | None]
    user.note    # str | None              — nullability preserved

Adds equals/not_equal/gt/gte/lt/lte/is_null on Field[T] and
contains/icontains/startswith/endswith on TextField, each returning Q.
New QuerySet.where(*conditions: Q) accepts them positionally with no
**kwargs, so a type checker can reject field typos and value-type
mismatches at the call site. Coexists with filter()/exclude().

Model field declarations across the monorepo drop their primitive
annotation (`name: str = types.TextField()` → `name = types.TextField()`).
The descriptor protocol handles both class- and instance-access typing.
@davegaeddert
Block typed-query comparisons on encrypted fields
1f02cf0 
Override equals/not_equal/gt/gte/lt/lte on EncryptedFieldMixin with a
`Never`-typed parameter so a type checker rejects the call at the use
site, and raise TypeError at runtime as a safety net. is_null remains
the only meaningful comparison since ciphertext is non-deterministic.

This makes the design's "method surface = capability" promise actually
load-bearing: misuse is a type error, not a runtime no-op or a silently
empty filter.
@davegaeddert
Close kwarg-path foot-gun and tighten encrypted-field block
0fc51f2 
Address review findings on the previous commit:

1. The typed `.equals()` block doesn't help users on the legacy kwarg
   path. `filter(api_key='x')` still resolved via the allowed `exact`
   lookup and silently returned zero rows (ciphertext is non-
   deterministic). Wrap the exact lookup class in `get_lookup()` so
   non-None right-hand values raise TypeError at lookup construction.
   None still passes through, preserving the exact-None → isnull rewrite.

2. Strengthen `is_null` test to inspect the Q's children instead of just
   asserting isinstance — a regression in `Field.is_null` would have
   slipped past the old check.

3. Add `assert self.name is not None` in the error-message helper, so a
   call on an unbound field fails loudly instead of rendering "None" in
   the error.

4. Change override return type from `Q` to `Never` to match the
   actually-unreachable return; `Never` is assignable to `Q` so call
   sites like `where(field.equals(...))` still type-check at the use
   site, with the `Never` parameter error as the surfaced diagnostic.

5. Parametrize the ordering-comparison test so each method (gt/gte/lt/
   lte) reports independently.
@davegaeddert
Add typed FK traversal for where() clauses
f76757c 
Extend the typed read API across forward foreign keys:

    Child.parent.name.equals("foo")     →  Q(parent__name="foo")
    Order.user.profile.city.equals(...)  →  Q(user__profile__city=...)

Runtime: new RelatedFieldRef / PrefixedFieldRef helpers proxy class-level
attribute access through to the related model's fields, accumulating a
lookup-path prefix as it goes. ForwardForeignKeyDescriptor gets a
__getattr__ that yields the initial RelatedFieldRef so chaining starts
from `Child.parent`. The SQL builder's existing names_to_path / join
machinery handles the rest — we just produce correctly-prefixed Q.

Typing: ForeignKeyField stub now returns a _ForeignKeyDescriptor[T, V]
with overloaded __get__ — class access yields `type[T]` (so the related
model's typed field surface is visible) and instance access yields V
(T or T | None). __set__ is overloaded to accept V | int so bare PK
assignment still type-checks.

Migration: drops `: ModelType = types.ForeignKeyField(...)` annotations
across the monorepo. The new stub provides the descriptor type and the
overloads handle both class- and instance-side typing.

Scope: forward FK only. Reverse FK (ReverseForeignKey) and M2M
traversal will follow the same pattern in a separate commit.
@davegaeddert
Fix FK traversal review findings
5a78e38 
1. Encrypted-field bypass: PrefixedFieldRef now calls _reject_if_blocked
   in equals/not_equal/gt/gte/lt/lte and the string lookups. If the
   wrapped field is an EncryptedFieldMixin instance, raise TypeError with
   the same "use .is_null() instead" hint the direct-access path gives.
   The SQL-layer block (_exact_for_encrypted) still catches it as a
   second line of defense, but the typed-API now fails at the call site
   for clearer stack traces.

2. Multi-hop coverage: added tests exercising the
   RelatedFieldRef → RelatedFieldRef → PrefixedFieldRef recursion via
   Grandchild.mid_parent.grandparent.name, both as Q construction and
   end-to-end query.

3. Bool slip-through: documented the Python `bool <: int` quirk in the
   _ForeignKeyDescriptor stub comment so future maintainers know the
   runtime check in ForwardForeignKeyDescriptor.__set__ is the only
   guard. (No clean way to exclude bool from `int` in Python's type
   system.)

4. Descriptor attribute shadowing: documented in the RelatedFieldRef
   docstring with a pinned test asserting the current behavior (where a
   field named `field` on the related model would be shadowed by the
   descriptor's own `.field`). The architectural fix — returning a fresh
   proxy from __get__(instance=None) — is bigger and deferred.
@davegaeddert
Close assignment-typing gap on Field.__set__
1807490 
Field.__set__ was already a data descriptor at runtime, but its
parameter was typed `value: Any` — so ty silently allowed wrong-type
assignment like `row.name = 123` on a TextField. Narrow to `value: T`
so the type checker enforces the declared field type at the call site.

Plain's runtime is more permissive (`to_python` converts strings → ints
etc.), but encouraging explicit conversion at the boundary catches a
real bug class and makes the new typed-field declarations actually pull
their weight.

Tests added: a TYPE_CHECKING-only block with deliberately wrong
assignments and `# ty: ignore[invalid-assignment]` markers. If the
type-check ever loosens, ty flags the markers as unused suppressions —
making regressions visible.
@davegaeddert
Merge master into typed-where
bc619bb 
Master's "Type fields as parameterized descriptors" overlaps heavily with
typed-where's foundation. Resolution keeps typed-where's typed-query
surface (.equals/.contains/...; where(); FK traversal via
_ForeignKeyDescriptor.__get__ -> type[T]) and adopts master's annotation
conventions where they're additive:

- JSONField / EncryptedJSONField: LHS-annotate explicitly (stubs return
  Any).
- String-arg ForeignKeyField: take master's split overloads — bare T
  with required LHS annotation — and apply across affected models.
- TextField.contains/icontains/startswith/endswith: kept (typed-where
  needs them; master had removed them).

Reordered tests/app/examples/models/relationships.py so WidgetTag.widget
uses a class-arg FK (Widget defined first, M2M through="WidgetTag")
preserving type-level FK traversal in test_typed_where_fk after master's
LHS-annotation convention.
@davegaeddert
Merge remote-tracking branch 'origin/master' into typed-where
9cf1090
@davegaeddert
Fix FK descriptor shadowing in typed where() traversal
f32c3ad 
Class-level access to a forward FK (Child.parent) now returns a fresh
RelatedFieldRef traversal proxy from __get__ instead of the descriptor
itself. Attribute lookup on the proxy reads the related model with
inspect.getattr_static, so a related field whose name collides with a
public descriptor attribute (field, is_cached, get_queryset,
get_prefetch_queryset) resolves to the field rather than silently
returning the descriptor attribute and building wrong SQL.

Prefetch machinery, the only code that needs the descriptor via
class-level access, now reaches it with inspect.getattr_static in
get_prefetcher to bypass the proxy.
@davegaeddert
Add is_in membership condition to typed where()
85717fd 
is_in builds a Q(field__in=[...]) membership condition on every field,
typed as an iterable of the field's value type so a wrong element type is
rejected at the call site. Negation composes with ~. FK traversal builds
the prefixed path, and encrypted fields block it with a Never-typed
parameter plus a runtime TypeError, matching the other comparisons.
@davegaeddert
Document typed where() conditions
3da7f53 
Add a Querying subsection covering where(), the condition methods on every
field and on text fields, negation and combination, foreign-key traversal,
and the encrypted-field restriction to is_null().
@davegaeddert
Consolidate typed where() traversal onto delegation and metadata reso…
c9e8beb 
…lution

PrefixedFieldRef now delegates each condition to the wrapped field's own
method and rewrites the resulting Q's leaf keys onto the relation path,
instead of hand-mirroring every condition method. This makes the traversed
surface exactly the field's own surface: a method the field doesn't define
(contains on a non-text field) raises AttributeError through traversal, and
encrypted-field blocking is enforced automatically by the field's own method
bodies — no separate reject hook.

RelatedFieldRef resolves names through the related model's metadata
(get_forward_field) rather than attribute lookup, which is shadowing-immune by
construction. Both refs now require a resolved model class; the dead
string-model branch is gone.

Field._build_q builds its Q via the positional-tuple constructor rather than
poking children directly.
@davegaeddert
davegaeddert marked this pull request as ready for review July 23, 2026 16:15
@pullapprove5

pullapprove5 Bot commented Jul 23, 2026

Copy link
Copy Markdown
PASS: 1 review scope passed
Scope Progress
✅ all 0/0

View in PullApprove

Next steps:

@davegaeddert davegaeddert changed the title Add typed where() with field-method conditions [merge 2] Add typed where() with field-method conditions Jul 23, 2026
@davegaeddert davegaeddert changed the title [merge 2] Add typed where() with field-method conditions [1b] Add typed where() with field-method conditions Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@davegaeddert