Skip to content

[3d] Add upsert() and remove update_or_create() - #88

Draft
davegaeddert wants to merge 7 commits into
bulk-upsertfrom
upsert
Draft

[3d] Add upsert() and remove update_or_create()#88
davegaeddert wants to merge 7 commits into
bulk-upsertfrom
upsert

Conversation

@davegaeddert

@davegaeddert davegaeddert commented Jul 23, 2026

Copy link
Copy Markdown
Member

Summary

Adds single-row QuerySet.upsert() and removes update_or_create() outright, migrating all internal callers. Stacked on bulk-upsert.

obj, created = Model.query.upsert(
    key="my-key",
    defaults={"value": data},              # both insert and conflict-update
    create_defaults={"source": "x"},       # insert-only
    conflict_defaults={"count": F("count") + 1},  # per-column SET override
    unique_fields=[Model.key],
)

One INSERT ... ON CONFLICT (unique_fields) DO UPDATE SET ... RETURNING <cols>, (xmax = 0) statement. Returns (obj, created) with obj hydrated from the post-write row (no second query); created comes from the (xmax = 0) flag (freshly-inserted tuples have xmax = 0).

How it works

  • created flag: the compiler's SQLInsertCompiler.as_sql appends a raw (xmax = 0) column straight onto the RETURNING clause in its returning_created branch — returning_columns() stays a pure fields→SQL helper with no extra-column argument. Converters only run on the real field columns, and upsert() pops the trailing bool.
  • conflict_defaults: the dialect's on_conflict_suffix_sql takes optional per-column SQL fragments; the insert compiler compiles each override value through the shared SQLCompiler._compile_assignment_value — the same helper UPDATE ... SET uses — so a value can be plain or an expression (F("count") + 1 resolves to the target row's existing column for an atomic counter). When every inserted column is a unique field there's nothing to SET, so a unique column is set to itself as a no-op — DO UPDATE (not DO NOTHING) is required for RETURNING to return the conflicting row.
  • SET columns: every non-unique, non-PK column from kwargs/defaultsEXCLUDED.col; create_defaults are insert-only and never in the SET; conflict_defaults override per column.
  • Validation: unique_fields take typed field references (Model.key, not "key") and must match the PK or a total UniqueConstraint; unique values must be non-null. Unknown keys in kwargs/defaults raise a clean FieldError (via _validate_model_field_names) before they fail later and more confusingly. No full_clean of the merged row (consistent with the bulk paths).

Caller migrations

  • plain-flags, plain-sessionsupsert().
  • FK reverse manager and M2M manager update_or_createupsert; the M2M wrapper keeps its add-if-created behavior (adds the through-row only on insert).

Tests

New test_upsert.py (insert/conflict, defaults, create_defaults insert-only, F-counter, unique-only idempotency, validation errors including string/wrong-model unique fields, single-statement checks) plus M2M/FK manager add-if-created tests. Full suite green; type-check plain-postgres clean.

Extends the insert compiler and dialect so a single INSERT ... ON CONFLICT DO
UPDATE can (a) override the SET clause per column with compiled expressions and
(b) append a raw "(xmax = 0)" column to RETURNING. Inert until upsert() uses it.
upsert(**values, unique_fields=[...], defaults=..., create_defaults=...,
conflict_defaults=...) runs one INSERT ... ON CONFLICT DO UPDATE ... RETURNING
and returns (obj, created), hydrating obj from the post-write row. defaults
apply on insert and conflict-update; create_defaults on insert only;
conflict_defaults override the SET per column (plain values or expressions such
as F("count") + 1). created comes from the RETURNING (xmax = 0) flag.

Removes update_or_create() and migrates every caller (flags, sessions, the FK
reverse and M2M related managers) to upsert(). The M2M manager keeps its
add-if-created behavior.
Extract one _compile_assignment_value helper for the UPDATE ... SET and
INSERT ... ON CONFLICT DO UPDATE SET value cascade, so conflict_defaults
now goes through the same related-field (prepare_database_save) branch.
Keep returning_columns() pure and move the (xmax = 0) created-flag trick
next to the returning_created flag. Share the unique_fields and non-null
key checks between upsert() and bulk_upsert(), and validate upsert()'s
merged keys with the same FieldError path get_or_create() uses.
Match the explicit-keyword style the upsert() wrappers already use,
instead of letting defaults ride along inside **kwargs.
@davegaeddert davegaeddert changed the title Add upsert() and remove update_or_create() [merge 6] Add upsert() and remove update_or_create() Jul 23, 2026
@davegaeddert davegaeddert changed the title [merge 6] Add upsert() and remove update_or_create() [3d] Add upsert() and remove update_or_create() Jul 23, 2026
QuerySet.upsert()'s unique_fields now accepts Field references (Model.field)
via the shared _validate_field_refs() check; names are derived at the
unique_fields_match_constraint boundary. defaults/create_defaults/
conflict_defaults/**kwargs stay string-keyed -- that's the kwargs idiom. The
related-manager upsert() wrappers forward list[Field]. Migrates the internal
callers: plain-sessions (Session.session_key) and plain-flags (Flag.name).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant