[3d] Add upsert() and remove update_or_create() - #88
Draft
davegaeddert wants to merge 7 commits into
Draft
Conversation
Extends the insert compiler and dialect so a single INSERT ... ON CONFLICT DO UPDATE can (a) override the SET clause per column with compiled expressions and (b) append a raw "(xmax = 0)" column to RETURNING. Inert until upsert() uses it.
upsert(**values, unique_fields=[...], defaults=..., create_defaults=...,
conflict_defaults=...) runs one INSERT ... ON CONFLICT DO UPDATE ... RETURNING
and returns (obj, created), hydrating obj from the post-write row. defaults
apply on insert and conflict-update; create_defaults on insert only;
conflict_defaults override the SET per column (plain values or expressions such
as F("count") + 1). created comes from the RETURNING (xmax = 0) flag.
Removes update_or_create() and migrates every caller (flags, sessions, the FK
reverse and M2M related managers) to upsert(). The M2M manager keeps its
add-if-created behavior.
Extract one _compile_assignment_value helper for the UPDATE ... SET and INSERT ... ON CONFLICT DO UPDATE SET value cascade, so conflict_defaults now goes through the same related-field (prepare_database_save) branch. Keep returning_columns() pure and move the (xmax = 0) created-flag trick next to the returning_created flag. Share the unique_fields and non-null key checks between upsert() and bulk_upsert(), and validate upsert()'s merged keys with the same FieldError path get_or_create() uses.
Match the explicit-keyword style the upsert() wrappers already use, instead of letting defaults ride along inside **kwargs.
QuerySet.upsert()'s unique_fields now accepts Field references (Model.field) via the shared _validate_field_refs() check; names are derived at the unique_fields_match_constraint boundary. defaults/create_defaults/ conflict_defaults/**kwargs stay string-keyed -- that's the kwargs idiom. The related-manager upsert() wrappers forward list[Field]. Migrates the internal callers: plain-sessions (Session.session_key) and plain-flags (Flag.name).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds single-row
QuerySet.upsert()and removesupdate_or_create()outright, migrating all internal callers. Stacked onbulk-upsert.One
INSERT ... ON CONFLICT (unique_fields) DO UPDATE SET ... RETURNING <cols>, (xmax = 0)statement. Returns(obj, created)withobjhydrated from the post-write row (no second query);createdcomes from the(xmax = 0)flag (freshly-inserted tuples havexmax = 0).How it works
SQLInsertCompiler.as_sqlappends a raw(xmax = 0)column straight onto the RETURNING clause in itsreturning_createdbranch —returning_columns()stays a pure fields→SQL helper with no extra-column argument. Converters only run on the real field columns, andupsert()pops the trailing bool.on_conflict_suffix_sqltakes optional per-column SQL fragments; the insert compiler compiles each override value through the sharedSQLCompiler._compile_assignment_value— the same helperUPDATE ... SETuses — so a value can be plain or an expression (F("count") + 1resolves to the target row's existing column for an atomic counter). When every inserted column is a unique field there's nothing to SET, so a unique column is set to itself as a no-op — DO UPDATE (not DO NOTHING) is required for RETURNING to return the conflicting row.kwargs/defaults→EXCLUDED.col;create_defaultsare insert-only and never in the SET;conflict_defaultsoverride per column.unique_fieldstake typed field references (Model.key, not"key") and must match the PK or a totalUniqueConstraint; unique values must be non-null. Unknown keys inkwargs/defaultsraise a cleanFieldError(via_validate_model_field_names) before they fail later and more confusingly. Nofull_cleanof the merged row (consistent with the bulk paths).Caller migrations
plain-flags,plain-sessions→upsert().update_or_create→upsert; the M2M wrapper keeps its add-if-created behavior (adds the through-row only on insert).Tests
New
test_upsert.py(insert/conflict, defaults, create_defaults insert-only, F-counter, unique-only idempotency, validation errors including string/wrong-model unique fields, single-statement checks) plus M2M/FK manager add-if-created tests. Full suite green;type-check plain-postgresclean.