Skip to content

fix(core): refuse a second condition on a key in KeyConditionExpression - #484

Open
LeeroyHannigan wants to merge 1 commit into
mainfrom
fix/key-condition-one-per-key
Open

LeeroyHannigan wants to merge 1 commit into
mainfrom
fix/key-condition-one-per-key

Conversation

@LeeroyHannigan

@LeeroyHannigan LeeroyHannigan commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Query served a KeyConditionExpression that carried two conditions on the same key attribute.
For pk = :a AND pk = :b the parser took the first equality as the partition key condition and the
second as the sort key condition, so on a table with pk HASH and sk RANGE the request returned
the items in partition :a whose sort key equalled :b. The same hole accepted two conditions on
the sort key, including BETWEEN or begins_with paired with another sort key condition.

DynamoDB rejects those shapes with one message, which the parser now returns, at the point in its
order of checks where DynamoDB reports it.

Fixes #430

What changed

crates/core/src/expression/key_condition.rs:

  • parse_key_condition_with_maps parses a request's key condition with the request's
    ExpressionAttributeNames and ExpressionAttributeValues. After the parser's checks for
    redundant parentheses, the clause grammar and nested attribute paths, and before any clause is
    read as the partition or the sort key condition, it refuses a key condition with two conditions on
    one key attribute, with the service's text: Invalid KeyConditionExpression: KeyConditionExpressions must only contain one condition per key. The tokenizer's reserved-word
    check runs before the parse.
  • Two clauses name the same attribute when their paths resolve to the same name, whether each is
    written as a plain name or as a #name reference. Two clauses on one #name reference count as a
    duplicate whether or not the request defines it.
  • When there is a duplicate, an undefined #name in any clause is reported first, with
    Invalid KeyConditionExpression: An expression attribute name used in the document path is not defined; attribute name: #u, and then an undefined :value, with Invalid KeyConditionExpression: An expression attribute value used in expression is not defined; attribute value: :a. A key condition without a duplicate takes the same path as on main.
  • key_condition_clause_operands, which the LeadingKeys extraction reads, takes the same two maps
    and runs the same check, so it names no operand for a duplicate.
  • parse_key_condition keeps its signature and does not look for a duplicate; neither the engine
    nor the LeadingKeys extraction calls it. The multi-attribute classifier keeps its own duplicate
    check, which a caller of parse_key_condition and resolve_multipart reaches.
  • crates/core/src/expression/key_condition.rs is over 600 lines (2323 with its tests, 1123 without
    them; it was 1905 and 996 on main). This change does not split it.

crates/engine/src/expression_helpers.rs and crates/engine/src/query.rs: parse_key_condition_expr
takes the request's expression maps and parses with parse_key_condition_with_maps, so the Query
handler reports a duplicate before resolve_key_schema_with runs the key value type check and before
the unused-value check.

crates/server/src/leading_keys.rs: the dynamodb:LeadingKeys extraction passes the request's names
and values to key_condition_clause_operands, and equality_pk_value parses with
parse_key_condition_with_maps. A duplicate, written with plain names or through a #name
reference, leaves dynamodb:LeadingKeys absent. That includes pk > :a AND #s = :b AND sk = :c,
whose non-equality partition operand main binds. The engine refuses every duplicate, so no item is
read whatever the policy decides. With the key absent, ExtendDB evaluates a ForAllValues: condition
on dynamodb:LeadingKeys as met and a ForAnyValue: condition as not met. main bound the first
equality's value, or the non-equality's operand. Compared with main, a principal allowed by
ForAllValues:StringEquals on another partition value now gets ValidationException instead of
AccessDeniedException. A principal allowed by ForAnyValue:StringEquals on the bound value now gets
AccessDeniedException, where main served the query or answered ValidationException. Where
DynamoDB runs its duplicate check relative to authorization was not measured.

No storage backend changed. The parser is shared, so the rejection applies on SQLite, PostgreSQL,
MongoDB and Cassandra alike. PartiQL statements do not use this parser.

Service measurements

24 October 2026, us-east-1, table bb-arn-main (pk S HASH, sk S RANGE) holding alice/bob and
alice/carol, boto3 with retries off. The cases the tests encode:

UTC time KeyConditionExpression DynamoDB answer
2026-10-24T00:27:56Z pk = :a OK, 2 items
2026-10-24T00:27:56Z pk = :a AND sk = :b OK, 1 item
2026-10-24T00:27:56Z sk = :b AND pk = :a OK, 1 item
2026-10-24T00:36:46Z pk = :a AND sk BETWEEN :b AND :c OK, 2 items
2026-10-24T00:36:46Z pk = :a AND begins_with(sk, :b) OK, 1 item
2026-10-24T00:27:56Z pk = :a AND pk = :b ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND pk = :a ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND pk = :b AND sk = :c ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND pk > :b ValidationException, one condition per key
2026-10-24T00:27:56Z pk > :b AND pk = :a ValidationException, one condition per key
2026-10-24T00:27:56Z :a = pk AND pk = :b ValidationException, one condition per key
2026-10-24T00:27:56Z (pk = :a) AND (pk = :b) ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND sk = :b AND sk = :c ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND sk = :b AND sk = :b ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND sk BETWEEN :b AND :c AND sk = :b ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND sk BETWEEN :b AND :c AND begins_with(sk, :b) ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND begins_with(sk, :b) AND sk = :c ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND sk > :b AND sk < :c ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND begins_with(sk, :b) AND begins_with(sk, :c) ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND pk = :b AND sk = :b AND sk = :c ValidationException, one condition per key
2026-10-24T00:27:56Z sk = :b AND sk = :c ValidationException, one condition per key
2026-10-24T00:27:56Z #p = :a AND pk = :b (#p -> pk) ValidationException, one condition per key
2026-10-24T00:36:46Z #p = :a AND #q = :b (both -> pk) ValidationException, one condition per key
2026-10-24T00:27:56Z pk = :a AND #s = :b AND sk = :c (#s -> sk) ValidationException, one condition per key
2026-10-24T00:30:39Z pk = :a AND zz = :b AND zz = :c ValidationException, one condition per key
2026-10-24T00:30:39Z pk = :a AND pk = :b on a hash-only table ValidationException, one condition per key
2026-10-24T00:30:39Z pk = :a AND pk.foo = :b ValidationException, conditions on nested attributes
2026-10-24T00:30:39Z ((pk = :a)) AND (pk = :b) ValidationException, redundant parentheses
2026-10-24T00:30:39Z #u = :a AND #u = :b (#u undefined) ValidationException, attribute name is not defined
2026-10-24T00:27:56Z pk = :a AND pk = :b AND other = :c ValidationException, reserved keyword: other
2026-10-24T00:27:56Z begins_with(pk, :a) AND sk = :b ValidationException, Query key condition not supported
2026-10-24T00:27:56Z pk BETWEEN :a AND :b ValidationException, Query key condition not supported

The exact message:

Invalid KeyConditionExpression: KeyConditionExpressions must only contain one condition per key

31 October 2026, us-east-1, a table with pk S HASH and sk S RANGE, AWS CLI, two samples of each
request at 17:37 UTC, both samples identical. :a, :b, :c are strings and :n is the number 1.
Every answer is a ValidationException:

KeyConditionExpression and parameters DynamoDB answer
pk = :a AND pk = :b one condition per key
pk = :a AND pk = :n one condition per key
#p = :a AND pk = :n (#p -> pk) one condition per key
pk = :n One or more parameter values were invalid: Condition parameter type does not match schema type
pk = :a AND #s = :b AND sk = :n (#s -> sk) one condition per key
#u = :a AND sk = :b AND sk = :c, no ExpressionAttributeNames Invalid KeyConditionExpression: An expression attribute name used in the document path is not defined; attribute name: #u
pk = :a AND pk = :b AND #u = :c, no ExpressionAttributeNames the same undefined-name text for #u
#u = :a AND #u > :b, no ExpressionAttributeNames the same undefined-name text for #u
pk = :a AND pk = :b, no ExpressionAttributeValues Invalid KeyConditionExpression: An expression attribute value used in expression is not defined; attribute value: :a
pk = :a AND pk = :b with an unused :z in ExpressionAttributeValues one condition per key
pk = :a AND pk = :b with IndexName nope, an index the table does not have one condition per key

Together the two sets place the one-condition-per-key check after redundant parentheses, nested
attribute paths, reserved keywords and undefined #name and :value references anywhere in the
expression, and before the partition key equality, the missing key schema element, the query
condition count, the index lookup, the key value type check and the unused-value check. Against the
key value type check the duplicate was measured in both spellings (#p = :a AND pk = :n,
pk = :a AND #s = :b AND sk = :n); the other orders were measured with plain names. The order of those other checks
among themselves was not measured. pk > :b AND pk = :a answers the duplicate message rather than
"Query key condition not supported", and sk = :b AND sk = :c answers it rather than "Query
condition missed key schema element". This branch runs the check at that point, except against the
index lookup, described under the last heading.

Tests

Unit tests in crates/core/src/expression/key_condition.rs:

  • duplicate_condition_on_one_key_attribute_rejected, over the 16 measured duplicate shapes.
  • duplicate_condition_through_name_reference_rejected,
    duplicate_condition_through_two_name_references_rejected and
    duplicate_condition_on_sort_key_through_name_reference_rejected, for the alias shapes.
  • undefined_name_reported_ahead_of_duplicate, undefined_value_reported_ahead_of_duplicate,
    duplicate_reported_ahead_of_type_and_unused_value_checks and
    nested_path_reported_ahead_of_duplicate, which pin the measured order.
  • clause_operands_reject_duplicate_key_attribute, which pins the refusal on
    key_condition_clause_operands, the entry point authorization reads, in both spellings.
  • distinct_key_attributes_accepted_after_duplicate_check, which keeps the ordinary and multi-part
    key shapes parsing and resolving under their key schemas.

key_condition_duplicate_check_order_matches_the_service in crates/engine/src/expression_helpers.rs
sends the 31 October requests that pin the order through parse_key_condition_expr: the three
undefined-name cases, the undefined-value case, the two number cases written through a #name
reference, the unused value, pk = :a AND pk = :b, and pk = :n, which parses.
query_duplicate_key_condition_leaves_the_key_absent in crates/server/src/leading_keys/tests.rs
checks that both spellings leave dynamodb:LeadingKeys absent, with an equality and with a
non-equality partition clause.

Wire tests in tests/test_query_key_condition_one_per_key.py, dual-target against DynamoDB and
ExtendDB: the duplicate partition key shapes, the duplicate sort key shapes, the alias shapes, the
accepted shapes with their row counts, the three 24 October check-order cases, and
TestDuplicateCheckOrder with seven of the 31 October requests: the three undefined-name cases, the
undefined-value case, the two number cases written through a #name reference, and the unused
value.

What fails without the fix: with the duplicate check disabled (reject_duplicate_key_attrs passing
every key condition) and nothing else changed, 8 of the 52 key_condition unit tests fail: the four
duplicate tests, clause_operands_reject_duplicate_key_attribute,
undefined_name_reported_ahead_of_duplicate, undefined_value_reported_ahead_of_duplicate and
duplicate_reported_ahead_of_type_and_unused_value_checks. The engine test fails, and so do
query_duplicate_key_condition_leaves_the_key_absent and
query_unreadable_partition_equality_binds_no_other_clause in the server. With the duplicate
reported before the undefined references instead, undefined_name_reported_ahead_of_duplicate,
undefined_value_reported_ahead_of_duplicate and the engine test fail. On a server built from
main, the wire tests for the duplicate shapes fail with the server answering 200 and Count 1 for
pk = :a AND pk = :b.

Rebase on #511 and #429

Rebased on main at f49c5f823 on 31 October. Two main PRs changed
crates/core/src/expression/key_condition.rs since this branch was cut. #511 moved the clause parsing
out of parse_key_condition into parse_clauses and added key_condition_clause_operands, which the
LeadingKeys extraction reads. #429 rewrote the multi-attribute key condition classifier, with its own
duplicate check by resolved attribute, and added resolve_key_schema_with, which the engine and the
LeadingKeys extraction call after the parse.

The duplicate check runs on the output of parse_clauses, in parse_key_condition_with_maps and in
key_condition_clause_operands, so both refuse a duplicate before #429's classification and its value
check. #511's description places the check at the end of parse_clauses, where both callers get it;
it runs in the two callers instead because it needs the request's names and values, which
parse_clauses does not take. parse_clauses is #511's code unchanged, and the clause classification
at the end of parse_key_condition moved into classify_clauses, which both parse functions call.
clause_operands_name_every_single_operand_clause (#511's) listed pk = :u AND pk > :v as an
accepted two-clause case; that expression names pk twice and is now in
clause_operands_reject_duplicate_key_attribute. Apart from that case and the two maps #511's two
tests now pass to key_condition_clause_operands, #511's and #429's tests in that file are as on
main.

#511's query_unreadable_partition_equality_binds_no_other_clause and
query_two_non_equality_partition_clauses_bind_nothing in crates/server/src/leading_keys/tests.rs,
and test_unreadable_partition_equality_does_not_bind_another_clause in
tests/test_fgac_leading_keys.py, send shapes that name the partition key twice. They now assert
what the duplicate check does with them: the key is absent, a principal allowed everything gets the
one-condition-per-key error, and the two LeadingKeys policies of the wire test get
AccessDeniedException. The comment in crates/server/src/leading_keys.rs that listed two
non-equality partition clauses as their own case now says the duplicate check leaves those absent.

On 4a2d3828a, cargo +1.97.0 fmt --all --check,
cargo +1.97.0 clippy --locked --workspace --all-targets -- -D warnings,
cargo +1.97.0 check --locked --workspace --all-targets and cargo +1.88.0 check --locked --workspace
pass with no warnings. The --lib tests: core 725 passed and 1 ignored, engine 184 and server 238. The 32 wire files that reference KeyConditionExpression, on a SQLite server and
a PostgreSQL 16 server built from the branch: 857 passed, 14 skipped, 2 failed and 5 errors on each,
with tests/test_query_key_condition_one_per_key.py and tests/test_fgac_leading_keys.py passing on
both. The 7 that do not pass are in tests/test_gsi_async.py, which runs a binary at
./target/release/extenddb that these dev builds do not produce, and tests/test_gsi_async_queue.py,
whose fixture needs EXTENDDB_TEST_PG_CONNECTION_STRING, which this harness does not set. They
answer the same on servers built from main at f49c5f823.

CI passed all 35 checks on 4a2d3828a.

Where ExtendDB still differs from DynamoDB on these requests

  • ExtendDB looks the index up before it parses the key condition, so pk = :a AND pk = :b with
    IndexName nope answers The table does not have the specified index: nope, where DynamoDB
    answers the duplicate.
  • A second condition on a non-key attribute, with no duplicate, is not refused on a single-attribute
    key schema. For pk = :a AND zz = :c on a table with pk HASH and sk RANGE, DynamoDB answers
    "Query condition missed key schema element: sk" and ExtendDB answers HTTP 200, applying the clause
    to the sort key column (Query on a single-attribute key schema applies a condition on a non-key attribute to the sort key #674). For the same shape on a hash-only table, DynamoDB answers "The
    number of query conditions (2) exceeds the number of key attributes defined in the schema (1)" and
    ExtendDB answers HTTP 200 with Count 1, ignoring the second condition.
  • For a key condition with an undefined :value and no duplicate, such as
    pk = :a AND sk = :missing, ExtendDB answers "An expression attribute value used in expression is
    not defined; attribute value: :missing" without the Invalid KeyConditionExpression: prefix
    DynamoDB puts on it.

…e key

A key condition that named the same key attribute twice was served. For
`pk = :a AND pk = :b` the parser took the first equality as the partition key
condition and the second as the sort key condition, so on a table with
pk HASH and sk RANGE the request returned the items in partition `:a` whose
sort key equalled `:b`. The same hole accepted two conditions on the sort key,
including BETWEEN or begins_with paired with another sort key condition.

DynamoDB rejects all of these with
"Invalid KeyConditionExpression: KeyConditionExpressions must only contain one
condition per key" (measured 2026-10-24, us-east-1). It resolves every `#name`
and `:value` reference before it looks for the duplicate, so an undefined name
or value is reported first, and it reports the duplicate ahead of the key value
type check and the unused-value check (measured 2026-10-31).

parse_key_condition_with_maps takes the request's expression attribute names
and values. After the parser's own checks, and before any clause is
classified, it refuses two conditions on one key attribute, written with plain
names or through `#name` references. When there is a duplicate it reports an
undefined `#name` and then an undefined `:value` first. A key condition
without a duplicate takes the same path as before. The engine's Query handler
parses through it, and the server's LeadingKeys extraction runs the same check
through key_condition_clause_operands, so a duplicate leaves
dynamodb:LeadingKeys absent whichever way it is written.

Fixes #430
@LeeroyHannigan
LeeroyHannigan force-pushed the fix/key-condition-one-per-key branch from 1632b19 to 4a2d382 Compare October 31, 2026 19:55
@navareamol navareamol added the v1 Needed for the 1.0 release label Nov 3, 2026
@navareamol navareamol added this to the 1.0 milestone Nov 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v1 Needed for the 1.0 release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Query accepts a KeyConditionExpression with two conditions on the partition key

2 participants