Skip to content

fix(server): check table names, index names and ARNs before authorization - #830

Open
LeeroyHannigan wants to merge 1 commit into
mainfrom
fix/367-table-name-validation
Open

LeeroyHannigan wants to merge 1 commit into
mainfrom
fix/367-table-name-validation

Conversation

@LeeroyHannigan

@LeeroyHannigan LeeroyHannigan commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

What

This PR checks every table name, index name and ARN in a request before the authorization lookups. A bad name now gets the ValidationException that DynamoDB returns. On PostgreSQL such a name answered 500 before this change.

A request's table names reach storage before the engine checks them. The authorization layer reads each table's key schema for dynamodb:LeadingKeys. It also reads the resource tags of the table ARN that it builds from the name. On PostgreSQL a name that holds U+0000 cannot bind as a TEXT parameter. So those lookups failed, and the request answered 500, "Internal error during authorization". DynamoDB answers 400 ValidationException. It also answers before the IAM decision. A caller with no permission gets the ValidationException for a bad name, and AccessDeniedException for the name of a table that exists (measured on 2026-10-24).

Two new modules run first in authorize_request (crates/server/src/request_helpers.rs), before the key info and tag lookups:

  • crates/server/src/table_name_gate.rs checks every field that carries a table or index name:

    • TableName and the RequestItems keys
    • the TransactItems table names, at transactItems.<n>.member.<op>.tableName
    • the restore SourceTableName and TargetTableName, and the ImportTable table name
    • ExclusiveStartTableName, and the Query and Scan IndexName

    A name outside [a-zA-Z0-9_.-]+ gets the member pattern sentence at the field's own location. When the name also breaks a length bound, the message has both clauses under "2 validation errors detected:", and the length clause comes first. ListStreams has its own sentence: "TableName must be at least 3 characters long and at most 255 characters long".

  • crates/server/src/arn_gate.rs checks an ARN in a name field:

    • A plain table ARN with a bad table segment gets "Table name of ARN must satisfy regular expression pattern: [a-zA-Z0-9_.-]+".
    • A bad account or region segment gets "Invalid subscriber ID" or "Invalid AWS region".
    • Any other shape gets the "Valid ARN format is ..." sentence. This includes an index ARN, a stream ARN and an ARN of another service.

    It also checks the ARN fields. A stream ARN gets "Invalid StreamArn", and a backup ARN gets "Invalid Backup ARN". A tag operation's ResourceArn gets "Invalid TableArn: Invalid ResourceArn provided as input ". The export TableArn gets "Invalid TableArn: Invalid Table ARN", and the restore SourceTableArn gets "Invalid Request: Invalid TableArn: Invalid Table ARN".

crates/engine/src/update_table.rs reports an index name outside the pattern at globalSecondaryIndexUpdates.<n>.member.create.indexName and at the delete form. Those are the paths DynamoDB uses.

DynamoDB reads the ListStreams TableName, the restore names and the ImportTable table name as names, not as ARNs. So those fields take the name rules. The cross-account layer on main already reads them the same way.

I measured these sentences against DynamoDB in us-east-1 on 2026-10-24, and again on 2026-11-04.

Where ExtendDB differs from DynamoDB after this change:

  • When RequestItems has more than one bad key, the gate names the first bad key of the parsed map. DynamoDB puts the whole map in its message. The parsed map keeps request order only when serde_json has its preserve_order feature. The MongoDB driver enables that feature, so the workspace build and the MongoDB image keep request order. The PostgreSQL, Cassandra and SQLite images build without the MongoDB driver and use key order. The tests use one bad key.
  • DescribeTable with an ARN of another service keeps the answer of the checks after the gate. DynamoDB answers AccessDeniedException for that shape (measured on 2026-10-24). ExtendDB answers 400, and the test checks only the status.

#833 covers how the catalog stores attribute names that hold U+0000.

Why

Part of #367. The issue reports two faults on PostgreSQL. This PR fixes the table name fault on every backend. #833 fixes the attribute name fault and closes the issue. Together the two PRs replace #531, which a review asked to split.

Testing done

Rust, on the branch (base: main at 25ee109):

  • cargo fmt --all -- --check: clean.
  • cargo +1.97.0 clippy --locked --all-targets -- -D warnings: clean.
  • cargo +1.88.0 build --locked --workspace: ok.
  • cargo test --locked -p extenddb-server -p extenddb-engine -p extenddb-core: core 765 passed (1 ignored), engine 231 passed, server 307 passed, 0 failed.
  • cargo test --locked --workspace --exclude extenddb-backup-store: 3063 passed, 0 failed, 61 ignored.

Wire tests through boto3 with SDK parameter validation off, against a local server per backend, built as the CI integration jobs build it. The 45 rows are in tests/python/test_nul_table_name_fields.py (18), test_nul_arn_shapes.py (6) and test_nul_table_names.py (21). The existing set is test_tables, test_gsi, test_control_plane, test_error_handling, test_batch, test_items, test_tagging, test_query_scan, test_transactions and test_streams.

Backend The 45 rows The existing set
SQLite 45 passed 182 passed
PostgreSQL 45 passed 182 passed
Cassandra (cassandra:4.1) 45 passed 182 passed
MongoDB (mongo:7 replica set) 45 passed 182 passed

The same 45 rows against a main 25ee109 binary, which has no gate:

  • PostgreSQL: 43 failed and 2 passed. Most failures were the 500 "Internal error during authorization".
  • SQLite: 32 failed and 13 passed. The failures got another sentence, another location or ResourceNotFoundException.

Against Amazon DynamoDB in us-east-1 on 2026-11-04, from 22:47 to 22:53 UTC: the 45 rows passed.

Checklist

  • I have read CONTRIBUTING.md
  • All tests pass (cargo test --workspace)
  • Code is formatted (cargo fmt --check)
  • Clippy is clean (cargo clippy -- -W clippy::pedantic)
  • I have added or updated tests for new functionality
  • I have updated documentation if behavior changed
  • Breaking changes are noted below (if any)
  • If this changes the wire protocol, Storage trait, auth model, on-disk
    format, or public CLI surface, an RFC has been accepted or is linked
    below. Otherwise, an ADR captures the decision (link below).

ADR / RFC: n/a. The change refuses the names that DynamoDB refuses, with the same messages. No document describes the old answers.


By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.

…tion

A request's table names reach storage before the engine checks them.
The authorization layer reads each table's key schema for
dynamodb:LeadingKeys. It also reads the resource tags of the table ARN
it builds from the name. On PostgreSQL a name that holds U+0000 cannot
bind as a TEXT parameter. So those lookups failed, and the request
answered 500, "Internal error during authorization". DynamoDB answers
400 ValidationException, and it answers before the IAM decision. A
caller with no permission gets the ValidationException for a bad name,
and AccessDeniedException for the name of a table that exists (measured
on 2026-10-24).

Two new modules in crates/server run first in authorize_request, before
the key info and tag lookups.

table_name_gate checks every field that carries a table or index name:
TableName, the RequestItems keys and the TransactItems table names. It
also checks the restore SourceTableName and TargetTableName, the
ImportTable table name, ExclusiveStartTableName, and the Query and Scan
IndexName. A name outside the pattern [a-zA-Z0-9_.-]+ gets the member
pattern sentence at the field's own location. A name that also breaks a
length bound gets both clauses, the length clause first. ListStreams has
its own sentence.

arn_gate checks an ARN in a name field, and the stream, backup, export,
restore and tag ARN fields. The faults are a bad table segment, a bad
account or region segment, and a shape that is not a plain table ARN.
Each fault gets the sentence DynamoDB gives for it.

UpdateTable reports an index name outside the pattern at
globalSecondaryIndexUpdates.N.member.create.indexName and at the delete
form. Those are the paths DynamoDB uses.

The sentences were measured against DynamoDB in us-east-1 on 2026-10-24
and again on 2026-11-04.

Part of #367

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant