Repository navigation
fix(server): check table names, index names and ARNs before authorization - #830
Open
LeeroyHannigan wants to merge 1 commit into
Open
LeeroyHannigan wants to merge 1 commit into
LeeroyHannigan wants to merge 1 commit into
Conversation
…tion A request's table names reach storage before the engine checks them. The authorization layer reads each table's key schema for dynamodb:LeadingKeys. It also reads the resource tags of the table ARN it builds from the name. On PostgreSQL a name that holds U+0000 cannot bind as a TEXT parameter. So those lookups failed, and the request answered 500, "Internal error during authorization". DynamoDB answers 400 ValidationException, and it answers before the IAM decision. A caller with no permission gets the ValidationException for a bad name, and AccessDeniedException for the name of a table that exists (measured on 2026-10-24). Two new modules in crates/server run first in authorize_request, before the key info and tag lookups. table_name_gate checks every field that carries a table or index name: TableName, the RequestItems keys and the TransactItems table names. It also checks the restore SourceTableName and TargetTableName, the ImportTable table name, ExclusiveStartTableName, and the Query and Scan IndexName. A name outside the pattern [a-zA-Z0-9_.-]+ gets the member pattern sentence at the field's own location. A name that also breaks a length bound gets both clauses, the length clause first. ListStreams has its own sentence. arn_gate checks an ARN in a name field, and the stream, backup, export, restore and tag ARN fields. The faults are a bad table segment, a bad account or region segment, and a shape that is not a plain table ARN. Each fault gets the sentence DynamoDB gives for it. UpdateTable reports an index name outside the pattern at globalSecondaryIndexUpdates.N.member.create.indexName and at the delete form. Those are the paths DynamoDB uses. The sentences were measured against DynamoDB in us-east-1 on 2026-10-24 and again on 2026-11-04. Part of #367
LeeroyHannigan
requested review from
amrith,
c33howard,
eliseong,
hitesh96db,
jcshepherd,
navareamol,
pdf-amzn and
robinnsc
as code owners
November 4, 2026 23:19
This was referenced Nov 4, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This PR checks every table name, index name and ARN in a request before the authorization lookups. A bad name now gets the
ValidationExceptionthat DynamoDB returns. On PostgreSQL such a name answered 500 before this change.A request's table names reach storage before the engine checks them. The authorization layer reads each table's key schema for
dynamodb:LeadingKeys. It also reads the resource tags of the table ARN that it builds from the name. On PostgreSQL a name that holds U+0000 cannot bind as aTEXTparameter. So those lookups failed, and the request answered 500, "Internal error during authorization". DynamoDB answers 400ValidationException. It also answers before the IAM decision. A caller with no permission gets theValidationExceptionfor a bad name, andAccessDeniedExceptionfor the name of a table that exists (measured on 2026-10-24).Two new modules run first in
authorize_request(crates/server/src/request_helpers.rs), before the key info and tag lookups:crates/server/src/table_name_gate.rschecks every field that carries a table or index name:TableNameand theRequestItemskeysTransactItemstable names, attransactItems.<n>.member.<op>.tableNameSourceTableNameandTargetTableName, and theImportTabletable nameExclusiveStartTableName, and theQueryandScanIndexNameA name outside
[a-zA-Z0-9_.-]+gets the member pattern sentence at the field's own location. When the name also breaks a length bound, the message has both clauses under "2 validation errors detected:", and the length clause comes first.ListStreamshas its own sentence: "TableName must be at least 3 characters long and at most 255 characters long".crates/server/src/arn_gate.rschecks an ARN in a name field:It also checks the ARN fields. A stream ARN gets "Invalid StreamArn", and a backup ARN gets "Invalid Backup ARN". A tag operation's
ResourceArngets "Invalid TableArn: Invalid ResourceArn provided as input ". The exportTableArngets "Invalid TableArn: Invalid Table ARN", and the restoreSourceTableArngets "Invalid Request: Invalid TableArn: Invalid Table ARN".crates/engine/src/update_table.rsreports an index name outside the pattern atglobalSecondaryIndexUpdates.<n>.member.create.indexNameand at the delete form. Those are the paths DynamoDB uses.DynamoDB reads the
ListStreamsTableName, the restore names and theImportTabletable name as names, not as ARNs. So those fields take the name rules. The cross-account layer on main already reads them the same way.I measured these sentences against DynamoDB in us-east-1 on 2026-10-24, and again on 2026-11-04.
Where ExtendDB differs from DynamoDB after this change:
RequestItemshas more than one bad key, the gate names the first bad key of the parsed map. DynamoDB puts the whole map in its message. The parsed map keeps request order only whenserde_jsonhas itspreserve_orderfeature. The MongoDB driver enables that feature, so the workspace build and the MongoDB image keep request order. The PostgreSQL, Cassandra and SQLite images build without the MongoDB driver and use key order. The tests use one bad key.DescribeTablewith an ARN of another service keeps the answer of the checks after the gate. DynamoDB answersAccessDeniedExceptionfor that shape (measured on 2026-10-24). ExtendDB answers 400, and the test checks only the status.#833 covers how the catalog stores attribute names that hold U+0000.
Why
Part of #367. The issue reports two faults on PostgreSQL. This PR fixes the table name fault on every backend. #833 fixes the attribute name fault and closes the issue. Together the two PRs replace #531, which a review asked to split.
Testing done
Rust, on the branch (base: main at 25ee109):
cargo fmt --all -- --check: clean.cargo +1.97.0 clippy --locked --all-targets -- -D warnings: clean.cargo +1.88.0 build --locked --workspace: ok.cargo test --locked -p extenddb-server -p extenddb-engine -p extenddb-core: core 765 passed (1 ignored), engine 231 passed, server 307 passed, 0 failed.cargo test --locked --workspace --exclude extenddb-backup-store: 3063 passed, 0 failed, 61 ignored.Wire tests through boto3 with SDK parameter validation off, against a local server per backend, built as the CI integration jobs build it. The 45 rows are in
tests/python/test_nul_table_name_fields.py(18),test_nul_arn_shapes.py(6) andtest_nul_table_names.py(21). The existing set istest_tables,test_gsi,test_control_plane,test_error_handling,test_batch,test_items,test_tagging,test_query_scan,test_transactionsandtest_streams.cassandra:4.1)mongo:7replica set)The same 45 rows against a main 25ee109 binary, which has no gate:
ResourceNotFoundException.Against Amazon DynamoDB in us-east-1 on 2026-11-04, from 22:47 to 22:53 UTC: the 45 rows passed.
Checklist
cargo test --workspace)cargo fmt --check)cargo clippy -- -W clippy::pedantic)Storagetrait, auth model, on-diskformat, or public CLI surface, an RFC has been accepted or is linked
below. Otherwise, an ADR captures the decision (link below).
ADR / RFC: n/a. The change refuses the names that DynamoDB refuses, with the same messages. No document describes the old answers.
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.