Skip to content

HIVE-29877: HMSCatalogAdapter fails to route Iceberg REST endpoints containing an optional {prefix} - #6749

Open
Aggarwal-Raghav wants to merge 1 commit into
apache:masterfrom
Aggarwal-Raghav:HIVE-29877
Open

HIVE-29877: HMSCatalogAdapter fails to route Iceberg REST endpoints containing an optional {prefix}#6749
Aggarwal-Raghav wants to merge 1 commit into
apache:masterfrom
Aggarwal-Raghav:HIVE-29877

Conversation

@Aggarwal-Raghav

@Aggarwal-Raghav Aggarwal-Raghav commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

HIVE-29877
Previously, the router expected URLs to match an exact length and crashed if an Iceberg client sent a {prefix} (like my_catalog). This PR introduces a mathematical offset that dynamically calculates the length of any prefix, allowing the server to gracefully accept and route multi-segment prefixes.

Why are the changes needed?

https://<workspace-instance>/api/2.1/unity-catalog/iceberg-rest/v1/catalogs/<uc_catalog_name>/namespaces/<uc_schema_name>/tables/<uc_table_name>
  • Spark client can inject prefix using --conf spark.sql.catalog.local.prefix=my_test_prefix

Does this PR introduce any user-facing change?

Yes (Positive Impact): Users can now successfully connect strict Iceberg REST clients that inject custom multi-segment prefixes into the HTTP paths without hitting HTTP 400 errors.

How was this patch tested?

Write Junit test and spark docker image testing (Screenshot attached below)

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor Author

BEFORE:

spark-sql ()> show databases;
26/09/01 19:21:04 ERROR SparkSQLDriver: Failed in [show databases]
org.apache.iceberg.exceptions.BadRequestException: Malformed request: No route for request: GET v1/my_test_prefix/namespaces
        at org.apache.iceberg.rest.ErrorHandlers$NamespaceErrorHandler.accept(ErrorHandlers.java:276)
        at org.apache.iceberg.rest.ErrorHandlers$NamespaceErrorHandler.accept(ErrorHandlers.java:266)
        at org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:242)
        at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:347)
        at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:299)
        at org.apache.iceberg.rest.BaseHTTPClient.get(BaseHTTPClient.java:77)
        at org.apache.iceberg.rest.RESTSessionCatalog.listNamespaces(RESTSessionCatalog.java:776)
        at org.apache.iceberg.catalog.BaseSessionCatalog$AsCatalog.listNamespaces(BaseSessionCatalog.java:139)
        at org.apache.iceberg.rest.RESTCatalog.listNamespaces(RESTCatalog.java:256)
        at org.apache.iceberg.catalog.SupportsNamespaces.listNamespaces(SupportsNamespaces.java:74)
        at org.apache.iceberg.spark.SparkCatalog.listNamespaces(SparkCatalog.java:383)
        at org.apache.spark.sql.execution.command.ShowNamespacesCommand.run(ShowNamespacesCommand.scala:44)
        at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:79)

AFTER:
Screenshot 2026-09-02 at 2 15 34 AM

HTTP request fired from spark -> HMS IRC: GET /iceberg/v1/my_test_prefix/namespaces?pageToken= HTTP/1.1

@sonarqubecloud

sonarqubecloud Bot commented Sep 2, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants