Core: Support CachingCatalog commit tracking for specialized tables

[ysung]

Stacked PR

This is a draft follow-up to #17388. Until #17388 merges, GitHub shows both commits in this PR; the incremental diff is the specialized-table and REST support described below.

Summary

• let specialized Table implementations opt in to CachingCatalog commit tracking without mutating delegate-owned instances
• preserve the concrete table type and validate that operations replacement returns an independent, correctly wired copy
• opt RESTTable in while preserving REST state and server-side scan planning
• preserve existing caching behavior for table implementations that do not opt in
• return contract-violating opt-in implementations normally but leave those instances uncached, with one actionable warning per concrete class

Change size relative to #17388

• production/business logic: 230 changed lines (+207/-23, net +184) across 3 files
• tests: 311 changed lines (+311/-0) across 2 files
• approximately 57% of the incremental changed lines are regression coverage

Design

#17388 installs commit-tracking operations on exact BaseTable instances. Specialized subclasses cannot be reconstructed as BaseTable without losing subtype behavior.

This follow-up adds SupportsOperationsReplacement. An implementation receives replacement operations and returns an independent copy of the same concrete class. Before caching that copy, CachingCatalog verifies that:

• it is non-null and distinct from the delegate-owned table
• it has the exact same concrete class
• it exposes the supplied operations
• the original table's operations were not mutated

RESTTable implements the contract by copying all REST-specific state along with the replacement operations. The REST regression test verifies that a commit overlapping a reload observes the committed snapshot and that planFiles() still uses server-side planning.

Implementations that do not opt in retain the same caching behavior as main, without commit tracking. This avoids turning downstream custom table types into permanent cache misses.

An opt-in implementation that returns null, the original instance, a different type, the wrong operations, mutates the original, or throws remains usable but that load is not cached. Each validation condition has direct regression coverage. The SPI also states that withOperations runs during cache computation and must not call back into the catalog or perform table operations.

Open API question for maintainers

The current draft uses a public SupportsOperationsReplacement interface, following the existing SupportsDistributedScanPlanning pattern. Despite its internal-use documentation, this still permanently adds a public Java type.

An alternative is a protected BaseTable copy-with-operations hook. The implementation and tests do not otherwise depend on which API shape maintainers prefer. Feedback on this choice is requested before this PR is marked ready for review.

Runtime behavior and tradeoffs

• supported specialized tables retain their concrete runtime type and remain cacheable
• RESTTable retains server-side scan planning
• operations() returns the commit-tracking wrapper rather than the delegate operations' concrete runtime class
• table implementations that do not opt in retain their existing cache behavior without post-commit reconciliation
• contract-violating opt-in implementations are returned normally but are not cached

Validation

• ./gradlew :iceberg-core:test --tests org.apache.iceberg.hadoop.TestCachingCatalog --tests org.apache.iceberg.rest.TestRESTScanPlanning.testCachingCatalogObservesCommitAfterConcurrentReloadWithServerSidePlanning
• ./gradlew :iceberg-core:spotlessApply
• ./gradlew :iceberg-core:revapi
• supported, non-opting, and contract-violating replacement implementations, including throwing and mutating implementations
• REST server-side planFiles() execution after wrapping and commit reconciliation

Part of #17338

---

AI Disclosure

• Model: GPT-5
• Platform/Tool: Codex
• Human Oversight: partially reviewed
• Prompt Summary: Split specialized-table and REST commit tracking from the core CachingCatalog race fix, preserve subtype behavior, and validate the operations-replacement contract.