Optimizer: Add commit-time TableStatsPublisher seam for Table Optimizer stats

[abhisheknath2011]

Summary

First (inert) step of the Table Optimizer "Stats Bridge": add a commit-time seam so per-commit table stats can later be published to the optimizer. OpenHouseInternalTableOperations.doCommit() now invokes a pluggable TableStatsPublisher right after a successful commit, with the just-committed TableMetadata in hand. The default implementation is a no-op, so behavior is unchanged until a concrete publisher is configured.

New Features / Internal API

• TableStatsPublisher — interface for publishing stats derived from a successful commit. Contract: best-effort, non- blocking, must not throw.
• NoOpTableStatsPublisher — default @Component implementation; keeps the hook inert in OSS/dev.
• OpenHouseInternalTableOperations.doCommit() — after commitStatus = SUCCESS, calls publishOnCommit(tableIdentifier, committedMetadata) inside a try/catch(Throwable) that only logs, so a
  publisher failure can never affect the commit.
• OpenHouseInternalCatalog.newTableOps() — injects the publisher bean into the table-ops constructor.

Refactoring

• Replaced @AllArgsConstructor on OpenHouseInternalTableOperations with two explicit constructors. The existing 7-arg constructor is preserved and delegates to NoOpTableStatsPublisher, so all existing callers/tests compile unchanged; a new 8-arg constructor accepts the publisher. Backward compatible.

Context
A concrete publisher (async call to the optimizer stats API, feature-gated) lands in follow-up PRs (stats extraction + LinkedIn internal publisher). This PR intentionally contains only the seam so the commit path is provably unaffected.

Follow up PR:
#713

Issue] Briefly discuss the summary of the changes made in this
pull request in 2-3 lines.

Changes

• Client-facing API Changes
• Internal API Changes
• Bug Fixes
• New Features
• Performance Improvements
• Code Style
• Refactoring
• Documentation
• Tests

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

./gradlew build

• Manually Tested on local docker setup. Please include commands ran, and their output.
• Added new tests for the changes made.
• Updated existing tests to reflect the changes made.
• No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
• Some other form of testing like staging or soak time in production. Please explain.

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Additional Information

• Breaking Changes
• Deprecations
• Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.

[cbb330]

I see the event is

• serverside
• built from tablemetadata

Given that, there is these two events have similar behavior: OpenHouseTableAuditEvent/OpenHouseServiceAuditEvent.

These are also commit time, server side, built from request/table metadata. Can they be deduplicated/extended for this purpose?

e.g. OpenHouseTableAuditEvent currently is emitted post-commit, which allows it to utilize some of the context from the request. post-commit as the seam could work for table optimizer as well, and we would prefer deduplicated events if the seam is equal in behavior

[abhisheknath2011]

This is published only on successful commits. This is moved to the finally block on the PR: #713

[abhisheknath2011]

I see the event is

• serverside
• built from tablemetadata

Given that, there is these two events have similar behavior: OpenHouseTableAuditEvent/OpenHouseServiceAuditEvent.

These are also commit time, server side, built from request/table metadata. Can they be deduplicated/extended for this purpose?

e.g. OpenHouseTableAuditEvent currently is emitted post-commit, which allows it to utilize some of the context from the request. post-commit as the seam could work for table optimizer as well, and we would prefer deduplicated events if the seam is equal in behavior

Yes as part of optimizer we are capturing commit event stats as async post commit (fire and forget) to store the events in mysql. Internally optimizer API is invoked to store in mysql. Yes we have server side events but we don't have table specific stats needed by Optimizer. Although we can enrich the existing server side event with required data we need a separate consumer pipeline to process these events and store in mysql (current stats and historical state). In future NRT stats could bridge the gap, but we are not there yet. So capturing async post commit event bridges this gap without adding additional load on tables service (as the optimizer stats collection API is part of optimizer service which is deployed separately) and this solution could potentially be used in the long term if needed.

Only successful commits are published, so no duplication. The publish part is moved to the finally block in this PR: #713

[mkuchenbecker]

I would model it as generic postcommit runs N postcommit operations with bounded guarantees such as max TTL, spawned in the background, etc. Then we know that e.g. a postcommit op at most consumes one thread for 2 seconds, and results / errors consistently consumed.

A caveat is we need to police the load / threads run and be careful on what we put here and what we guarentee because its by-definition best effort.

[mkuchenbecker]

Rather than a stats publisher can this take in a collection of N PostCommit operations where this is the only one?

The main thought is best-effort postcommit operation is a generic feature, and we plug-in stats or whatever. Postcommit can enforce bounded guarentees such as tight TTL and the observability while the actual buisiness logic remains agnostic.

[mkuchenbecker]

OnCommit isn't useful descriptor.

Should this just be a runnable interface? I am thinking that We might end up with a functional interface or similar Publisher.Publish or publisher.Run.