Async request-level token measurement for @tanstack/ai-compaction?

[Visualizeit]

@tanstack/ai-compaction currently uses a synchronous, per-message
estimateTokens callback, with characterCount / 4 as the default.

This is inexpensive, but:

• CJK content can be significantly underestimated.
• Provider token-count APIs are asynchronous.
• Accurate counting may need the complete request, including system prompts,
  tool schemas, multimodal content, and provider framing.
• onUsage arrives after the request, so it cannot protect the current call.

Would it make sense to support an async, request-level token measurement hook?

measureTokens?: (request) => number | Promise<number>

I think request-level measurement would be preferable to making the existing
per-message estimator async. Calling a remote API once per message could create
many requests, while provider count APIs generally measure the complete input.

Prior art:

• pi combines provider-reported usage
  with local estimation and overflow recovery.
• OpenCode primarily uses reported
  usage for overflow decisions, with estimation for compaction selection and
  provider errors as a fallback.

Both approaches still need heuristics because exact preflight measurement is
not available through the compaction abstraction.

Would this fit better as:

1. A measureTokens option in @tanstack/ai-compaction; or
2. A provider/adapter countTokens capability consumed by compaction?