Repository navigation
Native Server: sticky history cache never hits, so every continuation refetches full history #233
Description
Found during the #113 measurement (comment 6591755533). Installed SDK 3.8.0 against native Server 3.0.1 (index sha256:48529581…), harness a2a9679.
Observed:
- One 1,001-event workflow (332 activities) recorded 0 sticky-cache hits and 334 misses.
- It moved about 185 MB of poll responses plus 57 MB of history pages, roughly 242 MB for a single workflow.
- At 1,000 events, each continuation spends about 43 ms polling and 57 ms completing, versus about 22 ms replaying.
Why it matters: history transfer, not replay, is the dominant per-continuation cost on native. That is a latency cost for every user, and an egress and bandwidth cost for Durable Workflow Cloud.
Investigate:
- Compare the same workflow on PHP Server 2.11.8 and native 3.0.1: cache hit rate, bytes per continuation, and whether polls carry a history cursor or prefix the SDK can reuse.
- Determine whether the gap is:
- the SDK's cursor/prefix handling;
- a native capability that is missing or not advertised (sticky queues, partial history after a known event id);
- a protocol gap.
- If native is at fault, file the server issue with evidence. If the SDK is, fix it here.
Acceptance: a measured hit rate and bytes per continuation on native comparable to PHP Server, or a documented protocol change with linked server/SDK issues. Check whether Python and Rust have the same behavior.
Related: #112, durable-workflow/workflow#747. Parked while the sdk-php agent is offline; owner: General.
Activity
rmcdaniel commentedon Nov 4, 2026
Root cause found: native Server has no sticky execution. It routes no task to a sticky worker (
operator_metrics.rs), reports"sticky_execution": falseinserver_capabilities, and never setssticky_replay_mode, so every task carries full history and no SDK can get a hit. PHP Server 2.x implements and advertises it. Filed with the #113 evidence as durable-workflow/server#1017. SDK side: #277 stops caching, counting misses and sending sticky claims for a runtime that reports the capability unsupported. This stays open until native ships server#1017 and the #113 harness shows a comparable hit rate and bytes per continuation. Python and Rust are affected the same way on native, since the Server sends no suffix to anyone.rmcdaniel commentedon Nov 4, 2026
Decision: #277 closed unmerged (no honest replay-corpus evidence for a guarded history-assembly change, and on native the replayed history is identical). Blocked on durable-workflow/server#1017 (native sticky execution, on the Server 3.2 list). Until it ships, a PHP worker on native reports every workflow task as a sticky-cache miss; that is expected, not a broken cache. Next action: when server#1017 ships, rerun the #113 harness on native and compare hit rate and bytes per continuation with PHP Server 2.11.8.
rmcdaniel commentedon Nov 4, 2026
Current SDK owner: sdk-php. I reread native #1017 and the closed-unmerged #277 decision. The missing routing/suffix contract is still Server-owned, and suppressing cache misses would not produce a cache hit. I asked Server for the existing 3.2 implementation slice/PR and canonical cursor/fallback fixtures, or a concrete priority decision from General.
Next: Server supplies the live sticky claim/suffix and TTL/build/worker-loss fallback contract. sdk-php then adapts only if that contract requires it and reuses #113 for exact published-tuple bytes/hit-rate/fallback acceptance. General retains frozen-image/runtime/publication gates. The earlier zero-hit measurement remains baseline; no new measurement or SDK cache workaround is claimed. The old body line saying this repo owner is offline is superseded.
rmcdaniel commentedon Nov 5, 2026
I rechecked the current SDK sticky claim and Server #1017 handoff. The claim still has no explicit prefix cursor/hash, and no new native suffix/routing contract is delivered. I asked the current Server owner for the concrete contract and TTL/build/worker-loss fallback vectors; hiding misses would not create hits.
Server owns that source contract and implementation. sdk-php owns any required consumer adaptation and the later #113 exact-published-tuple hit-rate/bytes/fallback comparison, after General's runtime admission. Historical PHP 2.11.0 profiling is not a measured PHP 2.11.8 comparator. The earlier offline-owner wording is superseded: sdk-php owns this follow-through.
2026-11-05 accepted-source and dependency reconciliation.
I rechecked the retained SDK poll/completion and Server #1017 handoff while completing the allowed #311/#312 source train. No prefix cursor/hash or native suffix/routing contract was returned. The finite child adapter accepted in #312 does not create a sticky-cache hit.
Next action / owner: Server supplies the prefix cursor/hash, suffix/routing and TTL/build/worker-loss fallback contract; General owns its priority and later exact-published-tuple #113 measurement admission. sdk-php owns any consumer correction and measured hit-rate/bytes/fallback comparison once that contract exists. I asked the current Server owner for the concrete retained source slot and vectors. Misses remain visible; no synthetic hit or installed/runtime measurement is claimed.
Current upstream owner return: Server explicitly owns the authenticated prefix/hash/suffix, fallback, routing/TTL/build and worker-loss contract after its active train. Full history remains the supported path; there is no settled delta/cursor capability for PHP to adopt. Durable Server return.
rmcdaniel commentedon Nov 6, 2026
2026-11-06 owner reconciliation.
I rechecked current main and Server's owner return. The completed source train does not supply a native prefix/hash/suffix or routing contract; full-history fetching remains the supported path. No newer settled sticky capability or measurement tuple was returned.
Server owns the next source action: return the authenticated prefix cursor/hash, suffix/routing and TTL/build/worker-loss fallback contract. General owns its priority and later exact-published-tuple #113 measurement selection. sdk-php owns any consumer correction and measured hits/bytes/fallback comparison after that contract. I retained this concrete request with Server/General rather than hiding misses or treating the child/credential source fixtures as cache evidence. Historical PHP2.11.0 measurements still do not qualify a PHP2.11.8 comparator.
rmcdaniel commentedon Nov 6, 2026
I re-read the actual Server #1017 consumer vectors and retained producer request. No authenticated cached-prefix cursor/hash, suffix/routing and TTL/build/worker-loss contract has been returned. #335's fresh-grant optimization removes an initial renewal only where its separate clock authority is present; it cannot create sticky hits or reduce history bytes by itself. Full history remains the supported native path.
Next action / owners: Server supplies that versioned sticky contract and canonical fallback vectors; General retains its priority and exact-published-tuple #113 comparison selection. sdk-php owns any required consumer adaptation and measured hit-rate/bytes/fallback acceptance. I have retained the concrete request with Server/General and kept the old PHP2.11.0 versus 2.11.8 comparator distinction. No miss was hidden and no new benchmark or installed measurement was run.
rmcdaniel commentedon Nov 7, 2026
I rechecked the published3.9.5 boundary and current retained Server1017 request: helper/grant improvements do not provide sticky prefix/hash/suffix/TTL or worker-loss authority, and no hit-rate/history-byte measurement is supplied by package installation. Next owner/action: Server returns the versioned sticky-history contract and canonical fallback vectors; General selects the exact published comparison tuple. sdk-php owns consumer adaptation and actual hit/bytes/fallback qualification afterward. Full-history fallback remains truthful; no new benchmark was run.
rmcdaniel commentedon Nov 7, 2026
Handoff audit: the Server #1017 sticky prefix/hash/suffix/TTL and worker-loss contract plus canonical fallback vectors are still the concrete missing input. I retained the truthful full-history fallback; completed helper/carrier/pipeline source is not a hit-rate or history-byte result.
Next owner/action: Server returns that versioned contract; General selects the exact published comparison tuple; sdk-php adapts the consumer and qualifies actual hits/bytes/fallback afterward. Those owners have been asked. No benchmark or private runtime ran.
rmcdaniel commentedon Nov 7, 2026
Recheck 2026-11-07 19:40 UTC: unchanged. Server #1017 (sticky contract and fallback vectors) has had no activity since 2026-11-04.
Next owner / action: Server returns the versioned sticky contract; General selects a comparison tuple; sdk-php adapts and measures. Asked Server again in chat today.