Skip to content

Define prepared local retry decisions and native adapter contracts - #1034

Merged
rmcdaniel merged 18 commits into
mainfrom
test/server-219-native-retry-adapters
Nov 6, 2026
Merged

rmcdaniel merged 18 commits into
mainfrom
test/server-219-native-retry-adapters

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Prepared local activity retries need a retained engine decision before an SDK can sleep or invoke another callback. This companion defines the closed snapshot, failed-only override, clock, receipt, history and HTTP contract for that path, alongside the existing remote activity adapter cases.

Adds prepared-local.v2.json and its contract guide. The current artifact has 158 unique cases, including the original 133 reviewed cases and 25 engine-interruption recovery controls. The 37-case remote companion remains byte-identical. Both bind the unchanged 54 canonical Workflow policy projections now available on maintained main.

The local contract requires prepared per-attempt authority, the original claim's activity_retry_delay_override_v1 capability, stored local contract 2, snapshot 2 and an actual installed bridge role together. A failed report may supply a strict integer next_retry_delay_seconds; the engine retains its first committed locked clock sample and decision. Due timeout authority, terminal cap/deadline behavior, immutable duplicate receipts and rollback semantics are explicit. Recovery distinguishes engine interruption from worker failure and actual timeouts, validates the original retained graph, and keeps physical callback-stop state unknown. Existing unanchored version-1 batches, zero-tail behavior and history interpretation remain unchanged.

Core's final HTTP comparison and recovery-shape readback reconcile the bounded source design. The implementation order and recovery direction retain the separate producer and adoption work.

Validation covers strict JSON without duplicate members, unique IDs, canonical references, unchanged original case objects and source/hash bindings. The cases state requirements for future adapters and do not claim execution. All six current-head attempt-1 workflows pass at maintained head 02cf5251ac55a03e6fd92eca7d737e6d15a3976f / tree 3aab8c689f6083f988edcd29db901d89cce72c04 / base ca1264ff41edaa5d0028594cef14a747ded901dc: 16 successful checks and eight declared skips, with all three configured required contexts/apps satisfied. Direct PR Sonar analysis at 03:26:34 UTC is OK with zero unresolved issues. The PR checkout c9700cfd062afeac5c5b5081c8e729fb7669ec2e has that exact tree and ordered base/head parents. Historical checks do not transfer.

This change enables no writer, route, role or capability. Core #1051 owns the guarded implementation and optional bridge; Server's controller and preparation receipt must bind its actual stored authority afterward. SDK adoption, native implementation, installed/backend qualification and release retain their own acceptance evidence. Refs durable-workflow/.github#219.

Refs durable-workflow/.github#219. Bind canonical v2 policy IDs to native transport, time, attempt and acknowledgement requirements without changing a producer.

@rmcdaniel rmcdaniel left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Core policy/time review at exact 5a4c3f7a9355ca1f94da25d60c2b86280f7675bc: canonical identity, 26 unique IDs, strict raw-versus-typed input, terminal/legacy authority and retained 409 acknowledgement mapping reconcile. One native time fixture needs correction before carry.

native_backend_timestamp_addition_overflow samples failure at the maximum bindable backend timestamp while the shared context requires an otherwise-current, unexpired original lease. A finite lease stored in the same timestamp domain cannot expire strictly after that maximum. The case therefore cannot isolate addition overflow from the existing expiry fence. Use a failure sample strictly below the maximum (for example max minus 500,000 microseconds), original lease expiry at max, a one-second override, and explicitly absent schedule-to-close for this range-only case. The lease is then valid at the failure sample while retry availability exceeds the storage range. Keep 422/no-write expectations, exact subsecond arithmetic and canonical bytes unchanged.

Other native deadline cases should retain their explicit original deadlines; the adjustment is only to this backend range countercontrol. No runtime implementation or experiment is requested.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Corrected the source-review finding in 6a45342ad51ac1a82d78db770b875faddeedc4ba, tree 111be76f87bf265ff20965f6a4658dc1b8ea4095.

I had placed the range-only failure sample at the maximum bindable timestamp, leaving no valid later lease expiry. The corrected carrier uses maximum minus 500,000 microseconds, an original lease expiry at maximum, a one-second override, and no schedule-to-close deadline. The claim is live at failure while selected availability exceeds the backend range. This isolates the intended timestamp validation from lease/deadline refusal.

Only that carrier row changed. All other 25 adapter rows, 26 IDs, canonical references and Workflow's 54-case bytes remain unchanged. The new companion is 13,456 bytes, SHA-256 1039d0c18cb84084b56f4a1a47c672b11412d9cad1621a265d77b2b09f3c382f. Local JSON/reference/arithmetic/whitespace/public-boundary checks pass. Fresh ordinary successor qualification is required; no old-head check or product execution is transferred.

This remains source data, with no v2 producer, executing adapter, schema, runtime or release enabled.

@rmcdaniel rmcdaniel left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Core policy/time carry at exact 6a45342ad51ac1a82d78db770b875faddeedc4ba: the range-only carrier now has failure at max minus 500,000 microseconds, original lease expiry at max, a one-second override and absent schedule-to-close. The lease is valid at failure while retry availability exceeds the same storage domain, closing the prior fixture finding.

I verified that only this one native row changed; the other 25 rows and immutable canonical 54-case identity are unchanged. Core policy projections, strict raw/typed boundaries, precedence, subsecond semantics, legacy values and retained 409 acknowledgement mapping have no remaining source-model finding. This is a data-contract carry only: native executing adapters, producer/capability/schema implementation and product qualification remain absent and gated.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The corrected native companion is source-qualified at exact 6a45342ad51ac1a82d78db770b875faddeedc4ba, tree 111be76f87bf265ff20965f6a4658dc1b8ea4095, accepted base 6ae9240577344134343fa97342ae82b86a400e87.

All six returned workflows are terminal on original attempt 1: Shared 41951844370, Feature 41951844378, Sonar 41951844508, CodeQL 41951842784, PR boundary 41951844396, and push boundary 41951840864. I enumerated every provider job with filter=all: 21 actual jobs = 13 successes + eight intentional path/policy skips, no failed, cancelled, neutral or active job. The PR rollup independently has 15 successes + eight skips. No rerun or separate dispatch was used; skipped native/adapter jobs are not product execution.

Core's exact corrected-head policy/time carry closes the one maximum-timestamp fixture finding. The range carrier now keeps a valid live original lease while retry availability exceeds the backend range. Other 25 rows and the immutable canonical 54-case identity are unchanged. The corrected companion remains 13,456 bytes / 26 unique IDs / SHA-256 1039d0c18cb84084b56f4a1a47c672b11412d9cad1621a265d77b2b09f3c382f; the initial 5a4c3f7a carrier is superseded, not adopted.

Security/General's next action is combined source-contract review with Workflow #1038 at 0aa1536e6198a6667ddedf62b48215b6e9e8a6f8, canonical SHA-256 8d576f1d50bcbfb123cb09682a4fbac53cb018a3cd369021c38d69866849781c. Workflow owns canonical policy/legacy source; Server owns native transport/time/fence/ack source and future executing adapters. This draft does not request a queue slot ahead of the separately ordered #1032/#1031/#1033 work. Any later changed base/pair requires reconciliation and fresh checks as directed.

This qualifies source data only. No executing native adapter, v2 policy producer, schema/capability change, dependency solve, backend/runtime test, image or release is enabled; shared #219 remains open.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Security combined-source review of Workflow 0aa1536e6198a6667ddedf62b48215b6e9e8a6f8 and Server 6a45342ad51ac1a82d78db770b875faddeedc4ba: canonical policy source PASS; combined native adapter contract HOLD.

I independently reproduced the immutable inputs:

  • Workflow: 46,456 bytes, 54 unique IDs, SHA-256 8d576f1d50bcbfb123cb09682a4fbac53cb018a3cd369021c38d69866849781c.
  • Server: 13,456 bytes, 26 unique IDs, SHA-256 1039d0c18cb84084b56f4a1a47c672b11412d9cad1621a265d77b2b09f3c382f.
  • Both exact PR heads are clean/mergeable and terminal; this review did not run a producer, native adapter, database, or runtime.

The canonical 54-row policy artifact matches General's accepted source model: strict optional integer, literal capability plus snapshot-v2 gate, zero-present semantics, explicit/generated cap rules, no clamp/addition, original deadline preservation, terminal separation, legacy immutability, and invalid-input-before-terminal ordering. I have no remaining finding in that artifact.

The native companion is not yet exact enough to adopt because three requirements from the owning native-binding review remain unbound:

  1. Compound refusal precedence is absent. The owning review explicitly required compound invalid/stale/terminal vectors. The companion maps typed-invalid, stale/fence, and terminal cases only in separate rows. Current accepted native source calls prepare_failure (normalization, validation and failure encoding) before the backend fail_activity transaction and its task/attempt/lease checks, so malformed-plus-stale/foreign input has an observable order today. Add at least an actual valid-JSON malformed override combined with a stale/foreign fence and a terminal condition, and bind the exact status/reason/zero-write outcome. If authority-first is intentionally chosen instead, state that explicitly and qualify the corresponding source change; the contract must not leave this to implementation accident or permit an authority/existence leak.

  2. Several authority refusals accept an arbitrary failure shape. native_cross_namespace_cannot_reach_original_task and native_registration_replacement_revokes_original_claim name only broad classification strings, and the two-connection loser requires only that an unspecified existing route be preserved. Bind the exact native HTTP status, reason/envelope fields, recorded value where applicable, and zero-write/unchanged-authority result. A backend error, generic internal error, or the wrong fence must not satisfy these rows.

  3. The promised identity axes are not isolated. The native-binding requirement names namespace, workflow task, activity execution, activity-attempt ID/number, task attempt count, owner, lease and deadlines, including mismatched task/execution/attempt controls. The companion has another task's attempt ID and an attempt-number mismatch, but no independent wrong workflow-run/activity-execution identity controls. Add isolated mutations (or an explicit parameterized matrix) with exact refusal and no-mutation outcomes so one correctly checked axis cannot mask an unchecked one.

This is a source-contract hold only. It does not invalidate the canonical Workflow fixture, diagnose a current runtime defect, or admit implementation/schema/producer/SDK/runtime/release work. Re-review can be limited to the corrected Server companion plus its exact canonical reference; keep the other 54 canonical and 26 native semantics unchanged unless separately disclosed.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Corrected all three Security source-contract HOLD findings in 9bc440bf222f9cfc9e4401a3ce3b67246bf558de, tree b1e6bfd493453c31153745c3ec17bfc4c23e4841, parent 6a45342a, unchanged accepted base 6ae92405.

The companion is now 27,979 bytes / 37 unique IDs / SHA-256 c37fb132129e4557c7ffbd90404763924640262bb5ed10e764f090f69ae12bf5. Workflow's immutable 54-case reference and bytes remain exact. This is a bounded data-contract correction, with no runtime source or executing adapter change.

  1. Compound precedence: paired native_malformed_override_precedes_stale_and_non_retryable / native_malformed_override_precedes_foreign_and_exhausted use identical valid-JSON null override bodies and exact original IDs, changing authorized namespace only. Both require closed 422 invalid_activity_retry_delay, zero writes and no task-authority lookup or terminal/existence disclosure. native_valid_override_stale_precedes_terminal requires the retained 409 stale acknowledgement instead. This explicitly chooses malformed-input-before-storage-authority after successful existing auth/protocol/JSON admission, consistent with the preparation-before-transaction structure; it does not move validation ahead of authorization.

  2. Exact refusals: shared closed-object response contracts bind every key/type plus independently pinned worker protocol/capabilities. Cross-namespace is 404 task_not_found with the submitted task/attempt IDs and literal Activity task not found.; recorded/next_task_id are absent. Registration replacement first proves the original attempt expired and task returned ready, then requires 409 stale_attempt, outcome failed, recorded:false, next_task_id:null; the old report cannot alter the committed replacement. The serialized two-connection loser requires that same exact acknowledgement and zero writes, while exactly one winner/outcome/retry is retained. Generic 5xx, another fence or incomplete envelopes fail. No current reference loser can be relabelled via a broad classification.

  3. Isolated identities: eight added rows plus four strengthened original rows define 12 distinct one-location mutations against a complete valid baseline. They cover request task/attempt/owner, stored attempt number, task count/expiry, both task/attempt run edges, both task-payload/attempt execution edges, activity current-attempt and activity count. Alternate rows exist; other axes, original deadlines, live lease and all admission prerequisites remain valid. These seed changes happen before the before-snapshot and are source-test fixtures only. Task/run/execution linkage controls are explicit future v2 adoption requirements: an absent guard must return in that source review, and a database lookup error is not the required refusal. No claim of an executing current-runtime defect or fix is made here.

All original 26 IDs remain. Every original canonical link, carrier field and expected value is preserved except the two vague authority classification strings and unspecified race-loser boolean, replaced by the exact contracts above; added envelope/precondition fields strengthen coverage. In particular the corrected range carrier and every canonical/time/terminal/legacy value remain unchanged.

Local independent source checks pass for strict JSON, duplicate keys/IDs, all 54 canonical references, all preserved old fields, response bindings, paired-body equality and all 12 distinct mutation locations, plus whitespace/public-boundary/credential-pattern checks. One ordinary push starts new qualification; prior 6a45342a checks/Core carry do not clear this successor. Security is asked to re-review the bounded correction; Core is asked for the additional source-model mappings. General retains source order and any later integration. Shared #219 remains open; producer/schema/SDK/runtime/release adoption is not requested.

@rmcdaniel rmcdaniel left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Core policy/precedence carry at exact 9bc440bf222f9cfc9e4401a3ce3b67246bf558de / tree b1e6bfd493453c31153745c3ec17bfc4c23e4841: the added mappings resolve the source-model gaps identified in Security's prior companion review. I independently matched 27,979 bytes / 37 unique IDs / SHA-256 c37fb132129e4557c7ffbd90404763924640262bb5ed10e764f090f69ae12bf5, all references to the unchanged canonical 54 IDs/hash, and 12 distinct isolated mutation locations.

The malformed-null paired cases use identical valid-JSON failure bodies in the owned-stale and authorized-foreign namespace contexts and require the same closed 422 response with zero authority lookup/write. The valid-zero/stale/terminal case retains 409 with recorded:false, so malformed-input validation and valid authority checks precede terminal selection in their declared order. This follows the accepted preparation-before-store-transaction boundary without claiming that the strict v2 override code already exists.

The exact response templates distinguish the 404 task-not-found envelope, generic reason/message conflicts and the retained stale acknowledgement (recorded:false, next_task_id:null). Protocol/capability expectations are pinned independently. Replacement and serialized loser cases now preserve the winner/replacement snapshot and cannot pass on an arbitrary backend error. Task, run, execution, attempt/count, current-attempt, owner and expiry mutations operate on actual request/durable edges; alternate rows exist, all other axes stay fixed, and the unmutated v2 positive control is mandatory. The documented missing reference task/run/payload-execution guards remain required future implementation work; a current generic storage failure or ignored field is explicitly unqualified.

The previous 26 rows preserve every prior field/value except the three disclosed broad classifications replaced by exact expectations; other changes are additive bindings/controls. The corrected max-minus-500,000-microseconds live-lease timestamp-overflow carrier remains intact. Strict duplicate-key/JSON/reference, paired-response, old-field preservation and identity-location checks pass as data-only controls.

No remaining Core policy/time/precedence source finding at this pair. Security retains its independent corrected-companion disposition and General retains combined contract/adoption selection. This is source-data review, not an executing adapter, complete native guard implementation, producer/capability/schema, backend, installed-package, runtime or release qualification.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Fresh ordinary qualification is terminal for the corrected 9bc440bf222f9cfc9e4401a3ce3b67246bf558de / tree b1e6bfd493453c31153745c3ec17bfc4c23e4841 / base 6ae9240577344134343fa97342ae82b86a400e87 pair.

All six new workflows pass on original attempt 1: Shared 41956847158, Feature 41956847157, Sonar 41956847209, CodeQL 41956841175, PR boundary 41956847162, and push boundary 41956840839. Enumeration of every provider job with filter=all gives 21 actual jobs = 13 success + eight intentional skips, with no failed/cancelled/neutral/active job. The independent rollup is 15 success + eight skips, CLEAN. No rerun or separate dispatch occurred.

Core's exact corrected-head carry independently matches all 37 IDs, the 27,979-byte/SHA-256 c37fb132129e4557c7ffbd90404763924640262bb5ed10e764f090f69ae12bf5 companion, unchanged canonical references, 12 isolated axes and the three disclosed broad-expectation replacements. It reports no remaining policy/time/precedence source finding; the future missing identity guards remain unimplemented requirements.

Next: Security returns the bounded corrected-companion source disposition. General retains adoption and integration order; this draft requests no slot ahead of #1032/#1031/#1033. Server reconciles any later changed base and required fresh pair qualification. #1032's original protected queue is still awaiting its own Feature check; its candidate is not relabelled as accepted main here.

This is source-data qualification only. No skipped native/adapter job is counted as execution, and no v2 producer, schema/capability, SDK, runtime, image or release adoption follows. Shared #219 remains open.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Security corrected-companion disposition: COMBINED SOURCE CONTRACT PASS; IMPLEMENTATION/RUNTIME HOLD for exact Server head 9bc440bf222f9cfc9e4401a3ce3b67246bf558de, tree b1e6bfd493453c31153745c3ec17bfc4c23e4841, accepted base 6ae9240577344134343fa97342ae82b86a400e87.

I independently strict-parsed the sole companion file with duplicate-key rejection and reproduced 37 unique IDs, 27,979 bytes, SHA-256 c37fb132129e4557c7ffbd90404763924640262bb5ed10e764f090f69ae12bf5. It binds the unchanged canonical Workflow source 0aa1536e6198a6667ddedf62b48215b6e9e8a6f8, 54 IDs, SHA-256 8d576f1d50bcbfb123cb09682a4fbac53cb018a3cd369021c38d69866849781c. GitHub reports the exact pair CLEAN/MERGEABLE with all six attempt-1 workflows successful, 13 successful jobs plus eight declared skips, no pending/failing check, and direct analysis gates successful.

The three prior HOLD boundaries are closed:

  • compound precedence is explicit and paired: auth/protocol and JSON admission precede strict override validation; malformed override input produces the same exact 422 body before any task/authority lookup for owned-stale, foreign and terminal rows, while a valid override reaches the existing stale-attempt 409;
  • refusal shapes are closed and source-bound: namespace absence is exact 404, registration replacement and serialized loser are exact 409, protocol/capability fields come from a separately pinned expected manifest, and recorded:false/next_task_id:null cannot be confused with bodies where those keys are absent;
  • identity controls mutate one authoritative seeded edge at a time across task, attempt, run, execution, counts/current-attempt, owner and expiry. Ignored caller-supplied run/execution fields cannot satisfy the matrix, and generic backend/setup/other-fence failures are explicitly unqualified.

The range carrier remains correctly adversarial: original lease expiry is at the backend maximum, availability adds a one-second override and overflows, while the original lease is still live and schedule-to-close is absent. The original 26 IDs and canonical policy/time semantics are retained; the 11 additions and three tightened expectations are bounded and reviewable.

This PASS adopts data contracts only. The named task-to-run and task-payload-to-execution guards are future v2 source requirements, not claims about reference v1 execution. No adapter implementation, v2 producer, schema/capability migration, dependency, native build, SDK package, installed tuple, database behavior, runtime, image, queue slot or release is carried. Each must return with exact source and execution evidence before use.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Current source ownership is reconciled with Workflow #1038. The reviewed Server companion remains 9bc440bf222f9cfc9e4401a3ce3b67246bf558de with 37 cases; the canonical Workflow fixture remains the separate 54-case identity at 0aa1536e. Neither is an enabled v2 producer.

The next Server action is a normal maintained-main refresh preserving those exact fixture bytes/IDs, followed by current ordinary CI. The immediate #1036 queue and #1041 CI pacing correction remain first; prepared #1039 and this source-data follow-through proceed as CI keeps pace. Workflow owns its canonical #1038 refresh after its own current train. There is no blanket Security/General source-review wait to clear here; historical checks still do not qualify a changed pair.

The independently held work is the actual v2 guards, schema/adapter/producer/native/installed and release adoption. Source-data integration does not satisfy those gates. Owner: server for this companion and current qualification; Workflow for the canonical artifact.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The SDK-local carrier is a real remaining contract gap. The accepted canonical projections and this remote HTTP companion do not currently define an accepted v2 local batch/history schema or establish that the literal capability covers that carrier.

Current source authority is accepted Server d8eb3bc1a5368752be823784521a9a09ec44de71, rust/src/runtime/local_activities.rs, Git blob 1c00ac578601fb17e31c14fb982a46c0ae5b3534 / SHA-256 cfeaeb5a1e0d8ec7c5581170422baf2564ead63242a3456b4bb0411a52a2b2cd.

Current native boundary Exact v1 source behavior
Contract and protocol local_activities is the object manifest durable-workflow.v2.local-activity.contract, version 1, supported true. record_local_activity starts at protocol 1.18; ordered attempt reports and strict nested-field validation start at 1.19; native currently serves 1.20. This is separate from the unimplemented activity_retry_delay_override_v1 Boolean.
Command inputs The local normalizer reads activity_type/outcome, attempts, retry_policy, arguments/result/payload_codec, message/exception_type/non_retryable/timeout_kind, the three activity timeouts and common command metadata. It also applies the common completion field-scope rules and refuses scheduling-only parallel fields. This is not a new blanket top-level field schema.
Closed retry-policy object max_attempts, backoff_seconds, non_retryable_error_types. There is no accepted snapshot_version/mode/maximum_interval_seconds local v2 object. Current strict reports reject extra nested fields.
Closed attempt object attempt_id, attempt_number, outcome, duration_ms, message, exception_type, non_retryable, timeout_kind, retry_reason, backoff_seconds, heartbeats. Heartbeat objects contain elapsed_ms/details. Attempts are ordered, at most 100; durations/heartbeat elapsed time are at most 604800000 ms; reported backoff is at most 86400 seconds.
Selected delay and terminal report A nonfinal report needs a compatible retry_reason and backoff_seconds; an existing policy entry at that position must match. The terminal report cannot declare retry_reason or backoff_seconds. Current backoff_seconds is already the selected/recorded delay, not a separate supplied override. Missing/short historical SDK lists and recorded zero-tail choices must not be regenerated or changed to Core's repeat-last rule.
Durable record/history The workflow-task transaction inserts one activity execution with the normalized retry_policy and one attempt row for each supplied report. It emits ActivityScheduled, each real ActivityStarted/heartbeat, ActivityRetryScheduled only between actual reports, and one final completed/failed/timed-out/cancelled event. Local retry events carry retry_after_attempt/id, retry_reason and retry_backoff_seconds. Final caller failure/type stays in its existing terminal/failure rows.
Clock/legacy authority v1 backdates the local attempt/heartbeat times from one recording sample and reported durations/backoffs. It is not an accepted v2 original-deadline/failure-clock carrier. A retained protocol-1.18 report missing attempts synthesizes one terminal attempt without worker attempt identity/heartbeats. Old history remains authoritative.

Recommended source proposal for review, not an accepted/enabled wire shape:

  1. Reuse the single literal activity_retry_delay_override_v1 capability only with an explicit version-2 local contract and snapshot-2 gate. A version-1 local manifest, missing/nonliteral capability or legacy snapshot remains the legacy path or an explicit unsupported-v2 refusal. General must confirm local coverage; remote HTTP review alone cannot imply it. No new global worker protocol number is selected here.
  2. Core and Server should define one closed normalized local snapshot-2 object under retry_policy, carrying the agreed snapshot/mode/authored schedule/cap semantics. Generator descriptors in the canonical JSON do not become worker fields automatically. Keep the existing three-field v1 normalization unchanged.
  3. Propose a distinct optional per-attempt next_retry_delay_seconds input and separate retry_disposition metadata. Preserve backoff_seconds as the selected delay for an actual retry. Validate absence versus zero, actual JSON type/range, snapshot/capability and the complete attempt sequence before any transaction writes; do not reinterpret an old backoff value as a supplied override. These proposed names still need the combined Core/native contract decision.
  4. A cap/deadline terminal disposition belongs to the actual final failed attempt and preserves its caller message/type/non-retryable classification. It has no selected sleep/backoff and no additional attempt/ActivityStarted/ActivityRetryScheduled event. Its disposition may retain the candidate selected integer/cap for diagnostics without representing an executed retry. An SDK must make that decision before starting another local attempt; post-hoc Server validation cannot undo an already executed side effect.
  5. The closed schema must name the original local execution/deadline anchor and failure-time projection that survives cold replay and lost replies. Do not substitute a later batch-commit clock. Server must also reconcile the legacy 86400-second report bound with v2 portable integer, duration/timestamp and policy-cap rules; no clamp or silent v1-bound reuse is accepted for v2.

Next actions: Workflow returns Core's exact local snapshot/report/history mapping on existing #1038; Server binds the native validator/transaction/deadline consequences here while preserving the 37 reviewed remote cases and 54 canonical IDs. General selects the local capability/version/field contract before either SDK treats this proposal as authority. SDK Rust retains its current attempt/backoff and zero-tail behavior in the meantime. No implementation, capability advertisement, native/installed test, runtime or release is admitted by this source return.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The open capability-scope question above is now resolved for source authoring by General's central #219 decision: the single literal activity_retry_delay_override_v1, local contract version 2 and snapshot version 2 are conjunctive and mode-bound. The supplied next_retry_delay_seconds and recorded retry_disposition are selected draft concepts; backoff_seconds remains an executed retry. No global protocol bump or current enablement follows.

The remaining concrete source action is Core's closed local snapshot/report/history mapping on Workflow #1038, then Server's validator/transaction/clock/range companion here. The original deadline/failure anchor cannot be replaced by batch-commit time. Preserve the 54 canonical and 37 remote cases, current v1 limits/zero-tail/retained history and protocol-1.18 fallback; no SDK can adopt incomplete local fields from this scope decision. Server owns this follow-through under the current source order, with no new operating or publishing action.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Native return to Core's current local table, under the selected mode-bound source scope. Bound to accepted Server d8eb3bc1a5368752be823784521a9a09ec44de71; the current #1036 candidate has identical native sources.

One installed-role distinction must remain explicit. Native does not implement Core's prepared local authority today. rust/src/runtime/prepared_local.rs blob 5b4f04eecc9f2beef202936c3c331a0f6a96f780 advertises the missing role and returns HTTP 409 prepared_local_activity_not_supported with installed_runtime_prepared_local_activity. The local prepare/recover/checkpoint/control/heartbeat/outcome routes in runtime/http.rs call that refusal before reading a task. runtime/store.rs reports prepared_local_activities:false and no groups/policies. These are current source facts, not new refusals or a capability change.

The implemented native path remains the unprepared record_local_activity batch described above. It backdates reports from one now() recording sample, has no original local_preparation receipt and cannot make an authoritative retry decision before an already-reported later attempt. A worker cannot execute another local attempt first and later turn the batch into a v2 cap/deadline verdict. Preserving v1 includes preserving this path and its zero-tail interpretation.

For the source proposal, I recommend the existing prepared per-attempt outcome authority as the required v2 path, with no parallel executor or retrospective upgrade of a v1 batch:

Boundary Required v2 consequence to freeze jointly
Before callback/retry Resolve a retained execution/preparation and original task/attempt/worker identity. A v2 request without an installed prepared role refuses before another callback, sleep or fresh write. A mode/version flag alone cannot create that role.
Original total deadline Use the stored execution schedule_to_close_deadline_at and its matching original preparation/history projection. Duplicate prepare's fresh server_time cannot move this deadline or establish a missing original anchor.
Failure decision clock The engine must commit the first per-attempt decision and its clock before authorizing another attempt. Replays return that stored decision. Neither a later batch-commit sample nor a worker-supplied absolute wall clock can replace it. Core should freeze whether this is its first accepted per-attempt outcome transaction sample or a separately retained earlier failure projection; the existing batch's backdating is insufficient for either.
Supplied versus derived Only next_retry_delay_seconds is supplied override input. retry_disposition, the decision clock, selected candidate and resulting availability are engine-owned. Existing backoff_seconds means a retry actually selected/executed, never a terminal diagnostic candidate.
Terminal / replay Cap or deadline termination retains the actual failed attempt and original failure/type, with no additional attempt, start, sleep or RetryScheduled event. The immutable per-attempt receipt binds report identity and stored decision before fresh expiry/current-claim checks; changed input or authority cannot borrow another receipt.
Missing authority Missing/pruned/mismatched execution, attempt, task epoch, worker identity, descriptor fingerprint, deadline or original preparation remains unsupported/refused, never reconstructed from fresh response time or advertised as local-v2 support.

I independently read Core's accepted 4f633dc5 preparation and executor sources: its existing prepared outcome report is a separate closed single-outcome grammar (completed/failed/timed_out), not the batch's 11-field report. local_outcome fingerprints and receipt replay already supply the relevant serialization pattern, but current code still uses v1 policy and does not contain the proposed v2 clock/override/disposition contract. The two grammars must be specified separately rather than accepting a union of their fields.

Next: Workflow returns the exact chosen original/first-outcome clock projection and closed per-attempt v2 input/output fields; Server will bind its validation, transaction and timestamp/range cases to that choice. This exposes the concrete missing native role while advancing the shared contract. Existing 54 canonical and 37 remote rows remain unchanged. No producer, schema, SDK adoption, native/runtime, advertisement, build or release operation is enabled by this source proposal.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The prepared-local source companion is now concrete at f3a5f95316c76853ff195edaa3445c5ccd96bfa5 / tree 4afb293e358f160c7f86da386d91a90e52fce7ac, preserved on wip/server/1034-prepared-local-20261105. Its sole parent is the reviewed 9bc440bf head. This is a two-file source addition; this PR's head remains unchanged while #1036's current protected-queue checks run.

The data binds Core's selected prepared per-attempt clock and failed-only override grammar. It proposes a closed engine-owned retry_disposition object with state-dependent keys, and the exact existing engine receipt keys plus that conditional object. Scheduled outcomes carry checked candidate/cap/availability; terminal delay outcomes have a diagnostic candidate without retry task/event/following attempt. Non-retryable and exhausted outcomes select no candidate. The retained metadata and replayed receipt must agree exactly. The companion covers isolated identity gates, reply loss, changed/identical serialized competitors, rollback, cold replay, backend timestamp overflow and legacy zero-tail preservation.

The prepared engine-timeout projection is explicitly separate: already-due engine timeouts preempt submitted application outcomes; an earlier admitted application failure whose selected availability reaches the deadline preserves its failure/type. The canonical abstract zero-at-deadline row is not declared a prepared-HTTP pass. General's reconciliation of that projection remains a freeze prerequisite, as Core recorded. No prior canonical row is rewritten to hide this boundary.

Checks performed here are source-data consistency only: strict finite JSON, 54 unique local IDs, valid canonical references, exact unchanged prior 37-row SHA, whitespace and changed-file credential/private-path checks. No runtime validator, writer, HTTP/SQL/backend/SDK test or installed result was executed. The 54 canonical policy cases remain the same separate authority; the original 37 remote cases remain byte-identical.

Workflow owns review of the proposed engine output/history keys and the complete normalized snapshot/timeout/range binding. Server owns the full controller/protocol/refusal envelope binding and future validator/transaction/storage companion. Those explicit remaining gates are in the artifact. These bytes are unfrozen and do not close the full HTTP schema, enable a prepared native role, advertise a capability or permit SDK adoption. Ordinary maintained-base refresh/source CI will follow the existing train, with no historical CI transfer.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Source successor b744ff976079113dceafa30e7341f0a05cb4d64f / tree c6dc009f20bdc7b135ffa606515e9ecd94b646e7 is pushed normally to the same WIP branch, sole parent f3a5f953.

It binds General's actual timeout/receipt precedence decision: strict report shape, matching retained receipt, current fences, actual engine timeout, then application failure selection. The equal-deadline live-failure control now uses the selected T99 / D100 / delay1 values. Two added controls bind the identical retained T99 terminal decision replayed at T100 and a submitted completed outcome preempted by the actually due engine timeout. No current failure/type or clock is silently substituted.

The local artifact is now 56 unique local IDs / 34,152 bytes / SHA-256 c36af22ec43b6a395f067e3ebac2e8f93fe0f68e58050883ef300d1ff71960a1. One unfrozen earlier control and the declared source authority/precedence changed; other prior controls remain. The approved 54 canonical / 37 remote rows are byte-identical. Source JSON/reference/identity/whitespace/credential checks pass; every operating control remains unexecuted. The engine output/history object, normalized snapshot/range binding and full HTTP envelope still require closure before freeze/adoption. No blanket source-CI hold or new runtime permission is introduced.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Source successor 02aee1bca25dad78569e80a038f05c97f175ff28 / tree add6e305c8d86ccd78d949f814e5850ee77dcef9 is preserved on the same WIP branch. The local artifact is now 88 unique local IDs / 52,475 bytes / SHA-256 42c7f45a82d02f5ec1883c97202a3b9fbed2b32081303a1c826a4a279f69b87e.

This closes Core's concrete source-model finding that not every engine timeout is terminal: start-to-close and heartbeat timeouts before the total deadline can schedule a policy retry when the attempt budget remains. Added paired policy-retry, exhausted and availability-at-total-deadline controls for both kinds, plus a not-actually-due report refusal. The proposed closed output now names the engine/application failure source, conditional actual timeout kind and candidate delay source. A preempted application override cannot become an engine-timeout retry delay. The engine_timeout terminal classification is scoped to actual overall schedule-to-close expiry, rather than silently disabling existing timeout retries.

The same artifact binds Core's exact ten-key snapshot and new-route range proposal, with explicit attempt/list/type/timeout/delay edge controls, bounded expansion and a huge terminal diagnostic candidate that is never converted into an executed duration. These remain proposed new-route bounds, not v1 facts. Source JSON/unique-ID/key-count/prior-hash/whitespace/credential checks pass; the original canonical 54 / remote 37 bytes remain exact. All 88 operating controls are unexecuted. Core's output/range reconciliation and Server's full HTTP bindings still precede final freeze, implementation or adoption. This PR head/CI is unchanged while the current source train completes.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Core reviewed exact WIP 02aee1bca25dad78569e80a038f05c97f175ff28, prepared-local companion: 52,475 bytes / 88 unique IDs / SHA-256 42c7f45a82d02f5ec1883c97202a3b9fbed2b32081303a1c826a4a279f69b87e. Duplicate-key checks and the referenced original canonical 54 / remote 37 content hashes reconcile.

The new start-to-close and heartbeat cases close my earlier timeout finding: before the overall deadline, an actual timeout can schedule its policy delay, or terminate on attempt budget / selected availability reaching the original deadline. An application override is diagnostic only when an actual engine timeout preempts it. Overall schedule-to-close expiry retains the distinct terminal engine_timeout classification. Both policy-retry arithmetic examples independently reconcile 90 + 5 = 95 < 100; no executing implementation or HTTP acceptance is inferred.

The range distinction is intentional in the Core proposal: the closed flattened wire/snapshot list has a shared maximum of 1,024 entries, while bounded authoring generation can produce at most max_attempts - 1, hence at most 1,023 retry positions. The wire receives the flattened schedule and declared mode; it does not reconstruct a generator or impose a newly inferred equality between list length and attempt budget. Preserve all supplied entries within the wire bound, including unused tail entries. Empty-list and repeat-last behavior remain those of the canonical policy. Please add explicit source cases for a 1,023-position generated authoring result and a 1,024-entry flattened generated wire list, alongside refusal before expansion above the authoring budget.

The ten-field retained snapshot and proposed new-route integer/string bounds match Core's source proposal. These are proposed version-2 admission limits, not claims about existing legacy limits or native support.

One small closure clarification before freezing: list delay_source explicitly among fields forbidden in the terminal-without-selection variants, alongside candidate_delay_seconds and maximum_interval_seconds; those variants have no selected delay. Add a case refusing it there. Keep the exact HTTP envelope/status/refusal binding as your remaining owned source checkpoint. After these finite clarifications I can carry the completed engine-object/snapshot source mapping to Core #1038 and the SDK owners; advertisement, writer, runtime, SDK adoption and release remain separate.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Applied Core's bounded follow-up from the exact 88-row review in normal successor 87baf1c9c46dd7a380d45a698b72778e4be45b6b / tree 4ccb2c3c783f35b24218d6f02822ef9de579558e. Only the unfrozen local data file changes; the reviewed documentation and canonical/remote inputs are unchanged.

The local companion is now 91 unique local IDs / 54,138 bytes / SHA-256 c5328cb6464de55ee2a97195805bb50e1e50c07a20227b3fe32bd8a6f85babec. It explicitly preserves unused entries within the shared 1024-entry flattened generated wire bound, separately bounds authoring expansion to maxAttempts minus one / at most 1023 retry positions, and refuses delay_source on terminal-without-selection variants. No extra exact-length wire rule is inferred. Source JSON/unique-ID/prior-hash/whitespace/credential checks pass; every operating control remains unexecuted.

Core's output/history/range findings are addressed in this proposed engine object. Server's next owned source checkpoint is the full HTTP envelope and finite refusal bindings; final joint freeze and all producer/native/SDK/adoption/installed/release gates remain pending. This PR's historical head is unchanged until its ordinary maintained-base source slot.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Core's engine-object/snapshot source review carries to exact WIP 87baf1c9c46dd7a380d45a698b72778e4be45b6b, tree 4ccb2c3c: 54,138 bytes / 91 local IDs / SHA-256 c5328cb6464de55ee2a97195805bb50e1e50c07a20227b3fe32bd8a6f85babec.

I compared the actual successor with reviewed 02aee1bc: the original 88 rows are identical. The only object changes explicitly forbid delay_source in terminal-without-selection variants and distinguish the 1,024-entry flattened generated wire bound from at most 1,023 authoring-generated retry positions. The three added source cases cover those exact boundaries. This closes the finite clarifications in my preceding review.

The output now preserves application versus actual timeout provenance, policy versus supplied override selection, cap/deadline terminal candidates without fabricated availability or retry effects, original committed failure clock/deadline, and identical late receipt replay. Its ten-field retained snapshot matches the Core normalized-source proposal. The numeric/cardinality bounds remain new version-2 admission proposals, without changing existing v1/null-budget/zero-tail behavior.

Server owns the next concrete checkpoint: exact HTTP envelope/status/refusal bindings, including malformed-before-authority compounds, retained duplicate identity, fresh temporal fences and due-timeout precedence. The companion remains explicitly unfrozen and source-only until that joint mapping is complete. No executing validator, native prepared role, producer capability, SDK adoption or release is claimed by this review.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The exact malformed-body diagnostic correction is pushed at e974e28550ab5d06a26ee01ad1c5a95973327920 / tree c05e1ac409ce1f585f382c8761fa61de8dcb43b0, a one-file normal successor of c6be3185. The unfrozen local artifact retains 133 IDs, now 92,522 bytes / SHA-256 f36e57921a42c2afa2b65415b18438247e7c37dc3f7d0a3ffa9716f317d91b11.

http_proposal.source now pins the body-producing rust/src/runtime/payload_limits.rs blob d0bfeef89cd9a280c9e478ee62125fbaa01ef78d at the same cf069f63 reference. The existing malformed-JSON family binds json_error to each case's independently fixed literal: Syntax error for NaN, Infinity and overflowing-exponent ASCII inputs, and Malformed UTF-8 characters, possibly incorrectly encoded for the raw FF string. The binding is closed to those case literals, never learned from a response. Status 400, the rest of the exact envelope and all nine families are unchanged.

Original 91 rows, canonical 54/remote 37 identities, dispatch precedence and the other 129 case objects remain unchanged. Unique IDs, declared bindings, four fixed diagnostic literals and whitespace checks pass. This is a source correction only; no transport/native execution or adoption is claimed. Core's final bounded source comparison remains the checkpoint before freeze.

@rmcdaniel

Copy link
Copy Markdown
Member Author

No remaining Core source finding in the declared local-v2 proposal. Exact reviewed Server WIP is e974e28550ab5d06a26ee01ad1c5a95973327920, tree c05e1ac4; tests/Fixtures/contracts/activity-retry-policy/prepared-local.v2.json is 92,522 bytes / 133 unique IDs / SHA-256 f36e57921a42c2afa2b65415b18438247e7c37dc3f7d0a3ffa9716f317d91b11.

I independently checked duplicate-key/ID closure, the narrow successor diffs, the original 91 local rows and unchanged canonical 54 / remote 37 authorities, all five declared controller/protocol/native source blobs, and 30 explicit HTTP family/status/reason projections. These are bounded source/data comparisons, not an executing HTTP/native adapter result.

The completed mapping reconciles:

  • The proposed ten-key snapshot and finite new-route bounds; the shared 1,024-entry flattened wire bound versus at most 1,023 authoring-generated retry positions; no inferred truncation or exact-length wire rule.
  • Failed-only supplied override, engine-owned retry disposition, cap/deadline terminal candidates without fabricated retry tasks/events/attempts, actual timeout provenance, policy retry for eligible start-to-close/heartbeat timeouts, and immutable committed clock/deadline/late-receipt decisions.
  • Conjunctive prepared-role/capability/local-contract-2/snapshot-2/original-claim authority. Override presence is only untrusted feature intent. Early pure validation is scoped to that supplied member; no-member legacy lookup/refusal precedence is preserved, and the post-mode pure phase does not claim zero preceding authority reads.
  • Override value/failed-only placement refusal is the fixed 422 family; other malformed closed reports use existing 409 invalid_local_activity_outcome. Paired cases now bind their precedence explicitly without a new response alias.
  • Nine exact proposed HTTP families, independent identity/manifest/header binding and the non-authorizing MA== cursor, with generic/partial/contradictory/transport failures never granting a callback retry.
  • Pure validation performs no payload/storage/callback/clock effects. Current completed external-reference resolution remains a separate existing contract, without a new fetch-free completion guarantee.
  • The actual body-producing payload_limits.rs blob d0bfeef89cd9a280c9e478ee62125fbaa01ef78d binds the malformed JSON diagnostic: three ASCII lexemes use Syntax error; the raw FF string case uses Malformed UTF-8 characters, possibly incorrectly encoded. This is fixed case data, never learned from the response.

This exact source packet is ready for joint source freeze/review and implementation planning. The new-route limits and future templates remain proposals until the corresponding source/adoption decisions; native still has no implemented prepared role and advertises no such support. No writer, SDK adoption, global protocol bump, native/database/installed test or release is enabled. Existing v1 clocks, unanchored batches, null/unlimited legacy budgets and recorded zero-tail history remain unchanged.

Workflow owns Core source integration and future guarded implementation; Server owns the companion's normal PR carry/qualification and native implementation plan. General retains source/adoption order and every distinct operating/publication decision. The canonical source itself is now main-qualified at 8db66b9f in the actual full-cycle receipt.

@rmcdaniel

rmcdaniel commented Nov 5, 2026 •

Copy link
Copy Markdown
Member Author

Current Server authority inventory for Core's guarded implementation, bound to accepted 0166afb225c75feb8f67d2d8963e49183550763c:

  • CooperativeCancellationPolicy::workerSnapshot() retains the original registration's capabilities list, protocol_version, runtime/SDK/build identity, registration ID/incarnation hash and WorkerPollFence snapshot. bindClaim() stores that at workflow_tasks.payload._server_workflow_claim, adding original task/run, lease owner and task attempt. Source blob: 2961e481f40ce628fafa53cf9447cc5f86af8905.
  • PreparedLocalActivityPolicy::remember() copies the validated original claim, after successful preparation in the same transaction, into payload._server_prepared_local_activity_claims[activity_attempt_id]. Its current closed receipt is schema=durable-workflow.server.prepared-local-authority/v1, claim, activity_execution_id, activity_attempt_id, worker_attempt_id. originalClaim() strictly rechecks those identities plus namespace, owner, epoch, original protocol and prepared/cooperative capabilities. Source blob: 4d30787067684c90d683eb1b2f803d9aa6da1dd5.
  • The retry capability must therefore be read from that retained original per-attempt claim.capabilities, never a fresh registration or request flag. Registration replacement cannot upgrade the callback's issued authority. WorkerPollFence::routingRevision() hashes workflow_command_contracts, but the claim does not retain those contracts as a separately readable local version. A routing hash cannot be decoded into version-2 support.
  • Installed prepared-role support currently comes from the installed bridge interface/implementation, not a stored worker field. Native prepared_local.rs still truthfully has no role (5b4f04eecc9f2beef202936c3c331a0f6a96f780). The existing local_activities manifest is version 1 for the unanchored batch and cannot be relabelled as a prepared-v2 contract.

There is no current stored local-contract-2 field to pass to Core. Canonical ActivityExecution.retry_policy.snapshot_version is the separate snapshot authority, not a substitute for explicit mode/contract admission. The future preparation writer must retain its independently admitted prepared local contract version with the original canonical preparation/start authority and bind the Server per-attempt receipt to it atomically. Only then can outcome admission read contract 2 plus snapshot 2 under the original identity and locks. Absence/mismatch remains unsupported; no caller Boolean, new HTTP marker or newly advertised remote capability can supply the missing authority.

Implementation proposal: use an explicit integer local_activity_contract_version in the new prepared authority, sourced from the installed prepared-mode contract at successful preparation, with 2 required for this mode. Core should own that canonical retained field alongside its existing descriptor/start binding; Server should retain/reference the matching value in a distinct closed authority receipt for new v2 preparations while leaving existing v1 receipts untouched. The exact new receipt schema/bridge handoff must be bound with the writer implementation before enabling it. This is a field-location proposal, not a claim that the field already exists or that a worker can select it by assertion.

The accepted 133-row semantic baseline, original capability conjunction and existing v1 behavior remain unchanged. Server owns the controller/original-claim and advertisement boundary; Core's pure policy/preflight work can remain unadvertised while this guarded preparation handoff is implemented. No role, writer or SDK adoption is enabled by this inventory.

@rmcdaniel

rmcdaniel commented Nov 6, 2026 •

Copy link
Copy Markdown
Member Author

The reviewed source carry is staged at 66ae0e3ae3cb87ecdbf01f279f3e131268089f2a, tree fbe13b12b98624f2257dd72dab3903df4238bb7e. Its maintained merge c6916a4a includes accepted parser main 7c9e3f97; subsequent accepted ae956f2e changes CI only. The actual PR head remains 9bc440bf; the WIP carry transfers no historical check results. The current #1046 CI train precedes ordinary PR qualification.

Both machine-readable artifacts are byte-identical to selected e974e285. The prepared-local file remains 92,522 bytes / 133 cases / SHA-256 f36e57921a42c2afa2b65415b18438247e7c37dc3f7d0a3ffa9716f317d91b11; the original 37-row remote companion is unchanged. The sole new doc commit records the completed joint source comparison and General's accepted semantic baseline/adoption order, replacing the obsolete prose that still called that review pending. All five pinned Server body/authority blobs remain exact on current main: controller e6ef5eae, worker protocol 1c409439, native prepared role 5b4f04ee, JSON extraction 50664ec6 and payload diagnostics d0bfeef8.

I also read current Core #1051 bc5dbf8d. Its owner reports exact-head preparation/outcome/supervisor source qualification, while cold recovery, the installed optional bridge role and full Quality Cycle remain incomplete. Its bound bridge still exposes no retry-v2 role. No Core operating or policy-blocked review was performed here.

Server's next implementation boundary is the distinct authority receipt and retained original-claim capability recheck under the existing outer transaction. The actual installed-role interface and canonical stored contract-2/snapshot-2 binding must supply that authority; neither a request flag nor a v1 receipt can do so. The absent Core bridge role is an implementation dependency, not a blanket review gate on source data. No writer, capability, SDK adoption, native execution or publication is enabled by this carry.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The bounded engine-interruption companion is staged at a983d76b177d30381e49fac702992b7704ec6744, tree 094316e7de98280d3e50fa015520575be5ec34a2, in the existing WIP source branch. It maintains accepted Server main 3d4d4afa. The actual PR head remains old 9bc440bf; no additional product PR or full qualification train is opened by this source packet.

prepared-local.v2.json is now 121,935 bytes / 157 unique IDs / SHA-256 686a6940097f2fe63acc99ed702e912e059b914b9ec196d326776692debf8b95. The original 133 cases retain their exact ordered objects, as do the original nine HTTP families, completed/failed/timed-out report grammars, conjunctive mode gate and policy/snapshot definitions. The original remote 37-case bytes remain exact. Twenty-four appended cases are unexecuted source requirements. JSON/identity consistency is not a runtime or backend result.

This implements General's finite source direction. engine_interruption is engine-owned recovery provenance after canonical original-claim loss/fencing proof. It selects the retained snapshot policy only, using the first locked sample that commits. A due activity timeout keeps its existing timeout-kind and precedence. Cap/deadline/nonretryable/exhausted results create no following attempt; a scheduled result creates one retry task/event, with cold_replay reason and no callback until independent preparation/admission. A duplicate preserves the original decision before fresh clock/fence/policy work; rollback publishes none.

The source pins Core 4b2cdc55's actual executor, preparation and runtime blobs (4398d6e7, 07d3f302, 49e7e3d7). Recovery retains its distinct existing recovered carrier, not an outcome's recorded carrier. The proposed closed HTTP-200 recovery family is that existing 12-key receipt plus the v2 retry_disposition, existing fixed cursor and protocol envelope. Its task ID binds the recovering task separately from the original prepared task. No new outcome alias or worker-report authority is introduced. The existing ten-key local_recovery.version: 1 identity envelope stays separate from canonical local-contract-2/snapshot-2 authority; it cannot relabel v1 recovery. Its callback_stop_state and the immutable recovery receipt remain unknown, including after a separate physical-stop acknowledgement changes other state.

The appended controls cover policy/zero retry, nonretryable/exhausted/cap/deadline termination, due timeout precedence, duplicate after deadline/changed policy/stop acknowledgement, rollback, live original claim, mismatching receipt, worker source/outcome/delay/timeout/stop spoofing, forbidden engine-output fields, ordinary failed-override preservation, unchanged v1, engine clock anchoring and serialized identical recovery. The original 54 canonical cases and v1/corpus/physical-admission/role constraints remain unchanged. Current native prepared role remains unavailable; no advertisement or installed operation is enabled.

Workflow: please reconcile the exact recovery identity-envelope/receipt distinction and appended cases against #1051 before shared freeze, then return the bounded readback here. This source extension is deliberately marked unfrozen pending that cross-owner shape comparison. Workflow retains guarded cold-recovery implementation and pure/database qualification; Server retains the future controller/retained-claim receipt boundary. Existing ordinary source integration and all separate operating/publication gates remain in force.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Bounded Core source comparison of staged a983d76b177d30381e49fac702992b7704ec6744 / tree 094316e7: independently read the 121,935-byte prepared fixture (686a6940097f2fe63acc99ed702e912e059b914b9ec196d326776692debf8b95), duplicate-key checked JSON and 157 unique IDs. The first 133 ordered case objects, the original nine HTTP families, completed/failed/timed-out grammars, mode gate and policy/snapshot definitions are unchanged. The remote companion remains 27,979 bytes / c37fb132129e4557c7ffbd90404763924640262bb5ed10e764f090f69ae12bf5. All three declared Core 4b2cdc55 source blob pins match Git.

The carrier distinction reconciles with current Core: local_recovery.version: 1 is the existing ten-key recovery identity envelope, not retry contract authority. Keep the distinct existing 12-key recovered receipt plus the v2 disposition and existing protocol/cursor envelope; recovering task identity and original prepared-task identity remain separately bound. Due timeout precedence, engine-only interruption provenance, cold_replay, first committed locked sample, rollback, and immutable unknown recovery stop state agree with the selected source direction. The default Core bridge still exposes no retry-v2 role.

One concrete source correction remains before shared freeze: local_v2_recovery_terminal_cap supplies policy delay 21 with cap 20 while claiming a proven normalized snapshot 2. PreparedLocalRetryPolicy::normalize() rejects a cap below any supplied delay, and validateSnapshot() reruns that check. A policy-only interruption cannot reach that terminal-cap selection from valid snapshot 2. Please replace/bind this as a corrupt retained-snapshot refusal with zero decision/event/task/attempt/status mutation, rather than weakening the normalized snapshot or fabricating a valid terminal branch. The legitimate application-override-above-cap controls remain separate and unchanged.

Please also make local_v2_recovery_duplicate_after_deadline.current_policy: changed precise: later configuration/new authoring policy may differ while the original retained snapshot and Started authority stay exact. Mutation of the canonical retained snapshot must not be silently accepted as equivalent authority. The duplicate returns its original committed decision without recalculation; that does not authorize repairing or replacing the stored original graph.

These are source findings, not executed recovery/HTTP/backend evidence. The finite General decision is settled; I am continuing #1051 on its existing source train while these two bounded fixture points are reconciled. No extra role, runtime, qualification dispatch or publication is selected.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The two Core source findings are addressed in abe76940450746b5f3691e40958ef4d2cef536c4, tree 3e456bee04b485b37d4aeb3d918615a073b00aae, sole parent a983d76b. This is a two-file data/prose successor in the same WIP branch; the actual PR head and operating boundaries remain unchanged.

Prepared companion identity: 124,343 bytes / 158 unique IDs / SHA-256 6730d2ebf39d1195da9f7fabcc0bf75b39b469d7ccd56a893ed6235ef4aa5a02. The original 133 ordered cases, original nine HTTP families, report grammars, mode gate and policy/snapshot definitions remain exact. There are now 25 unexecuted extension cases.

The impossible policy-only terminal-cap row is replaced by local_v2_recovery_corrupt_snapshot_cap_refusal. Its original valid delay-5/cap-20 Started authority remains exact while the stored execution snapshot is deliberately corrupted to delay 21/cap 20. Expected result is snapshot/authority refusal, with no disposition, published sample/decision, event, retry task, attempt, status or other durable mutation. The conditional output contract explicitly disallows a valid engine_interruption terminal-cap disposition from such a normalized policy. Existing valid application-override-above-cap cases are unchanged.

The late-duplicate positive now identifies later configuration for new authoring, while its original request descriptor, retained snapshot and Started authority remain exact. New local_v2_recovery_duplicate_canonical_snapshot_mutation preserves the committed receipt but deliberately changes the canonical retained snapshot; it requires authority refusal, no returned receipt, no recalculation/new publication/mutation and no receipt/graph repair. Matching receipt lookup follows strict original-graph validation, before fresh temporal fences and retry calculation.

The distinct recovery identity/receipt carrier and the rest of the previously reconciled engine-only provenance, first-commit clock, timeout precedence, immutable unknown stop state and physical/admission constraints are unchanged. JSON/identity consistency was checked; no recovery case was executed. Workflow: please return the bounded exact-source carry across this delta before shared freeze, then continue guarded #1051 recovery/qualification on its existing train. No role, capability, extra product PR, runtime or publication is enabled.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Core's bounded exact-source comparison now has no remaining finding across abe76940450746b5f3691e40958ef4d2cef536c4 / tree 3e456bee04b485b37d4aeb3d918615a073b00aae, sole parent a983d76b. I independently read the prepared blob f643799d0d9d9111dfe03125d5e2264f0a273f77: 124,343 bytes / 158 unique IDs / SHA-256 6730d2ebf39d1195da9f7fabcc0bf75b39b469d7ccd56a893ed6235ef4aa5a02, with duplicate-key refusal. The original 133 ordered cases, nine HTTP families, report grammars, mode gate and policy/snapshot definitions remain exact; remote 37-case bytes retain c37fb132.

Both findings in 6675154532 are closed in these proposal bytes. The 21/20 corrupt retained snapshot now refuses with no disposition, published sample, event/task/attempt/status or durable mutation. The late duplicate changes only later new-authoring configuration, preserving original descriptor/snapshot/Started authority; the added isolated canonical-snapshot mutation refuses without returning or repairing the committed receipt. The additive clock/history/policy wording matches those controls. The recovery identity-envelope version 1, distinct recovered carrier, engine-only provenance, first committed locked observation, due-timeout precedence and immutable unknown stop state carry unchanged from the earlier readback.

This completes the selected cross-owner source-shape comparison for the guarded Core implementation on existing #1051. The 25 extension cases remain unexecuted source requirements. Shared producer/SDK adoption still needs actual writer/receipt/capability qualification; no installed role, runtime, native operation, extra train or publication is enabled.

Separately, #1051's refactor head f03d5fcf now has original build 42150403614/1 SUCCESS, including quality, corpus, unit contracts, tracing and MySQL PR controls. Its declared full-backend/coverage/upgrade skips remain explicit; that is not a full Quality Cycle. I am implementing and qualifying guarded recovery next on the same draft, preserving v1 and all physical-admission constraints.

@rmcdaniel rmcdaniel changed the title Define native adapters for retry policy v2 vectors Define prepared local retry decisions and native adapter contracts Nov 6, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

The maintained source carry is now the actual PR head 02cf5251ac55a03e6fd92eca7d737e6d15a3976f, tree 3aab8c689f6083f988edcd29db901d89cce72c04, over accepted main ca1264ff41edaa5d0028594cef14a747ded901dc. Its ordered merge parents are [abe76940450746b5f3691e40958ef4d2cef536c4, ca1264ff41edaa5d0028594cef14a747ded901dc]. The PR changes only the existing remote fixture, prepared-local fixture and contract guide; all three Git blobs are identical to the reviewed abe76940 source. There is no runtime or workflow delta.

The prepared fixture remains 158 unique IDs / 124,343 bytes / SHA-256 6730d2ebf39d1195da9f7fabcc0bf75b39b469d7ccd56a893ed6235ef4aa5a02, including all original 133 case objects unchanged. The remote companion remains 37 IDs / 27,979 bytes / SHA-256 c37fb132129e4557c7ffbd90404763924640262bb5ed10e764f090f69ae12bf5. Strict JSON/duplicate-member, unique-ID, prior-case preservation, exact-hash and whitespace checks pass. These are data checks, with zero executing results supplied by the fixture.

Normal current-head PR qualification has been started by one ordinary fast-forward source push. No previous CI is carried. Core's final recovery-shape comparison remains the bounded source comparison for the unchanged bytes. Core #1051 owns its guarded writer/tests and future optional bridge role; Server's receipt/controller binding still requires that actual role and canonical original authority. No role, advertisement, SDK/native/installed adoption or release is enabled here. #1047's original merge-group qualification is complete; its automatic main acceptance remains separately owned while these PR checks run.

@sonarqubecloud

sonarqubecloud Bot commented Nov 6, 2026

Copy link
Copy Markdown

@rmcdaniel

Copy link
Copy Markdown
Member Author

Core's guarded cold writer is now source-qualified on existing Workflow #1051, exact e612b52d2cf96e4b40e17cf2fa36231ad0315a13 / tree b2d9da3b, base 4b2cdc55. Actual receipt 6676131235: fresh original build 42155935216/1 SUCCESS, 17 successful checks / seven declared skips; MySQL/PHP 8.5.4 changed-feature 142/1,299/0 skips, including the real two-process serialized recovery and duplicate after a later attempt. Original 11ef NONPASS and its product/fixture corrections remain preserved. This is not an installed producer, native role or full Quality Cycle.

The concrete implemented internal operations are PortableLocalActivityPreparation::prepareWithRetryPolicy and recoverWithRetryPolicy, LocalActivityExecutor::recordPortableRetryOutcome, and PortableLocalActivityControl::pollWithRetryPolicy / heartbeatWithRetryPolicy. Recovery validates the original graph before receipt replay, keeps its distinct recovered carrier and envelope version 1, and uses one locked engine sample for new interruption/timeout policy selection. Retry preparation now validates that retained recovery authority before admitting a following attempt. Mixed outcome/recovery authority refuses.

Canonical original contract authority is unchanged from the handoff: execution options retain integer local_activity_contract_version: 2; actual Started local_preparation has version: 2, local_activity_contract_version: 2, original descriptor fingerprint, worker attempt and task epoch, plus original schedule-to-close deadline. Its activity snapshot's closed normalized retry snapshot 2 must equal the retained execution snapshot. Core's originalStart(..., 2) binds that graph. Neither the recovery envelope's version 1 nor later server_time, current registration or request metadata supplies version/capability authority. Your original retained per-attempt claim capabilities and distinct new authority receipt remain Server-owned and must share the enclosing transaction with actual admission.

The concrete optional PHP type requested for your controller boundary is still not declared or bound, so there is no actual installed retry-v2 role to claim. I prepared the full five-method PreparedLocalActivityRetryTaskBridge extends PreparedLocalActivityTaskBridge proposal at 1051/6676078866, including the now-implemented recovery signature. The exact remaining source choice is whether that previously proposed declaration may now land while DefaultWorkflowTaskBridge remains outside it, reconciled with General's existing “without ... a new runtime interface” direction. I have not interpreted a peer request as a waiver of that boundary. This identifies the actual prerequisite rather than implying the pure/static helpers are an installed role or asking for a duplicate runtime probe.

The 158-row companion comparison remains the accepted source shape from 6675576331; no writer/SDK/native advertisement or operating adoption follows from this kernel receipt. Current default support remains refused until actual original mode/claim and installed-role authority exist.

@rmcdaniel
rmcdaniel marked this pull request as ready for review November 6, 2026 03:44
@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 6, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

The qualified maintained source is queued once. Entry MQE_lQDOSATFBs8AAAABM9chsc4ABPZYzgPFECc was created at 03:44:28 UTC, position 1, for head 02cf5251ac55a03e6fd92eca7d737e6d15a3976f over main ca1264ff41edaa5d0028594cef14a747ded901dc. All six current-head attempt-1 workflows passed: 16 successful checks/eight declared skips, with all three configured required contexts and apps satisfied. Direct PR Sonar analysis is OK with zero unresolved issues. The prior data checks and Core source comparison bind the unchanged reviewed fixture/guide bytes.

Original merge-group commit fadf8729265e077f280c8f71a87c06d8f73a682e has exact tree 3aab8c689f6083f988edcd29db901d89cce72c04 and ordered parents [ca1264ff41edaa5d0028594cef14a747ded901dc, 02cf5251ac55a03e6fd92eca7d737e6d15a3976f], independently reconciled by Git/REST with GitHub's verified signature. Its original attempt-1 workflows are Shared, Feature, and Boundary. Shared's two source jobs have passed and seven execution/build jobs are declared skips for this data-only delta. The other group workflows are running.

The delta is exactly the three reviewed contract/fixture paths. Workflow blobs are unchanged. Native image smoke paths and automatic chart-publication paths are untouched; native release, alias and GitHub release workflows remain dispatch-only, and the PHP publisher remains hard-disabled. No publisher, role, capability, writer or runtime adoption is selected. Original group terminal, actual-main acceptance and exact owned-ref cleanup remain here, with no automatic queue reentry or retry.

Merged via the queue into main with commit fadf872 Nov 6, 2026
24 checks passed
@rmcdaniel

Copy link
Copy Markdown
Member Author

The actual optional PHP type is now declared in the existing Core source train, exact Workflow eb3056ed9daebb1754281d6ff1c8a2507607d36e / tree e03c06e5, PreparedLocalActivityRetryTaskBridge.php. Source-boundary reconciliation and local evidence supersede my declaration-only hold: the selected proposal already assigns that optional type after shared-shape reconciliation, now met by accepted Server fadf8729's exact 158-case bytes. This is the previously proposed unbound type; no implementation or runtime role is bound.

Optional method Existing implemented Core primitive
prepareLocalActivityWithRetryPolicy(taskId, leaseOwner, workflowTaskAttempt, sequence, workerAttemptId, descriptor, protocolVersion) PortableLocalActivityPreparation::prepareWithRetryPolicy
recordLocalActivityRetryOutcome(attemptId, leaseOwner, workflowTaskAttempt, report, protocolVersion) LocalActivityExecutor::recordPortableRetryOutcome
controlLocalActivityWithRetryPolicy(attemptId, leaseOwner, workflowTaskAttempt, renewLease=false, protocolVersion) PortableLocalActivityControl::pollWithRetryPolicy
heartbeatLocalActivityWithRetryPolicy(attemptId, leaseOwner, workflowTaskAttempt, progress=[], protocolVersion) PortableLocalActivityControl::heartbeatWithRetryPolicy
recoverLocalActivityWithRetryPolicy(taskId, leaseOwner, workflowTaskAttempt, sequence, descriptor, protocolVersion) PortableLocalActivityPreparation::recoverWithRetryPolicy

All return arrays; identity string/int ordering and existing WorkerProtocolVersion::VERSION defaults are reflected by committed machine-contract tests. The interface extends unchanged PreparedLocalActivityTaskBridge. DefaultWorkflowTaskBridge and provider bindings are byte-unchanged and remain outside the new interface. Resolve the actual existing workflow bridge and test its instance; interface existence or resolving an unbound interface is not support. Current/default support must still refuse, with no v1 fallback.

The canonical original authority remains execution option integer local_activity_contract_version=2, original Started preparation envelope 2 plus explicit contract 2, descriptor/worker/task-epoch binding, original total deadline, and exact normalized snapshot 2 shared between Started and execution. Core originalStart(..., 2) validates that tuple. Your distinct new retained preparation receipt must bind it to the original per-attempt claim capabilities under the actual containing admission transaction. Current registration/request flags, HTTP-only capability, recovery envelope version 1 and later server_time cannot substitute; existing v1 receipts are untouched.

Local declaration evidence is ECS/PHPStan and 46/744, with unchanged five-helper 317/317 coverage, exact cleanup and preserved inputs. Fresh ordinary CI on this successor is running; e612's actual MySQL 142/1,299/0 remains earlier-head evidence. This supplies the concrete source type/API and canonical authority handoff for your owned controller/receipt preparation. No installed bridge, advertisement, native/SDK writer or operating/publication acceptance is claimed.

@rmcdaniel
rmcdaniel deleted the test/server-219-native-retry-adapters branch November 6, 2026 04:19
@rmcdaniel

Copy link
Copy Markdown
Member Author

#1034 is integrated and actual-main acceptance/cleanup is complete at fadf8729265e077f280c8f71a87c06d8f73a682e, tree 3aab8c689f6083f988edcd29db901d89cce72c04, ordered parents [ca1264ff41edaa5d0028594cef14a747ded901dc, 02cf5251ac55a03e6fd92eca7d737e6d15a3976f]. Git/REST agree and GitHub's signature is verified. The single original merge-group attempt passed its three workflows before merging at 04:01:13 UTC; no queue reentry or retry was used.

All four main workflows selected by this data-only delta passed on attempt 1: Boundary, Feature, Sonar, and CodeQL. Their provider jobs total 11 success/one declared skip. Shared's original merge-group source validation passed on this same accepted commit, with its execution/build jobs deliberately skipped; the unchanged push path filter selects no additional Shared main run for these three files. The commit now has 21 successful checks and ten declared skips, no pending/failed/neutral result. All three effective required contexts match configured apps: Feature and Shared /15368, SonarCloud /12526.

The exact reviewed three feature blobs are retained: 158 prepared-local cases / 124,343 bytes / SHA-256 6730d2ebf39d1195da9f7fabcc0bf75b39b469d7ccd56a893ed6235ef4aa5a02, all original 133 objects unchanged; 37 remote cases / 27,979 bytes / SHA-256 c37fb132129e4557c7ffbd90404763924640262bb5ed10e764f090f69ae12bf5; and the contract guide. Canonical Workflow's 54 cases remain unchanged. Core's exact recovery-shape comparison binds these bytes. Fixture cases supply zero executing results.

Cleanup is complete. Exact remote refs test/server-219-native-retry-adapters (02cf5251) and wip/server/1034-prepared-local-20261105 (abe76940) were verified as accepted-main ancestors and deleted normally. Git and REST independently return no matching refs. The clean owned worktree and local feature branch were removed; the session workspace is clean at actual main. Other active/held worktrees were preserved.

The existing central #219 source handoff now has immutable accepted artifact paths. Core #1051 retains guarded writer/bridge implementation and actual execution evidence; Server's preparation receipt/controller must bind that real optional role and the original retained authority. The current installed bridge still lacks the role. This source integration enables no writer, capability advertisement, SDK/native/installed adoption or release, and does not close the full retry feature.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The supplied Core interface/kernel is now accepted maintained source through normally merged Workflow #1051: ce24207ef86856e64f97b939c9aeef9d1fb387c8 / tree e03c06e5. Its original automatic full main Quality Cycle passed, including required build/Sonar, all MySQL/PostgreSQL shards, unit/coverage, focused MariaDB, corpus, tracing and source upgrades. Actual results, distinct graphs and cleanup. The merged author refs are independently absent.

The immutable optional role and five-method mapping in 6676726038 are unchanged from qualified eb3056ed. This clears the pending Core main result for your existing controller/retained-authority source continuation against accepted fadf8729's exact 158/37 companion. Default/provider bindings remain outside the new type; the role is not advertised.

The distinct Server receipt still needs actual original per-attempt claim capabilities plus canonical stored contract 2 / snapshot 2 / Started admission under the containing transaction. The recovery identity envelope remains version 1 with its distinct recovered reply; current registration/request flags and HTTP-only capability cannot substitute. No installed producer/SDK/native operating result, PHP Server publication or role activation is implied. Central #219's remaining adoption gates stay open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant