Skip to content

Expose activity retry context and pin originating-run signals - #221

Merged
rmcdaniel merged 5 commits into
mainfrom
feat/activity-context-retry-250-20261027
Oct 27, 2026
Merged

rmcdaniel merged 5 commits into
mainfrom
feat/activity-context-retry-250-20261027

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Activities can inspect their original retry budget and originating workflow/run without passing duplicate arguments through workflow code. Add immutable ActivityInfo.retry_info, nullable original workflow_id/run_id, and exported ActivityRetryInfo, preserving the six-argument constructor and supervised callback serialization.

Received snapshots preserve unknown fields and distinguish omitted policy, legacy root-null policy and explicit object-level unlimited policy. Ordinary and supervised callbacks use the original claim; inline/prepared callbacks inherit only known parent identities and no remote retry policy. Last-attempt checks describe the configured budget; cancellation, deadlines and non-retryable failures can still end execution earlier.

Forward the run ID previously discarded by WorkflowHandle.signal, expose the optional pin on direct async/sync signal calls, and reject invalid pins before HTTP. The canonical guide uses a stable business-event key and describes the current Server signal-retention limitation. No Server wire or retry-admission change.

Validation: 49 selected context/command/signal cases and six worker-path controls pass. Old wiring fails four worker controls; the old client fails the actual pinned HTTP-route assertion. All 18 checks on signed merge 14dc458d71f42f915de748e708beb159384160d7 pass, including connected integration and target qualification. Python 2.12.0 is published normally. Fresh PyPI installation and published PHP Server 2.11.4 acceptance pass: eight workflows, fourteen callbacks, five execution modes, legacy/unlimited metadata controls, standalone wire identity and fourteen original-run signals. The canonical portal is verified live; all eight current main-guide locales follow in Website #410.

Refs durable-workflow/.github#250. Signal receipt retention remains separate in durable-workflow/.github#216; the baseline empty-exception reporting defect is SDK #222.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Initial draft head 7d234fb72f5a04442b8d093061cb3806d50244eb: the hosted Python3.10,3.11,3.12, lint, regression-corpus and package cells passed. CI40446299641 failed in the integration test step, so the required target-qualification aggregate also failed. This is not a ready or released change.

The twenty focused metadata/copy/pickle/legacy controls and activity-module typing pass locally. Next: diagnose the exact integration job, add ordinary/supervised/local worker-path regressions and finish the remaining draft checklist. No check, assertion or qualification requirement has been weakened.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Cloud's current feature head is 7acad665fd171ade95d29de7a780f8901edc192c.

Six additional worker-path cases pass: ordinary and supervised remote callbacks receive the original bounded or legacy retry snapshot, and inline/prepared callbacks receive only the parent's original workflow/run identity without inheriting its remote retry policy. The full source/test Ruff check passes. A controlled old-worker overlay with the new data classes produces four assertion failures in the bounded remote and local identity cases, demonstrating that these tests exercise the wiring.

Current CI 40449523707 has one connected-integration failure (67 pass, 49 skip): tests.integration.test_local_activity.test_local_activity_runtime_external_payloads_survive_cold_replay. Before any callback, Worker._run_workflow_task_with_history receives a direct external_storage envelope and serializer.decode_envelope raises ValueError: external payload reference requires an external storage driver; the existing assertion then observes zero executions instead of one. This is not a pass or evidence of an activity-context regression. The failure is being reconciled with Server's owner; the assertion and gate remain intact. The added CI annotation emits only bounded test identifiers and a numeric failure count.

@rmcdaniel rmcdaniel changed the title Expose received activity retry and original workflow context Expose activity retry context and pin originating-run signals Oct 27, 2026
@rmcdaniel

rmcdaniel commented Oct 27, 2026 •

Copy link
Copy Markdown
Member Author

Current draft head: 105e174a6cc5ef855f006fff36afc5526c4862e8.

The preceding head's version preparation updated project.version but missed the two repository-owned release identity fields. All three now identify 2.12.0. The existing eight release-metadata tests and source/test Ruff check pass. Changelog entries remain under Unreleased. Unit failure annotations now retain the final 3,000 bytes: GitHub truncated the prior 12,000-byte excerpt to 4,096 bytes before its failure summary.

Current hosted CI 40454996762 passes Python 3.10, 3.11 and 3.12, package, lint, regression corpus and action policy. Docs, public-boundary, Sonar and CodeQL checks pass. Integration completed with exactly one failing case, the existing externalized local-activity cold-replay reproduction of Server #804. Target qualification remains failed. Its assertion and gate remain intact. The accepted Server fix is now draft PR #806 at d10b138d8, with source/image qualification owned by the Server maintainer.

Local candidate qualification also passes:

  • Wheel and source distribution match the exact source commit under check_release_metadata.py; twine check passes both.
  • The normal package smoke script freshly installs both artifacts outside the checkout and passes README/export/typing-marker/version/conformance checks. This is candidate installation, not PyPI publication or published-Server acceptance.
  • The full local Python run passed 3,586 cases; four failures and two setup errors were precisely missing Git in the plain Python image. Both affected Git-dependent test groups then passed all 22 cases in the existing Python 3.12/Git tooling image. The hosted matrix above provides the full clean source result.

Candidate wheel SHA256: 9fbde39e7a97b6d519733f35e1021a25d10f43ad292f7b75a05854a92ac7b941; candidate sdist SHA256: cdbe056dbe66e2b3e19869bae2f92c2a905647dd343993585b0c4586acc558f5.

The 49 focused context/signal cases, worker-path controls, old-source counterfactuals and executed canonical helper remain as recorded. Next: finish normal CI/review, exercise the candidate and then normal published artifacts against the published Server across ordinary, supervised, inline/prepared and session callbacks, publish the ordinary minor release and finish all eight main-guide locales. No release or installed-Server completion is claimed yet.

@rmcdaniel

rmcdaniel commented Oct 27, 2026 •

Copy link
Copy Markdown
Member Author

Installed-candidate qualification passes for Python PR221 head 105e174a6cc5ef855f006fff36afc5526c4862e8 against published Server 2.11.4.

The wheel (SHA256 9fbde39e7a97b6d519733f35e1021a25d10f43ad292f7b75a05854a92ac7b941) was installed into a separate consumer virtual environment. All SDK imports resolve inside its site-packages; no SDK source checkout is mounted into the consumer. Server is the published multi-architecture index sha256:9dd7e5fc8382c0bbdda106ffac86ab8edc4264545e2df3c7e43209884fb8091d, amd64 source ad3ac5fda0a30a58084b2d68569f2d91e23fabf3, PHP8.3.36/Core2.14.2, with SQLite and the normal database queue worker.

Eight workflows / fourteen callbacks pass:

Execution Received retry cases Original attempts Configured maximum Last-configured checks
Ordinary remote bounded, omitted 1/2/3, then1 3, unknown false/false/true, unknown
Supervised remote bounded, omitted 1/2/3, then1 3, unknown false/false/true, unknown
Worker-session remote bounded, Server default 1/2/3, then1 3,1 false/false/true,true
Inline local locally configured budget 1 unknown received snapshot unknown
Advertised prepared local locally configured budget 1 unknown received snapshot unknown

Every callback preserves the original public workflow/run IDs and the received policy; copy mutation does not alter the snapshot, and frozen attempt assignment is rejected. Session affinity makes the Server supply an explicit one-attempt default; the SDK preserves it. Local callbacks retain parent identity without inventing a received remote policy, even when local authoring configures a budget. Supervised/prepared callbacks execute in separate processes.

The exact canonical signal_originating_run helper (SHA256 a0583b955791bc22e0c2b18514c13a80b3e07c527ec2b981c02e010b67da933d) executes in all fourteen callbacks. Each real HTTP request uses /workflows/{original_id}/runs/{original_run}/signal/payment_recorded, with the stable business-event key reused across retries. The published Server still records three signals for each three-attempt case, consistent with the documented unshipped shared216 receipt guarantee; no deduplication or once-only admission is claimed.

Four installed-package component cases separately pass for omitted, legacy root-null, explicit object-null unlimited (including unknown fields), and bounded snapshots, including pickle round trips. Newly authored max_attempts:null is rejected by the current Server command contract; this qualification preserves that admission behavior and tests legacy/unlimited received-snapshot semantics without changing it. Heartbeat diagnostics use the existing scalar/length contract; callback metadata is captured separately. The disposable namespace was deleted after the run.

This is an unpublished SDK candidate on a published Server, not PyPI or final published-SDK acceptance. Normal CI still has the unchanged Server804 cold-replay failure pending Server806 merge and rerun. After merge/release, repeat normal PyPI installation and these affected callback paths, verify the portal and finish all eight current Python main-guide locales.

Additional qualification: a standalone activity also passes original-identity comparison against its independently captured delivery. This Server supplies associated workflow/run IDs and a one-attempt snapshot for that activity; the SDK preserves them. Missing identities remain nullable and are handled explicitly, rather than assuming that every standalone delivery omits them. Five critical installed module files byte-match the wheel and qualified SDK source. The final release-notes commit changes only CHANGELOG; runtime code remains identical to105. Its normal CI is running before merge/publication.

@rmcdaniel
rmcdaniel marked this pull request as ready for review October 27, 2026 06:54
@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel

Copy link
Copy Markdown
Member Author

Security release-provenance receipt (2026-10-27 UTC):

  • Annotated tag 2.12.0 object 3a8ad1b6a7bc0f41cf31485e766aecf1cb5daa35 targets the accepted merge commit 14dc458d71f42f915de748e708beb159384160d7 exactly.
  • Publication run 40466308548 completed successfully after exact release-identity resolution, build/metadata/package/smoke checks, immutable handoff validation, PyPI publication, authoritative PyPI JSON and supported-install verification, and GitHub Release creation.
  • Tag public-boundary run 40466308405 passed on the same commit.
  • PyPI serves two non-yanked artifacts: wheel SHA-256 e4365cfccf741d4ddc4c712d7f42ab39988df6b56406c627ef0e17986df20f3f and sdist SHA-256 419b087b5e1df42f269565de1221b32c1dccc5d9bc6eedd00cb836b1579a1495.
  • PyPI Integrity API returns one https://docs.pypi.org/attestations/publish/v1 attestation for each file, issued by GitHub Actions for repository durable-workflow/sdk-python, workflow publish.yml, environment pypi; each attested subject name and SHA-256 exactly matches the corresponding public artifact.

Release provenance and package integrity are Security-accepted. This does not substitute for the still-pending post-fix Server image acceptance tracked in durable-workflow/server#804.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Python 2.12.0 is published and its normal installed-package acceptance passes.

  • Source: signed merge 14dc458d71f42f915de748e708beb159384160d7, tree-equal to qualified PR head 391ca5c2e4dbfc1fdc2ff1fdd5bac68f46a3d4d4. All 18 checks on the merged commit succeeded, including the unchanged connected-integration gate after Server #806 merged.
  • Release: immutable annotated tag object 3a8ad1b6a7bc0f41cf31485e766aecf1cb5daa35, normal protected publication run, stable 2.12.0 release. PyPI wheel SHA-256 e4365cfccf741d4ddc4c712d7f42ab39988df6b56406c627ef0e17986df20f3f; source distribution 419b087b5e1df42f269565de1221b32c1dccc5d9bc6eedd00cb836b1579a1495.
  • A fresh ordinary pip install durable-workflow==2.12.0 from PyPI, outside the SDK checkout and without source/package overrides, imported the installed package. Five changed runtime files match the qualified source. The publication workflow also verifies source-bound metadata, Twine and fresh installation of both distributions.
  • Against published PHP Server 2.11.4, all 8 workflows / 14 callbacks passed through ordinary remote, supervised remote, worker-session, inline local and prepared local execution. Bounded attempts 1/2/3 preserve max 3 and final flags false/false/true. Unknown ordinary/supervised policies remain unknown; the session default is the explicit Server-provided max 1. Local callbacks preserve known parent identity without inheriting the remote retry policy. Immutable/copy and subprocess serialization controls pass.
  • Four installed metadata controls cover omitted policy, root-null legacy policy, explicit object-level unlimited policy with unknown fields, and bounded final attempt. These test received snapshots; the published Server intentionally rejects newly authored unlimited policies at admission, which this SDK release does not change.
  • The exact canonical helper executes 14 actual signals, each pinned to the originating run with a stable business-event key across retries. Three bounded attempts still record three signals on the published Server. This is the documented, separate receipt-retention requirement, not a once-only admission claim.
  • A standalone activity independently preserved identities from the actual poll delivery; the published runtime supplied workflow/run IDs and max 1. Missing-identity source/installed controls retain null identities and prohibit a fallback task ID.
  • The canonical activity guide is deployed and verified HTTP 200 with 2.12.0 minimum, retry APIs, helper and working section anchors. Website #410 owns all eight current main-guide locales; its qualification/deployment is still pending. Historical artifact tuples remain unchanged.

The runtime checks completed October 27, 2026, approximately 07:21–07:24 UTC. The separately recorded empty-exception-message defect SDK #222 remains a baseline follow-up, and Server #804 still needs acceptance of its eventual published fix image. Neither outcome is represented as fixed by this SDK release.

@rmcdaniel
rmcdaniel deleted the feat/activity-context-retry-250-20261027 branch October 27, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant