Repository navigation
Audit conformance experiments for complete first-party SDK coverage #122
Description
Request
Audit the existing conformance experiments so portable behavior is tested across every meaningful PHP, Python and Rust direction. Keep language-specific and embedded-only behavior explicit.
Acceptance
- Inventory current experiment definitions and map them to the public conformance runbook.
- Replace unjustified language-only instructions with explicit PHP/Python/Rust coverage and every meaningful supported workflow/activity direction, including cross-language child and Nexus behavior where applicable.
- Identify genuinely language-specific or embedded-only cases explicitly; do not invent meaningless Cartesian combinations.
- Cover authoring, inputs/results, retries, timeouts, heartbeats, cancellation, signals/queries/updates, replay/restarts, upgrades, and failure recovery as applicable to each experiment.
- Keep executable instructions, prerequisites, expected outcomes, and cleanup clear for a new human or agent using only public repositories.
- Keep Cloud account data, infrastructure, provisioning and chaos procedures in the private Cloud repository; link the private follow-up there without exposing its details here.
- Run focused representative checks for changed instructions and report outcomes on GitHub. Do not commit generated per-run evidence or rebuild a conformance pipeline.
GitHub remains the source of truth. Reuse ordinary repository commands and GitHub Actions for public happy paths; maintainers handle exceptions directly.
Current audit state
PR #260 merged as 4fb961325779c7d39cfb0c35675020625194a7f4. All seven checks pass for its exact candidate 65511faf36f684dca88beeef9ae5443919b3c18a, with Sonar OK and zero open/confirmed PR findings. The published tree and both document bytes match that tested candidate. It documents the child image/bundled-package prerequisite, replay boundaries, protocol skew under one package tuple and migration’s host-evidence requirements. All six independent main checks pass, with direct Sonar analysis bound to that exact main revision. Delivery and published-byte verification.
PR #261 merged as d5d5782a2a79b4a289e687611a341feab6984ce5 after all seven checks passed for final head 44ca3909af642d21b949701c443231c8b4037ae5. The tested/published tree and both document bytes match. It adds the existing failover/MCP entry points, actors, inputs and cleanup, including the known cleanup-timeout result-loss limitation. Source audit. All six independent main checks pass, with direct Sonar analysis bound to that exact revision and quality gate OK. Delivery and published-byte verification. No runtime was exercised.
PR #263 merged as 64fd4a9c05f88e92ab65a90e3a152d8e17c22dd3 after all seven checks passed for integrated head eafe49575dabe1f62daff69b5a6f62dd5cad7d7b, with direct Sonar OK and zero open/confirmed PR findings. It names PHP/Python SDK parity commands, distinct result/cleanup contracts and Rust's separate starter journey. The tested/published tree and both document bytes match. Its pinned source/help audit also distinguishes PHP's managed loop and heartbeat timestamp presence from cadence/lifecycle qualification. Independent main checks remain pending. No runtime was exercised.
The product decision remains: correct instructions where the actual command/API already exists, and keep accepted missing capabilities open. Documentation and workarounds do not complete whole-history/list iteration #247, portable validators #234 or cross-SDK metrics delivery #233.
Published instruction corrections and their exact qualification remain in the owning PRs:
- Child/browser execution and actor scope: #244. Lifecycle host command and execution surfaces: #252, #253.
- Published Nexus API and separate Rust cohort: #246. Original-run CLI selection: #248. Manual observers, worker replay and client iterators: #249, #254.
- Validator authoring versus protocol/refusal controls: #251. Structured full/focused signals/query results: #255.
- Manual PHP heartbeat versus managed loop: #256. Schedule protocol helpers versus authored workers: #258. PHP Workflow-package sagas versus service SDKs: #259.
Current consumer evidence is bounded:
- Sample App main remains
40be451c020c6a0e8c9561585bcd96d38a352a75. Its tuple selects Server 2.10.0, PHP SDK 3.1.0, Python SDK 2.7.0, Rust SDK 4.1.0, CLI 2.3.0, Workflow 2.11.3 and Waterline 2.12.1. It separately declares Server-bundled Workflow 2.9.0. The current-source/child audit preserves those roles. Existing main run 39940001164 logs nine completion directions, five typed failures, five worker-recovery directions, fifteen cancellation recoveries and 45 browser views. Its raw ZIP was not independently reread. - Activity recovery and external-effect receipts retain their tested PR tree and recorded tuple. Timer/package/application and pending-activity upgrade receipts retain their original sources, selected package pairs and operations. Neither audit launched independent post-merge reruns or qualifies newer packages.
- The search-attribute raw artifact audit independently reads all nine reports and verifies the ZIP digest, original prefix, worker loss/replacement and acknowledged signal during absence. It covers one Rust original with PHP/Python observers. The namespace/principal audit reads logs and metadata for four named recoveries and eleven CLI/Waterline API views; its individual raw files were not reread. Those API views do not establish browser coverage.
- Frozen Nexus main-run notices corroborate eight external-payload completions, eight authored refusals and eight distinct uncaught-failure originals over four actual PHP/Python SDK directions. That Server 2.9.1 / PHP 2.5 / Python 2.6 tuple remains separate from Rust’s installed receipt and current packages; the raw reports are no longer returned by its artifact API.
- Published PHP Server 2.11.1 release audit binds the recorded image/Core inputs and passed readiness, Compose, TLS and 16-record catalog/provenance checks. Those checks do not replace SDK-worker experiment or native-runtime qualification.
- Frozen worker-versioning evidence and installed worker-affinity provenance #124 remain bound to their historical Server/SDK tuples. Current runner inputs are a route to a new qualification, not proof that it ran.
- Replay source and published-distribution checks match the actual Python 2.9 wheel and Rust 4.7 crate to the reviewed command code. Python reconstructs fixture history; Rust gracefully replaces Worker instances in one process. Skew/migration source checks distinguish protocol negotiation from package skew and evidence composition from an executed migration. These checks launched no runtime cohort.
The updated published API inventory verifies actual Python 2.10.0 wheel and Rust 4.9.0 crate identities. Python still reads run history/list pages explicitly and its standalone-activity list is separate from typed run-activity inspection. Rust retains lazy selected-run history/list APIs and has no schedule-creation binding. Named stream and schedule-history iterators remain separate capabilities. The earlier 2.9/4.7 source receipt remains its own cohort.
Open runner and qualification work:
- Sample App #268 remains open. The bounded source control reproduces its late-observer false refusal without a Server or SDK runtime. A separate successful original does not repair that observer or qualify failed originals.
- The failover report-preservation finding reproduces loss of the result write when project cleanup raises
TimeoutExpired, including after an original phase failure. The Server owner acknowledged it and prepared PR #700, exact head7d55abe211e86db0f610237480289ced3013b275. An independent scoped review passes all 38 constructed controls, retains five before-control errors with their distinct causes, and starts no Docker, network or subprocess work. That source candidate remains unmerged and unqualified as a published runner/runtime. No historical run or missing artifact is attributed to this source finding. - Server’s lifecycle anonymous-volume cleanup finding and missing managed PHP heartbeat qualification are handed to the owning runner scope. Owner acknowledgment/implementation remain pending. Existing volume ownership is unknown; the source finding does not justify deleting volumes.
- Keep native Server, newer artifact tuples, the four non-Rust child failure/recovery directions, three non-Rust PHP service-SDK saga directions, additional cancellation policies and untested external-effect boundaries unqualified until their own cases run. Client iteration, paged observers and paged worker replay remain separate capabilities. Accepted product work stays on its linked issues.
Next: finish independent main verification for #263, then continue the admitted source/current-consumer audit and obtain acknowledgment for the reserved runner findings. The unchecked acceptance criteria remain open. No new runtime cohort was launched by these source/evidence audits.
Activity
rmcdaniel commentedon Sep 8, 2026
Concrete audit finding from server#140: an exploratory unrestricted Server PHPUnit run includes legacy conformance-runner contract tests outside the current feature qualification command. The first failing assertion, ActivityConformanceRunnerContractTest::test_runner_rejects_local_repo_root_vendor_runtime_probe, expects the phrase "local product source probe" while the actual rejection identifies forbidden local source and missing pinned-image evidence. This reproduces identically on unchanged main e918f0c4; no Nexus change causes it. Include runner-test relevance and clean executable prerequisites in this existing experiment audit, alongside first-party SDK coverage. Do not turn obsolete message assertions into another release coordinator. Server release qualification is independently green on the documented 2,132-test command.
rmcdaniel commentedon Sep 23, 2026
Initial public-runner audit (2026-09-23):
WorkflowContext::start_child_workflowexists, so the five Rust-involving directions remain unexecuted. PR Clarify child-workflow conformance evidence scope #128 prevents this install-only pin from being read as behavioral proof.SagaAPI and unit replay tests, but this runner has no Rust execution shard. This is another applicable published-artifact coverage gap.Next action after PR #128: add real published-artifact Rust child parent/child cells, then Rust saga compensation cells. Keep these separate from the existing PHP/Python passes and do not treat artifact installation as runtime evidence.
rmcdaniel commentedon Sep 23, 2026
Merged PR #128 after the shared contract check passed. The public runbook now distinguishes child-workflow Rust artifact resolution from executed PHP/Python runtime cells. The issue remains open for real Rust child and saga published-artifact scenarios and the remaining experiment audit.
rmcdaniel commentedon Sep 23, 2026
Server PR #170 is merged as
91db02448ddacb70b3ea683c57a07962b7fb97b3. It repairs the stale activity conformance runner contract tests: synthetic evidence is exercised in explicit in-process mode, the extracted-source handoff remains separately tested, and local-source rejection asserts current diagnostics. The containerized contract suite passed 44 tests / 617 assertions; the Server PR feature suite and required checks passed. This is test-harness hygiene only, not new SDK runtime coverage. Rust child and saga published-artifact execution, plus the rest of the experiment audit, remain open.rmcdaniel commentedon Sep 23, 2026
Merged PR #130 (
d28c6746208dbb9f8aca0e6294f4f5f16ea03467). The stable conformance runbook now includes the existing published-artifact saga runner and states its actual PHP/Python workflow-compensation coverage. The current manifest and runner do not execute Rust saga handlers; Rust remains uncovered rather than inferred from artifact installation or the playground. Shared contract tests passed. This advances the experiment inventory, not the missing Rust execution cells.rmcdaniel commentedon Sep 24, 2026
Focused published-artifact child-workflow probe (2026-09-24), isolated local Docker stack:
Each returned the expected child language and input. For all five, persisted parent history contained StartAccepted, WorkflowStarted, ChildWorkflowScheduled, ChildRunStarted, ChildRunCompleted, WorkflowCompleted in that order; child runs completed. The PHP/Python-only directions remain covered by the existing published runner as noted above.
This is a focused local execution probe, not a claim that the published child runner now executes all nine directions. Remaining work: add these Rust-involving cells to ordinary reproducible public instructions/runner, cover restart/replay and failure/compensation cases, then run the separate Rust saga cells and continue the broader experiment audit. Scratch workers and stack are being removed after this probe.
rmcdaniel commentedon Sep 24, 2026
Merged durable-workflow/sample-app#101 at f843d3b961e1a3a38b3a4c9d610d08bebc734eab. The public polyglot/child-workflows/ example now contains PHP, Python, and Rust workers plus a client that starts all nine parent/child directions and checks result identity and persisted lifecycle history. It reuses the existing Sample App playground and Rust lockfile, with no new conformance coordinator.
Verified local published-artifact run on 2026-09-24 01:34:27-01:34:50 UTC: 9/9 directions completed, zero client failures. Each parent history had WorkflowStarted, ChildWorkflowScheduled, ChildRunStarted, ChildRunCompleted, WorkflowCompleted in order. Exact tuple: Server 2.3.14 (bundled Workflow 2.0.16), Waterline 2.0.3, CLI 2.0.0, PHP SDK 2.0.11, Python SDK 2.0.0, Rust SDK 2.0.7. PHP lint and Pint, Python syntax, Rust cargo check --locked, and all Sample App PR checks passed. The disposable local stack, workers, volumes, and scratch files were removed.
Scope remains explicit: the existing Server child-workflows-published-artifacts.sh runner still executes its PHP/Python matrix, not Rust; this Sample App example is a separate executable manual experiment. Restart/replay, failure/cancellation and Rust saga compensation cells remain open under this issue. Next: link the manual matrix from the public conformance runbook and then tackle the Rust saga case.
rmcdaniel commentedon Sep 24, 2026
The public conformance runbook now links the independently executed Sample App child-workflow matrix. Merged #131 at 3ef934b after Shared contract tests passed. The runbook explicitly keeps the Server runner's PHP/Python scenario result separate from the Sample App 9/9 execution result; no Rust pass is inferred from crate installation. Remaining work in this issue: Rust saga/compensation published-artifact execution and the broader experiment inventory, restart/replay and failure coverage.
rmcdaniel commentedon Sep 24, 2026
Rust saga coverage increment shipped: sample-app#102 merged at
5bcc3b78d444a6d9d4023c2f3a2001bfe3441cda; the public conformance runbook now links it (merged41c1df2330804970acb54863a752c18aadf664fb).Focused local published-artifact execution on 2026-09-24 around 02:57-03:00 UTC (exact wall-clock start/finish were not captured, so this is not a complete stable-release record): Server 2.3.14, Waterline 2.0.3, PHP SDK 2.0.11, Python SDK 2.0.0, Rust SDK 2.0.7. The Rust workflow ran three cases with Rust, PHP and Python compensation workers, twice. The stricter second run passed 3/3: planned decline had
ActivityFailed; both forward activities and both reverse-order compensation activities hadActivityCompleted; workflow completed with expected result. No Cloud/customer namespace was used. Rustcargo check --locked, PHP lint/Pint, Python syntax and all Sample App PR checks passed. The disposable local stack and workers were removed. Rust formatting was not checked because the prepared image lacks cargo-fmt.This is separate from the Server saga runner PHP/Python matrix and does not qualify external business side effects, restart/replay, duplicate delivery, compensation failure, or PHP/Python workflow with Rust compensation. Next experiment should exercise those inverse directions and restart/replay before claiming full SDK saga parity. The broader experiment inventory in this issue remains open.
rmcdaniel commentedon Sep 24, 2026
Rust-involving saga coverage update (pass, 2026-09-24 03:17:05-03:18:07 UTC): Sample App #103, merged runner commit
9206e513632c483ac63a4c98b724f74e02ed7bdc; conformance guide #133. A disposable local published-artifact stack completed 5/5 directions: Rust workflow -> Rust/PHP/Python compensation, PHP workflow -> Rust compensation, Python workflow -> Rust compensation. Each checked the intentional decline asActivityFailed, both reserves and reverse-order compensations as persistedActivityCompleted, andWorkflowCompletedwith the expected result. No customer namespace was used; task-owned workers, Compose services, volumes, and network were removed.Tuple: Server image
durableworkflow/server:2.3.14(bundled Workflow2.0.16), Waterline imagedurableworkflow/waterline:2.0.3, prepared Sample App CLI2.0.0, PHP SDK2.0.11, Python SDK2.0.0, Rust crate2.0.7. Command shape: start isolated published Server/Waterline stack withPLAYGROUND_COMPOSE_PROJECT=sample-app-saga-inverse scripts/playground rust; launchpolyglot/rust_workersaga binary pluspolyglot/sagas/php_worker.phpandpython_worker.py, then runpolyglot/sagas/client.pywith disposable runtime URL, namespace, queue, and tokens. The separate Server runner still contains only PHP/Python saga cells. Worker restart, duplicate delivery, and compensation failure are not proven by this run and remain open inventory work.rmcdaniel commentedon Sep 24, 2026
Rust saga restart increment (pass, 2026-09-24 approximately 03:56-04:00 UTC): Sample App #104, tested source commit
6062c3f02b54563e3cdfe85acdf4bda7b04c4a86, merged asdc657d023f46c98870acbc9919c306c273c12ae2; guide #134. Exact second-level start/finish times were not retained; future manually run failure-mode checks should capture them before removing the stack.Tuple: published Server image
durableworkflow/server:2.3.14(bundled Workflow2.0.16), Waterline imagedurableworkflow/waterline:2.0.3, prepared CLI2.0.0, PHP SDK2.0.11, Python SDK2.0.0, Rust SDK2.0.7. Disposable Compose projectsample-app-saga-restart, namespacedefault, queuesample-app-saga-restart. Command sequence: startsaga_compensation_worker; runpython polyglot/sagas/restart_client.py start; stop the first Rust worker process afterActivityCompleted(reserve-first)andSignalWaitOpened; runrestart_client.py signal <workflow-id>with no Rust worker present; start a new Rust worker process; runrestart_client.py verify <workflow-id>. First/new worker host process IDs differed (501160->507206). Run ID01m38s09gd35hj9pgf9e52xdbdcompleted with exactly one persisted signal wait and receipt, the expected five scheduled activities with no duplicate first reserve, one planned decline failure, four completed reserve/compensation activities in expected order, andWorkflowCompleted. Task-owned containers, network, and volumes were removed. This proves one Rust-workflow/Rust-compensation cold-replay direction; cross-language restarts, process loss during activity side effects, duplicate delivery, and compensation failure remain unproven.rmcdaniel commentedon Sep 24, 2026
Rust-parent saga restart matrix expanded and executed (2026-09-24 07:44:20-07:48:02 UTC). Sample App #105 merged as
416b39a9216f5497e1504d2f61344e3a6043f540; tested source commitf50d434. The runner now selects Rust, PHP, or Python compensation with--compensation-runtimeand verifies the selected type, result, and history.Published tuple: Workflow
2.0.16inside Server2.3.14(image digestsha256:252b85fd1e177bc40db9c5dc9a638d5d5c0bc465eab93e1024ebe5c7cd58bbc8), Waterline2.0.3, CLI2.0.0, PHP SDK2.0.11, Python SDK2.0.0, Rust SDK2.0.7. Disposable local Server, namespacedefault, queuesample-app-saga-directions. For each direction:python3 polyglot/sagas/restart_client.py start --compensation-runtime <runtime>; stop the Rust worker;signal <workflow-id>with no Rust worker; start a new Rust process;verify <workflow-id> --compensation-runtime <runtime>. Client/worker credentials were only the disposable playground token.All three passed: Rust-to-PHP run
01m395yq9hz0pkfwvfw6f7hrym, Rust-to-Python01m3961vqkxbzmc87p2edr0ks4, Rust-to-Rust01m3963xt557e4mz0jgp8h3t70. Each history had one persisted signal wait and receipt, no duplicate reserve, the planned decline failure, reverse-order compensation by the selected runtime, andWorkflowCompleted. Separate Rust worker containers/processes spanned each stop/resume; observed host PIDs included 730742, 739384, and 742846. The existing five-direction saga check also passed 5/5 on the same tuple. PR CI passed all ten checks. Task containers, network, and volumes were removed and their absence verified.This covers Rust-parent cold replay before compensation for three runtimes. PHP/Python-parent restarts, interruption during an activity side effect, duplicate delivery, and compensation failure remain unproven; #122 stays open.
rmcdaniel commentedon Sep 24, 2026
PHP- and Python-parent saga restart directions are now implemented and merged in Sample App #106 as
680ce55. Published local tuple: Server 2.3.14 (Workflow 2.0.16), PHP SDK 2.0.11, Python SDK 2.0.0, Rust SDK 2.0.7. PHP parent run01m397pj801ewhvbqwe1wdntrwand Python parent run01m397tdwma5t84bacn29m5f54each persisted the first reserve and condition wait, lost its parent worker, received the signal while that worker was offline, and completed after a fresh parent process with one signal, no duplicate reserve, the planned decline failure, and ordered Rust undo-second/undo-first compensation. The verifier distinguishes PHP/PythonConditionWaitOpenedfrom RustSignalWaitOpened. Existing five-direction saga check passed 5/5 on the same disposable stack; PHP syntax and Python compile passed, and all PR checks passed. Task containers and volumes were removed. This closes the PHP/Python-parent restart gap, not activity-side interruption, duplicate delivery or compensation failure; #122 remains open.rmcdaniel commentedon Sep 24, 2026
Terminal saga-compensation failure increment shipped on 2026-09-24: Sample App #107 merged as
5b5ecf6d6f3e5685a187ae1f6611bba8bdc0f854; conformance runbook #135 merged as08a375d7ddd4c8d4f64a6a443a867ed185876399. Both required check sets passed.Focused published-artifact local execution used Server
2.3.14(Workflow2.0.16), PHP SDK2.0.11, Python SDK2.0.0, Rust SDK2.0.7, an isolated MariaDB/Redis stack, and three registered workers. On the tested sourcecc7c3ea, all five Rust-involving directions passed the new--compensation-failuremode: the planned forward activity andundo-secondeach persistedActivityFailed, noundo-firstwas scheduled, and the workflow ended with persistedSagaCompensationFailedcontaining both failure identities. The default successful compensation client also passed 5/5 on the same tuple. PHP/Python syntax and locked Rust cargo check passed. The stack, workers, volume, network, and build scratch were removed. The manual result is not an automated Server-runner Rust shard.The issue remains open for the broader experiment inventory and failure modes not exercised here, especially process loss during a side effect, duplicate delivery, and real external-effect recovery. The PHP-parent result also exposed that a cross-SDK client should use the persisted
exception_typefor failure-type identity:exception_classcan be a broader runtime class. The experiment now asserts the persisted type and history rather than inferring type from the client class field.rmcdaniel commentedon Sep 24, 2026
Heartbeat inventory finding: the current public conformance row already names focused published-artifact heartbeat runners for PHP, Python, and Rust, plus a shared-server wave. It is incorrect to treat the old PHP/Python-only experiment wording as a missing SDK implementation, or to infer simultaneous cross-SDK visibility from three isolated passes. The local experiment guidance has been aligned to the actual public runner and CLI contract. No product heartbeat capability or new conformance pass is claimed by this documentation correction. Next inventory work should identify categories whose executable runner truly omits a supported SDK or direction, then run that cell with exact published artifacts.
79 remaining items
rmcdaniel commentedon Oct 23, 2026
Search-attribute receipt independently audited: existing main Action 39940000974, attempt 1, at Sample App
40be451c020c6a0e8c9561585bcd96d38a352a75. Unlike the earlier metadata-only artifact reads, this audit downloaded and parsed all nine retained JSON reports. ZIP SHA-256f66b65a33a82801569a62485cf5e95f0e9391bc4496ebb43c9c66f662c7b6498matches artifact 12496106826 metadata (sdk-search-attributes-39940000974-1, 10,473 bytes).The reports preserve one original workflow/run across Python and PHP observers. The complete four-event parked history is unchanged at the start of the nine-event terminal history; its next-page token is null. The terminal history contains two search-attribute upserts, one original signal wait, one received/applied signal and one WorkflowCompleted. PHP/Python attribute observations match before and after, across all seven declared types; eight initial attributes become seven after the deliberate deletion. The recorded result identifies Rust execution, and the pinned actor source uses Worker::new/register_workflow, ctx.upsert_search_attributes and Worker::run for that authored workflow.
Physical-loss reports show the same original container before/after SIGKILL, exit 137, not OOM, no running process, then a distinct running replacement. The acknowledged SignalReceived timestamp falls between original container termination and replacement startup. This corroborates cold original-run recovery from the retained raw evidence, rather than relying on the final pass line alone.
Run logs select Server 2.10.0 at
sha256:d69f9009d6bb3a4fb60c3516c7268a99c832937eacd24fe5a078564076b5f427, PHP 3.1.0, Python 2.7.0, Rust 4.1.0, CLI 2.3.0, selected Workflow 2.11.3 and Waterline 2.12.1. These are the recorded selections, not a new installed-version audit or a newer tuple. PHP/Python are metadata observers here, and this receipt does not qualify their workflow-side authoring, Rust 4.6+ list-client APIs, Waterline/browser views, namespace isolation, load latency, continuation inheritance or application-code upgrades. No runtime was rerun and no qualification cell outside this original was promoted.rmcdaniel commentedon Oct 23, 2026
Delivered #259, merge
c5219d98b02cf64403319dd74793d60b3c3f1317. All seven candidate and six independent main checks pass, with direct PR Sonar OK/zero unresolved findings and main Sonar OK at this exact revision. Candidate752772ac4f0a9c9cba1d4d484adca5bba09d16dfand merge share tree276a92247aba0c3fba91eaa722c14fe09fb36fda. Published coverage blobafe9ed777b24ba694d9ef179f40477ddce6b3440, SHA-2567e7f08f181cc2febb92853d48494be7b4dcbf790f6c920d7525404038cdc0a0e, independently matches tested bytes; the merged remote branch is absent.The guide preserves Server's real PHP Workflow-package and Python SDK compensation observations and Sample App's five Rust-involving service-SDK directions. It leaves the three non-Rust PHP service-SDK directions unqualified and keeps restart markers separate from external-effect recovery. No saga or runtime was executed for this prose correction. #122 remains open; the source/evidence audit continues.
rmcdaniel commentedon Oct 23, 2026
The child runner source has an execution-context prerequisite missing from the shared runbook. This is a documentation correction to an existing probe, not a new runtime capability or a passing cohort.
At Server
c36710e860a44c5d51f016eadabd49f5b85c451e,run_published_matrix_probeskips execution unless the runner is in source-free/appwith the bundled PHP runtime. It checks the selected public artifacts and invokes the image-bundled probe over an isolated SQLite scratch database. The PHP actor uses the Workflow package andWorkflowFiberRunner, while the Python actor uses its SDK adapter. The declared matrix namesworkflow-phpandsdk-python; this is not a PHP service-SDK Worker matrix.The runner blob
944a7feeb7dfe067aeb13c5db904a1bc7780e510still matches current Server main. I ran only its early--helppath: exit 0, 1,498 output bytes, SHA-256381e21cd4ce9e9b4b79aa2942e1d57001fd0f19dacfd992cfe306587e266b9b3. No artifacts were installed and no runtime probe was launched. Source requires install evidence, usable full-matrix evidence and passing required scenarios; a checkout skip or shell exit alone is not qualification.I am correcting the
.githubrunner table/prerequisites and actor scope. The existing Sample App host command remains the service-SDK/browser entry point. Its recorded results, the separate #268 observer failure and all unexecuted directions remain distinct.The artifact probe requires
DW_WORKFLOW_PHP_VERSIONto match the package already present in the image's Composer installed metadata. It does not install a replacement Workflow package. Python and CLI are installed separately; Rust and Waterline references are resolved without adding their runtime cells. Documentation correction is now draft #260.rmcdaniel commentedon Oct 23, 2026
Read-only replay provenance audit: scenario names containing “restart” or “upgrade” do not by themselves identify a process-loss or package-upgrade experiment.
6de015f2f5b28556d7149ef1db263ab3ab3aa403to source04116871c83e6a502c713789cb452e90917a3438. Its replay module blob721d9d9c7e63542efff748ecee5f8cc81fd079ffis also unchanged in current untagged candidate 4f853f3. The restart scenario reconstructs query state from golden history and explicitly emitsworker_restart_simulated: true. Its in-flight signal timing case replays a constructed partial history. These are installed replay/decision controls, not live signal arrival across an OS worker restart.5e2af20fde934e05bf18f64111dce8e590ea6c90includes the auto-discoveredsrc/bin/durable-workflow-replay-conformance.rsbinary, blobdce81427d4fa3ce2b96a7a5223151c93bdacdca5. It starts an actual managed Worker against the selected Server, completes an original and a continuation chain, then gracefully stops that Worker. It creates a second Worker in the same process, widens the version range from 2 to 3 and queries the original replayed state, requiring the original marker and one side-effect callback. It does not launch a second executable, kill an OS process or replace a registry package.This audit read source only. It did not install these commands or run a new Server/worker cohort. Existing source replay controls, installed runtime receipts, process-loss experiments and SDK/application upgrade records retain their separate evidence. The runbook already says corpus replay does not establish process recovery or migration; the source details above substantiate that distinction without opening a competing implementation slice.
Published-distribution verification now independently matches those reviewed source bytes. The actual PyPI
durable_workflow-2.9.0-py3-none-any.whlis 302,107 bytes, SHA-2561e7547e895485dc8f670bd1b9cc34184b135f249db8da98b16b250bfc6002f73; its replay module is blob721d9d9c7e63542efff748ecee5f8cc81fd079ff, and its console entry point isdurable-workflow-replay-conformance = durable_workflow.replay_conformance:main. The actual Rust 4.7.0 crate is 481,689 bytes, SHA-256dc6c0353e940a068f4a71eef8dc516d920c88be34f9c68e205f05ec10c34e65b; its packaged replay source is blobdce81427d4fa3ce2b96a7a5223151c93bdacdca5, normalized Cargo manifest declares that binary/path, and.cargo_vcs_info.jsonidentifies5e2af20f. Both archives were checked in memory against registry digests. Neither command was installed or executed in this verification.rmcdaniel commentedon Oct 23, 2026
Two more existing runner boundaries are now source-verified. They need clearer shared instructions; neither establishes a newly executed cohort.
Compatibility skew. Server c367’s scenario policy explicitly selects one exact current artifact tuple and records
historical_prerelease_packages_installed: false. Its pairing generator derives current/older/future/outside-window protocol shapes from the Server’s advertised authority. The four surfaces are CLI, Python SDK, PHP SDK worker and Waterline; Rust is not in the required artifact/surface list. “Backward/forward skew” here means protocol negotiation/refusal under that selected tuple, not installation of historical or future SDK packages. Package upgrades and supported older-package combinations require their own actual installed evidence. Existingrequest-response-captures.json, per-surface and per-pairing results carry the actual operation/refusal evidence.Migration.
migration-published-artifacts.sh --helpdescribes an evidence composer. It accepts full/sharded host evidence or an explicit foundation command plan for real v1 setup, migration and post-upgrade checks. The main composer reads that evidence and conditionally runs the supplied plan. Without a supplied plan, automatic plan execution returns without running migration commands. Its pass predicate rejects missing capabilities/cells, artifact and command failures; storage connectivity alone is insufficient. Do not report ordinary wrapper invocation or artifact resolution as a completed v1 durable-state migration.Only early
--helppaths ran, all exit 0 with no stderr: skew 2,912 bytes / SHA-2564215d1be7998d23e86aa03ea0ef59317fda2649ff478ccdcaee09c882d339aad; migration 6,930 bytes /0f2851f8707ad47fe186738e969646fb5a4242c520696d4511896bc7e5672cc1; replay 1,719 bytes /a18507a121df1f24d338b9b5a99c3e35b6edcf32d15c11a006649abe9e1f63b8. No host plan, candidate startup code, artifact installation or runtime was executed. Current helper blobs: skew mjs64d1e83942612c73ad5ac1526ee541324ed943b9, migration mjsab6736350164d6f7fd2d0f67e67292df2f25a5de. Existing migration and Rust skew coverage requirements remain open.The focused correction is included in combined PR #260, head
65511faf36f684dca88beeef9ae5443919b3c18a. Its two Markdown files cover child execution context and replay/skew/migration evidence boundaries. Normal checks must qualify that exact combined head; the prior child-only shared-test pass does not qualify the new tree. No runner or qualification cell changes.rmcdaniel commentedon Oct 23, 2026
Current consumer source and child receipt follow-up:
Sample App main still resolves to
40be451c020c6a0e8c9561585bcd96d38a352a75. Its qualified tuple, blob75676e32a09c824fd42549bf8ad30c85fef9ed29, SHA-256037c6d201c54ba8fa4c306b921cacacb64236465eed123511bd6b3da2f823aa1, selects Server 2.10.0, PHP SDK 3.1.0, Python SDK 2.7.0, Rust SDK 4.1.0, CLI 2.3.0, Workflow 2.11.3 and Waterline 2.12.1. It separately declares Server’s bundled Workflow as 2.9.0. The resolver preserves those separate roles. A selected standalone package pin must not be described as the image’s installed Core version. This follow-up reads manifest/log declarations, not another installed-image archive audit.I independently read the existing successful main push children run 39940001164, attempt 1, at that exact source. Structured log records contain:
outcome: passand 15 views: legacy, runtime and anonymous. Total 45 views.The corresponding artifact is
sdk-children-39940001164-1, ID12496835549, 7,079,279 bytes, API digestsha256:2ad36a68c5dd3987a32d023de3c3c5c6710ad7ba7743d2a2f6c747e60ee81972, retention through November 22. One bounded ZIP read timed out after 15 seconds. Individual raw histories, screenshots and browser response files were not independently reread in this follow-up. The normal run and logged assertions remain their original evidence; this was not a rerun.#268 is still open and the observer source remains unchanged. A successful original on the same source does not correct its independently reproduced late-observation false refusal or qualify the failed originals. This frozen PHP-Server tuple also does not qualify newer SDKs, native Server, the four non-Rust failure/recovery directions or additional cancellation policies.
Related source distinction: Server c367’s separate activity probe installs the selected public Composer packages and prepends their Workflow/Waterline/PHP SDK loader, then checks class locations with reflection. That published-package composition differs from the child probe, which uses the image-bundled Workflow package and requires matching installed metadata. Keep the loaded-package identity specific to each runner; a separate selected package pin is neither proof of the image’s bundled version nor proof that another probe loaded it. This is source inspection, not a new activity outcome or a claim about the frozen Sample App image contents.
rmcdaniel commentedon Oct 23, 2026
Published-client API follow-up confirms the existing creation/inspection gaps still hold at the reviewed releases. This reads package contents as data; it does not install or execute an SDK.
The actual Python 2.9.0 wheel (
1e7547e895485dc8f670bd1b9cc34184b135f249db8da98b16b250bfc6002f73) containsclient.pyblob97287b359f948bad4e1766b62df434ac2eb5a031, byte-identical to source04116871c83e6a502c713789cb452e90917a3438.create_scheduleis an actual client binding. Python schedule creation remains a supported authoring surface.list_activitieslists standalone namespace activities, not a workflow run’s activity projection. The accepted typed run-activity requirement #235 remains open.get_historyreturns one page, whilelist_workflowsrequires explicit next-token calls. ExistingScheduleHandle.iter_historyanditer_workflow_streamiterate schedule history and named stream items respectively. They do not establish an iterator over the full run audit/history or workflow list required by #247.The actual Rust 4.7.0 crate (
dc6c0353e940a068f4a71eef8dc516d920c88be34f9c68e205f05ec10c34e65b) contains library blobcbd5695f4310fd9f49c862de02d5e74fc70df721, matching source5e2af20fde934e05bf18f64111dce8e590ea6c90. Review of its Client implementations confirms no typed schedule-creation or run-activity inspection wrapper. The package’s/schedulesreferences occur in its metrics route vocabulary, generic payload handling and payload tests. Those references do not create a public schedule API or qualify a Rust schedule creator.The shared guide’s PHP/Python schedule-creator split remains accurate. Rust’s published history/list iterators and separate installed receipt retain their existing scope. No new contract, implementation slice, priority change or runtime qualification follows from this package/API inventory.
rmcdaniel commentedon Oct 23, 2026
Delivered #260: the shared instructions now identify the child probe’s image/bundled-package context, Python fixture versus Rust Worker-instance replay, protocol skew under one package tuple and migration’s required host evidence or supplied plan.
Exact candidate
65511faf36f684dca88beeef9ae5443919b3c18aand merge4fb961325779c7d39cfb0c35675020625194a7f4have equal tree6bb40c8aac2d663ed65fe5be47ab9b67254b4c5f. All seven candidate checks pass. All six independent main checks pass, including retained shared controls, Sonar and three CodeQL languages. Direct Sonar API reports PR gate OK / zero open or confirmed PR issues and main analysis1e60c79d-df14-4acb-a30d-81f499411d04at exact revision4fb9613, gate OK.Independent current-main readback matches the tested document bytes:
conformance/README.md75a542cae8ad17d72ea20eebf995cd5de2b8d1229a702a28134ab908add806409cf30681cb4a88e76fe44ea0c3e10ee1999ed21dconformance/sdk-coverage.md569ce0c167adb55ec920af392b5dcb3453ebe57bd0f74f671fa806c23f0144b4f3b89d2d015cdf1dd4565f9892cde13bec10d788Only these two Markdown files changed. The four early runner help paths exited 0, the new anchors resolve, and the reviewed Python/Rust replay command code matches actual registry archives. These checks launched no runtime cohort. Historical results, the #268 failed originals, unexecuted directions and newer/native tuples retain their separate qualification. #122 remains open for its unchecked acceptance criteria.
rmcdaniel commentedon Oct 23, 2026
Published Server release provenance follow-up, scoped to the ordinary PHP image record:
The latest GitHub release inspected is Server 2.11.1, tag/source
5336a98dd256e046edfdb67c81764cde4babd360. Its successful release run 39768142737, attempt 1, has two jobs: image build/publication and matching immutable Helm-chart publication. Both passed.The image job’s recorded build/exact-publication digest is
sha256:78df0eb915064d691773c6364329d997f1e429810514e14baaa41bcd8edf7e04. Its Workflow input is 2.10.2, source/dist commitaca7928484083635e68a5a384b2dcecef842108e, matching the release’s Composer lock entry (blob2c75e54120d0412c5eb8eaf6391995054455f525). This reads the recorded digest/provenance and locked inputs. It is not another independent image extraction or installed-file audit.The exact workflow pins both tooling and source to the tag run’s identity. Actual successful steps verify exact image publication, bare first-run readiness and source-free Compose bootstrap, database TLS/certificate refusals, catalog convergence and chart publication. The production Dockerfile uses the PHP base and Apache entry command. Native source changes listed in the changelog do not turn this release record into native-runtime qualification.
The catalog verifier reads Server discovery and the public catalog, checks their metadata/provenance and writes its own catalog result. The actual log reports catalog version 16, 16 capability records and the same Core 2.10.2 commit. This is catalog/discovery conformance. It does not replace the separate three-SDK authoring, cancellation/recovery, package-upgrade or browser experiment receipts, nor rerun the frozen Sample App tuple.
I read release metadata, step conclusions, the public PHP-image job logs and immutable source. Artifact
release-image-publish-evidence, ID12423746072, is 4,403 bytes with API digestsha256:3c73ff03feb87f93970fb0bfefd610bd6e011df38b96e7a3bd3fdb925e20a6f9; one bounded ZIP read timed out after 15 seconds, so the raw JSON files were not independently reread. No new image was pulled, runtime launched, source changed or release retried by this audit. Existing current-package acceptance remains on its own recorded actors and tuple.rmcdaniel commentedon Oct 23, 2026
Frozen Nexus main-run follow-up corroborates the documented PHP/Python actor scope without adding a current-package or Rust cell.
Latest inspected main execution is run 39114545513, successful push at source
c38f8e34f80977d25e3d7c83227ca7a58d40d1a4. Its workflow pins select Server 2.9.1, digestsha256:1631181c266cc74d70713e16b0688ea08b579cf2ed0e28f0ef5a3f3d786bc3d6, PHP SDK 2.5.0, Python SDK 2.6.0, Workflow 2.6.1, CLI 2.3.0 and Waterline 2.11.2.Actual bounded
Nexus SDK outcomesnotices in its logs record all four PHP/Python caller→service directions, each in shared and separate caller namespaces:uncaught_failure: true, failed caller status and successful verification processes. I independently checked eight distinct run IDs and eight distinct call IDs in those notices. All logged elapsed values are below the original 30-second bound; maximum 3.8681271076202393 seconds.The PHP caller and service worker import the service SDK’s Client/Worker/WorkflowContext. The Python actors import their SDK Client/Worker as well. These workflow-run actors remain distinct from the generic composer’s older Workflow-package invocable-carrier cells.
The workflow, generic Nexus wrapper and these four actor helper files match their current c367 source blobs. This comparison covers those six files, not a new whole-tree runtime qualification. The workflow’s declared retention is one day; its artifact API currently returns no artifacts. I did not independently reread
outcomes.json, full histories, omitted reports orcomplete-suite-exit-code.txt, and do not infer the complete suite’s exit code from this focused workflow’s green conclusion.No new workflow was dispatched or runtime launched. The complete scenario manifest retains its uncovered cells. This Server/PHP/Python tuple remains separate from Rust’s 72 selected installed HTTP originals, current SDK releases, catalog browser rendering and native Server qualification.
rmcdaniel commentedon Oct 23, 2026
The failover/operator-loop source audit found a shared documentation gap: both existing runners appear in the coverage map but are absent from the main entry-point table, and their actual execution inputs/actors need explicit instructions.
single-region-failover-published-artifacts.shrequires Bash, Python 3, Docker/Compose v2 and an exact public Server image. It resolves the Server/MySQL/Redis/Nginx images to digests. Its Python driver registers a worker and supplies start/poll/complete HTTP messages directly; theruntime: pythonregistration field does not establish Python SDK Worker execution. Its phases cover cross-node completion, API-node loss, database/Redis interruption, worker lease loss and scheduler restart. Cleanup calls project-scoped Compose down with volumes unless the stack is retained.agent-operability-published-artifacts.shfirst consumes usable Sample App metadata. With no such evidence and an explicitly provided MCP URL, it drives the MCP workflow tools. It starts a synthetic diagnostic workflow, reads its failure, diagnoses, requests repair and reads history. Missing inputs produce a coverage finding. It does not provision the Sample App. Its five stages include a guarded operating choice; this is separate from the six-case operator diagnosis drill and SDK-worker recovery.Only early help paths executed, from byte-identical temporary copies removed immediately afterward. Both exit 0 with no stderr: failover 1,447 bytes / SHA-256
8049acf5b71f542b51be7d5a368101c5c278933dbc9a0bb9eebf9bbf18634480; operator loop 1,521 bytes /6cf3b4532319df950216902632093c4b456ee25ee73951bf6a6002a33d9cf81d. No Docker stack, SDK, MCP endpoint or fault was exercised.Product decision: clarify these existing commands, inputs, result interpretation and cleanup. This adds no runtime capability and does not qualify new SDK/recovery cells. I am preparing the focused
.githubrunbook correction; no Server source change or new runtime matrix is needed for that documentation change.rmcdaniel commentedon Oct 23, 2026
Pinned SDK-matrix entry-point audit at Server main
3f448d3099848d6d8ec5971432bd8b60f26dfec3. These are source/help checks, not an executed SDK/runtime cohort.php-sdk-published-artifacts.sh, blobed0c37d24e9944a324bb0aa3d5e552d504bcb0ed, uses host PHP, Composer, Node and Python. It installs the selected published SDK into a disposable Composer project, then runs separate PHP client and managed Worker processes against an already prepared Server endpoint. The runner requires exact SDK/Server/image inputs; it does not provision that Server. Lifecycle, namespace and search-attribute scopes are distinct.--validate-definitionsstill installs the package but exits before Server execution. It does not qualify the runtime.php-sdk-conformance-result.json, the lifecycle sidecar, scope-specific assertions and consumed distribution identities. A successful shell exit is insufficient. Its exit trap terminates its captured worker, while result files, Composer project/cache and the separately owned Server remain. The lifecycle restart uses TERM followed by a new worker process; it is not a SIGKILL claim.python-published-artifacts.sh, bloba2c7c53bb3ca30b09d13293422a27735079dde2f, requires Docker Compose, Python with venv support and curl. It resolves exact public Server, CLI, Python SDK, Workflow and Waterline inputs, starts its own isolated stack and uses the installed SDK's composer/evaluator. Set the existing exact-version variables when qualifying a selected tuple, and retainpins.jsonplusexecuted-distribution-identities.json. It writes host, result, evaluation and record files; the evaluation must pass.not_covered, even when the standalone subresult passed. It cannot be promoted to an overall pass. Account and operational qualification stay private. The cleanup trap attempts namespace deletion and removes the same Compose stack/volumes. It retains the scratch root on failure or explicit retention;--keep-run-rootretains files, not the runtime stack. Successful runs remove that root only when results were written outside it.Both exact source copies passed
bash -nand their early--helpexited zero before dependency checks or provisioning. PHP help: 2,033 bytes, SHA-2568001ff1fd0f3d27352b6e896c830168b0295958151897f2b426c6dc77a35d27e. Python help: 690 bytes, SHA-256d721db075607745c40ab9e3ca10349532679a404bceb4f2186ce37de3f48b65e. Temporary copies were immediately removed. No dependencies were installed and no runtime was started.Product decision: these existing public entry points need concrete prerequisites, result interpretation and cleanup in the SDK-matrix instructions. This is an instruction gap, not another contract or a claim that the remaining SDK/failure cells have passed.
The same audit also reads Sample App
40be451c020c6a0e8c9561585bcd96d38a352a75scripts/playground, blob703ad17db011064eb99de97a34bebede30bdf8e7.scripts/playground rustis one caller-owned workflow/activity journey, with effective contract/result/evidence, not a complete Rust parity suite. It preserves existing source. Its finalizer terminates the worker but retains local services unlessPLAYGROUND_CLEANUP=1;scripts/playground down rustremoves the same selected project. Source and evidence remain. The actual Rust subcommand's early help exited zero from an exact temporary source copy, before journey execution. No toolchain, scaffold, runtime or managed request was started; the copy was removed. PR #263 includes this boundary and cleanup.PHP heartbeat scope within the same pinned parity entry point: the generated worker requests a heartbeat interval and calls
Worker::run(1). In PHP SDK sourcecb1d11e57767e20e1fdb8081024f67057ff66714,runtakes a poll-timeout argument and drives the managed loop;1is not a one-task execution limit. However, the parity aggregate'sworker_heartbeatassertion checks onlylast_heartbeat_atpresence on the two registration snapshots. Its readiness helper, blob91c0c134cd4e76b8a7796b0b673d33c753820524, waits for the authoritative command contract, without a timestamp-advancement assertion. This is an existing managed-worker fixture to reuse for focused heartbeat work, not evidence of renewal cadence, stale transition, deregistration or routing exclusion. The dedicated heartbeat qualification gap stays open. Source inspection launched no worker or Server.rmcdaniel commentedon Oct 23, 2026
Current published-API follow-up, keeping these distributions separate from frozen consumer/runtime cohorts:
62a78da1c5bdc2af607c11eede5c56a3d25c2159e5f2266c72176f5aa8dda115. Its client module has Git blob3c9040e6a1bd2427eb515bd81026006cabff86d3, matching the qualified/tagged source4f853f396dba5a530ef0815509b16bfddaa9ebc3.Client.get_historyreads one page andlist_workflowsrequires the explicit next token. Named workflow-stream and schedule-history iterators remain separate APIs.list_activitiesreturns standalone namespace activities, rather than typed activities for a selected workflow/run. Schedule creation is present. These method surfaces preserve the distinctions in Let SDK clients read whole run histories and workflow lists without hand-written paging #247 and List a run's activities from the Python and Rust clients #235; package publication does not complete those accepted capabilities.be21bb5b033cc8db364f1f9e3d789607eb6eff976126965cfc95072146c220a6. Its packaged VCS identity is93d3c3aa78be4347045f50833ffcdc02dcabc4ed, matching the immutable4.9.0tag. Inspection of the crate's 55 Rust source files finds no schedule-creation or run-activities client binding.iter_workflowsanditer_workflow_historyremain public lazy APIs. The latter selects one run; it does not follow continue-as-new or subscribe indefinitely. Source inspection is separate from the earlier installed Rust 4.6 multi-page receipt and Rust 4.7 metrics delivery baseline. Neither is described as the latest package.No packages were installed or executed and no runtime cohort was started. Both distributions were inspected in memory. Python 2.10.0 metrics delivery is now recorded separately in #233. This current API inventory neither upgrades Sample App's frozen
40be451ctuple nor qualifies native Server or a newer cross-SDK experiment.rmcdaniel commentedon Oct 23, 2026
PR #261 is delivered as merge
d5d5782a2a79b4a289e687611a341feab6984ce5. It names the existing failover/MCP entry points, actual actors, inputs, evidence and cleanup, and preserves the separate six-case operator drill and known cleanup-timeout result-loss limitation.All seven normal candidate checks passed for final head
44ca3909af642d21b949701c443231c8b4037ae5. All six independent main checks now pass: shared controls, Sonar analysis and three CodeQL languages. Direct Sonar analysis57be8f21-cca8-493e-bc36-a55bf9483c03identifies that exact main revision and reports quality gate OK. The candidate's direct Sonar gate was OK with zero open/confirmed PR findings.The tested candidate and published merge share whole tree
aa61ad21432ace444bbed984fb3a7ff56833db81. Independent published-byte verification confirms:conformance/README.md: blob4a2bfcd328494e745085a37f548534edbfd1d2c4, SHA-2562090ad5edcd8cc43cb894777b5cfbcf31e6a4675e1d9ac12e97e97b8641943e7.conformance/sdk-coverage.md: blob3458249d69f4e1c022b24cbc2274e3ac8c47c507, SHA-256e146fa2cee2a90e8be15eb472d35b02b6739d7bab6daeb5baf14d8be9e3f642d.Only those documents changed. The merged head branch is absent and its completed source checkout was retired. The early help/source checks started no Docker, SDK or MCP runtime. Remaining runtime cells, the reserved runner correction and #122 acceptance stay open.
rmcdaniel commentedon Oct 23, 2026
The existing runner documentation slice is delivered through #263, merged as
64fd4a9c05f88e92ab65a90e3a152d8e17c22dd3, whole759befc902f0a4b069c48eb86288efe8f17d44f4. All seven candidate checks and all six independent main checks pass. General rechecked the final Actions result, both public document blob identities and their exact sizes after accepting the owner handoff.This clarifies the PHP, Python and Rust entry points, caller-owned prerequisites, distinct exit/result contracts and scoped cleanup. It does not add a runtime acceptance cell. The managed heartbeat, frozen/native cohort and remaining installed-consumer gaps in this issue remain open; PHP metrics delivery is still tracked separately under #233.