Skip to content

Support slow payload hydration in cooperative workers without a five-second cutoff #150

Closed

Description

@rmcdaniel

Observed published failure

Normally Composer-installed PHP SDK 2.3.6 8ee9cc4c322a8a85e4005078f56d53a292edf01f, PHP 8.4.26 and immutable published Server 2.6.1 ef043e5c582761e486a64a2ab4226476e17ae4f71682ef59d6b9ddd784d246b6.

An explicitly cooperative Worker uses default protocol 1.20 and default run() polling. Before admission, authenticated operator/worker discovery verifies external storage available. A fresh original workflow carries an external 8 MiB input. Its encoded body is 11,184,852 bytes, with the response limited to 512000 bytes/second. The original run/execution budgets are 1800 seconds. Worker PHP limit is 512 MiB and container limit 1 GiB.

Original run 01m4saa5xs0ssc3bq1yy3am003, original task 01m4saa5y26p90qjdh7znk2qp6, attempt 1, is accepted. The initial lease expires at 00:25:02.829802 UTC. The Worker renews it once before reading. At 00:24:07.932 it fails while fetching the initial input: cURL timeout after 5001 milliseconds, with 2,469,384 of 11,184,852 bytes received. No workflow callback or activity is dispatched. The Worker exits 255 and deregisters, without OOM.

The ordinary Worker in the distinct paired run completes the same large input/activity/timer preparation and its subsequent three-reference query. The query returns the exact input/echo hashes in 69.416 seconds with continuous presence and one original registration.

Source mechanism

Client::request replaces the bounded poll request budget with a fresh five-second budget for the entire claimed response's hydration. RuntimePayloads takes the bounded download branch, which does not forward the incremental progress callback.

Client, RuntimePayloads and Psr18Transport are byte-identical between published 2.3.5 and 2.3.6. The live observation here is 2.3.6, not an independently executed earlier-release claim. This is separate from #146's ordinary query presence correction and does not establish any historical Cloud, provider or database cause.

Required outcome

  • Keep control RPCs bounded, while supporting valid slow task payloads under an explicit finite transfer budget.
  • Establish claim/presence/cancellation supervision before hydration and service it during bounded streaming. Cover workflow input/replay history, query history and activity input. Preserve original run, query, call and cleanup deadlines, claim identity and publication fencing.
  • Stop partial loading before application dispatch on shutdown, revoked registration, lost claim, cancellation or original budget expiry. Do not turn off bounded requests, restart deadlines or re-register to hide lost authority.
  • Qualify ordinary and cooperative modes against unchanged published Server, including slow bytes, integrity refusal and actual interruption. Add focused streamed regressions and pass normal gates.
  • Publish a compatible correction and verify normally installed consumers before advertising the cooperative slow-transfer outcome. Coordinate Cloud separately.

General owns this PHP correction. Rust SDK activity input supervision #90 remains separate, and Server native/shared implementation remains reserved to its owner.

Activity

  1. added
    kind:defectA public product behavior is incorrect
    priority:P1High-priority product or release risk
    on Oct 13, 2026
  2. rmcdaniel commented on Oct 13, 2026

    @rmcdaniel
    MemberAuthor

    PHP 2.3.7 is published at the immutable tag 275b3bf5d82fa334e6ff1f5ba21e2b4dd55ddf78. Release, stable qualification, exact merged CI.

    The PHP SDK maintainer now owns normal Composer installation, exact package metadata and affected ordinary/cooperative HTTP consumers, followed by the main package metadata update. Those checks remain pending and are not inferred from publication or source qualification. Sample App and broader consumer follow-through stay with their existing maintainer. #150 and #146 remain open until those outcomes are verified.

    The stable backport contains the payload supervision fix and 2.3.7 declarations only. Pending Nexus and held major work are excluded. All completed isolated source/stable runtime resources have been removed.

  3. rmcdaniel commented on Oct 13, 2026

    @rmcdaniel
    MemberAuthor

    PHP SDK 2.3.7 is now verified through a normal Composer distribution installation. Version, source and dist references identify exact 275b3bf5d82fa334e6ff1f5ba21e2b4dd55ddf78; the installed Composer manifest and quickstart identify 2.3.7, registration identifies durable-workflow-php/2.3.7, and the ordinary Client keeps default protocol 1.20.

    Published service-mode smoke 38333253204 passes the declared Server 2.5.0 baseline with a real class-oriented workflow result and normal worker shutdown. Its separately selected immutable Server 2.6.1 cell passes scoped cooperative discovery. Discovery alone is not an affected slow-transfer execution result.

    PR #153 aligns main's two release declarations with 2.3.7. Remaining work is the fresh normally installed ordinary/cooperative slow-payload consumers against unchanged published Server 2.6.1 and the affected shutdown/authority controls, followed by required Sample confirmation. Original query/lease/caller/timer/run budgets remain fixed; earlier source failures and limits remain intact. #150/#146 stay open through that follow-through.

  4. rmcdaniel commented on Oct 13, 2026

    @rmcdaniel
    MemberAuthor

    PHP 2.3.7 now has normal registry installation and affected published-HTTP qualification. This receipt is separate from the GitHub release and from the earlier source qualification.

    • Normal Composer installation resolves version 2.3.7, source and dist 275b3bf5d82fa334e6ff1f5ba21e2b4dd55ddf78, reviewed tree 314a3858f16cd260e46c735b7c25e1ad65fec49b. There is no path/VCS/source overlay. Installed product train and quickstart are 2.3.7; registration is durable-workflow-php/2.3.7, default protocol 1.20, minimum 1.19.
    • Published smoke passes the real installed class-oriented workflow against SDK-declared Server 2.5.0, including explicit SIGTERM shutdown. Its separate cooperative discovery against immutable Server 2.6.1 is discovery evidence.
    • Metadata PR #153 is merged at 3c597e477abb28aae0c011ec2af111a41b08153d. All 33 ordinary candidate gates and fresh merged-main CI pass; the live PHP quickstart declares 2.3.7. This metadata merge does not publish pending main features.

    The affected slow-transfer cells separately use normally installed 2.3.7/PHP 8.3.35 and unchanged published Server 2.6.1, digest sha256:ef043e5c582761e486a64a2ab4226476e17ae4f71682ef59d6b9ddd784d246b6. The selected budgets remain query 180s / initial query lease 185s / controller HTTP 210s / durable timer 900s / run and execution 1800s. Default protocol 1.20 and poll 5s remain unchanged; each external response is limited to 512000 bytes/s. Worker PHP limit is 512 MiB and container limit 1 GiB. Published maintenance/repair commands are installed before seeding.

    Ordinary original run 01m4sew8t2x0597z1smh249fbs and cooperative original run 01m4sf4z355d3pmtyxw97r62t5 both load the 8 MiB input, complete the remote echo activity once, replay the parent and reach its pending timer. The subsequent queries load all three complete external references and return the exact independently checked input/result SHA-256 1e0d523a7b808c938cdb364a98ac4fa0a1ac05ce07102b6f3c56d8e5be441d11. Ordinary query 01M4SEZDZQYK2FSRYDV2FR5NVZ completes in 83.419s wall, including queue/parent readiness; cooperative query 01M4SF9EZ6T3A8FKSGXZ6HBR7N completes in 72.676s. Each large body takes about 22.2s. These are correctness receipts, not comparative performance claims. Original registrations, first attempts, accepted budgets and one activity effect are preserved.

    Interruption and integrity receipts have distinct limits:

    • Explicit ordinary SIGTERM deregisters and stops with exit 0/no OOM. An earlier ordinary container control inherited the Server tooling image's SIGWINCH and reached SIGKILL/exit 137; that fixture error is retained and excluded from the clean-shutdown claim.
    • Explicit cooperative SIGTERM during query 01M4SFCKBVZRDZ16AQ18Z5E4ZX stops after 253448 of 11184852 bytes, before the stopped worker enters another callback; deregistration returns 200. The original caller times out at 180.346s. A cold same-ID replacement later leases the same query at attempt 2 and enters its callback after that caller timeout; completion is fenced with HTTP 409. This does not establish that the durable query deadline expired before dispatch: caller waiting budget, task deadline and replacement lease authority require separate interpretation. The cooperative process's final Docker exit/OOM state was unavailable, so no exact clean-exit claim is made for that control.
    • Namespace worker-role revocation during query 01M4SFQFWX0XT91S1V0QRRGHN4 returns 200; heartbeat 401 stops partial hydration at 1785352 bytes before a new callback/completion. The original query returns QueryFailed at 180.383s. Expected authority failure exits 255/no OOM; it is separate from clean shutdown.
    • The normally installed payload reader rejects an equal-length HTTP body alteration with external_payload_integrity_mismatch after 22.273s. The original stored object, reference and digest headers are unchanged. This tests the actual HTTP reader, not an executed corrupted workflow task.

    Both original runs are normally terminated and issued roles revoked. Namespace deletion first returns 503 with SQLite object-lock cleanup contention; after draining writers, retry on the same original state returns 200 and authenticated read returns 404. All task resources are removed. Earlier source early-cancellation terminal convergence remains unqualified; these receipts do not broaden it into a pass.

    Sample/cross-repository follow-through remains with General. The required Sample activity-recovery receipt is still pending, so parent closure is held for that handoff.

  5. rmcdaniel commented on Oct 13, 2026

    @rmcdaniel
    MemberAuthor

    Closing the delivered PHP 2.3.7 correction after normal Composer installation, affected ordinary/cooperative published-HTTP consumers, main metadata alignment and the required published Sample follow-through.

    The installed qualification records exact source/dist 275b3bf5d82fa334e6ff1f5ba21e2b4dd55ddf78, product train/registration 2.3.7 and unchanged default 1.20/minimum 1.19. SDK-declared Server 2.5.0 passes the normal published smoke; unchanged immutable Server 2.6.1 passes the separate affected slow input/activity/replay/three-reference query cells. Metadata #153 and fresh merged-main CI pass.

    General's final Sample #211 receipt confirms 45/45 original activity-recovery cases, the expanded matrix, all ten fresh-main workflows and actual installed PHP SDK/Laravel 2.3.7 in both native image architectures. Both public registries publish index sha256:8721e00e1bc5e841df444fcad063929ee6030ddfd768d0c28c4b74c405b8bb3f for merged 9d298b6161def330a2405288ee0164097ddec058.

    The original limits remain: the inherited SIGWINCH ordinary control ended 137 and is excluded from the later explicit SIGTERM exit-0 pass; the cooperative stopped control lacks an exact final Docker exit/OOM state. Its cold replacement enters a callback after the original caller timeout and completion is fenced 409. Earlier source early-cancellation terminal convergence remains unqualified. Credential revocation is an authority refusal, not clean shutdown. No Cloud, Nexus/shared cohort or frozen rewrite tuple is qualified by this closure. All task resources and credentials are retired.

    #112 cache/Octane and #116 Nexus replay/capability-cohort follow-through remain open and separately owned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:defectA public product behavior is incorrectpriority:P1High-priority product or release risk

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions