fix: bound action scheduler retention queries - #3087
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
COUNT(*)scans with metadata estimatesRoot cause
The old log deletion put
actionscheduler_logson the driving side of a join and appliedLIMITonly to the result. On the Events shape, MySQL could examine a substantial portion of the 24.6M-row logs table before finding a batch, so one SQL statement could consume minutes before PHP could observe its wall-clock deadline. The row-ceiling probe also sorted bylast_attempt_gmtfor a hook even though Action Scheduler has no(hook, last_attempt_gmt)index, and the post-pass guardrail ran exactCOUNT(*)scans over both oversized InnoDB tables.Algorithm and scale safety
Each retention iteration now:
status_last_attempt_gmtfor global windows orhook_status_scheduled_date_gmtfor per-hook windows.action_id IN (<bounded IDs>), with an additional rowLIMIT.The row-ceiling cutoff combines three status-specific index scans, each capped at
max_rows + 1, before applying the cross-status offset. This preserves the existing combined ceiling semantics while bounding the filesort input. InnoDB table-size guardrails now readinformation_schema.TABLES.TABLE_ROWS; SQLite retains exact counts.Locking, indexing, and compatibility
actionscheduler_logs.action_idindex and lock no more than one configured log batch per statement.^3.9schema; no production schema migration or table rebuild is introduced.OPTIMIZE TABLEbehavior remains opt-in and unchanged.Verification
php tests/retention-action-scheduler-batching-smoke.php- 39 assertions passedphp tests/action-scheduler-native-retention-smoke.php- 10 assertions passedphp -l inc/Engine/AI/System/Tasks/Retention/RetentionCleanup.php- passedphp -l tests/retention-action-scheduler-batching-smoke.php- passedhomeboy review data-machine lint --path /var/lib/datamachine/workspace/data-machine@fix-3084-bounded-retention --changed-only --summary- passed, no baseline drifthomeboy review test data-machine --path /var/lib/datamachine/workspace/data-machine@fix-3084-bounded-retention --changed-since origin/main --summary- real-WordPress smoke passed, 1 passed / 0 failedEXPLAINonly: global selector usedstatus_last_attempt_gmt; per-hook selector and all ceiling arms usedhook_status_scheduled_date_gmtThe first Homeboy umbrella run exposed inherited real-WordPress filter callbacks in the smoke fixture. The fixture now installs deterministic late-priority overrides and restores only its own callbacks; the final clean-checkout Homeboy test passes.
Operational follow-up
The existing Events backlog still requires repeated bounded catch-up passes; this change intentionally favors incremental progress over a single large drain. Operators should keep
OPTIMIZE TABLEdisabled during the drain unless they separately provision the lock and temporary disk space for a rebuild. Metadata row estimates may lag exact counts, so verify backlog progress through deleted-row results and periodic table-size observations rather than expecting per-pass file-size shrinkage.Fixes #3084