Skip to content

perf(test): add a per-suite timing option and a wall-clock baseline doc with the Linux column #10200

Open
Task
Open
perf(test): add a per-suite timing option and a wall-clock baseline doc with the Linux column#10200
Task
Labels
host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.priority: mediumReal value, no hard deadline; normal backlog flow.status: readyTriaged, unblocked, and fully specified; eligible to pick up.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

Description

@kyle-sexton
kyle-sexton
opened on Oct 26, 2026

Parent: #3716 (slice 1 of 3). Runs first; slice 2 waits on it. Slice 3 runs in parallel.
Runs in: lane. Linux only; the Windows column is slice 2.

Problem

#3716 criterion 1 asks for a committed baseline table of per-suite wall clock on Windows and on the Linux CI runner. The Linux numbers partly exist: scripts/suite-seconds.txt holds median seconds per suite, but they are inferred from CI log timestamps (its header comment), because scripts/run-plugin-tests.sh prints only PASS: <suite> and no per-suite time. A Windows host has no way to produce comparable numbers, and the gate scripts the issue names (scripts/check-purged-em-dashes.sh, scripts/check-changed-skills.sh, scripts/check-changelog-parity.sh, plugins/skill-quality/scripts/check-skill.sh, plugins/planning/scripts/check-open-questions.sh; #3717 merge comment) have no recorded time at all.

Fix

  1. Add a timing option to scripts/run-plugin-tests.sh (for example --timings <file>) that writes one TSV row per suite: suite path, wall seconds, exit code. Same option, same format, on Linux and Git Bash. No behavior change without the flag.
  2. Add a way to time the gate scripts above in their typical single-diff invocation into the same TSV format (a flag on an existing script or one small script; name it in the doc).
  3. Commit docs/test-wall-clock-baseline.md: how to produce each column, a Linux CI column filled from a whole-tree CI run with the new option (cite the run), the gate-script rows, and an empty Windows column that slice 2 fills. Link it from the header comment of scripts/affected-tests.sh.

Done when

  • bash scripts/run-plugin-tests.sh --timings <tmpfile> <one suite> exits 0 and writes one row with three tab-separated fields
  • test -f docs/test-wall-clock-baseline.md exits 0, and the file cites the CI run behind its Linux column
  • grep -c 'test-wall-clock-baseline.md' scripts/affected-tests.sh prints 1 or more
  • The doc has a row for each of the five gate scripts named above
  • The existing suites pass unchanged (ci-status is green on the PR head)

Split by the backlog HITL session (2026-10-25, owner rule Q93: split large work before a run picks it up). This was generated by AI.

Activity

  1. kyle-sexton
    added a parent issue on Oct 26, 2026
  2. github-actions
    added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 26, 2026

  3. github-actions commented on Oct 26, 2026

    @github-actions
    github-actionsBot
    on Oct 26, 2026 – with GitHub Actions
    Contributor

    Proposed work class: work-class: scoped
    Basis: automated intake triage; the lanes App applied this class.

    Size: Medium
    Complexity: Low
    Uncertainty: Known
    Flags: none
    Blocked by: none

    Understood: add a --timings <file> option to scripts/run-plugin-tests.sh that writes one TSV row per suite (path, wall seconds, exit code), and add a way to time the five named gate scripts into the same format. Also commit docs/test-wall-clock-baseline.md with a Linux CI column citing a whole-tree CI run, the gate-script rows, and an empty Windows column, linked from scripts/affected-tests.sh. The done-when items are concrete. The CI-run citation needs a real whole-tree run, which may have to happen after the change merges. No duplicates found among the open issues.

    Automated triage, applied by the lanes App.

  4. github-actions
    added
    priority: mediumReal value, no hard deadline; normal backlog flow.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 26, 2026
  5. github-actions
    added the
    Task
    issue type on Oct 26, 2026
  6. melodic-automation-lanes
    added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.
    admission-deferred
    on Oct 26, 2026
  7. kyle-sexton
    marked this as blocking on Oct 26, 2026
  8. melodic-software-github-iac
    removed
    admission-deferred
    on Oct 26, 2026
  9. melodic-automation-lanes
    added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Oct 29, 2026

  10. melodic-automation-lanes commented on Oct 29, 2026

    @melodic-automation-lanes
    melodic-automation-lanesBot
    on Oct 29, 2026
    Contributor

    This records a re-fire of an earlier refusal, not a new refusal. The lanes kill switch stopped run 40797684275, the work-resolve-issue run that agent-ready started, before it did any work; that run ended at 2026-10-29T01:33:05Z. work-unblock has now removed and added agent-ready back to start a fresh run. A kill-switch refusal is no fault in this issue or the lanes, so it never counts against later re-fires; after limits.limit-stops-per-issue of them since a person last added agent-ready or a run last reported, work-unblock waits longer before each one, up to 24 hours.

  11. melodic-automation-lanes
    added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    and removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Oct 29, 2026

  12. melodic-automation-lanes commented on Oct 29, 2026

    @melodic-automation-lanes
    melodic-automation-lanesBot
    on Oct 29, 2026
    Contributor

    The work-resolve-issue lane stopped on this issue (patch-protected-path): the change needs a path the issue lane may not push, which a maintainer session builds by hand.
    It added host-run and removed agent-ready: an agent run on a host builds it, and it waits on no decision from a person.

    In the run's words:

    The run stopped early (patch-protected-path): its change needs a path the issue lane may not push, which a person changes instead. In the run's words: patch-protected-path: .github/workflows/pr-require-checks.yml. Fix step 3 and the 'Done when' line 'the file cites the CI run behind its Linux column' need a whole-tree CI run that uses the new --timings option. The Linux suites run only from the test-bash job's 'Run this leg's shell suites' step (pr-require-checks.yml line 2925: scripts/run-plugin-tests.sh --jobs 3 --suites-from "$RUNNER_TEMP/leg-suites.txt"). That step must pass --timings and publish the TSV (print it or upload it as an artifact) before any CI run can produce the column. This lane cannot change that file, and the issue rules out an always-on change to the runner ('No behavior change without the flag'), so the workflow edit is the only route. A person who can edit workflows should either add that step change in this issue's PR, or split it off: steps 1 and 2 (the --timings option and the gate-script timer) plus the doc skeleton ship here, and the Linux column and its cited run follow once the workflow passes the flag.

    What the run says it did not do:

    • Fix step 1 (--timings option on scripts/run-plugin-tests.sh) and step 2 (gate-script timer) were not started, because the run stopped on the protected-path finding before any edit.
    • Fix step 3 (docs/test-wall-clock-baseline.md with a Linux CI column and the cited run, plus the link from scripts/affected-tests.sh) was not started. Its Linux column needs a CI run with --timings, which needs the protected workflow edit.
    • Tidy, review, verify and checks steps were not run, because the change is empty.

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40832357320

    To hand it back to the lane, remove host-run and add agent-ready again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.priority: mediumReal value, no hard deadline; normal backlog flow.status: readyTriaged, unblocked, and fully specified; eligible to pick up.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    Task

    Fields

    No fields configured for Task.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions