Skip to content

A background-task notification after the completion line fails a finished Run #1405

Closed

Description

@JacobStephens2

This was generated by AI during triage.

A Session that finishes its work and writes its completion line can still fail its Run if a background task it started earlier sends a notification afterwards. The notification wakes the Session for one more reply, and thirdshift reads its completion line from that reply alone.

Observed

Pickup run pickup-20261026T093001-0400 of #1363 on the thirdshift host, grok Harness, thirdshift 0.16.2:

  • 10:07: the implement Session started node --test … tests/inbox.test.cjs in the background from the repository root. It failed because the path is relative to console/. The Session reran it from console/ and it passed.
  • The Session pushed Stop Command details from re-parsing every historical Session trace #1377 (CI green) and ended its summary with Bug reproduced: fixed.
  • About 30 minutes later the background task's failure notification arrived. The Session replied: "That background failure is the first Console timeout run, started from the repository root… Pull request Stop Command details from re-parsing every historical Session trace #1377 is unchanged." That reply became the Session's single result.
  • The Run stopped at Implementation after 1h 10m with bug diagnosis ended without a reproduced or not reproduced completion line, so a finished, green PR needed a Recovery run.

Agent Brief

Category: bug
Summary: A late background-task notification replaces a Session's final message and loses its completion line

Current behavior:
not_reproduced in src/delivery.rs reads only the last line of the Harness interpretation's final_message. When a Session writes its completion line and is then woken by a background-task notification, final_message is the reply to that notification, and the completion line from the reply before it is lost. Other readers of completion lines from final_message (Architecture review, Security audit, close check, Security fix lines in src/prompt.rs) are exposed to the same thing.

Desired behavior:

  • A completion line the Session wrote is still found when a later reply in the same Session, written after a background notification, has no completion line.
  • A later reply that does carry a completion line wins over an earlier one, so a Session can still correct itself.
  • A Session that never wrote a completion line still fails the same way, with the same message.
  • The fix applies to every Harness that can deliver background notifications (at least grok and Claude), and to every completion-line reader, not only bug diagnosis. Put it where the Harness interpretation decides the final message, not in each caller.

Key interfaces:

  • src/harness/interpretation.rs: final_message and require_final_message. The interpretation knows the assistant messages in order and is the natural place to keep the last message that holds a completion line.
  • src/delivery.rs not_reproduced, and the other *_LINE readers listed above.
  • Harness stream fixtures in src/harness/interpretation_tests.rs.

Acceptance criteria:

  • A grok stream fixture (and a Claude one) with a reply ending Bug reproduced: fixed, then a background-notification turn whose reply has no completion line, makes bug diagnosis pass as reproduced.
  • The same fixture with Bug not reproduced: <comment URL> in the first reply reads as not reproduced with that URL.
  • A fixture where the later reply carries a different completion line uses the later one.
  • A fixture with no completion line in any reply still fails with bug diagnosis ended without a reproduced or not reproduced completion line.
  • At least one other completion-line reader (for example the Security audit lines) is covered by the same rule.

Out of scope:

Activity

  1. added
    bugSomething isn't working
    ready-for-agentFully specified, ready for an AFK agent
    in-progressA Claim: thirdshift has taken this issue
    and removed
    ready-for-agentFully specified, ready for an AFK agent
    on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions