Skip to content

Make the multi-recipient inbox benchmark test deterministic #1218

Description

@dahlia

The inboxRunner - rotates deliveries across multiple recipients test in packages/cli/src/bench/scenarios/inbox.test.ts failed in the Node.js CI job for #1215:

expected bob's inbox to be hit; hits: ["/users/alice/inbox"]

In the failed job, the test passed its success-rate assertion but recorded no delivery to Bob. A retry passed on the same commit (b879d6fe84a23a53231d5ffdff9acfae8de7dd29).

The test uses two recipients, concurrency 2, and a 300ms duration with the real clock. The load generator stops dispatching when that window expires, so scheduling delays may leave too few deliveries to exercise both recipients. This is a suspected cause; it has not been reproduced locally.

Make the test verify recipient rotation without relying on deliveries completing within 300ms. A controlled clock or explicit delivery-count synchronization could preserve coverage and reduce sensitivity to CI scheduling delays.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

component/ciCI/CD workflows and GitHub Actionscomponent/cliCLI tools relatedruntime/nodeNode.js runtime related

Type

Fields

Priority

None yet

Effort

None yet

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions