Skip to content

4.x: junitReporter still doesn't record helper _beforeSuite()/_afterSuite() failures (follow-up to #5645) #5660

Description

@mirao

Follow-up to #5645 and the fix in 5b2517c.

That fix listens on event.hook.failed, which is only emitted by fireHook() in lib/mocha/asyncWrapper.js, and fireHook() is only called from the injected() wrapper used for the test-file-defined BeforeSuite() / AfterSuite() / Before() / After() functions (registered in lib/mocha/ui.js).

A custom helper's _beforeSuite() / _afterSuite() lifecycle methods run through a different mocha hook — suite.beforeAll('codeceptjs.beforeSuite', suiteSetup(suite)) (lib/mocha/ui.js:106), which calls runAsyncHelpersHook('_beforeSuite', ...) (lib/listener/helpers.js:32). Looking at suiteSetup() / suiteTeardown() in asyncWrapper.js, on failure they call doneFn(err) directly and never call fireHook() — so event.hook.failed is never emitted for helper lifecycle failures, and junitReporter still produces an empty/incomplete report for these.

Repro

// steps_file.ts / a custom helper
import Helper from "@codeceptjs/helper";

class MyHelper extends Helper {
  async _beforeSuite() {
    throw new Error("Simulated error in helper _beforeSuite");
  }
}
export = MyHelper;
// My_test.ts
Feature("My");

Scenario("test something", ({ I }) => {
  I.amOnPage("https://example.com");
});
// codecept.conf.ts
export const config: CodeceptJS.MainConfig = {
  tests: "./*_test.ts",
  output: "./output",
  helpers: {
    Playwright: { browser: "chromium", url: "http://localhost", show: false },
    MyHelper: { require: "./my_helper.ts" },
  },
  noGlobals: true,
  plugins: { junitReporter: { enabled: true } },
  name: "my",
  require: ["tsx/cjs"],
};

Running codeceptjs run --verbose fails the suite ("before all" hook: codeceptjs.beforeSuite for "..."), but output/report.xml comes out as:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="CodeceptJS" tests="0" failures="0" skipped="0" errors="0" time="0.000" .../>

Same empty-report symptom as #5645, just triggered from a helper's lifecycle hook instead of the test file's BeforeSuite().

Suggested fix

We worked around it by listening on Mocha's own runner.on('fail', ...) event instead of (or in addition to) event.hook.failed. runner.on('fail') fires for every suite-level hook failure regardless of origin (test-file hooks and helper lifecycle hooks alike), so a single title-based filter (/^"(before all|after all)" hook:/) covers both cases without duplicating the entries already covered by event.hook.failed. Happy to share the patch we're running if useful for a PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions