Follow-up to #5645 and the fix in 5b2517c.
That fix listens on event.hook.failed, which is only emitted by fireHook() in lib/mocha/asyncWrapper.js, and fireHook() is only called from the injected() wrapper used for the test-file-defined BeforeSuite() / AfterSuite() / Before() / After() functions (registered in lib/mocha/ui.js).
A custom helper's _beforeSuite() / _afterSuite() lifecycle methods run through a different mocha hook — suite.beforeAll('codeceptjs.beforeSuite', suiteSetup(suite)) (lib/mocha/ui.js:106), which calls runAsyncHelpersHook('_beforeSuite', ...) (lib/listener/helpers.js:32). Looking at suiteSetup() / suiteTeardown() in asyncWrapper.js, on failure they call doneFn(err) directly and never call fireHook() — so event.hook.failed is never emitted for helper lifecycle failures, and junitReporter still produces an empty/incomplete report for these.
Repro
// steps_file.ts / a custom helper
import Helper from "@codeceptjs/helper";
class MyHelper extends Helper {
async _beforeSuite() {
throw new Error("Simulated error in helper _beforeSuite");
}
}
export = MyHelper;
// My_test.ts
Feature("My");
Scenario("test something", ({ I }) => {
I.amOnPage("https://example.com");
});
// codecept.conf.ts
export const config: CodeceptJS.MainConfig = {
tests: "./*_test.ts",
output: "./output",
helpers: {
Playwright: { browser: "chromium", url: "http://localhost", show: false },
MyHelper: { require: "./my_helper.ts" },
},
noGlobals: true,
plugins: { junitReporter: { enabled: true } },
name: "my",
require: ["tsx/cjs"],
};
Running codeceptjs run --verbose fails the suite ("before all" hook: codeceptjs.beforeSuite for "..."), but output/report.xml comes out as:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="CodeceptJS" tests="0" failures="0" skipped="0" errors="0" time="0.000" .../>
Same empty-report symptom as #5645, just triggered from a helper's lifecycle hook instead of the test file's BeforeSuite().
Suggested fix
We worked around it by listening on Mocha's own runner.on('fail', ...) event instead of (or in addition to) event.hook.failed. runner.on('fail') fires for every suite-level hook failure regardless of origin (test-file hooks and helper lifecycle hooks alike), so a single title-based filter (/^"(before all|after all)" hook:/) covers both cases without duplicating the entries already covered by event.hook.failed. Happy to share the patch we're running if useful for a PR.
Follow-up to #5645 and the fix in 5b2517c.
That fix listens on
event.hook.failed, which is only emitted byfireHook()inlib/mocha/asyncWrapper.js, andfireHook()is only called from theinjected()wrapper used for the test-file-definedBeforeSuite()/AfterSuite()/Before()/After()functions (registered inlib/mocha/ui.js).A custom helper's
_beforeSuite()/_afterSuite()lifecycle methods run through a different mocha hook —suite.beforeAll('codeceptjs.beforeSuite', suiteSetup(suite))(lib/mocha/ui.js:106), which callsrunAsyncHelpersHook('_beforeSuite', ...)(lib/listener/helpers.js:32). Looking atsuiteSetup()/suiteTeardown()inasyncWrapper.js, on failure they calldoneFn(err)directly and never callfireHook()— soevent.hook.failedis never emitted for helper lifecycle failures, andjunitReporterstill produces an empty/incomplete report for these.Repro
Running
codeceptjs run --verbosefails the suite ("before all" hook: codeceptjs.beforeSuite for "..."), butoutput/report.xmlcomes out as:Same empty-report symptom as #5645, just triggered from a helper's lifecycle hook instead of the test file's
BeforeSuite().Suggested fix
We worked around it by listening on Mocha's own
runner.on('fail', ...)event instead of (or in addition to)event.hook.failed.runner.on('fail')fires for every suite-level hook failure regardless of origin (test-file hooks and helper lifecycle hooks alike), so a single title-based filter (/^"(before all|after all)" hook:/) covers both cases without duplicating the entries already covered byevent.hook.failed. Happy to share the patch we're running if useful for a PR.