[serge-ivo]

Closes #108.

What changed

• --ci used flag ?? (ci ? 60 : config ?? 0), so .vcqa.json / package.json#vcqa failUnder was ignored in CI.
• New pure resolveFailUnder (src/fail-under.ts): --fail-under → config failUnder → --ci default 60 → 0. A config failUnder: 0 counts as an explicit choice (no gate).
• In --ci (non-JSON/markdown) the output now names the threshold and its source: Passing: score 82 ≥ 80 (config) / Failing: score 70 < 80 (config) / (flag) / (ci default). I found no consumer in action, github-app, app or mcp that parses the Failing: line.
• --help text for --ci / --fail-under updated, and so are the README flag rows.
• vcqa init: the scaffolded workflow runs --ci without --fail-under, and .vcqa.json carries the gate (failUnder: 70), so editing the config actually changes CI.

Verification

• pnpm build: clean. pnpm lint: 0 errors (62 warnings, same as the baseline on main).
• pnpm test: 65 files, 748/748 passing (main baseline 739, plus 9 new).
• src/fail-under.test.ts covers the precedence table. In src/cli.test.ts, new end-to-end tests run the built CLI:
  • config failUnder: 100 under --ci exits 1, and the test asserts the score was above 60, so the old code would have passed it.
  • config failUnder: 1 under --ci exits 0.
  • --fail-under 1 overrides config 100.
  • With no config, the 60 default applies.

The acceptance criteria are written as config 80 / score 70 and config 40 / score 50. The fixture's score is not pinned to those numbers, so the tests check the same thresholds-around-60 property with 100 and 1.

Follow-up

A docs PR on vibecodeqa/vibecodeqa updates docs/docs/ci.md:15, configuration.md:27 and scoring.md:42, which describe the old behaviour.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SmtSK7qDTyXhimJ9grap88

[vibecode-qa]

VibeCode QA

PR #109 (fix/108-ci-fail-under → main)

No workflow found. Add this to enable automatic PR scanning:

name: VibeCode QA
on:
  pull_request:
  workflow_dispatch:
permissions:
  contents: read
  pull-requests: write
  security-events: write
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: vibecodeqa/action@v1
        with:
          fail-under: "70"

Setup guide | Dashboard

[serge-ivo]

Adversarial review, round 1 (COMMENT only)

What I ran. I checked out head b4503a7 in a throwaway worktree and ran pnpm build, pnpm lint and pnpm test: build clean, lint 0 errors (62 warnings), 748/748 tests pass. I grepped every caller of the threshold, read src/commands/init.ts and vibecodeqa/action action.yml, and test-merged this branch with #110 and #111.

The core logic in resolveFailUnder is correct, and the precedence matches #108. --fail-under 0 and config failUnder: 0 both turn the gate off, and the output says so.

Findings

1. minor, CONFIRMED: src/cli.test.ts:275. The "config below 60" end-to-end test passes on main's behaviour too. The test above it asserts the fixture scores above 60. On main the --ci gate is 60, so this run also exits 0 there. The only thing that fails on main is the (config) text in the output. So "config failUnder: 40, score 50, exit 0" from the acceptance criteria is covered only by the unit test.
   Fix: add a fixture that scores below 60 (for example, seed it with lint or complexity findings) and assert that it exits 0 with config failUnder set under its score.

2. minor, CONFIRMED: src/commands/init.ts:35 and :88. The scaffold contradicts the new precedence. vcqa init writes a workflow that runs --ci --fail-under 70 and a .vcqa.json with failUnder: 60. Because the flag now always wins, the config value is dead in every scaffolded repo. Editing .vcqa.json changes nothing in CI, and the two files disagree (70 vs 60).
   Fix: drop --fail-under 70 from the workflow template and put the intended number in .vcqa.json. That makes the config the single source.

3. nit, CONFIRMED: README.md:257-258 still says --ci means "exit 1 if score < 60".
   Fix: use the same wording as the new --help text.

4. nit, CONFIRMED: this PR and #110 conflict in src/cli.ts (the printHelp flag block). git merge-tree pr109 pr110 reports a content conflict. Whichever merges second needs a rebase.

5. nit, pre-existing: src/config.ts:61 does not range-check config failUnder. "failUnder": "80" (a string) is silently dropped, and under --ci it falls back to 60. The new (ci default) source label makes this visible, which helps. 150 or -5 is accepted as is.
   Fix (optional): warn on a non-number or out-of-range value.

6. follow-up, outside this repo, CONFIRMED from vibecodeqa/action action.yml:76-79 and :104-107. The composite action runs --json without --ci. It turns the CLI's exit code into a failure only when its own fail-under input is non-zero, and that input defaults to 0. A config failUnder therefore still does not gate CI for anyone using the action. It belongs in an action issue, not this PR.

Verdict: no blockers. Items 1 and 2 are worth fixing before or soon after merge.

[vibecode-qa]

VibeCode QA

PR #109 (fix/108-ci-fail-under → main)

No workflow found. Add this to enable automatic PR scanning:

name: VibeCode QA
on:
  pull_request:
  workflow_dispatch:
permissions:
  contents: read
  pull-requests: write
  security-events: write
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: vibecodeqa/action@v1
        with:
          fail-under: "70"

Setup guide | Dashboard

[serge-ivo]

Round-1 review (5411684330) addressed in new commits; no force-push. Head is now 3016218.

1. Test didn't distinguish from main. Fixed in 5bd4219. The low-threshold test now scans a fixture that scores below 60 (52 locally) and asserts score < 60 with exit 0 under --ci and config failUnder: 1. Main's CI default of 60 would exit 1 on it.
2. init scaffold contradicted the new precedence. Fixed in 3016218. The workflow runs npx @vibecodeqa/cli --ci --sarif --badge, and .vcqa.json has failUnder: 70 (the gate the workflow enforced), with the help text updated. The init tests assert the workflow has no --fail-under and the config has 70.
3. README. Fixed in 3016218: the --ci / --fail-under rows match --help.
4. Conflict with fix(testing): treat a test-run timeout as not run, kill its process tree, make the limit configurable (#106) #110 in the printHelp block. Noted. Whichever merges second will merge main in and resolve it; no rebase on a reviewed branch.
5. Config failUnder range check. Not done here. It is pre-existing, and config.ts is under change in fix(testing): treat a test-run timeout as not run, kill its process tree, make the limit configurable (#106) #110. It's a candidate for a separate issue.
6. vibecodeqa/action doesn't pass --ci. Agreed; this needs a new issue in that repo. I'll raise it with the owner.

• PR body: the impossible 72 ≥ 80 example is now 82 ≥ 80.

Caveat: the init test assertion landed in 5bd4219, a commit ahead of the init.ts change, so one init test fails at that commit alone. It is green at 3016218. I left it rather than rewrite a branch under review.

Verified at 3016218: pnpm build is clean; pnpm lint has 0 errors (62 warnings, same as main); pnpm test passes 65 files, 748/748.

[serge-ivo]

Adversarial review, round 2 (final, COMMENT only)

What I ran, at head 3016218. I used a fresh worktree. Build is clean and lint shows 0 errors (62 warnings). pnpm test passes 65 files, 748/748.

I copied this branch's cli.test.ts into a main checkout and ran it against main's build. All 4 new --ci honours config failUnder tests fail there, as do the 2 updated init tests, so they test the change.

I then ran the built CLI over a full precedence matrix on a fixture that scores 69 (--skip-tests). That is 10 configs × 9 flag sets = 90 runs.

• Configs: none; .vcqa.json at 50 / 80; package.json#vcqa at 50 / 80; both files; malformed JSON; "80" as a string; -5; 0.
• Flag sets: none; --ci; --fail-under 50|80; --ci --fail-under 50|80|0|abc; --json --ci.

Round-1 findings

#	Status	Evidence
1 low-threshold test passed on main	FIXED	The new test exits 1 on main's build, because the fixture scores below 60.
2 init scaffold contradicted precedence	FIXED	vcqa init on a fresh dir writes run: npx @vibecodeqa/cli --ci --sarif --badge (parses as valid YAML) and .vcqa.json with failUnder: 70. Repos scaffolded earlier keep --fail-under 70 in their workflow, and the flag still wins, so their gate does not move.
3 README	FIXED	—
4 conflict with #110	still open	git merge-tree pr109 pr110 now conflicts in README.md as well as src/cli.ts.
5 config range check	NOT FIXED (deferred, pre-existing)	"failUnder": "80" falls back to Passing: score 69 ≥ 60 (ci default). The source label at least makes this visible.
6 vibecodeqa/action	out of scope	—

Precedence matrix: correct in all 90 runs. The order is flag, then config, then --ci 60, then none.

• --fail-under 0 turns the gate off.
• .vcqa.json and package.json#vcqa behave the same.
• --json --ci prints no gate line but keeps the exit code.

New findings (all nits)

1. nit, CONFIRMED: src/cli.ts:709. A config failUnder of 0 or below now turns off the --ci gate, and the message misstates a negative value.

   • On main, --ci with config failUnder: 0 still gated at 60. Here it prints No score gate: failUnder is 0 (config) and exits 0. That is consistent with the new precedence, but nothing documents that 0 means off.
   • Config -5 prints the same line, failUnder is 0, although the value is -5.

   Fix: print gate.threshold as given, and document 0 as "no gate" (vibecodeqa/vibecodeqa#57).

2. nit, CONFIRMED, pre-existing: src/cli.ts:52-63. A typo in --fail-under silently falls back. --ci --fail-under abc with config 50 gives Passing: score 69 ≥ 50 (config). #110 makes --test-timeout abc exit 2. Doing the same here would make the two flags consistent. The (config) label does at least show what happened.

3. nit, CONFIRMED, pre-existing: src/config.ts:20. A .vcqa.json without failUnder hides package.json#vcqa.failUnder. Matrix row "both": .vcqa.json = {"ignore":[]} and package.json#vcqa.failUnder = 80 give Passing: score 69 ≥ 60 (ci default). The loader is all-or-nothing. That is not new, but the docs now say ".vcqa.json / package.json#vcqa" as if the two were merged.

Verdict: READY TO MERGE. Resolve the #110 conflict (src/cli.ts help block and README.md flags table) and keep both sides' rows.

[serge-ivo]

Merged main (cec32ca) in; conflicts resolved in src/cli.ts (imports only: kept both resolveFailUnder and #113's isPartialScan). Head is now 2c63c30, a merge commit; no rebase or force-push. README.md, init.ts and cli.test.ts auto-merged.

Verified at 2c63c30: pnpm build is clean; pnpm lint has 0 errors (62 warnings); pnpm test passes 67 files, 826/826.

[vibecode-qa]

VibeCode QA

PR #109 (fix/108-ci-fail-under → main)

No workflow found. Add this to enable automatic PR scanning:

name: VibeCode QA
on:
  pull_request:
  workflow_dispatch:
permissions:
  contents: read
  pull-requests: write
  security-events: write
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: vibecodeqa/action@v1
        with:
          fail-under: "70"

Setup guide | Dashboard