From 96f1d8e813f47401ca0f8bd92cb8201fb5d27fb2 Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 1 Sep 2026 20:02:00 +0300 Subject: [PATCH 01/33] docs: spec + implementation plan for Ukrainian language support Co-Authored-By: Claude Fable 5 --- .../2026-09-01-ukrainian-language-support.md | 253 ++++++++++++++++++ ...09-01-ukrainian-language-support-design.md | 91 +++++++ 2 files changed, 344 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-01-ukrainian-language-support.md create mode 100644 docs/superpowers/specs/2026-09-01-ukrainian-language-support-design.md diff --git a/docs/superpowers/plans/2026-09-01-ukrainian-language-support.md b/docs/superpowers/plans/2026-09-01-ukrainian-language-support.md new file mode 100644 index 0000000..f7ba120 --- /dev/null +++ b/docs/superpowers/plans/2026-09-01-ukrainian-language-support.md @@ -0,0 +1,253 @@ +# Ukrainian Language Support Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add Ukrainian as a first-class language for transcription (Deepgram), answer generation (LLM prompt), and the knowledge base, selectable in the existing UI. + +**Architecture:** The app already has an end-to-end language flow: `settings.sttLanguage` feeds the `DeepgramLive` WebSocket client (`src/renderer/deepgram.js`), and `settings.answerLanguage` feeds the prompt builder (`src/main/prompt.js`). We add `uk` to that flow: a new option in three `` elements: `langSelect` ~line 54, `setSttLang` ~line 306, `setAnswerLang` ~line 322) + +**Interfaces:** +- Consumes: Task 1's `uk` branch in `buildPrompt` and Task 3's Nova-3 routing — the option values wire straight into existing `settings.sttLanguage` / `settings.answerLanguage` plumbing in `app.js` (no JS change). +- Produces: user-visible `Українська` choices. + +- [ ] **Step 1: Add the transcript-language options** + +In both the toolbar `langSelect` and the Settings `setSttLang` selects, insert after the `` line: + +```html + +``` + +(Match each select's existing indentation — the toolbar and Settings blocks differ.) + +- [ ] **Step 2: Add the answer-language option** + +In the `setAnswerLang` select, insert after ``: + +```html + +``` + +- [ ] **Step 3: Verify formatting and full test suite** + +Run: `npm run format:check && npm run lint && npm test` +Expected: all pass. If `format:check` complains about the edited lines, run `npm run format` and re-check. + +- [ ] **Step 4: Commit** + +```bash +git add src/renderer/index.html +git commit -m "feat: Ukrainian option in transcript- and answer-language selects" +``` + +--- + +### Task 5: Changelog + end-to-end manual verification + +**Files:** +- Modify: `CHANGELOG.md` (top `## [Unreleased]` → first `### Added` list) + +**Interfaces:** +- Consumes: everything from Tasks 1–4. +- Produces: release note; verified feature. + +- [ ] **Step 1: Add the changelog entry** + +In `CHANGELOG.md`, add as the first bullet under the first `### Added` in `## [Unreleased]`: + +```markdown +- **Ukrainian language support**: `Українська` is now selectable as the transcript language (Deepgram `nova-3`, `language=uk`) and as the answer language (answers generated in Ukrainian). Ukrainian documents in the Knowledge Base were already supported and are now covered by tests. +``` + +- [ ] **Step 2: Manual end-to-end check** + +Run: `npm start` (or `make run`), then: +1. Toolbar → Transcript language → `Українська`; Start Listening; speak a Ukrainian sentence with an English term (e.g. «Розкажіть про ваш досвід з Kubernetes») and confirm it appears in the live transcript. +2. Settings → Answer language → `Українська`; press the answer hotkey (default `Ctrl+A`) and confirm the generated answer is in Ukrainian. +3. Upload a Ukrainian-language text/PDF into the Knowledge Base and confirm a generated answer can draw on it. + +Expected: all three work. Requires a Deepgram key + one answer provider configured; if a live check isn't possible in your environment, report that explicitly instead of claiming verification. + +- [ ] **Step 3: Run the full suite one last time** + +Run: `npm test && npm run lint && npm run format:check` +Expected: all pass. + +- [ ] **Step 4: Commit** + +```bash +git add CHANGELOG.md +git commit -m "docs: changelog entry for Ukrainian language support" +``` diff --git a/docs/superpowers/specs/2026-09-01-ukrainian-language-support-design.md b/docs/superpowers/specs/2026-09-01-ukrainian-language-support-design.md new file mode 100644 index 0000000..cba12df --- /dev/null +++ b/docs/superpowers/specs/2026-09-01-ukrainian-language-support-design.md @@ -0,0 +1,91 @@ +# Ukrainian Language Support — Design + +**Date:** 2026-09-01 +**Status:** Approved (brainstormed in-session) + +## Goal + +Let a user run an interview session in Ukrainian end-to-end: live transcription +of Ukrainian speech (STT), Ukrainian documents in the knowledge base (the +"RAG" layer), and answers generated in Ukrainian (LLM). + +Target scenario (confirmed with the user): **mostly-Ukrainian speech with +English technical terms sprinkled in** — the typical Ukrainian tech interview. +Heavy whole-sentence switching between Ukrainian and English is out of scope +(Deepgram's `multi` auto-detect mode does not include Ukrainian). + +## Current state + +- **STT** (`src/renderer/deepgram.js`): a `DeepgramLive` class connects to + Deepgram's streaming API. Model selection is + `model: multi ? 'nova-3' : 'nova-2'`; the language code is passed through + as the `language` query param. The language is chosen in two ` @@ -314,6 +315,7 @@

Settings

+ @@ -323,6 +325,7 @@

Settings

+ From 4ca8c8cf3600d82fb597f98879c01ef5657b32f2 Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 1 Sep 2026 20:12:45 +0300 Subject: [PATCH 06/33] docs: changelog entry for Ukrainian language support Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 875710c..5c2ebe3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ All notable changes to this project are documented here. Format loosely follows ## [Unreleased] ### Added +- **Ukrainian language support**: `Українська` is now selectable as the transcript language (Deepgram `nova-3`, `language=uk`) and as the answer language (answers generated in Ukrainian). Ukrainian documents in the Knowledge Base were already supported and are now covered by tests. - **Windows & Linux support for running**: npm scripts are now cross-platform (`cross-env`), so `npm start` works on Windows (previously the `VAR=` prefix broke cmd). `make run` adapts to the OS — macOS builds & opens the signed app; Windows/Linux launch via `npm start`. Prebuilt `.exe` (Windows) and `.AppImage` (Linux) ship in Releases. ### Fixed From 4d724c3d1ef745c7da3506ffe129d7c60e79d117 Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 1 Sep 2026 20:14:40 +0300 Subject: [PATCH 07/33] style: prettier formatting for Cyrillic chunking test Co-Authored-By: Claude Fable 5 --- test/documents.test.js | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/test/documents.test.js b/test/documents.test.js index cdaa605..90f73dc 100644 --- a/test/documents.test.js +++ b/test/documents.test.js @@ -29,8 +29,9 @@ test('chunkText: an oversized single paragraph is hard-split', () => { }); test('chunkText: Cyrillic (Ukrainian) text chunks losslessly', () => { - const para = - 'Я працював над масштабуванням інференсу великих мовних моделей у Києві. '.repeat(20); + const para = 'Я працював над масштабуванням інференсу великих мовних моделей у Києві. '.repeat( + 20, + ); const text = Array(5).fill(para.trim()).join('\n\n'); const chunks = chunkText(text, 900, 150); assert.ok(chunks.length > 1); From 1ce636aa3af3da2b201e8c578c802953ebb99991 Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 1 Sep 2026 22:23:53 +0300 Subject: [PATCH 08/33] docs: design and implementation plan for academic answer style Co-Authored-By: Claude Opus 5 (1M context) --- .../plans/2026-09-01-academic-answer-style.md | 607 ++++++++++++++++++ ...2026-09-01-academic-answer-style-design.md | 177 +++++ 2 files changed, 784 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-01-academic-answer-style.md create mode 100644 docs/superpowers/specs/2026-09-01-academic-answer-style-design.md diff --git a/docs/superpowers/plans/2026-09-01-academic-answer-style.md b/docs/superpowers/plans/2026-09-01-academic-answer-style.md new file mode 100644 index 0000000..fc9e58e --- /dev/null +++ b/docs/superpowers/plans/2026-09-01-academic-answer-style.md @@ -0,0 +1,607 @@ +# Academic Answer Style Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make generated answers read as spoken academic prose for a dissertation defense — grounded in the user's own dissertation materials, and never formatted as bullet lists. + +**Architecture:** All answer shaping in this app lives in one dependency-free pure module, `src/main/prompt.js`, which `src/main/main.js` calls to build the system instruction and user text for every generation. We rewrite that module's answering doctrine (English instructions, defense persona, grounding rules ported from the sibling `dissertation/assistant` project's `answer_v1.md`), retune the question-extraction prompt for committee questions, and raise the answer-length budget that previously forced outline-form answers — including a one-time migration of the persisted setting. + +**Tech Stack:** Electron (plain JS, CommonJS), `node:test` for unit tests, ESLint + Prettier. + +**Spec:** `docs/superpowers/specs/2026-09-01-academic-answer-style-design.md` + +## Global Constraints + +- Node ≥ 18; tests run with `npm test` (`node --test`), lint with `npm run lint`, format check with `npm run format:check`. +- No new dependencies. +- `src/main/prompt.js` must stay a pure module with **no Electron imports** — that is what makes it unit-testable. +- Prompt instructions are written in **English**. The only exception is the per-language answer rule, which stays in its target language: `请用中文作答。` / `Answer in English.` / `Відповідай українською мовою.` +- The answer is rendered with `textContent` in the renderer, so the prompt must keep forbidding Markdown. +- New answer-length default is exactly `1200`; the old default being migrated from is exactly `500`. +- Settings schema version constant is `SETTINGS_VERSION = 2`; the field name is `schemaVersion`. +- Commit messages follow the repo's conventional style (`feat:`, `fix:`, `test:`, `docs:`, `style:`). +- Out of scope, do not implement: Interview/Defense mode switching, renaming the Job Description setting, evidence state / source excerpts in the UI, a "no question detected" extraction outcome. + +--- + +### Task 1: Rewrite the answer prompt for grounded academic prose + +**Files:** +- Modify: `src/main/prompt.js:5-77` (the whole `buildPrompt` function) +- Test: `test/prompt.test.js` (rewrite the six `buildPrompt` cases; leave the `extraction prompt helpers` case untouched — Task 2 owns it) + +**Interfaces:** +- Consumes: nothing from earlier tasks. +- Produces: `buildPrompt({ question, transcript, context, answerLanguage, maxChars, profile, jobDescription }) -> { systemInstruction: string, userText: string }`. The signature is **unchanged** — `src/main/main.js:315-323` keeps calling it exactly as it does today. Task 3 relies on `maxChars` still being interpolated into `systemInstruction` as a hard ceiling. + +- [ ] **Step 1: Write the failing tests** + +Replace the contents of `test/prompt.test.js` from line 1 down to (but not including) the `test('extraction prompt helpers', ...)` case with: + +```js +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert'); +const { buildPrompt, EXTRACTION_SYSTEM, buildExtractionUser } = require('../src/main/prompt'); + +const BASE = { + question: 'q', + transcript: '', + context: '', + answerLanguage: 'en', + maxChars: 1200, +}; + +test('buildPrompt: asks for flowing prose and forbids lists', () => { + const { systemInstruction } = buildPrompt(BASE); + assert.match(systemInstruction, /FLOWING PROSE/); + assert.match(systemInstruction, /3-8 sentences/); + assert.match(systemInstruction, /NEVER use bullet points/); + // The old outline instruction must be gone. + assert.doesNotMatch(systemInstruction, /大纲式/); + assert.doesNotMatch(systemInstruction, /要点/); +}); + +test('buildPrompt: maxChars is stated as a hard upper bound, not a target', () => { + const { systemInstruction } = buildPrompt({ ...BASE, maxChars: 900 }); + assert.match(systemInstruction, /900 characters is a hard upper bound/); +}); + +test('buildPrompt: casts the model as a defending PhD candidate', () => { + const { systemInstruction } = buildPrompt(BASE); + assert.match(systemInstruction, /PhD candidate defending your own dissertation/); + assert.match(systemInstruction, /examination committee/); +}); + +test('buildPrompt: carries the grounding rules from the dissertation spec', () => { + const { systemInstruction } = buildPrompt(BASE); + // Corpus primacy + verbatim figures. + assert.match(systemInstruction, /primary and authoritative source/); + assert.match(systemInstruction, /EXACTLY as they appear/); + // General knowledge must be labelled. + assert.match(systemInstruction, /not a result of this dissertation/); + // No fabrication. + assert.match(systemInstruction, /NEVER invent dissertation-specific facts/); + // Contradictions are reported, not silently resolved. + assert.match(systemInstruction, /contradict each other/); + // No relevant evidence → cautious, explicitly unconfirmed. + assert.match(systemInstruction, /not confirmed by the dissertation materials/); + // Plain text only (the renderer does not render Markdown). + assert.match(systemInstruction, /No Markdown/); +}); + +test('buildPrompt: answer-language rules stay in their target language', () => { + assert.match( + buildPrompt({ ...BASE, answerLanguage: 'en' }).systemInstruction, + /Answer in English\./, + ); + assert.match(buildPrompt({ ...BASE, answerLanguage: 'zh' }).systemInstruction, /请用中文作答。/); + assert.match( + buildPrompt({ ...BASE, answerLanguage: 'uk' }).systemInstruction, + /Відповідай українською мовою\./, + ); + assert.match( + buildPrompt({ ...BASE, answerLanguage: 'auto' }).systemInstruction, + /same language the question was asked in/, + ); +}); + +test('buildPrompt: explicit question mode includes the question and the transcript', () => { + const { userText } = buildPrompt({ + ...BASE, + question: 'Why this optimisation algorithm?', + transcript: 'Committee: tell us about the method\nCandidate: certainly', + }); + assert.match(userText, /Why this optimisation algorithm\?/); + assert.match(userText, /RECENT SESSION TRANSCRIPT/); + assert.match(userText, /QUESTION TO ANSWER/); +}); + +test('buildPrompt: extract mode (no question) answers the current question only', () => { + const { userText } = buildPrompt({ + ...BASE, + question: '', + transcript: 'Committee: how did you validate the results?', + }); + assert.match(userText, /RIGHT NOW/); + assert.match(userText, /first sentence must already be the answer/); + assert.doesNotMatch(userText, /QUESTION TO ANSWER/); // no explicit-question block +}); + +test('buildPrompt: knowledge-base context is included when provided', () => { + const { userText } = buildPrompt({ ...BASE, context: 'Section 3.2: RMSE = 0.041' }); + assert.match(userText, /DISSERTATION MATERIALS/); + assert.match(userText, /RMSE = 0\.041/); +}); + +test('buildPrompt: profile is injected as a highest-priority override', () => { + const { systemInstruction } = buildPrompt({ + ...BASE, + profile: 'The committee chair is a statistician.', + }); + assert.match(systemInstruction, /HIGHEST PRIORITY/); + assert.match(systemInstruction, /The committee chair is a statistician\./); +}); + +test('buildPrompt: job description is still injected as tailoring context', () => { + const { systemInstruction } = buildPrompt({ + ...BASE, + jobDescription: 'Defense of a thesis on adaptive control.', + }); + assert.match(systemInstruction, /ADDITIONAL TARGET CONTEXT/); + assert.match(systemInstruction, /adaptive control/); +}); +``` + +Keep the existing `test('extraction prompt helpers', ...)` case at the end of the file exactly as it is. + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: `npm test -- test/prompt.test.js` +Expected: FAIL — several `AssertionError [ERR_ASSERTION]: The input did not match the regular expression` failures, e.g. `/FLOWING PROSE/` and `/PhD candidate defending your own dissertation/`, because the current prompt is the Chinese interview prompt. + +- [ ] **Step 3: Write the implementation** + +Replace `src/main/prompt.js` lines 3–77 (the leading comment and the whole `buildPrompt` function) with: + +```js +// Pure functions that build the answering / question-extraction prompts. +// No Electron dependency, so this module is directly unit-testable. + +function buildPrompt({ + question, + transcript, + context, + answerLanguage, + maxChars, + profile, + jobDescription, +}) { + // The per-language rule is deliberately written in its own target + // language — it primes the model's output language far better than an + // English description of it would. + const langRule = + answerLanguage === 'zh' + ? '请用中文作答。' + : answerLanguage === 'en' + ? 'Answer in English.' + : answerLanguage === 'uk' + ? 'Відповідай українською мовою.' + : 'Answer in the same language the question was asked in.'; + + const lines = [ + 'You are the PhD candidate defending your own dissertation before an examination committee.', + "You will be given a committee member's question, a recent transcript of the session, and excerpts from your own dissertation materials.", + 'Answer in the first person, as if speaking aloud to the committee, in an academic but natural spoken register.', + 'Rules:', + `1) Write FLOWING PROSE — continuous sentences that read as speech. NEVER use bullet points, dashes as list markers, numbered lists, headings or any outline structure. Aim for about 3-8 sentences; ${maxChars} characters is a hard upper bound, not a target.`, + '2) Open with the direct answer to the question. No preamble, no "good question", no restating or summarising the question, no title such as "My answer".', + '3) The dissertation excerpts are the primary and authoritative source of facts about this work. Reproduce numerical values, formulas, algorithm names, constraints, experimental conditions and stated conclusions EXACTLY as they appear in the excerpts — never paraphrase, round or approximate them.', + '4) If the excerpts do not cover part of the question, you may add general knowledge from the subject area, but say explicitly that it is general background and not a result of this dissertation.', + '5) NEVER invent dissertation-specific facts, numbers, method names, citations or references that are not present in the excerpts.', + '6) If the excerpts contradict each other (different values for the same fact), state that discrepancy openly instead of silently picking one.', + '7) If no excerpt is relevant to the question, give a short, careful answer from general knowledge and state plainly that it is not confirmed by the dissertation materials.', + '8) Plain text only. No Markdown: no asterisks (**), no hash headings, no tables — the interface does not render Markdown and would show the raw characters.', + "9) The transcript is live speech recognition and may contain repetitions, cross-talk, misrecognised words and unfinished sentences. Tolerate that noise and answer only the committee's CURRENT core question.", + '10) Never reveal your reasoning — output only the final answer.', + `11) ${langRule}`, + ]; + if (jobDescription && jobDescription.trim()) { + lines.push( + '', + '================ ADDITIONAL TARGET CONTEXT (tailor the answer to it: align with its requirements, terminology and keywords) ================', + jobDescription.trim().slice(0, 6000), + ); + } + if (profile && profile.trim()) { + lines.push( + '', + '================ SESSION BACKGROUND AND ANSWERING STYLE (HIGHEST PRIORITY — follow it) ================', + profile.trim(), + ); + } + const systemInstruction = lines.join('\n'); + + const parts = []; + if (context) parts.push(`[DISSERTATION MATERIALS / KNOWLEDGE BASE EXCERPTS]\n${context}\n`); + + const q = (question || '').trim(); + if (q) { + if (transcript) { + parts.push( + `[RECENT SESSION TRANSCRIPT (~15 turns, speech recognition — may contain repeats or errors; use it only for context and continuity)]\n${transcript}\n`, + ); + } + parts.push(`[QUESTION TO ANSWER]\n${q}`); + parts.push('Answer it directly, using the material above.'); + parts.push('\nYour answer:'); + } else { + parts.push( + `[RECENT SESSION TRANSCRIPT (~15 turns, speech recognition — may contain repeats, cross-talk or errors)]\n${transcript || '(no dialogue yet)'}\n`, + ); + parts.push( + 'Work out silently which question the committee is asking RIGHT NOW, then answer it directly. Earlier turns are background context only — do not answer an older question that was already addressed.', + ); + parts.push( + 'Do not output any prefix or restatement — no "The core question is…", no "The committee is asking…", no "Your question is…". The first sentence must already be the answer itself.', + ); + parts.push('\nYour answer:'); + } + + return { systemInstruction, userText: parts.join('\n') }; +} +``` + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: `npm test -- test/prompt.test.js` +Expected: PASS — all cases in the file, including the untouched `extraction prompt helpers` case. + +- [ ] **Step 5: Run the full suite and the linters** + +Run: `npm test && npm run lint && npm run format:check` +Expected: all pass. If `format:check` reports `src/main/prompt.js` or `test/prompt.test.js`, run `npm run format` and re-check. + +- [ ] **Step 6: Commit** + +```bash +git add src/main/prompt.js test/prompt.test.js +git commit -m "feat: academic prose answers grounded in dissertation materials" +``` + +--- + +### Task 2: Retune question extraction for committee questions + +**Files:** +- Modify: `src/main/prompt.js:79-81` (the `EXTRACTION_SYSTEM` constant and its comment) +- Test: `test/prompt.test.js` (replace the `extraction prompt helpers` case) + +**Interfaces:** +- Consumes: `EXTRACTION_SYSTEM: string` and `buildExtractionUser(recentTranscript: string) -> string` from `src/main/prompt.js` (both already exported). +- Produces: `EXTRACTION_SYSTEM` with unchanged output contract — one clean question, no prefix, in the speaker's language — so `src/main/main.js:349-357`, which pipes the result straight into the Current Question box, needs no change. `buildExtractionUser` is not modified. + +- [ ] **Step 1: Write the failing test** + +In `test/prompt.test.js`, replace the existing case: + +```js +test('extraction prompt helpers', () => { + assert.match(EXTRACTION_SYSTEM, /ONLY that question/); + assert.match(buildExtractionUser('Interviewer: hi'), /Interviewer: hi/); + assert.match(buildExtractionUser('x'), /current core question is:/); +}); +``` + +with: + +```js +test('extraction prompt targets the committee question and keeps its contract', () => { + // Domain: a defense, not a job interview. + assert.match(EXTRACTION_SYSTEM, /dissertation defense/); + assert.match(EXTRACTION_SYSTEM, /committee member/); + assert.doesNotMatch(EXTRACTION_SYSTEM, /interview/i); + // Spec US3 behaviours. + assert.match(EXTRACTION_SYSTEM, /Drop introductory remarks/); + assert.match(EXTRACTION_SYSTEM, /multi-sentence question/); + assert.match(EXTRACTION_SYSTEM, /follow-up/); + assert.match(EXTRACTION_SYSTEM, /terminology/); + // Unchanged output contract. + assert.match(EXTRACTION_SYSTEM, /ONLY that question/); + assert.match(EXTRACTION_SYSTEM, /SAME language/); +}); + +test('buildExtractionUser embeds the transcript and the trailing cue', () => { + assert.match(buildExtractionUser('Committee: hi'), /Committee: hi/); + assert.match(buildExtractionUser('x'), /current core question is:/); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `npm test -- test/prompt.test.js` +Expected: FAIL — `/dissertation defense/` does not match, and `assert.doesNotMatch(EXTRACTION_SYSTEM, /interview/i)` fails because the current string says "noisy live interview transcripts". + +- [ ] **Step 3: Write the implementation** + +In `src/main/prompt.js`, replace lines 79–81: + +```js +// 问题提取(只看最近几轮) +const EXTRACTION_SYSTEM = + "You clean up noisy live interview transcripts. The transcript may contain repeats, cross-talk, ASR errors and half-sentences. Identify the interviewer's CURRENT core question and rewrite it as ONE clean, complete question. Output ONLY that question — no prefix, no quotes, no explanation. Write it in the SAME language the interviewer is speaking."; +``` + +with: + +```js +// Question extraction (looks only at the most recent turns). +const EXTRACTION_SYSTEM = + "You clean up noisy live transcripts of a dissertation defense. The transcript may contain repeats, cross-talk, speech-recognition errors and half-sentences. Identify the CURRENT core question a committee member is putting to the candidate and rewrite it as ONE clean, complete question. Drop introductory remarks, compliments and asides, but keep every meaningful part of a long multi-sentence question. If the latest turn is a short follow-up to an earlier question, return that follow-up rather than the earlier question. Preserve academic and technical terminology exactly as spoken. Output ONLY that question — no prefix, no quotes, no explanation. Write it in the SAME language the committee member is speaking."; +``` + +- [ ] **Step 4: Run the tests to verify they pass** + +Run: `npm test -- test/prompt.test.js` +Expected: PASS. + +- [ ] **Step 5: Run the full suite and the linters** + +Run: `npm test && npm run lint && npm run format:check` +Expected: all pass. + +- [ ] **Step 6: Commit** + +```bash +git add src/main/prompt.js test/prompt.test.js +git commit -m "feat: extract committee questions instead of interview questions" +``` + +--- + +### Task 3: Raise the answer-length budget and migrate saved settings + +**Files:** +- Modify: `src/main/settings.js` (add `SETTINGS_VERSION`, raise `maxChars`, add `migrate()`, use it in `load()`, export it) +- Modify: `src/renderer/app.js:518`, `:571`, `:592` (replace three hardcoded `500` fallbacks with one constant) +- Create: `test/settings.test.js` + +**Interfaces:** +- Consumes: `buildPrompt`'s `maxChars` parameter from Task 1 — it is now a stated ceiling, so the value can safely grow. +- Produces: `migrate(saved: object) -> object` exported from `src/main/settings.js`, plus `SETTINGS_VERSION = 2` and `DEFAULTS.maxChars = 1200`. `load()` and `save()` keep their existing signatures (`load() -> object`, `save(partial: object) -> object`), so `src/main/main.js` needs no change. + +- [ ] **Step 1: Write the failing test** + +Create `test/settings.test.js`: + +```js +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert'); +const { migrate, DEFAULTS, SETTINGS_VERSION } = require('../src/main/settings'); + +test('DEFAULTS: answer length is the academic-prose budget', () => { + assert.strictEqual(DEFAULTS.maxChars, 1200); + assert.strictEqual(DEFAULTS.schemaVersion, SETTINGS_VERSION); +}); + +test('migrate: a v1 file still on the old 500 default is lifted to the new default', () => { + const out = migrate({ maxChars: 500, provider: 'gemini' }); + assert.strictEqual(out.maxChars, 1200); + assert.strictEqual(out.schemaVersion, SETTINGS_VERSION); + assert.strictEqual(out.provider, 'gemini', 'unrelated settings are preserved'); +}); + +test('migrate: a v1 file with a user-chosen length keeps it', () => { + assert.strictEqual(migrate({ maxChars: 800 }).maxChars, 800); + assert.strictEqual(migrate({ maxChars: 200 }).maxChars, 200); +}); + +test('migrate: an already-migrated file is returned untouched', () => { + const saved = { maxChars: 500, schemaVersion: SETTINGS_VERSION }; + assert.strictEqual(migrate(saved).maxChars, 500, 'a deliberate 500 set after migrating survives'); +}); + +test('migrate: does not mutate its input', () => { + const saved = { maxChars: 500 }; + migrate(saved); + assert.strictEqual(saved.maxChars, 500); + assert.strictEqual(saved.schemaVersion, undefined); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `npm test -- test/settings.test.js` +Expected: FAIL — `TypeError: migrate is not a function` (the module currently exports only `load`, `save`, `DEFAULTS`, `settingsPath`). + +- [ ] **Step 3: Write the implementation** + +In `src/main/settings.js`, add the version constant above `DEFAULTS`: + +```js +// Bump when a stored settings value needs a one-time migration; see migrate(). +const SETTINGS_VERSION = 2; +``` + +Change the `maxChars` entry in `DEFAULTS` from: + +```js + // 答案字数上限 + maxChars: 500, +``` + +to: + +```js + // 答案字数上限(作为硬性上限,不是目标长度;学术散文需要更大预算) + maxChars: 1200, +``` + +Add `schemaVersion` as the last entry of `DEFAULTS` (after `jobDescription`): + +```js + // 设置结构版本,供 migrate() 做一次性迁移 + schemaVersion: SETTINGS_VERSION, +``` + +Add `migrate()` above `load()`: + +```js +// One-time upgrades of a settings file written by an older version. +// Takes the raw parsed file (before merging DEFAULTS, so a missing +// schemaVersion still reads as v1) and returns a new object. +function migrate(saved) { + const from = typeof saved.schemaVersion === 'number' ? saved.schemaVersion : 1; + if (from >= SETTINGS_VERSION) return saved; + const out = { ...saved }; + // v1 → v2: answers were outline-style and capped at 500 characters. + // Academic prose needs a larger budget, so lift the old default. A value + // the user chose themselves is left alone. + if (from < 2 && out.maxChars === 500) out.maxChars = DEFAULTS.maxChars; + out.schemaVersion = SETTINGS_VERSION; + return out; +} +``` + +Replace the body of `load()`: + +```js +function load() { + try { + const raw = fs.readFileSync(settingsPath(), 'utf8'); + return { ...DEFAULTS, ...migrate(JSON.parse(raw)) }; + } catch (_e) { + return { ...DEFAULTS }; + } +} +``` + +Extend the exports line: + +```js +module.exports = { load, save, migrate, DEFAULTS, SETTINGS_VERSION, settingsPath }; +``` + +- [ ] **Step 4: Run the test to verify it passes** + +Run: `npm test -- test/settings.test.js` +Expected: PASS, all five cases. + +- [ ] **Step 5: Align the renderer's fallback default** + +In `src/renderer/app.js`, add a constant next to the other module-level constants near the top of the file (just after `'use strict';` / the first `const` block): + +```js +// Keep in sync with DEFAULTS.maxChars in src/main/settings.js. +const DEFAULT_MAX_CHARS = 1200; +``` + +Then replace the three hardcoded fallbacks: + +- line 518, in `updateCounter()`: + ```js + const max = state.settings ? state.settings.maxChars || DEFAULT_MAX_CHARS : DEFAULT_MAX_CHARS; + ``` +- line 571, in `openSettings()`: + ```js + $('setMaxChars').value = s.maxChars || DEFAULT_MAX_CHARS; + ``` +- line 592, in `saveSettings()`: + ```js + maxChars: parseInt($('setMaxChars').value, 10) || DEFAULT_MAX_CHARS, + ``` + +No change is needed to the `` in `src/renderer/index.html:335` — 1200 is inside that range. + +No change is needed to `src/main/main.js:329-335` either: its Gemini output-token budget is `min(4096, max(160, ceil(maxChars * perChar * 1.15)))`, which at `maxChars = 1200` and `perChar = 1.1` gives 1518 tokens — well inside the 4096 ceiling — and already scales with `maxChars`. + +- [ ] **Step 6: Run the full suite and the linters** + +Run: `npm test && npm run lint && npm run format:check` +Expected: all pass. `npm run lint` would flag an unused `DEFAULT_MAX_CHARS`, so a pass confirms all three call sites were updated. + +- [ ] **Step 7: Commit** + +```bash +git add src/main/settings.js src/renderer/app.js test/settings.test.js +git commit -m "feat: raise answer length budget to 1200 chars with settings migration" +``` + +--- + +### Task 4: Update the documentation + +**Files:** +- Modify: `README.md:54` (English feature bullet), `README.md:171` (Chinese summary paragraph) +- Modify: `CHANGELOG.md` (`## [Unreleased]` section) + +**Interfaces:** +- Consumes: the behaviour delivered by Tasks 1–3 (prose answers, 1200-character default, committee-question extraction). +- Produces: nothing other tasks depend on. + +- [ ] **Step 1: Update the English feature bullet** + +In `README.md`, replace line 54: + +```markdown +- ✍️ **Concise, outline-style** answers (default ≤ 500 chars), shown alongside the detected question so you can edit and regenerate. +``` + +with: + +```markdown +- ✍️ **Spoken academic prose** — answers are continuous sentences (roughly 3–8, default ceiling 1200 chars), never bullet lists, grounded in your uploaded materials with figures, formulas and terminology reproduced verbatim; shown alongside the detected question so you can edit and regenerate. +``` + +- [ ] **Step 2: Update the Chinese summary** + +In `README.md`, replace the trailing clause of line 171: + +``` +生成简洁的第一人称大纲式答案(默认 ≤500 字)。 +``` + +with: + +``` +生成第一人称的学术口语化答案(连贯成段,不用要点列表;默认上限 1200 字符),数字、公式与术语严格照搬资料原文。 +``` + +Leave the rest of line 171 unchanged. + +- [ ] **Step 3: Add the changelog entry** + +In `CHANGELOG.md`, inside the first `## [Unreleased]` section, add a `### Changed` entry immediately above the existing `- **Renamed the app (display name)…** ` bullet (creating no new heading — `### Changed` already exists there): + +```markdown +- **Academic answer style**: answers are now flowing spoken prose (about 3–8 sentences) instead of the previous outline form with `- ` bullet points, and are cast as a dissertation-defense reply rather than a job-interview reply. The prompt now treats your Knowledge Base as the authoritative source, reproduces numbers, formulas, algorithm names and experimental conditions verbatim, labels anything drawn from general knowledge, reports contradictions between sources, and refuses to invent facts or references. Question extraction targets a committee member's current question, dropping preambles and preferring a genuine follow-up over an already-answered question. +- **Answer length default raised** from 500 to 1200 characters, since character count is now a hard ceiling rather than a target. Existing installs still on the old 500 default are migrated automatically (a length you chose yourself is kept); adjust it any time under Settings → Max characters. +``` + +- [ ] **Step 4: Verify the docs render and nothing else broke** + +Run: `npm run format:check && npm test` +Expected: PASS. If `format:check` flags `README.md` or `CHANGELOG.md`, run `npm run format` and re-check. + +- [ ] **Step 5: Commit** + +```bash +git add README.md CHANGELOG.md +git commit -m "docs: describe academic prose answers and the new length default" +``` + +--- + +## Manual verification (after Task 4) + +Not automatable — do this before considering the work done: + +1. Run `npm start`. +2. Open Settings, confirm **Max characters** shows `1200` (migrated from your saved `500`), set **Answer language** to `Українська`, and save. +3. Load one or more Ukrainian dissertation documents into the Knowledge Base. +4. Type a real defense question into the Current Question box and click **Generate Answer**. +5. Confirm the answer: is continuous prose with **no** lines beginning `- ` or `1.`, contains no `**` or `#`, opens with the answer itself rather than a preamble, and reproduces any figures from your documents exactly. +6. Ask a question your documents do not cover; confirm the answer says plainly that it is not confirmed by the dissertation materials rather than inventing a result. diff --git a/docs/superpowers/specs/2026-09-01-academic-answer-style-design.md b/docs/superpowers/specs/2026-09-01-academic-answer-style-design.md new file mode 100644 index 0000000..8018d47 --- /dev/null +++ b/docs/superpowers/specs/2026-09-01-academic-answer-style-design.md @@ -0,0 +1,177 @@ +# Academic Answer Style — Design + +**Date:** 2026-09-01 +**Status:** Approved (brainstormed in-session) + +## Goal + +Adapt this app's answer layer from a **job-interview** assistant to a +**dissertation-defense** assistant: answers should read as spoken academic +prose delivered to an examination committee, grounded in the candidate's own +dissertation materials — and must never be formatted as bullet lists. + +The doctrine being ported comes from the sibling project +`~/pcworkspace/dissertation/assistant` (a spec-driven Python/PySide6 +implementation of the same idea), specifically its answer prompt +`src/defense_assistant/infrastructure/llm/prompts/answer_v1.md` and its +spec `specs/001-defense-assistant/spec.md` (User Story 4, "Generate a +Dissertation-Grounded Answer with Evidence"). + +## Scope + +**In scope:** the prompt/answer layer only — `src/main/prompt.js`, the +answer-length setting that constrains it, and the docs that describe the +answer style. + +**Out of scope** (decided with the user): + +- A switchable Interview / Defense mode. The app becomes single-purpose at + the prompt level; UI labels are left alone. +- Renaming the Job Description setting to dissertation metadata, or any + other Settings/UI restructuring. +- The sibling spec's evidence machinery — per-answer source excerpts and an + evidence state (strongly / partially / insufficiently supported). That + needs retrieval, IPC and renderer work. +- The sibling spec's "no reliable question detected" extraction outcome + (US3, AC4). Useful only with renderer handling to stop the app answering a + non-question; that is a UI change. + +## Current state + +- `src/main/prompt.js` — a dependency-free pure module exporting + `buildPrompt()`, `EXTRACTION_SYSTEM` and `buildExtractionUser()`. Its + system instruction is written in Chinese and casts the model as **a + candidate in a job interview**. Rule 1 demands outline form verbatim: + "【大纲式,不要整段散文】:第一行一句话给结论/判断;随后 2-3 个以'- '开头的 + 精炼要点… 能用词组就别用整句" — one conclusion line plus 2–3 dash-prefixed + points, phrases preferred over sentences, targeting ~300 characters. This + is the direct cause of the list-formatted answers. +- `settings.js` — `maxChars: 500` default; `load()` merges the persisted + `settings.json` over `DEFAULTS`, so a saved value always wins and there is + no migration mechanism. +- `src/main/main.js:314-335` — calls `buildPrompt()` and derives Gemini's + `maxOutputTokens` as `min(4096, max(160, ceil(maxChars * perChar * 1.15)))` + with `perChar = 0.5` for English, `1.1` otherwise. OpenAI-compatible + providers get a flat 4096 because they may be reasoning models. +- `src/renderer/app.js` — displays the answer via `textContent` (no Markdown + rendering, which is why the prompt bans Markdown) and shows an + `N / max chars` counter, with `500` hardcoded as a fallback in three + places (lines 518, 571, 592). +- `test/prompt.test.js` — seven `node:test` cases, several of which assert + the Chinese prompt strings (`/对话历史/`, `/250 字符以内/`, + `/请用中文作答/`, `/最新\/当前/`, `/目标岗位 JD/`). + +## Design + +### 1. Prompt instructions move to English + +The rewritten system instruction is written in **English**. Rationale: it is +maintainable for this repo's owner, neutral across the four supported +providers, and keeps the *answer* language independently controlled by the +`answerLanguage` setting rather than coupling the prompt to one output +language. + +Exception: the per-language answer rules stay in their **target** language +(`Відповідай українською мовою.`, `请用中文作答。`, `Answer in English.`) +because writing the rule in the target language primes the output language. +Only the `auto` fallback branch is translated from Chinese to +`Answer in the same language the question was asked in.` + +### 2. Persona and answering doctrine + +The persona changes from "you are the candidate in a job interview" to "you +are the PhD candidate defending your own dissertation before an examination +committee, answering aloud." The rules are ported one-for-one from +`answer_v1.md`: + +| Rule | Source | +|---|---| +| Flowing prose, ~3–8 sentences, no bullets/dashes/numbers/headings | `answer_v1.md` rule 1 + the spec's "normally 3–8 sentences" | +| Open with the direct answer; no preamble, no restating the question | `answer_v1.md` rule 1 | +| Dissertation excerpts are the primary, authoritative source of facts | `answer_v1.md` rule 2 | +| Reproduce numbers, formulas, algorithm names, constraints and experimental conditions **verbatim** | `answer_v1.md` rule 2 | +| General knowledge allowed but must be explicitly labelled as not a result of this dissertation | `answer_v1.md` rule 3 | +| Never invent dissertation-specific facts, figures, method names or references | `answer_v1.md` rule 4 | +| Report contradictions between excerpts instead of silently choosing one | `answer_v1.md` rule 5 | +| No relevant excerpt → short cautious general answer, stated as unconfirmed | `answer_v1.md` rule 6 | +| Never reveal reasoning | `answer_v1.md` rule 7 | + +Carried over unchanged from the existing prompt: plain-text-only / no +Markdown, tolerance of speech-recognition noise, answer only the *current* +question, and the `profile` block as a highest-priority override. + +The `jobDescription` block stays wired (its header is translated to English) +so the existing Settings feature keeps working; it is simply empty in +defense use. + +### 3. Length: sentences primary, characters as a ceiling + +`maxChars` stops being a target and becomes a stated hard upper bound; the +prompt asks for approximately 3–8 sentences. Because the character budget no +longer drives the shape of the answer, the default rises from **500 to +1200** — roughly what 3–8 sentences of Ukrainian academic prose occupies. + +A persisted `settings.json` overrides `DEFAULTS`, so raising the default +alone would not reach existing installs. `settings.js` gains a +`schemaVersion` field and a `migrate()` step: on load, a settings file +predating this change (no `schemaVersion`) whose `maxChars` is still exactly +the old default of `500` is lifted to `1200`. Any other value is treated as +a deliberate user choice and left alone. The migrated value is persisted the +next time settings are saved; until then `migrate()` re-applies on every +load, which is idempotent. + +`src/renderer/app.js`'s three hardcoded `500` fallbacks are replaced by one +`DEFAULT_MAX_CHARS = 1200` constant so the character counter and the +Settings field agree with the new default. + +No change is needed to `main.js`'s token math: 1200 characters yields +`ceil(1200 * 1.1 * 1.15) = 1518` output tokens, comfortably inside the 4096 +ceiling, and the formula already scales with `maxChars`. + +### 4. Question extraction + +`EXTRACTION_SYSTEM` is retuned from "interviewer" to a committee member and +gains the handling the sibling spec's User Story 3 calls for: drop +introductory remarks, preserve every meaningful part of a long multi-sentence +question, prefer a genuine short follow-up over the earlier question already +answered, and keep academic terminology exactly as spoken. It keeps its +existing contract — output only the question, in the speaker's language — +so `main.js`'s back-fill of the Current Question box is unaffected. +`buildExtractionUser()` is unchanged. + +### 5. Documentation + +`README.md:54` ("Concise, outline-style answers (default ≤ 500 chars)") and +its Chinese counterpart at `README.md:171` ("简洁的第一人称大纲式答案(默认 +≤500 字)") both describe behaviour this change removes, and are updated to +describe grounded academic prose. A `CHANGELOG.md` entry goes under +`## [Unreleased] / ### Changed`. + +## Testing + +`test/prompt.test.js` is rewritten: the assertions on Chinese prompt strings +no longer apply. New coverage asserts the properties that matter rather than +exact wording where possible — that the prompt forbids lists, asks for +sentences, states the character ceiling, carries the grounding rules +(verbatim figures, no fabrication, labelled general knowledge, contradiction +reporting), and still injects context / question / transcript / profile / +job description. A new `test/settings.test.js` covers `migrate()`: a v1 file +at 500 is lifted, a v1 file with a user-chosen value is not, and an +already-migrated file is untouched. + +Manual verification: run the app against a real Ukrainian defense question +with dissertation documents in the Knowledge Base, and confirm the answer is +continuous prose with no dash-prefixed lines. + +## Risks + +- **Longer answers cost more tokens and take longer to stream.** Accepted: + it is inherent to the requested answer shape, and `maxChars` remains + user-adjustable down to 50. +- **A model may still emit a list despite the ban.** Mitigated by stating the + prohibition explicitly and positively ("flowing prose … continuous + sentences") rather than only negatively. Not enforced in code; if it proves + necessary, a post-processing strip is a follow-up, not part of this change. +- **The migration lifts a `maxChars` of 500 that a user set deliberately.** + Accepted: 500 is indistinguishable from the old default, and the value + stays editable in Settings. From 076cd9d525ea0a73cdb3d81be3130ebcf9bbb70f Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 1 Sep 2026 22:28:29 +0300 Subject: [PATCH 09/33] feat: academic prose answers grounded in dissertation materials Co-Authored-By: Claude Opus 5 (1M context) --- src/main/prompt.js | 54 ++++++++++------- test/prompt.test.js | 139 +++++++++++++++++++++++++++++--------------- 2 files changed, 123 insertions(+), 70 deletions(-) diff --git a/src/main/prompt.js b/src/main/prompt.js index d276947..8b47c58 100644 --- a/src/main/prompt.js +++ b/src/main/prompt.js @@ -1,6 +1,7 @@ 'use strict'; -// 纯函数:构造作答 / 问题提取的提示词。无 Electron 依赖,便于单测。 +// Pure functions that build the answering / question-extraction prompts. +// No Electron dependency, so this module is directly unit-testable. function buildPrompt({ question, @@ -11,6 +12,9 @@ function buildPrompt({ profile, jobDescription, }) { + // The per-language rule is deliberately written in its own target + // language — it primes the model's output language far better than an + // English description of it would. const langRule = answerLanguage === 'zh' ? '请用中文作答。' @@ -18,59 +22,65 @@ function buildPrompt({ ? 'Answer in English.' : answerLanguage === 'uk' ? 'Відповідай українською мовою.' - : '使用与问题相同的语言作答。'; + : 'Answer in the same language the question was asked in.'; const lines = [ - '你是正在参加面试的候选人本人。下面会给出面试现场的对话片段 / 问题,以及可能相关的个人资料/知识库内容。', - '请用第一人称、专业且自然的口吻,像在面试现场【口头作答】一样直接回答问题。', - '要求:', - `1) 严格控制在 ${maxChars} 字符以内(硬性上限),目标约 300 字、宁可更短。【大纲式,不要整段散文】:第一行一句话给结论/判断;随后 2-3 个以“- ”开头的精炼要点(关键词、工具、数字、取舍),能用词组就别用整句。`, - '2) 纯文本输出,禁止使用 Markdown 加粗/星号(**)、井号(#)、表格等标记(界面不渲染 Markdown,会显示成乱码);删掉所有铺垫和客套;', - '3) 直接开口作答,不要复述问题、不要写“我的回答”之类的标题、不要出现“根据资料/上文”之类措辞;', - '4) 若资料中有相关信息务必优先采用并保持事实准确,资料无关则用你的专业知识作答;', - '5) 给到的对话是实时语音识别结果,可能有重复、串音、口误、错别字——请自行容错,判断面试官【当前最可能在问的核心问题】,只回答这个问题;', - `6) ${langRule}`, + 'You are the PhD candidate defending your own dissertation before an examination committee.', + "You will be given a committee member's question, a recent transcript of the session, and excerpts from your own dissertation materials.", + 'Answer in the first person, as if speaking aloud to the committee, in an academic but natural spoken register.', + 'Rules:', + `1) Write FLOWING PROSE — continuous sentences that read as speech. NEVER use bullet points, dashes as list markers, numbered lists, headings or any outline structure. Aim for about 3-8 sentences; ${maxChars} characters is a hard upper bound, not a target.`, + '2) Open with the direct answer to the question. No preamble, no "good question", no restating or summarising the question, no title such as "My answer".', + '3) The dissertation excerpts are the primary and authoritative source of facts about this work. Reproduce numerical values, formulas, algorithm names, constraints, experimental conditions and stated conclusions EXACTLY as they appear in the excerpts — never paraphrase, round or approximate them.', + '4) If the excerpts do not cover part of the question, you may add general knowledge from the subject area, but say explicitly that it is general background and not a result of this dissertation.', + '5) NEVER invent dissertation-specific facts, numbers, method names, citations or references that are not present in the excerpts.', + '6) If the excerpts contradict each other (different values for the same fact), state that discrepancy openly instead of silently picking one.', + '7) If no excerpt is relevant to the question, give a short, careful answer from general knowledge and state plainly that it is not confirmed by the dissertation materials.', + '8) Plain text only. No Markdown: no asterisks (**), no hash headings, no tables — the interface does not render Markdown and would show the raw characters.', + "9) The transcript is live speech recognition and may contain repetitions, cross-talk, misrecognised words and unfinished sentences. Tolerate that noise and answer only the committee's CURRENT core question.", + '10) Never reveal your reasoning — output only the final answer.', + `11) ${langRule}`, ]; if (jobDescription && jobDescription.trim()) { lines.push( '', - '================ 目标岗位 JD(请据此定制回答:对齐岗位要求、技术栈与关键词,突出匹配点)================', + '================ ADDITIONAL TARGET CONTEXT (tailor the answer to it: align with its requirements, terminology and keywords) ================', jobDescription.trim().slice(0, 6000), ); } if (profile && profile.trim()) { lines.push( '', - '================ 本次面试背景与作答风格(最高优先级,务必遵循) ================', + '================ SESSION BACKGROUND AND ANSWERING STYLE (HIGHEST PRIORITY — follow it) ================', profile.trim(), ); } const systemInstruction = lines.join('\n'); const parts = []; - if (context) parts.push(`【可参考的个人资料 / 知识库】\n${context}\n`); + if (context) parts.push(`[DISSERTATION MATERIALS / KNOWLEDGE BASE EXCERPTS]\n${context}\n`); const q = (question || '').trim(); if (q) { if (transcript) { parts.push( - `【最近约15轮面试对话历史(语音识别,可能有重复/错误,仅供你理解上下文、保持连贯)】\n${transcript}\n`, + `[RECENT SESSION TRANSCRIPT (~15 turns, speech recognition — may contain repeats or errors; use it only for context and continuity)]\n${transcript}\n`, ); } - parts.push(`【需要回答的问题】\n${q}`); - parts.push('请结合上面的对话上下文直接作答。'); - parts.push('\n请直接给出你的回答:'); + parts.push(`[QUESTION TO ANSWER]\n${q}`); + parts.push('Answer it directly, using the material above.'); + parts.push('\nYour answer:'); } else { parts.push( - `【最近约15轮面试对话历史(语音识别,可能有重复/串音/口误)】\n${transcript || '(暂无对话)'}\n`, + `[RECENT SESSION TRANSCRIPT (~15 turns, speech recognition — may contain repeats, cross-talk or errors)]\n${transcript || '(no dialogue yet)'}\n`, ); parts.push( - '请在心里判断面试官【最新/当前】正在问的核心问题,然后【直接作答】;较早的轮次只作为背景上下文,用来让回答更贴合、连贯,不要去回答更早的旧问题。', + 'Work out silently which question the committee is asking RIGHT NOW, then answer it directly. Earlier turns are background context only — do not answer an older question that was already addressed.', ); parts.push( - '严禁任何前缀或复述,例如不得出现“The core question is…”“面试官在问…”“你的问题是…”,第一句话就是你的回答本身。', + 'Do not output any prefix or restatement — no "The core question is…", no "The committee is asking…", no "Your question is…". The first sentence must already be the answer itself.', ); - parts.push('\n你的回答:'); + parts.push('\nYour answer:'); } return { systemInstruction, userText: parts.join('\n') }; diff --git a/test/prompt.test.js b/test/prompt.test.js index 47115f1..16440a9 100644 --- a/test/prompt.test.js +++ b/test/prompt.test.js @@ -4,69 +4,112 @@ const test = require('node:test'); const assert = require('node:assert'); const { buildPrompt, EXTRACTION_SYSTEM, buildExtractionUser } = require('../src/main/prompt'); -test('buildPrompt: explicit question mode includes the question and history', () => { - const { systemInstruction, userText } = buildPrompt({ - question: 'Why vLLM?', - transcript: 'Interviewer: tell me about inference\nCandidate: sure', - context: '', - answerLanguage: 'en', - maxChars: 500, - profile: '', - }); - assert.match(systemInstruction, /500/); - assert.match(systemInstruction, /Answer in English\./); - assert.match(userText, /Why vLLM\?/); - assert.match(userText, /对话历史/); +const BASE = { + question: 'q', + transcript: '', + context: '', + answerLanguage: 'en', + maxChars: 1200, +}; + +test('buildPrompt: asks for flowing prose and forbids lists', () => { + const { systemInstruction } = buildPrompt(BASE); + assert.match(systemInstruction, /FLOWING PROSE/); + assert.match(systemInstruction, /3-8 sentences/); + assert.match(systemInstruction, /NEVER use bullet points/); + // The old outline instruction must be gone. + assert.doesNotMatch(systemInstruction, /大纲式/); + assert.doesNotMatch(systemInstruction, /要点/); +}); + +test('buildPrompt: maxChars is stated as a hard upper bound, not a target', () => { + const { systemInstruction } = buildPrompt({ ...BASE, maxChars: 900 }); + assert.match(systemInstruction, /900 characters is a hard upper bound/); +}); + +test('buildPrompt: casts the model as a defending PhD candidate', () => { + const { systemInstruction } = buildPrompt(BASE); + assert.match(systemInstruction, /PhD candidate defending your own dissertation/); + assert.match(systemInstruction, /examination committee/); }); -test('buildPrompt: extract mode (no question) instructs to find the latest question', () => { +test('buildPrompt: carries the grounding rules from the dissertation spec', () => { + const { systemInstruction } = buildPrompt(BASE); + // Corpus primacy + verbatim figures. + assert.match(systemInstruction, /primary and authoritative source/); + assert.match(systemInstruction, /EXACTLY as they appear/); + // General knowledge must be labelled. + assert.match(systemInstruction, /not a result of this dissertation/); + // No fabrication. + assert.match(systemInstruction, /NEVER invent dissertation-specific facts/); + // Contradictions are reported, not silently resolved. + assert.match(systemInstruction, /contradict each other/); + // No relevant evidence → cautious, explicitly unconfirmed. + assert.match(systemInstruction, /not confirmed by the dissertation materials/); + // Plain text only (the renderer does not render Markdown). + assert.match(systemInstruction, /No Markdown/); +}); + +test('buildPrompt: answer-language rules stay in their target language', () => { + assert.match( + buildPrompt({ ...BASE, answerLanguage: 'en' }).systemInstruction, + /Answer in English\./, + ); + assert.match(buildPrompt({ ...BASE, answerLanguage: 'zh' }).systemInstruction, /请用中文作答。/); + assert.match( + buildPrompt({ ...BASE, answerLanguage: 'uk' }).systemInstruction, + /Відповідай українською мовою\./, + ); + assert.match( + buildPrompt({ ...BASE, answerLanguage: 'auto' }).systemInstruction, + /same language the question was asked in/, + ); +}); + +test('buildPrompt: explicit question mode includes the question and the transcript', () => { const { userText } = buildPrompt({ - question: '', - transcript: 'Interviewer: what is a KV cache?', - context: '', - answerLanguage: 'auto', - maxChars: 500, + ...BASE, + question: 'Why this optimisation algorithm?', + transcript: 'Committee: tell us about the method\nCandidate: certainly', }); - assert.match(userText, /最新\/当前/); - assert.doesNotMatch(userText, /需要回答的问题/); // no explicit-question block + assert.match(userText, /Why this optimisation algorithm\?/); + assert.match(userText, /RECENT SESSION TRANSCRIPT/); + assert.match(userText, /QUESTION TO ANSWER/); }); -test('buildPrompt: maxChars and profile are injected', () => { - const { systemInstruction } = buildPrompt({ - question: 'q', - transcript: '', - context: '', - answerLanguage: 'zh', - maxChars: 250, - profile: 'You are interviewing at Google.', +test('buildPrompt: extract mode (no question) answers the current question only', () => { + const { userText } = buildPrompt({ + ...BASE, + question: '', + transcript: 'Committee: how did you validate the results?', }); - assert.match(systemInstruction, /250 字符以内/); - assert.match(systemInstruction, /You are interviewing at Google\./); - assert.match(systemInstruction, /请用中文作答/); + assert.match(userText, /RIGHT NOW/); + assert.match(userText, /first sentence must already be the answer/); + assert.doesNotMatch(userText, /QUESTION TO ANSWER/); // no explicit-question block }); test('buildPrompt: knowledge-base context is included when provided', () => { - const { userText } = buildPrompt({ - question: 'q', - transcript: '', - context: 'RESUME: senior engineer', - answerLanguage: 'en', - maxChars: 500, + const { userText } = buildPrompt({ ...BASE, context: 'Section 3.2: RMSE = 0.041' }); + assert.match(userText, /DISSERTATION MATERIALS/); + assert.match(userText, /RMSE = 0\.041/); +}); + +test('buildPrompt: profile is injected as a highest-priority override', () => { + const { systemInstruction } = buildPrompt({ + ...BASE, + profile: 'The committee chair is a statistician.', }); - assert.match(userText, /RESUME: senior engineer/); + assert.match(systemInstruction, /HIGHEST PRIORITY/); + assert.match(systemInstruction, /The committee chair is a statistician\./); }); -test('buildPrompt: job description is injected as tailoring context', () => { +test('buildPrompt: job description is still injected as tailoring context', () => { const { systemInstruction } = buildPrompt({ - question: 'q', - transcript: '', - context: '', - answerLanguage: 'en', - maxChars: 500, - jobDescription: 'Senior Forward Deployed Engineer — must know vLLM and GCP.', + ...BASE, + jobDescription: 'Defense of a thesis on adaptive control.', }); - assert.match(systemInstruction, /目标岗位 JD/); - assert.match(systemInstruction, /must know vLLM and GCP/); + assert.match(systemInstruction, /ADDITIONAL TARGET CONTEXT/); + assert.match(systemInstruction, /adaptive control/); }); test('extraction prompt helpers', () => { From 7ae63149d5631fbcaa34aca7120731a7cfbee219 Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 1 Sep 2026 22:29:53 +0300 Subject: [PATCH 10/33] feat: extract committee questions instead of interview questions Co-Authored-By: Claude Opus 5 (1M context) --- src/main/prompt.js | 4 ++-- test/prompt.test.js | 18 ++++++++++++++++-- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/src/main/prompt.js b/src/main/prompt.js index 8b47c58..0e41dcb 100644 --- a/src/main/prompt.js +++ b/src/main/prompt.js @@ -86,9 +86,9 @@ function buildPrompt({ return { systemInstruction, userText: parts.join('\n') }; } -// 问题提取(只看最近几轮) +// Question extraction (looks only at the most recent turns). const EXTRACTION_SYSTEM = - "You clean up noisy live interview transcripts. The transcript may contain repeats, cross-talk, ASR errors and half-sentences. Identify the interviewer's CURRENT core question and rewrite it as ONE clean, complete question. Output ONLY that question — no prefix, no quotes, no explanation. Write it in the SAME language the interviewer is speaking."; + 'You clean up noisy live transcripts of a dissertation defense. The transcript may contain repeats, cross-talk, speech-recognition errors and half-sentences. Identify the CURRENT core question a committee member is putting to the candidate and rewrite it as ONE clean, complete question. Drop introductory remarks, compliments and asides, but keep every meaningful part of a long multi-sentence question. If the latest turn is a short follow-up to an earlier question, return that follow-up rather than the earlier question. Preserve academic and technical terminology exactly as spoken. Output ONLY that question — no prefix, no quotes, no explanation. Write it in the SAME language the committee member is speaking.'; function buildExtractionUser(recentTranscript) { return `Recent turns:\n${recentTranscript}\n\nThe interviewer's current core question is:`; diff --git a/test/prompt.test.js b/test/prompt.test.js index 16440a9..e558301 100644 --- a/test/prompt.test.js +++ b/test/prompt.test.js @@ -112,9 +112,23 @@ test('buildPrompt: job description is still injected as tailoring context', () = assert.match(systemInstruction, /adaptive control/); }); -test('extraction prompt helpers', () => { +test('extraction prompt targets the committee question and keeps its contract', () => { + // Domain: a defense, not a job interview. + assert.match(EXTRACTION_SYSTEM, /dissertation defense/); + assert.match(EXTRACTION_SYSTEM, /committee member/); + assert.doesNotMatch(EXTRACTION_SYSTEM, /interview/i); + // Spec US3 behaviours. + assert.match(EXTRACTION_SYSTEM, /Drop introductory remarks/); + assert.match(EXTRACTION_SYSTEM, /multi-sentence question/); + assert.match(EXTRACTION_SYSTEM, /follow-up/); + assert.match(EXTRACTION_SYSTEM, /terminology/); + // Unchanged output contract. assert.match(EXTRACTION_SYSTEM, /ONLY that question/); - assert.match(buildExtractionUser('Interviewer: hi'), /Interviewer: hi/); + assert.match(EXTRACTION_SYSTEM, /SAME language/); +}); + +test('buildExtractionUser embeds the transcript and the trailing cue', () => { + assert.match(buildExtractionUser('Committee: hi'), /Committee: hi/); assert.match(buildExtractionUser('x'), /current core question is:/); }); From 4983bcd6d5401bc1f6322702dce44c4997cfab86 Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 1 Sep 2026 22:31:25 +0300 Subject: [PATCH 11/33] feat: raise answer length budget to 1200 chars with settings migration Co-Authored-By: Claude Opus 5 (1M context) --- src/main/settings.js | 28 ++++++++++++++++++++++++---- src/renderer/app.js | 9 ++++++--- test/settings.test.js | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 64 insertions(+), 7 deletions(-) create mode 100644 test/settings.test.js diff --git a/src/main/settings.js b/src/main/settings.js index 5e93ab7..4263e2a 100644 --- a/src/main/settings.js +++ b/src/main/settings.js @@ -4,6 +4,9 @@ const fs = require('fs'); const path = require('path'); const { app } = require('electron'); +// Bump when a stored settings value needs a one-time migration; see migrate(). +const SETTINGS_VERSION = 2; + const DEFAULTS = { deepgramApiKey: process.env.DEEPGRAM_API_KEY || '', // 答案 Provider: deepseek / gemini / openai / ollama @@ -25,24 +28,41 @@ const DEFAULTS = { hotkey: 'Control+A', // 自动作答:监测到面试官问完一个问题就自动触发(无需按热键) autoAnswer: false, - // 答案字数上限 - maxChars: 500, + // 答案字数上限(作为硬性上限,不是目标长度;学术散文需要更大预算) + maxChars: 1200, // 答案语言: auto(跟随问题) / zh / en answerLanguage: 'auto', // 面试背景与作答风格(注入到系统提示,最高优先级) interviewProfile: '', // 目标岗位 JD(持久化;上传或粘贴,作答时据此定制) jobDescription: '', + // 设置结构版本,供 migrate() 做一次性迁移 + schemaVersion: SETTINGS_VERSION, }; function settingsPath() { return path.join(app.getPath('userData'), 'settings.json'); } +// One-time upgrades of a settings file written by an older version. +// Takes the raw parsed file (before merging DEFAULTS, so a missing +// schemaVersion still reads as v1) and returns a new object. +function migrate(saved) { + const from = typeof saved.schemaVersion === 'number' ? saved.schemaVersion : 1; + if (from >= SETTINGS_VERSION) return saved; + const out = { ...saved }; + // v1 → v2: answers were outline-style and capped at 500 characters. + // Academic prose needs a larger budget, so lift the old default. A value + // the user chose themselves is left alone. + if (from < 2 && out.maxChars === 500) out.maxChars = DEFAULTS.maxChars; + out.schemaVersion = SETTINGS_VERSION; + return out; +} + function load() { try { const raw = fs.readFileSync(settingsPath(), 'utf8'); - return { ...DEFAULTS, ...JSON.parse(raw) }; + return { ...DEFAULTS, ...migrate(JSON.parse(raw)) }; } catch (_e) { return { ...DEFAULTS }; } @@ -58,4 +78,4 @@ function save(partial) { return merged; } -module.exports = { load, save, DEFAULTS, settingsPath }; +module.exports = { load, save, migrate, DEFAULTS, SETTINGS_VERSION, settingsPath }; diff --git a/src/renderer/app.js b/src/renderer/app.js index a58506c..580d13b 100644 --- a/src/renderer/app.js +++ b/src/renderer/app.js @@ -2,6 +2,9 @@ const $ = (id) => document.getElementById(id); +// Keep in sync with DEFAULTS.maxChars in src/main/settings.js. +const DEFAULT_MAX_CHARS = 1200; + const state = { listening: false, settings: null, @@ -515,7 +518,7 @@ function endGenerate() { function updateCounter(text) { const n = charCount(text); - const max = state.settings ? state.settings.maxChars || 500 : 500; + const max = state.settings ? state.settings.maxChars || DEFAULT_MAX_CHARS : DEFAULT_MAX_CHARS; const el = $('charCounter'); el.textContent = `${n} / ${max} chars`; el.classList.toggle('over', n > max); @@ -568,7 +571,7 @@ function openSettings() { $('setOllamaModel').value = s.ollamaModel || 'llama3.1'; $('setSttLang').value = s.sttLanguage || 'en-US'; $('setAnswerLang').value = s.answerLanguage || 'auto'; - $('setMaxChars').value = s.maxChars || 500; + $('setMaxChars').value = s.maxChars || DEFAULT_MAX_CHARS; $('setHotkey').value = s.hotkey || 'Control+A'; $('setProfile').value = s.interviewProfile || ''; $('setJD').value = s.jobDescription || ''; @@ -589,7 +592,7 @@ async function saveSettings() { ollamaModel: $('setOllamaModel').value.trim() || 'llama3.1', sttLanguage: $('setSttLang').value, answerLanguage: $('setAnswerLang').value, - maxChars: parseInt($('setMaxChars').value, 10) || 500, + maxChars: parseInt($('setMaxChars').value, 10) || DEFAULT_MAX_CHARS, hotkey: $('setHotkey').value.trim() || 'Control+A', interviewProfile: $('setProfile').value, jobDescription: $('setJD').value, diff --git a/test/settings.test.js b/test/settings.test.js new file mode 100644 index 0000000..f300281 --- /dev/null +++ b/test/settings.test.js @@ -0,0 +1,34 @@ +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert'); +const { migrate, DEFAULTS, SETTINGS_VERSION } = require('../src/main/settings'); + +test('DEFAULTS: answer length is the academic-prose budget', () => { + assert.strictEqual(DEFAULTS.maxChars, 1200); + assert.strictEqual(DEFAULTS.schemaVersion, SETTINGS_VERSION); +}); + +test('migrate: a v1 file still on the old 500 default is lifted to the new default', () => { + const out = migrate({ maxChars: 500, provider: 'gemini' }); + assert.strictEqual(out.maxChars, 1200); + assert.strictEqual(out.schemaVersion, SETTINGS_VERSION); + assert.strictEqual(out.provider, 'gemini', 'unrelated settings are preserved'); +}); + +test('migrate: a v1 file with a user-chosen length keeps it', () => { + assert.strictEqual(migrate({ maxChars: 800 }).maxChars, 800); + assert.strictEqual(migrate({ maxChars: 200 }).maxChars, 200); +}); + +test('migrate: an already-migrated file is returned untouched', () => { + const saved = { maxChars: 500, schemaVersion: SETTINGS_VERSION }; + assert.strictEqual(migrate(saved).maxChars, 500, 'a deliberate 500 set after migrating survives'); +}); + +test('migrate: does not mutate its input', () => { + const saved = { maxChars: 500 }; + migrate(saved); + assert.strictEqual(saved.maxChars, 500); + assert.strictEqual(saved.schemaVersion, undefined); +}); From d72a645c8bb1424be55b3a5d77d1b3f482782795 Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 1 Sep 2026 22:32:31 +0300 Subject: [PATCH 12/33] docs: describe academic prose answers and the new length default Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 2 ++ README.md | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5c2ebe3..f43284c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,8 @@ All notable changes to this project are documented here. Format loosely follows - System-audio capture without Screen Recording permission now fails quietly with a clear prompt instead of throwing `Failed to get sources` / `Video was requested…` unhandled rejections. When the permission is already denied, the app skips the capture attempt and guides you to grant it (continuing mic-only). ### Changed +- **Academic answer style**: answers are now flowing spoken prose (about 3–8 sentences) instead of the previous outline form with `- ` bullet points, and are cast as a dissertation-defense reply rather than a job-interview reply. The prompt now treats your Knowledge Base as the authoritative source, reproduces numbers, formulas, algorithm names and experimental conditions verbatim, labels anything drawn from general knowledge, reports contradictions between sources, and refuses to invent facts or references. Question extraction targets a committee member's current question, dropping preambles and preferring a genuine follow-up over an already-answered question. +- **Answer length default raised** from 500 to 1200 characters, since character count is now a hard ceiling rather than a target. Existing installs still on the old 500 default are migrated automatically (a length you chose yourself is kept); adjust it any time under Settings → Max characters. - Renamed the app (display name) to **Real Time Interview Copilot**. The npm package, bundle id (`com.interview.copilot`), repo slug, and local data directory are unchanged, so existing settings/keys/JD/Knowledge Base are preserved. ### Added diff --git a/README.md b/README.md index 407d973..40f780f 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ Want **zero cloud**? Use **Ollama** as the answer provider (runs locally). A ful - 🧠 **Context-aware** — answers are grounded in the last ~15 turns of dialogue + your uploaded résumé / JD / notes (a persistent Knowledge Base you can add to, update, and clear). - 🎯 **JD customization** — paste or upload a job description in Settings; it's persisted and tailors every answer to the target role. - 🔌 **Switchable providers** — DeepSeek, Gemini, OpenAI, or local Ollama, with automatic retry + model fallback. -- ✍️ **Concise, outline-style** answers (default ≤ 500 chars), shown alongside the detected question so you can edit and regenerate. +- ✍️ **Spoken academic prose** — answers are continuous sentences (roughly 3–8, default ceiling 1200 chars), never bullet lists, grounded in your uploaded materials with figures, formulas and terminology reproduced verbatim; shown alongside the detected question so you can edit and regenerate. - 🖥️ **Single, clean UI** — live transcript on the left; question / answer / knowledge base on the right. ## Requirements @@ -168,7 +168,7 @@ If Real Time Interview Copilot is useful to you, please **⭐ star the repo** ## 中文 -基于 **Deepgram**(实时转写)+ **DeepSeek / Gemini / OpenAI / Ollama**(答案生成)的桌面应用:双声道识别(麦克风=面试者、系统声音=面试官),按 `Ctrl+A` 自动识别面试官当前的问题,并结合最近约 15 轮对话 + 你上传的资料,生成简洁的第一人称大纲式答案(默认 ≤500 字)。 +基于 **Deepgram**(实时转写)+ **DeepSeek / Gemini / OpenAI / Ollama**(答案生成)的桌面应用:双声道识别(麦克风=面试者、系统声音=面试官),按 `Ctrl+A` 自动识别面试官当前的问题,并结合最近约 15 轮对话 + 你上传的资料,生成第一人称的学术口语化答案(连贯成段,不用要点列表;默认上限 1200 字符),数字、公式与术语严格照搬资料原文。 **⚠️ 免责声明**:本工具用于**面试练习、复盘、模拟面试与无障碍辅助**。在**真实面试**中未经对方知情使用实时答题,可能违反对方公司政策、协议、学术诚信规则或当地法律,且通常被视为不诚信行为。**如何使用、是否合规由你自行负责**,作者不为此背书,软件按「现状」提供、不附带任何担保。 From cb2c67c13ee2979d216ad9b8cd7966261b0096e4 Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 1 Sep 2026 22:35:06 +0300 Subject: [PATCH 13/33] fix: drop interview framing from the extraction cue The trailing cue in buildExtractionUser is concatenated directly after EXTRACTION_SYSTEM, which now describes a dissertation defense, so asking for "the interviewer's current core question" contradicted it inside a single request. Also removes a duplicate Ukrainian answer-language test superseded by the target-language case, and corrects the design doc's claim that buildExtractionUser needed no change. Co-Authored-By: Claude Opus 5 (1M context) --- .../2026-09-01-academic-answer-style-design.md | 6 +++++- src/main/prompt.js | 2 +- test/prompt.test.js | 16 ++++------------ 3 files changed, 10 insertions(+), 14 deletions(-) diff --git a/docs/superpowers/specs/2026-09-01-academic-answer-style-design.md b/docs/superpowers/specs/2026-09-01-academic-answer-style-design.md index 8018d47..0ce6b00 100644 --- a/docs/superpowers/specs/2026-09-01-academic-answer-style-design.md +++ b/docs/superpowers/specs/2026-09-01-academic-answer-style-design.md @@ -137,7 +137,11 @@ question, prefer a genuine short follow-up over the earlier question already answered, and keep academic terminology exactly as spoken. It keeps its existing contract — output only the question, in the speaker's language — so `main.js`'s back-fill of the Current Question box is unaffected. -`buildExtractionUser()` is unchanged. +`buildExtractionUser()`'s trailing cue changes only its domain noun — +"The interviewer's current core question is:" becomes "The committee +member's ..." — because that cue is concatenated directly after +`EXTRACTION_SYSTEM` and would otherwise reintroduce the job-interview +framing the same request just dropped. ### 5. Documentation diff --git a/src/main/prompt.js b/src/main/prompt.js index 0e41dcb..e8df48c 100644 --- a/src/main/prompt.js +++ b/src/main/prompt.js @@ -91,7 +91,7 @@ const EXTRACTION_SYSTEM = 'You clean up noisy live transcripts of a dissertation defense. The transcript may contain repeats, cross-talk, speech-recognition errors and half-sentences. Identify the CURRENT core question a committee member is putting to the candidate and rewrite it as ONE clean, complete question. Drop introductory remarks, compliments and asides, but keep every meaningful part of a long multi-sentence question. If the latest turn is a short follow-up to an earlier question, return that follow-up rather than the earlier question. Preserve academic and technical terminology exactly as spoken. Output ONLY that question — no prefix, no quotes, no explanation. Write it in the SAME language the committee member is speaking.'; function buildExtractionUser(recentTranscript) { - return `Recent turns:\n${recentTranscript}\n\nThe interviewer's current core question is:`; + return `Recent turns:\n${recentTranscript}\n\nThe committee member's current core question is:`; } module.exports = { buildPrompt, EXTRACTION_SYSTEM, buildExtractionUser }; diff --git a/test/prompt.test.js b/test/prompt.test.js index e558301..1f0ab57 100644 --- a/test/prompt.test.js +++ b/test/prompt.test.js @@ -129,16 +129,8 @@ test('extraction prompt targets the committee question and keeps its contract', test('buildExtractionUser embeds the transcript and the trailing cue', () => { assert.match(buildExtractionUser('Committee: hi'), /Committee: hi/); - assert.match(buildExtractionUser('x'), /current core question is:/); -}); - -test('buildPrompt: answerLanguage uk produces a Ukrainian answer instruction', () => { - const { systemInstruction } = buildPrompt({ - question: 'q', - transcript: '', - context: '', - answerLanguage: 'uk', - maxChars: 500, - }); - assert.match(systemInstruction, /Відповідай українською мовою\./); + assert.match(buildExtractionUser('x'), /committee member's current core question is:/); + // The cue is concatenated after EXTRACTION_SYSTEM, so it must not + // reintroduce the job-interview framing that prompt just dropped. + assert.doesNotMatch(buildExtractionUser('x'), /interviewer/i); }); From 7d8fcc5910aec95cc6afecd432f4cbd60170a1d7 Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 8 Sep 2026 17:10:14 +0300 Subject: [PATCH 14/33] docs: add spec and plan for macOS system-audio loopback fix Diagnoses why the Interviewer channel captures nothing on macOS: the getDisplayMedia loopback path needs two Chromium feature flags that main.js never sets, so the stream comes back video-only. On headphones the interviewer is never transcribed; on speakers their speech reaches only the microphone, which is hardwired to the "You" label. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HzVW3YgFohygg7bM1Hxxyb --- .../2026-09-08-macos-system-audio-loopback.md | 405 ++++++++++++++++++ ...9-08-macos-system-audio-loopback-design.md | 177 ++++++++ 2 files changed, 582 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-08-macos-system-audio-loopback.md create mode 100644 docs/superpowers/specs/2026-09-08-macos-system-audio-loopback-design.md diff --git a/docs/superpowers/plans/2026-09-08-macos-system-audio-loopback.md b/docs/superpowers/plans/2026-09-08-macos-system-audio-loopback.md new file mode 100644 index 0000000..8e4b5bc --- /dev/null +++ b/docs/superpowers/plans/2026-09-08-macos-system-audio-loopback.md @@ -0,0 +1,405 @@ +# macOS System-Audio Loopback Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make the Interviewer channel actually capture system audio on macOS, so the other party's speech stops being transcribed under the `You` label. + +**Architecture:** The existing `setDisplayMediaRequestHandler` in `src/main/main.js` already answers with `audio: 'loopback'` — the correct shape. What is missing is the pair of Chromium feature flags that unlock loopback capture on macOS, which must be appended to the command line before app ready. We add a dependency-free pure module that builds the `enable-features` value per platform (unit-testable without booting Electron), call it at module scope in `main.js`, enable echo cancellation on the microphone so speaker output stops leaking into the candidate channel, and make the renderer throw when the system stream comes back without an audio track instead of silently succeeding. + +**Tech Stack:** Electron 33 (plain JS, CommonJS), Chromium `enable-features` command-line switch, WebRTC `getDisplayMedia` / `getUserMedia`, `node:test` for unit tests, ESLint + Prettier. + +**Spec:** `docs/superpowers/specs/2026-09-08-macos-system-audio-loopback-design.md` + +## Global Constraints + +- Node ≥ 18; tests run with `npm test` (`node --test`), lint with `npm run lint`, format check with `npm run format:check`. +- **No new dependencies.** The flag names are copied from `alectrocute/electron-audio-loopback`; the library itself is not installed. +- Electron stays pinned at `^33.2.0`. Do not bump it. +- The feature-flag switch key is exactly `enable-features`. +- macOS flag names are exactly `MacLoopbackAudioForScreenShare` and `MacSckSystemAudioLoopbackOverride`. The Linux flag name is exactly `PulseaudioLoopbackForScreenShare`. Windows gets no flags. +- Flags must be applied **before** `app.whenReady()` — Chromium reads command-line switches at startup and ignores later changes. +- `getDisplayMedia` must keep requesting `video: true`; loopback fails outright without it, even though the video track is discarded. +- New source files under `src/main/` are `'use strict';` CommonJS modules, like every existing file there. +- Commit messages follow the repo's conventional style (`feat:`, `fix:`, `test:`, `docs:`). + +--- + +### Task 1: Pure module that builds the platform feature-flag value + +**Files:** +- Create: `src/main/audioLoopback.js` +- Create: `test/audioLoopback.test.js` + +**Interfaces:** +- Consumes: nothing. This module has no imports — it must stay loadable under plain `node --test` without Electron. +- Produces: `FEATURE_SWITCH` (the string `'enable-features'`) and `buildLoopbackFeatures({ platform, existing })`, which returns a comma-joined feature string, or `null` when the platform needs no flags. Task 2 imports both by these exact names. + +- [ ] **Step 1: Write the failing test** + +Create `test/audioLoopback.test.js`: + +```js +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert'); +const { FEATURE_SWITCH, buildLoopbackFeatures } = require('../src/main/audioLoopback'); + +test('FEATURE_SWITCH is the Chromium enable-features switch', () => { + assert.strictEqual(FEATURE_SWITCH, 'enable-features'); +}); + +test('darwin gets both macOS loopback flags', () => { + assert.strictEqual( + buildLoopbackFeatures({ platform: 'darwin' }), + 'MacLoopbackAudioForScreenShare,MacSckSystemAudioLoopbackOverride', + ); +}); + +test('linux gets the PulseAudio loopback flag', () => { + assert.strictEqual( + buildLoopbackFeatures({ platform: 'linux' }), + 'PulseaudioLoopbackForScreenShare', + ); +}); + +test('win32 needs no flags (loopback is native there)', () => { + assert.strictEqual(buildLoopbackFeatures({ platform: 'win32' }), null); +}); + +test('an unknown platform needs no flags', () => { + assert.strictEqual(buildLoopbackFeatures({ platform: 'freebsd' }), null); +}); + +test('an existing enable-features value is preserved, required flags appended', () => { + assert.strictEqual( + buildLoopbackFeatures({ platform: 'darwin', existing: 'SomeOtherFeature' }), + 'SomeOtherFeature,MacLoopbackAudioForScreenShare,MacSckSystemAudioLoopbackOverride', + ); +}); + +test('a flag that is already present is not duplicated', () => { + assert.strictEqual( + buildLoopbackFeatures({ platform: 'darwin', existing: 'MacLoopbackAudioForScreenShare' }), + 'MacLoopbackAudioForScreenShare,MacSckSystemAudioLoopbackOverride', + ); +}); + +test('empty, whitespace and absent existing values produce no stray separators', () => { + const expected = 'MacLoopbackAudioForScreenShare,MacSckSystemAudioLoopbackOverride'; + assert.strictEqual(buildLoopbackFeatures({ platform: 'darwin', existing: '' }), expected); + assert.strictEqual(buildLoopbackFeatures({ platform: 'darwin', existing: ' , ' }), expected); + assert.strictEqual(buildLoopbackFeatures({ platform: 'darwin', existing: undefined }), expected); +}); + +test('surrounding whitespace in an existing value is trimmed', () => { + assert.strictEqual( + buildLoopbackFeatures({ platform: 'darwin', existing: ' A , B ' }), + 'A,B,MacLoopbackAudioForScreenShare,MacSckSystemAudioLoopbackOverride', + ); +}); + +test('called with no arguments it returns null instead of throwing', () => { + assert.strictEqual(buildLoopbackFeatures(), null); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `node --test test/audioLoopback.test.js` +Expected: FAIL — `Cannot find module '../src/main/audioLoopback'`, because the module does not exist yet. + +- [ ] **Step 3: Write minimal implementation** + +Create `src/main/audioLoopback.js`: + +```js +'use strict'; + +// 系统声音(面试官)走 getDisplayMedia 的 loopback 采集。 +// Electron 31.0.1–38.x 在 macOS/Linux 上需要显式打开 Chromium 的 feature flag, +// 且必须在 app ready 之前写进命令行;Electron 39+ 已内置,无需这些开关。 +// Flag 名称取自 alectrocute/electron-audio-loopback (src/config.ts)。 +const FEATURE_SWITCH = 'enable-features'; + +const LOOPBACK_FEATURES = { + // MacSckSystemAudioLoopbackOverride 让 loopback 走 ScreenCaptureKit(macOS 13.2+)。 + darwin: ['MacLoopbackAudioForScreenShare', 'MacSckSystemAudioLoopbackOverride'], + linux: ['PulseaudioLoopbackForScreenShare'], + // win32 原生支持 loopback,不需要开关。 +}; + +// 合并而不是覆盖:保留命令行上已有的 enable-features 值,只补上缺失的开关。 +function buildLoopbackFeatures({ platform, existing } = {}) { + const required = LOOPBACK_FEATURES[platform]; + if (!required || !required.length) return null; + + const merged = (existing || '') + .split(',') + .map((f) => f.trim()) + .filter(Boolean); + + for (const f of required) { + if (!merged.includes(f)) merged.push(f); + } + return merged.join(','); +} + +module.exports = { FEATURE_SWITCH, LOOPBACK_FEATURES, buildLoopbackFeatures }; +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `node --test test/audioLoopback.test.js` +Expected: PASS — 10 tests, 0 failures. + +- [ ] **Step 5: Run the full suite and lint** + +Run: `npm test && npm run lint` +Expected: PASS — the new module is standalone, so nothing else changes. + +- [ ] **Step 6: Commit** + +```bash +git add src/main/audioLoopback.js test/audioLoopback.test.js +git commit -m "feat: add platform loopback feature-flag builder" +``` + +--- + +### Task 2: Apply the flags in the main process before app ready + +**Files:** +- Modify: `src/main/main.js:16-23` (the `require` block — append one import beneath it, plus the new function and its call) + +**Interfaces:** +- Consumes: `FEATURE_SWITCH` and `buildLoopbackFeatures({ platform, existing })` from `src/main/audioLoopback.js` (Task 1). +- Produces: nothing importable. The observable effect is that `getDisplayMedia` with `audio: 'loopback'` yields a real audio track on macOS, which Task 3's audio-track guard depends on. + +This task has no unit test: `main.js` calls `require('electron')` at load time and cannot be loaded by `node --test`. The logic it depends on is already covered by Task 1; this step is pure wiring, verified by lint and by the manual run in Task 5. + +- [ ] **Step 1: Add the import** + +In `src/main/main.js`, immediately after the existing line +`const { PROVIDERS } = require('./config');` (line 23), add: + +```js +const { FEATURE_SWITCH, buildLoopbackFeatures } = require('./audioLoopback'); +``` + +- [ ] **Step 2: Add the flag-application function and call it at module scope** + +Directly beneath the import added in Step 1 — and above the existing +`// 按当前 Provider(config.js 注册表)解析出:...` comment — insert: + +```js +// macOS/Linux 上系统声音采集需要 Chromium feature flag,且必须在 app ready 之前设置, +// 所以这里在模块顶层立即执行,不能挪进 whenReady()。 +function applyLoopbackFeatureFlags() { + const merged = buildLoopbackFeatures({ + platform: process.platform, + existing: app.commandLine.getSwitchValue(FEATURE_SWITCH), + }); + if (!merged) return; + if (app.commandLine.hasSwitch(FEATURE_SWITCH)) app.commandLine.removeSwitch(FEATURE_SWITCH); + app.commandLine.appendSwitch(FEATURE_SWITCH, merged); +} + +applyLoopbackFeatureFlags(); +``` + +- [ ] **Step 3: Verify the call sits before app ready** + +Run: `grep -n "applyLoopbackFeatureFlags();\|app.whenReady()" src/main/main.js` +Expected: two lines, with the `applyLoopbackFeatureFlags();` line number **smaller** than the `app.whenReady()` line number. If it is not, move the call up — flags set after ready are ignored by Chromium and the fix silently does nothing. + +- [ ] **Step 4: Run tests and lint** + +Run: `npm test && npm run lint` +Expected: PASS — no test touches `main.js`; lint must report no unused-variable or undefined-variable errors for the new import. + +- [ ] **Step 5: Confirm the app still boots** + +Run: `npm start` +Expected: the window opens with no error in the terminal. Close it. (System audio is not verified here — the dev build launched from a terminal cannot hold the Screen Recording grant; that is Task 5's job with a packaged app.) + +- [ ] **Step 6: Commit** + +```bash +git add src/main/main.js +git commit -m "fix: enable Chromium loopback feature flags for macOS system audio" +``` + +--- + +### Task 3: Echo cancellation on the mic, and fail loudly on a silent system stream + +**Files:** +- Modify: `src/renderer/app.js:305-313` (`getMicStream`) +- Modify: `src/renderer/app.js:332-341` (`getSystemStream`) + +**Interfaces:** +- Consumes: nothing from earlier tasks at the code level; it relies on Task 2 having made a real audio track available, otherwise the new guard will (correctly) throw on every macOS run. +- Produces: `getSystemStream(value)` now rejects with an `Error` when the loopback stream carries no audio track. The existing `catch` in `startListening` (`app.js:395-398`) and `handleSystemCaptureError` (`app.js:314-330`) already handle a thrown error — no new call site is added. + +This task has no unit test: the repo has no DOM or renderer test harness (`test/` covers `src/main/` only), and adding one is out of scope per the spec. Verified manually in Task 5. + +- [ ] **Step 1: Enable echo cancellation on the microphone** + +In `src/renderer/app.js`, in `getMicStream`, change the constraint at line 310 +from `echoCancellation: false` to: + +```js + // 开启回声消除:否则外放时麦克风会听到对方的声音,并被标成 You。 + echoCancellation: true, +``` + +Leave `noiseSuppression: true` and the `deviceId` spread exactly as they are. + +- [ ] **Step 2: Drop the video track properly and reject a stream with no audio** + +In `src/renderer/app.js`, replace the whole `getSystemStream` function +(lines 332–341) with: + +```js +async function getSystemStream(value) { + if (value === '__loopback__') { + // 通过主进程的 displayMediaRequestHandler 抓取系统声音。 + // 必须请求 video:true,否则 loopback 直接失败——拿到之后立刻丢弃视频轨。 + const stream = await navigator.mediaDevices.getDisplayMedia({ video: true, audio: true }); + stream.getVideoTracks().forEach((t) => { + t.stop(); + stream.removeTrack(t); + }); + // 没拿到音频轨说明 loopback 没生效(缺 feature flag / 缺屏幕录制权限)。 + // 这里必须抛错,否则会静默地只跑麦克风,把面试官的话标成 You。 + if (!stream.getAudioTracks().length) { + stream.getTracks().forEach((t) => t.stop()); + throw new Error('system audio returned no audio track'); + } + return stream; + } + return navigator.mediaDevices.getUserMedia({ audio: { deviceId: { exact: value } } }); +} +``` + +- [ ] **Step 3: Run tests, lint and format check** + +Run: `npm test && npm run lint && npm run format:check` +Expected: PASS on all three. If `format:check` complains, run `npm run format` and re-check. + +- [ ] **Step 4: Commit** + +```bash +git add src/renderer/app.js +git commit -m "fix: cancel mic echo and reject a system stream with no audio" +``` + +--- + +### Task 4: Documentation + +**Files:** +- Modify: `README.md:110` (the macOS row of the Platform support table) +- Modify: `CHANGELOG.md` (the `## [Unreleased]` / `### Fixed` section) + +**Interfaces:** +- Consumes: the behaviour delivered by Tasks 1–3. +- Produces: nothing code-facing. + +- [ ] **Step 1: Note the permission's current macOS name in the README table** + +In `README.md`, replace line 110: + +```markdown +| macOS 13+ | `make run` (builds & opens the app) or a `.dmg` | ✅ | ✅ Screen-Capture loopback — grant Screen Recording | +``` + +with: + +```markdown +| macOS 13+ | `make run` (builds & opens the app) or a `.dmg` | ✅ | ✅ ScreenCaptureKit loopback (reliable on 13.2+) — grant Screen Recording, shown as "Screen & System Audio Recording" on recent macOS. The screen-recording indicator appears even though only audio is captured. | +``` + +The platform column stays `macOS 13+` so the table keeps agreeing with +`README.md:59` and the Requirements section; the 13.2 floor is a loopback +reliability detail and belongs in the note. + +- [ ] **Step 2: Add the changelog entry** + +In `CHANGELOG.md`, under `## [Unreleased]` → `### Fixed`, add as the first bullet: + +```markdown +- **System audio is now actually captured on macOS.** The loopback path needs two Chromium feature flags (`MacLoopbackAudioForScreenShare`, `MacSckSystemAudioLoopbackOverride`) that were never set, so `getDisplayMedia` returned a video-only stream: on headphones the interviewer was never transcribed at all, and on speakers their speech leaked into the microphone and was labelled **You**. The flags are now applied before app startup, the microphone runs with echo cancellation so speaker output no longer bleeds into the candidate channel, and a system stream that arrives without an audio track now raises a visible error instead of silently degrading to mic-only. +``` + +- [ ] **Step 3: Verify the docs are formatted** + +Run: `npm run format:check` +Expected: PASS. If it fails, run `npm run format` and re-check. + +- [ ] **Step 4: Commit** + +```bash +git add README.md CHANGELOG.md +git commit -m "docs: describe the macOS system-audio loopback fix" +``` + +--- + +### Task 5: Manual verification on the packaged app + +**Files:** none — verification only. + +**Interfaces:** +- Consumes: the complete change from Tasks 1–4. +- Produces: a go/no-go on the fix. Nothing downstream depends on it. + +This is the only step that can prove the fix, because the flags, the TCC permission and the ScreenCaptureKit path all exist outside the reach of `node --test`. + +- [ ] **Step 1: Build and open the signed app** + +Run: `make app` +Expected: builds into `dist/` and opens `Real Time Interview Copilot.app`. + +- [ ] **Step 2: Re-grant Screen Recording** + +Open System Settings → Privacy & Security → Screen & System Audio Recording, and +enable **Real Time Interview Copilot**. The app is ad-hoc signed, so every +rebuild changes its signature and macOS treats it as a new binary — an existing +grant from a previous build does not carry over. Remove any stale entry, add the +fresh one, and restart the app when prompted. + +- [ ] **Step 3: Verify with headphones** + +Start a Teams call with audio routed to headphones, click **Start Listening**, +and have the other party speak. +Expected: their speech appears under **Interviewer**. Nothing appears under +**You** while they are talking. (Before this change there was no `Interviewer` +transcript at all in this setup.) + +- [ ] **Step 4: Verify with the built-in speakers** + +Switch output to the MacBook speakers and repeat. +Expected: their speech still appears under **Interviewer**, and — this is the +part echo cancellation is responsible for — it does **not** also appear as a +duplicate **You** line. + +- [ ] **Step 5: Verify the failure is now visible** + +Revoke the Screen Recording permission, restart the app, and click +**Start Listening**. +Expected: a toast appears (either the "needs Screen Recording permission" one or +"Couldn't capture system audio: … — continuing with mic only") instead of the +app looking healthy while mislabelling everything. + +- [ ] **Step 6: Re-grant the permission** + +Re-enable Screen Recording so the app is left in a working state. + +- [ ] **Step 7: Commit nothing, report results** + +No commit. Report which of Steps 3–5 passed. If Step 3 fails — no `Interviewer` +transcript on headphones — the flags did not take effect: re-check Task 2 Step 3 +(the call must precede `app.whenReady()`), and confirm the running binary is the +freshly built one rather than a stale copy in `dist/`. diff --git a/docs/superpowers/specs/2026-09-08-macos-system-audio-loopback-design.md b/docs/superpowers/specs/2026-09-08-macos-system-audio-loopback-design.md new file mode 100644 index 0000000..578d578 --- /dev/null +++ b/docs/superpowers/specs/2026-09-08-macos-system-audio-loopback-design.md @@ -0,0 +1,177 @@ +# macOS System-Audio Loopback — Design + +**Date:** 2026-09-08 +**Status:** Approved (brainstormed in-session) + +## Goal + +Make the **Interviewer** channel actually capture system audio on macOS. Today +it captures nothing: the interviewer's speech either disappears entirely (when +the user wears headphones) or leaks into the microphone and is transcribed +under the **You** label (when the user listens on the built-in speakers). + +## Scope + +**In scope:** the audio-capture path — the Chromium feature flags that unlock +loopback capture, the microphone constraints that let speaker output be +mistaken for the candidate, the renderer's silent-failure behaviour when the +system stream carries no audio, and the docs that describe platform support. + +**Out of scope** (decided with the user): + +- Upgrading Electron to 39+, where loopback is supported natively without + flags. A major-version bump of the runtime is a separate change with its own + risk surface; the flags work on the pinned Electron 33. +- Bundling `electron-audio-loopback` as a dependency. This repo needs only the + flag list from it, not its IPC layer — the existing + `setupDisplayMediaLoopback()` handler is already shape-identical to the + library's. +- The Core Audio Taps path (`MacCatapSystemAudioLoopbackCapture`, or + AudioTee.js). It captures pre-mixer audio and needs only the narrower + "System Audio Recording Only" permission, but it is the newer and less + proven of the two Chromium paths. +- Any UI work: a dedicated per-channel capture indicator, a device picker + redesign, or surfacing the loopback state in the status bar. +- Speaker-diarization or cross-channel de-duplication to catch mislabelled + speech after the fact. Fixing the capture removes the cause. + +## Current state + +- `src/main/main.js:169-188` — `setupDisplayMediaLoopback()` registers + `session.defaultSession.setDisplayMediaRequestHandler` and answers with + `callback({ video: sources[0], audio: 'loopback' })`. This is the correct + shape. +- `src/main/main.js` has **no `app.commandLine.appendSwitch` call anywhere**. + This is the defect. The installed Electron's own typings + (`node_modules/electron/electron.d.ts:21334-21339`) state: *"Specifying a + loopback device will capture system audio, and is currently only supported on + Windows."* On macOS the handler therefore resolves with a video-only stream + and the audio track is silently absent. +- `src/renderer/app.js:333-339` — `getSystemStream()` requests + `getDisplayMedia({ video: true, audio: true })`, stops the video tracks, and + returns the stream **without checking that an audio track survived**. A + video-only stream is treated as success. +- `src/renderer/app.js:305-313` — `getMicStream()` sets + `echoCancellation: false` explicitly, so the microphone transcribes whatever + the speakers play. +- `src/renderer/app.js:406` / `:417` — the microphone stream is hardwired to + the `interviewee` role (rendered as **You**, `app.js:105`) and the system + stream to `interviewer`. The roles are per-stream, so any speech reaching the + mic is attributed to the candidate by construction. +- `src/renderer/app.js:388-398` — when system capture throws, `startListening` + toasts and continues mic-only. Correct as a fallback, but unreachable in the + video-only case above, so the user gets no signal at all. +- `extend-info.plist` already declares `NSScreenCaptureUsageDescription`, and + `main.js:387` reads `systemPreferences.getMediaAccessStatus('screen')`. The + permission plumbing is in place and needs no change. +- `README.md:110` claims macOS system audio works via "Screen-Capture + loopback", and `README.md:59` calls macOS the most seamless platform. Both + describe behaviour the code does not currently deliver. + +### Why both reported symptoms follow from one cause + +| User setup | Observed | Explanation | +|---|---|---| +| Headphones | No `Interviewer` transcript at all | Loopback yields no audio; the mic cannot hear headphones. | +| Built-in speakers | Interviewer's speech appears as `You` | Loopback yields no audio; the mic hears the speakers, and the mic channel is labelled `You`. | + +## Design + +### 1. Enable the Chromium loopback feature flags + +Loopback capture on macOS 13.2+ is gated behind two Chromium features that +must be enabled on the command line **before app ready**: +`MacLoopbackAudioForScreenShare` (the general screen-share loopback path) and +`MacSckSystemAudioLoopbackOverride` (routes it through ScreenCaptureKit). Linux +has an equivalent, `PulseaudioLoopbackForScreenShare`. Windows needs none — its +support is native, which is exactly what the Electron typings describe. Flag +names and the darwin/linux split follow +[`alectrocute/electron-audio-loopback`](https://github.com/alectrocute/electron-audio-loopback/blob/main/src/config.ts), +the reference implementation for Electron 31.0.1–38.x. + +A new dependency-free module `src/main/audioLoopback.js` owns the flag list and +exports a pure `buildLoopbackFeatures({ platform, existing })`. Keeping the +logic out of `main.js` follows this repo's existing pattern (`prompt.js`, +`config.js`) and makes it reachable from `node --test`, which cannot boot +Electron. It returns `null` for platforms needing no flags, so the caller has a +single unambiguous "nothing to do" signal. + +The function merges rather than overwrites: it preserves any `enable-features` +value already present on the command line (a user's `--enable-features=…`, or +another switch set elsewhere) and appends only the required flags that are +missing. `main.js` calls it at module scope — before `app.whenReady()` — reads +the current value with `app.commandLine.getSwitchValue`, removes the switch if +present, and re-appends the merged value. + +### 2. Enable echo cancellation on the microphone + +`echoCancellation` flips from `false` to `true`. Without this, the fix above is +only half a fix: with loopback working and the meeting playing through the +built-in speakers, the interviewer's voice still reaches the microphone and +still renders as **You**, alongside a now-correct `Interviewer` line. Echo +cancellation references the system output and suppresses exactly that path. + +`noiseSuppression: true` stays as it is. + +### 3. Fail loudly instead of returning a silent stream + +`getSystemStream()` gains a post-condition: after the video tracks are stopped +they are also removed from the stream (the reference implementation does both; +a stopped-but-attached track is still enumerable), and if no audio track +remains the remaining tracks are stopped and an `Error` is thrown. Throwing — +rather than returning `null` — routes the failure into `startListening`'s +existing `catch` and `handleSystemCaptureError`, which already distinguishes +"Screen Recording not granted" from other causes and already tells the user the +session is continuing mic-only. No new error-handling branch is introduced. + +This turns the current failure mode — a session that looks healthy while +mislabelling every word the interviewer says — into a visible one. + +### 4. Documentation + +`README.md:110`'s macOS row and `README.md:59` are left claiming loopback +works, because after this change they are true; the row gains a note that the +permission is "Screen Recording", which on recent macOS is presented as +"Screen & System Audio Recording". A `CHANGELOG.md` entry goes under +`## [Unreleased]` / `### Fixed`. + +## Testing + +`test/audioLoopback.test.js` (new, `node:test`, matching the existing suites) +covers `buildLoopbackFeatures` as a pure function: darwin produces both macOS +flags; linux produces the PulseAudio flag; win32 and unknown platforms produce +`null`; an existing value is preserved and the required flags appended; a flag +already present is not duplicated; and empty, whitespace-only, and absent +`existing` values are all handled. + +The `main.js` wiring and the renderer changes have no test harness in this repo +(there is no Electron-runtime or DOM test setup, and none is added here), so +they are verified manually: + +1. Rebuild and re-sign the app (`make app`) — the ad-hoc signature changes on + every build, so macOS treats it as a new binary and the Screen Recording + grant must be re-issued under System Settings → Privacy & Security. +2. With **headphones**, start a Teams call and confirm the other party's speech + appears under `Interviewer` and nothing appears under `You` while they talk. +3. With **built-in speakers**, confirm the same, and specifically that the + interviewer's speech no longer produces a duplicate `You` line. +4. Revoke Screen Recording and confirm the app now surfaces a toast and falls + back to mic-only rather than appearing to work. + +## Risks + +- **Echo cancellation may degrade microphone quality.** Chromium's AEC puts the + input into a voice-processing mode with a narrower response. Accepted: the + audio's only consumer is speech-to-text, which AEC is tuned for, and + correctly separated speakers matter more here than fidelity. +- **macOS shows a screen-recording indicator although only audio is captured.** + Inherent to the ScreenCaptureKit path — `getDisplayMedia` must request + `video: true` for loopback to work at all, even though the video track is + discarded immediately. Documented in the README rather than worked around. +- **The flag names are undocumented Chromium internals.** They can be renamed + or removed in a future Chromium, which would silently return the app to + today's behaviour. Mitigated by change 3: the failure becomes visible instead + of silent. The durable fix is Electron 39+, which is out of scope here. +- **Flags are ignored if set after app ready.** Mitigated by calling at module + scope and by keeping the call in a named function directly beneath the + `require` block, where its ordering constraint is stated in a comment. From 683ede1a2f7d321393cb5c58dac62e3d9fd2293d Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 8 Sep 2026 17:36:04 +0300 Subject: [PATCH 15/33] docs: add spec and plan for the Electron 44 upgrade Supersedes the macOS loopback feature-flag design, whose premise was wrong: Electron 33's Chromium contains no macOS system-audio capture at all, and silently ignores the unknown feature names it was passed. Electron 44 captures system audio through Apple's CoreAudio Tap API by default, which needs no flags but does need the NSAudioCaptureUsageDescription Info.plist key on both packaging paths. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HzVW3YgFohygg7bM1Hxxyb --- .../plans/2026-09-08-electron-44-upgrade.md | 450 ++++++++++++++++++ .../2026-09-08-electron-44-upgrade-design.md | 249 ++++++++++ 2 files changed, 699 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-08-electron-44-upgrade.md create mode 100644 docs/superpowers/specs/2026-09-08-electron-44-upgrade-design.md diff --git a/docs/superpowers/plans/2026-09-08-electron-44-upgrade.md b/docs/superpowers/plans/2026-09-08-electron-44-upgrade.md new file mode 100644 index 0000000..7fc3602 --- /dev/null +++ b/docs/superpowers/plans/2026-09-08-electron-44-upgrade.md @@ -0,0 +1,450 @@ +# Electron 44 Upgrade Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Move the app to Electron 44 and declare `NSAudioCaptureUsageDescription`, so macOS captures system audio through Chromium's CoreAudio Tap path and the Interviewer channel finally works. + +**Architecture:** Electron 44's Chromium uses Apple's CoreAudio Tap API by default for desktop audio capture, so no feature flags are involved — the capture code in `main.js` and `app.js` is already correct and stays as it is. What changes is the runtime version, the Info.plist declaration that CoreAudio Tap requires on macOS 14.2+, and the deletion of the inert feature-flag module from the superseded attempt. Because a missing plist key fails silently with no error from Electron, and because the two packaging paths read two different Info.plist sources by hand, a unit test enforces that both sources declare the same keys. + +**Tech Stack:** Electron 44 (plain JS, CommonJS), `@electron/packager` 20 + `electron-builder` 26, macOS CoreAudio Tap via `getDisplayMedia`, `node:test` for unit tests, ESLint + Prettier. + +**Spec:** `docs/superpowers/specs/2026-09-08-electron-44-upgrade-design.md` + +## Global Constraints + +- Node ≥ 22.12.0 for the dev toolchain; tests run with `npm test` (`node --test`), lint with `npm run lint`, format check with `npm run format:check`. +- **No new dependencies.** The Info.plist test parses XML with a regular expression rather than adding a plist library. +- Electron target version is exactly `^44.2.0`. +- The Info.plist key is exactly `NSAudioCaptureUsageDescription`, and it must be added to **both** `extend-info.plist` and `package.json` → `build.mac.extendInfo`, with identical description text. +- Do **not** add any `app.commandLine.appendSwitch` call. Electron 44 needs no feature flags for system audio; the only flag in this area (`MacCatapLoopbackAudioForScreenShare`) opts *out* of the working path. +- Do **not** change `setupDisplayMediaLoopback()` in `src/main/main.js`, the `getDisplayMedia({ video: true, audio: true })` call, or the dual-stream role assignment. They are already correct on Electron 44. +- Keep `echoCancellation: true` (`src/renderer/app.js:310`) and the audio-track guard in `getSystemStream` (`src/renderer/app.js:332-351`). The guard is the app's only failure signal, because `systemPreferences.getMediaAccessStatus()` has no media type for system audio. +- New source files under `src/main/` and `test/` are `'use strict';` CommonJS, like every existing file. +- Commit messages follow the repo's conventional style (`feat:`, `fix:`, `test:`, `docs:`, `chore:`). + +### Starting state — read this before Task 1 + +The working tree contains an **uncommitted, superseded** change (from +`docs/superpowers/plans/2026-09-08-macos-system-audio-loopback.md`). Nothing is +committed. Parts of it are deleted by Task 2 and parts are deliberately kept: + +| Artifact | Fate | +|---|---| +| `src/main/audioLoopback.js` | **Deleted** (Task 2) — inert on 33, unnecessary on 44 | +| `test/audioLoopback.test.js` | **Deleted** (Task 2) | +| `applyLoopbackFeatureFlags()` + import, `src/main/main.js:24-38` | **Reverted** (Task 2) | +| `echoCancellation: true`, `src/renderer/app.js:310` | **Kept** | +| Audio-track guard in `getSystemStream`, `src/renderer/app.js:332-351` | **Kept** | +| README macOS row + CHANGELOG bullet describing feature flags | **Rewritten** (Task 4) — they describe a fix that does not work | + +--- + +### Task 1: Enforce and add the system-audio Info.plist key + +**Files:** +- Create: `test/packaging.test.js` +- Modify: `extend-info.plist` +- Modify: `package.json` (the `build.mac.extendInfo` object) + +**Interfaces:** +- Consumes: nothing from other tasks. +- Produces: the invariant that `extend-info.plist` and `package.json` → `build.mac.extendInfo` declare an identical set of `NS*UsageDescription` keys including `NSAudioCaptureUsageDescription`. Task 5's manual verification depends on this key being present in the packaged app. + +This task comes first because it is the only part of the change with a testable +property, and because it is independent of the Electron version — the key is +required regardless of when the runtime bump lands. + +- [ ] **Step 1: Write the failing test** + +Create `test/packaging.test.js`: + +```js +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert'); +const fs = require('fs'); +const path = require('path'); + +const ROOT = path.join(__dirname, '..'); + +// extend-info.plist feeds @electron/packager (`make app`); build.mac.extendInfo +// feeds electron-builder (`npm run dist:mac`). They are maintained by hand and +// must not drift: a key present in only one produces an app that captures audio +// when built one way and silently fails when built the other. +function plistUsageKeys() { + const xml = fs.readFileSync(path.join(ROOT, 'extend-info.plist'), 'utf8'); + return [...xml.matchAll(/(NS\w*UsageDescription)<\/key>/g)].map((m) => m[1]).sort(); +} + +function builderUsageKeys() { + const extendInfo = require(path.join(ROOT, 'package.json')).build.mac.extendInfo; + return Object.keys(extendInfo) + .filter((k) => /^NS\w*UsageDescription$/.test(k)) + .sort(); +} + +test('extend-info.plist declares the system-audio usage description', () => { + assert.ok( + plistUsageKeys().includes('NSAudioCaptureUsageDescription'), + 'CoreAudio Tap capture fails silently on macOS 14.2+ without this key', + ); +}); + +test('electron-builder config declares the system-audio usage description', () => { + assert.ok( + builderUsageKeys().includes('NSAudioCaptureUsageDescription'), + 'CoreAudio Tap capture fails silently on macOS 14.2+ without this key', + ); +}); + +test('both packaging paths declare the same usage-description keys', () => { + assert.deepStrictEqual(plistUsageKeys(), builderUsageKeys()); +}); + +test('every usage description is a non-empty string', () => { + const extendInfo = require(path.join(ROOT, 'package.json')).build.mac.extendInfo; + for (const [key, value] of Object.entries(extendInfo)) { + if (!/^NS\w*UsageDescription$/.test(key)) continue; + assert.strictEqual(typeof value, 'string', `${key} must be a string`); + assert.ok(value.trim().length > 0, `${key} must not be empty — macOS shows it in the prompt`); + } +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `node --test test/packaging.test.js` +Expected: FAIL with **exactly 2 of the 4 tests failing** — the two `NSAudioCaptureUsageDescription` assertions, because neither source declares the key yet. The `deepStrictEqual` sync test passes at this point (both sources hold the same two keys) and so does the non-empty-string test. If you instead see 3 failures, the two sources have already drifted; read both files and reconcile them before continuing. + +- [ ] **Step 3: Add the key to `extend-info.plist`** + +In `extend-info.plist`, insert this pair immediately after the closing +`` of `NSMicrophoneUsageDescription` and before the +`NSScreenCaptureUsageDescription` line: + +```xml + NSAudioCaptureUsageDescription + Real Time Interview Copilot captures your computer's audio (the interviewer) so it can be transcribed. +``` + +- [ ] **Step 4: Add the same key to the electron-builder config** + +In `package.json`, inside `build.mac.extendInfo`, add the key with **identical** +text so the sync test passes: + +```json + "NSAudioCaptureUsageDescription": "Real Time Interview Copilot captures your computer's audio (the interviewer) so it can be transcribed.", +``` + +Place it before the existing `"NSMicrophoneUsageDescription"` entry so the object +reads alphabetically. + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `node --test test/packaging.test.js` +Expected: PASS — 4 tests, 0 failures. + +- [ ] **Step 6: Run the full suite and lint** + +Run: `npm test && npm run lint && npm run format:check` +Expected: PASS. Note the total test count — it still includes the 10 `audioLoopback` tests that Task 2 removes. + +- [ ] **Step 7: Commit** + +```bash +git add test/packaging.test.js extend-info.plist package.json +git commit -m "feat: declare NSAudioCaptureUsageDescription on both packaging paths" +``` + +--- + +### Task 2: Delete the inert feature-flag module + +**Files:** +- Delete: `src/main/audioLoopback.js` +- Delete: `test/audioLoopback.test.js` +- Modify: `src/main/main.js:24-38` (remove the import, the function, and its call) + +**Interfaces:** +- Consumes: nothing. +- Produces: a `src/main/main.js` with no `app.commandLine` usage at all. Task 3's Electron bump relies on there being no feature flags to re-validate. + +- [ ] **Step 1: Delete the module and its test** + +```bash +rm src/main/audioLoopback.js test/audioLoopback.test.js +``` + +- [ ] **Step 2: Remove the wiring from `main.js`** + +In `src/main/main.js`, delete these lines in full — the import, the comment +block, the function, and the bare call (currently lines 24–38): + +```js +const { FEATURE_SWITCH, buildLoopbackFeatures } = require('./audioLoopback'); + +// macOS/Linux 上系统声音采集需要 Chromium feature flag,且必须在 app ready 之前设置, +// 所以这里在模块顶层立即执行,不能挪进 whenReady()。 +function applyLoopbackFeatureFlags() { + const merged = buildLoopbackFeatures({ + platform: process.platform, + existing: app.commandLine.getSwitchValue(FEATURE_SWITCH), + }); + if (!merged) return; + if (app.commandLine.hasSwitch(FEATURE_SWITCH)) app.commandLine.removeSwitch(FEATURE_SWITCH); + app.commandLine.appendSwitch(FEATURE_SWITCH, merged); +} + +applyLoopbackFeatureFlags(); +``` + +Leave the `const { PROVIDERS } = require('./config');` line above it and the +`// 按当前 Provider(config.js 注册表)解析出:...` comment below it untouched. + +- [ ] **Step 3: Verify no trace of the module remains** + +Run: `grep -rn "audioLoopback\|applyLoopbackFeatureFlags\|FEATURE_SWITCH\|commandLine" src/ test/` +Expected: **no output at all.** Any hit means the removal is incomplete. + +- [ ] **Step 4: Verify `main.js` still parses** + +Run: `node --check src/main/main.js` +Expected: no output, exit 0. + +- [ ] **Step 5: Run tests and lint** + +Run: `npm test && npm run lint && npm run format:check` +Expected: PASS, with the total 10 lower than in Task 1 Step 6 — the `audioLoopback` suite is gone. + +- [ ] **Step 6: Commit** + +```bash +git add -A src/main test +git commit -m "fix: drop the inert loopback feature-flag module" +``` + +--- + +### Task 3: Bump Electron to 44 and clear the stale runtime cache + +**Files:** +- Modify: `package.json` (`devDependencies.electron`, `engines.node`) +- Modify: `.nvmrc` +- Delete: `.electron-cache/` (gitignored download cache) + +**Interfaces:** +- Consumes: the flag-free `src/main/main.js` from Task 2. +- Produces: an installed Electron 44 runtime whose Chromium uses CoreAudio Tap by default. Task 5's manual verification depends on it. + +- [ ] **Step 1: Clear the stale Electron zip cache** + +The Makefile passes the first subdirectory of `.electron-cache/` to the packager +as `--electron-zip-dir`, and that directory currently holds only +`electron-v33.4.11-darwin-arm64.zip`. Leaving it there points the packager at the +old runtime — the exact way to "upgrade" and observe no change. + +```bash +rm -rf .electron-cache dist +``` + +- [ ] **Step 2: Raise the Node floor to what the toolchain already requires** + +`@electron/packager@^20` declares `"node": ">= 22.12.0"`, so the current `.nvmrc` +of `20` cannot package the app at all. In `package.json`, change: + +```json + "engines": { + "node": ">=22.12.0" + }, +``` + +And write `.nvmrc` as a single line: + +``` +22 +``` + +- [ ] **Step 3: Bump the Electron dependency** + +In `package.json`, under `devDependencies`, change the `electron` entry to: + +```json + "electron": "^44.2.0", +``` + +- [ ] **Step 4: Install** + +Run: `npm install` +Expected: completes without an `EBADENGINE` error, and downloads the Electron 44 binary (this takes a while — it is a fresh ~100 MB download because Step 1 cleared the cache). + +- [ ] **Step 5: Verify the installed runtime is actually 44** + +Run: `node -e "console.log(require('./node_modules/electron/package.json').version)"` +Expected: a `44.x.y` version string. Anything starting with `33.` means the install did not take — re-check Step 3 and re-run `npm install`. + +- [ ] **Step 6: Verify the new Chromium has the system-audio capability** + +Run: + +```bash +strings -a node_modules/electron/dist/Electron.app/Contents/Frameworks/Electron\ Framework.framework/Versions/Current/Electron\ Framework | grep -c "MacCatapSystemAudioLoopbackCapture" +``` + +Expected: a count of **1 or more**. This is the check that proves the upgrade delivered what Electron 33 lacked — the same search returned zero on 33. A count of `0` means this Electron still cannot capture system audio and you must STOP and report it rather than proceeding to build. + +- [ ] **Step 7: Run tests and lint** + +Run: `npm test && npm run lint && npm run format:check` +Expected: PASS. The unit suites are pure modules and are unaffected by the runtime version. + +- [ ] **Step 8: Verify the app boots on the new runtime** + +Run: `node --check src/main/main.js` +Expected: no output, exit 0. + +Do **not** run `npm start` here — it opens a GUI window that will not close on its own in an automated session, and a terminal-launched dev build cannot hold the Screen Recording grant anyway. The real launch is Task 5. + +- [ ] **Step 9: Commit** + +```bash +git add package.json package-lock.json .nvmrc +git commit -m "chore: upgrade Electron 33 -> 44 for macOS CoreAudio Tap audio capture" +``` + +--- + +### Task 4: Documentation + +**Files:** +- Modify: `README.md:59` (Requirements line) +- Modify: `README.md:110` (macOS row of the Platform support table) +- Modify: `CHANGELOG.md` (replace the superseded `### Fixed` bullet) + +**Interfaces:** +- Consumes: the behaviour delivered by Tasks 1–3. +- Produces: nothing code-facing. + +- [ ] **Step 1: State the two different macOS floors in Requirements** + +In `README.md`, replace line 59: + +```markdown +- **macOS 13+**, **Windows**, or **Linux** — runs on all three; system-audio capture is most seamless on macOS (see [Platform support](#platform-support)). +``` + +with: + +```markdown +- **macOS 13+**, **Windows**, or **Linux** — runs on all three. Capturing the interviewer's audio needs **macOS 14.2+**; on macOS 13–14.1 use the [virtual-device fallback](#platform-support). +``` + +- [ ] **Step 2: Rewrite the macOS row of the Platform support table** + +In `README.md`, replace line 110 — which currently describes the ScreenCaptureKit +loopback path that this change replaces: + +```markdown +| macOS 13+ | `make run` (builds & opens the app) or a `.dmg` | ✅ | ✅ ScreenCaptureKit loopback (reliable on 13.2+) — grant Screen Recording, shown as "Screen & System Audio Recording" on recent macOS. The screen-recording indicator appears even though only audio is captured. | +``` + +with: + +```markdown +| macOS 13+ | `make run` (builds & opens the app) or a `.dmg` | ✅ | ✅ on **macOS 14.2+** via CoreAudio Tap — grant **Screen & System Audio Recording**, and allow system-audio recording when macOS asks. macOS shows a screen-recording indicator even though only audio is captured. On macOS 13–14.1 use the virtual-device fallback below. | +``` + +- [ ] **Step 3: Replace the superseded changelog bullet** + +In `CHANGELOG.md`, under `## [Unreleased]` → `### Fixed`, replace the bullet that +begins `- **System audio is now actually captured on macOS.** The loopback path +needs two Chromium feature flags` — it credits a fix that provably does not work +— with: + +```markdown +- **System audio is now actually captured on macOS.** The Chromium shipped in Electron 33 had no macOS system-audio capture at all, so `getDisplayMedia` handed back a stream with no usable audio: on headphones the interviewer was never transcribed, and on speakers their speech reached only the microphone and was labelled **You**. The app now runs on Electron 44, whose Chromium captures system audio through Apple's CoreAudio Tap API (macOS 14.2+), and declares the `NSAudioCaptureUsageDescription` permission that API requires. The microphone also runs with echo cancellation so speaker output no longer bleeds into the candidate channel, and a system stream that arrives without an audio track now raises a visible error instead of silently degrading to mic-only. +``` + +- [ ] **Step 4: Verify formatting** + +Run: `npm run format:check` +Expected: PASS. If it fails, run `npm run format` and re-check. + +- [ ] **Step 5: Commit** + +```bash +git add README.md CHANGELOG.md +git commit -m "docs: describe the Electron 44 system-audio upgrade" +``` + +--- + +### Task 5: Manual verification on the packaged app + +**Files:** none — verification only. + +**Interfaces:** +- Consumes: the complete change from Tasks 1–4. +- Produces: a go/no-go. Nothing downstream depends on it. + +This is the only step that can close the bug. It needs a human at the machine: a +real TCC grant, a real meeting, and a GUI. + +- [ ] **Step 1: Build and open the signed app** + +Run: `make app` +Expected: packages into `dist/Real Time Interview Copilot-darwin-/` and opens the app. + +- [ ] **Step 2: Replace the installed copy** + +macOS attaches permission grants to a specific path, and +`/Applications/Real Time Interview Copilot.app` is what actually launches on this +machine. Quit the running app, then replace that copy with the freshly built one +from `dist/` and launch it from `/Applications`. Verifying the `dist/` copy while +the stale `/Applications` copy is the one being granted permissions is how the +previous round produced a confusing result. + +- [ ] **Step 3: Grant both permissions** + +In System Settings → Privacy & Security: +- **Screen & System Audio Recording** — remove any stale + `Real Time Interview Copilot` entry and add the new build. The ad-hoc signature + changes on every build, so macOS treats each build as a new binary and prior + grants do not carry over. +- When macOS prompts to allow recording system audio, allow it. This prompt is + new — it is the `NSAudioCaptureUsageDescription` key from Task 1 taking effect, + and its absence is what made the failure silent before. + +Restart the app after granting. + +- [ ] **Step 4: Verify with headphones** + +Start a Teams call with audio in headphones, click **Start Listening**, and have +the other party speak. +Expected: their speech appears under **Interviewer**, and nothing appears under +**You** while they talk. Before this change there was no `Interviewer` transcript +at all in this setup. + +- [ ] **Step 5: Verify with the built-in speakers** + +Switch output to the MacBook speakers and repeat. +Expected: their speech still appears under **Interviewer**, and does **not** also +appear as a duplicate **You** line — that half is echo cancellation's job. + +- [ ] **Step 6: Verify the failure is visible** + +Revoke the Screen & System Audio Recording permission, restart the app, and click +**Start Listening**. +Expected: a toast appears — either the "needs Screen Recording permission" one or +"Couldn't capture system audio: … — continuing with mic only" — rather than a +session that looks healthy while mislabelling everything. + +- [ ] **Step 7: Re-grant and report** + +Re-enable the permission so the app is left working. No commit. + +Report which of Steps 4–6 passed. If Step 4 fails — still no `Interviewer` +transcript on headphones — do **not** start adding feature flags. Report the +result with the output of Task 3 Step 6 and confirmation that the app launched +from `/Applications` is the new build (check its Info.plist contains +`NSAudioCaptureUsageDescription`). diff --git a/docs/superpowers/specs/2026-09-08-electron-44-upgrade-design.md b/docs/superpowers/specs/2026-09-08-electron-44-upgrade-design.md new file mode 100644 index 0000000..a9d4288 --- /dev/null +++ b/docs/superpowers/specs/2026-09-08-electron-44-upgrade-design.md @@ -0,0 +1,249 @@ +# Electron 44 Upgrade — Design + +**Date:** 2026-09-08 +**Status:** Approved (brainstormed in-session) +**Supersedes:** `2026-09-08-macos-system-audio-loopback-design.md`, whose central +premise was wrong. See *Why the previous attempt failed*. + +## Goal + +Make the **Interviewer** channel capture system audio on macOS by moving to an +Electron whose Chromium actually implements macOS system-audio capture, and by +declaring the Info.plist key that capture requires. + +## Why the previous attempt failed + +The previous design assumed Electron 33 could be made to capture system audio by +enabling two Chromium feature flags. It cannot. Searching the shipped binary +`Electron Framework.framework/Versions/Current/Electron Framework` (155 MB, +Electron 33.4.11) for the feature names finds nothing: + +``` +MacCatapSystemAudioLoopbackCapture ABSENT +MacSckSystemAudioLoopbackOverride ABSENT +MacLoopbackAudioForScreenShare ABSENT +``` + +The method is sound — known features of the same binary (`MacWebContentsOcclusion`, +`ScreenCaptureKitStreamPickerSonoma`, `SpareRendererForSitePerProcess`) are all +found. No string containing `SystemAudio` exists in it at all. + +Chromium **silently ignores unknown feature names** passed to `--enable-features`. +That is why the flags appear verbatim in the running process's arguments while +changing nothing: + +``` +--enable-features=MacLoopbackAudioForScreenShare,MacSckSystemAudioLoopbackOverride, + ScreenCaptureKitPickerScreen,ScreenCaptureKitStreamPickerSonoma +``` + +Electron 33's own typings state the position plainly +(`node_modules/electron/electron.d.ts:21334-21339`): loopback audio is +*"currently only supported on Windows."* The community library that supplied the +flag names targets a version range in which later Chromium builds gained them; +its range is not evidence about this binary. + +## Scope + +**In scope:** the Electron dependency version, the Info.plist declarations that +system-audio capture requires on both packaging paths, removal of the dead +feature-flag code from the superseded attempt, the Node floor the current +toolchain already needs, and the docs describing all of it. + +**Out of scope** (decided with the user): + +- Any change to the renderer's capture flow beyond deleting nothing from it. The + `getDisplayMedia({ video: true, audio: true })` call, the + `setDisplayMediaRequestHandler` answering `{ video: sources[0], audio: 'loopback' }`, + and the dual-stream role assignment are all still correct on Electron 44 and + stay untouched. +- Opting back into the older ScreenCaptureKit path via + `disable-features=MacCatapLoopbackAudioForScreenShare`. CoreAudio Tap is the + default and the recommended path on this user's macOS. +- Removing the BlackHole / virtual-device fallback from the README. It remains + the answer for macOS 13–14.1, where CoreAudio Tap is unavailable. +- Notarization, a paid Developer ID, or CI release-matrix changes. +- Windows and Linux capture behaviour, which this change does not alter. + +## Current state + +- `package.json` — `"electron": "^33.2.0"`; installed 33.4.11. Latest stable + Electron is **44.2.0**. +- The working tree carries the **uncommitted** superseded change: new + `src/main/audioLoopback.js` and `test/audioLoopback.test.js`, the + `applyLoopbackFeatureFlags()` wiring at `src/main/main.js:24-38`, the renderer + edits, and README/CHANGELOG text describing the flags. Nothing is committed. +- `src/renderer/app.js:310` — `echoCancellation: true` (from the superseded + change). +- `src/renderer/app.js:332-351` — `getSystemStream()` throws when no audio track + survives (from the superseded change). +- **Two independent packaging paths, each with its own Info.plist source:** + `extend-info.plist`, passed by the Makefile via + `--extend-info=extend-info.plist` to `@electron/packager` (`make app`, + `make universal`, `make run`); and `package.json` → `build.mac.extendInfo`, + used by `electron-builder` (`npm run dist:mac`). Both currently declare + `NSMicrophoneUsageDescription` and `NSScreenCaptureUsageDescription`, and + **neither declares `NSAudioCaptureUsageDescription`**. They are maintained by + hand and can drift apart unnoticed. +- `Makefile:17-18` — `ZIP_DIR := $(firstword $(wildcard .electron-cache/*/))`, + passed as `--electron-zip-dir`. `.electron-cache/` currently holds exactly one + zip: `electron-v33.4.11-darwin-arm64.zip`. +- `.nvmrc` says `20` and `package.json` `engines.node` says `>=18`, but the + already-installed `@electron/packager@^20` declares `"node": ">= 22.12.0"`. + The floor is stale today, independently of this change. +- `systemPreferences.getMediaAccessStatus()` accepts only `microphone`, `camera` + and `screen` in Electron 44 — there is **no** media type for system-audio + capture, so the audio permission's state cannot be queried from code. +- `src/main/main.js:169-188` — `setupDisplayMediaLoopback()`. Unchanged by this + design; Electron 44 introduces no breaking change to + `setDisplayMediaRequestHandler`, `desktopCapturer.getSources`, or + `systemPreferences`. + +## Design + +### 1. Electron `^33.2.0` → `^44.2.0` + +As of Electron v39.0.0-beta.4, Chromium made Apple's **CoreAudio Tap API** the +default for desktop audio capture on macOS. Electron 44 is therefore the first +*stable* major line at or above that floor that is also current, and it needs +**no feature flags at all** — the flag that exists in this area +(`MacCatapLoopbackAudioForScreenShare`) is for opting *out* of the new path, and +is documented as removed upstream and inert as of Electron 45. + +Breaking changes across 34–44 that touch this app's surface are limited to: +Electron 44 requires **macOS 13+** (the README already claims macOS 13+, and the +user is on 26.6), and Electron 36 lowercases switches passed through +`app.commandLine`. The latter is a second, independent reason the superseded +flag code could never have worked as written, and it disappears with change 2. +Nothing else the app uses — `app`, `BrowserWindow`, `ipcMain`/`ipcRenderer`/ +`contextBridge` under `contextIsolation: true`, `dialog`, `globalShortcut`, +`session.setDisplayMediaRequestHandler`, `desktopCapturer.getSources`, +`systemPreferences`, `shell`, `BrowserWindow.loadFile` — has a breaking change in +that range. + +`.electron-cache/` must be cleared as part of the bump. The Makefile passes the +first subdirectory it finds there as `--electron-zip-dir`, and that directory +holds only the 33.4.11 zip; leaving it in place points the packager at the old +runtime and risks reproducing exactly the "I upgraded and nothing changed" +confusion this whole spec exists to end. The directory is gitignored and is a +pure download cache, so deleting it is free. + +### 2. Delete the feature-flag module + +`src/main/audioLoopback.js`, `test/audioLoopback.test.js`, the import at +`src/main/main.js:24`, and the `applyLoopbackFeatureFlags()` function and its +call at `src/main/main.js:26-38` are removed outright. On Electron 44 the flags +are unnecessary; on Electron 33 they were inert. Keeping code that looks like a +fix but is provably a no-op is worse than not having it. + +The two renderer changes from the superseded attempt are **kept**, and change 3 +explains why the second one becomes load-bearing rather than merely defensive. + +### 3. Declare `NSAudioCaptureUsageDescription` on both packaging paths + +On macOS 14.2 and higher the CoreAudio Tap path requires the +`NSAudioCaptureUsageDescription` Info.plist key for `desktopCapturer` to capture +audio. Electron's own documentation is explicit that when the key is absent, +audio-stream creation fails and **"no warnings or errors are displayed"** — the +same silent failure the user has been living with, arriving by a different route. +There is also no automatic fallback to the older ScreenCaptureKit path if tap +creation fails. + +The key is added to **both** `extend-info.plist` and `package.json` → +`build.mac.extendInfo`, with identical wording, because the two packaging paths +read different files and a key added to only one produces an app that works when +built one way and silently fails when built the other. + +Because that drift is invisible at runtime and Electron reports nothing, a unit +test enforces it: `test/packaging.test.js` parses both sources and asserts they +declare the same set of `NS*UsageDescription` keys, and that the set contains +`NSAudioCaptureUsageDescription`. This is the one property of this change that +*can* be tested automatically, and it guards precisely the failure mode that is +undetectable by hand. + +`extend-info.plist` is parsed without a dependency: the test extracts `` +names with a regular expression rather than adding a plist library, matching this +repo's no-new-dependencies constraint. + +The existing `NSScreenCaptureUsageDescription` stays. `getDisplayMedia` is still +called with `video: true` — required for loopback to be granted at all — and +`desktopCapturer.getSources({ types: ['screen'] })` still enumerates screens, so +Screen Recording remains necessary alongside the new audio permission. + +### 4. The renderer's audio-track guard becomes the only failure signal + +`systemPreferences.getMediaAccessStatus()` has no media type for system audio, so +the app cannot ask macOS whether the audio-capture permission was granted. With +Electron documenting that a failed tap is silent, the guard added in the +superseded change — throw when `getSystemStream()` yields a stream with no audio +track, routing into the existing `handleSystemCaptureError` toast — is the only +thing standing between the user and another silent mislabelling session. It is +kept unchanged and its rationale is recorded here rather than left implicit. + +The existing `getScreenPermission()` gate in `startListening` is also kept +unchanged: it guards the video half of the request, which is still required. + +### 5. Node floor and documentation + +`.nvmrc` goes from `20` to `22`, and `engines.node` from `>=18` to `>=22.12.0`, +matching what the already-installed `@electron/packager@^20` requires. This is a +pre-existing inaccuracy rather than something Electron 44 introduces, but it sits +directly in the build path this change asks people to re-run, and a contributor +following `.nvmrc` today cannot package the app at all. + +README changes: the Requirements line and the Platform support table state that +the app needs macOS 13+ to run but **macOS 14.2+ for system-audio capture**, that +the permission appears as "Screen & System Audio Recording", and that macOS +13–14.1 users need the existing BlackHole fallback. The CHANGELOG entry written +by the superseded attempt is replaced — it currently describes the feature flags +as the fix, which is false — with one describing the Electron upgrade. + +## Testing + +`test/packaging.test.js` (new, `node:test`) covers the Info.plist invariant from +change 3: both sources declare `NSAudioCaptureUsageDescription`; both declare the +same set of `NS*UsageDescription` keys; and the descriptions are non-empty +strings (macOS shows them in the permission prompt, and an empty string is a +silently bad prompt). + +`test/audioLoopback.test.js` is deleted with its module. The remaining suites +(`prompt`, `settings`, `documents`, `llm`, `sse`, `store`) are pure modules +untouched by an Electron version change and must continue to pass unchanged. + +No automated test can prove system audio is captured — it needs a real macOS +permission grant, a real audio source, and a GUI. That is manual, and it is the +only thing that actually closes this bug: + +1. `rm -rf .electron-cache dist`, `npm install`, `make app`. +2. Replace the copy at `/Applications/Real Time Interview Copilot.app` — the + user's TCC grants are attached to that path, and a stale copy there is what + gets launched. +3. Grant both **Screen & System Audio Recording** and, when macOS prompts for it, + system-audio recording. The ad-hoc signature changes on every build, so macOS + treats each build as a new binary and prior grants do not carry over. +4. With **headphones**: the other party's speech appears under `Interviewer`, and + nothing appears under `You` while they speak. +5. With **built-in speakers**: the same, and no duplicate `You` line. +6. Revoke the permission and confirm a toast appears rather than a healthy-looking + session. + +## Risks + +- **A major-version jump of 11 releases.** Mitigated by how small and stable this + app's Electron surface is — ten imported APIs, none with a breaking change in + the 34–44 range — and by the fact that the full unit suite plus a real launch + are both cheap to run. If the app fails to boot, the bisect space is the eleven + majors, not the codebase. +- **Chromium 44's renderer may change layout or media behaviour subtly.** The UI + is a single static HTML file with no framework; visual regressions would be + obvious on first launch and are covered by the manual verification. +- **CoreAudio Tap requires macOS 14.2+, so this fixes nothing for macOS 13–14.1 + users.** Accepted: those users keep the BlackHole fallback, which change 5 + documents explicitly rather than leaving them to discover it. +- **The Info.plist sync test asserts equality of key sets, so adding a key to one + file deliberately will fail the suite.** Intended — the failure is the prompt to + add it to both, which is exactly the drift being prevented. +- **`electron-builder@26` and `@electron/packager@20` are not pinned to Electron + 44.** Both resolve the Electron version from the installed dependency rather + than bundling one, and neither declares an upper bound. If either tool breaks on + 44, the fallback is to package with the other, since the repo supports both. From 1a7b5c4ce15301b287cabba55bc2dd1443cd7e7d Mon Sep 17 00:00:00 2001 From: Danylo Lubenets Date: Tue, 8 Sep 2026 17:41:22 +0300 Subject: [PATCH 16/33] docs: add comment-translation task to the Electron 44 plan Task 0 translates source comments to English before the Electron work, so later tasks do not write comments that immediately need translating. It carries an explicit do-not-touch list: the repo also contains Chinese that is user-facing or load-bearing (the Chinese answer-language rule, the language