Skip to content

Sweep fix: speaker lab shared DB no longer leaks voices between runs - #2105

Merged
r3dbars merged 4 commits into
mainfrom
claude/sweep-fix-speaker-lab-shared-db
Oct 8, 2026
Merged

r3dbars merged 4 commits into
mainfrom
claude/sweep-fix-speaker-lab-shared-db

Conversation

@r3dbars

@r3dbars r3dbars commented Oct 7, 2026

Copy link
Copy Markdown
Owner

Split out of #2101 (one PR per bug). Confirmed bug from the Haiku sweep.

What was wrong

meeting-series kept <work>/shared-db forever. A --force rerun, or a fresh run over a work root an earlier run used, started with every voice the last run learned. fresh_db=false meetings then matched people they were supposed to meet for the first time, and lab_result.json / naming rows looked better than they were.

Fix

New MeetingLabSharedDB.swift. Before the first meeting, prepare() reads a lab-shared-db.json marker in shared-db.

  • The marker records the set, work root, shared meeting order and DB file name. It also records every meeting-series flag except --only/--limit/--force (flag order doesn't matter), and sha256 of the --embedder-thresholds file and the TRANSCRIPTED_LAB_KNOBS_FILE knobs file.
  • Before a shared meeting can write voices, the marker gets inProgress: <id>. recordApplied() clears it and adds the meeting once its lab_result.json is written.

How a run decides:

  • It resumes only when there's no --force, the marker matches this run, nothing is marked in progress, and the meetings in the DB are exactly the finished ones (first N in series order, no gap).
  • It starts empty on a brand-new run (nothing finished yet) or with --force. It deletes only after checking the path is under the run's own work root (refuses .. escapes and the root itself).
  • It stops and asks for --force, leaving the DB untouched, for any other mismatch while finished meetings exist. Examples: a meeting killed mid-run, retrying a failed mid-series meeting after later ones finished, a different --work, changed flags (--calendar-naming, --separation, --sep-*, --speaker-hint, --embedder-*, --backend), edited thresholds or knobs files, or an old DB with no marker. The error says why. It no longer runs the rest of the series on an empty DB.
  • Every run logs resumed or starting empty (<reason>).

Behavior change to know: --force --only m3 now runs m3 against an empty shared DB, not one holding m1/m2's voices. To rerun one mid-series meeting properly, rerun the series from the start with --force.

Failing test

runMeetingLabSharedDBSelfTests(), run by speaker-eval-harness autoeval-self-test. The package has no XCTest target. It drives prepare/beginMeeting/recordApplied/fingerprintArgs/fileHashes against real files in a temp work root and lists every failed promise.

  • Each of the three review rounds went red first, then green. The last round's red listed 14 broken promises against the previous code, including the interrupted meeting, the refusals without --force, --calendar-naming, and the thresholds/knobs hashes.
  • I also broke the config check, the in-order check, the in-progress check, the in-progress clear and the path guard on purpose; each went red.

Checks

  • bash build-deps.sh --force, swift build --package-path Tools/SpeakerEvalHarness: pass
  • speaker-eval-harness autoeval-self-test: PASS, run by hand only. The test-matrix command is BLOCKED (invalid-command) by agent-check.py, which won't run a bare-path command (already true on main), and SpeakerEvalHarness is in TOOLS_WITHOUT_CI. Neither check.sh nor CI runs it.
  • bash check.sh: everything else passed; only that BLOCKED entry.

Independent review: approve with nits (3 rounds, reviewed as part of #2101). Nits, all still open:

  • --series and --work go into the fingerprint as typed, so the same folder given as a relative path one time and an absolute path the next refuses a resume. --force gets past it.
  • A shared meeting skipped for a missing truth.json or a failed audio copy leaves a gap, so the next resume refuses and asks for --force. The gap itself predates this PR, and refusing is the safe outcome.
  • The runMeetingSeries call sites (prepare before the loop, beginMeeting before the pipeline, recordApplied after lab_result.json, stopping on refusal) aren't covered by any test, because they need Parakeet and the diarizer models.

🤖 Generated with Claude Code

meeting-series kept <work>/shared-db forever, so a --force rerun (or a
fresh run over a work root an earlier run used) started with every voice
the last run learned, and fresh_db=false meetings silently matched people
they should have met for the first time.

shared-db now carries a lab-shared-db.json marker: set, work root, shared
meeting order, DB file, every flag except --only/--limit/--force (order
free), sha256 of the --embedder-thresholds and lab knobs files, the
finished meetings, and the meeting in progress. A run resumes only on a
full match with nothing interrupted. A new run or --force empties it
(only under the run's own work root). Any other mismatch while finished
meetings exist stops the run and asks for --force instead of running the
rest of the series on an empty DB. Promise checks run in
autoeval-self-test.
@r3dbars
r3dbars marked this pull request as ready for review October 8, 2026 10:22
@r3dbars r3dbars added the waiting-on-human Needs a human (Justin) to look before anything else happens label Oct 8, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-08T13:38:23.074648Z 95e9de8 Manual request
🔒 Security Review ✅ Completed 2026-10-08T10:28:00.863998Z f0446be Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f0446be827

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Tools/SpeakerEvalHarness/Sources/speaker-eval-harness/MeetingLab.swift Outdated
Comment thread Tools/SpeakerEvalHarness/Sources/speaker-eval-harness/MeetingLabSharedDB.swift Outdated
…, env overrides, stale results, marker writes)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1f890c7e2b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Tools/SpeakerEvalHarness/Sources/speaker-eval-harness/MeetingLabSharedDB.swift Outdated
A forced run empties the shared DB, then beginMeeting still required the
first series meeting, so the documented --force --only m3 deleted the old
run and aborted. Forced runs may now begin any shared meeting; a DB built
that way is out of series order, so it is never resumed.
@r3dbars

r3dbars commented Oct 8, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b4e93a347

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@r3dbars

r3dbars commented Oct 8, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 95e9de8669

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@r3dbars r3dbars closed this Oct 8, 2026
@r3dbars r3dbars reopened this Oct 8, 2026
@r3dbars r3dbars closed this Oct 8, 2026
@r3dbars r3dbars reopened this Oct 8, 2026
@r3dbars
r3dbars merged commit a5d1d6b into main Oct 8, 2026
22 of 33 checks passed
@r3dbars
r3dbars deleted the claude/sweep-fix-speaker-lab-shared-db branch October 8, 2026 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-on-human Needs a human (Justin) to look before anything else happens

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant