Match scanned files when the on-disk name drops a leading article

[dny238]

Problem

The per-audiobook filesystem scan (ScanFileDiscovery.FindMatchingAudioFiles) links a file to the book only when the filename contains the full library title or the path contains the full author string (case-insensitive substring). That misses very common real-world naming:

• The file/folder dropped a leading article — Language of Emotions.m4b for the book The Language of Emotions.
• The author folder dropped post-nominal credentials or an initial — Karla McLaren\ for author M.Ed. Karla McLaren, or John M. Gottman\ vs library author John Gottman **PhD**.
• The title carries a subtitle the folder omits — Developing Mind\ for The Developing Mind, Third Edition.

In these cases the audio file sits on disk but the scan matches nothing, so the book stays fileless and Scan Folder appears to do nothing even though the file is right there.

Fix

Adds tolerant, token-based matching on top of the existing exact checks — purely additive, so any file that matched before still matches:

• Titles compared with leading articles (the/a/an) and punctuation/subtitles normalized away.
• Authors compared with honorifics/credentials (PhD, MD, M.Ed., CFP, Jr, …) and single-letter initials dropped.
• Order-independent token-subset comparison (either side may be the fuller string), which handles subtitles and middle initials.

Matching is scoped to the audiobook's own scan folder (scanRoot = audiobook.BasePath), so the extra leniency can't pull in unrelated books.

Tests

New ScanFileDiscoveryMatchTests covering: exact-title baseline (unchanged), dropped leading "The", author-with-credentials, author-with-middle-initial, title-with-subtitle vs base-title folder, and a negative case (unrelated file must not match).

• dotnet build clean (0 warnings)
• New tests 6/6; broader Scanning/Library suites 68/68 pass

🤖 Generated with Claude Code

[m4bard]

Worth flagging up front: this file is rewritten on #717. Matches is gone there, replaced by FileNameMatchesExpectedTitle(...) && HasAuthorContext(...) — title required, author demoted to corroboration — and the whole-directory grouping is gone too. I had a PR against this same method (#766) and just closed it as superseded; this one is in the same position. That's most of why I'm commenting rather than suggesting changes.

I built your branch and ran it against its own base (4555ad21, which is also current canary), so the only variable is this diff. One audiobook added, BasePath cleared so the scan root falls back to the library root, scan, then every linked file mapped back to its true owner. Library is ten public-domain books laid out {Author}/{Title}.

scanned	canary 4555ad21	this branch
Ghost Stories of an Antiquary — M. R. James	1 file (its own)	3 files — also Henry James' The Turn of the Screw and James M. Barrie' Peter Pan in Kensington Gardens
Faust I — Goethe	2 files (1 foreign: Faust I + II)	3 files — also Faust

The first row is a new misattribution, and it also moves BasePath from /audiobooks/M. R. James/Ghost Stories of an Antiquary to /audiobooks/ — the common parent of files that no longer belong together. The second row is partly pre-existing: canary already claims Faust I + II by substring, and this branch adds Faust on top.

The mechanism for the first is NormalizeAuthor dropping single-letter tokens, so M. R. James collapses to {james}, which is then subset-matched against the whole path via authorTokens.IsSubsetOf(TokenSet(NormalizeText(file))). Any file under any path containing "james" qualifies. That inherits the problem in filePathMatchesAuthor — it says "somewhere on this author's shelf", not "this book" — which the grouping above then amplifies, since one matching file adopts its whole directory. The second row is IsSubsetEitherWay accepting a subset in either direction, so an omnibus and its volumes claim each other.

Then I built the two shapes your description is about, to check what the change buys:

shape	canary	this branch	#766 (title-required)
author folder is a variant — James M. Barrie vs recorded J.M. Barrie — filename keeps the title	links	links	links
leading article dropped from folder and filename	links	links	0 files — misses

Both already work on canary, so on these two shapes the tolerant matching isn't adding recall. But the second only works on canary because of the author arm: the title arm needs the full recorded title as a substring and Turn of the Screw doesn't contain The Turn of the Screw, so the author arm is what carries it. #766 removes exactly that arm and drops to zero — which is your bug report, reproduced against my PR rather than yours.

That's the part I'd flag as worth keeping. Your NormalizeTitle handles that case directly: article-stripped, The Turn of the Screw and Turn of the Screw normalize to the same token set, so IsSubsetEitherWay matches on the title alone with no author involved. I checked that statically rather than at runtime, because on this branch filePathMatchesAuthor returns first and the tolerant title code never runs for that input.

So once #717 removes the author arm, article-dropped names lose the thing currently carrying them, and the title half of this is what replaces it. It's the author half — whole-path, initials stripped — that produces the misattributions above, and dropping it doesn't cost either shape I tested.

Repro, if it's useful — from listenarr-testdata:

./tools/validate_scan_attribution.sh --image <your-build> --asin B004FOLXEO \
    --layout author-title --only-asin B004FOLXEO,B01ATTZF38,B0C6FJ6L34

It generates the library, drives a real container, and exits non-zero if the scan claims a file belonging to another book. Happy to run it against any revision you push.

[m4bard]

Ran the harness against 0ac2326d. The misattributions I measured before are gone, and your own cases still link.

Same setup as last time. One book added, BasePath cleared so the scan walks the library root, then every linked file mapped back to its true owner.

scanned	canary 4555ad21	f9a93076	0ac2326d
Ghost Stories of an Antiquary (M. R. James)	1 file, its own	3 files	1 file, its own
Faust I (Goethe)	2 files, 1 foreign	3 files, 2 foreign	2 files, 1 foreign
Anne of Green Gables (L. M. Montgomery)	1 file, its own	2 files	1 file, its own

BasePath on the first row is back to /audiobooks/M. R. James/Ghost Stories of an Antiquary rather than collapsing to /audiobooks/. The foreign file still showing against Faust I is Faust I + II, which canary already claims by substring, so it predates this branch.

Your own shapes, same build:

shape	result
author folder is a variant (James M. Barrie vs recorded J.M. Barrie), filename keeps the title	links its own file
leading article dropped from folder and filename	links its own file

SetEquals is a better fit here than the subset test was. The article case still works, because The Turn of the Screw and Turn of the Screw normalise to the same token set. An omnibus and its volumes no longer claim each other, because theirs differ. One caveat on that first point: I confirmed it by construction rather than at runtime, since filePathMatchesAuthor still returns first for that input and the tolerant branch never runs.

You can run this yourself rather than waiting on me:

git clone https://github.com/m4bard/listenarr-testdata && cd listenarr-testdata
python3 -m venv .venv && .venv/bin/pip install -e .
./tools/vet-against.sh \
    --repo https://github.com/dny238/Listenarr.git --branch fix/scan-file-match-tolerant \
    --tool attribution \
    --asin B004FOLXEO --layout author-title \
    --only-asin B004FOLXEO,B01ATTZF38,B0C6FJ6L34

That clones your branch, builds it, generates a library holding those three books, scans, and exits non-zero if the scan claimed a file belonging to another book. It needs podman or docker, plus ffmpeg and curl.

The three ASINs are the M. R. James, Henry James and James M. Barrie books from the table, which are useful together because the names overlap. Anything in corpus/corpus.json works instead. Standalone books do need --layout author-title, because the default {author}/{series}/{title} cannot render a book with no series and skips it without saying anything.

[dny238]

Thanks for the thorough repro — the M. R. James → {james} → whole-path subset match is exactly the failure mode you measured, and you're right that dropping single-letter initials made it worse.

Pushed 0ac2326: removed the author arm entirely and tightened the title rule to strict token-set equality after article/punctuation normalization (no more either-way subset). So the tolerant path now only fires when the on-disk name equals the recorded title modulo a leading article — e.g. Language of Emotions for The Language of Emotions — and never attributes on author grounds. Added a regression test for the shared-surname cross-book case (The Turn of the Screw under Henry James/ must not match Ghost Stories of an Antiquary).

I've left the pre-existing filePathMatchesAuthor substring check alone since that's canary behavior and, as you note, #717 is already reworking it toward title-required matching. If #717 lands, the useful residue of this PR is precisely the NormalizeTitle equality — the thing that carries article-dropped names once the author arm is gone — and I'm glad to fold just that into #717's approach and close this if you'd prefer. Your call on whether it's worth keeping as a standalone or superseding.

Happy to run your validate_scan_attribution.sh against the new revision.

[therobbiedavis]

#717 replaces the old broad scan matcher with ownership/stable-identifier/book-boundary attribution and incorporates the #766/#765 author-overmatch fix. The tolerance in this PR (leading articles, credentials/initials, token-subset matching) is still additional behavior and is not superseded. Please rebase on #717 and add the tolerance inside the new book-boundary matching rules rather than reintroducing broad path matching.

[dny238]

@m4bard thanks for re-running your harness against the reworked branch — good to have it confirmed that the misattributions are gone while the real-world cases still link.

@therobbiedavis understood — I'll rebase this on #717 and add the tolerance (leading articles, credentials/initials, title token-set equality) inside the new book-boundary matching rules rather than reintroducing broad path matching. I'll wait for #717 to merge so I'm folding it into the final rules. The tolerance is title-only SetEquals now — no author fallback — so it should slot in as a normalization step rather than a widening of what counts as a match.

[m4bard]

Heads up that the thing this was waiting on has landed: #717 merged on 2026-08-12, and #819 has gone in since, which moved a lot of the same area again.

No pressure on the rebase from me. I mention it only because the wait was for a specific event and that event is now a week and a half old, so it seemed worth saying rather than assuming you had seen it.

The harness that produced the earlier numbers still exists and still runs, so when you have the tolerance folded into the book-boundary rules I am happy to re-run it against the rebased branch the same way as before: the shared-surname cross-book case, the leading-article case, and the credentials and initials cases, reported as attributed against not attributed per case rather than as a single pass or fail. If it would be more useful to have that as a check you can run yourself rather than as a comment from me, say so and I will point you at it.

[dny238]

@m4bard thanks — I've pushed the reworked branch. It's rebased on canary and the leading-article tolerance is now folded into #717's book-boundary matcher (SegmentMatchesExpectedTitle): a folder segment and the expected title are treated as equal when they differ only by a leading article ("The"/"A"/"An") on either side. It stays a full-title equality modulo the article — no substring or author-based matching — so the same-author / different-book boundary guards you and @therobbiedavis added remain intact. Three new tests in ScanFileDiscoveryTests cover dropped-article, added-article, and a same-author sibling-book guard (The Reckoning must not collapse onto The Awakening).

If you're still up for re-running your harness against this branch, that'd be hugely appreciated — the shared-surname cross-book, leading-article, and credentials/initials cases, reported per-case attributed/not-attributed as before. Heads-up on expectations: this rework is deliberately title-only (it drops the author-based fallback per the earlier review), so the credentials/initials cases should now report as not attributed — that's intended, not a regression.

[m4bard]

Ran the harness against cd29725. Results per case below, each one also run against ghcr.io/listenarrs/listenarr:canary on the same library so the two columns are comparable.

The scan-attribution check adds exactly one book, clears its BasePath so the whole library root gets walked, scans, then maps every linked file back to its true owner from the generator's manifest. So "foreign" below means a file the scan claimed for the scanned book that actually belongs to a different one.

case	canary	cd29725
shared surname, two authors, both on disk, one in the DB	1 linked, own, 0 foreign	1 linked, own, 0 foreign
on-disk name drops the article, same-author sibling present	0 linked	1 linked, own, 0 foreign
author credited in full, on-disk in initials, sibling present	0 linked	0 linked

The article case does what you describe, and the boundary holds

This is the one worth having. The book is The Hound of the Baskervilles, the folder on disk is Hound of the Baskervilles, and The Sign of Four by the same author sits next to it in the same author folder.

On canary the scan links nothing at all: the book is simply not found. On your branch it links one file, its own, and BasePath resolves to /audiobooks/Arthur Conan Doyle/Hound of the Baskervilles. The sibling is not touched.

That is your The Reckoning / The Awakening test holding at runtime rather than in isolation, which was the part I could not tell from reading the diff. Full-title equality modulo the article is narrow enough that the same-author neighbour never comes into range.

The shared-surname case is identical on both sides, so nothing regressed there either.

The initials case is unchanged rather than broken

I want to be careful here, because I nearly reported this as a failure.

Author credited as Lucy Maud Montgomery in the record, folder on disk L. M. Montgomery, with Anne of Green Gables sitting under that same folder as the sibling. Your branch links nothing. So does canary. It is not a regression, it is a case neither version covers.

Reading the diff, that is expected: the change strips a leading article from both sides of a title comparison and says so explicitly, and the comment is clear that it does not widen into author matching. So nothing in cd29725 would move this case.

Worth raising anyway, because the PR title still says "or author credentials" and the implementation on this revision is article-only. Whether the title should narrow or the credentials half is still to come is your call, but as it stands somebody reviewing against the title will look for something that is not in the diff.

One caveat on that case: I chose full given names against initials as my reading of "credentials/initials". If you meant an honorific or a suffix, Dr. or Sir or a postnominal, that is a different comparison and my null result should not be taken as covering it. Say which and I will run that instead.

How to reproduce

The two prepared cases are not a checked-in scenario, so this is the honest recipe rather than one command. Both start from the public corpus in the test-data repo.

# build an image from the PR head
tools/vet-against.sh --repo https://github.com/dny238/Listenarr.git \
    --branch fix/scan-file-match-tolerant --tool attribution \
    --layout author-title

# shared surname, no preparation needed
tools/validate_scan_attribution.sh --image <IMAGE> \
    --asin B004FOLXEO --only-asin B004FOLXEO,B01ATTZF38 --layout author-title

For the article and initials cases: generate a two-book library with
tools/generate_library.py --layout author-title --only-asin <PAIR> --out <DIR> --seed 1,
rename the target book's folder and file on disk to the variant form, rewrite the matching
path in that library's manifest.json, then pass --library <ABSOLUTE DIR> to the validator.
The pairs are B0036HXZCO,B0036I51QQ for the article case and B002V8L2UQ,B073JR7W68 for the
initials one, dropping the leading The and shortening the author folder to initials respectively.

I am going to add those two variants to the generator so they stop being hand-built, and I will
follow up here with a single command once they are.

[m4bard]

Following up on the last paragraph of my earlier comment: the two cases are no longer hand-built, so here are the single commands.

./tools/validate_scan_attribution.sh \
    --image ghcr.io/listenarrs/listenarr:canary \
    --asin B0036HXZCO --layout author-title \
    --only-asin B0036HXZCO,B0036I51QQ \
    --folder-variant drop-leading-article:B0036HXZCO

./tools/validate_scan_attribution.sh \
    --image ghcr.io/listenarrs/listenarr:canary \
    --asin B002V8L2UQ --layout author-title \
    --only-asin B002V8L2UQ,B073JR7W68 \
    --folder-variant author-initials:B002V8L2UQ

Swap --image for a build of your branch, or go from source in one step:

./tools/vet-against.sh --repo https://github.com/dny238/Listenarr.git \
    --branch fix/scan-file-match-tolerant --tool attribution \
    --asin B0036HXZCO --layout author-title \
    --only-asin B0036HXZCO,B0036I51QQ \
    --folder-variant drop-leading-article:B0036HXZCO

--folder-variant KEY:ASIN spells one book's folder differently from its record while the record and the embedded tags keep the canonical form. Scoping it to one ASIN is the point: the sibling stays in ordinary form, so the same run measures both halves at once, whether the matcher reaches the variant folder and whether it stops there.

It sits on the generator's hazard axis rather than being a layout, because a layout is uniform and this case needs one book varied beside one that is not. A variant a book cannot express is recorded as unapplied rather than claimed, so Anne of Green Gables having no article to drop shows up as such instead of silently counting as a pass.

The numbers are the same ones I reported before, which is how I checked the generated tree matches what I had built by hand: article case, canary links nothing and your branch links one own file with none foreign; initials case, nothing on either.

Two things I got wrong the first time, in case you hit them. --library needs an absolute path, because a relative one is read as a container volume name and fails with a message about volume naming that has nothing to do with paths. And vet-against.sh forwards flags to whichever --tool you picked, so a flag from a different tool used to surface only after the image build finished. Both are fixed: the path is absolutised, and forwarded flags are checked against the runner before anything is cloned.

[dny238]

@m4bard thank you — running it at runtime against canary on the same library is exactly the confirmation I couldn't get from the diff, and I appreciate the care on the initials case (calling it "unchanged, not a regression" rather than quietly a pass).

You're right about the title, and I've retitled the PR to match the diff: it's now "Match scanned files when the on-disk name drops a leading article". This revision is deliberately article-only — I dropped the author/credentials arm during the earlier review because the surname-based matching was the source of the cross-book misattributions you originally caught, so keeping it strictly full-title-modulo-article is what makes the boundary hold.

On what "credentials" was meant to cover: author-folder differences like dropped post-nominals (MD/PhD) and initials-vs-full-name (L. M. vs Lucy Maud). I'm intentionally leaving that out here — it needs author-token tolerance that's much easier to get wrong (exactly the failure mode you measured before), so it belongs in a separate, separately-validated change rather than riding along on this one. If there's appetite for it I'll open it as a follow-up, and I'd take you up on running your honorific/suffix and initials variants against that branch.

Thanks also for folding the two variants into the generator and posting the single commands — that's a genuinely useful harness.

[m4bard]

The retitle matches the diff now, and I think dropping the author arm was the right call rather than a concession. The failure I measured before came from surname-based matching pulling in the wrong book, and full-title-modulo-article does not have that failure mode: the title still has to match in full, so there is no room for a near-miss to win. Keeping the two apart is what makes this one reviewable on its own.

On the follow-up, and what I can actually offer

Yes, and I would rather be precise about what exists so you are not waiting on something I have not built.

The generator in the harness currently has two folder variants:

drop-leading-article    rewrites the folder title without a leading article
author-initials         rewrites the author folder as initials, "L. M. Montgomery" -> "L. M."

So the initials-versus-full-name half of what you described is ready now. Post-nominals are not: there is no MD or PhD variant, and adding one is a small change to the same table rather than new machinery. If you open the follow-up I will add it and run both against your branch.

One thing worth deciding before you write it, since it bit the earlier version. Initials tolerance and post-nominal tolerance are not the same risk. Stripping PhD from a folder name is close to lossless, because almost nothing distinguishes two authors by post-nominal alone. Matching L. M. to Lucy Maud is a genuine widening: it accepts any author whose initials collide, and the library does not have to be large before that happens. If they land as one change they will be judged by the riskier half.

I have no opinion on whether you split them, only that the initials half is the one that needs the evidence.

The harness

Glad it is useful. Both variants are on the public generator, so anything I run against your branch is reproducible from the same commands rather than from my library. If it would help, say which ASINs you want them applied to and I will post the exact invocation alongside the result.

Worked through with Claude Code at my direction. The variant list above is read from the generator as it stands today, and the post-nominal gap is stated because it is not there yet. I reviewed this before posting.