Repository navigation
Conversation
Memory pages join the one recall the prompt router and `rules --prompt` already run for notes (spc-2610111627213138 step 3). A page whose frontmatter declares a `recall:` list is an item keyed page:<filename>, read from the project's .abcd/memory/ through the store handle and, read only, from ~/.abcd.noindex/memory/ through the home scope's guarded open, each machine page held to the guards the machine memory.json gets. A page in the project's store suppresses the machine's page of the same name, and pages follow notes in the id order until the ranking lands. The recall still opens only its four declared places: a test records every open through inotify, with a host memory folder of matching text under the home, and finds none there. A page or page store that cannot be read is named on stderr and in the diagnostic's unreadable list, and the rules and every other item still come back. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
…silent A Settled here section under the spec's Decisions carries each choice step 3 made, dated and marked as an agent's choice for review: which pages the recall reads and what a page's title is, a workspace page shadowing the machine's page of the same name, the guards a machine page is read with, and how a page that cannot be read is named. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
When only some matched notes and pages fit the per-turn budget, a relevance ranking now chooses which, in place of the id order (spc-2610111627213138 step 4). The score is BM25F in the standard library: a term counts three times in an item's recall words, twice in its title and once in its body, weighted frequencies summed before saturation, with k1 1.2, b 0.75 and idf over every item in reach, so a rare matching word outranks words every item shares. Ties break on the narrower scope, then the more recent note or page, then the key. The query terms come from the loader's own prompt index, and an item's words are stemmed by the same rule, so the ranking weighs the words the matcher matched. `rules --prompt` shows each candidate's rank and score and, under it, every matched term with the fields it was found in and what it added. A golden file pins the scores of a small fixed corpus. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
…silent The spec's Settled here section gains step 4's choices, dated and marked as an agent's choice for review: a page's recency is its latest ingested_at, how the ranking counts terms and lengths (with the two rules exports it needed), the ranked take keeping step 2's stop at the first item that does not fit, and the diagnostic ranking notes and pages in one order. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
The machine page scope was opened through the home scope's walk, which refuses a symlinked directory but judges nothing about who can write one, so a ~/.abcd.noindex every account could write had its notes file refused and its pages beside it read and injected. The scope is now opened through fsutil.OpenHomeDeclarationScope, the walk with the directory judgement ReadHomeDeclaration applies, and absence is decided first, as it decides it, so an exposed home with no page scope still costs nothing. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
A page's title is the first line of its body, the line its index summary comes from, and the body the recall weighed and rendered still carried it, so a title word counted in the title and again in the body (as much as a recall word) and the injected block repeated it. The page's body is now the rest of the page after that line. The golden corpus models pages the same way, and its scores are regenerated. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
A page can be as large as 4 MiB while a turn carries 64 KiB, and the take stopped at the first candidate that did not fit, so one oversized page ranked first held back every item below it on every turn, each reported as "tried again on the next" though no turn could change it. An item whose block alone is larger than the whole per-turn budget is now set aside as too large, never recorded, without stopping the take: the hook names it on stderr, and `rules --prompt` gives it the form too-large. An item that only misses this turn's remainder still stops the take, as before. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
The ranking's query terms were keyed on each token's stem, but the term rule matches a token by several candidate roots, so "merge before merging" made two terms that both claimed an item's "merge" and scored it twice, each with its own idf. Tokens whose roots overlap are now one term, so each root belongs to one term and an item's word is found under one term at most. The terms are built with a root-to-term map, so a long prompt costs no more than its tokens. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
The project's page of a name suppressed the machine's page of that name only when the project's page was read, so one the read refused (over the page cap, say) let the machine's page come back in its place, reversing the narrower-scope rule. A refused workspace page now shadows as a read one does, and a test holds both that and a silent workspace page (no recall list) shadowing. The read's comment now says the pages it reads are those the index is rendered from, and that it never reads index.md. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
The ranked order compared scores exactly, so two scores equal in arithmetic but for the last bits a machine's float operations may leave (a fused multiply-add on one architecture and not another) could order differently between machines. Scores are now compared at a billionth before the ties break on scope, recency and key. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
… fixes The Settled here bullets now say what the review fixes changed: a page's body leaves out its title line, a refused workspace page still shadows, the machine page scope's directories are judged as the machine notes file's are, one word in two inflections is one ranking term, an item larger than the whole budget is set aside without stopping the take, and scores tie at a billionth. Refs: itd-2610091918433290, itd-39 Assisted-by: Claude:claude-opus-5-5
Collaborator
Author
|
Heads-up from the local reviewer: step 2's base branch has review fixes that will change the code you are building on. They are committed locally and will be pushed to
When the base branch moves, I will post another comment here. Keep going, and do not arm or merge. |
Main carries step 2's review fixes. The recall now judges a personal note's tie lazily, after matching (recall.Matching), so the ranking calls Matching in place of candidates and hands back the root lookup's error: Rank takes the RootFunc, and Recall and `rules --prompt` name a lookup git did not answer in time once, as main's step 2 does. Main's per-layer labels and panic recovery are kept beside the branch's TooLarge set, and the hook's stderr line names notes and pages as items. The id-order test step 4 replaced stays removed, and two tests that read the id order now judge which items came back, not their order. The spec's settled choice on the ranking corpus is amended: N and avg_len run over every item read, a note tied to another repository among them, since the tie is no longer judged before matching. Assisted-by: Claude:claude-opus-5-5
Main's provenance labels named every machine item "personal note, agent-written" and every project item "project file", so a page from the machine's memory store read as an agent's note. A page's head now says "project page, committed in this repository" or "machine page, this account's memory store". Assisted-by: Claude:claude-opus-5-5
Since main's step 2 review the recall judges a tied note against the RootFunc it is handed, and a nil one answers no root, so the test's note tied to this repository was left out and the want list failed on Linux. The test now asks git itself, which also keeps git's own opens among those recorded. Assisted-by: Claude:claude-opus-5-5
PromptTerms cut the prompt by hand; main's step 2 review made rules.CapPrompt the one cap every matcher input goes through. Assisted-by: Claude:claude-opus-5-5
After the merge the ranking calls the tie-judging match and ReadItems calls the note read, so nothing outside the recall package reaches either and the exported-reach audit refused them. Assisted-by: Claude:claude-opus-5-5
The merged stderr line labels each item with its layer, as main's step 2 does for notes. Assisted-by: Claude:claude-opus-5-5
Assisted-by: Claude:claude-opus-5-5
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This builds steps 3 and 4 of spc-2610111627213138. Steps 1 and 2 landed in #1003, so this pull request's base is
main, and the spec now marks steps 1 to 4 as landed with their commits.Step 3 adds memory pages to the recall that already brings notes back. A page whose frontmatter declares a
recall:list comes back on a prompt that matches it. Pages come from the project's.abcd/memory/and, read only, from~/.abcd.noindex/memory/. Machine pages are read with the same guards as the machinememory.json. Notes and pages come back together in one## MEMORYblock, deduplicated per session through the same ledger, and a project page suppresses the machine's page of the same name. The recall still opens only its four declared places. A Linux test records every open with inotify, puts a host memory folder full of matching text under the test home, and finds nothing opened there.rules --promptlists matched pages beside notes.Step 4 adds the relevance ranking. When only some matched items fit the per-turn budget, BM25F decides which stay, in place of the id order. It uses the standard library only:
rules --promptshows each candidate's rank and score, and under it each matched term with its fields and what it added.Two fresh-agent reviews of the whole diff against the spec's criteria found nothing blocking. Their should-fix and nit findings, and a later takeover review's, are each fixed in their own commit. Where the spec is silent, each choice is recorded under its "Settled here" section and marked as an agent's choice for review. Latest
mainis merged in.Steps 5 to 8 (the fill reader, the brackets, the docs and the live close) follow in later pull requests; the intents stay planned until then.
Held unarmed until the security and bug-fix release is tagged, so that release carries none of this feature.
Refs: itd-2610091918433290, itd-39
Assisted-by: Claude:claude-opus-5-5