Repository navigation
fix(python): keep batch fact ids with their own facts when a group is skipped (#751) - #752
Merged
Merged
Conversation
… skipped (#751) remember_batch returns the ids of the stored facts only. A group skipped as a duplicate contributes none, so callers that paired the list with the input by position moved every later id onto the wrong fact: auto-extraction could tombstone the old fact of an UPDATE whose replacement was never written, and the #747 drain silently lost skipped queued facts. - remember_batch returns BatchIds: still a list of the stored ids, plus batch_results (one id or None per input fact, the same shape as RelayWriteQuotaExceeded.batch_results). - Auto-extraction and the drain pair ids with facts through it. A skipped queued fact goes back to the queue with one more attempt counted. - A store error is a duplicate only when 409 is a whole word, not digits inside a hex hash. The managed relay never returns 409 for writes, so the old substring match only ever fired by accident. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #751.
Problem
TotalReclaw.remember_batchreturned only the ids of the facts it stored. A group skipped as a "duplicate" contributes no ids, so after a skip the list is shorter than the input. Two callers paired the list with the input by position:_process_batch_results). Every id after the skip was matched to the wrong fact. AnUPDATEwhose replacement was skipped could still tombstone its old fact, which lost data. Facts that were actually stored were logged as "not stored", and theirUPDATEs left the old fact in place._drain_entries,zip(to_store, ids)). Skipped queued facts had already been taken off the queue and were never put back, so they were lost silently.On the managed service the skip only fires by accident.
_store_group_adaptivetreated'409'anywhere in an error message as a duplicate, including the digits inside a hex hash or address in an unrelated bundler error. The relay's/v1/bundlerroute never returns 409.Changes
client.py:remember_batchreturns aBatchIds. It is still alist[str]of the stored ids, so existing callers and==comparisons are unchanged. It also carriesbatch_results, with one entry per input fact holding the id orNone. This is the same shapeRelayWriteQuotaExceeded.batch_resultsalready has for a quota refusal. The data comes from the per-factstored_atmap that Hermes: write-quota exhaustion (403 on /v1/bundler) never surfaces to user or agent — auto-extraction facts silently dropped (44+ since Nov 9) #747 already built.agent/lifecycle.py:_per_fact_idslines ids up with facts. A plain list from a stub client is taken as already aligned.UPDATEtombstones nothing.attempts + 1, through a new_retry_laterhelper shared with the existing failure path. It is dropped at the existing limit of 5 attempts. If the fact really is already in the vault, the next drain's write gate drops it.operations.py: newis_duplicate_rejection(msg)counts409only as a whole word (\b409\b);duplicateandfingerprintstill match as before. It is used in_store_group_adaptiveand in the import engine's per-fact path. Existing duplicate tests ("HTTP 409: duplicate fingerprint", "409 fingerprint duplicate") still pass.Not changed: the MCP server's
import-from.tshas the sameincludes('409')match. It is the self-hosted per-fact path, where the server really returns 409, and has no alignment problem, so it is left for a separate change.Testing
python/tests/test_batch_id_alignment_751.py, run against the realremember_batchover a patchedstore_fact_batch.forgetwas awaited for the skippedUPDATE. In the drain test, the id was recorded against the wrong text. The hex-409error was swallowed.api-staging.totalreclaw.xyz.remember_batchreturnsbatch_resultsthat match the ids 1:1, and each id decrypts to its own fact text from the staging subgraph.-32500, which the UserOp layer retries 3 times and the batch writer then halves. The first single op gets HTTP 409 and the second goes to staging. The drain stores 1 fact. The skipped fact is still queued withattempts == 1, and the stored fact is indexed on staging.🤖 Generated with Claude Code