You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Python: a skipped batch group shifts fact ids onto the wrong facts (auto-extraction can tombstone a fact whose replacement was not stored) #751
TotalReclaw.remember_batch returns only the ids of the facts it stored. When a batch group is skipped as a "duplicate", that group's facts get no id, so the list is shorter than the input. Two callers in agent/lifecycle.py still pair the list with the input by position:
Auto-extraction (_process_batch_results, called from store_extracted_facts) assumes "fact_ids[i] is the id of chunk[i]".
So after a skipped group, every later id is matched to the wrong fact.
What goes wrong
Example: a 15-fact auto-extraction chunk with embeddings goes over the 32 KB byte cap and is split into two groups. Group 1 (facts 0–7) is skipped and group 2 (facts 8–14) is stored, so 7 ids come back. Then:
Facts 0–6 are treated as stored, using the ids of facts 8–14.
Their text goes into the recent-write ledger and stored_texts under the wrong ids.
If one of them is an UPDATE, its old fact is deleted (tombstoned) although the replacement was never written. This loses data.
Facts 8–14 are logged as "not stored" although they were stored. Old facts they were meant to replace through an UPDATE are not tombstoned, so the vault keeps both.
The drain loses skipped queued facts silently. They were already taken off the queue, are not stored, and are not put back.
Why a group gets skipped on the managed service
operations._store_group_adaptive treats a store error as a duplicate, and drops it with no ids and no error, if its message contains '409', 'duplicate' or 'fingerprint' anywhere. The relay's /v1/bundler route never returns 409 or a duplicate error; only the self-hosted server did, and the Python client does not support self-hosted. So on the managed service the skip only fires by accident. For example, '409' appears inside a hex hash or address in a bundler error message. That accidental match drops a group silently and triggers the misalignment above.
Found by the review of #750. This bug comes from the #747 drain and the older auto-extraction path, not from #750.
Fix
remember_batch keeps returning the stored ids, as before, but the returned list also carries batch_results: one entry per input fact, in input order, holding the fact id or None when that fact was not stored. This matches what RelayWriteQuotaExceeded.batch_results already provides on a quota refusal.
Auto-extraction and the drain read batch_results, so each id stays with its own fact. A fact that was not stored is not recorded and its UPDATE does not tombstone anything. In the drain, it goes back to the queue with attempts + 1, capped by the existing attempt limit.
_store_group_adaptive treats 409 as a duplicate only when it appears as a whole word, as in HTTP 409, not as three digits inside a hex string.
Activity
fix(python): keep batch fact ids with their own facts when a group is…
fix(python): keep the facts a partly failed batch did store (#753) (#754