Skip to content

Fix memory integrity and introduce project-centered dashboard workflows - #203

Open
Coding-Dev-Tools wants to merge 13 commits into
mainfrom
codex/rework-memory-foundation
Open

Fix memory integrity and introduce project-centered dashboard workflows#203
Coding-Dev-Tools wants to merge 13 commits into
mainfrom
codex/rework-memory-foundation

Conversation

@Coding-Dev-Tools

@Coding-Dev-Tools Coding-Dev-Tools commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Description

Concurrent governed writes could create multiple successors and delayed sync publication could restore erased vectors. Packing could also discard repository-specific facts, and synthesis could remove exceptions while still reporting a grounded answer. This change makes those transitions authoritative and gives project-memory tasks a consistent dashboard journey.

  • Add atomic, idempotent revisions, schema 18 command receipts/source claims, version conflicts and paginated record lineage. Slow revision preparation happens before the writer reservation. Committed correction/revision/approval retries replay without embeddings and preserve erased/retired-result conflicts. Approval retries retain the original recorded reviewer; malformed lineage metadata cannot crash history or create false links. Completed session promotions/merges replay after closure without embeddings, and project history retains promoted workspace versions without widening caller access. The inspector and editor send the selected project to history and cancel stale responses on context changes; broader roots remain readable within a project.
  • Repair indexes from current canonical generations, retaining erasure and retry semantics. Native index failures roll back canonical changes. Cleanup is processed before blocked vector updates, including with a one-attempt budget; failed work remains durable.
  • Preserve distinct scoped facts and complete cited evidence units. Answer coverage defaults to unknown, independently of support and citations. Numerical ranking and backend defaults remain unchanged.
  • Add portable scoped cursors, bounded file-backed browsing snapshots, durable startup transformation markers, content-free diagnostics and package/build identity.
  • Organize Home, Ask, Library and Connections around project context, visible ownership, atomic editing and history. Preserve drafts and independent answer/preview retries. Pause hidden Explore views through the existing renderer APIs; retain CSP, themes and renderer algorithms.
  • Add strict corpus, paired-result and complete-engine capacity measurement contracts. Refresh the measured offline payload artifact and visible figures without weakening the evidence tolerance. These tools do not establish 100k capacity or independent task quality.

The original checkout's active graph/layout work stays separate. The implementation and remaining dependency order are recorded in docs/REWORK_EXECUTION.md.

Type

  • Bug fix
  • New additive interfaces
  • Documentation and evaluation infrastructure
  • Packaging gate coverage

Verification

  • All checks passed on 31da32c06a5ade989608504472e00dfca6f13f94: CodeQL and ci. This includes the NumPy-only Python 3.9 floor, full Python 3.10–3.14 stacks, encryption matrix, coverage, browser, packaging, Docker, type and hygiene gates. The Python 3.12 full suite passed 5,067 tests with 18 optional/platform skips.
  • The approval, lineage, session-retry, lost-response/embedding and erasure-starvation counterexamples failed on their prior source and pass after the fixes. Additional focused governance, poisoning, inspection, REST/history and workspace-isolation suites passed; independent review found no remaining blocker in those changes.
  • Seven documented offline evaluation gates, Ruff, Pyright, commercial manifest, generated MCP contract, packaged CSP assets and whitespace checks passed.
  • Complete browser CI passed. The final focused history/workflow run passed 25 checks, including real-service project isolation, stale request cancellation and correction/history smoke in Firefox and WebKit. Human agent restart and screen-reader testing are not claimed.
  • Built wheel and sdist passed archive verification. A fresh Windows Python 3.12 environment containing only NumPy and the installed wheel migrated a real schema-17 fixture, restarted, retried an atomic revision, rejected conflicting reuse, and paged preserved history. Database/backup integrity, all new assets, dependency compatibility and all 16 console help entrypoints passed on the wheel rebuilt from this exact commit.

Migration and remaining release gates

Schema 18 uses the verified pre-migration backup path. Never downgrade a migrated live database in place: restore into a separate fenced location and reconcile later erasures/revocations before serving it. Production restore/cutover, both physical capacity targets, the independent 400-task corpus, human usability and the consented pilot remain open. Paid evaluations require the separate concrete proposal's approval. No merge, release or deployment is included.

Related private processing/recovery work: Coding-Dev-Tools/engraphis-cloud#67. Website release contracts: Coding-Dev-Tools/engraphis.com#12.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-06T07:41:58.920843Z 31da32c New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 82357fc02c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/engine.py
Comment on lines +3211 to +3215
return {
"id": candidate.id,
"approved_from": old.id,
"op": "noop",
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the stored reviewer when reusing an approval

When a pending memory already has a live approved successor—for example, after a lost response—the retry path returns only the successor ID and op. approve_for_prompt() then reports the reviewer supplied by the retry rather than the reviewer stored in the successor's approval metadata. A retry made under a different reviewer therefore falsely attributes the existing approval while its audit record still names the original reviewer; return the stored reviewer from this branch.

Useful? React with 👍 / 👎.

Comment thread engraphis/service.py Outdated
if memory_id in (meta.get("supersedes") or []) or meta.get("corrects") == memory_id:
if (memory_id in (meta.get("supersedes") or [])
or meta.get("corrects") == memory_id or meta.get("approved_from") == memory_id
or memory_id in (meta.get("promoted_from") or [])):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate promoted_from before scanning successors

When caller-controlled metadata contains a non-list promoted_from value, such as 1, this membership test raises TypeError. Because the preceding SQL selects any same-workspace metadata containing the target ID, an otherwise unrelated record like {"note":"<target-id>","promoted_from":1} makes both inspection and the new history endpoint fail for that target. Type-check promoted_from as is already done in predecessors() before testing membership.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 957f52aae0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/vector_repair.py Outdated
Comment on lines +104 to +105
if not vector_writes_ready:
break

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Continue past blocked upserts to process deletions

During a bulk/startup repair, if vector writes are unavailable and the earliest queue entry is an upsert, this break exits the entire repair loop before later deletion entries are inspected. After an external deletion failure, an older upsert can therefore leave a securely erased or quarantined vector searchable until embedding compatibility is restored, even when the provider could now process deletes; skip blocked upserts while continuing cleanup work.

AGENTS.md reference: AGENTS.md:L172-L173

Useful? React with 👍 / 👎.

Comment thread engraphis/core/engine.py
Comment on lines +3353 to +3356
def validate_promotion() -> Optional[dict]:
replay = command.validate()
if replay is not None:
return replay

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Replay session promotions before checking session activity

An identical retry of a successfully committed session-memory promotion cannot reach this receipt replay after the source session has closed, because the earlier session-status check raises first. This is reproducible by promoting a session memory, ending the session, and repeating the promotion: it raises ValueError despite the committed receipt, so perform replay before applying transient session-state preconditions.

Useful? React with 👍 / 👎.

Comment thread engraphis/core/engine.py
Comment on lines +3672 to +3675
def validate_merge() -> Optional[dict]:
replay = command.validate()
if replay is not None:
return replay

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Replay session merges before checking session activity

A retry of a successful session-scoped merge after that session ends fails at the earlier active-session check and never reaches this durable receipt replay. Thus a caller retrying after a lost response receives an error even though the merge committed; check for the existing command result before requiring the session to remain active.

Useful? React with 👍 / 👎.

Comment thread engraphis/service.py Outdated
if root is None:
raise MemoryConflict("memory was erased while opening history")
members = [record for record in self._chain_for(root, wid)
if (rid is None or record.repo_id == rid)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve wider-scope successors in repository history

When a repository memory is promoted to workspace scope, its successor intentionally has repo_id=None, so this exact-repository filter removes it whenever history is requested with the source repository. The endpoint consequently reports an incomplete lineage and incorrect total_count, while inspect() returns both records for the same request; authorize the root against repo, then retain visible same-workspace lineage members or apply ancestor-scope semantics.

AGENTS.md reference: AGENTS.md:L174-L178

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cc3642d4b7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread engraphis/core/engine.py
Comment on lines +1010 to +1013
if _transactional_validator is not None:
replay = _transactional_validator()
if replay is not None:
return replay

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Replay committed revisions before embedding

When a revision/correction has committed but its response is lost, the durable receipt is not checked until this transactional validator runs; remember_with_resolution() has already called self.embedder.embed(...). If the configured embedder is temporarily unavailable on the retry, the identical request raises that backend error instead of returning its committed result, defeating the new recoverable/idempotent revision contract. Check the command receipt before entering the write preparation path, while retaining this in-transaction validation for races.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c5dc4137ce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

root.setAttribute('aria-busy', 'true');
status.textContent = 'Loading saved versions…';
try {
const params = new URLSearchParams({ workspace, limit: '50' });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include the selected project in history requests

When a project is selected, this request still sends only workspace; MemoryService.memory_history() consequently receives repo=None and follows lineage pointers across every repository in that workspace. For example, a sibling-project record whose metadata points at the inspected memory will appear in the selected project's history, even though the service already supports filtering by repository. Pass the active project into this component and add it to these parameters.

AGENTS.md reference: AGENTS.md:L177-L178

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant