fix(file-search): store workspace file revisions at millisecond precision by waleedlatif1 · Pull Request #7941 · simstudioai/sim

waleedlatif1 · 2026-09-17T20:41:18Z

Summary

workspace_files.content_updated_at is a revision identity that round-trips through JavaScript Date and JSON (both millisecond), while now() stores microseconds. A sub-millisecond revision stops comparing equal to its own round-trip, so every SQL equality keyed on it matches zero rows — the file is never claimed, never indexed, and never cleaned up, because the only cleanup path (markTerminal) is itself gated behind being dispatched.
Migration 0357 truncates the column default and adds a BEFORE INSERT OR UPDATE trigger for the writers a default cannot reach: explicit CURRENT_TIMESTAMP expressions, raw SQL inserts, and any UPDATE. It is DDL-only, so it stays idempotent under the autocommit path a fresh install takes after 0355's embedded COMMIT, and holds its lock for DDL rather than for a data rewrite.
Script migration 0018 retires the rows minted before the invariant existed, keyset-paginated and batched. Each batch is one transaction so the secret_provenance_version that workspace_files_secret_provenance_demote nulls is restored with its sidecar, instead of being lost if a run dies midway.
Scoped to context = 'workspace', live and soft-deleted: those are the rows the search index keys on, and a soft-deleted row is restored by paths that do not rewrite the column. Other contexts are never search-indexed and only compare this column in JavaScript, where both sides are already millisecond; the trigger normalizes them on their next content write.
The trigger name sorts before workspace_files_secret_provenance_demote, so that trigger's WHEN clause compares the normalized value and a write that does not change the millisecond revision no longer demotes provenance. There is a test for both directions.
No application code changes. Complements fix(file-search): bound dispatcher claims and skip locked rows #7940, which bounds the claim query: that made the dispatcher cheap, this makes the rows claimable.

Type of Change

Bug fix

Testing

10 PostgreSQL 17 tests covering the invariant (default expression, explicit microsecond write, CURRENT_TIMESTAMP restore write, trigger ordering vs provenance demotion in both directions) and the repair (scoping, sidecar realignment, orphaned index/segment rows, replay no-op, keyset progress). Wired into the existing packages/db CI step.
Verified the repair against a seeded 5.6M-row workspace_files: 94,196 rows repaired in 2.4s, replay a 0.5s no-op, other contexts untouched, tracked provenance versions and sidecars preserved.
Verified 0357 applies and replays cleanly both in autocommit and inside a transaction.
bun run lint, all 46 audits, docs-manifest:check, check:migrations origin/staging, and type-check pass.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…sion The content revision round-trips through JavaScript Date and JSON, both of which truncate to milliseconds, while now() stores microseconds. A sub-millisecond revision stops comparing equal to its own round-trip, so every SQL equality keyed on it matches zero rows and the file is never claimed, indexed, or cleaned up. Truncate at the column default and enforce it with a BEFORE trigger for the writers a default cannot reach, then retire the rows minted before the invariant existed in a batched script migration.

vercel · 2026-09-17T20:41:24Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 17, 2026 9:03pm UTC

greptile-apps · 2026-09-17T20:45:23Z

Confidence Score: 5/5

The PR appears safe to merge; no unresolved correctness, migration-safety, security, or repository-rule issue was identified.

Summary

This PR makes workspace-file content revisions safe to round-trip through JavaScript’s millisecond-precision timestamps and repairs legacy workspace rows.

Changes the column default and adds a database trigger that normalizes revisions on inserts and updates.
Adds a bounded, resumable script migration that repairs legacy revisions while preserving provenance and removing obsolete search projections.
Adds PostgreSQL coverage for precision enforcement, trigger ordering, repair scope, replay safety, and projection cleanup.
Runs the new PostgreSQL suite in the existing database CI job.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[SQL migration 0358] --> B[Install millisecond-normalizing trigger]
    B --> C[New inserts and updates store millisecond revisions]
    A --> D[Run script migration 0018]
    D --> E[Select bounded legacy workspace-file page]
    E --> F[Lock selected files in ID order]
    F --> G[Normalize revisions]
    G --> H[Realign provenance sidecars]
    H --> I[Restore tracked provenance versions]
    I --> J[Remove obsolete search index and segment rows]
    J --> K{More legacy rows?}
    K -- Yes --> E
    K -- No --> L[Record script migration complete]

_{Reviews (3) · Last reviewed commit: "chore(db): renumber the content revision..."}

cubic-dev-ai

All reported issues were addressed across 8 files

_{Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Fix all with cubic | Re-trigger cubic}

A chat upload is materialized into a workspace file by setting context alone, so the revision is never written and the search-index trigger would key the promoted file to a value nothing can claim. Fire the normalizer on every update; `UPDATE OF` is evaluated against the statement's target columns, so this does not demote provenance on a metadata write. Accept the shared integration database in the repair suite the way the sibling suites do, and record the new script migration in the registry order assertions.

waleedlatif1 · 2026-09-17T20:53:51Z

@greptile

waleedlatif1 · 2026-09-17T20:53:52Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-17T20:53:58Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 10 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic}

Staging took 0357 for the outbox autovacuum migration. The SQL is byte for byte the previously reviewed file; only the number and the references to it changed.

waleedlatif1 · 2026-09-17T21:01:59Z

@greptile

waleedlatif1 · 2026-09-17T21:02:00Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-17T21:02:07Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 10 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic}

vercel Bot deployed to Preview September 17, 2026 20:42 View deployment

greptile-apps Bot reviewed Sep 17, 2026

View reviewed changes

Comment thread .github/workflows/test-build.yml

cubic-dev-ai Bot reviewed Sep 17, 2026

View reviewed changes

Comment thread .github/workflows/test-build.yml

Comment thread packages/db/migrations/0357_workspace_file_content_version_precision.sql Outdated

Comment thread packages/db/script-migrations/index.ts

vercel Bot deployed to Preview September 17, 2026 20:54 View deployment

cubic-dev-ai Bot reviewed Sep 17, 2026

View reviewed changes

waleedlatif1 added 3 commits September 17, 2026 13:59

chore(db): drop the 0357 migration ahead of renumbering

4ad1df1

Merge staging into fix/workspace-file-content-revision-precision

dfbfb4b

chore(db): renumber the content revision migration to 0358

cc01970

Staging took 0357 for the outbox autovacuum migration. The SQL is byte for byte the previously reviewed file; only the number and the references to it changed.

vercel Bot deployed to Preview September 17, 2026 21:03 View deployment

cubic-dev-ai Bot reviewed Sep 17, 2026

View reviewed changes

waleedlatif1 merged commit 7931dcc into staging Sep 17, 2026
34 checks passed

waleedlatif1 deleted the fix/workspace-file-content-revision-precision branch September 17, 2026 21:10

waleedlatif1 mentioned this pull request Sep 17, 2026

v0.8.42: knowledgebase queueing improvemnets, file search time fixes, outbox fixes #7945

Merged

Navigation Menu

fix(file-search): store workspace file revisions at millisecond precision - #7941

fix(file-search): store workspace file revisions at millisecond precision#7941
waleedlatif1 merged 5 commits into
stagingfrom
fix/workspace-file-content-revision-precision

waleedlatif1 commented Sep 17, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Sep 17, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Sep 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Sep 17, 2026

Uh oh!

waleedlatif1 commented Sep 17, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 17, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

waleedlatif1 commented Sep 17, 2026

Uh oh!

waleedlatif1 commented Sep 17, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 17, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

waleedlatif1 commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Sep 17, 2026

Uh oh!

waleedlatif1 commented Sep 17, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 17, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Sep 17, 2026

Uh oh!

waleedlatif1 commented Sep 17, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 17, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

waleedlatif1 commented Sep 17, 2026 •

edited

Loading

vercel Bot commented Sep 17, 2026 •

edited

Loading

greptile-apps Bot commented Sep 17, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading