Skip to content

Commit e5dea1b

Browse files
committed
fix(knowledge): store connector sync schedules at millisecond precision
The scheduler round-trips next_member_sync_at through a JavaScript Date and claims the run by matching the value back exactly. Date carries milliseconds while PostgreSQL stored microseconds, so a schedule written in SQL rather than by the application became unmatchable the moment it landed on a fractional millisecond: the connector stayed permanently due, every claim was refused, and its members never synced. Narrow both connector schedule columns to timestamp(3) so the two ends compare the same value. The rewrite also rounds the stored values, so an already-wedged row recovers on the next scheduler tick. Also stop the claim diagnosis from asserting a queued run it cannot see — it now reads the lock token and says so plainly when no condition explains the refusal, which is what let this hide as ordinary contention.
1 parent bad0ce4 commit e5dea1b

9 files changed

Lines changed: 27913 additions & 27 deletions

File tree

‎.github/workflows/test-build.yml‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -94,11 +94,14 @@ jobs:
9494
working-directory: packages/db
9595
run: bun run db:migrate
9696

97-
- name: Verify retired-column contract migration in PostgreSQL
97+
- name: Verify schema contract migrations in PostgreSQL
9898
working-directory: packages/db
9999
env:
100-
RETIRED_COLUMNS_TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_auth_scim
101-
run: bunx vitest run scripts/retired-columns.postgres.test.ts
100+
MIGRATION_CONTRACT_TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_auth_scim
101+
run: >-
102+
bunx vitest run
103+
scripts/retired-columns.postgres.test.ts
104+
scripts/connector-sync-schedule-precision.postgres.test.ts
102105
103106
- name: Verify OAuth lifecycle and SCIM membership guards in PostgreSQL
104107
working-directory: apps/sim

‎apps/sim/lib/knowledge/connectors/member-queue.ts‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,7 @@ async function describeUnacceptedMemberSync(
161161
memberSyncStatus: knowledgeConnector.memberSyncStatus,
162162
nextMemberSyncAt: knowledgeConnector.nextMemberSyncAt,
163163
syncLockToken: knowledgeConnector.syncLockToken,
164+
memberSyncLockToken: knowledgeConnector.memberSyncLockToken,
164165
archivedAt: knowledgeConnector.archivedAt,
165166
deletedAt: knowledgeConnector.deletedAt,
166167
})
@@ -181,7 +182,14 @@ async function describeUnacceptedMemberSync(
181182
) {
182183
return 'The member sync schedule changed after this run was scheduled'
183184
}
184-
return 'A member sync is already queued or running for this connector'
185+
if (row.memberSyncLockToken) {
186+
return 'A member sync is already queued or running for this connector'
187+
}
188+
/**
189+
* No checked condition explains the refusal. Naming a cause here instead once hid a connector
190+
* that was refused on every attempt for days, because the reason read as ordinary contention.
191+
*/
192+
return 'The connector refused the claim while no lock or status explains it'
185193
}
186194

187195
/**
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
-- The scheduler round-trips these schedules through a JavaScript `Date` and claims a run by
2+
-- matching the value back exactly. `Date` carries milliseconds while PostgreSQL stores
3+
-- microseconds, so a schedule written in SQL rather than by the application became unmatchable the
4+
-- moment it landed on a fractional millisecond: the connector stayed permanently due and every
5+
-- claim was refused. Narrowing to the precision the round trip can carry makes both ends compare
6+
-- the same value, and rounds the stored values so an already-wedged row recovers. Both columns are
7+
-- narrowed in one statement so the rewrite is a single pass. `knowledge_connector_member`'s
8+
-- `next_attempt_at` is written by the same backfill but is only ever range-compared, never claimed
9+
-- by equality, so it is left alone.
10+
--
11+
-- migration-safe: every deployed reader and writer of these columns goes through a JavaScript
12+
-- `Date`, which already truncates to milliseconds, so the discarded precision is unobservable to
13+
-- the running version and this needs no earlier deploy.
14+
ALTER TABLE "knowledge_connector"
15+
ALTER COLUMN "next_member_sync_at" SET DATA TYPE timestamp (3),
16+
ALTER COLUMN "next_sync_at" SET DATA TYPE timestamp (3);

0 commit comments

Comments
 (0)