[mbani01]

Why

The nightly shadowDiff runs 10 channels at a time under a 30-minute execution timeout. That fits today's 110 mirrored repos (6-8 minutes a night) but not the automated mirror sweep that follows this PR: its first night alone brings the pool to ~610 channels, which is 35-45 minutes at the current fan-out. The diff would time out, write no sync_diff_summary rows for the day, and the switch sweep's gate would then refuse every repo for lack of a diffed day.

What

• CHANNEL_CONCURRENCY 10 → 50 in workflows/shadowDiff.ts (exported, sized for ~5,000 channels in about an hour).
• connectors-shadow-diff schedule workflowExecutionTimeout 30 minutes → 4 hours.
• On ScheduleAlreadyRunning the worker now reconciles the existing schedule's action (same pattern as the docs-readiness schedules), so the new timeout applies to the live schedule after deploy instead of only on first creation.
• One workflow test: 120 channels fan out to a peak of exactly 50 in flight, and a failed channel is counted without stopping the run.

Rollout

Deploy before the mirror sweep (next PR) runs its first night.

Refs linuxfoundation/lfx-cdp-insights-ops#833

[cursor]

PR Summary

Medium Risk
Raises nightly shadow-diff load (50 parallel channels) and changes Temporal schedule behavior on deploy; failures are isolated per channel but longer runs increase worker/activity pressure.

Overview
Scales nightly shadow diff capacity so larger channel pools can finish within the scheduled window: parallel channel processing rises from 10 to 50 (CHANNEL_CONCURRENCY, now exported), and the Temporal schedule’s workflow execution timeout moves from 30 minutes to 4 hours.

Schedule registration refactors the start-workflow action into a shared constant and, when the schedule already exists (ScheduleAlreadyRunning), updates the live schedule to reconcile the action (timeout and workflow config) instead of only logging—so deploys pick up the new limits without recreating the schedule.

Adds a workflow unit test that asserts batched concurrency (peak 50 in flight over 120 channels) and that one channel failure is counted without aborting the run.

^{Reviewed by Cursor Bugbot for commit 2666125. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

🟢 Approval recommended

The changes are focused, regression coverage addresses the concurrency behavior, and no blocking issues were identified.

No open issues 🎉

What changed in this PR

Scales nightly shadow diffs to support the upcoming mirror sweep and a larger channel pool.

Changes:

• Raises channel concurrency from 10 to 50.
• Extends the execution timeout to four hours and reconciles existing schedules.
• Adds coverage for concurrency limits and continued processing after a channel failure.

File	Description
services/​apps/​connectors_worker/​src/​workflows/​shadowDiff.ts	Exports and raises the concurrency limit.
services/​apps/​connectors_worker/​src/​workflows/​shadowDiff.test.ts	Tests 120 channels, peak concurrency, and failure counting.
services/​apps/​connectors_worker/​src/​schedules/​shadowDiff.ts	Updates the timeout for new and existing schedules.

_{Review effort: Balanced}