Skip to content

fix(datasource): wait for the discovered list to settle before onboarding - #19

Open
mivds wants to merge 1 commit into
mainfrom
fix/discovered-list-lag
Open

mivds wants to merge 1 commit into
mainfrom
fix/discovered-list-lag

Conversation

@mivds

@mivds mivds commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator

Problem

Soda Cloud is moving the discovered-datasets list to a read replica, so the list can trail the database by a few seconds. Two commands assume it is always current.

sodacli datasource onboard stops waiting at the first non-empty list. If the replica has copied only part of a discovery, the user is offered only some of the tables to onboard.

sodacli dataset list hides discovered datasets that are marked onboarded. Right after an onboarding, the replica still marks them as not onboarded, so for a few seconds they show up twice: once as onboarded and once as not onboarded.

Change

datasource onboard waits until the discovered count is the same on two polls in a row, which costs one extra poll of 5 seconds. The 5-minute timeout stays.

dataset list also hides a discovered dataset when its qualified name matches an onboarded dataset in the same output.

Testing

go build, go vet and the offline CLI tests pass. There are no unit tests for these commands, and I have not run them against a live org.

🤖 Generated with Claude Code

…ding

The discovered-datasets list can trail the discovery that fills it, and
its onboarded flag can trail an onboarding. datasource onboard now waits
until the count is the same on two polls in a row, and dataset list no
longer shows a just-onboarded dataset a second time as not onboarded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mivds mivds added the bug Something isn't working label Oct 22, 2026
@mivds mivds self-assigned this Oct 22, 2026
@mivds
mivds marked this pull request as ready for review October 22, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant