Skip to content

fix: always append the unique __id tie-breaker to the sort so paginated results are deterministic - #122

Open
ThibaudDauce wants to merge 1 commit into
mainfrom
fix/deterministic-order-for-pagination
Open

fix: always append the unique __id tie-breaker to the sort so paginated results are deterministic#122
ThibaudDauce wants to merge 1 commit into
mainfrom
fix/deterministic-order-for-pagination

Conversation

@ThibaudDauce

Copy link
Copy Markdown

Downloading the metrics CSV of an organization from the front gives duplicated rows. And it's worse than duplicates: some rows are simply missing.

Following the links.next like the front does, on organization 534fff94a3a7292c64a77fc1 (355 pages):

sort rows received distinct rows
metric_month__sort=desc (the one used by the front) 17748 13533
__id__sort=asc 17748 17748

The front asks for metric_month__sort=desc, but metric_month is not unique (~1500 rows share the same month here) and we only added the __id tie-breaker when no sort was requested. Postgres doesn't guarantee any order between the ties, and it changes from one query to the next, so with an offset pagination some rows come back on the following page and others are never returned.

__id.asc is now always appended at the end of the ORDER BY, except with aggregators since it's not in the GROUP BY.

The sorts are also gathered in a single order= now: two __sort in the same query were producing two competing order= parameters and only one was kept.

Nothing to change on the front, metric_month__sort=desc can stay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant