Skip to content

Fix zero-offset limit skipping first deduplicated row - #5701

Open
ruanyl wants to merge 1 commit into
opensearch-project:mainfrom
ruanyl:codex/fix-limit-dedupe-first-row
Open

Fix zero-offset limit skipping first deduplicated row#5701
ruanyl wants to merge 1 commit into
opensearch-project:mainfrom
ruanyl:codex/fix-limit-dedupe-first-row

Conversation

@ruanyl

@ruanyl ruanyl commented Aug 18, 2026

Copy link
Copy Markdown
Member

Description

Fix LimitOperator.open() so a zero offset does not probe its child iterator.

When PPL fetch_size adds a zero-offset limit around a stateful operator such as
DedupeOperator, the existing condition calls input.hasNext() even though no
rows need to be skipped. DedupeOperator.hasNext() consumes and caches the first
unique row, so the subsequent normal iteration replaces it with the second row
and omits the first result.

This change checks the offset before calling the child iterator and adds a
regression test covering a zero-offset limit around deduplication.

Related Issues

Resolves #5699

Related: opensearch-project/OpenSearch-Dashboards#12606

Testing

  • Confirmed the new regression test fails on main with expected: <3> but was: <2>.
  • ./gradlew :core:test
  • ./gradlew :core:spotlessCheck
  • ./gradlew :core:test --tests org.opensearch.sql.planner.physical.LimitOperatorTest

Check List

  • New functionality includes testing.
  • New functionality has been documented. Not applicable; this restores expected iterator behavior.
  • New functionality has javadoc added. Not applicable.
  • New functionality has a user manual doc added. Not applicable.
  • New PPL command checklist all confirmed. Not applicable.
  • API changes companion pull request created. Not applicable.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created. Not applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Yulong Ruan <ruanyl@amazon.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@github-actions

Copy link
Copy Markdown
Contributor

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Increment counter after consuming row

The count variable is incremented before consuming the row with input.next(). This
creates a logical inconsistency where count represents rows to skip rather than rows
already skipped. Consider incrementing count after input.next() to maintain semantic
clarity and prevent potential off-by-one errors in future modifications.

core/src/main/java/org/opensearch/sql/planner/physical/LimitOperator.java [42-45]

 while (count < offset && input.hasNext()) {
+  input.next();
   count++;
-  input.next();
 }
Suggestion importance[1-10]: 3

__

Why: While the suggestion improves semantic clarity by incrementing count after consuming the row, the current implementation is functionally correct. The order of count++ and input.next() doesn't affect the behavior since both operations must complete within the loop iteration. This is a minor style improvement rather than a bug fix.

Low

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Dashboard variables definition with PPL on prometheus datasource swallows first row

1 participant