Document that PPL fetch_size limits aggregation buckets - #5671
Conversation
fetch_size applies to the final rows of the query, so on an aggregating query it limits the number of buckets returned rather than sampling documents. The existing docs described it only as a row limit, which reads as DSL size -- where aggregation results are returned in full. Contrast both DSL analogues: unlike terminate_after it does not corrupt aggregate values, and unlike size it does truncate aggregation results. Signed-off-by: Kai Huang <ahkcs@amazon.com>
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Explore these optional code suggestions:
|
| * Kmeans | ||
| * `show datasources` and command | ||
| * SQL queries with `fetch_size` parameter (cursor-based pagination). Note: PPL's `fetch_size` (response size limiting, no cursor) is supported in Calcite Engine. | ||
| * SQL queries with `fetch_size` parameter (cursor-based pagination). Note: PPL's `fetch_size` (response size limiting, no cursor) is supported in Calcite Engine. It limits the final rows of the query, so on an aggregating query it limits the number of buckets returned. |
There was a problem hiding this comment.
As this is PPL limitations doc, Remove following sentence
SQL queries with
fetch_sizeparameter (cursor-based pagination). Note:
Description
fetch_sizeis applied to the final rows of a PPL query, so on an aggregating query it limits the number of aggregation buckets returned rather than sampling documents. The existing docs described it only as "limits the number of rows returned," which reads like DSLsize— where aggregation results come back in full.This adds a short section covering:
fetch_sizeis equivalent to appending| head <fetch_size>, so forstats/timechart/top/rare/patterns ... mode=aggregationit truncates buckets.terminate_after, which undercounts because documents are never collected. But unlike DSLsize, aggregation results are truncated. It matches neither analogue exactly, which is what made the behavior easy to misread.Raised in review of opensearch-project/OpenSearch-Dashboards#12511, where this behavior caused Discover to silently drop buckets from PPL visualizations.
Issues Resolved
None; documentation gap found during review of OpenSearch-Dashboards#12511.
Check List
Docs-only change.
docs/user/interfaces/is not covered by the doctest suite, so no queries here are executed as assertions; the example values were confirmed by hand against a local 3.8.0-SNAPSHOT cluster.