Add BanyanDB trace tail sampling monitoring - #14052
Merged
Merged
Conversation
Add otel-rules/banyandb/banyandb-trace-sampling.yaml, 41 rules covering the banyandb_trace_pipeline_* / banyandb_trace_tst_pipeline_* catalog a sampler plugin chain emits. Modeled at Service scope with `group` kept as a metric label rather than at Endpoint scope, so one cluster-wide page can render per-group series and cluster totals alike. Add the e2e case test/e2e-v2/cases/banyandb/trace-sampling, registered in the e2e-test matrix. It runs a real sw-trace-sampler chain on the plugin-capable BanyanDB image and asserts the 17 rules a healthy, fully-dropping run can produce; the other 24 cover failure counters, the keep verdict, FINALIZE-only state and back-pressure guards that cannot occur in a short healthy run. The existing BanyanDB self-observability case cannot cover any of this: its cluster runs no sampler plugins, so the wire families never register. Fix a cross-rule CounterWindow key collision in the v2 MAL engine. rate() / increase() / irate() resolved their lower bound from a process-wide window keyed on the counter name plus its post-.sum(...) label set, with nothing identifying the evaluating rule, so two rules reading one wire family and dropping the label they filtered on collapsed onto one slot and differenced against each other's values. The window is now keyed by owning rule as well. Auditing the shipped rules gives 10 colliding keys over ~25 rules. Fix meter_rabbitmq_node_outgoing_messages_total double-counting rabbitmq_global_messages_delivered_get_auto_ack_total, which appeared twice among the six summed delivery terms.
There was a problem hiding this comment.
🔵 Needs a closer look
It changes core v2 MAL rate/increase/irate windowing semantics that can affect many existing meter rules, so it warrants final human verification despite added regression coverage.
Pull request overview
This PR extends SkyWalking’s self-observability for BanyanDB by adding a dedicated MAL/OTel ruleset and an end-to-end e2e case for trace tail-sampling (sampler plugin chain) metrics, while also fixing a correctness bug in the v2 MAL counter windowing logic and correcting a RabbitMQ node metric overcount.
Changes:
- Add
banyandb-trace-sampling.yaml(41 rules) to expose BanyanDB trace tail-sampling pipeline/plugin metrics at Service scope (cluster), retaininggroupas a metric label. - Add a new e2e-v2 case (
test/e2e-v2/cases/banyandb/trace-sampling) and register it in the GitHub Actions e2e matrix to validate key healthy “fully-dropping” metric families end-to-end. - Fix v2 MAL
CounterWindowkey collisions across rules by including the owning rule (output metric name) in the window key; fix RabbitMQ outgoing message delivery double-counting.
File summaries
| File | Description |
|---|---|
| test/e2e-v2/cases/banyandb/trace-sampling/trace-sampling-cases.yaml | Defines swctl queries/assertions for the trace sampling monitoring e2e scenario. |
| test/e2e-v2/cases/banyandb/trace-sampling/otel-collector-config.yaml | Collector config to scrape BanyanDB Prometheus metrics and inject cluster label for rule evaluation. |
| test/e2e-v2/cases/banyandb/trace-sampling/expected/labels-group.yml | Expected shape for group-labeled time series results. |
| test/e2e-v2/cases/banyandb/trace-sampling/expected/labels-group-result.yml | Expected shape for group+result-labeled time series results. |
| test/e2e-v2/cases/banyandb/trace-sampling/expected/labels-group-plugin.yml | Expected shape for group+plugin_name-labeled time series results. |
| test/e2e-v2/cases/banyandb/trace-sampling/expected/labels-group-plugin-rule.yml | Expected shape for group+plugin_name+rule-labeled time series results. |
| test/e2e-v2/cases/banyandb/trace-sampling/expected/labels-group-plugin-rule-verdict.yml | Expected shape for group+plugin_name+rule+verdict-labeled time series results. |
| test/e2e-v2/cases/banyandb/trace-sampling/expected/labels-group-plugin-result.yml | Expected shape for group+plugin_name+result-labeled time series results. |
| test/e2e-v2/cases/banyandb/trace-sampling/expected/labels-group-plugin-p.yml | Expected shape for histogram percentile results with p label (per plugin). |
| test/e2e-v2/cases/banyandb/trace-sampling/expected/labels-group-p.yml | Expected shape for histogram percentile results with p label (chain-level). |
| test/e2e-v2/cases/banyandb/trace-sampling/expected/labels-group-lane-p.yml | Expected shape for histogram percentile results split by lane. |
| test/e2e-v2/cases/banyandb/trace-sampling/expected/active-samplers.yml | Expected output asserting sampler activation (1 vs 0) per trace group. |
| test/e2e-v2/cases/banyandb/trace-sampling/e2e.yaml | New e2e-v2 harness for trace sampling monitoring validation (retry/timeout tuned for merge + scrape + bucket). |
| test/e2e-v2/cases/banyandb/trace-sampling/docker-compose.yml | Compose topology for plugin-capable BanyanDB + OTel collector + OAP + trace mocker. |
| oap-server/server-starter/src/main/resources/otel-rules/rabbitmq/rabbitmq-node.yaml | Removes a duplicated delivery counter term in outgoing_messages_total. |
| oap-server/server-starter/src/main/resources/otel-rules/banyandb/banyandb-trace-sampling.yaml | Adds the new BanyanDB trace tail sampling rule catalog (service-scoped, group retained as label). |
| oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/CrossRuleCounterWindowTest.java | Regression test proving cross-rule counter window isolation for increase() on frozen counters. |
| oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/counter/CounterWindowTest.java | Updates unit test calls for the new (owner, name, labels, ...) CounterWindow API. |
| oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/SampleFamily.java | Passes rule identity (context.metricName) into increase/rate/irate windowing calls. |
| oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/Sample.java | Extends counter window calls to include owning rule for proper keying. |
| oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/counter/ID.java | Adds owner field to CounterWindow key material (rule identity). |
| oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/counter/CounterWindow.java | Keys counter windows by (owner, counter name, labels) to avoid cross-rule collisions. |
| oap-server/analyzer/meter-analyzer-scripts-test/src/test/resources/scripts/mal/test-otel-rules/rabbitmq/rabbitmq-node.data.yaml | Updates expected value for RabbitMQ outgoing messages after de-duplication. |
| docs/en/changes/changes.md | Changelog entries for the new BanyanDB sampling metrics, CounterWindow fix, and RabbitMQ rule correction. |
| docs/en/banyandb/tail-sampling.md | Points operators to the OAP-collected sampling metric catalog and anchor link in dashboards doc. |
| docs/en/banyandb/dashboards-banyandb.md | Documents the full trace tail sampling metric catalog and adds the rule file to the BanyanDB list. |
| .github/workflows/skywalking.yaml | Registers the new BanyanDB trace sampling monitoring e2e case in CI matrix. |
Review details
- Files reviewed: 27/27 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
wu-sheng
approved these changes
Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add otel-rules/banyandb/banyandb-trace-sampling.yaml, 41 rules covering the banyandb_trace_pipeline_* / banyandb_trace_tst_pipeline_* catalog a sampler plugin chain emits. Modeled at Service scope with
groupkept as a metric label rather than at Endpoint scope, so one cluster-wide page can render per-group series and cluster totals alike.Add the e2e case test/e2e-v2/cases/banyandb/trace-sampling, registered in the e2e-test matrix. It runs a real sw-trace-sampler chain on the plugin-capable BanyanDB image and asserts the 17 rules a healthy, fully-dropping run can produce; the other 24 cover failure counters, the keep verdict, FINALIZE-only state and back-pressure guards that cannot occur in a short healthy run. The existing BanyanDB self-observability case cannot cover any of this: its cluster runs no sampler plugins, so the wire families never register.
Fix a cross-rule CounterWindow key collision in the v2 MAL engine. rate() / increase() / irate() resolved their lower bound from a process-wide window keyed on the counter name plus its post-.sum(...) label set, with nothing identifying the evaluating rule, so two rules reading one wire family and dropping the label they filtered on collapsed onto one slot and differenced against each other's values. The window is now keyed by owning rule as well. Auditing the shipped rules gives 10 colliding keys over ~25 rules.
Fix meter_rabbitmq_node_outgoing_messages_total double-counting rabbitmq_global_messages_delivered_get_auto_ack_total, which appeared twice among the six summed delivery terms.
If this is non-trivial feature, paste the links/URLs to the design doc.
Update the documentation to include this new feature.
Tests(including UT, IT, E2E) are added to verify the new feature.
If it's UI related, attach the screenshots below.
If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
Update the
CHANGESlog.