Add OpenTelemetry-native host and process monitoring

[mike-realuptime]

Description

Resolves #14046.

This PR adds OpenTelemetry-native host and process monitoring for Linux and Windows while preserving the existing Prometheus exporter based monitoring paths.

The implementation keeps the final SkyWalking data model and metric aggregation in OAP/MAL. The OpenTelemetry Collector is responsible for collection, normalization of source attributes, transport, and local process cardinality control.

Host monitoring

The existing host MAL rules are extended instead of introducing parallel rules owning the same SkyWalking target metrics:

• vm.yaml supports both:

  • Prometheus node-exporter metrics
  • OpenTelemetry hostmetrics for Linux

• windows.yaml supports both:

  • Prometheus windows-exporter metrics
  • OpenTelemetry hostmetrics for Windows

Equivalent source metrics are mapped to the same canonical SkyWalking metrics only when their semantics match.

The existing node-exporter/windows-exporter monitoring paths remain supported.

Process monitoring

This PR adds:

• process-hostmetrics-linux.yaml
• process-hostmetrics-windows.yaml

Processes are modeled as logical SkyWalking instances using a normalized process name rather than PID.

This avoids PID-driven entity churn and keeps process cardinality bounded.

Multiple operating-system processes belonging to the same normalized process group are aggregated before being sent to OAP. SkyWalking then performs the final metric modeling through MAL.

Process metrics include:

• process count
• thread count
• CPU utilization
• resident memory usage
• resident memory utilization
• open handles / file descriptors
• oldest process uptime

OpenTelemetry Collector

The reference configuration targets OpenTelemetry Collector Contrib 0.158.0.

The Collector:

• uses the hostmetrics receiver
• detects host.name
• adds the SkyWalking job identity
• normalizes process names
• groups equivalent processes
• aggregates process metrics before OTLP export
• sends host and process metrics through OTLP to OAP

Backward compatibility

The existing node-exporter path remains intact.

The existing vm.yaml rule is extended rather than replaced, so existing Linux VM metrics continue using their current canonical SkyWalking metric names.

Node-exporter-specific metrics without an equivalent OpenTelemetry hostmetric remain sourced from node-exporter.

Validation

A new E2E case validates the OpenTelemetry hostmetrics path using OpenTelemetry Collector Contrib 0.158.0.

It verifies:

• Linux host service discovery
• memory metrics
• CPU metrics
• logical CPU count
• normalized CPU utilization
• process logical-instance discovery
• process count aggregation
• thread aggregation
• resident memory
• open handles
• oldest process uptime

The process test starts three operating-system processes with the same executable name and verifies that SkyWalking exposes one logical process instance with a process count of 3.

The E2E test passes with:

12 passed
0 failed
0 skipped

The existing node-exporter path was also regression-tested with the unified vm.yaml, including successful OS_LINUX service discovery and values for host memory, CPU, and node-exporter-specific tcp_alloc.

Scope

This PR focuses on OAP/MAL, Collector configuration, and E2E coverage.

Horizon UI dashboards for the new process metrics will be submitted separately to the SkyWalking Horizon UI repository.

[Copilot]

🟡 Changes recommended

Several YAML files have an incomplete Apache 2.0 license header (missing the leading “Licensed to the Apache Software Foundation (ASF)…” line).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds OpenTelemetry-native host and process monitoring support (Linux + Windows) by extending existing MAL rules to accept either exporter-based metrics or OTel hostmetrics, and introduces a new E2E case that validates the OTel hostmetrics + pre-aggregated process pipeline end-to-end.

Changes:

• Extend Linux vm.yaml and Windows windows.yaml MAL rules to map both exporter and OTel hostmetrics sources into the same canonical SkyWalking meter targets (avoiding rule ownership collisions).
• Add new process MAL rules (process-hostmetrics-{linux,windows}.yaml) modeling normalized process groups as logical SkyWalking instances.
• Add a new E2E case (vm/otel-hostmetrics) including collector config, compose wiring, and workflow integration.

File summaries

File	Description
test/e2e-v2/cases/vm/prometheus-node-exporter/otel-rules/vm.yaml	Extends Linux VM MAL to accept both node-exporter and OTel hostmetrics under vm-monitoring.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/windows.yaml	Adds Windows VM MAL rules for the OTel-hostmetrics E2E case.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/vm.yaml	Adds Linux VM MAL rules for the OTel-hostmetrics E2E case.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/process-hostmetrics-windows.yaml	Adds Windows process MAL rules mapping normalized process groups to instances.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/process-hostmetrics-linux.yaml	Adds Linux process MAL rules mapping normalized process groups to instances.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-collector-config.yaml	Adds an otelcol-contrib config for host + process collection, normalization, grouping, and aggregation.
test/e2e-v2/cases/vm/otel-hostmetrics/expected/process-count-3.yml	Adds an assertion verifying grouped process count aggregation (=3).
test/e2e-v2/cases/vm/otel-hostmetrics/entrypoint.sh	Starts 3 sleep processes and runs the collector in the E2E container.
test/e2e-v2/cases/vm/otel-hostmetrics/e2e.yaml	Defines the new E2E verification steps for hostmetrics + process metrics.
test/e2e-v2/cases/vm/otel-hostmetrics/Dockerfile.otelcol	Builds a minimal image bundling otelcol-contrib plus the E2E entrypoint.
test/e2e-v2/cases/vm/otel-hostmetrics/docker-compose.yml	Wires OAP + BanyanDB + collector container; mounts MAL rules and collector config.
oap-server/server-starter/src/main/resources/otel-rules/windows.yaml	Updates production Windows infrastructure MAL to accept both windows-exporter and OTel hostmetrics sources.
oap-server/server-starter/src/main/resources/otel-rules/vm.yaml	Updates production Linux infrastructure MAL to accept both node-exporter and OTel hostmetrics sources.
oap-server/server-starter/src/main/resources/otel-rules/process-hostmetrics-windows.yaml	Adds production Windows process MAL rules for normalized process groups.
oap-server/server-starter/src/main/resources/otel-rules/process-hostmetrics-linux.yaml	Adds production Linux process MAL rules for normalized process groups.
.github/workflows/skywalking.yaml	Adds the new “VM OpenTelemetry Hostmetrics” E2E job entry and pins otelcol-contrib version via env.

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[wu-sheng]

Thanks for the contribution. I reviewed the current head and reproduced the Collector path with otelcol-contrib:0.158.0. I think the following should be addressed before merge:

1. Fix the MAL unit-test data. All current unit-test jobs fail with 12 EMPTY results. This is not an existing exporter-output regression: the new VM/Windows hostmetrics-only expressions have no matching input in vm.data.yaml and windows.data.yaml. Please extend those fixtures, and add runtime fixtures for the two new process MAL files (which are currently compile-only).

2. The advertised Collector-side process aggregation does not occur. Every scrape still exports three process.count=1 datapoints for the three sleep processes, rather than one process.count=3 datapoint. The datapoints have slightly different timestamps, and aggregate_on_attributes includes the timestamp in its grouping key. MAL then sums the three values within the report, so the final OAP value is correctly 3 and repeated reports are averaged rather than accumulated. Please either remove the ineffective Collector aggregation and document that MAL performs the value aggregation, or implement actual pre-OTLP aggregation and assert the raw Collector output.

3. The native Windows path does not match the MAL inputs. windows.yaml filters/groups system_cpu_time by mode, but hostmetrics emits state; idle is therefore included and the per-mode result collapses. process-hostmetrics-windows.yaml expects process_open_handles, while native Windows hostmetrics emits process.handles (process_handles in MAL). No Windows normalization configuration or Windows test is included.

4. The process MAL rules are not enabled by the default OAP configuration. enabledOtelMetricsRules contains vm and windows, but neither process-hostmetrics-linux nor process-hostmetrics-windows. The E2E masks this by explicitly enabling the Linux process rule. Please either enable them by default or document the required opt-in configuration.

5. The existing node-exporter TopN-with-attributes assertion is regressed. The rewritten cpu_load1 expression removes the previous decorate({ me -> me.attr0 = me.layer.name() }), while the E2E still queries top_n(..., attr0='OS_LINUX'). Restore the decoration or intentionally update the contract and assertion.

6. Several mappings described as equivalent have different semantics:

   • tcp_curr_estab maps only state=established, whereas Tcp_CurrEstab includes ESTABLISHED and CLOSE-WAIT.
   • OTel CPU states interrupt and wait are copied unchanged, while the existing node-exporter label values are irq and iowait.
   • OTel/gopsutil cached memory includes SReclaimable, while node_memory_Cached_bytes does not, so memory_buff_cache changes meaning depending on its source.

7. process.count inherits the {threads} unit. copy_metric copies the descriptor from process.threads; the exact runtime output reports process.count Unit: {threads}. Please set the unit explicitly to {process}.

8. The production/reference configuration and Windows coverage are missing. The only Collector configuration added is explicitly Linux E2E configuration, despite the PR advertising Linux and Windows production support. Please add deployable reference documentation/configuration and Windows OTLP fixture coverage.

The process_memory_utilization * 100 expression is intentional fixed-point scaling for UI precision, so I do not consider it an output bug. Its comment should describe the basis-point scaling, and the corresponding UI expression should divide by 100.