Skip to content

Use ConcurrentHashtable for telemetry log deduplication - #12367

Draft
bric3 wants to merge 1 commit into
masterfrom
perf/log-collector-concurrent-hashtable
Draft

Use ConcurrentHashtable for telemetry log deduplication#12367
bric3 wants to merge 1 commit into
masterfrom
perf/log-collector-concurrent-hashtable

Conversation

@bric3

@bric3 bric3 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Replaces LogCollector's ConcurrentHashMap dedup storage with a bounded ConcurrentHashtable entry that carries its own atomic count. Duplicate lookups no longer construct RawLogMessage.

Migrates the direct tests to JUnit 5 and strengthens concurrency, capacity, and drain coverage.

Motivation

Telemetry logging can be reached repeatedly from error paths. The previous duplicate-hit path allocated a RawLogMessage on every call and retained a separate AtomicInteger per distinct message.

Additional Notes

Stacked on #11675.

The eight-thread duplicate-hit benchmark reduced normalized allocation from 48.000 B/op to approximately 0 B/op (about 1e-4 B/op measured).

Contributor Checklist

  • Format the title according to the contribution guidelines
  • Assign the type: and comp: or inst: labels in addition to any other useful labels
  • Avoid using close, fix, or linking keywords when referencing an issue; use solves instead
  • Update CODEOWNERS on source file addition, migration, or deletion
  • Update public documentation with any new configuration flags or behaviors
  • Once approved, use merge queue to merge the PR

@bric3 bric3 added type: feature Enhancements and improvements comp: telemetry Telemetry tag: performance Performance related changes tag: no release notes Changes to exclude from release notes tag: ai generated Largely based on code generated by an AI or LLM and removed type: feature Enhancements and improvements labels Sep 1, 2026
@datadog-datadog-prod-us1-2

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.96 s 13.93 s [-0.5%; +1.0%] (no difference)
startup:insecure-bank:tracing:Agent 12.91 s 13.01 s [-1.4%; -0.1%] (maybe better)
startup:petclinic:appsec:Agent 17.01 s 16.86 s [-0.1%; +1.9%] (no difference)
startup:petclinic:iast:Agent 16.96 s 17.04 s [-1.2%; +0.3%] (no difference)
startup:petclinic:profiling:Agent 16.83 s 16.83 s [-1.1%; +1.0%] (no difference)
startup:petclinic:sca:Agent 17.00 s 16.54 s [+1.7%; +3.9%] (significantly worse)
startup:petclinic:tracing:Agent 16.17 s 16.16 s [-0.9%; +1.0%] (no difference)

Commit: c1af8194 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@bric3

bric3 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T21:19:42.623545Z 1367128 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 13671282ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Base automatically changed from feat/concurrent-hashtable to master September 1, 2026 21:22
@bric3
bric3 force-pushed the perf/log-collector-concurrent-hashtable branch from 1367128 to c1af819 Compare September 4, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: telemetry Telemetry tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes tag: performance Performance related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant