Repository navigation
Conversation
TagMap bucketed a known tag by folding its serial, so which tags shared one of the 16 buckets was luck, and each shared bucket costs a BucketGroup (48 B). Slot coloring already gives tags that appear together on a span type distinct slots, so bucketing a known tag by its slot keeps them in distinct buckets more often. Trace-level tags reuse span-level slot numbers, so the level bit moves them to the other half; known tags without a slot and custom tags keep their hashes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SpanScenarioBenchmark builds spans and traces the way today's decorators do (prototype in afterStart, then request or connection tags by name): bare spans, a server span, the server span via the builder, a jdbc span, and a server trace with 1, 5 or 20 jdbc children. TraceShapeBenchmark builds one web-request-shaped trace three ways: every tag by name, the decorators' calls, and the newest span APIs. Both finish what they start, so allocation per op covers whole spans and traces. Single-span results can hide per-child costs that only the trace scenarios show. TraceShapeBenchmark is the same file #12918 adds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TraceShapeBenchmark takes the number of jdbc children (1, 5, 20), so each arm shows how a trace's cost grows with its span count: single-span and small-trace results can hide costs that land on every child span. It covers what SpanScenarioBenchmark's trace scenario added, and the rest of that benchmark overlapped SpanCreationBenchmark (same scenario names, different semantics), so SpanScenarioBenchmark goes. TagMapBucketHashTest pins down the bucket hash: known tags get distinct, nonzero hashes, a slotted tag lands in its slot's bucket (trace-level in the other half), and tags without a slot and custom tags keep their hashes. The inline comment now says the level bit flips bit 3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HeaderPathBenchmark runs one request per op through the propagation paths: extract the inbound headers with the default propagator, start the server span from what was extracted, run a few children, inject into an outbound call, finish everything. Inbound headers vary: none, Datadog, W3C (with dd, OpenTelemetry and third-party tracestate members), both, and both plus baggage. The existing propagation benchmarks each measure one step, so they can't see costs the extracted state causes later on every span; this one can. A setup check fails the run if nothing is extracted or injected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
setTag(tag, null) already removes a tag from a span, but TagMap's own setters stored a null-valued entry. Internal writers that go straight to a TagMap could therefore leave null tags on a span, and the serializer sent them as a msgpack nil under meta. TagMap.put / set / getAndSet (by name or id, and putAll / fromMap through set) now remove the tag on a null value and return its prior value. A null Entry or reader is still a no-op. The one writer that relied on null tags was payload tagging: a JSON null in a request or response body produced a tag with a null value. It now produces no tag, so such a value also no longer counts toward the max-tags limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kets Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 23, 2026
This comment has been minimized.
This comment has been minimized.
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
Eager post-processors run on the application thread as a span finishes. They now take a SpanStorage (a span's tags as stored) instead of a TagMap, and DDSpanContext implements SpanStorage directly, so no TagMap view needs to be built for them once span storage stops being a single TagMap. SpanStorage has its own verbs, load and store, apart from TagMap's get/set and the span's intercepted getTag/setTag, so DDSpanContext can offer the raw and the intercepted operations without a clash. Stores bypass interception; thread safety is left to each implementation (DDSpanContext's are unsynchronized, called under the tag lock). TagMap implements it too, through its own get and set, for standalone maps. This is the first slice: load, loadString and two stores. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pure move: the loops over one bucket array (find, put, remove, copy, merge, the forEach variants, fillMap / fillStringMap, count, isEmpty, the bucket-position check, toInternalString) become static methods of HashSupport, moved verbatim. Each reports what it did through its return value (the entry put replaced, the entry remove removed, how much merge grew the map), and TagMap keeps its fields, its size counter, and the read-through and tombstone handling around them. No new objects or fields, and no behavior change: allocation per trace in TraceShapeBenchmark matches the base within fork-to-fork noise, and the TagMap and core span suites pass unchanged. This prepares the split of known and custom tags into separate stores, which reuse these mechanics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dougqh
force-pushed
the
dougqh/tagmap-store-facade
branch
from
October 23, 2026 16:57
75ec723 to
77b8ac8
Compare
IdStore keeps known tags by id; its one implementation today is HashIdStore, built through IdStoreFactory so a JVM loads a single IdStore subclass and calls on it stay devirtualized. HashStore keeps custom tags by name. Both reuse HashSupport's bucket mechanics and read through to a frozen parent's store, hiding an inherited tag with a tombstone: one shared, immutable Entry per known tag (Entry.tombstoneOf) in the id store, and a removed-name set in the name store. The hash id store's buckets are its exact state, so it needs no slot masks. Nothing uses the stores yet: TagMap and the span keep their current storage. IdStoreContractTest pins the stores' behavior directly. The only TagMap changes are to its nested Entry, EntryReader and BucketGroup, which the stores share: the tombstone flag and a lookup of a bucket chain by id. HashSupport.checkBuckets now indexes a group's entries by entry rather than by bucket, so the integrity check really inspects groups. The tag registry gains _dd.agent_psr as a trace-level known tag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TagMap keeps known tags in an IdStore (by id) and custom tags in a HashStore (by name), instead of one bucket array. A name is classified once, at the boundary, and routed to its store; known and custom tags never share a key, so read-through and precedence work per store and never cross between them. The name store is created on the first custom write; until then a map borrows its read-through parent's store read-only. Behavior is unchanged: the TagMap suites pass as they are, and Entry objects are still shared across copies. The span still holds a single TagMap; it starts owning the stores directly in the next change. TagMap.Internals stays for now, writing straight to the id store, and goes away with that change. Reads by id go straight to the id store. stream() and spliterator() yield stable Entries, and getEntryReader gives a momentary reader for typed reads. TagStoresTest covers the stores through TagMap, and IdStoreContractTest gains the view cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…agmap-hash-support
Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com>
A span isn't conceptually a TagStore: the store is the storage beneath it, as a Map was beneath a TagMap. It implements one only so post-processing can be handed its storage without allocating, since the processor chain's dispatch loop keeps escape analysis from removing a wrapper. Drop the TagStore Javadoc's note about this being a first slice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ssors-span-context
…ssors-span-context # Conflicts: # dd-trace-core/src/main/java/datadog/trace/core/tagprocessor/HttpEndpointPostProcessor.java
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…agmap-hash-support
EagerTagsPostProcessor and EagerPostProcessorChain still called it storage after the rename to TagStore, while every processor calls it unsafeTags. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…agmap-hash-support
parentEntryVisible still masked the hash into a bucket by hand before calling findInBucket, the one place outside HashSupport that knew the bucket array's power-of-2 layout. It now calls HashSupport.find, and findInBucket becomes private. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…asses # Conflicts: # internal-api/src/main/java/datadog/trace/api/HashSupport.java
…acade # Conflicts: # internal-api/src/main/java/datadog/trace/api/TagMap.java
HashSupport.forEach checked every entry's tombstone flag and walked bucket groups through the _entryAt switch, on the path the serializer takes for every span, though only HashIdStore ever holds a tombstone. forEach goes back to the unrolled BucketGroup walk with no tombstone check, for TagMap and HashStore. HashIdStore uses the new forEachLive, which skips tombstones, and only while its hasTombstones hint is set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HashCursor held a separate BucketWalker, so a cursor cost two allocations where TagMap's iterator costs one. It now extends the walker, whose step becomes nextEntry so it can't be mistaken for the cursor's filtering next, and next() returns Entry covariantly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The coverage check flagged IdStoreFactory, which no test built a store through, and HashSupport.BucketWalker, which only a small cursor test reached. IdStoreFactoryTest builds stores through the JVM's factory, checking behavior rather than the implementation, and a contract test walks a cursor over every bucket shape and a parent level. HashStoreTest grows to the typed setters, removal, copying, merging, sizing and a read-through over bucket groups. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
TagMapkeeps known tags in anIdStore(by id) and custom tags in aHashStore(by name), instead of one bucket array.stream()andspliterator()yield stableEntrys, andgetEntryReadergives a momentary reader for typed reads.TagMap.Internalsstays for now, writing straight to the id store. It goes away in the next PR, when the span owns its stores.TagMap's new view methods.Motivation
This is the second of three PRs that replace the span's single
TagMapwith two stores split by tag kind.TagMaphad picked up name canonicalization, known-versus-custom classification and direction awareness. Moving storage into the stores leavesTagMapa thin facade, and lets the span use the same stores directly in the next PR.Additional Notes
TagMapsuites pass as they are, including theInternalstests, which now exercise the shim.Entryobjects are still shared across copies.TagStoresTestcovers the stores throughTagMap, andIdStoreContractTestgains the view cases.TracerConnectionReliabilityTestcases, which need Docker.Where this is going
This PR is one step in replacing the span's single
TagMapwith storage that knows each tag's kind. The goal is to cut what every span allocates on the application thread, without changing behavior.The end state
TagMapbecomes a thin view for code outside the span.Entryobject per tag, and each span's store is presized from a per-operation hint. With the flag on, a web-request-shaped trace allocates about 30% less on the application thread than on master.TagStore, not aTagMap. That covers post-processors now and serializers later, so noTagMapview gets built for them.TagStorehas its own verbs,loadandstore, apart fromTagMap'sget/setand the span's interceptedgetTag/setTag, soDDSpanContextcan offer both. The tag stores themselves use the same verbs (Add id and name tag stores, not yet used (dense store - part 5) #13006). It starts with what the eager processors use, by name and by id, and grows typed-load and primitive-store variants. Its intercepted counterpart,SpanData(Take the span's tag lock once in setTagsFrom (tag api - part 1) #13030), is whatsetTagsFrom,TagContributorandTagExtractortake, with one lock per batch.Later, not in this stack: cheaper read-through to parent spans, trace-level tags in a per-trace holder (
TraceSegment), and primitive tag values without boxing.The stack. This replaces #12918, split into pieces that can each be reviewed, and measured, on their own.
TagMap. This is the stack's only behavior change, kept separate on purpose.TagMap, and addTraceShapeBenchmarkTagStorefor the eager post-processorsTagMap's bucket mechanics intoHashSupport(a pure move)TagMapa facade over the two storestrace.experimental.dense.tags.enabled(off by default)Every PR after #12999 is behavior-preserving.
The pieces' allocation figures come from a deterministic per-thread probe (
ThreadMXBeanallocated bytes) that builds the same 7-span trace asTraceShapeBenchmark. The #12992 and #12926 figures come fromTraceShapeBenchmarkitself.🤖 Generated with Claude Code