ref(tsdb): remove the dead Redis cmsketch frequency tables - #122479
Open
strongs wants to merge 1 commit into
Open
ref(tsdb): remove the dead Redis cmsketch frequency tables#122479strongs wants to merge 1 commit into
strongs wants to merge 1 commit into
Conversation
RedisTSDB carried a third data type next to counters and set counters, a frequency table backed by a 649-line Lua count-min sketch. It is dead code. Both routing layers send all three frequency models to the dummy backend on write, and SnubaTSDB serves the reads. Neither dedicated TSDB Redis host has run a single eval or evalsha across the full 15 months of telemetry we retain, while hincrby ran trillions of times on the same hosts over the same window. BaseTSDB already declares all four frequency methods as NotImplementedError stubs, so RedisTSDB drops its overrides and inherits those. No caller changes. This frees no memory. The keys with no TTL on the us1 tsdb Redis stay until somebody deletes them, which is a separate decision. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
github-actions
Bot
added
the
Scope: Backend
strongs
marked this pull request as ready for review
August 24, 2026 19:33
valencitas
approved these changes
Aug 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
RedisTSDBcarries a third data type next to counters and set counters: a "frequency table", the thing that answered "which releases show up most often in this issue". It is backed by a Lua count-min sketch, which just so happens to be dead code.The reads moved to Snuba. SnubaTSDB.get_frequency_series serves them off the events dataset. The Redis implementation is no longer on the path for any read.
The writes go to a no-op backend. There are three live frequency models:
frequent_issues_by_project,frequent_releases_by_group,frequent_environments_by_group. Both routing layers agree on where they go, and both send them todummyon write:redissnubafor every cell except US, andgetsentryfor the USServiceDelegator.No Lua script has ever run on a TSDB Redis.
cmsketch.luais the only script the backend loads and the only file insrc/sentry/scripts/tsdb/, so on a dedicated TSDB Redis anyEVALorEVALSHAis a cmsketch call and nothing else. Across the full 15 months of Datadog retention, neither dedicated host reports one, whilehincrbyon the same hosts, over the same window, is in the billions and still climbing. Thathincrbyline is the control: it proves the metric, the host tag and the window all work, because the live counter path used them on the same hosts the whole time.Out of scope
404:,407:and408:prefixes.event_manager.pystill builds a frequencies list on every event and hands it to a no-op backend. That is real waste in a hot path, but it is a separate change with different owners and a different risk profile.Refs INFRENG-458.
🤖 Generated with Claude Code