Repository navigation
fix(transform): count computed-column failures - #334
Merged
Merged
Conversation
A failed CEL expression silently skipped its computed column. Each failure now counts in dfe_loader_computed_eval_failures_total, by table, column and stage (compile, context or eval). Each table and column pair logs its first runtime error once per process, across rebuilds. Further failures for a field missing from every message cost a counter increment and an atomic load. They produce no log line. Every loader-owned metric name now comes from metrics::names, which applies the dfe_loader_ or loader_ prefix in one macro. No emitted name changes. The unused Metrics::record_max_dynamic_paths_exceeded is gone, because the inserter emits that counter itself. Metrics still emit both names because the engine's dashboards read buffer_rows, batches_flushed_total, insert_latency_seconds and loader_messages_by_table_total.
catinspace-au
force-pushed
the
fix/computed-metric
branch
from
November 12, 2026 07:06
2bc7f65 to
d749573
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A failed CEL expression dropped the computed column without a metric or log, making it indistinguishable from an unfilled column.
dfe_loader_computed_eval_failures_total{table, column, stage}. The stage iscompile,contextoreval. The counter is registered in the manifest beside the other loader counters.metrics::names, which applies thedfe_loader_orloader_prefix in one macro. No emitted name changes.Metrics::record_max_dynamic_paths_exceededis gone. The inserter emits that counter itself.buffer_rows,batches_flushed_total,insert_latency_secondsandloader_messages_by_table_total.dfe-infra's argocd, helm and docs trees reference none of these metric names.
The evaluate tests use one test table, with each runtime table on its own thread. Rows cover a missing field (6 failures over 2 rebuilds, 1 log line) and an invalid expression (counted per compile). Another row covers two tables failing on one column at once (2 log lines, 2 series).
Done when CI's Quality and Test jobs pass.