Fix telemetry aggregation and add lab-aware device health detection#708
Merged
Conversation
PyMongo's native async collection methods return awaitables for index creation and aggregation. Calling them without await leaves indexes uncreated and makes aggregation endpoints fail when they try to consume a coroutine as a cursor. Await both operations and add focused tests that verify the collection calls and cursor conversion are awaited. Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com>
Device-type aggregation can hide a single dead board behind healthy peers, while mixing job and test events dilutes boot failures with many per-test results. Add a physical-device anomaly scope backed only by job results, expose device_id in stats, and return constant-failure flags plus the latest error context. Document the public query and cover grouping, filtering, classification, and validation. Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com>
LAVA runtime names currently identify labs, and device anomalies already use runtime as part of their grouping key. Exposing only the runtime field makes that isolation easy for API consumers to miss. Add lab as an explicit stats filter and grouping alias, include it in anomaly rows, reject conflicting lab/runtime filters, and document that results from different labs remain separate. Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com>
The HTML page handlers duplicate template path resolution, file reads, response construction, metrics, and cache headers. Some also use PlainTextResponse while manually overriding the content type. Introduce an allowlisted template helper that consistently returns HTMLResponse, centralize the cache policy and template directory, and reuse it across public pages. Update the stale root test and cover page content types, cache headers, and rejection of non-allowlisted templates. Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com>
Running the focused API checks currently requires manually creating isolated MongoDB and Redis services, assembling API and core dependencies in a disposable Python environment, and cleaning up each resource. Add a configurable Docker-based runner that performs the targeted pytest suite and pinned Ruff checks with read-only source mounts and guaranteed cleanup. It accepts custom pytest arguments and supports alternate core checkout and image paths through environment variables. Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com>
nuclearcat
force-pushed
the
remove-cloudevents
branch
from
July 21, 2026 04:40
78fc20a to
35ef89e
Compare
MongoDB rejects creation when the same index definition already exists under a different name. Staging retains the event-history TTL definition as ttl_timestamp, so startup fails with IndexOptionsConflict when the API requests the generated timestamp_1 name. Inspect existing index keys and behavior-affecting options before creation, reuse only equivalent definitions regardless of name, and recover from equivalent indexes created concurrently. Preserve startup failures for genuinely different TTL, uniqueness, or filter options. Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes broken telemetry aggregation after the PyMongo async migration and adds public API support for identifying physical devices that consistently fail to boot.
Device health is grouped by lab, device type, and physical device ID, ensuring that a successful boot in one lab does not hide failures in another.
Also we optimize HTML templates and add local tests.