feat(eventing): add typed signal-to-event projection architecture - #363
Draft
robbiemu wants to merge 13 commits into
Draft
feat(eventing): add typed signal-to-event projection architecture#363robbiemu wants to merge 13 commits into
robbiemu wants to merge 13 commits into
Conversation
robbiemu
force-pushed
the
codex/issue-222-alerting-core
branch
from
August 11, 2026 22:35
6fb2377 to
2f5ac1c
Compare
robbiemu
force-pushed
the
codex/issue-222-alerting-core
branch
from
August 11, 2026 22:40
2f5ac1c to
0212b99
Compare
# Conflicts: # apps/api/src/services/alerts/AlertsService.ts # apps/api/src/services/integrations/planetscale/PlanetScaleWebhookQueue.ts # apps/cli/src/server/serve.ts # apps/cli/test/server-network.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a host-neutral typed signal-to-event projection architecture and uses it for three producer paths:
It also completes the transport-neutral Maple Local outbox boundary with durable named-consumer leases and acknowledgements.
The latest commits expand the generic GitLab vertical to its complete frozen factual vocabulary: project, issue, comment, merge request, pipeline, deployment, job, ref, and release facts. This remains generic Maple functionality. The PR does not implement Matrix delivery, local room policy, deployment-specific paths, credentials, or agent authorization.
Related to #222.
Why there are two event paths
Immediate per-occurrence path
authenticated OTLP/provider input → decode once → typed normalized signal → bounded selector → pure projector → durable outbox
The original telemetry continues through the warehouse encoder. In Maple Local a matched event is staged before the chDB write and marked ready only after the warehouse write succeeds. A retry with the same source occurrence recomputes the same Maple event ID.
Scheduled aggregate path
warehouse query → observation → alert lifecycle evaluation → factual alert event → existing delivery outbox
Rates, thresholds, percentiles, absence, recovery, and flap suppression remain scheduled conclusions over a window. They are not modeled as individual ingest-time facts.
Core architecture
Source adapters normalize authenticated inputs into typed signals and publish a field catalog. Projection configuration stores a bounded typed predicate AST. Projection revisions compile into immutable registry snapshots after source fields/operators and projector configuration are validated.
Projectors are pure, versioned functions: they declare an ID/version, source kinds, output type/schema, and closed configuration decoder. They perform no I/O and do not create issues, route rooms, send messages, or mutate providers.
Canonical CloudEvents and three-level identity
Maple event IDs remain SHA-256 hashes over a length-delimited tuple of tenant, source kind, source, source occurrence ID, projection ID, and projection revision.
This PR adds two optional backward-compatible CloudEvents extensions:
sourceoccurrenceid: source delivery/record identity;sourceidentityquality:source,derived, ornone.New projected events carry both. Historical envelopes lacking them remain valid. A bridge can persist source occurrence ID → immutable Maple CloudEvent ID → its own deterministic transport transaction ID without parsing event data.
For GitLab, the producer uses the stable hook UUID (
X-Gitlab-Event-UUIDor equivalent) and indexes multi-record deliveries as<delivery-id>:<zero-based-index>. OTLP carries it asevent.idandgitlab.event.id; payload hashes are audit-only.Complete GitLab v1 factual contracts
The original
gitlab.issue.created@1projector anddev.maple.gitlab.issue.created.v1behavior remain intact. New families require positivegitlab.project.id, boundedgitlab.project.path, and one explicitevent.name. Project ID is routing identity; paths are mutable metadata.gitlab.project.lifecycle@1dev.maple.gitlab.project.lifecycle.v1project_create,project_update,project_rename,project_transfer,project_archive,project_unarchive,project_deletion_request,project_destroygitlab.issue.lifecycle@1dev.maple.gitlab.issue.lifecycle.v1issue_open,issue_update,issue_close,issue_reopengitlab.issue.comment@1dev.maple.gitlab.issue.comment.v1issue_commentgitlab.merge-request.lifecycle@1dev.maple.gitlab.merge-request.lifecycle.v1merge_request_open,merge_request_update,merge_request_close,merge_request_reopen,merge_request_merge,merge_request_reviewgitlab.merge-request.comment@1dev.maple.gitlab.merge-request.comment.v1merge_request_comment,merge_request_review_commentgitlab.pipeline.completed@1dev.maple.gitlab.pipeline.completed.v1ci_pipeline_completedwith `successgitlab.deployment.lifecycle@1dev.maple.gitlab.deployment.lifecycle.v1deployment_running,deployment_success,deployment_failed,deployment_canceled,deployment_blocked,deployment_manualgitlab.job.lifecycle@1dev.maple.gitlab.job.lifecycle.v1ci_job_<status>allowlistgitlab.ref.lifecycle@1dev.maple.gitlab.ref.lifecycle.v1gitlab.release.lifecycle@1dev.maple.gitlab.release.lifecycle.v1release_create,release_update,release_deleteThe ref vocabulary preserves the production receiver contract: raw
push,tag_push, andrepository_updatedeliveries normalize and deduplicate into six semantic branch/tag transitions before Maple.Additional bounded facts include:
mergeRequestIid, and up to 20 failed-job summaries with total/truncated metadata;Every data payload contains
project{id,path,oldPath?},sourceEvent, its relevant bounded factual object, and optional actor/result/service name. All action/status vocabularies are explicit and fail closed. Deployment/job source actions must agree with their status fields.Data-safety bounds
The GitLab projector boundary enforces:
The machine-readable output set contains 11 complete CloudEvents plus a companion identity fixture. Tests recompute every event ID from its checked-in occurrence/projection tuple and validate every envelope.
Durable Local outbox and consumers
Maple Local stores immutable projection revisions, active pointers, bounded projection failures, staged/ready events, and durable consumer state in a private SQLite control database.
The existing draft-branch migration adds one strict
event_consumerstable and index while preserving projection/outbox rows. Named consumers support explicitbeginningorlatestregistration, whole-batch claims under bounded leases, exact acknowledgement, replay after expiry, fail-closed stale/wrong/partial acknowledgement, and pruning through the lowest active-consumer acknowledgement. Claims use a separate least-privilege credential.Checkpoint manifests bind the control snapshot alongside the chDB backup. The latest GitLab expansion changes no storage schema or consumer API.
Existing producer convergence
Verified PlanetScale webhooks run through a registered source adapter/selector/projector before durable queueing while retaining current downstream behavior.
The host-neutral alert core still owns scheduled aggregate observation evaluation, trigger/resolve/renotify planning, flap suppression, no-data safety, scheduling helpers, delivery idempotency, and retry policy. Existing alert delivery payloads carry an additive factual alert CloudEvent.
Deliberate boundaries
This PR does not:
The GitLab receiver owns webhook authentication/normalization, source UUID indexing, duplicate semantic-transition suppression, bounded excerpt preparation, failed-job lookup/truncation, and emission of the documented fields. Maple validates and projects facts that are present.
Review guide
Primary surfaces:
packages/eventing-core: model, predicates, source/projector registries, deterministic identity, optional source-identity extensions, schemas and fixtures;apps/cli/src/server/eventing: OTLP normalization, GitLab projectors, runtime, SQLite state, outbox and consumer protocol;apps/cli/src/server/serve.ts: decode-once integration and authenticated control/consumer endpoints;apps/cli/src/server/checkpoints.ts: eventing-control checkpoint participation;packages/alerting-coreand hosted alert services;docs/gitlab-event-projectors.mdanddocs/signal-to-event-projection.md.Validation
Published head:
b25fe1661516334be31a856c1713a2fd225c1a86.git diff --checkpassed.mainquery-engine errors only: missingtoStartOfMinuteinservices.ts, and a missingCompiledQuerytype argument inports.ts. No current error points to this PR's eventing changes.