Skip to content

feat(eventing): add typed signal-to-event projection architecture - #363

Draft
robbiemu wants to merge 13 commits into
MapleTechLabs:mainfrom
robbiemu:codex/issue-222-alerting-core
Draft

feat(eventing): add typed signal-to-event projection architecture#363
robbiemu wants to merge 13 commits into
MapleTechLabs:mainfrom
robbiemu:codex/issue-222-alerting-core

Conversation

@robbiemu

@robbiemu robbiemu commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR introduces a host-neutral typed signal-to-event projection architecture and uses it for three producer paths:

  • immediate facts derived from authenticated telemetry, demonstrated in Maple Local with normalized GitLab OTLP logs;
  • verified provider events, demonstrated by PlanetScale webhooks; and
  • scheduled aggregate alerts, which keep their query-driven lifecycle while producing the common CloudEvents envelope.

It also completes the transport-neutral Maple Local outbox boundary with durable named-consumer leases and acknowledgements.

The latest commits expand the generic GitLab vertical to its complete frozen factual vocabulary: project, issue, comment, merge request, pipeline, deployment, job, ref, and release facts. This remains generic Maple functionality. The PR does not implement Matrix delivery, local room policy, deployment-specific paths, credentials, or agent authorization.

Related to #222.

Why there are two event paths

Immediate per-occurrence path

authenticated OTLP/provider input → decode once → typed normalized signal → bounded selector → pure projector → durable outbox

The original telemetry continues through the warehouse encoder. In Maple Local a matched event is staged before the chDB write and marked ready only after the warehouse write succeeds. A retry with the same source occurrence recomputes the same Maple event ID.

Scheduled aggregate path

warehouse query → observation → alert lifecycle evaluation → factual alert event → existing delivery outbox

Rates, thresholds, percentiles, absence, recovery, and flap suppression remain scheduled conclusions over a window. They are not modeled as individual ingest-time facts.

Core architecture

Source adapters normalize authenticated inputs into typed signals and publish a field catalog. Projection configuration stores a bounded typed predicate AST. Projection revisions compile into immutable registry snapshots after source fields/operators and projector configuration are validated.

Projectors are pure, versioned functions: they declare an ID/version, source kinds, output type/schema, and closed configuration decoder. They perform no I/O and do not create issues, route rooms, send messages, or mutate providers.

Canonical CloudEvents and three-level identity

Maple event IDs remain SHA-256 hashes over a length-delimited tuple of tenant, source kind, source, source occurrence ID, projection ID, and projection revision.

This PR adds two optional backward-compatible CloudEvents extensions:

  • sourceoccurrenceid: source delivery/record identity;
  • sourceidentityquality: source, derived, or none.

New projected events carry both. Historical envelopes lacking them remain valid. A bridge can persist source occurrence ID → immutable Maple CloudEvent ID → its own deterministic transport transaction ID without parsing event data.

For GitLab, the producer uses the stable hook UUID (X-Gitlab-Event-UUID or equivalent) and indexes multi-record deliveries as <delivery-id>:<zero-based-index>. OTLP carries it as event.id and gitlab.event.id; payload hashes are audit-only.

Complete GitLab v1 factual contracts

The original gitlab.issue.created@1 projector and dev.maple.gitlab.issue.created.v1 behavior remain intact. New families require positive gitlab.project.id, bounded gitlab.project.path, and one explicit event.name. Project ID is routing identity; paths are mutable metadata.

Projector CloudEvent type Accepted normalized source events
gitlab.project.lifecycle@1 dev.maple.gitlab.project.lifecycle.v1 project_create, project_update, project_rename, project_transfer, project_archive, project_unarchive, project_deletion_request, project_destroy
gitlab.issue.lifecycle@1 dev.maple.gitlab.issue.lifecycle.v1 issue_open, issue_update, issue_close, issue_reopen
gitlab.issue.comment@1 dev.maple.gitlab.issue.comment.v1 issue_comment
gitlab.merge-request.lifecycle@1 dev.maple.gitlab.merge-request.lifecycle.v1 merge_request_open, merge_request_update, merge_request_close, merge_request_reopen, merge_request_merge, merge_request_review
gitlab.merge-request.comment@1 dev.maple.gitlab.merge-request.comment.v1 merge_request_comment, merge_request_review_comment
gitlab.pipeline.completed@1 dev.maple.gitlab.pipeline.completed.v1 ci_pipeline_completed with `success
gitlab.deployment.lifecycle@1 dev.maple.gitlab.deployment.lifecycle.v1 deployment_running, deployment_success, deployment_failed, deployment_canceled, deployment_blocked, deployment_manual
gitlab.job.lifecycle@1 dev.maple.gitlab.job.lifecycle.v1 explicit ci_job_<status> allowlist
gitlab.ref.lifecycle@1 dev.maple.gitlab.ref.lifecycle.v1 `branch_create
gitlab.release.lifecycle@1 dev.maple.gitlab.release.lifecycle.v1 release_create, release_update, release_delete

The ref vocabulary preserves the production receiver contract: raw push, tag_push, and repository_update deliveries normalize and deduplicate into six semantic branch/tag transitions before Maple.

Additional bounded facts include:

  • issue ID/IID/title/URL/state and up to 50 labels of 256 UTF-8 bytes each;
  • merge-request IID/title/URL/branches/commit/review state;
  • sanitized issue/MR comment excerpts and canonical comment URLs, never full bodies;
  • canonical pipeline URL, optional unambiguous mergeRequestIid, and up to 20 failed-job summaries with total/truncated metadata;
  • deployment identity/environment/status/revision/URL;
  • job identity/name/stage/status/URL/pipeline/ref/revision/duration/allow-failure;
  • branch/tag before and after revisions, including all-zero SHAs;
  • release ID/tag/name/URL, excluding arbitrary descriptions.

Every data payload contains project{id,path,oldPath?}, sourceEvent, its relevant bounded factual object, and optional actor/result/service name. All action/status vocabularies are explicit and fail closed. Deployment/job source actions must agree with their status fields.

Data-safety bounds

The GitLab projector boundary enforces:

  • scalar text ≤ 4 KiB;
  • comment excerpt ≤ 1,024 UTF-8 bytes after control-character removal and whitespace normalization;
  • canonical HTTP(S) URL ≤ 2,048 bytes, without credentials/query strings (comment note anchors are allowed);
  • positive object IDs and non-negative counts/durations;
  • hexadecimal revisions, including zero SHAs;
  • failed jobs ≤ 20 with only ID/name/stage/status/canonical URL;
  • issue labels ≤ 50, each ≤ 256 bytes;
  • no variables, logs, secrets, raw payloads, full comments, or arbitrary descriptions.

The machine-readable output set contains 11 complete CloudEvents plus a companion identity fixture. Tests recompute every event ID from its checked-in occurrence/projection tuple and validate every envelope.

Durable Local outbox and consumers

Maple Local stores immutable projection revisions, active pointers, bounded projection failures, staged/ready events, and durable consumer state in a private SQLite control database.

The existing draft-branch migration adds one strict event_consumers table and index while preserving projection/outbox rows. Named consumers support explicit beginning or latest registration, whole-batch claims under bounded leases, exact acknowledgement, replay after expiry, fail-closed stale/wrong/partial acknowledgement, and pruning through the lowest active-consumer acknowledgement. Claims use a separate least-privilege credential.

Checkpoint manifests bind the control snapshot alongside the chDB backup. The latest GitLab expansion changes no storage schema or consumer API.

Existing producer convergence

Verified PlanetScale webhooks run through a registered source adapter/selector/projector before durable queueing while retaining current downstream behavior.

The host-neutral alert core still owns scheduled aggregate observation evaluation, trigger/resolve/renotify planning, flap suppression, no-data safety, scheduling helpers, delivery idempotency, and retry policy. Existing alert delivery payloads carry an additive factual alert CloudEvent.

Deliberate boundaries

This PR does not:

  • implement Matrix delivery, room/Space topology, or agent policy;
  • add environment-specific paths, room IDs, credentials, or deployment configuration;
  • call GitLab from a projector or reconstruct missing producer fields;
  • add NATS/Kafka or another required broker;
  • replace scheduled aggregate alerts with ingest selectors;
  • expose arbitrary SQL or executable projection configuration;
  • claim exactly-once external side effects;
  • activate projections automatically.

The GitLab receiver owns webhook authentication/normalization, source UUID indexing, duplicate semantic-transition suppression, bounded excerpt preparation, failed-job lookup/truncation, and emission of the documented fields. Maple validates and projects facts that are present.

Review guide

Primary surfaces:

  • packages/eventing-core: model, predicates, source/projector registries, deterministic identity, optional source-identity extensions, schemas and fixtures;
  • apps/cli/src/server/eventing: OTLP normalization, GitLab projectors, runtime, SQLite state, outbox and consumer protocol;
  • apps/cli/src/server/serve.ts: decode-once integration and authenticated control/consumer endpoints;
  • apps/cli/src/server/checkpoints.ts: eventing-control checkpoint participation;
  • packages/alerting-core and hosted alert services;
  • PlanetScale webhook runtime/queue;
  • docs/gitlab-event-projectors.md and docs/signal-to-event-projection.md.

Validation

Published head: b25fe1661516334be31a856c1713a2fd225c1a86.

  • Full Maple CLI suite with loopback listeners enabled: 478 passed, 0 failed across 35 files.
  • Focused CLI eventing integration: 38 passed, 0 failed.
  • Eventing core: schema freshness + 29 passed, 0 failed; package typecheck passed.
  • Alerting core: 10 passed, 0 failed; package typecheck passed.
  • GitLab projector suite: 15 passed, 0 failed, covering all explicit events/actions, invalid fields, labels, excerpts, URLs, failed-job bounds, pipeline→MR association, status agreement, semantic ref transitions, indexed source occurrence IDs, and all 11 output fixtures.
  • Source lint, formatting, generated-schema check, and git diff --check passed.
  • CLI-wide typecheck reaches two pre-existing refreshed-main query-engine errors only: missing toStartOfMinute in services.ts, and a missing CompiledQuery type argument in ports.ts. No current error points to this PR's eventing changes.

@robbiemu robbiemu changed the title refactor(alerting): extract a host-neutral alert core feat(eventing): add typed signal-to-event projection architecture Aug 8, 2026
@robbiemu
robbiemu force-pushed the codex/issue-222-alerting-core branch from 6fb2377 to 2f5ac1c Compare August 11, 2026 22:35
@robbiemu
robbiemu force-pushed the codex/issue-222-alerting-core branch from 2f5ac1c to 0212b99 Compare August 11, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant