Repository navigation
feat(guardrails): guardrail events reach the platform with fleet counts, and credentials in tool output are hidden from the model (abilityai/trinity-enterprise#165) - #4425
Conversation
… its store (Abilityai/trinity-enterprise#165) One row per guardrail event an agent reported, unique per (agent, event_id) so a resend is idempotent (ON CONFLICT DO NOTHING ... RETURNING on both dialects). SQLite migration plus Alembic 0123 chained after 0122 (one head), DDL in schema.py/tables.py, indexes for the per-agent read, the per-execution counts and the retention sweep. execution_id carries no FK. Rows follow the agent through AGENT_REFS (CASCADE: renamed with it, purged with its hard delete).
…/trinity-enterprise#165) POST /api/agents/{name}/guardrail-events takes a batch from the agent's own key only (the heartbeat predicate, outside get_current_user): 413 past 500 events or 256 KiB, 422 only for a malformed envelope, 429 + Retry-After past the per-agent cap. Each event is validated on its own: an unknown kind or hook is stored as 'other' with the raw value, code is an open string, a malformed event is counted in 'rejected' and never fails the batch, an over-long execution_id is nulled. Only allowlisted fields are kept, free text sanitized and bounded. GET is owner tier with the uniform 404 and returns events newest first, a keyset cursor and the reporting state (never / reporting / stale) kept in Redis and cleared on teardown. count_by_execution and count_by_agent are the seams the execution-status and fleet consumers call. Both route censuses list the routes.
…us guard (Abilityai/trinity-enterprise#165) guardrail_event_retention_days (default 90, 0 disables) is a registered retention window: validated write path, reported by GET /api/settings/retention, not a community-floor key. The cleanup cycle prunes rows past it by the server receive time (an agent-reported timestamp must not make a row immortal), behind its own _guard_allows site, in bounded chunks.
…rchitecture catalogs and the guardrails flow (Abilityai/trinity-enterprise#165)
…cycle harness (Abilityai/trinity-enterprise#165) The happy-path harness mocks db wholesale, so a new sweep's prune count must be a real int or the WAL-checkpoint sum raises. Same shape as the ent#433 and ent#478 sweeps; the test now also asserts the guardrail prune is reached.
…ls its batch or the owner's read (Abilityai/trinity-enterprise#165) A NUL in a raw-stored field made psycopg2 refuse the whole multi-row insert, and a lone surrogate did the same on SQLite: the batch 500ed on every retry and the forwarder stalled behind it. Inside details a surrogate was stored JSON-escaped and then made every later GET of that agent's events fail while the response was encoded. Display text (tool, verb, tier, turn_kind, rule, path) now carries U+FFFD in place of such a character; an id (execution_id, session_id, tool_use_id) that holds one is dropped, like an over-long one.
…dy is in (Abilityai/trinity-enterprise#165) Auth was decided before the body was read, so a body streamed slowly could outlive an agent delete or rename: its rows landed under the retired name after the cascade, and its report re-marked that name as reporting after the reporter was cleared, so a later agent of the same name would read as covered. The key rows follow the agent (cascaded on delete, re-keyed on rename), so the second check refuses the write with the same 403.
…er rule in the guardrails flow (Abilityai/trinity-enterprise#165)
…e time, not counted in every window (Abilityai/trinity-enterprise#165) An agent whose clock runs ahead sent events whose occurred_at sat after every rolling-window cutoff until the row was pruned, so the 24 h counts (and any since-read) kept counting them. Ingest now clamps occurred_at to received_at when the agent's ts is later, and keeps the agent's own value in details.occurred_at_reported. The event is never rejected: a skewed clock must not cost the agent its record.
…ed, so history before it is not read as clean (Abilityai/trinity-enterprise#165) A consumer of reporting could not tell a run from before the agent's first report (never covered) from a covered run with no events: both read as zero. The reporter hash now records first_report_at with HSETNX (first write wins), and reporting carries it: the stored value, else the oldest stored row's received_at when the fast store lost it, else the last report. Every fallback is at or after the true first report, so a lost fast store only moves coverage later, the safe direction. It is null exactly when the state is never. The response model carries the field so the GET does not drop it.
…ARD-005, the API catalog and the guardrails flow (Abilityai/trinity-enterprise#165)
… after the last report, and is read before the counts (Abilityai/trinity-enterprise#165) Three gaps a second review found in first_report_at, each proven by a test that failed first: - A recycled name inherited its predecessor's reporter record when the teardown's fail-open Redis clear failed, or an ingest in flight wrote it back, so a new agent's unreported history read as covered. The create path now clears it beside the #1560 breakers, before the container. - Two racing reports could leave the start after a later-dated last report. The start is capped at last_report_at: the agent did report then, so the earlier value is still at or after the truth. - count_by_execution read the counts before the coverage, so an agent's first batch landing between the two reads paired a clean 0 with the coverage that batch set up. Coverage is now read first.
… the read order in GUARD-005 and the guardrails flow (Abilityai/trinity-enterprise#165)
…ilityai/trinity-enterprise#165) The guardrail hooks' records never left the container: §28.2 said they were logged to Vector, which reads container stdout only. State what the spool record carries, what leaves the container, how the forwarder delivers, and the loss window and authenticity limits. Also corrects the hook file names (.py, not .sh) and the scanner's pattern claim.
…y deny (Abilityai/trinity-enterprise#165) lib.log_event now adds v, event_id, a derived kind and hook, the turn's execution_id (read from the claude process's launch environment), and the hook input's session and tool-use ids. kind and hook are closed sets owned by lib.py and derived through one frozen table, so later hook events need no lib.py edit. The existing record fields are unchanged. Every deny call site passes a stable code. The turn-tier guard records the turn's kind as turn_kind, since kind is now the record's own field. The image smoke checks a deny record carries the contract, and the skill gate's log comment no longer claims Vector reads the file.
…e agent's own key (Abilityai/trinity-enterprise#165) A new agent-server loop tails /logs/guardrails.jsonl from a persisted offset and posts an allowlisted, sanitized projection of each guardrail event: never the command text, a traceback, error text or the deny prose. Delivery is debounced to within 2 s of an append, since no shutdown flush runs (bash is PID 1 and has no SIGTERM handler). Batches are bounded by count and bytes; 413 halves the batch, 422 drops and counts it, 429 honours Retry-After, 404/401/403 keep the spool and retry hourly, 5xx backs off with jitter. An empty batch every 5 minutes is a keepalive. The spool rotates at 5 MiB and stays bounded while the platform is unreachable, with the overflow reported as dropped. Gated on the heartbeat's rule: no agent key, no loop. Fails open.
… the forwarder (Abilityai/trinity-enterprise#165) A hook input can carry an escaped lone surrogate, for example in a model-written file path. It survives the spool's json round trip, but httpx encodes the request body as strict UTF-8, so the POST raised before it was sent, on every attempt. The batch holding that record never left, the records after it never did either, and the keepalives stopped with them. The projection now replaces a lone surrogate with U+FFFD in text and drops an id that holds one, the rule the platform's ingest applies. The test fake encodes bodies with httpx's own encoder, so the suite sees the real wire.
…nted, not retried forever (Abilityai/trinity-enterprise#165) A JSON integer too large for a float made math.isfinite raise OverflowError in the projection, outside every guard, so the forwarder step failed on the same line every few seconds and nothing after it was delivered. The check now sits inside the guard, and the line is counted in `dropped` like any other record that does not parse into an event.
…ated spool (Abilityai/trinity-enterprise#165) Two corner cases of the spool rotation: - A hook write cut short (a full disk, a kill mid-write) left a line with no newline at the end of `.1`. Nothing appends to `.1` once its grace is over, so the forwarder waited there, delivering none of the live spool until the next 5 MiB rotation, while its keepalives kept the agent reading as reporting. Past the grace that line is now counted in `dropped` and the forwarder moves on. - A second rotation inside the first one's 2 s grace replaced `.1` while a hook could still be appending to it, so that write landed in an unlinked file, neither delivered nor counted. Rotation now waits out the grace.
…ord check (Abilityai/trinity-enterprise#165) - 401/403 keep the spool and retry hourly. Only 404 was pinned, so a 403 that fell through to the 5xx backoff stayed green. The platform answers 403 to a key that is not the agent's own and to one revoked while the body uploads. - The image smoke's deny-record check is called from main() after the bash rows, and it fails a record missing any one of v, event_id, kind, hook or code. Before, dropping the call or the kind check stayed green.
…l tail in the guardrails flow (Abilityai/trinity-enterprise#165)
…urfaces with honest empty states (Abilityai/trinity-enterprise#165)
…sted with its coverage (Abilityai/trinity-enterprise#165)
… Recent events in Settings > Guardrails, with honest empty states (Abilityai/trinity-enterprise#165)
…the catalogs, the guardrails flow and the user docs (Abilityai/trinity-enterprise#165)
…uardrail events surfaces (Abilityai/trinity-enterprise#165)
…in the flow and user docs (Abilityai/trinity-enterprise#165)
…its output schema requires (Abilityai/trinity-enterprise#165) The CLI drops a hook rewrite that fails the tool output schema and shows the original output, so a withhold with the wrong field types fails open. The Read withhold tests checked field names only, and nothing exercised the WebFetch number fix-up; both now fail when those fields stop being integers.
…what the server returns (Abilityai/trinity-enterprise#165) The scanner skips every mcp__trinity__ tool. The docs said that output is platform-authored and sanitized by the backend; get_agent_logs returns an agent's raw container logs, so that is not true for every tool. The real reason for the skip is that some tools hand the agent a credential on purpose.
…the scanner's shared left boundary (Abilityai/trinity-enterprise#165)
…165-guardrail-events
…165-guardrail-events # Conflicts: # docs/memory/feature-flows.md
…165-guardrail-events # Conflicts: # docker/base-image/hooks/guard002-smoke.py # docs/memory/feature-flows.md # docs/memory/feature-flows/agent-guardrails.md # tests/registry.json
…bilityai/trinity-enterprise#165) GET /api/ops/guardrail-stats is require_person and picks the fleet-wide roster with is_fleet_admin, so an agent key never reaches fleet_agent_names. List it in the fleet-reach guard's allowlist with that reason.
… agent's own key (Abilityai/trinity-enterprise#165) POST /api/agents/{name}/guardrail-events reads the bearer itself and accepts only the agent's own key, like the heartbeat. List it with that reason and drive it in the door sweep: the ops_write key gets the own-key 403, the agent's key gets 200.
… not wall time (Abilityai/trinity-enterprise#165) A shared CI runner stalled the 8 MiB backslash case to 11.3 s of wall time for a scan that takes about 2.7 s of CPU. Measure the hook process's CPU so the bound tests the scanner, not the runner.
|
Heads-up from an automated build chain (trinity-pm): #4431 also adds an Alembic revision on |
|
|
|
Resolve by running |
|
Resolve by merging |
dev now carries 0123_agent_ownership_default_model on 0122_channel_client_blocks, so the guardrail event store's revision moves to 0124_guardrail_events, chained after it, and the version-line keeps one head. The SQLite entry follows dev's agent_ownership_default_model. Conflicts kept both sides: file-guardrail.py has dev's .env denial sentence and the placeholder write-back refusal; tests/registry.json holds both sets of entries.
…pe can start (Abilityai/trinity-enterprise#165) Output that is all backslashes or `ESC [` passes the left boundary at every position, so the one matcher tried all fourteen shapes eight million times on an 8 MiB result. On a CI runner that hook took 10.9 s of CPU, against a 30 s registered timeout and the test's 10 s bound; the previous change to CPU time had assumed a stalled runner, and the CPU figure shows the cost was the scan. When every shape shows the character it starts with, a lookahead on those characters now follows the boundary. No shape can match anywhere else, so the matches are unchanged; a position that starts none of them fails at one test. The reader is conservative: an optional first character, a top-level `|`, or a comment, flag or conditional group reads as "not shown", and then the matcher has no lookahead. In process, 8 MiB of backslashes drops from 1.4 s to 0.5 s and `ESC [` from about 2 s to 0.7 s; the whole hook on backslashes from 2.8 s to 0.9 s of CPU. Tests pin the lookahead for the shipped shapes, the reader on each trap with a counterexample the engine matches, the fallback without it, and identical spans with and without it.
|
Thanks — #4431 landed first, so this PR moved: |
Brings #4447 (httpcore pinned against a sys.modules split). No conflicts.
…65-guardrail-events # Conflicts: # docs/memory/feature-flows.md
…SK (Abilityai/trinity-enterprise#165) CodeQL read the variable name SECRET as sensitive data and flagged the persisted-output check, which writes that made-up key into a temp file the smoke deletes. The value and every check are unchanged.
Closes abilityai/trinity-enterprise#165 (GUARD-004 and GUARD-005). This PR carries the issue's whole scope. The keyword names another tracker, so it does not move the issue's status; that is set by hand after merge.
Draft until the manual UI sign-off is given. The two new UI surfaces (the Guardrails card on Operations › Health and Recent events in Settings › Guardrails) have not been visually accepted in a browser, in either theme. This PR stays a draft until that sign-off.
Order and couplings.
devand depends on no other open PR.devso the full CI runs on it, and it therefore also carries this PR's commits. Merge this one first.tests/registry.json; whichever of the two merges second keeps both sets of entries (the only conflict between them, checked withgit merge-tree).0124_guardrail_events, whose parent isdev's head0123_agent_ownership_default_model(feat(agents): an agent's own default model, one resolver for every turn path (ent#1202) #4431). One head. The SQLite entryguardrail_events_tablefollowsagent_ownership_default_model.Why
The guardrail hooks wrote every decision (a Bash or file deny, a turn-tier refusal) only to
/logs/guardrails.jsonlinside the agent container. Nothing read that file, it is lost on recreate, and it is readable only while the container runs. So the platform could not say that a run was refused, or that an agent had stopped reporting: #4281 is the concrete case, 78 hook denials over about 25 hours while every execution readsuccess.The PostToolUse output scanner only logged a credential match. The model still received the value, and from there it could reach a reply, a commit or the next tool call.
This PR makes guardrail decisions a platform record with an honest fleet view, and turns the scanner into an enforcing control: a credential in tool output reaches the model as a placeholder.
What changes
Guardrail events: a versioned record in the agent, forwarded under the agent's own key
hooks/lib.py).log_eventaddsv,event_id(uuid4),kindandhook(derived through one frozen table),code(defaults to the legacy event name),execution_id(read once from the parent process'sTRINITY_EXECUTION_ID), and the CLI's conversation and tool-use ids. The derived fields win over a caller's, and a fault in the derivation still writes the legacy record. Existing fields are unchanged (additive only). Public enums:EVENT_KINDS,EVENT_HOOKS,SPOOL_SCHEMA_VERSION.codeat every deny site: bash (bash_deny_pattern,bash_agent_rule), file (protected_path,path_agent_rule), read-only (read_only_path), turn-tier (shell_owner_only,shell_visitor_off,mcp_owner_only,mcp_visitor_off,owner_only_path,owner_only_search,tier_unreachable), skill gate (skill_refused,skill_check_unreachable). The bash, file and read-only denies also passrule=, so a forwarded deny says which baseline row fired;turn-tier-guard.pypasses the turn's kind asturn_kind=. An AST guard fails any hook call that passes a reserved field (kind,v,event_id,hook,execution_id) or a deny withoutcode=.agent_server/guardrail_events.py, wired inmain.pythe way the heartbeat is). It tails the spool with a persisted offset, projects each record to an allowlist (no command text, traceback or error is ever sent), sanitizesrule/path, and posts batches of ≤500 events and ≤255 KiB withAuthorization: Bearer <TRINITY_MCP_API_KEY>.runtime, so a reachable agent never crosses the platform's 15-minute stale window (pinned: keepalive + max backoff < 900 s).Retry-After(clamped 1 s–1 h); 401/403/404 keep the spool, log once and retry hourly; 5xx and transport errors back off from 2 s to 5 minutes and keep the data.vare skipped. A lone surrogate becomes U+FFFD (an id holding one is dropped) and a timestamp outside float range is counted as dropped, so no single line can stall the queue.TRINITY_MCP_API_KEYorTRINITY_BACKEND_URLthe loop does not start and logs "disabled". Runtimes other than Claude Code run no hooks and send keepalives only.Guardrail events on the platform: store, own-key ingest, owner read
guardrail_eventson both migration tracks (Invariant Fix git pushing bug #9): SQLite migrationguardrail_events_tableand Alembic0124_guardrail_events, plusdb/schema.pyanddb/tables.py.UNIQUE(agent_name, event_id); indexes on(agent_name, occurred_at),(execution_id)and(received_at);execution_idcarries no foreign key.db/guardrail_events.pywithdatabase.pyfacade methods. The insert ison_conflict_do_nothing(...).returning(id), so accepted and duplicate counts are exact on both dialects. Reads page by keyset on(occurred_at, id). Serviceservices/guardrail_event_service.py, routerrouters/guardrail_events.py(# mcp: noneheader), models inmodels.py.POST /api/agents/{agent_name}/guardrail-eventsaccepts the agent's ownTRINITY_MCP_API_KEYonly (a sibling agent's key, a user key and a connector key get 403). Body{"schema":1,"runtime":…,"events":[≤500],"dropped":n}, ≤256 KiB. Answers200 {accepted, duplicates, rejected}, 413 over the caps, 422 for a malformed envelope only, 429 withRetry-Afterpast 60 posts per minute per agent.kindorhookis stored asotherwith the raw value kept; a malformed event is counted inrejected; an over-longexecution_idis nulled;ruleandpathgo through the credential sanitizer; a NUL or lone surrogate in a display field is stored as U+FFFD, and an id holding one is dropped, so one unstorable character can neither fail the batch nor break the owner's later read.details.occurred_at_reported; it is never rejected, so a skewed agent clock loses nothing.GET /api/agents/{agent_name}/guardrail-events, owner tier throughOwnedAgentByName(uniform 404, SEC: User and Agent Enumeration via differential API responses #186). Filterskind,since/until,limit(≤200), keyset cursor.{state: reporting|stale|never, last_report_at, first_report_at, runtime, dropped_total}.staleafter 15 minutes without a report.first_report_atis the receive time of the agent's first accepted report (first write wins in Redis; the oldest stored row is the fallback; capped atlast_report_at), so a consumer can tell "before this agent was covered" from "covered and clean". A lost Redis moves it later, never earlier, and the create path clears a recycled name's reporter state so a new agent never inherits a predecessor's start.count_by_execution(agent_name, execution_ids), the read-time seam A schedule created by a non-owner-tier person silently loses the shell on every run, and the run still reports success #4281 uses:{reporting, executions: {id: {by_kind, by_hook_code}}}, reading the reporting state before the counts and filtering by the authenticated agent.AGENT_REFS); the reporter key is cleared on teardown (CLEARED_KEYSPACES). Retention is 90 days onreceived_at(so an agent-reported date cannot keep a row forever), through the guarded prune (guardrail_event_retention_daysinOPS_SETTINGS_DEFAULTS, bounds 0–3650).selfinagent_route_tiers.jsonandAGENT_CALLABLE_ANYin_route_census.py; the GET is owner tier;FROZEN_BASELINE_COUNTis unchanged. The fleet-reach guard (test_ent872_fleet_reach.py) lists the fleet read's roster function,guardrail_event_service.fleet_agent_names, in its allowlist with the reason: its one caller is therequire_personroute below, which decides the fleet-wide roster withdependencies.is_fleet_admin.Fleet counts and two UI surfaces
GET /api/ops/guardrail-stats?hours=(ops_router). GateDepends(require_person): a browser login or the person's ownuserkey; agent, system, ops, connector and portal-delegate keys get 403, and a Workspace-only person is refused by the operator floor. An admin sees every live agent; anyone else sees the agents they own, not those shared with them.hoursis 1–2160 (the retention window), checked in the route and again in the service.fleet_statsreturns{hours, since, events, by_kind, coverage: {reporting, stale, never, no_hooks}, agents: [...]}. Every agent in scope is a row, with or without events, loudest first; coverage isno_hookswhen the container's runtime label is not Claude Code, otherwise the reporting state.reporting_states(names)is a batch read (one Redis pipeline plus one groupedMIN/MAX(received_at)fallback query) built on the same helper as the single-agent read, so the two cannot drift. ModelsGuardrailStatsAgentandGuardrailStatsResponse.GuardrailStatsCard.vue, admin tab): the total and a badge per kind (the total reads—, with the reason, when no agent reports), a tally per coverage state, and a bounded, scrolling list of agents with events or without coverage, each opening that agent's Settings › Guardrails. The tab's 30 s refresh reloads it in place.GuardrailEventsSection.vue, owner tier): one coverage notice, then up to 50 rows from the last 24 h (kind badge,hook · code, tool and path/rule/patterns, time).stores/guardrailEvents.js(fleet and per-agent state with their own loaded and error flags; a late read for a previous agent is dropped) andutils/guardrailEvents.js. A skeleton on the first load only (no bare loading gate),LoadFailedon a failed first read, the stale banner on a failed refresh (which keeps the data). Both surfaces sit last in their container, so their variable-height lists move nothing above them. Design-system tokens only; the new files carry zero raw palette classes.The output scanner hides credentials from the model
hooks/output-scanner.py, rewritten). On a successful call it returns a shape-preservingupdatedToolOutput: every string leaf (lists andstructuredContentincluded) is scanned and each match becomes[REDACTED:<shape>]. Clean output prints nothing and is byte-identical.|, a comment, flag or conditional group) turns the lookahead off. This keeps escape-dense output cheap: 8 MiB of backslashes took 10.9 s of CPU on a CI runner before it; in process it drops from 1.4 s to 0.5 s, andESC [from about 2 s to 0.7 s.tool-resultsfile, the file is redacted in place (temp file +os.replace), only for a regular file under/home/developerreached without a symlink. Over 8 MiB, or when the rewrite fails, the file is cut to a notice.PostToolUseas well asPostToolUseFailure, with the response as an error string. The scanner rewrites that string as a Bash object ({stdout, stderr, interrupted}), which the CLI accepts. The cost: the model receives that result without its error flag (the text still startsError: Exit code N). A newPostToolUseFailureregistration adds a warning to the model and recordscredential_unredacted.OUTPUT_SCANNER_CLI_VERIFIED = "2.1.281"with a pin guard and the list of CLI facts to re-check on a bump.credential_redacted,credential_unredacted,scanner_error,scanner_oversize, with shape names only, never a value.managed-settings.json).PostToolUseonRead|Grep|WebFetch,PostToolUseon^mcp__, andPostToolUseFailureon^mcp__, in the same exec form. On CLI 2.1.281 (read from the binary) a matcher of only letters, digits,_and|is an exact list of names and anything else is an unanchored regex; so the exact list cannot reachNotebookRead,ReadMcpResourceToolor similar, and the MCP matcher is anchored. A test pins both.is_platform_mcp_toolpredicate the turn-tier guard uses (a test checks the two agree), because some of them hand the agent a credential on purpose.base64,image_data, adatakey besidetype: base64) are never rewritten and do not count toward the 8 MiB cap, so images and PDFs pass untouched.result, MCP keeps its list or string form. Event codes are per tool (read_output,grep_output,webfetch_output,mcp_output,mcp_failure_output, and the Bash ones).guardrails-baseline.jsoncredential_patternsholds 14 shapes: the vendor-prefix part ofSECRET_VALUE_PATTERNSwithoutBearer/Basicand without anyKEY=valueheuristic. One sharedcredential_boundarydecides where a key may start, and all shapes compile into one matcher, so the leftmost key wins in a single pass.-, sopod/task-runner-deployment-…,flask-…,disk-…and?sig=share links are untouched. It does start after_, after a literal\n/\r/\tin JSON text, after a%XXURL escape, and after an ANSI escape (CSI or a short escape, raw or written as\^[/\x1b/\033/\e); the escape is kept in the output. The Slack forms need 20+ characters after the prefix, so short placeholders in fixtures and docs pass.-and_included: 43 URL-safe characters (whattoken_urlsafe(32)mints) that end there, or the older alphanumeric form. The same shape is used in both copies ofSECRET_VALUE_PATTERNS(backend and agent log sanitizers) and in the Workspace composer guard; inSECRET_VALUE_PATTERNSit moved first, so a log line never keeps part of a key that happens to contain another shape's prefix.credential_escapebaseline entry, equal toESCAPE_SEQUENCEin both sanitizers, and_split_keysis AST-identical in all three files (pinned by tests). Plain text skips the second look.AIza…) joinsSECRET_VALUE_PATTERNSin both copies and the composer guard (google).file-guardrail.pyrefuses new content holding[REDACTED:(Write, Edit, MultiEdit, NotebookEdit) andbash-guardrail.pyrefuses a command holding it, both withcode="redaction_writeback_refused", so the placeholder never replaces a real value on disk. The message tells the model toEditonly the lines it needs rather than rewrite the whole file. Write and Edit to.env,.env.*,.mcp.json,.credentials.encand the CLI's credentials file stay refused by the existing path guard whatever the content (now pinned by a test); anEditof another line matches against the real file, so the real key stays byte-identical.guard002-smoke.py) now requires a deny record carryingv/event_id/kind/hook/code, and checks the scanner's rewrite payload, the absence of the test key on stdout and stderr, the failed-call object, the persisted file, the failure warning, a splitskkey redacted whole, Read and MCP redaction, the platform tool left alone, the refusal rows, and (through the matcher rule) that the scanner reaches Read, Grep, WebFetch, MCP and both Bash events.Docs
requirements/security.md§28.2 (spool and forwarder; the "logged to Vector" line corrected), §28.4 (GUARD-004, status 🚧: fail-safe table, coverage and what it does not reach) and §28.5 (GUARD-005, status 🚧: the event contract, fleet read and empty states);architecture/agent-runtime.md,backend.md,api-endpoints.md,database.md,frontend.md; the guardrails, monitoring, settings and credential-in-chat feature flows; the user docsagents/agent-guardrails.md(what the scanner does and does not cover) andoperations/monitoring.md. GUARD-005 stays 🚧 because the requirement text also promises a blocked/allowed ratio, top patterns, notifications and export, which are outside this issue's acceptance criteria (listed under follow-ups).The issue's acceptance criteria
guardrails.jsonl), the ingest and the owner-tier GETGET /api/ops/guardrail-statsand the Health card; silence reads asnever/stale/no_hooks, never as 0.env.exampleplaceholders,?sig=links, coloured lines) stays byte-identical; shapes reuseSECRET_VALUE_PATTERNSno_hooks; the user doc says what other runtimes getTests
New files:
test_ent165_guardrail_event_store.py(real rows on SQLite and PostgreSQL),test_ent165_guardrail_event_route.py(the real router over a real DB, only the key lookup stubbed),test_ent165_guardrail_event_retention.py,test_ent165_guardrail_event_contract.py(the backend's kind and hook enums against the hooks'EVENT_KINDS/EVENT_HOOKS),test_ent165_guardrail_spool.py(89),test_ent165_guardrail_forwarder.py(37; the test backend encodes bodies with httpx's own encoder, so tests see the real wire),test_ent165_guardrail_stats.py(the real router over a real DB with fakeredis, single-vs-batch reporting parity on the Redis and the fallback path),test_ent165_output_scanner.py(290),test_ent165_scanner_other_tools.py(62; each case runs the hook only if a port of the CLI's matcher rule says the managed registration selects it), and the frontend specsguardrailEventsSection.mount.spec.js,guardrailStatsCard.mount.spec.js,guardrailEventsUtils.spec.js(mounted over the real store withapistubbed). Updated:test_ent787_guardrail_exec_form.py,test_guardrails.py.env -i … python3 -I -Sinvocation. Coverage includes the benign corpus with a precondition proving the untightened shapes would damage it; a three-class partition test that fails whenSECRET_VALUE_PATTERNSgains a shape not classified here; withhold on a forced exception without exit 2; 2000 minted MCP keys found whole; an 8 MiB case well inside the 30 s hook timeout; and keys split by colour codes (at the prefix, mid-key, every N characters, every-/_, only at the end) for thesk, GitHub, Google, AWS, Slack and MCP shapes in GNU grep, the image's grep,rgandtputstyles, raw and JSON-written, through the scanner and both sanitizers; 49 coloured benign lines stay unchanged.dev's scanner the scanner set is red for behaviour (no rewrite printed, guards absent). On the Bash-only scanner, 47 of 61 first-draft other-tools tests fail, and 25 still fail with only the registration applied, so they are red for the scanner's behaviour, not only the matcher.other; noON CONFLICT; any valid key accepted; counts not filtered by agent; GET at accessor tier; prune onoccurred_at; sweep not in the cleanup cycle; retention default lowered; theAGENT_REFSentry removed; no rate or byte cap; path and rule not sanitized; no receive-time clamp;first_report_atlast-wins, read from the newest row or dropped from the model; the create-path clear removed (red intest_1484); the unstorable-character handling removed; the second own-key check removed. Forwarder and spool:command_prefixallowlisted, sanitize removed, 3 s debounce, no rotation grace, rotation loss not counted, 413 never splits, 404 backs off, offset committed on failure, no keepalive, 422 not counted, no backoff growth, records withoutvforwarded,Retry-Afterignored, 401/403 falling into the 5xx backoff, a caller overriding a derived field, a constantevent_id, a deny losing itscode, the smoke's record check skipped. Fleet read and UI: a non-admin sees every agent (7 red); shared agents included (5);require_personreplaced byget_current_user(5); coverage ignores runtime (4); only agents with events listed (8); the batch read drops the fallback (2); the empty copy shown whatever the coverage (3); stale read as never (3); a failed first read shown as loaded; the skeleton on every fetch; a failed refresh blanks the card. Scanner: no left boundary; withhold keeps stderr; no symlink check; exception not caught; no persisted redaction; no failed-call Bash object; no write-back guard; the MCP shape back to alphanumeric in the baseline or either sanitizer copy or the composer guard; the old pattern order;_,\n/\t,%XXor the escape lead dropped; shapes run one after another; no second look (56 red); the split step not called in either sanitizer (18 each); escapes not written back (55); the start not mapped back (32); spans not merged (1); the agent copy of_split_keysdrifting (parity); no platform exemption (5); no binary skip (6); unanchoredmcp__matcher (22);Readdropped from the matcher (18); Read withhold numbers as strings (2). Lookahead: none built for the shipped shapes (2); a top-level|(3), an optional first character (3), any group form (2) or a class whose first member is an escape (1) read as a first character.Validation: each area was re-run on
devat13d32e22fbefore the areas were merged into this branch — the event store, ingest and retention tests with seeds 11, 22 and 4242: 103 passed, 1 skipped (the contract test, which needs the agent half), and with the fleet read 137 passed, 1 skipped, each time; the forwarder, spool and neighbour set (test_agent_heartbeat.py,test_3106_agent_server_fastapi_telemetry.py,test_2114_subscription_shadow_guard.py,test_2338_journey_catalog.py) 219 passed at the same seeds; the hook suites 2059 passed;test_ent165_*plus the credential-sanitizer suites 383 passed (Bash scanner) and 445 passed (other tools); the guards (#2996 ×5, ent#926 ×3, #1560 ×2,test_agent_cleanup_parity, schema parity, 186, 1310 ×2, models-centralized, 297, cleanup sweeps, 1644, 2085, 1638, retention floor,test_1484, 293, 2323, 837 floor) 1897 passed; frontend specs plus the raw-colour, loading-gate and source-text ratchets 65 passed with no baseline change;npm run check:tokensandnpm run buildpass;check_alembic_heads.pyone head;check_alembic_parity.pypass; a realalembic upgrade headon PostgreSQL reached the revision (then numbered0123_guardrail_events) with all indexes, and a downgrade and re-upgrade rebuilt them.test_ent926_turn_tier_hook.pyfails 39 cases on macOS only (/homeresolves under/System/Volumes/Data), identically on untoucheddev; it passed 163/163 in a Linux container (Python 3.13). The enum contract test, which needs the agent and backend halves on one tree and skips otherwise, runs and passes on this branch (2 passed). The merges into this branch conflicted only in docs,tests/registry.jsonandguard002-smoke.py(both the deny-record and the scanner checks kept). After mergingdevat97a86f1dc(conflicts infile-guardrail.py—dev's.envdenial sentence and the write-back refusal both kept —db/migrations.pyandtests/registry.json):check_alembic_heads.pyone head (0124_guardrail_events),check_alembic_parity.pyandcheck_sqlite_upgrade_parity.pypass; schema parity, the Alembic guards, thetest_ent165_*suites, the file-guardrail and guardrail-registration suites anddev's ent#1158 and ent#1202 suites 1041 passed, 2 skipped; the scanner suites and the files that read the guardrails flow 716 passed.devatb99c23f6e(#4447, tests only) was merged after that without conflicts.CodeQL flags
guard002-smoke.py(clear-text storage): the build smoke writes its own synthetic key ("sk-ant-api03-" + "A" * 93, namedSECRET) into a temp file it deletes, to prove the scanner redacts a persisted output file. The query tags the value by its variable name only; no credential is involved.Mutation: n/a — not a fix
Unverified: the split-key change on a live agent (proven by the tests and the image-build smoke only; the live run found the case and predates the change);
rgoutput (not installed in the agent image); WebFetch live (the CLI refuses a dotless host and upgrades plain HTTP to TLS, and no TLS page holding a test key was available; unit-tested with the shape captured from the CLI); Grep live (the pinned CLI ships no Grep tool; the shape comes from the binary's output schema); a PDF Read (unit only); a lone surrogate written by the real CLI (unit-tested through httpx's encoder); an agent-server restart within 2 s of a rotation; a real turn on the other non-Claude runtime (its CLI reached the provider, which refused the test account); Python 3.13 and the full unit suite (CI); the UI look (pending sign-off below); the scanner's first-character lookahead in an agent container (unit-tested and measured in process only). The live run predates the merge ofdevat97a86f1dcand the renumbering to0124; the PostgreSQL upgrade under the new number is CI'spg-migrationsjob.Verified on a live stack
Production images built from one tree holding these changes together with the public ent#863 and #4281 changes on the
devof that time, on the prod compose stack (PostgreSQL 16, Redis with ACL users, two backend workers), with real agent containers on Claude Code CLI 2.1.281. The first pass used a scripted model endpoint (CLI, hooks, agent server and platform real; every request the CLI sent was logged); the scanner cases were then run again with real model turns through a pass-through logging proxy that recorded every tool result the model received. Also checked: the stored stream-json, thetool-resultsfiles, the transcripts and the agent's guardrail log.devimage and on a fresh SQLite install; the realinit_databaseon PostgreSQL moved from0122to the guardrail revision (then0123_guardrail_events, now0124on0123_agent_ownership_default_model) with one head and the table and indexes present.duplicates. The owner and an admin read rows and state; a shared non-owner gets a 404 byte-identical to a missing agent's. A NUL intoolstored as U+FFFD on PostgreSQL; a surrogate inpathdoes not break the GET. Cleanup cycles ran the sweep with no prune error; rows followed a rename; the hard purge left no rows and no reporter key. An event sent three days ahead was stored at its receive time;first_report_atstayed put across later reports, a Redis restart (AOF on) and a backend restart; after a realFLUSHALLthe database fallback gave the oldest row's receive time; a same-name recreate got its own start.guard002-smoke.pyandskill-gate.py --self-testpassed in the image build. A Bash deny landed as a row withhook=bash_guardrail,code=bash_deny_patternand the execution id about 1.2 s after the hook fired; a shared person's turn landedshell_owner_onlywithturn_kind: person; no command text in the stored row or the posted body. About 5 s after start the agent readreporting(runtimeclaude-code); a Gemini agent sent keepalives only; an agent without forwarder settings logged "disabled" and never posted; an agent on an older image stayednever. Against a 404 backend: one INFO line, spool kept; after a restart against a 200 backend the backlog was delivered.docker restartresumed at the same offset with no resend; a recreate started a fresh spool; the spool rotated past 5 MiB with no gap; a stop right after a deny still delivered it within the 10 s grace.hours=0andhours=2161get 422; a Workspace-only user gets 403workspace_only. A Gemini agent readsno_hooks; an agent stopped 14.5 minutes still readreportingand at 15.8 minutesstale. The grouped fallback andfleet_statsran on PostgreSQL; after a realFLUSHALLthe fleet read gave the samefirst_report_atas the per-agent read.GUARDRAILS: ERRORandguardrails_registration: ok;rm -rf /still denied,lsran. A key on stdout read[REDACTED:sk], on stderr[REDACTED:github]; a 39 KB output's persisted file was redacted; an exit-3 call read[REDACTED:sk]with the warning and acredential_unredactedrecord. The agent's own MCP key (with-/_in its first 16 characters) readBearer [REDACTED:trinity_mcp]throughcatandReadof~/.mcp.json, with no fragment of it anywhere on record; Read of.env→[REDACTED:sk]; a background command's output read back from its tasks file was redacted; an owner MCP result and error were redacted (the error with the warning).grep --color=alwaysoutput kept its colour codes around the placeholder; the JSON\n/\t,%3D,KEY_andprefix_cases were redacted; the CLI accepted every rewrite. A model-typed key in a command was stored as***REDACTED***in both copies of the execution log.mcp__trinity__list_agentsuntouched; a realshare_filelink kept its?sig=;pod/task-runner-…passed byte-identical. Read → Edit kept the real key on disk; a whole-file Write of[REDACTED:sk]was refused with the placeholder message. In a shared person's turn, Bash, the owner MCP tool and the.envpath were refused by the turn-tier guard and an allowed Read still redacted. 8 MiB through the managed hook in a 1-CPU agent container: 2.1 s for 8 MiB ofESC[, 0.6 s for log text, 1.5 s for backslashes (before the split-key change; in-process that change movesESC[from 1.80 to 2.05 s and backslashes from 1.26 to 1.37 s, and leaves log text at 0.40 s). A Gemini agent started and ran a real turn (no hooks on that runtime, as documented).Pending
Unfiled follow-ups
guardrail_event_retention_daysis not inretentionFields.jsor the monitoring retention table.dropped_totalon Recent events and the Health card — lost events are counted but not shown.dropped_totalis cumulative with no time, so a consumer cannot tell which runs a drop may cover.last_report_atis last-write-wins — a delayed report can move it earlier, so an agent can readstaleearly.opskey to the fleet read, if the ops agent should read guardrail counts (a reviewed scope grant).startup.sh— let the agent server flush on stop instead of being killed after the grace._count_eventsby the line limit — a line far beyond any hook record is read whole on the event loop.stale).TRINITY_BACKEND_URL, so its coverage always readsnever.pending.grep --coloroutput oftask-runner-…names untouched..envvalue split by colour codes in the agent log sanitizer.sk-andxoxb-shapes —/var/cache/sk-learn-…, ansk-gateway-…host, long placeholders and the AWS docs example key read as keys; the_/escape boundary extends this tomy_sk-learn-….skshape — it refuses ordinary text such as a message naming atask-runner-…pod.[REDACTED:, and a grep for it, are refused.credential_unredactedonly when the failed call was not also redacted — it overstates counts.Error:.utils/scrub.js) onto the whole-key MCP shape.mcp_validator's literal-secret list withSECRET_VALUE_PATTERNS— its comment no longer claims they mirror.requirements/security.mdstill says GUARD-004 is not started in one summary line outside §28.4.Glob,WebSearch, MCP resource reads and the results of agent-spawning tools (Agent/Task).get_agent_logs— either narrow the platform-MCP exemption to the credential-delivering tools or sanitize/api/agents/{name}/logs.Edit.Journey Impact
Journey Impact: none: adds guardrail event forwarding, storage and reads, an Operations card and a settings list, and credential redaction in tool output; tool calls run as before and no catalog promise covers guardrail events