From 1963228c32e0948de0937ba984a5bacfc543ebd0 Mon Sep 17 00:00:00 2001 From: Maxime Lamothe-Brassard Date: Wed, 30 Sep 2026 22:48:33 +0000 Subject: [PATCH 01/10] Email Security docs: Gmail quarantine label, per-user counters, content-read audit event Co-Authored-By: Claude Sonnet 5.5 --- docs/email-security/api-reference.md | 1 + docs/email-security/automation.md | 114 ++++++++++++++++ docs/email-security/messages.md | 125 +++++++++++++++++- docs/email-security/pipeline.md | 4 +- .../provider-setup/google-workspace.md | 2 +- docs/email-security/providers.md | 2 +- 6 files changed, 244 insertions(+), 4 deletions(-) diff --git a/docs/email-security/api-reference.md b/docs/email-security/api-reference.md index 96888e9f9..48e5a496f 100644 --- a/docs/email-security/api-reference.md +++ b/docs/email-security/api-reference.md @@ -102,6 +102,7 @@ that fires on volume. A refused attempt carries `result: refused` and a | `eml_never_stored` | The message exists but no raw copy was written at ingest | | `eml_expired` | The raw copy aged out of its retention lane | | `read_failed` | The object is there and could not be read | +| `audit_write_failed` | The access record for this download could not be written (or an earlier record for it could not be read), so nothing was served | | `eml_store_not_configured` | This deployment has no raw-message store | | `internal_error` | The service could not complete the read (an index-store failure, not an object failure) | diff --git a/docs/email-security/automation.md b/docs/email-security/automation.md index c7f92762e..7987e4db5 100644 --- a/docs/email-security/automation.md +++ b/docs/email-security/automation.md @@ -195,6 +195,120 @@ From there the detection flows into Cases, Outputs and everything else that consumes detections. A report is the highest-signal thing your users will ever hand you, so treating it as a first-class detection is usually right. +## Counting events per mailbox or per user + +A rule that fires on one event is often not what you want. "One malicious message +landed in a mailbox" is routine; "the same mailbox received five in an hour" is an +attack on a person. LimaCharlie's D&R [suppression](../8-reference/response-actions.md#suppression) +does the counting, and Email Security events carry the identity to count by, so no +mail-specific feature is needed. + +The pattern is a `report` action whose suppression is **global** and whose `keys` +include the mailbox or user. Global means the counter is shared across the +organization, so it is scoped by the key alone. The action is skipped until the +count reaches `min_count`, and then fires up to `max_count` times in the `period`. +Setting both to the same number fires once, on the Nth event, and stays quiet for +the rest of the window. + +| Parameter | Use for per-user counters | +|---|---| +| `is_global` | `true`. The counter is organization-wide and the key decides what is counted together | +| `keys` | A constant label, so two rules never share a counter, then the field to count by, for example `'{{ .event.mailbox.address }}'` | +| `min_count`, `max_count` | The threshold. Set both to `N` to fire once when the Nth event arrives | +| `period` | The window: `s`, `m` or `h`, from 1 second to 720 hours | + +The window is fixed, not sliding: it starts at the first counted event for a key and +the counter resets when it expires. See the platform's +[Behavioral Detection](../3-detection-response/behavioral-detection.md) page for the +full set of patterns and its limitations. + +The field to count by depends on the event: + +| Event | Field | Holds | +|---|---|---| +| `EMAIL_MESSAGE`, `EMAIL_VERDICT`, `EMAIL_ACTION` | `event/mailbox/address` (template `{{ .event.mailbox.address }}`) | The protected mailbox the event is about | +| `EMAIL_USER_REPORT` | `event/reporter` (template `{{ .event.reporter }}`) | The address that sent the report to the abuse mailbox, or `unknown` when the report had no usable sender | + +### Example: five malicious messages to one mailbox in an hour + +```yaml +# Detect +op: and +rules: + - op: is + path: routing/event_type + value: EMAIL_MESSAGE + - op: is + path: event/verdict/verdict + value: malicious + - op: is + path: event/direction + value: inbound +``` + +```yaml +# Respond +- action: report + name: email-mailbox-malicious-burst + priority: 3 + suppression: + is_global: true + min_count: 5 + max_count: 5 + period: 1h + keys: + - 'email-malicious-per-mailbox' + - '{{ .event.mailbox.address }}' +``` + +The detection fires once, when a mailbox receives its fifth malicious inbound +message inside the hour, and carries the triggering `EMAIL_MESSAGE` so the +responder can see the mailbox and the message. It counts the verdict the rule pack +gave at ingest. A message that only becomes malicious later, through an analyst, AI +or detonation revision, arrives as an `EMAIL_VERDICT` and is not counted by this +rule. + +### Example: three user reports from one person in a day + +```yaml +# Detect +op: and +rules: + - op: is + path: routing/event_type + value: EMAIL_USER_REPORT + - op: exists + path: event/automated_sender + not: true +``` + +```yaml +# Respond +- action: report + name: email-reporter-repeat + priority: 2 + suppression: + is_global: true + min_count: 3 + max_count: 3 + period: 24h + keys: + - 'email-reports-per-reporter' + - '{{ .event.reporter }}' +``` + +`automated_sender` is present, and `true`, only on reports that came from a +machine, so the second condition leaves those out of the count. A person who +reports three messages in a day is either being targeted or is the most alert +member of your staff, and in both cases an analyst wants to know. + +!!! tip "Chain a counter onto a detection" + The same suppression can count detections instead of events, using the + `target: detection` chaining described in + [Behavioral Detection](../3-detection-response/behavioral-detection.md#cardinality-detection). + That is how you count *distinct* values, for example the number of different + senders that hit one mailbox, rather than the number of events. + ## Watching your own coverage `EMAIL_INGEST_ERROR` is the event to alert on. A mail security product that diff --git a/docs/email-security/messages.md b/docs/email-security/messages.md index 712a5f798..e7aa7f83c 100644 --- a/docs/email-security/messages.md +++ b/docs/email-security/messages.md @@ -228,7 +228,7 @@ provider, and audited. | Action | Effect | |---|---| -| `quarantine_message` | Out of the inbox into a product-owned quarantine location — restorable, invisible to the user | +| `quarantine_message` | Out of the inbox into a product-owned quarantine location, restorable. On Microsoft 365 the location is a hidden folder, so the user does not see the message. On Google Workspace it is a visible `LC Quarantine` label, so the user can still find the message under that label | | `trash_message` | To the provider's recoverable trash | | `move_to_spam` | To the provider's junk/spam location | | `restore_message` | Back to where it was before we moved it, falling back to the Inbox when that is unknown | @@ -238,6 +238,16 @@ provider, and audited. The per-provider mechanics differ and are documented in [Connecting Providers](providers.md#capability-differences-between-providers). +!!! warning "Quarantine is hidden from the user on Microsoft 365 only" + On Microsoft 365 the message moves to a hidden `LC Quarantine` folder that + the user does not see in Outlook. On Google Workspace, quarantine removes + the message from the inbox and adds an `LC Quarantine` label that is shown + in the label list and in message lists, so the user can open the label and + read the message. If your process assumes the recipient cannot reach a + quarantined message, that holds for Microsoft 365 mailboxes only. On + Workspace, treat quarantine as "out of the inbox", and use `trash_message` + if you need the message out of the user's normal view. + ```bash limacharlie mailsec message action \ --action quarantine_message --reason "confirmed credential phish" --oid $OID @@ -340,6 +350,119 @@ limacharlie mailsec message eml \ Read a justification back with `mailsec action get `. +## Who read a message: the content-read audit event + +Reading a message's content is recorded, not only downloading it. Each time +someone reads the body of a message, Email Security writes one +**`mailsec_message_content_read`** event to the organization's +[audit log](../7-administration/access/user-access.md#4-what-access-related-changes-have-been-made-and-by-whom), +the same log that records configuration and user changes across the platform. It +answers "who read this person's mail", which the action audit above does not: that +trail records what was *done* to a message and who took the original bytes out, not +who looked at the text. + +This is an audit log event, not an `EMAIL_*` event. It is not emitted on the mail +connection's sensor, it does not reach D&R rules as a sensor event, and it is +not part of the message index. You read it through the audit log, and you can +forward the whole `audit` stream with an +[Output](../5-integrations/outputs/stream-structures.md#3-audit-stream-structure). + +### What counts as a read + +There is one event type, and the `content` field says which kind of content was read. + +| `content` | What was served | Needs | +|---|---|---| +| `mdm` | The parsed message, including its HTML and plain-text body. This is what the drawer shows, and what `mailsec message get` and the matching API route return | `mailsec.get` | +| `eml` | The original message bytes, through the [justified download](#downloading-the-original-message) | `mailsec.get` and `mailsec.get.eml`, plus a justification | + +Only a read that actually served content is recorded. A message whose raw copy has +expired, an unknown `msg_uuid` and a refused download serve nothing and write +nothing here. A refused download is still recorded in the action audit and as an +`EMAIL_ACTION` event, as before. + +### Fields + +| Field | Meaning | +|---|---| +| `etype` | Always `mailsec_message_content_read` | +| `ident` | Who read it: the authenticated identity the request ran as. `origin` carries the same value | +| `time`, `ts` | When the read was recorded | +| `msg` | A human-readable sentence naming the message and the mailbox | +| `entity.msg_uuid` | The message that was read | +| `entity.mailbox_address` | The mailbox the message was read from | +| `mtd.content` | `mdm` or `eml` | +| `mtd.actor_kind` | The kind of credential behind `ident`: `user` for an interactive session, `user_api_key` for a user's personal API key, `org_api_key` for an organization API key, or `unknown` when the request carried no identity | +| `mtd.provider` | `m365` or `gworkspace`, as indexed for the message | +| `mtd.verdict` | The message's verdict at the time of the read | +| `mtd.mdm_source` | `mdm` reads only: `stored` for the model the engine judged with, `eml_reparse` for a fresh parse of the raw copy. See [Which model you are looking at](#which-model-you-are-looking-at) | +| `mtd.bytes` | `eml` reads only: the size of the download | +| `mtd.justification` | `eml` reads only: the justification that was supplied | + +The event never carries the subject, the sender or any of the message body. A +record of who read mail must not become a second copy of it in a stream you may +forward to a SIEM with its own retention. `ident` and `mtd.actor_kind` are what +tell an analyst's console session apart from an automation or an +[AI triage](ai-triage.md) agent working through an API key. + +### Finding the events + +Reading the audit log needs the `audit.get` permission. In the web console, open +**Audit Logs** in the organization and filter on the event type. From the CLI: + +```bash +limacharlie audit list --event-type mailsec_message_content_read \ + --start $(date -d '7 days ago' +%s) --end $(date +%s) --oid $OID +``` + +An entry for a person reading a message in the console looks like this: + +```json +{ + "oid": "", + "etype": "mailsec_message_content_read", + "msg": "read the parsed content of message 4f0c2b1e-9d7a-4c55-8a3e-6b1f2d9e7a10 in mailbox alice@example.com", + "ident": "analyst@example.com", + "origin": "analyst@example.com", + "time": 1790000000000, + "entity": { + "msg_uuid": "4f0c2b1e-9d7a-4c55-8a3e-6b1f2d9e7a10", + "mailbox_address": "alice@example.com" + }, + "mtd": { + "content": "mdm", + "actor_kind": "user", + "provider": "m365", + "verdict": "suspicious", + "mdm_source": "stored" + } +} +``` + +A download has `content: eml` and adds `bytes` and `justification` to `mtd`. + +### What to rely on + +- **One event per reader, message and kind, per hour.** The drawer re-fetches the + message on tab switches, and several screens open the same drawer, so recording + every request would report one analyst reading one message ten times. A second + read of the same message by the same identity inside the hour is not recorded + again. A different identity, including the same person through a personal API + key, is a different reader. A download with a different justification is + recorded separately, and so is an `eml` read after an `mdm` read of the same + message. This limits how often the event is written and never limits access: + nothing is refused because it was already recorded. +- **The event is best effort.** If the audit log cannot be written at that moment, + the read is still served and the gap is recorded in the service's own logs. The + drawer must stay usable while the audit service restarts, so a failed audit write + does not block an `mdm` read. +- **A download fails closed, on the action audit.** This is separate from the event + above. Before any byte of the original message is read, its record in the action + audit must be written. If it cannot be, the download is refused and recorded as + `refused_reason: audit_write_failed`; nothing is served. So a raw download never + happens without an audit record, even if the `mailsec_message_content_read` event + for it could not be written. + ## Sender profiles ```bash diff --git a/docs/email-security/pipeline.md b/docs/email-security/pipeline.md index d5a0983f2..92025a7a8 100644 --- a/docs/email-security/pipeline.md +++ b/docs/email-security/pipeline.md @@ -222,7 +222,9 @@ Opening the drawer serves the **sealed judged model** where it exists, labelled included. The fallback re-parses the encrypted raw message with today's parser (`mdm_source: eml_reparse`) and carries no enrichments at all, and the response always says which one you are reading. Neither needs a justification: the model -is the product's structured view of the message. +is the product's structured view of the message. Each read is still recorded in the +organization's audit log, see +[Who read a message](messages.md#who-read-a-message-the-content-read-audit-event). The **original bytes** are gated separately. Downloading the EML requires the `mailsec.get.eml` permission on top of `mailsec.get`, plus a written diff --git a/docs/email-security/provider-setup/google-workspace.md b/docs/email-security/provider-setup/google-workspace.md index f55cda2bd..c0417e55a 100644 --- a/docs/email-security/provider-setup/google-workspace.md +++ b/docs/email-security/provider-setup/google-workspace.md @@ -454,7 +454,7 @@ before lifecycle can pass. It is idempotent and the watch expires on its own. | Action | What happens in Gmail | |---|---| -| `quarantine_message` | `INBOX` removed, an `LC Quarantine` label added — restorable, and out of the user's inbox | +| `quarantine_message` | `INBOX` removed, an `LC Quarantine` label added. Restorable, and out of the user's inbox. The label is created **visible** in the label list and in message lists, so the user can still see and open the quarantined message (unlike Microsoft 365, where the folder is hidden) | | `trash_message` | `TRASH` added. The product's own quarantine label is removed afterwards, so the message's placement reads as trashed rather than still quarantined | | `move_to_spam` | `SPAM` added, resolved through Gmail's own identifiers | | `restore_message` | The labels are inverted | diff --git a/docs/email-security/providers.md b/docs/email-security/providers.md index 8092444d1..7fc570eec 100644 --- a/docs/email-security/providers.md +++ b/docs/email-security/providers.md @@ -138,7 +138,7 @@ over. | | Microsoft 365 | Google Workspace | |---|---|---| -| **Quarantine** | Move to a hidden `LC Quarantine` folder — restorable, invisible to the user | Remove `INBOX`, add an `LC Quarantine` label | +| **Quarantine** | Move to a hidden `LC Quarantine` folder. Restorable, and the user does not see the folder | Remove `INBOX`, add a **visible** `LC Quarantine` label. Restorable, and the user can still find the message under that label | | **Trash** | Move to Recoverable Items — invisible to the user, recoverable by an admin. Distinct from Deleted Items | Add `TRASH` | | **Move to spam** | Move to the Junk Email folder | Add `SPAM` | | **Restore** | Move back to the folder we recorded, falling back to the Inbox | Invert the labels | From fc759ceae5bfb32a4ea05e45a6f5927f3a5ede38 Mon Sep 17 00:00:00 2001 From: Maxime Lamothe-Brassard Date: Wed, 30 Sep 2026 22:51:36 +0000 Subject: [PATCH 02/10] Email Security docs: data residency, encryption and data flows page Co-Authored-By: Claude Sonnet 5.5 --- docs/email-security/data-residency.md | 126 ++++++++++++++++++++++++++ docs/email-security/index.md | 1 + docs/email-security/pipeline.md | 3 + mkdocs.yml | 1 + 4 files changed, 131 insertions(+) create mode 100644 docs/email-security/data-residency.md diff --git a/docs/email-security/data-residency.md b/docs/email-security/data-residency.md new file mode 100644 index 000000000..08df3cbd9 --- /dev/null +++ b/docs/email-security/data-residency.md @@ -0,0 +1,126 @@ +# Data Residency, Encryption and Data Flows + +--8<-- "includes/email-security-beta.md" + +This page answers the questions a security or privacy review asks about Email +Security: where your mail is stored, how it is protected, how long it is kept, and +what can leave the region your organization lives in. Every statement here is about +behavior of the product as built. Where a question cannot be answered from that, +the page says so in [What this page does not cover](#what-this-page-does-not-cover). + +See [Storage and privacy](pipeline.md#storage-and-privacy) for the short version of +the encryption and retention facts, and +[Data retention and deletion](policy.md#data-retention-and-deletion) for purge. + +## Where your data lives + +Email Security runs as its own deployment in **each LimaCharlie datacenter**, and +an organization's mail is processed and stored in the datacenter that hosts the +organization, the one chosen as the +[Data Residency Region](../8-reference/faq/general.md#where-will-my-data-be-processed-and-stored) +when the organization was created. Requests for an organization are served by that +datacenter's deployment, and so is the work of reading its mailboxes. + +Each datacenter's deployment has its own stores, all in that datacenter's region: + +| Data | Where it is held | Notes | +|---|---|---| +| Message index | A single-region database in the datacenter | One row per message per protected mailbox: sender, mailbox, subject, verdict, hashes used for clustering, campaign and remediation state. It does not hold the message body. Also holds campaigns, sender profiles, the action audit and user reports | +| Raw messages | A single-region storage bucket dedicated to Email Security, one per datacenter | The original message, and beside it the parsed model the engine judged with. Both are encrypted, see below | +| Key that wraps the encryption root | A key in the cloud key management service, in the same region as the bucket | The service holds the root only as ciphertext, see below | +| Link-detonation results | A separate single-region bucket in the datacenter | Short-lived, see [What can leave the region](#what-can-leave-the-region) | +| Short-lived caches | The datacenter's own cache | Holds shared facts about public domains, such as a registration date, and short-lived coordination state | + +Email Security does not copy any of these stores to another datacenter. Each +datacenter has its own database, bucket and wrapping key, created in its own region. + +The `EMAIL_*` events are ordinary LimaCharlie telemetry. They are ingested by the +same datacenter and stored in your organization's telemetry lake like any other +event, under the platform's own residency guarantees and your retention settings. + +## Encryption of stored messages + +The full raw message is stored encrypted, and so is the parsed copy the engine +judged it with. Each object is compressed and then sealed with **AES-256-GCM**. + +- **Per-organization keys.** Every organization has its own data-encryption key. + It is derived with **HKDF** (SHA-256) from the datacenter's root key, with the + organization id as the salt, so two organizations never share a key. The derived + key is computed in memory when needed and is not stored. +- **A KMS-wrapped root.** The root key is held as ciphertext that can only be + unwrapped by calling the cloud key management service, and the permission to do so + is scoped to the Email Security services that write or read raw messages. A + service configured without the key management key fails to start rather than fall + back to an unwrapped root. +- **Authenticated, and bound to its location.** GCM detects any change to the + stored bytes. The object's own storage path is part of the authenticated data, so + an object copied to another path, another message's or another organization's, + does not open. The bytes an analyst receives are the bytes that were stored. +- **Access to the bucket is not access to the mail.** The bucket holds ciphertext. + The service that answers API reads can read the bucket, and has no code path + that writes to it. + +The message index is stored in the regional database, not in the encrypted +bucket, and holds metadata rather than message bodies, as listed above. + +## Retention + +| Lane | Kept | Holds | +|---|---|---| +| Transient | **35 days** | Every message | +| Retained | up to **400 days** | Flagged messages and the evidence attached to them | + +The two windows are enforced by lifecycle rules on the raw-message bucket itself, +in addition to Email Security's own sweeps, so the data ages out even if a sweep is +delayed. Your [`retention`](policy.md#retention) policy can only shorten them. A +tenant purge removes everything the product holds for an organization at once, +including stored raw messages and detonation results. + +`EMAIL_*` telemetry follows your ordinary telemetry retention, not these lanes. + +## What can leave the region + +Most of Email Security's work happens inside the datacenter: reading mail, +parsing, scoring, clustering, attachment inspection and storing. The table lists +everything that sends data out of that datacenter, or to a party other than you, +and what it sends. + +| Flow | What is sent, and to whom | You control it with | +|---|---|---| +| **Your mail provider** | Email Security reads mail from Microsoft 365 or Google Workspace and sends remediation calls back (move, label, banner). That traffic is between the datacenter and your provider | The provider connection, its scopes and its scope of mailboxes | +| **`EMAIL_*` events to your Outputs and rules** | Event bodies, including parsed message text (each body part is capped at about 256 KB). They go wherever you send them: an Output destination, a D&R response action or a webhook | Your [Outputs](../5-integrations/outputs/index.md) and D&R rules | +| **AI triage** | If you build and enable it, the agent reads parsed messages through the Email Security API and sends what it reads to the AI provider you configured. See [AI Triage](ai-triage.md). Nothing is sent unless you create the agent | Whether you create the agent, which model provider you give it, and the permissions of its API key. Do not grant it `mailsec.get.eml` | +| **Domain registration lookups** | Domain-age enrichment asks the domain's public registry, using the open RDAP protocol, when the domain was registered. The query contains the domain name, taken from a message's links or sender, and nothing else from the message. It does not go through an aggregator. The list of registries comes from the public IANA RDAP bootstrap file. Answers are cached for all organizations in the datacenter, so one domain is looked up once rather than once per message | Not configurable per organization | +| **Link detonation** | Where it is deployed, a suspicious link is fetched from an isolated environment in the same datacenter region, so the destination server sees a request from that region. The request is for the link as written in the message, after a mail gateway's rewrite is removed, including its query string. The isolated environment holds no credential and is not told which organization the link came from. Only a bounded summary comes back: redirect chain, certificate facts, a hash, a title and a short text excerpt, never the page body. Links carrying embedded credentials, and non-web addresses, are refused | Detonation is an enrichment. It runs automatically only for [suspicious messages](detections.md#link-detonation), and otherwise when an analyst, a rule or an AI triage agent asks for it with `crawl_link`. A link that identifies the recipient in its address identifies them to its destination, as it would if they clicked it | +| **Usage metering** | A daily per-organization count of protected mailboxes goes to platform usage metering. It carries no message data | Not configurable | + +Two things people expect to be on this list are not: + +- **Attachment inspection stays in the datacenter.** Attachments are opened by a + scanning service that runs in the datacenter's own cluster and is reached over the + datacenter's internal network. Email Security sends the bytes to it, scans, and + drops them when the message leaves the pipeline. What Email Security keeps is a + summary and hashes, not the attachment. +- **Threat-intelligence feeds are not queried per message.** Email Security does + not send message content, links or hashes to a third-party reputation service. A + feed such as a malicious-URL list reaches rules as an ordinary + [lookup in your organization](ioc-feeds.md), and is matched inside the + datacenter. The popularity ranking used to spot unranked domains is downloaded + into the service; downloading it sends no message data. + +## What this page does not cover + +These are not claimed either way, because they go beyond what Email Security +itself does: + +- Where the platform's telemetry lake, usage metering and billing systems + store their data. That is the platform's own residency behavior, described in + the general [FAQ](../8-reference/faq/general.md#where-will-my-data-be-processed-and-stored). +- The path a response takes from the datacenter to the person or tool that asked + for it, through the platform's API gateway. A message you read in the console or + through the API, including a raw download, is returned over that path. +- Where AI Sessions, if you use it for triage, runs, and what your chosen model + provider does with the content it is given. +- The handling of change notifications your mail provider sends to LimaCharlie, + and of the provider's own copy of your mail. +- Backups and operational access by LimaCharlie staff. diff --git a/docs/email-security/index.md b/docs/email-security/index.md index cfae262f7..df38a341f 100644 --- a/docs/email-security/index.md +++ b/docs/email-security/index.md @@ -116,4 +116,5 @@ Managing connections and policy uses the ordinary Hive permissions for the | [Policy Reference](policy.md) | Every `mailsec_policy` record type | | [Events & Automation](automation.md) | The `EMAIL_*` events and wiring them to D&R | | [Command Line Interface](cli.md) · [API Reference](api-reference.md) | The programmable surface | +| [Data Residency & Encryption](data-residency.md) | Where mail is stored, how it is encrypted, how long it is kept, and what can leave the region | | [Troubleshooting](troubleshooting.md) | What each failure looks like, and where it is reported | diff --git a/docs/email-security/pipeline.md b/docs/email-security/pipeline.md index 92025a7a8..c69800f03 100644 --- a/docs/email-security/pipeline.md +++ b/docs/email-security/pipeline.md @@ -168,6 +168,9 @@ single "status" would lose every one of those distinctions. A mail security product holds the most sensitive data in the tenant, so it is worth being precise about what is kept, where, and who can read it. +For where the data lives and what can leave the region, see +[Data Residency, Encryption and Data Flows](data-residency.md). + ### The raw message The full original message is stored, compressed and then encrypted with diff --git a/mkdocs.yml b/mkdocs.yml index 231add15c..5b519ae6d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -612,6 +612,7 @@ nav: - AI Assistants (MCP): email-security/mcp.md - API Reference: email-security/api-reference.md - AI Triage: email-security/ai-triage.md + - Data Residency & Encryption: email-security/data-residency.md - Troubleshooting: email-security/troubleshooting.md - Cloud Security: From ea3dd6fc61ccf0dafa5d6c7560a759510893ec89 Mon Sep 17 00:00:00 2001 From: Maxime Lamothe-Brassard Date: Wed, 30 Sep 2026 23:05:10 +0000 Subject: [PATCH 03/10] Email Security docs: branded warning banners, per-verdict wording, action text, banner preview --- docs/email-security/api-reference.md | 3 +- docs/email-security/automation.md | 20 ++++++- docs/email-security/messages.md | 2 +- docs/email-security/policy.md | 87 +++++++++++++++++++++++----- docs/email-security/remediation.md | 2 +- 5 files changed, 96 insertions(+), 18 deletions(-) diff --git a/docs/email-security/api-reference.md b/docs/email-security/api-reference.md index 48e5a496f..ac495e4b6 100644 --- a/docs/email-security/api-reference.md +++ b/docs/email-security/api-reference.md @@ -310,7 +310,7 @@ faster, which is worth doing for its own sake. | Route | Does | |---|---| -| `POST /messages/{msg_uuid}/actions` | Perform a typed action on one message. Body: `action` (`quarantine_message`, `trash_message`, `move_to_spam`, `restore_message`, `banner_message`, `unbanner_message`), optional `force` (boolean; see [alert-only overrides](#explicit-override-in-alert-only-mode)), optional `reason`, optional `attempt` (idempotency token — omit to collapse onto the existing attempt). `banner_message` uses the organization's own banner, rendered from its `mailsec_policy` record of type `banners`; the body's `banner` field is **deprecated and ignored** and will be removed. Requires `mailsec.act` | +| `POST /messages/{msg_uuid}/actions` | Perform a typed action on one message. Body: `action` (`quarantine_message`, `trash_message`, `move_to_spam`, `restore_message`, `banner_message`, `unbanner_message`), optional `force` (boolean; see [alert-only overrides](#explicit-override-in-alert-only-mode)), optional `reason`, optional `attempt` (idempotency token — omit to collapse onto the existing attempt). `banner_message` uses the organization's own banner, rendered from its `mailsec_policy` record of type `banners`; its optional `text` (plain text, at most 512 characters; refused on any other action) replaces the wording for that one banner. No caller supplies HTML. Requires `mailsec.act` | | `POST /campaigns/{campaign_id}/actions` | Sweep a campaign. Same body plus `confirm`. **Without `confirm` this previews** and changes nothing, returning the member ids, the distinct mailboxes, the counts and a `confirm` token derived from that exact member set. With `confirm` it executes exactly that set; a campaign that grew since the preview is refused. Capped at 500 members. `reason` is recorded on **every member's** audit row and on the sweep's own row (`action_id` in the response); `attempt` (bounded at 128 characters, refused not truncated) mints a new row per member, so a deliberate retry is recorded beside what it retried instead of over it. Neither is part of the `confirm` token. Requires `mailsec.act` | | `POST /actions/bulk/execute` | Execute a previewed bulk remediation. Returns a `bulk_id` immediately and the provider work proceeds in the background. Requires `mailsec.act`. See [Bulk Remediation](remediation.md) | | `POST /reports/{report_id}/resolve` | Record a triage outcome. Body: `disposition` — one of `true_positive`, `false_positive`, `benign`. Resolving an already-resolved report succeeds and reports `already_resolved`, so two analysts clicking at once is not an error. Requires `mailsec.set` | @@ -400,6 +400,7 @@ close a report must be able to reopen one, or a mis-click is permanent. |---|---| | `POST /analyze` | Parse a raw message into the Message Data Model and judge it with the organization’s enabled `dr-mail` rules and resolved scoring policy. **Nothing is ingested or stored**: no index row is written, no raw copy kept, and the organization's mail history is unchanged. Body: `eml_b64` (preferred) or `eml`, plus optional `org_domains` and `direction` — one of `inbound`, `internal`, `outbound`; anything else is refused with a `400` rather than analysed, because many default detections apply to inbound mail only and a mistyped direction would silently answer a lower verdict. Omit it to judge with no direction. Tenant context it cannot have — your sender history, your VIP list — is named explicitly in the payload rather than silently missing. Requires `mailsec.get` | | `POST /actions/bulk/preview` | Preview a bulk remediation over a caller-supplied selection: reports each message's current state and the distinct-mailbox blast radius, and mints the `confirm` token derived from that exact selection. **Nothing is changed and no job is created** — it is a `POST` only because up to 500 message ids do not belong in a query string. Requires `mailsec.get`, like the campaign preview it mirrors. See [Bulk Remediation](remediation.md) | +| `POST /banner/preview` | Render a candidate [`banners` policy](policy.md#banners) exactly as recipients would see it, without saving. Body: `banner` (the record's fields, without `policy_type`), optional `verdict` (which variant to preview) and optional `text` (an action's wording). Answers 200 with `valid: true` and the rendered `html`, or `valid: false` and the validator's reason. Requires `mailsec.get` | | `POST /rules/validate` | Compile a candidate `dr-mail` rule and report its errors without saving it. Body: `rule` (object), optional `rule_id`. Runs the same validator the `dr-mail` Hive applies on save, including lookup existence checks when the API's Hive metadata access is configured. Response blocks receive shape and size checks; full response compilation happens in the collector. See [Custom Rules](custom-rules.md#validation). An invalid rule is a `200` carrying `valid: false` and the reason, not an error response. Requires `mailsec.get` | | `POST /rules/backtest` | Evaluate a candidate `pre_verdict` rule over re-parsed stored messages. Original pipeline enrichments are not reconstructed. Lookups use current records when the Hive resolver is configured; `post_verdict` rules are refused. See [backtest limitations](custom-rules.md#what-a-backtest-can-evaluate). Body: `rule`, optional `rule_id`, `since`, `until`. Every response carries a `coverage_note` and counts what it could not examine (`skipped_no_raw`, `skipped_unparse`, `truncated`). `precision` is `null` — not `0` — when nothing it matched has an analyst disposition yet. Every message in the window is re-read from storage, which makes this the most expensive read on the surface: it is subject to the [replay budget](#the-replay-budget). Requires `mailsec.get` | diff --git a/docs/email-security/automation.md b/docs/email-security/automation.md index 7987e4db5..6859190cc 100644 --- a/docs/email-security/automation.md +++ b/docs/email-security/automation.md @@ -158,8 +158,24 @@ idempotency all apply unchanged — there is exactly one remediation path in thi product. A rule does not supply banner HTML: `banner_message` uses the organization's own -banner from its [`banners` policy record](policy.md#banners), rendered -server-side into a fixed escaped template. Automated bannering also requires +banner from its [`banners` policy record](policy.md#banners) (title, colour, +logo and the wording for the message's verdict), rendered server-side into a +fixed escaped template. A rule may add one plain-text `text` (at most 512 +characters, no `<` or `>`) that replaces the wording for that banner only, for +example to name the reason the rule fired: + +```yaml +respond: + - action: extension request + extension name: ext-email-security + extension action: banner_message + extension request: + msg_uuid: '{{ .event.msg_uuid }}' + text: "Payment details changed in this message. Confirm by phone before paying." +``` + +`text` is refused, with the reason in the rule's result, if it contains markup +or control characters or is sent on any action other than `banner_message`. Automated bannering also requires `enabled` on the [`banners` record](policy.md#banners); without it a rule's `banner_message` is decided and audited but the mailbox is not touched (`alert_only`). Bannering asked for by a person — console, API, CLI — is not diff --git a/docs/email-security/messages.md b/docs/email-security/messages.md index e7aa7f83c..7bad7778a 100644 --- a/docs/email-security/messages.md +++ b/docs/email-security/messages.md @@ -232,7 +232,7 @@ provider, and audited. | `trash_message` | To the provider's recoverable trash | | `move_to_spam` | To the provider's junk/spam location | | `restore_message` | Back to where it was before we moved it, falling back to the Inbox when that is unknown | -| `banner_message` | Prepend the organization's warning banner. Its wording comes from the `banners` [policy record](policy.md#banners) and is escaped into a fixed template — no caller supplies HTML | +| `banner_message` | Prepend the organization's warning banner. Its look and default wording come from the `banners` [policy record](policy.md#banners) and are escaped into a fixed template; an optional plain-text `text` replaces the wording for this one banner. No caller supplies HTML | | `unbanner_message` | Remove it | The per-provider mechanics differ and are documented in diff --git a/docs/email-security/policy.md b/docs/email-security/policy.md index 0b1d399b3..5efe9767a 100644 --- a/docs/email-security/policy.md +++ b/docs/email-security/policy.md @@ -343,28 +343,87 @@ through [Mail Rules](custom-rules.md). ## `banners` -The warning banner's text and switch. +The warning banner's look, wording and switch. ```yaml policy_type: banners enabled: true +title: "Acme IT security" +color: red text: "External sender. Verify before clicking links or opening attachments." +logo_url: "https://cdn.example.com/brand/logo.png" +logo_alt: "Acme IT" +variants: + malicious: + title: "Do not open" + text: "Our systems judged this message malicious. Do not click or reply; report it." + color: red + suspicious: + text: "This message looks suspicious. Check the sender before you act." ``` | Field | Default | | |---|---|---| | `enabled` | `false` | Bannering rewrites the customer's mail, and nothing in this product modifies mail by default | -| `text` | A packaged warning | **Plain text only** — no `<` or `>` — and capped at 512 characters | - -The HTML template is fixed and sanitized in code; policy contributes only the -text, and it is HTML-escaped when the banner is rendered. Accepting markup here -would turn a configuration field into stored HTML injection against your own -users, so it is refused at the record and escaped again at the render. - -**This record is the only source of a banner's wording.** No API call, CLI flag -or D&R rule supplies banner HTML — the `banner` field on the action routes and -the `--banner` flag are deprecated and ignored, and will be removed. If you -change the wording here, every subsequent `banner_message` uses it. +| `text` | A packaged warning | **Plain text only** — no `<` or `>` — at most 512 characters | +| `title` | `Security warning` | The bold heading. Plain text, at most 80 characters | +| `color` | `yellow` | One of `yellow`, `red`, `orange`, `blue`, `green`, `gray`. A name from a fixed palette, never a CSS value | +| `logo_url` | none | An `https://` URL of one image, at most 512 characters. See [the logo](#the-logo) | +| `logo_alt` | empty | Alternative text for the logo, at most 80 characters | +| `variants` | none | Overrides of `title`, `text` and `color` per verdict: `malicious`, `suspicious`, `graymail`, `benign`, `unknown` | + +The HTML template is fixed and sanitized in code. Policy contributes plain-text +strings, one colour *name*, and one image URL; nothing you write is ever +interpreted as HTML or CSS. Text is escaped when the banner is rendered, and +accepting markup here would turn a configuration field into stored HTML +injection against your own users, so it is refused when the record is written +and neutralized again at render time. Control characters, right-to-left +overrides and zero-width characters are refused too, because they let a warning +read differently from what it says. Tab and newline are allowed in the wording +and show as a space. + +The banner is placed **outside** the container that holds the sender's own HTML +and stylesheets, so a sender cannot hide, restyle or cover it, whatever the +message contains. Your branding lives inside that protected block. + +### Which wording a message gets + +For each message, most specific first: + +1. the `text` the action itself carried (an API call, a D&R rule, or the console's + "Banner wording" box; see [Remediation](remediation.md)), for that one banner; +2. the `variants` entry for the message's **current verdict**; +3. the record's `text`; +4. the packaged sentence. + +`title` and `color` follow the same order, minus step 1. The logo belongs to the +organization and does not vary by verdict. A verdict without a variant uses the +defaults. A message that already carries a banner keeps it: `banner_message` +is idempotent, so a later verdict change does not swap the wording on messages +that were already bannered. Un-banner and banner again if you want that. + +### The logo + +The logo is one image, shown 32 pixels high (at most 128 wide) at the start of +the heading, with the alt text as its description. To keep it safe: + +- Only `https://` URLs are accepted. `http:`, `data:`, `cid:` and other schemes + are refused, as are URLs carrying credentials, a port, an IP address or a + single-label host name, and anything that is not plain ASCII (percent-encode + the rest). +- Mail clients fetch the image from **your** host each time a message is + opened, and several block remote images until the reader allows them. The + banner's text always stands on its own: treat the logo as decoration and + never as the only thing that says "warning". A roughly square logo looks best; + a very wide one is scaled down. + +### Previewing + +The console's Policy page shows the banner exactly as recipients get it, from +the same renderer and validator the collector uses, before you save. The same +preview is available from the API as `POST /banner/preview`. + +### Switch `enabled` is what lets **automation** banner this organization's mail: with it off, an automation, a D&R rule or the AI triage agent asking for @@ -379,7 +438,9 @@ wording is the packaged sentence. Bannering also needs the provider capability: `Mail.ReadWrite` is enough on Microsoft 365 (edited in place), while Google Workspace additionally needs the -optional `https://mail.google.com/` scope and **replaces** the message. +optional `https://mail.google.com/` scope and **replaces** the message. On Google +Workspace, a plain-text part of a message can only carry text, so there the banner is +two lines (title, then wording) and the logo and colour do not apply. --- diff --git a/docs/email-security/remediation.md b/docs/email-security/remediation.md index f9927e946..53dfc0f4e 100644 --- a/docs/email-security/remediation.md +++ b/docs/email-security/remediation.md @@ -281,7 +281,7 @@ route and renders each member's outcome by name. The console's bulk action list is a deliberate subset: `banner_message` is offered (a bulk banner still uses the organization's own -[banner policy](policy.md#banners); no client supplies HTML), and +[banner policy](policy.md#banners), and may carry one plain-text `text` that replaces the wording for the whole job; no client supplies HTML), and `unbanner_message` is not — un-bannering is a per-message follow-up taken from a bannered row's timeline, not a sweep. The API accepts all six. From b27f16ea297aa14a6d6072cbacf5d170851c6a4d Mon Sep 17 00:00:00 2001 From: Maxime Lamothe-Brassard Date: Wed, 30 Sep 2026 23:25:45 +0000 Subject: [PATCH 04/10] Email Security docs: a rule's invalid banner text is dropped, not fatal --- docs/email-security/automation.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/email-security/automation.md b/docs/email-security/automation.md index 6859190cc..db1571ffb 100644 --- a/docs/email-security/automation.md +++ b/docs/email-security/automation.md @@ -174,8 +174,11 @@ respond: text: "Payment details changed in this message. Confirm by phone before paying." ``` -`text` is refused, with the reason in the rule's result, if it contains markup -or control characters or is sent on any action other than `banner_message`. Automated bannering also requires +A `text` that contains markup, control characters or is over 512 characters is +**dropped** and the banner goes out with the organization's own wording, because +a rule's values are usually templated from the message and a sender must not be +able to decide whether the warning appears. `text` on any action other than +`banner_message` is refused as a rule error. Automated bannering also requires `enabled` on the [`banners` record](policy.md#banners); without it a rule's `banner_message` is decided and audited but the mailbox is not touched (`alert_only`). Bannering asked for by a person — console, API, CLI — is not From 0128c6e482ff7db6a8906c4ea09da99c69bad840 Mon Sep 17 00:00:00 2001 From: Maxime Lamothe-Brassard Date: Wed, 30 Sep 2026 23:40:31 +0000 Subject: [PATCH 05/10] Email Security docs: stray text is ignored; do not template sender-controlled fields into banner text --- docs/email-security/automation.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/email-security/automation.md b/docs/email-security/automation.md index db1571ffb..83f5e9d3d 100644 --- a/docs/email-security/automation.md +++ b/docs/email-security/automation.md @@ -178,7 +178,10 @@ A `text` that contains markup, control characters or is over 512 characters is **dropped** and the banner goes out with the organization's own wording, because a rule's values are usually templated from the message and a sender must not be able to decide whether the warning appears. `text` on any action other than -`banner_message` is refused as a rule error. Automated bannering also requires +`banner_message` is ignored. Because `text` replaces the organization's +wording, avoid templating sender-controlled fields (the display name, the +subject) into it: whatever you put there is shown to the recipient as part of +the warning. Automated bannering also requires `enabled` on the [`banners` record](policy.md#banners); without it a rule's `banner_message` is decided and audited but the mailbox is not touched (`alert_only`). Bannering asked for by a person — console, API, CLI — is not From f56f2a4e34b889eed900806b723be0ae88442ebd Mon Sep 17 00:00:00 2001 From: Maxime Lamothe-Brassard Date: Thu, 1 Oct 2026 00:03:24 +0000 Subject: [PATCH 06/10] Email Security docs: which invisible characters banner wording allows --- docs/email-security/policy.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/email-security/policy.md b/docs/email-security/policy.md index 5efe9767a..2ee28dfbb 100644 --- a/docs/email-security/policy.md +++ b/docs/email-security/policy.md @@ -377,9 +377,12 @@ strings, one colour *name*, and one image URL; nothing you write is ever interpreted as HTML or CSS. Text is escaped when the banner is rendered, and accepting markup here would turn a configuration field into stored HTML injection against your own users, so it is refused when the record is written -and neutralized again at render time. Control characters, right-to-left -overrides and zero-width characters are refused too, because they let a warning -read differently from what it says. Tab and newline are allowed in the wording +and neutralized again at render time. Control characters, bidirectional +overrides and isolates, and characters that hide text (zero-width space, word +joiner, byte-order mark, soft hyphen) are refused too, because they let a +warning read differently from what it says. The joiners and the left-to-right, +right-to-left and Arabic letter marks that Persian, Hebrew, Arabic and Indic +writing need are allowed. Tab and newline are allowed in the wording and show as a space. The banner is placed **outside** the container that holds the sender's own HTML From e0a3c657073ca48abf791588163fd7e9d7c31336 Mon Sep 17 00:00:00 2001 From: Maxime Lamothe-Brassard Date: Thu, 1 Oct 2026 00:47:40 +0000 Subject: [PATCH 07/10] Email Security docs: counter keys follow the mailbox identity object (id, address, upn) --- docs/email-security/automation.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/email-security/automation.md b/docs/email-security/automation.md index 83f5e9d3d..7877a2bc9 100644 --- a/docs/email-security/automation.md +++ b/docs/email-security/automation.md @@ -248,8 +248,8 @@ The field to count by depends on the event: | Event | Field | Holds | |---|---|---| -| `EMAIL_MESSAGE`, `EMAIL_VERDICT`, `EMAIL_ACTION` | `event/mailbox/address` (template `{{ .event.mailbox.address }}`) | The protected mailbox the event is about | -| `EMAIL_USER_REPORT` | `event/reporter` (template `{{ .event.reporter }}`) | The address that sent the report to the abuse mailbox, or `unknown` when the report had no usable sender | +| Any event about a mailbox (`EMAIL_MESSAGE`, `EMAIL_VERDICT`, `EMAIL_ACTION`, and the other mailbox-scoped events) | `event/mailbox/address` (template `{{ .event.mailbox.address }}`) | The protected mailbox the event is about. The same `mailbox` object also carries `id` (the provider's stable handle) and `upn` (the sign-in name, which can differ from the address on Microsoft 365), so `{{ .event.mailbox.upn }}` can key a counter by sign-in identity | +| `EMAIL_USER_REPORT` | `event/reporter` (template `{{ .event.reporter }}`) | The address that sent the report to the abuse mailbox, or `unknown` when the report had no usable sender. Its `mailbox` is the abuse mailbox the report arrived in, not the person who reported, so count reports per person with `reporter` | ### Example: five malicious messages to one mailbox in an hour From 8fec365801bef61913f82e05cf7787176df4e2b0 Mon Sep 17 00:00:00 2001 From: Maxime Lamothe-Brassard Date: Thu, 1 Oct 2026 03:26:40 +0000 Subject: [PATCH 08/10] Describe mailbox sensors in automation overview --- docs/email-security/automation.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/docs/email-security/automation.md b/docs/email-security/automation.md index 7877a2bc9..03214d2e8 100644 --- a/docs/email-security/automation.md +++ b/docs/email-security/automation.md @@ -8,11 +8,16 @@ cloud and identity data — which is what makes "a phish was delivered, and then that user's endpoint ran a new binary" one rule instead of two products and a spreadsheet. -## The sensor - -Each mail connection appears as **one cloud sensor** on platform `email`. The -mailbox is a field on the event, not an identity: a ten-thousand-mailbox tenant -is one sensor, not ten thousand. +## The sensors + +Each protected mailbox appears as its own sensor on platform `email`, keyed by +the provider's stable mailbox id and named by its normalized primary address. +Mailbox-scoped events carry `mailbox: {id, address, upn}`; `upn` is present when +known and can differ from the address on Microsoft 365. Events without a mailbox +use the connection's sensor (`mailsec-`). The `email` platform +does not count against the sensor quota. Watch coverage and `EMAIL_INGEST_ERROR` +to assess ingestion health; the connection sensor's online state reflects only +its own events. ## The events From 8cca5fb5b8ebc666e74be7f49098883d2aae229c Mon Sep 17 00:00:00 2001 From: Maxime Lamothe-Brassard Date: Thu, 1 Oct 2026 03:55:17 +0000 Subject: [PATCH 09/10] Explain body hunting and search cost confirmation --- docs/email-security/api-reference.md | 2 +- docs/email-security/automation.md | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/docs/email-security/api-reference.md b/docs/email-security/api-reference.md index ac495e4b6..9491cff5d 100644 --- a/docs/email-security/api-reference.md +++ b/docs/email-security/api-reference.md @@ -400,7 +400,7 @@ close a report must be able to reopen one, or a mis-click is permanent. |---|---| | `POST /analyze` | Parse a raw message into the Message Data Model and judge it with the organization’s enabled `dr-mail` rules and resolved scoring policy. **Nothing is ingested or stored**: no index row is written, no raw copy kept, and the organization's mail history is unchanged. Body: `eml_b64` (preferred) or `eml`, plus optional `org_domains` and `direction` — one of `inbound`, `internal`, `outbound`; anything else is refused with a `400` rather than analysed, because many default detections apply to inbound mail only and a mistyped direction would silently answer a lower verdict. Omit it to judge with no direction. Tenant context it cannot have — your sender history, your VIP list — is named explicitly in the payload rather than silently missing. Requires `mailsec.get` | | `POST /actions/bulk/preview` | Preview a bulk remediation over a caller-supplied selection: reports each message's current state and the distinct-mailbox blast radius, and mints the `confirm` token derived from that exact selection. **Nothing is changed and no job is created** — it is a `POST` only because up to 500 message ids do not belong in a query string. Requires `mailsec.get`, like the campaign preview it mirrors. See [Bulk Remediation](remediation.md) | -| `POST /banner/preview` | Render a candidate [`banners` policy](policy.md#banners) exactly as recipients would see it, without saving. Body: `banner` (the record's fields, without `policy_type`), optional `verdict` (which variant to preview) and optional `text` (an action's wording). Answers 200 with `valid: true` and the rendered `html`, or `valid: false` and the validator's reason. Requires `mailsec.get` | +| `POST /banner/preview` | Render a candidate [`banners` policy](policy.md#banners) exactly as recipients would see it, without saving. Body: `banner` (the record's fields, without `policy_type`), optional `verdict` (which variant to preview) and optional `text` (an action's wording). Answers 200 with `valid: true`, the rendered `html`, supported `colors` and field `limits`, or `valid: false` and the validator's reason. Requires `mailsec.get` | | `POST /rules/validate` | Compile a candidate `dr-mail` rule and report its errors without saving it. Body: `rule` (object), optional `rule_id`. Runs the same validator the `dr-mail` Hive applies on save, including lookup existence checks when the API's Hive metadata access is configured. Response blocks receive shape and size checks; full response compilation happens in the collector. See [Custom Rules](custom-rules.md#validation). An invalid rule is a `200` carrying `valid: false` and the reason, not an error response. Requires `mailsec.get` | | `POST /rules/backtest` | Evaluate a candidate `pre_verdict` rule over re-parsed stored messages. Original pipeline enrichments are not reconstructed. Lookups use current records when the Hive resolver is configured; `post_verdict` rules are refused. See [backtest limitations](custom-rules.md#what-a-backtest-can-evaluate). Body: `rule`, optional `rule_id`, `since`, `until`. Every response carries a `coverage_note` and counts what it could not examine (`skipped_no_raw`, `skipped_unparse`, `truncated`). `precision` is `null` — not `0` — when nothing it matched has an analyst disposition yet. Every message in the window is re-read from storage, which makes this the most expensive read on the surface: it is subject to the [replay budget](#the-replay-budget). Requires `mailsec.get` | diff --git a/docs/email-security/automation.md b/docs/email-security/automation.md index 03214d2e8..4461f6359 100644 --- a/docs/email-security/automation.md +++ b/docs/email-security/automation.md @@ -366,6 +366,18 @@ The **Email Security → Hunt** screen runs ordinary LCQL search over filters can also be opened in the Query Console. Other emitted `EMAIL_*` events are searchable in the Query Console and through `limacharlie search`. +**Body contains** matches a case-insensitive phrase in any of the message's +current authored thread, visible HTML text or plain-text part. The phrase is +limited to 256 characters and cannot contain control characters or line breaks. +It searches the body text retained in the event, subject to ingestion limits; +it does not fetch the original EML. Narrow the time window and other filters +before searching bodies, because the search reads message text across the window. + +Before running, Hunt estimates the search cost. Small priced searches can start +immediately; larger searches ask you to confirm. If the estimate is unavailable +or unpriced, Hunt says so and asks before running, rather than treating the +search as free. The estimate is a guide; the final charge can differ. + These searches cover retained telemetry, independently of the Email Security message index and raw-message retention. They can find older emitted messages when telemetry is retained longer than the index. Initial historical backfill From 47f78e28bddbf95e3c1103c99699707090ab8e8d Mon Sep 17 00:00:00 2001 From: Maxime Lamothe-Brassard Date: Thu, 1 Oct 2026 04:15:12 +0000 Subject: [PATCH 10/10] Use canonical report outcomes in CLI examples --- docs/email-security/api-reference.md | 2 +- docs/email-security/cli.md | 4 ++-- docs/email-security/user-reports.md | 10 ++++++---- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/email-security/api-reference.md b/docs/email-security/api-reference.md index 9491cff5d..1607583a7 100644 --- a/docs/email-security/api-reference.md +++ b/docs/email-security/api-reference.md @@ -313,7 +313,7 @@ faster, which is worth doing for its own sake. | `POST /messages/{msg_uuid}/actions` | Perform a typed action on one message. Body: `action` (`quarantine_message`, `trash_message`, `move_to_spam`, `restore_message`, `banner_message`, `unbanner_message`), optional `force` (boolean; see [alert-only overrides](#explicit-override-in-alert-only-mode)), optional `reason`, optional `attempt` (idempotency token — omit to collapse onto the existing attempt). `banner_message` uses the organization's own banner, rendered from its `mailsec_policy` record of type `banners`; its optional `text` (plain text, at most 512 characters; refused on any other action) replaces the wording for that one banner. No caller supplies HTML. Requires `mailsec.act` | | `POST /campaigns/{campaign_id}/actions` | Sweep a campaign. Same body plus `confirm`. **Without `confirm` this previews** and changes nothing, returning the member ids, the distinct mailboxes, the counts and a `confirm` token derived from that exact member set. With `confirm` it executes exactly that set; a campaign that grew since the preview is refused. Capped at 500 members. `reason` is recorded on **every member's** audit row and on the sweep's own row (`action_id` in the response); `attempt` (bounded at 128 characters, refused not truncated) mints a new row per member, so a deliberate retry is recorded beside what it retried instead of over it. Neither is part of the `confirm` token. Requires `mailsec.act` | | `POST /actions/bulk/execute` | Execute a previewed bulk remediation. Returns a `bulk_id` immediately and the provider work proceeds in the background. Requires `mailsec.act`. See [Bulk Remediation](remediation.md) | -| `POST /reports/{report_id}/resolve` | Record a triage outcome. Body: `disposition` — one of `true_positive`, `false_positive`, `benign`. Resolving an already-resolved report succeeds and reports `already_resolved`, so two analysts clicking at once is not an error. Requires `mailsec.set` | +| `POST /reports/{report_id}/resolve` | Record a triage outcome. Body: `disposition` — one of `malicious`, `spam`, `graymail`, `benign`, `simulation`. Resolving an already-resolved report succeeds and reports `already_resolved`, so two analysts clicking at once is not an error. Requires `mailsec.set` | | `POST /reports/{report_id}/reopen` | Put a resolved report back in the queue — see [`POST /reports/{report_id}/reopen`](#post-reportsreport_idreopen). Requires `mailsec.set` | | `POST /messages/{msg_uuid}/verdict` | Re-judge one message — see [`POST /messages/{msg_uuid}/verdict`](#post-messagesmsg_uuidverdict). Requires `mailsec.act` | | `POST /connections/{record}/test` | Probe a configured connection and report each requirement independently: the credential, each scope, a real directory read, and — for Google Workspace — the notification subscription and topic. Every check carries `id`, `name`, `required`, `status`, and on failure `detail` and `remediation`. A failed **optional** check leaves `ok` true. Body: `include_watch` (Workspace only; the one probe with a side effect — it establishes an idempotent, self-expiring push watch). Takes a **record name, not a credential**. Requires `mailsec.act` | diff --git a/docs/email-security/cli.md b/docs/email-security/cli.md index a771ea0e8..8dc006ddf 100644 --- a/docs/email-security/cli.md +++ b/docs/email-security/cli.md @@ -111,7 +111,7 @@ limacharlie mailsec action get # Abuse-mailbox reports limacharlie mailsec report list --status open --oldest-first limacharlie mailsec report get -limacharlie mailsec report resolve --disposition true_positive +limacharlie mailsec report resolve --disposition malicious limacharlie mailsec report reopen # Custom rules @@ -399,7 +399,7 @@ limacharlie mailsec campaign action "$CAMPAIGN" --action quarantine_message \ # Resolve the oldest open report REPORT=$(limacharlie mailsec report list --status open --oldest-first --limit 1 \ --output json | jq -r '.reports[0].report_id') -limacharlie mailsec report resolve "$REPORT" --disposition true_positive +limacharlie mailsec report resolve "$REPORT" --disposition malicious ``` Because the CLI is the whole surface, it is also how an diff --git a/docs/email-security/user-reports.md b/docs/email-security/user-reports.md index b69fc661c..5e055a342 100644 --- a/docs/email-security/user-reports.md +++ b/docs/email-security/user-reports.md @@ -68,14 +68,16 @@ as a gap so you can tell "we could not find it" from "we did not look". ## Resolving ```bash -limacharlie mailsec report resolve --disposition true_positive --oid $OID +limacharlie mailsec report resolve --disposition malicious --oid $OID ``` | Disposition | Meaning | |---|---| -| `true_positive` | It was malicious | -| `false_positive` | We flagged it and it was fine | -| `benign` | It was never a threat | +| `malicious` | It was malicious | +| `spam` | Unwanted mail | +| `graymail` | Legitimate bulk or marketing mail | +| `benign` | Legitimate, harmless mail | +| `simulation` | A known training or simulation message | Resolving requires `mailsec.set`, **not** `mailsec.act`: it changes triage state the product owns, and touches nobody's mailbox. That is the line `mailsec.act`