diff --git a/docs/email-security/detections.md b/docs/email-security/detections.md index 0fa15ac40..541b7a1d8 100644 --- a/docs/email-security/detections.md +++ b/docs/email-security/detections.md @@ -285,6 +285,51 @@ attachment threats, suspicious content, detonation evidence and graymail. They are ordinary D&R rules over the Message Data Model, not a separate engine. See [Mail Rules](custom-rules.md) for the format, IaC and explicit restoration. +### Callback phishing and HTML smuggling + +Two attack classes are invisible to a scanner that only looks at links and known-bad +files, so the defaults read them from structure. + +**Callback phishing** (telephone-oriented attack delivery) is an invoice, renewal or +"your device is infected" notice whose only action is a phone number. The defaults +read the number as a fact ([PhoneNumbers](rule-reference.md#phonenumbers)) from the +body, from the text of attached images, from numbers in a PDF, and from attached +messages, and combine it with a call to action, billing vocabulary, how short the +message is, and whether the sender looks odd (a free-mail address, a young domain, a +failing DMARC result, a Reply-To elsewhere). A legitimate vendor's receipt carries the +same words and a support number, which is why a sender oddity is required and an +established sender is never read as a lure. The callback rules describe one +observation and do not add up: a message that trips all of them scores as the +strongest. A PDF's wording is not available to rules, so the PDF rule judges a short +PDF by its shape and its numbers; numbers in a PDF are recognised for North American +formats only. + +**HTML smuggling** is a web page, often an `.html` or `.svg` attachment, that builds +the real payload in the victim's browser. The parser scans every HTML-like attachment +and HTML body in full and reports encoded data, the type that data decodes to, +decoding and download primitives, redirects and password forms +([HTMLIndicators](rule-reference.md#htmlindicators)). Defaults flag a page that +decodes encoded data and saves it, a page whose encoded data is an archive or +executable, a page that builds its own decoder, an HTML sign-in page delivered as a +file, a tiny redirect page, an SVG that carries script, and a message body that runs +a decoder. A single-file report or export tool that embeds data and offers a download +button matches the same facts as a smuggling page and is scored as suspicious, not +malicious, unless its data also decodes to a recognisable payload. Credential-page +and tiny-redirect defaults require a browser-file extension; source templates and +files with unconventional names can fall outside those two checks. + +In managed pack `0.6.0`, these 22 new rules carry explicit severity. Older managed +rules currently use the informational fallback. Severity is independent of the +verdict, so a malicious verdict from an older rule can still have informational +severity. + +Display-name brand impersonation ("PayPal Support" over an unrelated address), +advance-fee and extortion text, voicemail and fax lures, free-hosting and +open-redirector links, internationalised look-alike domains, OneNote files, locked +PDFs with the password in the message, and web pages hidden inside archives from a +stranger are covered by further defaults. **Email Security → Rules** shows every rule's +conditions and false-positive notes. + A verdict's `engine_version` is a SHA-256 fingerprint of the scoring rules, resolved thresholds, exclusions, VIPs, threat-feed references and clustering policy, and linked parsing/enrichment library build. @@ -292,6 +337,31 @@ Changing rule content or scoring policy changes the fingerprint. It identifies the decision configuration; it is not a promise that an external lookup feed or other message enrichment is unchanged. +### Outbound PII detections + +The optional email DLP pack adds outbound detections for validated payment card +numbers, IBANs and US Social Security numbers, plus a bulk detection when any +one kind has at least ten distinct values. Bulk counts are per kind: four cards, +four IBANs and four SSNs do not meet the bulk threshold. Bulk detections fire +alongside the matching single-kind detection. These are platform D&R rules on +`EMAIL_MESSAGE`, separate from the engine verdict; installing them does not +change the verdict or automatically move mail. + +The rules read [PIIFindings](rule-reference.md#piifindings), which stores counts +only. A detection still carries the originating email event, whose body can +contain the actual sensitive values; the count facts do not redact that body. +Plan detection access and outputs accordingly. IBANs commonly occur on ordinary +invoices, and dashed SSN-shaped internal IDs can match. Tune the optional rules +for your organization rather than treating a match as proof of malicious intent. +The detector covers message text, OCR and attached messages, but does not inspect +text inside ordinary document, spreadsheet or PDF files. Counts remain lower +bounds for these excluded sources even when `enrichments/pii/truncated` is absent. +Known incomplete extraction or parsing, and inspection limits, set that flag; +the counts still describe only the available text. +Deferred attachment scans refresh the stored facts but do not replay the initial +`EMAIL_MESSAGE` evaluation. A DLP match therefore describes evidence available +when that event was emitted, rather than every later attachment result. + ## Link detonation Static link features answer what a URL *looks* like. Detonation answers where it diff --git a/docs/email-security/rule-reference.md b/docs/email-security/rule-reference.md index e8cb83b3a..c1e255fd0 100644 --- a/docs/email-security/rule-reference.md +++ b/docs/email-security/rule-reference.md @@ -75,6 +75,12 @@ Scoring classes require `weight` from 1 to 100; graymail records must omit it. | What did attachment analysis actually inspect? | Scope `attachments`; inspect `explode/scanners` before interpreting scanner-specific results | | Is this a known sender? | `enrichments/sender_profile/prevalence` (`none`, `new`, `rare`, `common`) | | Is the sender impersonating an organization? | `enrichments/lookalike/org_domain_distance`, `enrichments/lookalike/vip_hit` | +| Is the display name a well-known brand over an address that is not the brand's? | `enrichments/lookalike/display_name_brand` | +| Does it contain validated payment cards, IBANs or US Social Security numbers? | `enrichments/pii/card_numbers`, `ibans`, `us_ssns`; counts only, with `truncated` for incomplete inspection. See [PIIFindings](#piifindings) | +| Does the message tell the reader to call a number (callback phishing)? | `enrichments/phone_numbers/body` and `enrichments/phone_numbers/attachments`; read `call_to_action`, `lure_terms`, `toll_free`. See [PhoneNumbers](#phonenumbers) | +| Is this attachment a web page that builds a file in the browser (HTML smuggling)? | Scope `attachments`; read `html/base64_bytes`, `html/payload_types`, `html/blob_download`, `html/atob`. See [HTMLIndicators](#htmlindicators) | +| Is a PDF short, locked, or carrying phone numbers? | Scope `attachments`; read `explode/pdf`. See [PDFInfo](#pdfinfo) | +| How much of the message does the reader actually see? | `body/current_thread/visible_chars` | | What happened after a link was fetched? | `enrichments/detonation`; see [Link detonation](detections.md#link-detonation) | | Was parsing or analysis incomplete? | `_meta/truncations`, `_meta/errors`, `_meta/explode_timeout`, `body/truncated` | @@ -315,6 +321,7 @@ the pipeline includes the whole body in `current_thread` for an unverified reply | `inner_text` | string | Non-empty | | `display_text` | string | Non-empty | | `charset` | string | Non-empty | +| `indicators` | [HTMLIndicators](#htmlindicators) | When the scan found something | ### PlainBody @@ -329,8 +336,15 @@ the pipeline includes the whole body in `current_thread` for an unverified reply | `text` | string | Non-empty | | `renderings` | array of [ThreadRendering](#threadrendering) | Non-empty | | `visible_text` | string | Non-empty | +| `visible_chars` | integer | Non-empty | | `links` | array of [Link object](#link) | Non-empty | +`visible_chars` is the number of non-whitespace characters in `visible_text`. It +measures how much the reader is shown, and blank-line padding cannot inflate it. +A mail rule cannot compute a length itself because its regular expressions are +limited to short repeat counts, so use this field for "the message is a two-line +note" conditions. + ### ThreadRendering | Field | Type | Presence | @@ -399,8 +413,67 @@ and a decoded destination; when both links are emitted, it is set on both. | `tlsh` | string | Non-empty | | `magic_type` | string | Non-empty | | `is_inline` | boolean | Non-empty | +| `html` | [HTMLIndicators](#htmlindicators) | When the part is HTML-like | | `explode` | [Explode](#explode) | When set | +`html` is computed from the attachment's own bytes when the message is parsed, so +it does not depend on attachment analysis. It is present exactly when the part +was recognised as HTML-like (an HTML or SVG file, or any part whose first bytes +open like a web page, whatever it is called). An absent block means "not a web +page", never "a clean web page". + +### HTMLIndicators + +Structural facts from a bounded scan of an HTML-like document: an attachment, or +the HTML body. HTML smuggling is a structure, not a string: a large blob of +encoded data, a few lines of script that decode it, and a browser call that +saves the result as a download. No single field below is a finding, because HTML +exports embed images as base64 and ordinary pages call `atob`. Combine them. +Keyword fields are matched on a normalised view of the document (lower case, +whitespace and quotes removed), so spacing and case do not matter, but splitting a +word across string pieces (`'at'+'ob'`) defeats them. The encoded data and its +decoded type are reported for that reason. + +| Field | Type | Presence | +|---|---|---| +| `scripts` | integer | Non-empty | +| `event_handlers` | boolean | Non-empty | +| `base64_bytes` | integer | Non-empty | +| `base64_max_run` | integer | Non-empty | +| `numeric_array_bytes` | integer | Non-empty | +| `payload_types` | array of string | Non-empty | +| `atob` | boolean | Non-empty | +| `eval` | boolean | Non-empty | +| `from_char_code` | boolean | Non-empty | +| `unescape` | boolean | Non-empty | +| `document_write` | boolean | Non-empty | +| `blob_download` | boolean | Non-empty | +| `download_attr` | boolean | Non-empty | +| `auto_click` | boolean | Non-empty | +| `js_redirect` | boolean | Non-empty | +| `meta_refresh` | boolean | Non-empty | +| `password_input` | boolean | Non-empty | +| `remote_form_action` | boolean | Non-empty | +| `truncated` | boolean | Non-empty | + +- `scripts` counts `