diff --git a/docs/assets/showcase/demo.gif b/docs/assets/showcase/demo.gif new file mode 100644 index 0000000..96890ba Binary files /dev/null and b/docs/assets/showcase/demo.gif differ diff --git a/docs/commercial/phi-handling.md b/docs/commercial/phi-handling.md new file mode 100644 index 0000000..900d0a4 --- /dev/null +++ b/docs/commercial/phi-handling.md @@ -0,0 +1,81 @@ +# PHI handling in one page + +How OpenAdapt handles protected health information (PHI) and other regulated +identifiers, end to end: what is scrubbed, what is deliberately retained and +where, what a human reviews before anything leaves your boundary, and why the +shareable receipt cannot carry PHI at all. Every mechanism below is inspectable +open source; the deeper dossier is +[Security and data handling](../guides/security-and-data-handling.md). + +## 1. Scrubbing on the persist and log paths + +PHI/PII scrubbing is provided by +[openadapt-privacy](https://github.com/OpenAdaptAI/openadapt-privacy) +(Presidio-backed named-entity recognition) through a single choke point in the +engine: + +- The shareable `REPORT.md` passes every free-text field through the scrubber. +- Persisted step and heal frames are routed through image redaction. +- Drift-oracle console output is scrubbed before printing. +- A regulated deployment pins `OPENADAPT_FLOW_SCRUB=on` and **fails closed**: a + missing scrubbing capability aborts the run instead of writing plaintext. +- There is no silent plaintext: in the default mode without the privacy extra, + writing identity-like free text emits an explicit `PlaintextPHIWarning`. + +**The honest boundary.** The recorded identity evidence and the identity audit +trail intentionally retain literal identifiers — scrubbing them would defeat +the wrong-record check they exist to power. Those artifacts are governed as +PHI-at-rest **inside your boundary** (filesystem controls, retention, +full-disk encryption, opt-in AES-256-GCM sealing), and the published privacy +map says so explicitly. + +## 2. The local review gate before any upload + +Nothing raw is uploaded. The only artifact lane to the hosted control plane is +the sanitized-derivative pipeline, and it puts a human decision between your +data and the wire: + +1. `openadapt flow sanitize` builds a **derivative** with a file inventory and + recorded transformations. +2. `openadapt flow review-sanitized` presents the derivative for **operator + review** in a local viewer. +3. `openadapt flow approve-sanitized` records the explicit approval; only then + can the derivative be pushed, and the control plane verifies the manifest, + review state, and exact archive SHA-256 before accepting a byte. + +Content the sanitizer cannot fully handle — databases, video, audio, nested +archives, symlinks, unknown binaries — refuses the **entire** derivative +rather than passing through. Sanitizer success is not treated as proof of +de-identification: the operator review is the gate. + +## 3. The receipt: an allow-list, not a redaction + +The shareable run receipt is generated **additively from a closed allow-list, +never redacted subtractively** from the rich operator report. Every field is a +closed enum, a bounded count, a digest, or a strictly validated version +string; an unknown key is refused rather than silently dropped. + +Structurally unrepresentable in a receipt: screenshots, OCR text, typed +values, parameters, URLs, hostnames, coordinates, application name, +organization name, user name, workflow name, step intents, and halt free text. +The same principle governs the hosted attended-decision envelope (closed +enums, bounded integers, booleans — no string field, no image) and the hosted +break-report descriptor (hashed, coarse, no free text). + +## Where each artifact can live + +| Artifact | PHI posture | Boundary | +|---|---|---| +| Recording, bundle, `report.json`, identity evidence | Retains literal identifiers by design (audit + wrong-record check) | Stays inside your boundary; seal with AES-256-GCM for at-rest protection | +| `REPORT.md` | Scrubbed free text; redacted frames under the regulated pin | Your boundary; review before any sharing | +| Sanitized derivative | Operator-reviewed, transformation-manifested | May cross to the hosted control plane after explicit approval | +| Run receipt | Allow-list only; PHI structurally unrepresentable | Shareable | +| Hosted halt descriptor / decision envelope | Closed enums, counts, digests; no strings, no images | Hosted control plane | + +## Related pages + +- [Security packet](security-packet.md) — the reviewer summary. +- [Subprocessors and hosted data retention](subprocessors.md) +- [Fail-closed regulated execution](../concepts/regulated-execution.md) +- Engine [PRIVACY.md](https://github.com/OpenAdaptAI/openadapt-flow/blob/main/docs/PRIVACY.md) — + the complete path-by-path PHI map. diff --git a/docs/commercial/security-packet.md b/docs/commercial/security-packet.md index 805c6ee..3be5403 100644 --- a/docs/commercial/security-packet.md +++ b/docs/commercial/security-packet.md @@ -4,7 +4,10 @@ The honest current state of OpenAdapt's security posture, written for a security or vendor-risk reviewer. Deeper technical detail: [Security and data handling](../guides/security-and-data-handling.md) and the reviewer-oriented -[Security and deployment review](../guides/security-review.md). Nothing on +[Security and deployment review](../guides/security-review.md). Companion +pages: [Subprocessors and hosted data retention](subprocessors.md), +[Vulnerability disclosure](vulnerability-disclosure.md), and +[PHI handling in one page](phi-handling.md). Nothing on this page is a compliance determination. ## Architecture in one paragraph @@ -49,7 +52,8 @@ append-only, hash-chained audit log. | SOC 2 | **Not attested.** OpenAdapt does not hold a SOC 2 report and does not claim certification. Request the current security-controls packet for implementation evidence and remaining gaps. | | HIPAA / BAA | No standing BAA offering. Deployments that touch PHI use a customer-controlled boundary; BAA and counsel review are engagement-specific. | | Penetration test | Request current status directly; do not infer from documentation. | -| Vulnerability disclosure | Coordinated disclosure via private GitHub advisories, acknowledgment target within 5 business days (engine `SECURITY.md`). | +| Vulnerability disclosure | Coordinated disclosure: private GitHub advisories for the open-source packages, `hello@openadapt.ai` for hosted surfaces; acknowledgment target within 5 business days. Full channel and scope: [vulnerability disclosure](vulnerability-disclosure.md). | +| Subprocessors | Hosted surfaces only; current provider list and roles: [subprocessors and hosted data retention](subprocessors.md). Local and on-prem deployments use none at run time. | | Supply chain | GitHub Actions pinned by commit SHA; Dependabot; public Desktop releases ship `SHA256SUMS`, a CycloneDX SBOM, and build-provenance attestations. Windows/Linux installers are currently unsigned and macOS is ad-hoc signed; verify checksums and provenance. | If a security questionnaire needs a signed answer on any row, request the @@ -80,6 +84,31 @@ Sensitive live observations never route into the shared managed boundary; a recording that was sanitized for upload does not make runtime data sanitized, and the documentation says so explicitly rather than implying otherwise. +What crosses each boundary, per shape: + +```mermaid +flowchart TB + subgraph localb [Fully local / air-gapped on-prem] + LE[Engine + recorder + verifier] --> LT[Target application] + end + subgraph byocb [Customer-controlled runner / BYOC] + BE[Runner in customer VM or cloud account] --> BT[Target application] + end + subgraph hostedb [Hosted control plane] + HC[Accounts, workflow versions,
run history, billing] + end + subgraph managedb [Managed browser runner] + MR[Public-HTTPS,
non-regulated targets only] + end + localb -. "nothing at run time;
signed updates on operator-pulled media" .-> hostedb + byocb -. "declared results, evidence permitted by the
deployment data boundary, minimized control metadata;
raw frames and PHI/PII stay inside" .-> hostedb + hostedb -. "bounded authorization
and control metadata" .-> byocb + hostedb --- MR +``` + +The PHI-specific narrative — scrubbing, the operator review gate, and the +allow-list receipt — is one page: [PHI handling](phi-handling.md). + ## What we will not claim - Architecture documentation is not a HIPAA, PHIPA, SOC 2, or other diff --git a/docs/commercial/subprocessors.md b/docs/commercial/subprocessors.md new file mode 100644 index 0000000..e97c9f9 --- /dev/null +++ b/docs/commercial/subprocessors.md @@ -0,0 +1,93 @@ +# Subprocessors and hosted data retention + +This page lists the third-party service providers the **hosted** OpenAdapt +surfaces use, and the current hosted retention and deletion behavior. It is +informational documentation for a vendor-risk reviewer, not a contractual +subprocessor notice; request the current signed artifact for anything a +contract must reference. + +**Scope.** Fully local and on-prem deployments use none of these providers at +run time: a healthy local replay makes zero outbound calls, and the no-egress +posture is operator-verifiable. This page describes the hosted control plane +(`app.openadapt.ai`), the managed browser runner, and the public web +properties. + +## Service providers in use + +As read from the hosted control plane's deployment configuration: + +| Provider | Role | What it can process | +|---|---|---| +| Netlify | Hosting for the `app.openadapt.ai` control plane | Application traffic to the control plane: account and session data in transit, and the metadata/digest surfaces described in the [security packet](security-packet.md). | +| Supabase | Database, authentication, and object storage for the control plane | Accounts, organizations, workflow versions, run metadata, sanitized artifact derivatives, retention/erasure receipts. | +| Modal | Compute for the managed browser runner | Managed browser execution for explicitly initiated, public-HTTPS, non-regulated workloads only — not a lane for PHI/PII. | +| Stripe | Payments and billing | Payment and subscription data. Card data is entered on Stripe's surfaces, not OpenAdapt's. | +| Resend | Transactional email (organization invites, purchase alerts) | Recipient email addresses and the fixed-template message content. Purchase alerts carry purchase metadata only, never workflow evidence. | +| GitHub | Source hosting, CI, release distribution | Public source, build artifacts, and CI logs. No customer workload data. | +| PostHog | Product analytics on the public web properties and control plane (when enabled) | Usage events on OpenAdapt's own pages. Not wired into the local engine or run path. | +| GlitchTip | Error monitoring for the control plane (when enabled) | Control-plane error reports. | +| Google Analytics | Web analytics on the public website and docs | Page-view analytics on public pages only. | + +The local engine has no telemetry, analytics, license check, or update ping in +the run path; analytics providers above apply to OpenAdapt's own hosted pages, +not to your workflows. + +## Hosted retention and deletion + +The hosted service applies a **versioned, explicitly configured retention +policy** — there is no implicit retention duration. The policy names its +version and sets explicit windows for recordings, reports, and run metadata +(run metadata is never retained shorter than reports), plus a backup recovery +window and a maximum restore-drill age. + +Current behavior: + +- **Scheduled deletion is fail-closed.** Destructive scheduled retention + refuses to run without a recent receipt proving a complete database **and** + private object storage restore into an isolated scratch environment. +- **Legal holds pause eligible deletion** for the held organization. +- **Tenant erasure is organization-scoped** and produces an append-only, + PHI/PII-free receipt with identifiers, counts, and digests — never deleted + payloads. +- The public [readiness endpoint](https://app.openadapt.ai/api/health/ready) + reports the configured retention component separately from the + destructive-operation gate. + +The concrete day windows are deployment configuration, reviewed with the +policy version; request the current policy version and its windows directly +rather than citing this page. + +## Data-flow summary + +What crosses each boundary, by deployment shape (full narrative: +[Security packet](security-packet.md) and +[deployment boundaries](deployment-boundaries.md)): + +```mermaid +flowchart TB + subgraph local [Fully local / on-prem] + L1[Recorder + engine + verifier] --> L2[Your target app] + end + subgraph byoc [Customer-controlled runner / BYOC] + B1[Runner in your VM or cloud] --> B2[Your target app] + end + subgraph hosted [Hosted control plane] + H1[Accounts, versions, run history, billing] + end + local -. "nothing at run time" .-> hosted + byoc -. "declared results + bounded control metadata;
raw frames and PHI/PII stay inside" .-> hosted + hosted -. "bounded authorization and
control metadata" .-> byoc + hosted --> P[(Supabase)] + hosted --> N[(Netlify)] + hosted --> S[(Stripe)] + hosted --> M[(Modal:
managed browser runner,
non-regulated only)] + hosted --> R[(Resend)] +``` + +## Related pages + +- [Security packet](security-packet.md) — posture summary for reviewers. +- [Vulnerability disclosure](vulnerability-disclosure.md) — how to report. +- [PHI handling](phi-handling.md) — the end-to-end PHI narrative. +- [Security and data handling](../guides/security-and-data-handling.md) — the + full technical dossier, including hosted retention detail. diff --git a/docs/commercial/vulnerability-disclosure.md b/docs/commercial/vulnerability-disclosure.md new file mode 100644 index 0000000..f989f1a --- /dev/null +++ b/docs/commercial/vulnerability-disclosure.md @@ -0,0 +1,56 @@ +# Vulnerability disclosure + +OpenAdapt is deployed next to sensitive systems. Security reports are taken +seriously and handled through coordinated disclosure. Please do not open a +public issue, discussion, or pull request containing vulnerability details. + +## Where to report + +| Component | Channel | +|---|---| +| Open-source engine and packages (`openadapt-flow`, `openadapt-capture`, `openadapt-privacy`, and the other public repositories) | GitHub's private advisory channel on the affected repository: **Security tab → Advisories → Report a vulnerability** (for the engine: [report directly](https://github.com/OpenAdaptAI/openadapt-flow/security/advisories/new)). | +| Hosted control plane (`app.openadapt.ai`), managed runner, hosted recorder | Email **hello@openadapt.ai** with "Security" in the subject. | + +If you cannot use the private GitHub channel, open a public issue that contains +**no details** and asks a maintainer to open a private channel with you. + +## What to include + +The affected route, component, or repository; the impact; reproduction steps; +and any suggested remediation. Do **not** include live customer payloads, +credentials, screenshots, or other sensitive evidence in the first message — +an appropriate transfer channel will be arranged if needed. + +## What to expect + +- Acknowledgment target: within **5 business days**. +- Affected versions are confirmed and the reporter is kept informed of the + remediation plan. +- Reporters who want recognition are credited after a fix is released. +- Fixes ship forward on the latest released version; there is no + long-term-support branch at this stage. + +## Scope + +Explicitly in scope for the hosted surfaces: tenant-isolation failures, +authentication or MFA bypasses, secret exposure, artifact-admission bypasses, +managed-runner escape or egress failures, cross-organization access, +billing-entitlement bypasses, and deployment supply-chain findings. + +Engine-specific scope notes: + +- The compiled bundle and `report.json` intentionally retain literal + identifiers (for the identity check and audit trail) behind a documented + boundary. A report that these are exposed **outside** that boundary is in + scope. +- Identity crops sent to the on-prem VLM appliance are deliberately not + scrubbed; the control is on-prem-only plus no-retention. Reports of + retention or off-prem transmission are in scope. +- Supply chain: GitHub Actions are pinned by commit SHA and dependencies flow + through Dependabot; reports of a pinning gap are welcome. + +## Related pages + +- [Security packet](security-packet.md) +- [Security and data handling](../guides/security-and-data-handling.md) +- Engine [SECURITY.md](https://github.com/OpenAdaptAI/openadapt-flow/blob/main/SECURITY.md) diff --git a/docs/get-started/first-workflow.md b/docs/get-started/first-workflow.md index a04ffb6..6c1b762 100644 --- a/docs/get-started/first-workflow.md +++ b/docs/get-started/first-workflow.md @@ -2,24 +2,78 @@ This walks through compiling a workflow on **your own** web app: record what you do, compile it, replay it, and read the report. It takes about five minutes and -makes zero model calls. Web is the quickest substrate to start on; the same +makes zero model calls. Web is the quickest +[substrate](../reference/glossary.md#substrate) to start on; the same record, compile, replay loop drives native Windows, macOS, or Linux applications and RDP or Citrix sessions by choosing a [backend](../reference/cli.md#backend). -## Prerequisites +Here is the loop you are about to run — record once, replay deterministically, +and watch the run heal or halt under drift: + +![Record, compile, and replay a workflow with OpenAdapt](../assets/showcase/demo.gif) + +## Prerequisites and install - **Python 3.10-3.12.** The engine declares `requires-python >=3.10,<3.13`. + Check yours with `python --version` (on some systems `python3 --version`). - **macOS, Linux, or Windows.** This walkthrough selects the Playwright-driven browser capability, so it has no OS-specific steps. Its matching Chromium provisions automatically on the first web action; native, RDP, and Citrix paths do not install it. +- **Use a virtual environment.** It keeps the install isolated and avoids the + stale-package problems a shared or Conda base environment causes. -Install the CLI if you have not already: +Install the CLI for your shell — the quoting around `openadapt[browser]` +differs per shell, and getting it wrong is the most common first failure: -```bash -pip install 'openadapt[browser]' -``` +=== "macOS / Linux (bash, zsh)" + + ```bash + python3 -m venv .venv && source .venv/bin/activate + pip install 'openadapt[browser]' + ``` + + The quotes matter: unquoted square brackets are glob characters in zsh + (`no matches found`) and can misbehave in bash. Single or double quotes + both work here. + +=== "Windows PowerShell" + + ```powershell + py -m venv .venv; .\.venv\Scripts\Activate.ps1 + pip install "openadapt[browser]" + ``` + + PowerShell accepts single or double quotes; double quotes are shown for + consistency with cmd.exe. + +=== "Windows cmd.exe" + + ```bat + py -m venv .venv && .venv\Scripts\activate.bat + pip install "openadapt[browser]" + ``` + + Use **double** quotes. cmd.exe passes single quotes through literally, so + `pip install 'openadapt[browser]'` fails with an *Invalid requirement* + error that starts with `'openadapt`. + +=== "Linux native desktop (AT-SPI)" + + The browser walkthrough on this page needs no extra system packages. To + later record or replay **native Linux applications** (`--backend linux`), + install the AT-SPI runtime and build prerequisites first (Debian/Ubuntu): + + ```bash + sudo apt-get install \ + gcc pkg-config python3-dev libcairo2-dev libgirepository-2.0-dev \ + gir1.2-atspi-2.0 libatspi2.0-0 + pip install 'openadapt[linux]' + ``` + + The built-in driver uses X11; Wayland requires an operator-approved XDG + portal session. Or use the installer script from the landing page, which installs [uv](https://docs.astral.sh/uv/) if needed and sets up a persistent @@ -29,19 +83,28 @@ Or use the installer script from the landing page, which installs curl -fsSL https://openadapt.ai/install.sh | sh -s -- browser ``` -For local development and CI only, `openadapt flow demo-record --out rec` uses -the bundled MockMed fixture. It is not a production workflow or product outcome. +!!! tip "No app to record against yet?" + You do not need your own target to try the loop. The engine bundles + **MockMed**, a synthetic demo clinic app (fake data only): + `openadapt flow demo-record --out rec` serves it locally and records the + canonical triage demo, and `openadapt flow replay bundle` with no `--url` + serves it again as the replay target. It is a local development fixture, + not a production workflow or product outcome — but it is a real, running + web app, so every step below works against it unchanged. ## 1. Record -`record --url` opens a headed browser pointed at your app and watches what you -do: real clicks, typing, key presses, and scrolls. It writes the same recording -format that `compile` consumes. +`record --backend web --url` opens a headed browser pointed at your app and +watches what you do: real clicks, typing, key presses, and scrolls. It writes +the same recording format that `compile` consumes. ```bash -openadapt flow record --url https://your.app --out rec +openadapt flow record --backend web --url https://your.app --out rec ``` +(Omitting `--backend` defaults to `web` with a printed notice; production +profiles require it explicitly.) + Perform the task once. When you are done, press ++ctrl+c++ or close the browser window to finish. The recording is written to `rec/`. @@ -58,10 +121,11 @@ openadapt flow compile rec --out bundle --name my-task Compilation turns the recording into a **workflow bundle**: an ordered list of steps, each carrying the evidence needed to re-find its target (a template crop, -an OCR label, geometry landmarks) and postconditions derived from what the demo -actually changed on screen. Write-shaped clicks (save, submit, create, delete) -are auto-classified as irreversible so they refuse to act on a low-confidence -match. +an OCR label, geometry landmarks — the +[capability ladder](../reference/glossary.md#capability-ladder)) and +postconditions derived from what the demo actually changed on screen. +Write-shaped clicks (save, submit, create, delete) are auto-classified as +irreversible so they refuse to act on a low-confidence match. ## 3. Lint @@ -70,11 +134,21 @@ openadapt flow lint bundle ``` `lint` reports coverage gaps before you trust the bundle: clicks that act with -no identity check, steps that assert nothing, writes that may be -under-classified. Each finding carries a severity. It is advice, not a gate. See +no [identity check](../reference/glossary.md#identity-gate), steps that assert +nothing, writes that may be under-classified. Each finding carries a severity. +It is advice, not a gate. See [Write and enforce a policy](../guides/policy-and-certification.md) for the `certify` gate that refuses an unsafe bundle outright. +!!! success "A nonzero exit here is expected, not broken" + `lint` exits `1` when any finding reaches `error` severity — an unarmed or + vacuous **irreversible** step. That is the safety boundary working: it is + telling you a write-shaped click would act without a wrong-record guard. + Review the findings, then continue to step 4; replay still runs, and the + [`certify` gate](../guides/policy-and-certification.md) is where a failing + bundle is actually refused. All exit codes are listed in + [Run outcomes and halt reasons](../reference/run-outcomes.md#cli-exit-codes). + ## 4. Replay ```bash @@ -83,7 +157,9 @@ openadapt flow replay bundle --url https://your.app Recorded parameter values are the defaults; override any of them with `--param key=value`. The run is deterministic and local. On the healthy path it -makes zero model calls and finishes in seconds. +makes zero model calls and finishes in seconds. `replay` exits `0` on success +and `1` on a [halt](../reference/glossary.md#halt) — a halt is the fail-closed +refusal to guess, not a crash. ## 5. Read the report @@ -92,7 +168,9 @@ illustrated `REPORT.md` and a machine-readable `report.json`. The report tells you, per step, which rung of the resolution ladder resolved the target, whether the identity check was armed and what it verified, which postconditions passed, and any heals that were applied. See [What you get](what-you-get.md) and -[Read and audit run reports](../guides/run-reports.md). +[Read and audit run reports](../guides/run-reports.md). Every outcome and halt +reason the report can show is defined in +[Run outcomes and halt reasons](../reference/run-outcomes.md). ## What is next diff --git a/docs/get-started/index.md b/docs/get-started/index.md index 8bc87e3..5e17053 100644 --- a/docs/get-started/index.md +++ b/docs/get-started/index.md @@ -9,6 +9,11 @@ description: >- Start with one complete local result. Then choose the guide for your target surface. You do not need to understand the package layout first. +This is the whole loop — record a demonstration once, compile it, and replay +it deterministically: + +![OpenAdapt records, compiles, and replays a demonstrated workflow](../assets/showcase/demo.gif) + ## First success: two commands The fastest path needs no account, target application, API key, or @@ -20,10 +25,12 @@ openadapt quickstart ``` The command records the bundled synthetic MockMed task, compiles its observed -effect contract, certifies it with the shipped clinical-write policy, and runs -it under the Standard profile. A separate read-only API confirms the saved -record outside the screen that performed the write. The healthy run returns -`VERIFIED` with no model or Cloud call. OpenAdapt writes all artifacts to +[effect contract](../reference/glossary.md#effect-contract), certifies it with +the shipped clinical-write [policy](../reference/glossary.md#policy), and runs +it under the Standard [profile](../reference/glossary.md#profile). A separate +read-only API confirms the saved record outside the screen that performed the +write. The healthy run returns +[`VERIFIED`](../reference/run-outcomes.md) with no model or Cloud call. OpenAdapt writes all artifacts to `openadapt-quickstart/` and refuses to overwrite that directory. You now have: @@ -87,10 +94,15 @@ openadapt flow replay openadapt-quickstart/bundle \ --run-dir openadapt-quickstart-halt ``` -The command returns a non-zero exit code because the expected outcome is a -halt. Open `openadapt-quickstart-halt/REPORT.md` to see the retained evidence. -Do not retry a possibly dispatched write. Reconcile it against an independent -system of record first. +!!! success "The nonzero exit is the demonstration succeeding" + The command exits `1` **because the expected outcome is a + [halt](../reference/glossary.md#halt)** — the safety boundary refusing to + act on a screen state the compiled program has no branch for. If you see + `Replay HALTED`, the fail-closed gate worked; continue below. Open + `openadapt-quickstart-halt/REPORT.md` to see the retained evidence. Do not + retry a possibly dispatched write; reconcile it against an independent + system of record first. Every outcome is defined in + [Run outcomes and halt reasons](../reference/run-outcomes.md). ## Install a different execution surface @@ -143,6 +155,13 @@ openadapt flow run bundle-prod --config deployment.yaml --dry-run openadapt flow run bundle-prod --config deployment.yaml ``` +!!! success "If `certify` exits nonzero here, the gate is working" + A failing certification exits `2` and prints each violated requirement. + That is the point of the gate: an unsafe bundle is refused before it can + ship. Close the gaps it names (see + [Write and enforce a policy](../guides/policy-and-certification.md)), then + re-run `certify` and continue. + Follow [Run a deployment](../guides/run-a-deployment.md), then complete the [security and deployment review](../guides/security-review.md). Do not promote a bundle just because the sample-app tour passed. `seal` preserves the source, diff --git a/docs/guides/data-driven-loops.md b/docs/guides/data-driven-loops.md index 464fb78..b1b95d1 100644 --- a/docs/guides/data-driven-loops.md +++ b/docs/guides/data-driven-loops.md @@ -10,7 +10,7 @@ demonstration, and [`replay --worklist`](../reference/cli.md#replay) drives it. Record and compile the task once, as you would for a single run: ```bash -openadapt flow record --url https://your.app --out rec +openadapt flow record --backend web --url https://your.app --out rec openadapt flow compile rec --out bundle --name intake ``` diff --git a/docs/guides/induce-a-program.md b/docs/guides/induce-a-program.md index 58ffa47..84ef81c 100644 --- a/docs/guides/induce-a-program.md +++ b/docs/guides/induce-a-program.md @@ -13,9 +13,9 @@ Record the task a few times, varying the values you intend to be parameters (a different note, a different record) and keeping the intended path the same: ```bash -openadapt flow record --url https://your.app --out rec-1 -openadapt flow record --url https://your.app --out rec-2 -openadapt flow record --url https://your.app --out rec-3 +openadapt flow record --backend web --url https://your.app --out rec-1 +openadapt flow record --backend web --url https://your.app --out rec-2 +openadapt flow record --backend web --url https://your.app --out rec-3 ``` ## Induce diff --git a/docs/guides/parameters-and-secrets.md b/docs/guides/parameters-and-secrets.md index 7d5971c..a2ba2e5 100644 --- a/docs/guides/parameters-and-secrets.md +++ b/docs/guides/parameters-and-secrets.md @@ -10,7 +10,7 @@ Mark a typed field as a parameter at record time. Its demonstrated value becomes the default, overridable at replay: ```bash -openadapt flow record --url https://your.app --out rec --param note +openadapt flow record --backend web --url https://your.app --out rec --param note openadapt flow compile rec --out bundle --name my-task openadapt flow replay bundle --url https://your.app --param note="Follow-up in 2 weeks" ``` @@ -48,7 +48,7 @@ log, the compiled bundle, or the saved frames (its on-screen region is redacted) At replay it is injected from the environment, and a missing one fails fast. ```bash -openadapt flow record --url https://your.app --out rec --secret password +openadapt flow record --backend web --url https://your.app --out rec --secret password export OPENADAPT_FLOW_SECRET_PASSWORD='...' # supplied at replay time openadapt flow replay bundle --url https://your.app ``` diff --git a/docs/guides/record-your-app.md b/docs/guides/record-your-app.md index 2b7f57f..ea578bf 100644 --- a/docs/guides/record-your-app.md +++ b/docs/guides/record-your-app.md @@ -8,12 +8,12 @@ target flags in place of `--url`. ## Record -On the web substrate, `record --url` opens a headed browser on your app and +On the web substrate, `record --backend web --url` opens a headed browser on your app and watches what you do: clicks, typing, key presses, and scrolls. It writes the same recording format `compile` consumes. ```bash -openadapt flow record --url https://your.app --out rec +openadapt flow record --backend web --url https://your.app --out rec ``` Perform the workflow the way you want it replayed, then press ++ctrl+c++ or close diff --git a/docs/guides/run-reports.md b/docs/guides/run-reports.md index 911c625..3c4afdd 100644 --- a/docs/guides/run-reports.md +++ b/docs/guides/run-reports.md @@ -29,7 +29,10 @@ For each step, the report records: verified against the system of record, with its verdict. - **Model calls**: any call to a grounding or state-verification model, so the $0 property is observable, not assumed. -- **Outcome**: success, or a halt naming the violated expectation. +- **Outcome**: success, or a halt naming the violated expectation. Every + terminal outcome (`VERIFIED`, `COMPLETED_UNVERIFIED`, `HALTED`, `FAILED`, + `ROLLED_BACK`), the transaction refinement, and every typed halt reason are + defined in [Run outcomes and halt reasons](../reference/run-outcomes.md). ## The identity-coverage line diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md index 6ed9f8b..6575189 100644 --- a/docs/guides/troubleshooting.md +++ b/docs/guides/troubleshooting.md @@ -2,7 +2,9 @@ First-week failure modes, keyed by symptom → cause → fix. Other docs explain *why* the runtime halts ([the identity gate](../concepts/identity-gate.md)); this page is -the operational "it halted, now what." +the operational "it halted, now what." For the definition of every terminal +outcome and every typed halt reason — with per-reason remediation — see +[Run outcomes and halt reasons](../reference/run-outcomes.md). ## Capture is blank or black @@ -131,7 +133,8 @@ into a clean virtualenv instead. See - Re-read the run's `REPORT.md` and `report.json`: the halt reason, the resolver rung, and the drift signature are recorded there. See - [Read and audit run reports](run-reports.md). + [Read and audit run reports](run-reports.md) and + [Run outcomes and halt reasons](../reference/run-outcomes.md). - Check what a bundle is missing before you deploy it with [`lint`](../reference/cli.md#lint) and [`certify`](../reference/cli.md#certify). - Ask on the [Discord](https://discord.gg/yF527cQbDG) or open an issue on diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 08c180a..a8648fd 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -103,7 +103,7 @@ Citrix through their exact target flags. The example below records the web substrate. ```bash -openadapt flow record --url https://your.app --out rec +openadapt flow record --backend web --url https://your.app --out rec ``` | Flag | Description | @@ -239,7 +239,9 @@ default off, so an unconfigured replay behaves exactly as before): | `--durable` | Enable the Tier-3 [durable runtime](../concepts/durable-runtime.md): checkpoint each verified step, durably pause on halt, resumable via `resume` | | `--allow-model-grounding` | **Model-egress opt-in** (PHI audit REM-3): permit wiring an off-box model grounder / identity-VLM / state-verifier; screenshots may leave the box. Off by default: replay makes no model-service calls; target and effect-verifier traffic stays deployment-defined. | -Exits 0 on success, 1 on a halt. With no model component wired, replay makes no +Exits 0 on success, 1 on a halt (every terminal outcome and halt reason is +defined in [Run outcomes and halt reasons](run-outcomes.md)). With no model +component wired, replay makes no model-service calls; target and effect-verifier traffic follows the deployment config. The on-prem VLM appliance engages only when `--allow-model-grounding` is passed **and** [`OPENADAPT_FLOW_VLM_URL`](configuration.md) is set. @@ -389,7 +391,9 @@ openadapt flow certify bundle --config deployment.yaml | `--config YAML` | [Deployment config](deployment-config.md) to read the policy from when `--policy` is omitted, so one file both certifies and runs the bundle | Provide `--policy` or a `--config` that sets `policy.policy`; certify errors if -neither supplies a policy. Exits 2 when the bundle fails certification. +neither supplies a policy. Exits 2 when the bundle fails certification — the +gate refusing an unsafe bundle, not an error in your setup +([exit codes](run-outcomes.md#cli-exit-codes)). ## seal diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index 4419f6b..488ffd7 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -21,3 +21,87 @@ source code. It does not grant unrestricted access to OpenAdapt Cloud. See [Deployment boundaries](../commercial/deployment-boundaries.md) and [Integrate OpenAdapt Execute](../commercial/execute-api.md). + +## Capability ladder + +The ordered set of resolution strategies the replayer tries when re-finding a +step's target, from the cheapest deterministic rung (structural DOM/UIA/AX +identity, then template crop, OCR label, geometry landmarks) up to an optional +grounding model. The healthy path resolves on a deterministic rung with zero +model calls; a rung change is recorded in the run report. See +[The capability ladder](../concepts/capability-ladder.md). + +## Certification + +The **enforced gate** over a bundle: `openadapt flow certify` evaluates a +[policy](#policy) against the bundle and exits nonzero when it fails, so CI +and deploy gates can refuse an unsafe bundle. Certification makes "runnable" +distinct from "certified safe"; sealing a bundle expires certification +inherited from its source. Contrast with [qualification](#qualification), +which is evidence that the workflow behaves correctly on its target. See +[Policy and certify](../concepts/policy-and-certify.md). + +## Effect contract + +A step's typed declaration of the **business effect** its write must produce +in the system of record (for example: this record exists with this value). +Effect contracts are checked by an independent read — API, database, or +document hash, not the pixels — and a non-confirmed verdict halts the run. +Reports record them as one-way SHA-256 digests. See +[Effect verification](../concepts/effect-verification.md). + +## Halt + +The runtime's fail-closed refusal to act: when identity, a postcondition, an +effect verdict, a policy gate, or an unhandled screen state does not match the +compiled expectation, the run stops and records what it observed instead of +guessing. A halt is a governed outcome, not a crash; it can be answered by an +operator (durable pause) or resolved permanently with `teach`. See +[Run outcomes and halt reasons](run-outcomes.md) and +[The halt-learn loop](../concepts/halt-learn-loop.md). + +## Identity gate + +The pre-click check on consequential steps that verifies the on-screen record +identifier against the run's expected identity evidence before acting — the +wrong-record guard. A conflict or an unreadable identity band halts the run +rather than clicking into the wrong record. See +[The identity gate](../concepts/identity-gate.md). + +## Policy + +A reviewable YAML document (or built-in, such as `clinical-write`) stating +what a bundle must satisfy to be trusted: which risks need identity arming, +what postconditions and effect strength writes require, and what is refused +outright. A policy is enforced by [certification](#certification) at the gate +and by the run gate at execution. See +[Write and enforce a policy](../guides/policy-and-certification.md). + +## Profile + +A named runtime posture — `demo`, `standard`, or `regulated` — that selects +which requirements the run gate enforces (certification, identity coverage, +effect contracts and their minimum tier, encryption, durability) and how the +outcome may be described. Only Standard and Regulated runs can report +`VERIFIED`. See [Run outcomes](run-outcomes.md) and +[Fail-closed regulated execution](../concepts/regulated-execution.md). + +## Qualification + +Structured **evidence** that one workflow behaves correctly on its target +application and surface: representative cases plus deterministic fault cases +that must halt, executed and recorded (`openadapt flow qualify`). A +qualification belongs to the exact workflow, application, and environment; it +is the artifact a pilot or deployment reviews. Contrast with +[certification](#certification), the policy gate. See +[Qualify a workflow](../guides/qualify-a-workflow.md). + +## Substrate + +The surface a workflow is recorded and executed on: web (browser DOM), native +Windows (UIA), macOS (AX), Linux (AT-SPI), or a remote display such as RDP or +Citrix (pixels). The substrate determines what structural evidence exists for +the capability ladder and the identity gate; bundles are surface-bound and a +cross-surface run is never silent. See +[The substrate model](../concepts/substrate-model.md) and +[backends](../concepts/backends.md). diff --git a/docs/reference/run-outcomes.md b/docs/reference/run-outcomes.md new file mode 100644 index 0000000..d2ecf29 --- /dev/null +++ b/docs/reference/run-outcomes.md @@ -0,0 +1,122 @@ +# Run outcomes and halt reasons + +Every run terminates in exactly one **execution outcome**, refined by a +**transaction outcome** that states what is known about the business effect. +This page is the single reference for every terminal state a run report or the +hosted Execute API can show, what each one means, and what to do next. + +All values below are read from the engine's typed report models +(`execution_outcome`, `transaction_outcome`, and the effect-verification +evidence in `report.json`) and, for the hosted lifecycle, from the published +Execute status contract. + +## Execution outcomes + +The coarse, evidence-qualified result in `report.json` +(`execution_outcome`) and on the final console line. + +| Outcome | Meaning | What to do next | +|---|---|---| +| `VERIFIED` | Execution completed **and** every declared contract passed: governed authorization, identity coverage, postconditions, and every effect confirmed at or above the required tier. Only possible under the Standard or Regulated profile. The only production success. | Nothing. Archive the run directory; the report and receipt are the audit evidence. | +| `COMPLETED_UNVERIFIED` | Execution reached the end, but the run lacked the evidence to claim `VERIFIED` — typically a Demo-profile replay with no independent effect verifier. Never billable, never a production success. | Fine for development. For production, wire an effect verifier and run under the Standard or Regulated profile so the same workflow can terminate `VERIFIED`. See [Run a deployment](../guides/run-a-deployment.md). | +| `HALTED` | The runtime **refused to act** on a governed check: identity, postcondition, effect verdict, an unhandled state, or a policy gate. A halt is fail-closed behavior, not a crash. | Read the halt reason in `REPORT.md` (categories below). Usually: [`teach`](cli.md#teach) the correction, or fix the target/parameters and re-run. Do not blind-retry a run whose transaction outcome is `RECONCILIATION_REQUIRED`. | +| `FAILED` | A non-governed runtime failure (for example the browser or agent connection died) rather than a safety refusal. | Check the environment (target reachable, backend agent up, permissions), then re-run. If the transaction outcome is `FAILED_PLATFORM`, no business effect occurred. | +| `ROLLED_BACK` | A detected duplicate or collateral write was **compensated** and the compensation was re-verified. Non-success, but the system of record was restored. | Review the compensation entries in the effect journal, confirm the record state, then address the root cause before re-running. | + +## Transaction outcomes + +`transaction_outcome` refines the execution outcome: it states what the +evidence proves about the **business effect** when a run stops. + +| Outcome | Meaning | What to do next | +|---|---|---| +| `VERIFIED` | Every declared effect passed at or above the required tier under a production profile. | Nothing; this is the billable success. | +| `HALTED_BEFORE_EFFECT` | The run stopped and the evidence proves **no consequential write landed**: every consequential step was verified absent or stopped before delivery was attempted. | Safe to fix and re-run. This is the honest version of "nothing happened". | +| `RECONCILIATION_REQUIRED` | Delivery or persistence is **uncertain, conflicting, or unverifiable**. A write may have half-landed. | Do **not** re-run yet. Reconcile against the independent system of record first (find or rule out the record), then re-run. The per-step effect journal in the report shows which step is uncertain. | +| `FAILED_PLATFORM` | An OpenAdapt/platform failure before any possible business effect. Never billable. | Re-run after the platform issue is resolved; report persistent cases. | +| `CANCELED` | The run was canceled before any business effect could occur. | Re-run when ready. | +| `REJECTED_POLICY` | Authorization, identity coverage, qualification, or environment gates refused execution **before** any business effect. | Fix the gate: certify the bundle, supply the required authorization, or adjust the [deployment configuration](deployment-config.md). | +| `COMPLETED_UNVERIFIED` | Demo-only completion with no production-grade effect evidence. | Same as the execution outcome above: acceptable in development only. | +| `ROLLED_BACK` | A duplicate/collateral write was compensated and re-verified. | Review the compensation evidence, then address the root cause. | + +## Effect-verifier verdicts + +Each declared effect on a consequential step gets an independent read of the +system of record, recorded per effect in the report's evidence: + +| Verdict | Meaning | What to do next | +|---|---|---| +| `confirmed` | The verifier independently observed the intended effect in the system of record. | Nothing; this is what `VERIFIED` is built from. | +| `refuted` | The verifier affirmatively observed the effect is **absent** (or wrong). The run halts. | The write did not land as intended. If the observed effect is `absent`, the run maps to `HALTED_BEFORE_EFFECT` and is safe to re-run after fixing the cause; if `conflicting`, reconcile the duplicate/wrong record first. | +| `indeterminate` | The verifier could not establish presence or absence (unreachable, ambiguous read). The run halts — an unreachable verifier is **never** treated as success. | Check verifier connectivity and configuration (`--effects-kind`, `--effects-base-url`). Treat the write as uncertain: reconcile before re-running. | + +The evidence also records what the verifier **observed** about the record, +independent of the verdict: `present`, `absent`, `conflicting` (a record was +written but is a duplicate or wrong value), or `unknown` (fail-safe default; +absence cannot be claimed). + +## Halt reasons and remediation + +When a run reports `HALTED`, `REPORT.md` names the violated expectation and +`report.json` carries a typed refusal code. The categories: + +| Halt reason (code) | Stage | What happened | Remediation | +|---|---|---|---| +| `target_ambiguous` | target resolution | More than one (or zero) candidate matched the step's recorded evidence; acting would be a guess. | Re-record the step with cleaner evidence, or [`teach`](cli.md#teach) the disambiguation. Prefer a structural backend (DOM / UIA / AX / AT-SPI) over pixels where the app exposes one. | +| `identity_conflict` | identity verification | The pre-click identity check read a **different record identifier** than the run's parameters expect — the wrong-record gate firing. | First verify the parameters or worklist row are correct. If the UI legitimately changed, `teach` the correction. Do not lower the gate: see [Troubleshooting](../guides/troubleshooting.md#it-halts-too-much-over-halting-on-citrix-pixel-only). | +| `identity_unverifiable` | identity verification | The identity band could not be read confidently (common on pixel-only substrates where OCR meets confusable glyphs). | Raise substrate fidelity (structural backend), re-capture the identifier crop, or `teach` the specific case. Over-halting beats a silent wrong write. | +| `actuation_observation_changed` | actuation revalidation | The screen changed between resolving the target and acting on it; the runtime refused to click a stale observation. | Usually transient (a late toast or dialog): re-run. If it recurs at the same step, `teach` the interstitial state so the program handles it. | +| `api_path_unavailable` | API admission | A step bound to the API actuation tier could not use it. | Check `--api-base-url` / the deployment config's actuation section and the credentials it references. | +| `effect_strength_insufficient` | effect strength | The configured verifier cannot meet the minimum verification tier the policy or profile requires for this write. | Wire a stronger verifier in the [deployment configuration](deployment-config.md), or revisit the required tier in the policy — a deliberate governance decision, not a tweak. | +| `effect_verifier_missing` | effect verification | The step declares effects but no verifier is configured; the profile refuses to treat the write as verified. | Configure `--effects-kind` (`rest`, `fhir`, `document-hash`) or, in Demo development only, explicitly approve the unverified write. | + +Beyond the typed refusal codes, a run also halts on an **unhandled state**: a +resolution failure, a dead-end branch, an unmet `halt` guard, a non-confirmed +effect, or an explicit `halt` terminal in the program. The report records where +it stopped, what unexpected on-screen state it observed, and the steps that +succeeded before it — exactly the evidence [`teach`](cli.md#teach) consumes. +See [The halt-learn loop](../concepts/halt-learn-loop.md). + +`report.json` also carries a machine-readable `failure_category` per failed +step: `governed_refusal` and `safety_halt` (the gates above), `runtime_failure` +(a non-governed error), or `continuation_preempted` (a durable run superseded +this attempt). + +## Hosted Execute lifecycle + +The hosted [Execute API](../commercial/execute-api.md) reports a run through a +published status contract. Lifecycle states: + +| State | Meaning | What to do next | +|---|---|---| +| `queued` | Accepted, not yet running. | Poll or wait for the webhook. | +| `running` | Executing under its governed profile. | Wait. | +| `decision_required` | The run paused for an attended human decision (a durable halt an operator can answer). | Answer the decision from the runner-local portal or the hosted lane; the engine re-verifies live state before continuing. See [Attend a paused run](../concepts/halt-learn-loop.md). | +| `waiting_for_reconciliation` | A consequential write is uncertain; the run waits for reconciliation against the system of record. | Reconcile; do not re-submit the same request without its idempotency key. | +| `terminal` | The run ended; `terminal_outcome` carries the result. | Read the outcome and evidence receipt. | + +Terminal outcomes mirror the engine taxonomy in lowercase: `verified`, +`halted_before_effect`, `reconciliation_required`, `rejected_policy`, +`failed_platform`, and `rolled_back_verified`. + +## CLI exit codes + +| Command | Exit codes | +|---|---| +| [`replay`](cli.md#replay) | `0` success (`VERIFIED` under Standard/Regulated); `1` on a halt or failure. | +| [`run`](cli.md#run) | Same as `replay` once admitted; `2` when a fail-closed gate (certification, policy, profile requirement) refuses the run before it starts. | +| [`tutorial`](../get-started/index.md) | `0` when `VERIFIED`; `1` on any other outcome; `2` when the tutorial is refused before running. | +| [`lint`](cli.md#lint) | `0` clean (or advice only); `1` once a finding reaches `error` severity (`--strict`: also on warnings). | +| [`certify`](cli.md#certify) | `0` pass; `2` when the bundle fails certification — the gate refusing an unsafe bundle. | +| [`teach`](cli.md#teach) | `0` correction promoted; `1` governed refusal (nothing written); `2` unusable inputs. | + +A nonzero exit from `lint`, `certify`, or a halted `replay` is the safety +boundary doing its job — the fail-closed design refuses to guess. See the +per-reason remediation above before changing any gate. + +## Still stuck? + +- [Troubleshooting](../guides/troubleshooting.md) covers first-week symptoms. +- [Read and audit run reports](../guides/run-reports.md) explains the report. +- Ask on [Discord](https://discord.gg/yF527cQbDG) or open an issue on + [GitHub](https://github.com/OpenAdaptAI). diff --git a/mkdocs.yml b/mkdocs.yml index f604ca2..d7547cf 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -143,6 +143,7 @@ nav: - Operate: - Run a deployment: guides/run-a-deployment.md - Read and audit run reports: guides/run-reports.md + - Run outcomes and halt reasons: reference/run-outcomes.md - Workflow catalog and halt map: guides/workflow-catalog.md - Attend a paused run and teach a correction: concepts/halt-learn-loop.md - Governed self-healing: concepts/self-healing.md @@ -159,6 +160,8 @@ nav: - Fail-closed regulated execution: concepts/regulated-execution.md - Settings and policy governance: concepts/settings-governance.md - Security and data handling: guides/security-and-data-handling.md + - PHI handling in one page: commercial/phi-handling.md + - Vulnerability disclosure: commercial/vulnerability-disclosure.md - Commercial: - commercial/index.md - Qualification Sprint: commercial/qualification-sprint.md @@ -166,6 +169,7 @@ nav: - Qualification report outline: commercial/qualification-report-outline.md - Acceptance matrix template: commercial/acceptance-matrix.md - Security packet: commercial/security-packet.md + - Subprocessors and data retention: commercial/subprocessors.md - Deployment boundaries: commercial/deployment-boundaries.md - External Citrix zero-install brief: commercial/citrix-external-brief.md - OpenAdapt Execute private-pilot guide: commercial/oem-brief.md