FEAT: Save targets created through the API (part 8)

.pyrit_conf_example

@@ -161,6 +161,14 @@ allow_custom_initializers: false
 # as scenario_presets_source. Defaults to ~/.pyrit/scenario_suites.
 # scenario_suites_source: https://account.blob.core.windows.net/container/scenario_suites
 
+# Optional storage for saved targets, converters, scorers, and techniques, so they
+# come back when the backend restarts. Each kind has its own folder under the source.
+# This supports the same local directory and Azure Blob sources as
+# scenario_presets_source. If a saved entry can't be restored, the backend still
+# starts, lists the failure in its runtime status, and refuses to use that entry's
+# name until the file is fixed or deleted. Defaults to ~/.pyrit/registry_recipes.
+# registry_recipes_source: https://account.blob.core.windows.net/container/registry_recipes
+
 # Upload Folder for API-Created Targets
 # -------------------------------------
 # Folder that targets created through the REST API may upload local files from

doc/getting_started/pyrit_conf.md

@@ -194,6 +194,18 @@ Suites store ordered references to scenario presets and resolve each preset's cu
 
 A SAS query string may be included; otherwise, PyRIT uses `DefaultAzureCredential`. The default is `~/.pyrit/scenario_suites`. Local writes use the same hidden lock-file and atomic replacement behavior as scenario presets. Startup scans the configured source so malformed suite documents are reported immediately without preventing the backend from starting.
 
+### `registry_recipes_source`
+
+Stores saved targets, converters, scorers, and techniques in a local directory or Azure Blob container, so they come back when the backend restarts. Targets created through `POST /api/targets` with `"persist": true` are saved here. An Azure URI may include a blob-name prefix:
+
+```yaml
+registry_recipes_source: https://account.blob.core.windows.net/pyrit-storage/registry-recipes
+```
+
+Each kind has its own folder under the source (`targets/`, `converters/`, `scorers/`, `techniques/`), with one JSON file per saved entry named after the entry. A file holds what's needed to rebuild the entry, and never a credential: restore refuses a file that stores a value for a parameter its class marks as a credential, such as `api_key`. A saved target also can't use a key from this server: it signs in with Microsoft Entra ID (`"auth_mode": "identity"`), which uses this server's Azure identity for everyone using the app, or needs no key. Targets from the server's configuration that it uses, such as a round-robin's members, keep their own sign-in. So restore refuses a target file that would read an empty key from an environment variable or leave an address to this server's settings. The addresses each target type declares (such as `endpoint`, `container_url` for Blob storage, or an A2A agent card path) can't hold a user name, password, SAS token, fragment, path parameters (`;`, `&`, `=`), or a query other than `api-version`, and a path can't have a query at all. An address the server would otherwise fill in must be a full http(s) or ws(s) address. In identity mode, an OpenAI target file sets `"headers": "{}"` so that the server's `OPENAI_ADDITIONAL_REQUEST_HEADERS` aren't added. The OpenAI SDK adds the headers in `OPENAI_CUSTOM_HEADERS` to every request, so while the server sets it, OpenAI targets can't be saved and saved ones aren't restored. The SDK's `OPENAI_ORG_ID` and `OPENAI_PROJECT_ID` still apply to every OpenAI target on this server. GitHub Copilot, Hugging Face, HackAPrompt, LiteLLM, HTTP, and Copilot WebSocket targets can't be saved yet. A SAS query string may be included in the source; otherwise, PyRIT uses `DefaultAzureCredential`. The default is `~/.pyrit/registry_recipes`.
+
+The backend restores saved entries at startup and after a live apply, once the configured initializers have registered theirs. If a saved entry can't be restored, the backend still becomes ready and lists each failure with its reason in `GET /api/config/runtime`. The failed entry keeps its name: looking it up, or starting a run that uses it, fails with the reason (409) instead of finding nothing or something else, and nothing else can take the name. While a saved technique is broken, a launch has to name its techniques, because choosing them by `all`, `default`, or a tag could quietly leave it out. If a whole folder can't be listed, nothing is restored and every name of that kind that isn't registered is held the same way. Fix or delete the listed files, then apply or restart. Files whose names aren't valid entry names (up to 64 lowercase letters, digits, and underscores, starting with a letter) are listed as ignored.
+
 ### `initialization_scripts`
 
 Local paths to custom Python scripts containing `PyRITInitializer` subclasses. Paths can be absolute or relative to the current working directory.

frontend/e2e/fixtures/manual_send_backend.py

@@ -4,6 +4,7 @@
 """Offline backend for manual-send browser tests, with no real configuration or providers."""
 
 import os
+import tempfile
 from collections.abc import AsyncGenerator
 from contextlib import asynccontextmanager
 from unittest.mock import AsyncMock, patch
@@ -27,19 +28,22 @@ async def _initialize_offline_async(self: ConfigurationLoader, *, raise_on_initi
 
 @asynccontextmanager
 async def _offline_lifespan_async(application: FastAPI) -> AsyncGenerator[None, None]:
-    config = ConfigurationLoader(memory_db_type="in_memory", env_files=[], env_akv_ref=[])
-    with (
-        patch.dict(os.environ, {"PYRIT_AUTH_MODE": "local", "PYRIT_DEV_MODE": "true"}, clear=True),
-        patch.object(
-            ConfigurationFileService,
-            "_read_source_async",
-            new=AsyncMock(return_value="memory_db_type: in_memory\nenv_files: []\ninitializers: []\n"),
-        ),
-        patch.object(ConfigurationLoader, "load_with_overrides", return_value=config),
-        patch.object(ConfigurationLoader, "initialize_pyrit_async", new=_initialize_offline_async),
-    ):
-        async with lifespan(application):
-            yield
+    with tempfile.TemporaryDirectory() as recipes:
+        config = ConfigurationLoader(
+            memory_db_type="in_memory", env_files=[], env_akv_ref=[], registry_recipes_source=recipes
+        )
+        with (
+            patch.dict(os.environ, {"PYRIT_AUTH_MODE": "local", "PYRIT_DEV_MODE": "true"}, clear=True),
+            patch.object(
+                ConfigurationFileService,
+                "_read_source_async",
+                new=AsyncMock(return_value="memory_db_type: in_memory\nenv_files: []\ninitializers: []\n"),
+            ),
+            patch.object(ConfigurationLoader, "load_with_overrides", return_value=config),
+            patch.object(ConfigurationLoader, "initialize_pyrit_async", new=_initialize_offline_async),
+        ):
+            async with lifespan(application):
+                yield
 
 
 app.router.lifespan_context = _offline_lifespan_async

frontend/playwright.config.ts

@@ -1,3 +1,7 @@
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import path from "node:path";
+
 import { defineConfig, devices } from "@playwright/test";
 
 import type { UserPreferences } from "./src/types";
@@ -24,6 +28,28 @@ const VITE_SERVER_OPTIONS =
 const E2E_BACKEND_PORT = process.env.PYRIT_E2E_BACKEND_PORT ?? "18000";
 const E2E_BACKEND_URL = `http://127.0.0.1:${E2E_BACKEND_PORT}`;
 
+/**
+ * Copies the seeded backend's configuration into a fresh folder that also holds
+ * its saved targets, converters, scorers, and techniques, so a run never reads
+ * or writes the ones saved on the machine running it.
+ */
+function createSeededBackendConfigFile(): string {
+  const folder = mkdtempSync(path.join(tmpdir(), "pyrit-seeded-backend-"));
+  // The runner stops the backend before it exits, so nothing still uses the folder by then.
+  process.once("exit", () => rmSync(folder, { recursive: true, force: true }));
+  const configFile = path.join(folder, "browser_config.yaml");
+  const config = readFileSync(new URL("../tests/end_to_end/browser_config.yaml", import.meta.url), "utf-8");
+  const recipesSource = JSON.stringify(path.join(folder, "registry_recipes"));
+  writeFileSync(configFile, `${config}\nregistry_recipes_source: ${recipesSource}\n`);
+  return configFile;
+}
+
+// Workers load this file too, but only the main process starts the backend.
+const SEEDED_BACKEND_CONFIG_FILE =
+  SEEDED_MODE && process.env.TEST_WORKER_INDEX === undefined
+    ? createSeededBackendConfigFile()
+    : "";
+
 export default defineConfig({
   testDir: "./e2e",
   testIgnore: ["**/analytics.spec.ts", "**/scenario-hierarchy.spec.ts", "**/scenario-datasets.spec.ts", "**/scenario-techniques.spec.ts"],
@@ -113,7 +139,7 @@ export default defineConfig({
           command:
             `cd .. && uv run --no-sync python -m pyrit.backend.pyrit_backend ` +
             `--auth-mode local --host 127.0.0.1 --port ${E2E_BACKEND_PORT} --log-level warning ` +
-            "--config-file tests/end_to_end/browser_config.yaml",
+            `--config-file "${SEEDED_BACKEND_CONFIG_FILE}"`,
           env: { PYRIT_DEV_MODE: "true" },
           url: `${E2E_BACKEND_URL}/api/health`,
           reuseExistingServer: false,

pyrit/backend/README.md

@@ -54,6 +54,39 @@ its database collation.
 ### Targets
 - `GET /api/targets` - List available prompt targets
 - `GET /api/targets/{id}` - Get target details
+- `POST /api/targets` - Create a target
+
+A created target is Temporary: a restart or a configuration apply removes it. With `"persist": true` (and `register`
+left on), the target is saved under `registry_recipes_source` and restored at every start, and the signed-in user is
+recorded as the one who saved it. A saved target can't hold or use a key: it signs in with Microsoft Entra ID
+(`"auth_mode": "identity"`, through this server's Azure identity) or needs no key. An OpenAI target's empty `headers`
+is set to `"{}"` first, so a saved one doesn't send this server's `OPENAI_ADDITIONAL_REQUEST_HEADERS`.
+
+| A request with `persist` that... | Gets |
+|---|---|
+| names the target with anything but up to 64 lowercase letters, digits, and underscores, starting with a letter | **422** |
+| would save a key, or use one of this server's keys or address settings | **400** |
+| gives an address that can carry a credential, or, where this server's setting would apply, one that isn't a full http(s) or ws(s) address | **400** |
+| saves a target type that can't be saved, or one that only a scan's launch scripts added | **400** |
+| saves an OpenAI target while this server sets `OPENAI_CUSTOM_HEADERS`, whose headers the OpenAI SDK adds to every request | **400** |
+| uses a Temporary target | **400** |
+| takes a name that a target or a saved file already has | **409** |
+| takes the name of a saved entry that couldn't be restored | **409** |
+| can't be written to storage | **503** |
+
+A request refused with **400** can be sent again without `persist`. When storage can't be written, the server log names
+the target and its folder, since the target comes back at the next start if the write reached storage anyway.
+
+Each listed target's `origin` is `saved`, `configuration` (registered by the server's startup scripts), or `temporary`
+(gone after a restart or a configuration apply, such as a target created without `persist` or registered by a scan's
+launch scripts). A saved target can only use targets that come back after a restart; targets from configuration keep
+their own sign-in.
+
+A saved target records its class and constructor parameters as the API accepted them, so it follows the class's
+constructor. A class that moves within PyRIT still restores. Renaming or removing a class or a constructor parameter
+changes the saved format: such a change raises `schema_version` and adds a migration for older files, with a test that
+restores one. A file that still doesn't match fails to restore with the reason, and its name stays held, so nothing
+else stands in for it.
 
 ### Manual attack creation
 
@@ -337,8 +370,9 @@ The chat preserves failed drafts across replacement, but requires converter choi
 to be reviewed again rather than restoring outputs tied to the previous registry.
 
 The offline browser fixture `frontend.e2e.fixtures.manual_send_backend:app` uses the real
-backend lifecycle with isolated in-memory SQLite, no environment files, and no default
-providers. The recovery tests register deterministic loopback targets. For example, start
+backend lifecycle with isolated in-memory SQLite, no environment files, no saved registry
+entries, and no default providers. The recovery tests register deterministic loopback
+targets. For example, start
 it with `uv run python -m uvicorn frontend.e2e.fixtures.manual_send_backend:app --host 127.0.0.1 --port 18213`
 (set `PYTHONUTF8=1` on Windows). In another shell, set `PYRIT_BACKEND_URL=http://127.0.0.1:18213`
 and `E2E_FRONTEND_PORT=31213`, then run `npx playwright test chat-recovery --project seeded --workers 1`

pyrit/backend/mappers/target_mappers.py

@@ -15,7 +15,7 @@
 from typing import Any
 
 from pyrit.models import TargetIdentifier
-from pyrit.models.catalog import TargetInstance
+from pyrit.models.catalog import RegistryEntryOrigin, TargetInstance
 from pyrit.prompt_target import CapabilityName, PromptTarget
 from pyrit.registry import TargetRegistry
 from pyrit.registry.registry import Reconstructable
@@ -55,7 +55,9 @@ def _target_specific_params(target_identifier: TargetIdentifier) -> dict[str, An
     return {**extra, **explicit_specific} or None
 
 
-def target_object_to_instance(target_registry_name: str, target_obj: PromptTarget) -> TargetInstance:
+def target_object_to_instance(
+    target_registry_name: str, target_obj: PromptTarget, *, origin: RegistryEntryOrigin | None = None
+) -> TargetInstance:
     """
     Build a TargetInstance DTO from a registry target object.
 
@@ -66,6 +68,7 @@ def target_object_to_instance(target_registry_name: str, target_obj: PromptTarge
     Args:
         target_registry_name: The human-friendly target registry name.
         target_obj: The domain PromptTarget object from the registry.
+        origin: Where the registered target comes from, or None for an inner target.
 
     Returns:
         TargetInstance DTO with metadata derived from the object.
@@ -100,6 +103,7 @@ def target_object_to_instance(target_registry_name: str, target_obj: PromptTarge
         supports_history_replay=supports_history_replay,
         target_specific_params=_target_specific_params(target_identifier),
         inner_targets=_build_inner_targets(target_obj),
+        origin=origin,
     )
 
 

pyrit/backend/middleware/error_handlers.py

@@ -11,9 +11,12 @@
 from fastapi.exceptions import RequestValidationError
 from fastapi.responses import JSONResponse
 
+from pyrit.backend.middleware.auth import has_admin_access
 from pyrit.backend.middleware.request_size import RequestTooLargeError, request_too_large_response
 from pyrit.backend.models import FieldError, ProblemDetail
 from pyrit.exceptions.analytics_exception import AnalyticsException
+from pyrit.registry import SavedEntryUnavailableError, TechniqueSelectionUnavailableError
+from pyrit.registry.file_document_storage import DocumentStorageUnavailableError, DocumentUnreadableError
 
 logger = logging.getLogger(__name__)
 
@@ -169,6 +172,68 @@ async def permission_error_handler(  # pyrit-async-suffix-exempt
             content=problem.model_dump(exclude_none=True),
         )
 
+    @app.exception_handler(DocumentStorageUnavailableError)
+    async def storage_unavailable_handler(  # pyrit-async-suffix-exempt
+        request: Request,
+        exc: DocumentStorageUnavailableError,
+    ) -> JSONResponse:
+        """
+        Handle stored documents that can't be read or written as 503 Service Unavailable.
+
+        The log names the storage location, without credentials, and the reason, with the
+        traceback at debug level. The response never carries a path or URL.
+
+        Returns:
+            JSONResponse: RFC 7807 problem detail response with 503 status.
+        """
+        logger.warning(f"{exc} ({request.method} {request.url.path})")
+        logger.debug("Storage error details", exc_info=exc)
+        if isinstance(exc, DocumentUnreadableError):
+            detail = f"{exc.label} '{exc.name}' can't be read: {exc.reason}"
+        else:
+            detail = f"{exc.label} storage is unavailable"
+        problem = ProblemDetail(
+            type="/errors/storage-unavailable",
+            title="Service Unavailable",
+            status=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail=detail,
+            instance=str(request.url.path),
+        )
+
+        return JSONResponse(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            content=problem.model_dump(exclude_none=True),
+        )
+
+    @app.exception_handler(SavedEntryUnavailableError)
+    async def saved_entry_unavailable_handler(  # pyrit-async-suffix-exempt
+        request: Request,
+        exc: SavedEntryUnavailableError,
+    ) -> JSONResponse:
+        """
+        Handle a name held for a saved entry that couldn't be restored as 409 Conflict.
+
+        Administrators get the full message; anyone else gets a fixed explanation of the reason,
+        since the message can quote a server setting.
+
+        Returns:
+            JSONResponse: RFC 7807 problem detail response with 409 status.
+        """
+        return _conflict(request=request, detail=exc.describe(with_details=_shows_details(request)))
+
+    @app.exception_handler(TechniqueSelectionUnavailableError)
+    async def technique_selection_unavailable_handler(  # pyrit-async-suffix-exempt
+        request: Request,
+        exc: TechniqueSelectionUnavailableError,
+    ) -> JSONResponse:
+        """
+        Handle a broad technique selection made while a saved technique couldn't be restored as 409 Conflict.
+
+        Returns:
+            JSONResponse: RFC 7807 problem detail response with 409 status.
+        """
+        return _conflict(request=request, detail=str(exc))
+
     @app.exception_handler(NotImplementedError)
     async def not_implemented_handler(  # pyrit-async-suffix-exempt
         request: Request,
@@ -223,3 +288,33 @@ async def generic_exception_handler(  # pyrit-async-suffix-exempt
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             content=problem.model_dump(exclude_none=True),
         )
+
+
+def _conflict(*, request: Request, detail: str) -> JSONResponse:
+    """
+    Build a 409 Conflict problem response.
+
+    Returns:
+        JSONResponse: RFC 7807 problem detail response with 409 status.
+    """
+    problem = ProblemDetail(
+        type="/errors/conflict",
+        title="Conflict",
+        status=status.HTTP_409_CONFLICT,
+        detail=detail,
+        instance=str(request.url.path),
+    )
+    return JSONResponse(status_code=status.HTTP_409_CONFLICT, content=problem.model_dump(exclude_none=True))
+
+
+def _shows_details(request: Request) -> bool:
+    """
+    Return whether the caller may see a restore failure's full message, which can quote a server setting.
+
+    Returns:
+        bool: True for an administrator; False otherwise, including when no authentication policy is set.
+    """
+    try:
+        return has_admin_access(request)
+    except RuntimeError:
+        return False