Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,10 @@ Every public module has a committed API report at `packages/<package>/api-report

PRs are squash-merged with the PR title as the commit message, and release-please builds each package's changelog from those messages.

- **A change to an experimental feature uses that feature's scope:** `skills` (Agent Skills) and `evaluations` (Evaluations from Code). `feat`, `fix` and `perf` commits with one of these scopes are listed under **Experimental** (`changelog-sections` in `release-please-config.json`), for example `feat(skills): watch skills for revocations`. Any other scope, such as `client`, lists the change as a core feature or fix.
- **Use exactly one scope.** `feat(skills,client):` matches no Experimental row, so it lands under Features.
- **A change to an experimental feature uses that feature's scope:** `evaluations` (Evaluations from Code). `feat`, `fix` and `perf` commits with that scope are listed under **Experimental** (`changelog-sections` in `release-please-config.json`), for example `feat(evaluations): score rows in parallel`. Any other scope, such as `client` or `skills`, lists the change as a core feature or fix. Agent Skills is core, so `feat(skills):` lands under Features.
- **Use exactly one scope.** `feat(evaluations,client):` matches no Experimental row, so it lands under Features.
- **A PR that changes both core and experimental behaviour** should be split. If it can't be, add a `BEGIN_COMMIT_OVERRIDE` block to its description with one conventional-commit line per change. An override also corrects a merged PR's title, as long as the release PR hasn't merged yet.
- **A breaking experimental change** uses `!` while the package is 0.x: `feat(skills)!:` bumps the minor version, which is what an experimental break is allowed. After 1.0, `!` would force a major release. Use `feat(skills):` without `!` instead, and say the change is breaking in the title.
- **A breaking experimental change** uses `!` while the package is 0.x: `feat(evaluations)!:` bumps the minor version, which is what an experimental break is allowed. After 1.0, `!` would force a major release. Use `feat(evaluations):` without `!` instead, and say the change is breaking in the title.
- **When a feature is promoted to core,** remove its rows from `changelog-sections`.

The rule is specified in `TESTING.md` §0.3 in `launchdarkly/ai-sdks-monorepo`.
Expand Down
16 changes: 13 additions & 3 deletions packages/ai/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,16 +52,26 @@ if result["enabled"]:

Never raises. Returns `{"enabled": bool, "config": dict | None, "meta": dict | None}`.

## Agent Skills

Agent Skills (`get_skill`, `write_skills`, `watch_skills` and the rest) are re-exported from this
package's root. They need `launchdarkly-ai-server` 1.0.0 or later, the first release with Agent
Skills in core. This package does not pin `launchdarkly-ai-server`, so upgrading it alone can leave
an older one installed, and `from launchdarkly_ai_python import get_skill` then raises
`ImportError`. Upgrade both:

```bash
pip install --upgrade launchdarkly-ai-python "launchdarkly-ai-server>=1.0.0"
```

## Experimental features

Experimental features are not re-exported from this package. They live in
`launchdarkly-ai-server`, the package this one depends on, and need a `launchdarkly-ai-server`
release that includes them. Import them from `launchdarkly_ai_server.experimental`; if that
import fails, upgrade `launchdarkly-ai-server` itself, since upgrading this package alone can
leave an older `launchdarkly-ai-server` in place. The experimental features are:
leave an older `launchdarkly-ai-server` in place. The experimental feature is:
Comment thread
jeffdupont marked this conversation as resolved.
Comment thread
jeffdupont marked this conversation as resolved.

- **Agent Skills**, in `launchdarkly_ai_server.experimental.skills`; see the
[Agent Skills guide](https://github.com/launchdarkly/python-ai-sdk/blob/main/packages/client/README.md#agent-skills-experimental).
- **Evaluations from Code**, in `launchdarkly_ai_server.experimental.evaluations`; see the
[evaluations guide](https://github.com/launchdarkly/python-ai-sdk/blob/main/packages/client/README.md#run-an-evaluation-from-code-experimental).

Expand Down
23 changes: 8 additions & 15 deletions packages/client/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -471,11 +471,7 @@ asyncio.run(main())

---

### Agent Skills (experimental)

> **Experimental.** Import Agent Skills from `launchdarkly_ai_server.experimental.skills`;
> none of these names is exported from the package root. They may change in a minor release,
> so review the changelog when you upgrade.
### Agent Skills

Skills are versioned `SKILL.md` documents managed in LaunchDarkly and attached to AI Config
variations by reference. The SDK tells you which skills a config references, retrieves their
Expand All @@ -487,9 +483,9 @@ import asyncio
import hashlib
from pathlib import Path

from launchdarkly_ai_server import init_client, inspect_config
from launchdarkly_ai_server.experimental.skills import (
InMemorySkillStore, get_skill, set_skill_store, skill_refs, write_skills,
from launchdarkly_ai_server import (
InMemorySkillStore, get_skill, init_client, inspect_config, set_skill_store,
skill_refs, write_skills,
)

SKILL_MD = "---\nname: PDF Extraction\n---\nExtract text from PDFs.\n"
Expand Down Expand Up @@ -623,9 +619,8 @@ ERROR ld.skills.integrity_failure {"action":"withheld","event":"ld.skills.integr
```

**Match on `ld.skills.integrity_failure`.** It is the name LaunchDarkly's AI SDKs share for this
record. While Agent Skills is experimental, the event name, its fields, and its `reason_code`
values may change in a minor release; any such change gets a changelog entry under
**Experimental**, so review it before upgrading. JSON keys are sorted, so the line is byte-identical across LaunchDarkly's
record. A change to the event name, its fields, or its `reason_code` values gets a changelog
entry, so review the changelog before upgrading. JSON keys are sorted, so the line is byte-identical across LaunchDarkly's
AI SDKs for the same input.

| Field | Description |
Expand Down Expand Up @@ -683,7 +678,7 @@ retrieval, verification, and telemetry as `get_skill`, but reports which of five
happened instead of collapsing them all to `None`.

```python
from launchdarkly_ai_server.experimental.skills import get_skill_result
from launchdarkly_ai_server import get_skill_result

outcome = await get_skill_result("pdf-extraction")

Expand Down Expand Up @@ -731,9 +726,7 @@ authenticated with the environment's server-side SDK key.
```python
import os

from launchdarkly_ai_server.experimental.skills import (
FDv2SkillStore, set_skill_store, watch_skills,
)
from launchdarkly_ai_server import FDv2SkillStore, set_skill_store, watch_skills

store = FDv2SkillStore(os.environ["LD_SDK_KEY"]).start()
if not store.wait_for_skills(timeout=10):
Expand Down
40 changes: 20 additions & 20 deletions packages/client/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,8 @@ from launchdarkly_ai_server import execute_and_track, execute_and_stream, wrap_t
from launchdarkly_ai_server import config, graph, resolve_graph
from launchdarkly_ai_server.experimental.evaluations import init_evaluations

# Agent Skills (experimental: none of these is exported from the package root)
from launchdarkly_ai_server.experimental.skills import (
# Agent Skills
from launchdarkly_ai_server import (
set_skill_store, SkillStore, InMemorySkillStore, FDv2SkillStore, StoreDiagnostics,
skill_refs, get_skill, get_skill_result, get_skills, all_skills, write_skills,
watch_skills, SkillWatcher,
Expand All @@ -90,12 +90,12 @@ from launchdarkly_ai_server.experimental.skills import (
)
```

An experimental feature is exported only from its module under
`launchdarkly_ai_server.experimental`, and its names may change in a minor release. No core
public type, function or `init_client` option may name it. Core reaches it only through an
internal hook (as `shutdown` clears the skill store), and an error there is logged, never
raised into the core call. The experimental part is the SDK's API, not the LaunchDarkly data
model: `AiConfigRep` still documents the `skills` references a config carries.
An experimental feature (today only Evaluations from Code) is exported only from its module
under `launchdarkly_ai_server.experimental`, and its names may change in a minor release. No
core public type, function or `init_client` option may name it. Agent Skills is core and is
exported from the package root; `shutdown` still reaches its state only through an internal
hook (it clears the skill store), and an error there is logged, never raised into
`shutdown`.

`MAX_SKILL_CONTENT_BYTES` and `SKILL_OBJECT_KIND` are deliberately **not** exported; both stay
internal to `skills_core`:
Expand All @@ -106,7 +106,7 @@ internal to `skills_core`:
would imply otherwise. An adapter that needs it imports it from
`launchdarkly_ai_server.skills_core`.

When adding a new core export, add it to `__init__.py`'s imports and `__all__`. An Agent Skills export goes in `experimental/skills.py` instead, and in `EXPERIMENTAL_SKILLS_SURFACE` in `tests/test_skills.py`. Handler packages must never import from sub-paths (e.g. `launchdarkly_ai_server.client`).
When adding a new core export, add it to `__init__.py`'s imports and `__all__`, and to `ROOT_SURFACE` in `tests/test_public_surface.py`. An Agent Skills export also goes in `SKILLS_SURFACE` in `tests/test_skills.py`. Handler packages must never import from sub-paths (e.g. `launchdarkly_ai_server.client`).

---

Expand Down Expand Up @@ -212,7 +212,7 @@ Three layers, in increasing order of blast radius:
(including `null`, or one bad entry) raises `ValueError` rather than returning a partial
list that would authorize a prune. So does a config that is not a dict, including the
`None` a failed `inspect_config` returns; only a dict with no `skills` key yields `[]`. `parse_ai_config` deliberately does not check `skills`,
so an experimental field cannot fail a core config call (TESTING.md §0.3).
so a malformed field cannot fail a config call that does not use skills.
2. **Content accessors** — `get_skill`, `get_skill_result`, `get_skills`, `all_skills` read
through the `SkillStore` seam. Configure a store with `set_skill_store(store)`; with
none configured the accessors raise
Expand Down Expand Up @@ -318,12 +318,12 @@ inside its known-event branches. Empty data on a known event is not JSON either,
connection the same way: read as `None`, a `delete-object` would be ignored and a
`payload-transferred` would commit with no selector.

**A put or delete that is valid JSON but has no usable `key` is warned and ignored, for the
experimental stage.** Its transfer still commits, so a keyless `delete-object` loses its
**A put or delete that is valid JSON but has no usable `key` is warned and ignored, and
delivery keeps going.** This is the accepted behaviour, not a provisional one. Its transfer still commits, so a keyless `delete-object` loses its
Comment thread
jeffdupont marked this conversation as resolved.
revocation and a keyless `put-object` in an `xfer-full` revokes that skill by omission (and
`write_skills("*")` prunes it). `ldclient` interrupts the stream instead. Interrupting is not
strictly safer, because a server that keeps resending the object would then block every later
change, so the choice is to be revisited before 1.0 (TESTING.md §3.25). Pinned by
change. Pinned by
`test_a_delete_with_no_usable_key_is_ignored_and_its_transfer_commits` and
`test_a_put_with_no_usable_key_in_an_xfer_full_revokes_by_omission`; changing the rule should fail
them.
Expand Down Expand Up @@ -625,8 +625,9 @@ Do not undo any of these as a simplification:
store), and the stdlib's default formatter drops `extra`, so an `extra`-only record is
invisible under a plain `logging.basicConfig()`.
- **`ld.skills.integrity_failure` is documented for customers to match on.** Do not rename it
casually. The experimental stage allows a rename in a minor release, but only in every SDK
at once and with a changelog entry, because a rename silently breaks customers' alerts.
casually. Agent Skills is core, so a rename is a breaking change: it needs a `!` commit (a
minor bump while on 0.x, a major after 1.0) in every SDK at once and a changelog entry,
because a rename silently breaks customers' alerts.
- **`sort_keys=True` makes the line byte-identical across SDKs** (modulo `language`), since
the other implementations build the object in alphabetical key order.
- **Optional fields are omitted, never nulled**, so a SIEM field-existence check means
Expand Down Expand Up @@ -689,10 +690,9 @@ the allowlist is enforced in one place.
`_clear_state` delegate to `skills_core`. `set_skill_store` and tests use those; none
should reach into `skills_core` directly.

**Core never imports this feature.** `skills_core` registers `clear_state` with
`shutdown_hooks.register_shutdown_hook` when it loads, and `shutdown()` runs the registered
hooks. `lifecycle.py` imports no skills module (TESTING.md §0.3, pinned by
`test_core_does_not_import_experimental_code`).
**Lifecycle never imports this feature.** `skills_core` registers `clear_state` with
`shutdown_hooks.register_shutdown_hook` when it loads (the package root imports it), and
`shutdown()` runs the registered hooks. `lifecycle.py` imports no skills module.

### Descriptor-pinned filesystem access

Expand Down Expand Up @@ -984,6 +984,6 @@ a conversation is out of reach at this layer either way.
- `Skill.content` is opaque `bytes`. Do not add anything that parses or interprets it — no YAML library in this package's dependencies at any tier, and no accessor that decodes content.
- Do not route skills telemetry through `client.track()`, and do not introduce an LD context anywhere in the skills path. Signals go through the `skills_core.py` emitter seam, whose default is a no-op, and only via its `record_*` functions.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rule, and the Telemetry seam section at lines 586–590 ("Do not use client.track()… No LD context is involved anywhere in this feature"), now contradict the report_usage calls added in 4d6b3cb, which send $ld:ai:sdk:usage through client.track() with SDK_INFO_CONTEXT. The code follows §3.27, so I think the docs should exempt it explicitly. For example: the rule covers the three skills signals; the SDK-wide helper usage event (§3.27, once per helper per client) is the one client.track() path, and it carries no skill data. Otherwise a future contributor following this line could remove the usage reporting as a violation.

- Do not add a signal name outside the three in the Agent Skills table above — the list is an allowlist. `AgentControl Skill SDK Reference Returned` and `AgentControl Skill Content Retrieved` were considered and deliberately excluded from SDK emission.
- Do not rename `ld.skills.integrity_failure` in one language only or without a changelog entry, and do not add an eleventh `reason_code` in one language only — both are documented compatibility surfaces, though the experimental stage lets either change in a minor release. See "The integrity-failure log record" above.
- Do not rename `ld.skills.integrity_failure` in one language only or without a changelog entry, and do not add an eleventh `reason_code` in one language only — both are documented compatibility surfaces. See "The integrity-failure log record" above.
- Do not relax any of the `write_skills` filesystem defenses (local key re-validation, symlink refusal, manifest-authorized destruction, corrupt-manifest fail-closed, atomic `0644` writes). Each is a deliberate security property with abuse-case tests attached.
- Do not make `SkillStore` lookups key-only. Version is part of the lookup identity because a payload holds several versions of one key; a key-only seam cannot express a version-pinned reference.

This file was deleted.

Loading
Loading