Skip to content

feat(monitoring): add business_diagnostic v0.1 — scenario ledger and calibration (#338) - #344

Draft
mrmasa88 wants to merge 3 commits into
ARPAHLS:mainfrom
0x-AO-Protocol:feat/issue-338-business_diagnostic
Draft

feat(monitoring): add business_diagnostic v0.1 — scenario ledger and calibration (#338)#344
mrmasa88 wants to merge 3 commits into
ARPAHLS:mainfrom
0x-AO-Protocol:feat/issue-338-business_diagnostic

Conversation

@mrmasa88

@mrmasa88 mrmasa88 commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Closes #338

Scope

v0.1 only — adjudicate and calibrate. Stateless, stdlib-only, offline; schemas shipped as documentation with explicit runtime validation; fail-closed contract errors. Projections against declared targets and a report skeleton are out of scope here and will come as separate Skill Upgrade issues after this merges.

Bundle (skills/monitoring/business_diagnostic/)

  • manifest.yaml, skill.py, instructions.md, card.json, __init__.py
  • schemas/ — framework, marks, observations, outcome, output (documentation; runtime uses explicit checks)
  • kb/demo_scenarios.json — synthetic framework, timestamped and sourced
  • fixtures/ — E2E adjudicate/calibrate, honest state, one fixture per error code
  • test_skill.py — 36 offline tests: loader, manifest identity, E2E equality against fixtures, every error code from its fixture (registry set == fixture set), non-exhaustive update, negative observations, re-mark boundary, honest state, Brier trend ordering, identity without indicators, bit-identical repeat, no clock in source

Modeled on monitoring/kpi_gate (#318).

Repo ripples

docs/skills/business_diagnostic.md (catalog page) · docs/skills/README.md row · docs/usage/agent_loops.md · docs/usage/install_extras.md + pyproject.toml extras (via scripts/sync_extras.py) · tests/fixtures/card_ui_schema/monitoring__business_diagnostic.json · examples/business_diagnostic_demo.py (+ README, smoke test) · host-simulation stress discovery assert · CHANGELOG.md

Deviations from the issue text (small, deliberate)

  1. as_of is a required input — the skill never reads the clock, so identical input returns identical output.
  2. Contract violations use the same shape as kpi_gate (status: error + error{code, detail}); the contract_errors: [] array in the issue's example is not used.
  3. A duplicate observation for the same indicator is a contract error (hosts de-duplicate); observations dated on/before marked_on or after adjudication_date are rejected — the ledger cannot double-count.
  4. The error registry has 14 codes: an INVALID_REQUEST envelope code was added so that UNKNOWN_ACTION means exactly that. Validation order is fixed and documented.
  5. Output carries three top-level booleans for when: gates in chains (fired_any, marks_stale, insufficient).

Tests

python3 -m pytest skills/monitoring/business_diagnostic tests/test_skill_issuer.py tests/test_registry_identity.py tests/test_card_ui_schema.py tests/test_registry_docs.py tests/test_extras_sync.py tests/test_examples_smoke.py -q → 91 passed
Full suite: 721 passed / 5 skipped. black --check and flake8 clean; scripts/sync_extras.py --check OK. Wheel smoke: run locally — 21 registry skills verified in the built wheel, monitoring/business_diagnostic loadable from a fresh base install, 4 unrelated skills deferred (optional extras not installed).


Type of Change

  • New Skill — new registry bundle under skills/
  • Skill Upgrade — changes to an existing skill under skills/
  • Bug Fix — incorrect runtime or framework behavior
  • Documentation — docs, README, CONTRIBUTING only
  • Framework Featureskillware/core/ loader, env, adapters
  • CLIskillware/cli.py, docs/usage/cli.md
  • Examplesexamples/*.py, agent loops, examples/README.md
  • Packaging — PyPI wheel, pyproject.toml, MANIFEST.in
  • RFC / meta — templates, labels, CI, or large design doc

Checklist (all PRs)

  • Linked GitHub issue (Fixes #… or Refs #…)
  • Scope matches the issue — no unrelated refactors
  • python -m black --check . and flake8 pass locally (or CI-equivalent subset)
  • pytest skills/ and pytest tests/ pass locally when relevant
  • CHANGELOG.md updated under [Unreleased] when user-visible behavior changes
  • examples/README.md updated if this PR adds, renames, or removes a runnable script
  • Ran pytest tests/test_registry_docs.py when skills, examples index, or agent-loops matrix changed

New or updated skill

Bundle and metadata

  • Skill at skills/<category>/<skill_name>/ (from templates/python_skill/ or equivalent)
  • manifest.yaml: name (full ID), version, description, parameters, constitution, real issuer
  • Optional: short_description, issuer.github, issuer.org, requirements, env_vars

Effect, Directive, Assurance

  • Deterministic skill.py (Effect; no ad-hoc LLM-generated execution paths)
  • instructions.md (Directive) explains when and how to use the skill
  • card.json (Presentation) issuer matches manifest when present
  • test_skill.py (Assurance) covers execution and schema expectations
  • SkillLoader.load_skill("<category>/<skill_name>") succeeds (or deps documented)

Documentation and catalog

  • docs/skills/<skill_name>.md and row in docs/skills/README.md
  • Usage Examples for Gemini, Claude, OpenAI, DeepSeek, Ollama per skill usage template

Constitution and safety (skills only)

Compute only: no data fetching, no network, no clock, no side effects. Operator marks are never overwritten; model-implied values are labeled as what the declared weights imply, not forecasts. Undeclared ids are contract errors; what cannot be computed honestly returns insufficient_data with a reason code.

Related Issues

Closes #338 · Refs #318 (kpi_gate, the template for this bundle)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[New Skill]: monitoring/business_diagnostic — deterministic scenario ledger and calibration (companion to kpi_gate)

1 participant