diff --git a/.DS_Store b/.DS_Store deleted file mode 100644 index 73c57fc..0000000 Binary files a/.DS_Store and /dev/null differ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 17260e0..e74140e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -72,3 +72,41 @@ jobs: with: subject-path: dist/* - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 + + # Rule 3: every tag gets a GitHub Release. A tag is a git object; a Release is what a + # human and a maintenance scorer can read, and cadence is a signal tags do not carry. + # Every mirror had every tag and zero Releases until 1.1.0, whose six were created by + # hand -- which is how the rule was being met, and therefore how it would stop being met. + github-release: + needs: publish + runs-on: ubuntu-latest + permissions: + contents: write # the Release is the only thing this workflow writes to the repository + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - name: Create the GitHub Release from the tag + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + version="${GITHUB_REF_NAME#v}" + if gh release view "$GITHUB_REF_NAME" >/dev/null 2>&1; then + echo "Release $GITHUB_REF_NAME already exists; nothing to do." + exit 0 + fi + # This version's entry, from its heading to the next one. A tag whose version has + # no entry would publish a Release nobody can read, so that fails the job rather + # than shipping an empty one. + awk -v want="## [$version]" ' + index($0, want) == 1 { inside = 1; next } + inside && /^## \[/ { exit } + inside { print } + ' CHANGELOG.md > release-notes.md + if [ ! -s release-notes.md ]; then + echo "CHANGELOG.md carries no entry for $version" >&2 + exit 1 + fi + gh release create "$GITHUB_REF_NAME" \ + --title "Packvium $version" \ + --notes-file release-notes.md \ + --verify-tag diff --git a/CHANGELOG.md b/CHANGELOG.md index 648baf5..1831154 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,113 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). As of `1. public API, the request and result schemas, the numeric and units policy, the validation rules and the compatibility policy are frozen: breaking any of them costs a major version. +## [1.2.0] + +An additive release. A packing result can now be turned into a document an operator can +work from, in all four languages; and, in Python, a proposed catalog change can be +compared, replayed against history and published through an approval step rather than +argued about. + +Nothing here breaks 1.1.0. Every schema change is additive, the frozen Python surface is +unchanged, and the PHP surface grows one new namespace and nothing else. + +### Added + +- **Execution plans, in all four languages.** `packvium.execution`, + `Packvium\Execution\Plan`, `packvium_core::execution` and `@packvium/engine`'s + `execution.js` derive a work order from an already validated result: what to lift, why a + carton was chosen, and what was not packed with its proof level unsoftened. The adapter + calls no solver and no validator — a test in each language asserts that — and given the + same result all four emit a byte-identical canonical form. + + Two properties are part of the format rather than notes about it. Everything the solver + or validator decided sits under `facts`; every human-readable sentence sits under + `presentation` and names the fields it came from in `cites`, so a consumer that reads + only `facts` loses nothing it may rely on. And the step order is **injected**: supply + `loading_orders` and each step is numbered, omit it and the container reports + `order: "unavailable"` with every placement still listed. There is no third behaviour, + because presenting the order the solver happened to walk its candidates in as an order + that is safe to lift boxes in would be a claim nothing supports. + +- **Operator locks, in Python.** `packvium.locks` lets an operator pin a placement and get + a *second* result beside the approved one — never an edit of it, so "what was approved" + and "what was proposed" stay two artifacts. A lock becomes an ordinary placement + constraint, so the re-solve is the same portfolio under the same independent validator: + **a lock cannot produce a placement the engine would otherwise refuse**, and the + strongest thing it can do is reserve its own slot. A lock the solve cannot honour is + reported as unpreserved and names the lock rather than raising; a lock set that + contradicts itself is refused before any solve runs. + + Python-only, deliberately: a lock has no representation in the request schema, so there + is nothing to hand another engine. + +- **Scenario comparison, recommendations and historical replay, in Python.** + `packvium.simulation` compares two pinned scenarios order by order and returns a Pareto + report per order — never a blended delta, because a single number can name a winner that + is worse on the axis a caller actually cares about. `packvium.recommendations` turns that + comparison into a proposal, or returns nothing at all when the paired cohort is too + small, and publishes only through an explicit approval step. `packvium.holdout` replays a + proposal against decisions held out of the evidence that produced it, over the append-only + ledger in `packvium.outcomes`; a packing the injected validator rejects is that arm's + failure at any cost, and realised damage, returns and repacks are reported against the + carton that actually shipped rather than credited to the one that was never tried. + + This surface is supported and **not yet signature-frozen** — a parameter may be renamed + or a returned value gain a field in a minor release, announced here rather than blocked + by a gate. Pin the version if you depend on its exact shape; `docs/PUBLIC-API.md` says + which surfaces carry the stronger promise. + +- **`pack_from_dict` accepts a keyword-only `extensions` registry (Python).** Additive: it + defaults to `None` and every existing call keeps its meaning and its answer. It exists so + a lock can enter the engine as an ordinary request-derived constraint rather than through + a second, lock-aware search path. The registry **adds to** the compiled policy rules and + does not replace them. + +- **Two new worked examples.** `execution` (Python, PHP and Node) turns a result into + numbered steps and shows what "byte-identical" does and does not promise; `intelligence` + (Python) proves a carton change is worth publishing before publishing it. + +### Changed + +- **The solvers do less repeated work, again.** Byte-identical results everywhere. Group + batching in the Rust extreme-point solver drops two quadratic scans over group members + for a single pass, and the PHP and Python grid-admission and load-ordering paths reuse a + prototype profile and a canonical order instead of rebuilding them per candidate. + +- **Every example names its own time budget.** An example that solved on the library default + could print a different answer on a busy machine. Each now sets `time_limit_ms`, far above + what its solve needs; every printed answer is unchanged. + +### Fixed + +- **The JavaScript engine could return fewer placements than it reported packed.** When the + per-container beam search stopped early — on its node limit, the deadline or the effort + budget — it chose the surviving leader without the batches it had not reached yet, so those + items were neither placed nor listed as unpacked while the result still said `feasible`. It + needs `solver_profile: "quality"` or an explicit `container_plan_beam_width` above 1, and it + shipped in `@packvium/engine` `1.0.0` and `1.1.0`. The default profiles never reach that + path, and Python, PHP and Rust were never affected. If you run the JavaScript engine on the + `quality` profile, check `packed_item_count` against the placements you actually received + from those versions. + +- **The intelligence modules would not have imported on Python 3.9**, which is the minimum + the package declares. Six modules used `dataclass(slots=True)` from the standard library; + that argument does not exist before 3.10, so importing any of them raised `TypeError` on a + supported runtime. They now route through the package's own compatibility shim, as the + rest of the package already did. + +- **`@packvium/engine` did not expose its execution plan.** The module was named in the + package manifest but not in `exports`, so `import '@packvium/engine/execution.js'` failed + with `ERR_PACKAGE_PATH_NOT_EXPORTED` and the main entry re-exported neither function. The + subpath is now declared, with TypeScript declarations beside it. + +- **A non-finite metric is refused rather than ranked as optimal.** `NaN` makes both `>` and + `<` false, so an axis carrying one was silently counted equal and a candidate whose metrics + were all `NaN` arrived on the Pareto frontier beside a clean one. It is now refused by a + named error that says which candidate and which axis. Infinities are still accepted: they + are ends of the number line, and a caller encoding "unpriceable" as an infinite cost gets + the answer they mean. + ## [1.1.0] An additive release on the 1.0.0 freeze. Route-aware unloading becomes a property of the diff --git a/README.md b/README.md index d5e6b06..e41b4d5 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ dependencies**, exact integer geometry. Full documentation, the constraint reference and benchmarks live at [packvium.com](https://packvium.com). -> **Version 1.1.0 — the public API is frozen.** Field names, status codes and the +> **Version 1.2.0 — the public API is frozen.** Field names, status codes and the > objective vector do not change without a major version, so any `1.x` is a safe upgrade > from any earlier `1.x`. > Read [docs/GUARANTEES.md](https://github.com/toxakara/packvium-python/blob/main/docs/GUARANTEES.md) before relying on a result. @@ -66,6 +66,8 @@ in `constraints.py` have failed you. | [`shapes.py`](https://github.com/toxakara/packvium-python/blob/main/examples/shapes.py) | Items that are not their box: complementary wedges sharing one crate as `convex_hull`, and a cushion that compresses under load until the crush limit refuses it. | | [`nested.py`](https://github.com/toxakara/packvium-python/blob/main/examples/nested.py) | Units into cartons, cartons onto a pallet, in one call. | | [`commerce.py`](https://github.com/toxakara/packvium-python/blob/main/examples/commerce.py) | Rate a shipment, apply an eligibility rule, and pin a catalog version. | +| [`execution.py`](https://github.com/toxakara/packvium-python/blob/main/examples/execution.py) | Turn a result into dock instructions: solver facts kept apart from screen text, a step order that is injected or honestly absent, and an operator lock that yields a second plan rather than editing the approved one. | +| [`intelligence.py`](https://github.com/toxakara/packvium-python/blob/main/examples/intelligence.py) | Prove a carton change is worth publishing: two scenarios compared order by order, a proposal that refuses to exist on thin evidence, and a replay against held-out history where a cheaper packing your validator rejects still counts as a regression. | | [`extensions.py`](https://github.com/toxakara/packvium-python/blob/main/examples/extensions.py) | A rule the schema has no field for — and an honest account of what you give up by writing one. | ```bash @@ -88,6 +90,16 @@ PYTHONPATH=src python3 examples/objectives.py containers. - **Extensible.** Register your own constraints, item orderings, candidate scorers, container selectors or complete solvers. +- **Work orders, not just coordinates.** `packvium.execution` turns a validated result + into a plan: solver facts kept apart from the text that cites them, an injected loading + order or an honest `unavailable`, and a canonical form four engines emit byte for byte. + `packvium.locks` lets an operator pin a placement and get a *second* plan beside the + approved one — never an edit of it, and never a placement the engine would refuse. +- **Decide before you publish.** `packvium.simulation` and `packvium.recommendations` + compare two catalog scenarios order by order and propose a change only when the paired + cohort supports it; `packvium.holdout` replays that proposal against history it has not + seen. Python-only, and supported but not yet signature-frozen — see + [PUBLIC-API.md](https://github.com/toxakara/packvium-python/blob/main/docs/PUBLIC-API.md). ## Documentation diff --git a/docs/GUARANTEES.md b/docs/GUARANTEES.md index 34bed67..44a9cea 100644 --- a/docs/GUARANTEES.md +++ b/docs/GUARANTEES.md @@ -48,7 +48,7 @@ silently — if you need them, they belong in your own layer above this library. ## Status of this release -Version `1.1.0` freezes the public API. Field names, status codes, the objective +Version `1.2.0` freezes the public API. Field names, status codes, the objective vector, the numeric policy and the validation rules do not change without a major version, so any `1.x` is a safe upgrade from any earlier `1.x`. A caret or tilde constraint on `1.0` is enough; an exact pin is no longer required. diff --git a/docs/PUBLIC-API.md b/docs/PUBLIC-API.md index 9996b03..ecf9af0 100644 --- a/docs/PUBLIC-API.md +++ b/docs/PUBLIC-API.md @@ -113,6 +113,43 @@ move backwards. This is a testing seam, not a serialized request field. catalog was involved), preserved by Python, PHP, Rust and JavaScript; - warnings and top-K alternatives. +### `alternatives`, and what it is not + +Each entry in `alternatives[]` is a **complete result in its own right**, with its own score +vector, containers and unpacked items — the schema says `$ref: "#"` and means it. + +**`configuration.alternatives` counts the winner.** It is the size of the ranked set the +portfolio keeps, so the list of runners-up holds at most `alternatives - 1` entries. The +schema's own minimum, `alternatives: 1`, therefore returns an **empty** list. Ask for `2` to +receive one. + +**The list is per-engine and is not part of the cross-language contract.** Implementations +are not required to explore the same number of portfolio starts — under this engine's own +profiles a request may run one start, nine, or twenty-two — so PHP, Python, Rust and +JavaScript will not generally return the same runners-up, the same number of them, or the +same scores. Do not diff `alternatives` across implementations. Three properties *are* +required of every implementation: + +1. each entry is a complete result that satisfies the result schema on its own; +2. no alternative's score is lexicographically better than the winner's — if one were, it + would be the winner; +3. the list holds at most `configuration.alternatives - 1` entries. + +**An empty list is normal, and it does not mean "nothing else was possible".** Read +`termination.starts`, which names every portfolio start, whether it completed, and which was +selected. Four situations produce an empty list, and only the last is about the difficulty of +the request: + +| Situation | What `termination.starts` shows | +| --- | --- | +| `solver_profile: fast` runs a single solver | one start — this profile can never rank a runner-up | +| `alternatives: 1` | any number of starts, and a cap of zero runners-up | +| only one start finished within the budget | several starts, `completed: true` on one | +| the grid lattice packed everything, so the portfolio stopped | one start, `grid:volume`, selected | + +If you want alternatives, ask for a profile that explores (`balanced` or `quality`) and a +count of at least `2`. + ### Solver metrics Every result, including each alternative, contains an `algorithm.metrics` object: @@ -421,6 +458,103 @@ tariff effective at that instant, no rate for that zone -- is a successful call `"status": "rejected"` with a code from a closed set, exactly as an infeasible packing request returns a result with a status rather than raising. +## Execution plan + +A document derived from an already validated packing result: what to do first, why this +carton, and what was not packed. It is a view, not a decision — it calls no solver and no +validator, and a test in each language asserts that. The contract is +EXECUTION-PLAN.md. + +| Language | Entry point | +| --- | --- | +| Python | `packvium.execution.build_execution_plan(request, result, loading_orders=...)`, `.canonical_plan_json(plan)` | +| PHP | `Packvium\Execution\Plan::build(array $request, array $result, array $loadingOrders = [])`, `::canonicalJson(array $plan)` | +| Rust | `packvium_core::execution::build_plan_json(result_json, loading_orders_json)` | +| JavaScript | `buildExecutionPlan(request, result, {loadingOrders})`, `canonicalPlanJson(plan)` from `@packvium/engine`'s `execution.js` | + +All four are held to **byte-identical** output on the canonical form, over the whole golden +corpus. That is stricter than the packing contract, which allows two engines to place items +differently and compares an objective floor: a plan is derived from a result, so there is +nothing left to differ about. The plan's own JSON Schema is closed — an extra key in one +implementation is a divergence, not a nicety. + +**The step order is injected.** The engines compute a safe loading order from geometry the +adapter never sees, so `loading_orders` is optional: supply it and each step is numbered, +omit it and the container reports `order: "unavailable"` with every placement still listed. +There is no third behaviour, because falling back to the order placements appear in would +present an artifact of how the solver walked its candidates as an order safe to lift boxes +in. + +**Facts and presentation are separate in the output.** Anything the solver or validator +decided is under `facts`; every human-readable sentence is under `presentation` and names +the authoritative fields it was built from in `cites`. A downstream system that reads only +`facts` loses nothing it is entitled to rely on, and an `unpacked_items[].proof.level` of +`observed` stays `observed` in both. + +**Operator locks re-solve; they never edit the plan.** An operator who pins a placement +gets a *new* result beside the approved one rather than an edit of it, so "what was +approved" and "what was proposed" stay two artifacts instead of two states of one. + +| Language | Entry point | +| --- | --- | +| Python | `packvium.locks.resolve_with_locks(request, locks)`, `.locks_from_plan(plan, ...)`, `.lock_registry(locks)` | +| PHP, Rust, JavaScript | **not exported** | + +Each lock becomes an ordinary placement constraint, so the re-solve is the same portfolio +under the same independent validator that any other request gets. There is no lock-aware +solver and no relaxed validation: **a lock cannot produce a placement the engine would +otherwise refuse**, and the strongest thing it can do is reserve its own slot. A lock the +solve cannot honour comes back on `LockedResolve` as unpreserved and names the lock — it is +an outcome, not an exception — while a lock set that contradicts itself, by overlapping or +by naming an item the request does not contain, is refused with `LockSetError` before any +solve runs. + +Python only, and deliberately so: a lock has no representation in the request schema, since +that field is held for the next contract freeze, so there is nothing to hand another engine. + +## Scenario and recommendation API + +Scenario what-if comparison and catalog/rule recommendation publication, exported as +`packvium.simulation` and `packvium.recommendations`. The full contract is +INTELLIGENCE-API.md; what belongs here is the surface and its two +limits, both of which are part of the API rather than notes about it. + +| Language | Entry point | +| --- | --- | +| Python | `packvium.simulation.run_scenario(...)`, `.compare_scenarios(...)`; `packvium.recommendations.propose_recommendation(...)`, `.approve_catalog(...)`, `.approve_policy(...)`; `packvium.holdout.evaluate_on_history(...)` over a `packvium.outcomes.OutcomeLedger` | +| PHP, Rust, JavaScript | **not exported** | + +**Python only, and held to no cross-language conformance.** Every other contract in this +document is proved by driving four engines as subprocesses over JSON. `run_scenario` takes +an `OrderEvaluator` *callback* as its central argument, and a callback does not cross a +pipe -- so the harness that proves the packing and commerce contracts cannot express a +scenario at all. Whether the data-in/data-out half ever crosses runtimes is an open +question, not an omission; INTELLIGENCE-API.md scopes it. + +**Historical replay ships with the ledger it reads.** `evaluate_on_history` scores a +recommendation against decisions held out by an explicit `split_at`, and could not be +exported until `packvium.outcomes` was, because a function whose central argument a caller +cannot construct is worse than an unexported one. Two rules are enforced as control flow +rather than documented: a packing the injected validator rejects is that arm's failure at +any cost and its metrics never reach the comparison, and the validator is handed the +request/result pair with no score in it. Only the deterministically recomputable part of an +outcome is scored -- realised damage, returns, repacks and operator overrides happened +under the carton that actually shipped, so they are reported against the baseline and never +credited to the treatment. + +`compare_scenarios` returns a Pareto report per order and never a blended delta, and +`propose_recommendation` returns `None` rather than a low-confidence proposal when the +paired cohort is too small. Neither writes to a registry: publishing goes only through +`approve_catalog` / `approve_policy`, and a proposal is handed no registry to write to. + +**Supported, and not yet frozen.** This surface and `packvium.locks` are reached by +importing the submodule, which puts them outside the frozen API snapshot the packing and +commerce surfaces are held to. They work, they are tested, and they will not be withdrawn +inside 1.x — but a parameter may be renamed or a returned value gain a field in a minor +release, announced in the changelog rather than blocked by a gate. Pin the version if you +depend on the exact shape. COMPATIBILITY.md states why this one surface +is held that way and what would change it. + ## JSON API Python, PHP, Rust and the JavaScript fallback accept the same top-level keys: `units`, diff --git a/examples/basic.py b/examples/basic.py index 6b531ab..712f033 100644 --- a/examples/basic.py +++ b/examples/basic.py @@ -22,7 +22,10 @@ from packvium import Container, Dimensions, Item, Packer, PackingConfig -result = Packer(PackingConfig.balanced()).pack( +# An example must not change answer merely because the host was busy. This solve needs +# a fraction of the budget below; the generous wall-clock value is only a safety fuse, so +# a loaded machine cannot cut the multi-start portfolio short and let a different start win. +result = Packer(PackingConfig.balanced(time_limit_ms=60_000)).pack( [ Item.create("book", Dimensions.mm("210", "140", "30"), "450 g", quantity=4), Item.create("mug", Dimensions.inches("4", "4", "5"), "12 oz", quantity=2, keep_upright=True), diff --git a/examples/constraints.py b/examples/constraints.py index ed13fa7..78ac8b4 100644 --- a/examples/constraints.py +++ b/examples/constraints.py @@ -23,6 +23,11 @@ explain_unpacked_item, ) +#: An example must not change answer merely because the host was busy. These solves need +#: a fraction of the budget; the generous wall-clock value is only a safety fuse, so a +#: loaded machine cannot cut the multi-start portfolio short and let a different start win. +SAFETY_FUSE_MS = 60_000 + items = [ # `keep_upright` forbids every rotation that would tip the item over. An open tub of # paint is the usual reason. @@ -84,7 +89,7 @@ ), ] -result = Packer(PackingConfig.balanced()).pack(items, containers) +result = Packer(PackingConfig.balanced(time_limit_ms=SAFETY_FUSE_MS)).pack(items, containers) def millimetres(ticks: int) -> str: @@ -135,7 +140,7 @@ def millimetres(ticks: int) -> str: def compare(rule: str, without: list[Item], with_rule: list[Item], containers: list[Container]) -> None: print(f"\n{rule}") for label, variant in (("without the rule", without), ("with the rule ", with_rule)): - outcome = Packer(PackingConfig.balanced()).pack(variant, containers) + outcome = Packer(PackingConfig.balanced(time_limit_ms=SAFETY_FUSE_MS)).pack(variant, containers) placements = sum(len(container.placements) for container in outcome.containers) print( f" {label}: {len(outcome.containers)} container(s), " diff --git a/examples/execution.py b/examples/execution.py new file mode 100644 index 0000000..e6c000f --- /dev/null +++ b/examples/execution.py @@ -0,0 +1,211 @@ +"""Turn a packing result into instructions someone can follow on a dock. + +Run it: + + PYTHONPATH=src python3 examples/execution.py + +`pack()` answers where every box goes. That answer is not yet a work order: it does not +say what to lift first, it does not separate what the solver *decided* from what a screen +should *say*, and it has no way for the operator who is standing there to tell you the +printer must go in the corner. + +The execution plan is that second document. It is derived from an already validated +result -- it calls no solver and no validator, and there is a test in each language that +asserts so. Anything it could decide on its own would be a decision made twice. +""" + +from __future__ import annotations + +import json + +from packvium import pack_from_dict +from packvium.execution import build_execution_plan, canonical_plan_json +from packvium.locks import ( + LockSetError, + PlacementLock, + locks_from_plan, + resolve_with_locks, +) + +# -------------------------------------------------------------------------------------- +# One crate, a printer that must stay upright, four toner cartridges, and a pallet jack +# that was never going to fit. The last one is deliberate: an execution plan has to say +# what is *not* going on the truck as clearly as what is. +# -------------------------------------------------------------------------------------- +REQUEST = { + "units": {"length": "mm"}, + "configuration": { + "objective": "default", + "profile": "balanced", + "seed": 42, + # A safety fuse, not a target -- nothing in this scene comes close to it. + "time_limit_ms": 60_000, + }, + "items": [ + {"id": "printer", "quantity": 1, "weight": "9 kg", "keep_upright": True, + "dimensions": {"length": "420", "width": "340", "height": "260"}}, + {"id": "toner", "quantity": 4, "weight": "900 g", + "dimensions": {"length": "180", "width": "120", "height": "100"}}, + {"id": "pallet-jack", "quantity": 1, "weight": "80 kg", + "dimensions": {"length": "1200", "width": "550", "height": "1200"}}, + ], + "containers": [ + {"id": "crate", "quantity": 1, "max_payload": "30 kg", + "inner_dimensions": {"length": "600", "width": "400", "height": "400"}}, + ], +} + +result = pack_from_dict(REQUEST) +plan = build_execution_plan(REQUEST, result) +container = plan["containers"][0] + + +print("=" * 78) +print("1. What the solver decided, kept apart from what a screen says") +print("=" * 78) +print() + +print(f" format: {plan['format']}") +print(f" status: {plan['facts']['status']}") +print(f" containers used: {plan['facts']['container_count']}") +print(f" score: {plan['facts']['score']}") +print(f" utilization: {container['facts']['volume_utilization']}") +print() +print(" Everything above is under `facts`. It is the solver's own answer, copied and") +print(" not re-derived, so a downstream system that reads only `facts` loses nothing it") +print(" is entitled to rely on. The score stays a vector: collapsing five axes into one") +print(" number is a judgement about your priorities that this document does not make.") +print() + + +print("=" * 78) +print("2. The step order is injected, or it is honestly absent") +print("=" * 78) +print() + +print(f" order: {container['order']}") +for step in container["steps"]: + reference = step["placement"] + ticks = reference["position_ticks"] + print(f" {reference['item_type']:<9} {reference['orientation']} " + f"at ({ticks['x']}, {ticks['y']}, {ticks['z']})") +print() +print(" Every placement is listed and not one is numbered. The engines compute a safe") +print(" loading order from geometry this adapter never sees, so without one it says") +print(" `unavailable` rather than guessing.") +print() +print(" There is no third behaviour on purpose. Falling back to the order placements") +print(" happen to appear in would present an artifact of how the solver walked its") +print(" candidate points as an order that is safe to lift boxes in. It is not.") +print() + +ordered = build_execution_plan( + REQUEST, result, + loading_orders={0: list(range(len(container["steps"]))[::-1])}, +) +print(f" order: {ordered['containers'][0]['order']}") +for step in ordered["containers"][0]["steps"]: + print(f" {step['sequence']}. {step['placement']['item_type']}") +print() +print(" Hand it an order and each step is numbered. The order above is reversed on") +print(" purpose, to show that the sequence is the one you supplied and not one the") +print(" adapter re-derived behind your back.") +print() + + +print("=" * 78) +print("3. Every sentence names the fields it was built from") +print("=" * 78) +print() + +for entry in plan["unplaced"]: + facts = entry["facts"] + print(f" facts: item_type={facts['item_type']!r}") + print(f" reason={facts['reason']!r} proof_level={facts['proof_level']!r}") + print(f" presentation: {entry['presentation']['summary']}") + print(f" cites: {', '.join(entry['presentation']['cites'])}") +print() +print(" `proven` is a claim about a search, not a summary of one: no orientation of the") +print(" pallet jack fits any offered crate, so nothing was tried and nothing needed to") +print(" be. A reason with no citation would be a sentence nobody can check, which is") +print(" why `cites` is part of the format rather than a convention.") +print() + + +print("=" * 78) +print("4. One plan, four engines, the same bytes") +print("=" * 78) +print() + +canonical = canonical_plan_json(plan) +print(f" canonical form: {len(canonical)} bytes, first 68 of them") +print(f" {canonical[:68]}...") +print() +print(" Hand the same *result* to all four adapters and they emit the same bytes. That") +print(" is stricter than the packing contract, and it can be: a plan is derived from a") +print(" result, so there is nothing left to differ about.") +print() +print(" It does not follow that four engines packing the same *request* agree. Python") +print(" and PHP are held to identical placements and do produce this exact form; Rust") +print(" and JavaScript are held to a valid answer at or above the objective floor, and") +print(" `examples/execution.mjs` prints 1280 bytes here rather than 1877 for that") +print(" reason. Whose packing is better is a question the plan never answers.") +print() + + +print("=" * 78) +print("5. The operator pins a box, and gets a second plan — never an edited one") +print("=" * 78) +print() + +locks = locks_from_plan(plan, container_index=0, item_types=["printer"]) +lock = locks[0] +print(f" locked: {lock.item_type} {lock.orientation} at {lock.position_ticks}") +print(" The lock is read out of the plan's own placement reference. That is the only") +print(" reason the two can be trusted to mean the same box: there is no second address") +print(" format to drift.") +print() + +resolved = resolve_with_locks(REQUEST, locks) +print(f" preserved: {resolved.preserved}") +print(f" the approved plan is untouched: " + f"{canonical_plan_json(build_execution_plan(REQUEST, result)) == canonical}") +print() +print(" A lock becomes an ordinary placement constraint, so the re-solve is the same") +print(" portfolio under the same independent validator. Nothing here can produce a") +print(" placement the engine would otherwise refuse — the strongest thing a lock can do") +print(" is reserve its own slot and make the search work around it.") +print() + + +print("=" * 78) +print("6. Two ways a lock can fail, and they are not the same kind of thing") +print("=" * 78) +print() + +# Far outside the crate: a single lock that is individually impossible. The solve still +# runs and still answers; it simply cannot honour this one. +unreachable = PlacementLock(container_index=0, item_type="printer", orientation="LWH", + position_ticks=(99_000_000, 0, 0)) +outcome = resolve_with_locks(REQUEST, (unreachable,)) +print(f" a lock the solve cannot honour -> preserved={outcome.preserved}, " + f"{len(outcome.missing)} lock reported back") +print(f" {outcome.missing[0].item_type} at {outcome.missing[0].position_ticks}") +print(" Not an exception. The operator asked for something the geometry does not allow,") +print(" and what they need back is a valid plan plus the news that their pin was not") +print(" honoured — not a stack trace and no plan at all.") +print() + +# An item the request never contained: the lock set is wrong about the world, and no +# solve could make it right. +try: + resolve_with_locks(REQUEST, (lock, PlacementLock(container_index=0, item_type="scanner", + orientation="LWH", + position_ticks=(0, 0, 0)))) +except LockSetError as refusal: + print(f" a lock set that contradicts itself -> {refusal}") +print() +print(" Refused before any solve runs, because no result could satisfy it. The same goes") +print(" for two locks whose boxes overlap. The distinction is worth keeping: the first") +print(" case is an answer the operator may not like, the second is a question that has") +print(" no answer, and reporting them the same way would hide which one happened.") diff --git a/examples/extensions.py b/examples/extensions.py index 50424c8..e15fd7c 100644 --- a/examples/extensions.py +++ b/examples/extensions.py @@ -20,6 +20,11 @@ from packvium.constraints import ConstraintContext, ConstraintResult from packvium.extensions import DefaultSolutionScorer, ExtensionRegistry +#: An example must not change answer merely because the host was busy. These solves need +#: a fraction of the budget; the generous wall-clock value is only a safety fuse, so a +#: loaded machine cannot cut the multi-start portfolio short and let a different start win. +SAFETY_FUSE_MS = 60_000 + # --------------------------------------------------------------------------------- # A custom placement constraint. `max_top_load` caps what may rest on an item, and # `must_be_on_floor` pins one to the bottom -- but neither says "nothing fragile above @@ -61,14 +66,14 @@ def evaluate(self, context: ConstraintContext) -> ConstraintResult: ] containers = [Container.create("column", Dimensions.mm("400", "400", "1300"), max_payload="200 kg", quantity=1)] -unrestricted = Packer(PackingConfig.balanced()).pack(items, containers) +unrestricted = Packer(PackingConfig.balanced(time_limit_ms=SAFETY_FUSE_MS)).pack(items, containers) highest_vase = max( p.position.z for c in unrestricted.containers for p in c.placements if p.instance.item.id == "vase" ) print("without the rule, the highest vase sits at", Length(highest_vase).decimal("mm"), "mm") restricted = Packer( - PackingConfig.balanced(), + PackingConfig.balanced(time_limit_ms=SAFETY_FUSE_MS), ExtensionRegistry(placement_constraints=(FragileHeightLimit(Length.parse("400 mm")),)), ).pack(items, containers) @@ -110,7 +115,7 @@ def score(self, solution) -> tuple[int, ...]: print() for label, scorer in (("default", None), ("evenly loaded", EvenlyLoadedContainers())): - result = Packer(PackingConfig.balanced(), solution_scorer=scorer).pack(lopsided_items, two_boxes) + result = Packer(PackingConfig.balanced(time_limit_ms=SAFETY_FUSE_MS), solution_scorer=scorer).pack(lopsided_items, two_boxes) contents = [sorted(p.instance.item.id for p in c.placements) for c in result.containers] weights = [c.payload_weight.decimal("kg") + " kg" for c in result.containers] print(f"{label:>15}: {contents} -> {weights}") diff --git a/examples/intelligence.py b/examples/intelligence.py new file mode 100644 index 0000000..9e34fc3 --- /dev/null +++ b/examples/intelligence.py @@ -0,0 +1,263 @@ +"""Prove a carton change is better before you publish it. + +Run it: + + PYTHONPATH=src python3 examples/intelligence.py + +Every other example asks the engine to pack one shipment. This one asks a different +question, the one that comes up when you already pack well and want to change something: +*is the new carton set actually an improvement, and how would I know?* + +Four functions answer it, and none of them returns a single blended number. A score that +folds cost and utilisation together can name a "winner" that is worse on the axis you +actually care about, so what you get back is a per-order Pareto report: which orders +improved, which regressed, and which genuinely traded one axis for another. + +Nothing here reads the clock or the network. The scenarios are pinned to catalog versions +you supply, so the same comparison replays to the same answer next year. +""" + +from __future__ import annotations + +from packvium.holdout import ( + OrderEvaluationArtifact, + SupportingEvidenceInHoldoutError, + ValidationVerdict, + evaluate_on_history, +) +from packvium.outcomes import OutcomeEvent, OutcomeEventType, OutcomeLedger +from packvium.recommendations import propose_recommendation +from packvium.simulation import ( + OrderRunResult, + ScenarioVersionPin, + compare_scenarios, + run_scenario, +) + +# -------------------------------------------------------------------------------------- +# Two carton sets over the same five orders. You would measure these by packing each +# order twice; they are written out here so the example has nothing to hide. +# +# `cost_minor` is in cents and lower is better. `utilisation` is filled volume over +# container volume and higher is better. Order 5 is the interesting one: the proposed +# cartons cannot pack it at all. +# -------------------------------------------------------------------------------------- +MEASURED = { + "order-1": {"baseline": (1240, 0.71), "treatment": (1120, 0.76)}, + "order-2": {"baseline": (980, 0.68), "treatment": (910, 0.72)}, + "order-3": {"baseline": (1500, 0.80), "treatment": (1600, 0.83)}, + "order-4": {"baseline": (2100, 0.64), "treatment": (1890, 0.69)}, + "order-5": {"baseline": (1350, 0.70), "treatment": None}, +} +ORDER_IDS = tuple(MEASURED) + +#: Both directions are stated explicitly. There is no default, because guessing that +#: runtime or cost is "higher is better" would silently invert the whole report. +HIGHER_IS_BETTER = {"cost_minor": False, "utilisation": True} + +#: A pin is what makes a comparison replayable: it names the catalog version each arm +#: was run against rather than whatever is current when you happen to run it. +BASELINE_PIN = ScenarioVersionPin(catalog_version=1) +TREATMENT_PIN = ScenarioVersionPin(catalog_version=2) + + +def arm(name: str): + """An evaluator for one arm. In your code this calls `pack()`; here it replays the + measurements above so the example teaches the comparison and not the packing.""" + + def evaluate(order_id: str, version_pin: ScenarioVersionPin) -> OrderRunResult: + measured = MEASURED[order_id][name] + if measured is None: + return OrderRunResult( + order_id=order_id, succeeded=False, + failure_reason="no carton in the proposed set fits this order", + ) + cost, utilisation = measured + return OrderRunResult( + order_id=order_id, succeeded=True, + metrics={"cost_minor": float(cost), "utilisation": utilisation}, + ) + + return evaluate + + +print("=" * 78) +print("1. Two scenarios, compared order by order") +print("=" * 78) +print() + +baseline = run_scenario("current-cartons", BASELINE_PIN, ORDER_IDS, arm("baseline")) +treatment = run_scenario("proposed-cartons", TREATMENT_PIN, ORDER_IDS, arm("treatment")) + +for report in compare_scenarios(baseline, treatment, HIGHER_IS_BETTER): + if report.winner is not None: + print(f" {report.profile}: {report.winner}") + else: + print(f" {report.profile}: no winner — {' and '.join(report.pareto_optimal)} " + f"are both Pareto-optimal") + +print() +print(" order-3 has no winner because the trade-off is real: the proposed cartons cost") +print(" more and pack denser. A blended score would have picked one and hidden that.") +print() +print(" order-5 names `baseline` as winner for a duller reason — the proposed cartons") +print(" produced no answer there, so there was only one candidate to be optimal. That") +print(" is not a baseline win, and the next step is careful not to count it as one.") +print() + +print("=" * 78) +print("2. A proposal, or an explicit refusal to make one") +print("=" * 78) +print() + +# The paired cohort is four orders out of five, so confidence is 0.8. Ask for more than +# the evidence supports and you get `None` rather than a proposal with a caveat attached. +strict = propose_recommendation( + "rec-cartons-2024", "adopt the proposed carton set", baseline, treatment, + constraints=("no placement the validator would reject",), + rollback_plan="republish catalog version 1", + minimum_cohort_size=4, minimum_confidence=0.9, +) +print(f" minimum_confidence=0.9 -> {strict}") +print(" Four of five orders are comparable, so confidence is 0.80 and the bar is not") +print(" met. `None` is the whole answer: there is no low-confidence proposal to weigh.") +print() + +recommendation = propose_recommendation( + "rec-cartons-2024", "adopt the proposed carton set", baseline, treatment, + constraints=("no placement the validator would reject",), + rollback_plan="republish catalog version 1", + minimum_cohort_size=4, minimum_confidence=0.75, +) +assert recommendation is not None +print(f" minimum_confidence=0.75 -> {recommendation.recommendation_id}") +print(f" supported by: {', '.join(recommendation.supporting_order_ids)}") +print(f" confidence: {recommendation.confidence:.2f}") +for delta in recommendation.expected_deltas: + print(f" {delta.metric:<12} {delta.baseline_mean:>8.3f} -> {delta.treatment_mean:>8.3f}") +print() +print(" order-5 is absent from the supporting orders. A delta is a paired comparison,") +print(" so an order only one arm could pack contributes to neither mean.") +print() + +print("=" * 78) +print("3. Replay against history the proposal has never seen") +print("=" * 78) +print() + +# The ledger is append-only and records what actually happened to a shipment. Its +# timestamps are what a holdout split is made of. +# +# Each event type accepts a closed set of fields and nothing else -- `damage` takes a +# reason and a severity, not a carton id. A ledger that accepted arbitrary payloads would +# be a log; the point of this one is that a replay can rely on what it finds there. +SHIPPED = OutcomeEventType.ACTUAL_CARTON +ledger = OutcomeLedger() +HISTORY = ( + ("order-1", SHIPPED, 100, {"carton_id": "box-a", "catalog_version": 1}), + ("order-2", SHIPPED, 120, {"carton_id": "box-a", "catalog_version": 1}), + ("shipment-101", SHIPPED, 900, {"carton_id": "box-b", "catalog_version": 1}), + ("shipment-102", SHIPPED, 910, {"carton_id": "box-a", "catalog_version": 1}), + ("shipment-103", SHIPPED, 920, {"carton_id": "box-c", "catalog_version": 1}), + ("shipment-104", SHIPPED, 930, {"carton_id": "box-b", "catalog_version": 1}), + # A real bad outcome under the carton that actually shipped. It is reported against + # the baseline and never credited to the treatment, because it happened. + ("shipment-104", OutcomeEventType.DAMAGE, 940, + {"reason_code": "crushed_corner", "severity": "minor"}), +) +for index, (decision_id, event_type, recorded_at, payload) in enumerate(HISTORY, start=1): + ledger.record(OutcomeEvent( + event_id=f"e{index}", decision_id=decision_id, event_type=event_type, + payload=payload, recorded_at=recorded_at, + )) + +REPLAY = { + "shipment-101": {"baseline": (1400, 0.66), "treatment": (1210, 0.73)}, + "shipment-102": {"baseline": (1150, 0.74), "treatment": (1180, 0.77)}, + # Cheaper under the proposed cartons -- and unstackable, which the validator catches. + "shipment-103": {"baseline": (1600, 0.62), "treatment": (1050, 0.81)}, + "shipment-104": {"baseline": (990, 0.69), "treatment": (940, 0.71)}, +} +INVALID = {("shipment-103", "treatment"): ("unsupported_item",)} + + +def replay(decision_id: str, version_pin: ScenarioVersionPin) -> OrderEvaluationArtifact: + """Re-pack one historical decision under whichever carton set the pin names.""" + name = "baseline" if version_pin == BASELINE_PIN else "treatment" + cost, utilisation = REPLAY[decision_id][name] + run = OrderRunResult( + order_id=decision_id, succeeded=True, + metrics={"cost_minor": float(cost), "utilisation": utilisation}, + ) + # The request/result pair is what the validator is handed. It deliberately carries no + # score, so nothing it decides can be influenced by how good the packing looked. + return OrderEvaluationArtifact(run=run, request={"decision_id": decision_id}, + result={"arm": name}) + + +def validator(request, result) -> ValidationVerdict: + """Your own independent check, run over the packing rather than over its score.""" + codes = INVALID.get((request["decision_id"], result["arm"]), ()) + return ValidationVerdict(valid=not codes, codes=codes) + + +evaluation = evaluate_on_history( + recommendation, ledger, + decision_ids=("order-1", "order-2", "shipment-101", "shipment-102", + "shipment-103", "shipment-104"), + baseline_pin=BASELINE_PIN, treatment_pin=TREATMENT_PIN, + evaluator=replay, validator=validator, higher_is_better=HIGHER_IS_BETTER, + split_at=500, +) + +print(" split_at=500 — required, with no default, because how much history is enough") +print(" is a claim about your business that this library cannot make for you.") +print() +print(f" training: {', '.join(evaluation.training_decision_ids)}") +print(f" held out: {', '.join(evaluation.holdout_decision_ids)}") +print() +for decision in evaluation.decisions: + note = "" + if decision.treatment_codes: + note = f" (treatment rejected: {', '.join(decision.treatment_codes)})" + elif decision.baseline_realised_risk: + note = (f" (baseline actually suffered: " + f"{', '.join(decision.baseline_realised_risk)})") + print(f" {decision.decision_id:<14} {decision.verdict}{note}") +print() +print(f" improved {evaluation.improved}, regressed {evaluation.regressed}, " + f"traded off {evaluation.traded_off}, unpackable {evaluation.unpackable}") +print() +print(" shipment-104's note is negative evidence about the baseline, not a reason the") +print(" treatment won. Damage, returns, repacks and operator overrides happened under") +print(" the carton that actually shipped, so they are reported against that arm and") +print(" never credited to the one that was never tried.") +print() +print(" shipment-103 is the one to read twice. The proposed cartons packed it for 1050") +print(" against 1600 and denser besides — and the independent validator rejected the") +print(" placement, so it counts as a regression. There is no cost at which a packing") +print(" your validator refuses becomes an improvement; the metrics are discarded, not") +print(" discounted.") +print() + +print("=" * 78) +print("4. A proposal cannot be scored on its own evidence") +print("=" * 78) +print() + +# Move the split earlier and order-1 -- which the recommendation cites -- lands in the +# held-out set. That is marking your own homework, so it is refused rather than reported. +try: + evaluate_on_history( + recommendation, ledger, + decision_ids=("order-1", "shipment-101"), + baseline_pin=BASELINE_PIN, treatment_pin=TREATMENT_PIN, + evaluator=replay, validator=validator, higher_is_better=HIGHER_IS_BETTER, + split_at=50, + ) +except SupportingEvidenceInHoldoutError as refusal: + print(f" refused: {refusal}") +print() +print(" The refusal is the feature. A holdout score is worth exactly as much as the") +print(" separation between the evidence that produced the proposal and the evidence") +print(" that tests it, and nothing else in the call can enforce that separation.") diff --git a/examples/objectives.py b/examples/objectives.py index 3ade749..2d8f4f5 100644 --- a/examples/objectives.py +++ b/examples/objectives.py @@ -17,6 +17,11 @@ from packvium import Container, Dimensions, Item, Packer, PackingConfig from packvium.models import RateTable, UnratedWeightError +#: An example must not change answer merely because the host was busy. These solves need +#: a fraction of the budget; the generous wall-clock value is only a safety fuse, so a +#: loaded machine cannot cut the multi-start portfolio short and let a different start win. +SAFETY_FUSE_MS = 60_000 + WIDGETS = [Item.create("widget", Dimensions.mm("100", "100", "100"), "500 g", quantity=8)] @@ -32,14 +37,14 @@ def solve(config: PackingConfig, containers) -> tuple[str, tuple[int, ...]]: snug = Container.create("snug", Dimensions.mm("300", "300", "300"), max_payload="20 kg", cost_minor=500) roomy = Container.create("roomy", Dimensions.mm("400", "400", "400"), max_payload="20 kg", cost_minor=150) -print("default ", solve(PackingConfig.balanced(), [snug, roomy])) +print("default ", solve(PackingConfig.balanced(time_limit_ms=SAFETY_FUSE_MS), [snug, roomy])) # --------------------------------------------------------------------------------- # `lowest_cost` -- the cheapest *packaging*. `cost_minor` is what the box itself costs # you, so this is the objective for a warehouse buying cartons, not for a shipper paying # a carrier. Here it prefers the roomy box precisely because the snug one costs more. # --------------------------------------------------------------------------------- -print("lowest_cost ", solve(PackingConfig(objective="lowest_cost"), [snug, roomy])) +print("lowest_cost ", solve(PackingConfig(objective="lowest_cost", time_limit_ms=SAFETY_FUSE_MS), [snug, roomy])) # --------------------------------------------------------------------------------- # `shipping_cost` -- carrier-billable *weight*. Billed weight is the greater of actual @@ -55,6 +60,7 @@ def solve(config: PackingConfig, containers) -> tuple[str, tuple[int, ...]]: dimensional_weight_divisor=5000, dimensional_weight_length_unit="cm", dimensional_weight_weight_unit="kg", + time_limit_ms=SAFETY_FUSE_MS, ) print("shipping_cost ", solve(by_weight, [snug, roomy])) @@ -80,6 +86,7 @@ def solve(config: PackingConfig, containers) -> tuple[str, tuple[int, ...]]: dimensional_weight_divisor=5000, dimensional_weight_length_unit="cm", dimensional_weight_weight_unit="kg", + time_limit_ms=SAFETY_FUSE_MS, ) print("landed_cost ", solve(by_money, [dear_per_gram, cheap_per_gram])) @@ -98,7 +105,7 @@ def solve(config: PackingConfig, containers) -> tuple[str, tuple[int, ...]]: # `open_dimension_height` -- pack into the shortest stack. For a container with no lid, # or a pallet whose height you are trying to keep under a doorway. # --------------------------------------------------------------------------------- -print("open_dimension ", solve(PackingConfig(objective="open_dimension_height"), [snug, roomy])) +print("open_dimension ", solve(PackingConfig(objective="open_dimension_height", time_limit_ms=SAFETY_FUSE_MS), [snug, roomy])) # --------------------------------------------------------------------------------- # `maximum_value` -- when not everything fits, leave the *cheap* things behind. Ranked by @@ -112,7 +119,7 @@ def solve(config: PackingConfig, containers) -> tuple[str, tuple[int, ...]]: Item.create("gold", Dimensions.mm("100", "100", "100"), "500 g", quantity=2, value=90_000), Item.create("gravel", Dimensions.mm("100", "100", "100"), "500 g", quantity=2, value=10), ] -result = Packer(PackingConfig(objective="maximum_value")).pack(mixed, tiny) +result = Packer(PackingConfig(objective="maximum_value", time_limit_ms=SAFETY_FUSE_MS)).pack(mixed, tiny) kept = sorted(p.instance.item.id for c in result.containers for p in c.placements) left = sorted(u.instance.item.id for u in result.unpacked) print("maximum_value ", "packed:", kept, "left behind:", left) diff --git a/examples/shapes.py b/examples/shapes.py index 83ca716..ab17e17 100644 --- a/examples/shapes.py +++ b/examples/shapes.py @@ -32,7 +32,7 @@ def summarise(label: str, request: dict) -> None: `pack_from_dict` answers in the same JSON shape the other three engines return, so everything read here is the cross-language contract rather than a Python attribute. """ - result = pack_from_dict(request) + result = pack_from_dict({**request, **SAFETY_FUSE}) containers = result["containers"] placed = sum(len(container["placements"]) for container in containers) print( @@ -48,6 +48,10 @@ def crate(length: str, width: str, height: str) -> list: MM = {"units": {"length": "mm"}} +# An example must not change answer merely because the host was busy. These solves need a +# fraction of the budget; the generous wall-clock value is only a safety fuse, so a loaded +# machine cannot cut the multi-start portfolio short and let a different start win. +SAFETY_FUSE = {"configuration": {"time_limit_ms": 60_000}} # ------------------------------------------------------------------ convex_hull @@ -125,7 +129,7 @@ def brick(kilograms: int) -> dict: def load(label: str, kilograms: int) -> None: """One crate, one cushion, one brick -- only the brick's mass changes.""" - result = pack_from_dict({**MM, "items": [cushion(100), brick(kilograms)], + result = pack_from_dict({**MM, **SAFETY_FUSE, "items": [cushion(100), brick(kilograms)], "containers": crate("100", "100", "200")}) unused = result["score"][3] print(f" {label:22s} {len(result['containers'])} container(s), " diff --git a/examples/units.py b/examples/units.py index 718ce4b..35c21c6 100644 --- a/examples/units.py +++ b/examples/units.py @@ -89,7 +89,10 @@ print() print("on the wire:", Length.parse("12 3/8 in").to_dict(), Weight.parse("2 3/4 lb").to_dict()) -result = Packer(PackingConfig.balanced()).pack( +# An example must not change answer merely because the host was busy. This solve needs +# a fraction of the budget below; the generous wall-clock value is only a safety fuse, so +# a loaded machine cannot cut the multi-start portfolio short and let a different start win. +result = Packer(PackingConfig.balanced(time_limit_ms=60_000)).pack( [Item.create("shelf", Dimensions.inches("12 3/8", "9 1/2", "3/4"), "2 3/4 lb", quantity=3)], [Container.create("carton", Dimensions.inches("13", "10", "4"), max_payload="20 lb")], ) diff --git a/pyproject.toml b/pyproject.toml index 0d01aa7..bc524c4 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "packvium" -version = "1.1.0" +version = "1.2.0" description = "Deterministic, extensible 3D cartonization and rectangular bin-packing library" readme = "README.md" requires-python = ">=3.9" diff --git a/src/packvium/constraints.py b/src/packvium/constraints.py index 2a37ed0..fa386e4 100644 --- a/src/packvium/constraints.py +++ b/src/packvium/constraints.py @@ -267,7 +267,7 @@ class LoadSupportGraph: order, preserving ContactGraph's integer-remainder and traversal contract. """ - __slots__ = ("_supporters", "_children", "_face", "_units", "_nested") + __slots__ = ("_supporters", "_children", "_face", "_units", "_nested", "_descending", "_prior_descending") def __init__(self, units: Sequence[LoadUnit], cell_hint: int = 1): face = ContactGraph([unit.box for unit in units], cell_hint=cell_hint) @@ -289,9 +289,11 @@ def __init__(self, units: Sequence[LoadUnit], cell_hint: int = 1): self._face = face self._units = tuple(units) self._nested = bool(nesting) + self._descending: tuple[int, ...] | None = None + self._prior_descending: tuple[int, ...] | None = None if not nesting: - self._supporters = tuple(face.supporters(index) for index in range(len(units))) - self._children = tuple(face.children(index) for index in range(len(units))) + self._supporters = () + self._children = () return supporters = [list(face.supporters(index)) for index in range(len(units))] for lower, upper in zip(nesting, nesting[1:]): @@ -343,24 +345,51 @@ def with_unit(self, unit: LoadUnit, cell_hint: int = 1) -> "LoadSupportGraph": """ if self._nested or unit.nesting_item_id is not None: return LoadSupportGraph(self._units + (unit,), cell_hint=cell_hint) - index = len(self._units) face = self._face.with_box(unit.box) # Without nesting this graph *is* the face graph, so read the edges straight off # it rather than patching a copy of the old ones. Re-deriving them by hand would # be a second implementation of the same rule, free to drift from the first. graph = LoadSupportGraph.__new__(LoadSupportGraph) - graph._supporters = tuple(face.supporters(i) for i in range(index + 1)) - graph._children = tuple(face.children(i) for i in range(index + 1)) + graph._supporters = () + graph._children = () graph._face = face graph._units = self._units + (unit,) graph._nested = False + graph._descending = None + graph._prior_descending = self._descending_indices() return graph + def _descending_indices(self) -> tuple[int, ...]: + """Canonical load order, sharing the settled order across candidate siblings.""" + if self._descending is not None: + return self._descending + if self._prior_descending is None: + self._descending = tuple(sorted( + range(len(self._units)), + key=lambda i: (-self._units[i].box.z2, -self._units[i].box.origin.z, i), + )) + return self._descending + order = self._prior_descending + index = len(self._units) - 1 + box = self._units[index].box + key = (-box.z2, -box.origin.z, index) + lo, hi = 0, len(order) + while lo < hi: + mid = (lo + hi) // 2 + other = order[mid] + other_box = self._units[other].box + if key < (-other_box.z2, -other_box.origin.z, other): + hi = mid + else: + lo = mid + 1 + self._descending = order[:lo] + (index,) + order[lo:] + return self._descending + def supporters(self, index: int) -> tuple[ContactEdge, ...]: - return self._supporters[index] + return self._supporters[index] if self._nested else self._face.supporters(index) def children(self, index: int) -> tuple[int, ...]: - return self._children[index] + return self._children[index] if self._nested else self._face.children(index) def non_stackable_failure( @@ -402,7 +431,7 @@ def top_loads(units: Sequence[LoadUnit], graph: LoadSupportGraph | None = None) """ graph = graph if graph is not None else LoadSupportGraph(units) loads = [0] * len(units) - descending = sorted(range(len(units)), key=lambda i: (-units[i].box.z2, -units[i].box.origin.z, i)) + descending = graph._descending_indices() for upper_index in descending: supports = graph.supporters(upper_index) total_area = sum(edge.area for edge in supports) diff --git a/src/packvium/execution.py b/src/packvium/execution.py new file mode 100644 index 0000000..b815833 --- /dev/null +++ b/src/packvium/execution.py @@ -0,0 +1,241 @@ +"""An execution plan derived from an already validated packing result. + +`docs/EXECUTION-PLAN.md` is the contract. A packing result answers *what goes where*; an +operator needs *what to do first, and why this carton*. This module turns the first into +the second and is built so that it cannot do anything else. + +It imports no solver and no validator, holds no registry and reads no clock. Everything it +emits is a function of the request and result it was handed, so the same pair yields the +same plan forever -- which is the only reason a plan can be printed, signed and audited. + +Two rules do most of the work. + +**Authoritative facts and presentation text are separated in the output, not just in the +prose.** Anything the solver or validator decided -- a position, a score vector, a +`feasibility.code`, an `unpacked_items[].proof` and its `level` -- appears under `facts`. +Anything a human reads appears under `presentation`, and every entry there names the +authoritative fields it was derived from in `cites`. A reason with no citation is not a +reason, and a downstream system that only ever reads `facts` loses nothing it is entitled +to rely on. + +**A placement is referenced by what the cross-language contract promises, not by its id.** +A live result does carry `item_id` (`cube#1`), and `conformance/canonical.py` drops it, +along with the container's `id`, from the projection two implementations are diffed +against -- an id there is "an instance count rather than a semantic property". Four engines +need not number instances alike, and this adapter is held to byte-identical Python and PHP +output, so citing `item_id` would fail in the worst way for an operator: two correct +systems disagreeing about which box a label names. The reference is derived from fields the +projection keeps, and uses `position.*.ticks` -- the exact integer -- rather than `value`, +the rendering the same `exactScalar` also carries. +""" + +from __future__ import annotations + +import json +from typing import Any, Mapping, Optional, Sequence + +__all__ = [ + "FORMAT", + "ExecutionPlanError", + "build_execution_plan", + "canonical_plan_json", + "placement_reference", +] + +#: The plan's own format tag. It is not the packing schema's version and does not move +#: with it: this document is derived from a result, and a result can gain fields without +#: changing what a plan says. +FORMAT = "packvium-execution-plan/v1" + +#: What `score` indices mean is a property of the request's objective, and the adapter +#: does not know it. Naming an index it cannot explain would be inventing meaning, so an +#: unnamed index is reported as an index. +UNNAMED_AXIS = "unnamed objective axis" + + +class ExecutionPlanError(Exception): + """The adapter was handed something it cannot describe.""" + + +def placement_reference(container_index: int, placement: Mapping[str, Any]) -> dict[str, Any]: + """A reference two languages agree on, for one placement in one container. + + `container_index` is the container's position in the result, not its `id`: the id is an + instance counter that the cross-language projection drops. Position is read from + `ticks`, never from `value`. + """ + try: + position = placement["position"] + return { + "container_index": container_index, + "item_type": placement["item_type"], + "orientation": placement["orientation"], + "position_ticks": { + axis: int(position[axis]["ticks"]) for axis in ("x", "y", "z") + }, + } + except (KeyError, TypeError) as error: + raise ExecutionPlanError( + f"placement is missing a field the reference is built from: {error}" + ) from error + + +def _steps(container_index: int, container: Mapping[str, Any], + loading_order: Optional[Sequence[int]]) -> dict[str, Any]: + """The operator sequence for one container, or an honest absence of one. + + The engines compute a loading order under support and accessibility rules, from domain + objects this adapter never sees. It is therefore *injected*: a caller who has the order + passes it, and one who does not gets no order at all. The alternative -- falling back to + the order placements happen to appear in -- would present an artifact of how the solver + walked its candidates as if it were a safe order to lift boxes in. + """ + placements = list(container.get("placements") or ()) + if loading_order is None: + return { + "order": "unavailable", + "steps": [ + {"placement": placement_reference(container_index, placement)} + for placement in placements + ], + } + if sorted(loading_order) != list(range(len(placements))): + raise ExecutionPlanError( + f"loading order for container {container_index} is not a permutation of its " + f"{len(placements)} placements" + ) + return { + "order": "loading", + "steps": [ + {"sequence": step, "placement": placement_reference(container_index, placements[index])} + for step, index in enumerate(loading_order, start=1) + ], + } + + +def _first_difference(winner: Sequence[int], loser: Sequence[int]) -> Optional[dict[str, Any]]: + """The first index at which two score vectors differ, and by how much. + + Never a blended number. The score is compared lexicographically by the portfolio that + produced it, so the first differing index *is* the decision; summing or weighting the + vector would replace a decision that was made with one that was not. + """ + for index, (a, b) in enumerate(zip(winner, loser)): + if a != b: + return {"index": index, "winner": a, "alternative": b, "difference": b - a} + if len(winner) != len(loser): + raise ExecutionPlanError( + "score vectors of different length cannot be compared lexicographically" + ) + return None + + +def _alternative(index: int, winner_score: Sequence[int], + alternative: Mapping[str, Any]) -> dict[str, Any]: + difference = _first_difference(winner_score, list(alternative.get("score") or ())) + facts = { + "alternative_index": index, + "score": list(alternative.get("score") or ()), + "status": alternative.get("status"), + "first_difference": difference, + } + if difference is None: + text = ("This option scored identically to the chosen one on every objective axis; " + "the score does not record why one was taken.") + else: + text = (f"This option differs first at objective axis {difference['index']} " + f"({UNNAMED_AXIS}): chosen {difference['winner']}, this {difference['alternative']}.") + return { + "facts": facts, + # Deliberately not "it lost because it is taller". The solver recorded a score, not + # a cause; a sentence naming a cause would be a claim nothing in the result supports. + "presentation": {"summary": text, "cites": ["score", "alternatives[].score"]}, + } + + +def build_execution_plan( + request: Mapping[str, Any], + result: Mapping[str, Any], + *, + loading_orders: Optional[Mapping[int, Sequence[int]]] = None, +) -> dict[str, Any]: + """Derive the execution plan for one validated result. + + `loading_orders` maps a container's index to the engine-computed order its placements + should be loaded in. It is optional and injected because the order is a statement about + physics that this adapter must not make for itself; see `_steps`. + """ + if result.get("status") is None: + raise ExecutionPlanError("a result without a status is not a validated result") + + orders = dict(loading_orders or {}) + containers = list(result.get("containers") or ()) + winner_score = list(result.get("score") or ()) + + plan_containers = [] + for index, container in enumerate(containers): + sequence = _steps(index, container, orders.get(index)) + plan_containers.append({ + "container_index": index, + "facts": { + "container_type": container.get("container_type"), + "placement_count": len(container.get("placements") or ()), + "volume_utilization": container.get("volume_utilization"), + }, + **sequence, + }) + + unplaced = [ + { + "facts": { + "item_type": item.get("item_type"), + "reason": item.get("reason"), + # The proof's `level` is carried through unchanged. Softening `observed` + # into "could not fit" would turn an honest limit into a false certainty. + "proof_level": (item.get("proof") or {}).get("level"), + "details": list(item.get("details") or ()), + }, + "presentation": { + "summary": f"Not packed: {item.get('reason')} " + f"({(item.get('proof') or {}).get('level')}).", + "cites": ["unpacked_items[].reason", "unpacked_items[].proof.level"], + }, + } + for item in (result.get("unpacked_items") or ()) + ] + + alternatives = [ + _alternative(index, winner_score, alternative) + for index, alternative in enumerate(result.get("alternatives") or ()) + ] + + return { + "format": FORMAT, + "objective": result.get("objective"), + "facts": { + "status": result.get("status"), + "score": winner_score, + "feasibility": result.get("feasibility"), + "optimality": result.get("optimality"), + "container_count": len(containers), + }, + "containers": plan_containers, + # Often empty, and not for one reason. Four produce an empty list: the `fast` + # profile runs a single solver, stops the start loop once the grid lattice + # packs everything, only one start completed, or `alternatives: 1` -- the cap counts + # the winner. Measured over the corpus, 165 of 399 requests do carry one, so this is + # not the rare case an earlier draft of this comment claimed. An empty + # list is well-formed and is never an error. + "alternatives": alternatives, + "unplaced": unplaced, + } + + +def canonical_plan_json(plan: Mapping[str, Any]) -> str: + """The one byte-comparable spelling of a plan. + + Cross-language equality is asserted on this string rather than on a parsed object, so + key order and whitespace cannot make two identical plans look different -- the same + discipline `packvium.commerce.canonical_json` applies to a quote. + """ + return json.dumps(plan, sort_keys=True, separators=(",", ":"), ensure_ascii=False) diff --git a/src/packvium/holdout.py b/src/packvium/holdout.py new file mode 100644 index 0000000..55c378c --- /dev/null +++ b/src/packvium/holdout.py @@ -0,0 +1,338 @@ +"""Historical replay and holdout evaluation for a recommendation. + +`docs/INTELLIGENCE-API.md` is the contract. The question this answers is whether a +recommendation *would have* improved past decisions, scored on a slice it was not derived +from -- so a proposal can be argued about before it is ever applied forward. + +Three rules shape every line below, and all three are the difference between a backtest +and a story told about one. + +**The split is by time, and the training view is as of that time.** A decision is held out +when its earliest recorded event is at or after `split_at`. The training side folds only +events recorded strictly before it (`packvium.outcomes.OutcomeLedger.view_as_of`), so a +correction filed +later stays in the immutable ledger but does not reach backwards into what was knowable +when the recommendation was formed. Held-out decisions may accumulate later events, +because those are precisely the outcomes being evaluated. A recommendation whose +`supporting_order_ids` reach into the holdout is refused rather than scored: it would be +citing its own answer sheet. + +**Most of the ledger cannot be scored counterfactually, and pretending otherwise is the +failure mode this module exists to avoid.** The ledger records what happened under the +carton that actually shipped. Re-running the treatment produces a different carton, so +only the deterministically recomputable part -- the packing decision and what the pinned +tariff prices from it -- can be compared. `DAMAGE`, `RETURN`, `REPACK` and +`OPERATOR_OVERRIDE` were realised under a box the treatment did not choose; nothing in the +ledger says what would have happened in another one. They are carried as negative evidence +about the baseline and never as a predicted benefit of the treatment. The obvious +implementation -- diff every recorded metric -- silently credits a recommendation with +avoiding damage it was never tested against. + +**The validator runs unconditionally, before any metric can be compared, and its verdict +is not an input to a score.** An arm the validator rejects has failed for that decision, +whatever it cost. There is no threshold at which that changes, and `validator` is handed +no score, so nothing learned from history can reach it. That is the project's fourth +release gate expressed as control flow rather than as a promise. + +The aggregate is a count of decisions, never a mean delta. A mean over a cohort is one +number that a few large orders can carry by themselves, and refusing that collapse is why +the Pareto comparator exists in the first place. +""" + +from __future__ import annotations + +from ._compat import dataclass +from typing import Any, Callable, Mapping, Optional, Sequence + +from .outcomes import OutcomeEventType, OutcomeLedger +from .recommendations import Recommendation +from .simulation import ( + OrderRunResult, + ScenarioResult, + ScenarioVersionPin, + compare_scenarios, +) + +__all__ = [ + "DecisionOutcome", + "HoldoutError", + "HoldoutEvaluation", + "OrderEvaluationArtifact", + "SupportingEvidenceInHoldoutError", + "ValidationVerdict", + "evaluate_on_history", +] + + +#: Ledger facts that describe something a human or the world did to the carton that +#: actually shipped. Under a different carton they may not have happened at all, so they +#: are reported against the baseline and never credited to the treatment. +NOT_COUNTERFACTUAL = frozenset({ + OutcomeEventType.DAMAGE, + OutcomeEventType.RETURN, + OutcomeEventType.REPACK, + OutcomeEventType.OPERATOR_OVERRIDE, +}) + +IMPROVED = "improved" +REGRESSED = "regressed" +TRADED_OFF = "traded_off" +UNPACKABLE = "unpackable" + + +class HoldoutError(Exception): + """Base class for every error raised by this module.""" + + +class SupportingEvidenceInHoldoutError(HoldoutError): + """The recommendation cites a decision that the split holds out. + + Refused rather than scored: a recommendation evaluated on the evidence it was derived + from measures how well it remembers, not whether it generalises. + """ + + +@dataclass(frozen=True, slots=True) +class ValidationVerdict: + """An independent validator's answer about one packing. + + Deliberately carries no score and no metric. It is produced by a callback that is + never handed one, which is what keeps a learned number from reaching the verdict. + """ + + valid: bool + codes: tuple[str, ...] = () + + def __post_init__(self) -> None: + if self.valid and self.codes: + raise ValueError("a valid verdict cannot carry rejection codes") + if not self.valid and not self.codes: + raise ValueError("a rejection must say which rules it failed") + + +@dataclass(frozen=True, slots=True) +class OrderEvaluationArtifact: + """One arm's run of one decision, plus the pair an independent check needs. + + The forward path's `OrderEvaluationArtifact` retains score, rates and ids but not + placement geometry, so claiming it can be independently validated would be false. + This envelope requires the request and result explicitly for that reason -- the + validator is given the packing, not a summary of it. + """ + + run: OrderRunResult + request: Any + result: Any + + def __post_init__(self) -> None: + if self.request is None or self.result is None: + raise ValueError( + "a holdout artifact must carry the request/result pair the validator reads; " + "a run summary alone cannot be independently validated" + ) + + +#: `(decision_id, pin) -> OrderEvaluationArtifact`. +HoldoutEvaluator = Callable[[str, ScenarioVersionPin], OrderEvaluationArtifact] + +#: `(request, result) -> ValidationVerdict`. Separate from the evaluator on purpose: an +#: engine that graded its own homework would make the gate a comment. +IndependentValidator = Callable[[Any, Any], ValidationVerdict] + + +@dataclass(frozen=True, slots=True) +class DecisionOutcome: + """What the replay found for one held-out decision, retained rather than summarised. + + The release gate about performance claims requires raw artifacts, not just counts, so + every verdict keeps the validator codes that produced it and the realised-risk events + the baseline actually incurred. + """ + + decision_id: str + verdict: str + baseline_valid: bool + treatment_valid: bool + baseline_codes: tuple[str, ...] = () + treatment_codes: tuple[str, ...] = () + baseline_realised_risk: tuple[str, ...] = () + + def __post_init__(self) -> None: + if not self.decision_id: + raise ValueError("decision_id is required") + if self.verdict not in (IMPROVED, REGRESSED, TRADED_OFF, UNPACKABLE): + raise ValueError(f"unknown verdict {self.verdict!r}") + if self.verdict == IMPROVED and not self.treatment_valid: + raise ValueError( + "a packing the validator rejected can never be recorded as an improvement" + ) + + +@dataclass(frozen=True, slots=True) +class HoldoutEvaluation: + """The complete record of one holdout evaluation. + + Retains `split_at`, both pins and every per-decision verdict -- not just the counts -- + so the run can be argued with rather than merely believed. + """ + + recommendation_id: str + split_at: int + baseline_pin: ScenarioVersionPin + treatment_pin: ScenarioVersionPin + training_decision_ids: tuple[str, ...] + holdout_decision_ids: tuple[str, ...] + decisions: tuple[DecisionOutcome, ...] = () + + @property + def improved(self) -> int: + return sum(1 for d in self.decisions if d.verdict == IMPROVED) + + @property + def regressed(self) -> int: + return sum(1 for d in self.decisions if d.verdict == REGRESSED) + + @property + def traded_off(self) -> int: + return sum(1 for d in self.decisions if d.verdict == TRADED_OFF) + + @property + def unpackable(self) -> int: + return sum(1 for d in self.decisions if d.verdict == UNPACKABLE) + + +def _earliest_event_time(ledger: OutcomeLedger, decision_id: str) -> Optional[int]: + events = ledger.events_for_decision(decision_id) + if not events: + return None + return min(event.recorded_at for event in events) + + +def _realised_risk(ledger: OutcomeLedger, decision_id: str) -> tuple[str, ...]: + """The baseline's realised bad outcomes, as recorded. Negative evidence only.""" + return tuple(sorted({ + event.event_type.value + for event in ledger.current_view(decision_id) + if event.event_type in NOT_COUNTERFACTUAL + })) + + +def _arm( + decision_id: str, pin: ScenarioVersionPin, + evaluator: HoldoutEvaluator, validator: IndependentValidator, +) -> tuple[OrderRunResult, ValidationVerdict]: + """Run one arm and validate it. The verdict is produced before any metric is read.""" + artifact = evaluator(decision_id, pin) + verdict = validator(artifact.request, artifact.result) + if not verdict.valid: + # The arm failed for this decision. Its metrics are discarded rather than + # down-weighted: there is no cost at which a rejected placement competes. + return ( + OrderRunResult( + order_id=decision_id, succeeded=False, + failure_reason="independent validation rejected the placement: " + + ", ".join(verdict.codes), + ), + verdict, + ) + return artifact.run, verdict + + +def _verdict_for( + decision_id: str, baseline_run: OrderRunResult, treatment_run: OrderRunResult, + baseline_pin: ScenarioVersionPin, treatment_pin: ScenarioVersionPin, + higher_is_better: Mapping[str, bool], +) -> str: + if not baseline_run.succeeded and not treatment_run.succeeded: + return UNPACKABLE + if not treatment_run.succeeded: + return REGRESSED + if not baseline_run.succeeded: + # The treatment packed a decision the baseline could not, and its own packing + # passed the validator. That is an improvement in the only sense available here. + return IMPROVED + + # Both arms are valid, so the comparison is the same per-decision Pareto report the + # forward path uses -- one profile, two engines, never a blended delta. + reports = compare_scenarios( + ScenarioResult(scenario_id="baseline", version_pin=baseline_pin, + order_ids=(decision_id,), runs=(baseline_run,)), + ScenarioResult(scenario_id="treatment", version_pin=treatment_pin, + order_ids=(decision_id,), runs=(treatment_run,)), + higher_is_better, + ) + report = reports[0] + if report.winner == "treatment": + return IMPROVED + if report.winner == "baseline": + return REGRESSED + return TRADED_OFF + + +def evaluate_on_history( + recommendation: Recommendation, + ledger: OutcomeLedger, + decision_ids: Sequence[str], + baseline_pin: ScenarioVersionPin, + treatment_pin: ScenarioVersionPin, + evaluator: HoldoutEvaluator, + validator: IndependentValidator, + higher_is_better: Mapping[str, bool], + *, + split_at: int, +) -> HoldoutEvaluation: + """Score `recommendation` against the decisions held out by `split_at`. + + `split_at` is required and has no default. Any default would be this library making a + claim about how much history is enough for a business it cannot see. + + Raises `SupportingEvidenceInHoldoutError` when the recommendation cites a held-out + decision, and `ValueError` when a decision has no recorded events -- an unrecorded + decision cannot be placed on either side of a split by time. + """ + if split_at < 0: + raise ValueError("split_at cannot be negative") + if not decision_ids: + raise ValueError("at least one decision id is required") + + training: list[str] = [] + holdout: list[str] = [] + for decision_id in decision_ids: + earliest = _earliest_event_time(ledger, decision_id) + if earliest is None: + raise ValueError( + f"decision {decision_id!r} has no recorded events, so it cannot be split by time" + ) + (holdout if earliest >= split_at else training).append(decision_id) + + cited_in_holdout = sorted(set(recommendation.supporting_order_ids) & set(holdout)) + if cited_in_holdout: + raise SupportingEvidenceInHoldoutError( + f"recommendation {recommendation.recommendation_id!r} cites held-out " + f"decision(s) {cited_in_holdout}; it may not be scored on its own evidence" + ) + + outcomes: list[DecisionOutcome] = [] + for decision_id in holdout: + baseline_run, baseline_verdict = _arm(decision_id, baseline_pin, evaluator, validator) + treatment_run, treatment_verdict = _arm(decision_id, treatment_pin, evaluator, validator) + outcomes.append(DecisionOutcome( + decision_id=decision_id, + verdict=_verdict_for(decision_id, baseline_run, treatment_run, + baseline_pin, treatment_pin, higher_is_better), + baseline_valid=baseline_verdict.valid, + treatment_valid=treatment_verdict.valid, + baseline_codes=baseline_verdict.codes, + treatment_codes=treatment_verdict.codes, + baseline_realised_risk=_realised_risk(ledger, decision_id), + )) + + return HoldoutEvaluation( + recommendation_id=recommendation.recommendation_id, + split_at=split_at, + baseline_pin=baseline_pin, + treatment_pin=treatment_pin, + training_decision_ids=tuple(training), + holdout_decision_ids=tuple(holdout), + decisions=tuple(outcomes), + ) diff --git a/src/packvium/locks.py b/src/packvium/locks.py new file mode 100644 index 0000000..8b3e17c --- /dev/null +++ b/src/packvium/locks.py @@ -0,0 +1,392 @@ +"""Operator locks, expressed as request-derived constraints. + +`docs/EXECUTION-PLAN.md` is the contract. A lock is an operator saying *this item stays in +this box, in this place*, and the whole design problem is that honouring one must not become +a way around the solver or the validator. + +It is not. A lock is a `PlacementConstraint` supplied through the existing +`ExtensionRegistry`, and everything after that is the ordinary path: the same portfolio, the +same constraints, the same independent validation. There is no lock-aware solver, no relaxed +validator and no special case anywhere in the engine -- which is why a lock cannot produce a +placement the engine would otherwise refuse. The strongest thing a lock can do is *forbid* +alternatives to itself. + +Four properties, and the first is what makes the rest safe. + +**The approved plan is never mutated.** `resolve_with_locks` returns a `LockedResolve` +carrying a *separate* result. The original is untouched and citable, so "what was approved" +and "what was proposed after the lock" are two artifacts rather than two states of one. + +**A physically invalid lock is rejected, not accommodated.** The lock constraint refuses +every candidate that is not the locked one; it never asserts that the locked one is legal. +Support, top-load, overlap and every other rule still run, so a lock that would float an item +simply leaves it unplaced -- and the result says so with the engine's own vocabulary. + +**Infeasibility is an answer, not an exception.** A lock set the solver cannot satisfy comes +back as `preserved=False` with the ordinary `unpacked_items[].proof`, not as a raised error. +A well-formed request the model cannot answer has always been a result with a status here. + +**Preservation is verified, not assumed.** Forbidding alternatives does not make the solver +try the locked point; it only stops it succeeding anywhere else. Whether the lock actually +held is therefore checked against the returned result, and reported. + +## A lock reserves a slot; it does not demand priority + +The reference `docs/EXECUTION-PLAN.md` derives is `(container_index, item_type, +position_ticks, orientation)` -- deliberately not `item_id`, because an id is an instance +counter four engines need not agree on. That address names a *placement*, and the constraint +has to mean the same thing: locking one of eight identical cubes must leave the other seven +free to go wherever they fit. + +Two earlier readings of that failed, and both failed by emptying a container that had been +full. Read as a rule about the *item type* -- no cube may be anywhere but the locked point -- +a container that held eight came back holding one. Read as a rule about *order* -- the locked +slot must be filled before any other instance -- a lock on the far corner came back holding +nothing, because candidate points are extreme points derived from what is already placed and +an empty container offers only the origin. A lock the search cannot reach yet is not an +infeasible lock. + +So the rule reserves rather than forces. While a lock is outstanding, its box is off limits +to every other candidate of that type; the candidate that *is* the lock is admitted, and +every candidate that does not touch the reserved volume is untouched. The locked slot is +therefore still empty when the search finally reaches it, which is what makes the lock hold +without any part of the engine knowing a lock exists. + +Nothing here forces the slot to be filled, and that is the honest shape: a reservation the +solve never used comes back as a `missing` lock rather than as a load with seven items +deleted from it. + +## What a lock costs, and what it does not + +Locking is not free, and the cost is not in this module. Registering *any* caller constraint +takes a request off `GridSolver` (`solvers.py`), because the grid places by formula and never +evaluates the constraint chain -- a grid solver that ignored a lock would be the bypass +property 4 forbids. On an exactly tiling request that is a real jump in candidates evaluated, +3.5x at 8 items and 26x at 125, and a do-nothing constraint pays every bit of it. + +The lock's own cost, measured against that baseline in `benchmarks/lock_scaling.py`, is a +ratio of 1.00 at every size: the same candidates are evaluated, some with a different verdict. +Locking changes which solver answers, not how the answer scales. + +`context.placements` is per-container search state, and the search is deterministic, so the +outstanding set is a deterministic function of the request and the lock set. + +The reservation covers other instances of the locked type, which is what the address makes +computable: a lock names an `item_type`, and the box it occupies follows from that type's +dimensions under the locked orientation. A *different* type taking the reserved volume is +not refused, because the constraint has no way to size a box it is not currently evaluating +-- it is caught after the solve, as a lock reported `missing`. + +## The one thing a constraint cannot see + +A lock names a container index; a constraint runs inside a search that has no index yet -- +`container_index` is a position in the *result's* sorted container order, which does not +exist until the solve finishes. The constraint therefore applies the outstanding rule in +every container the locked box could physically occupy, and `resolve_with_locks` verifies +preservation at the exact index afterwards. Over-applying can only *forbid*, never place, +so the failure mode is a reported `missing` lock and never a placement the engine would have +refused. +""" + +from __future__ import annotations + +from ._compat import dataclass +from typing import Any, Mapping, Optional, Sequence + +from .constraints import ConstraintContext, ConstraintResult +from .extensions import ExtensionRegistry +from .geometry import AxisAlignedBox, Point, Rotation + +__all__ = [ + "LOCK_VIOLATED", + "LockSetError", + "LockedResolve", + "PlacementLock", + "lock_registry", + "locks_from_plan", + "resolve_with_locks", +] + +#: The rejection code a locked item's non-locked candidate carries. Named like every other +#: constraint code so that a trace, a diagnostic and an `unpacked_items[].details` entry +#: read the same way whether the refusal came from physics or from an operator. +LOCK_VIOLATED = "operator_lock" + +_Slot = tuple[str, tuple[int, int, int]] + + +class LockSetError(ValueError): + """A lock set that no request could satisfy, raised before any solve. + + The distinction this draws is the one that decides what an operator is told. A lock the + *solve* could not honour is an answer -- `preserved=False`, with the lock named in + `missing` -- because whether it fits is a question about a request. A lock set that + contradicts *itself* is not a question at all: two locks whose boxes overlap cannot both + hold against any request, in any container, under any solver. Answering it with a solve + would return an emptied container and call it a result. + + This is the same line `packvium.execution` draws when it refuses a loading order that is + not a permutation. + """ + + +@dataclass(frozen=True, slots=True) +class PlacementLock: + """One operator lock: an item type pinned to an exact position and orientation. + + Addressed the way the execution plan addresses a placement -- by `container_index`, + `item_type`, exact integer `position_ticks` and `orientation`, never by `item_id`. The + reason is the one `docs/EXECUTION-PLAN.md` gives: an id is an instance counter that four + engines need not agree on, and a lock that meant different boxes in different runtimes + would be worse than no lock. + """ + + container_index: int + item_type: str + orientation: str + position_ticks: tuple[int, int, int] + + def __post_init__(self) -> None: + if self.container_index < 0: + raise ValueError("a lock must name a container index") + if not self.item_type: + raise ValueError("a lock must name an item type") + if self.orientation not in Rotation.__members__: + raise ValueError(f"{self.orientation!r} is not an orientation this engine emits") + if len(self.position_ticks) != 3: + raise ValueError("a lock's position must have three axes") + if any(not isinstance(t, int) or isinstance(t, bool) for t in self.position_ticks): + raise ValueError("a lock's position must be exact integer ticks") + if any(t < 0 for t in self.position_ticks): + raise ValueError("a lock's position cannot be negative") + + @property + def slot(self) -> _Slot: + return (self.orientation, tuple(self.position_ticks)) + + +def locks_from_plan(plan: Mapping[str, Any], *, container_index: int, + item_types: Sequence[str]) -> tuple[PlacementLock, ...]: + """Read locks out of an execution plan's own placement references. + + The plan is where an operator sees the placements, so it is where they point at one. + This reads the references the plan already emits rather than inventing a second address + format, which is the only reason the two can be trusted to mean the same box. + """ + wanted = set(item_types) + locks = [] + for container in plan.get("containers") or (): + if container.get("container_index") != container_index: + continue + for step in container.get("steps") or (): + reference = step.get("placement") or {} + if reference.get("item_type") not in wanted: + continue + ticks = reference.get("position_ticks") or {} + locks.append(PlacementLock( + container_index=container_index, + item_type=str(reference["item_type"]), + orientation=str(reference["orientation"]), + position_ticks=(int(ticks["x"]), int(ticks["y"]), int(ticks["z"])), + )) + return tuple(locks) + + +@dataclass(frozen=True, slots=True) +class _LockConstraint: + """Refuses a locked type's candidates while that type still owes a locked slot. + + Deliberately one-directional. It can say *no*, and it can say nothing else: there is no + branch in which it returns `allow()` for a placement the other constraints would have + refused, because it never runs instead of them -- it runs alongside them, and every + constraint must allow a candidate for it to be placed. + + Three narrowings keep a lock from costing more than it asks for. An item type with no + lock is untouched. A lock whose box cannot fit this container is not applied here at all. + And a candidate that does not touch a reserved volume is allowed wherever it lands, which + is what leaves the seven unlocked cubes free. + + Complexity: O(L) per candidate for L locks naming the item's type, plus one walk of the + container's placements when such a lock exists -- the same O(P) walk the support and + overlap rules already make, so `docs/ALGORITHMS-AND-COMPLEXITY.md`'s per-candidate bound + is unchanged. A request with no locks never constructs this constraint at all. + + Measured, in `benchmarks/lock_scaling.py`: against a caller-constraint baseline, a lock + evaluates exactly the same number of candidates at every size -- ratio 1.00 from 8 items + to 125 -- which is the constant factor the bound requires. + """ + + locks: tuple[PlacementLock, ...] + + def _applicable(self, lock: PlacementLock, context: ConstraintContext) -> bool: + """Whether this container could hold the locked box at all. + + A constraint cannot know which container index it is packing, so without this a lock + read from one container would steer the first instance in *every* container -- and in + a container too small for the locked origin, would refuse every candidate and leave + the type unpacked. The test is the physical box rather than the clearance envelope: + it exists to drop locks that plainly cannot apply here, and a lock this admits is + still judged by the ordinary geometry rules. + """ + box = context.item.dimensions.rotated(Rotation[lock.orientation]) + inner = context.container.inner_dimensions + x, y, z = lock.position_ticks + return (x + box.length.ticks <= inner.length.ticks + and y + box.width.ticks <= inner.width.ticks + and z + box.height.ticks <= inner.height.ticks) + + def _reserved_box(self, lock: PlacementLock, + context: ConstraintContext) -> AxisAlignedBox: + """The volume the locked item will occupy, sized from its own type. + + The physical box rather than the clearance envelope: a clearance is a margin the + ordinary clearance rule already enforces around whatever ends up here, and reserving + it twice would refuse neighbours the engine is willing to place. + """ + dimensions = context.item.dimensions.rotated(Rotation[lock.orientation]) + return AxisAlignedBox(Point(*lock.position_ticks), dimensions) + + def evaluate(self, context: ConstraintContext) -> ConstraintResult: + item_type = context.item.item.id + relevant = [lock for lock in self.locks + if lock.item_type == item_type and self._applicable(lock, context)] + if not relevant: + return ConstraintResult.allow() + + filled = { + (placement.rotation.name, + (placement.position.x, placement.position.y, placement.position.z)) + for placement in context.placements + if placement.instance.item.id == item_type + } + outstanding = [lock for lock in relevant if lock.slot not in filled] + if not outstanding: + return ConstraintResult.allow() + + candidate_slot = (context.rotation.name, + (context.point.x, context.point.y, context.point.z)) + candidate_box = AxisAlignedBox(context.point, context.dimensions) + for lock in outstanding: + if lock.slot == candidate_slot: + continue + if candidate_box.intersects(self._reserved_box(lock, context)): + return ConstraintResult.reject( + LOCK_VIOLATED, + f"{item_type} is locked to {lock.orientation}@{lock.position_ticks}", + ) + return ConstraintResult.allow() + + +def lock_registry(locks: Sequence[PlacementLock], + base: Optional[ExtensionRegistry] = None) -> ExtensionRegistry: + """The caller's extensions plus the lock constraint. + + A registry rather than a new solver argument, because that is the extension point this + engine already has and using it means the locked run is the ordinary run. `base` is + preserved so an application's own constraints are not silently dropped by locking. + """ + registry = base or ExtensionRegistry() + if not locks: + return registry + constraint = _LockConstraint(tuple(locks)) + return ExtensionRegistry( + placement_constraints=(*registry.placement_constraints, constraint), + item_order_strategies=registry.item_order_strategies, + solvers=registry.solvers, + container_selector=registry.container_selector, + ) + + +@dataclass(frozen=True, slots=True) +class LockedResolve: + """A re-solve under a lock set, beside the plan it came from -- never replacing it. + + `preserved` is measured against `result`, not promised by the constraint: forbidding + alternatives stops the solver succeeding elsewhere, it does not make it try the locked + point. `missing` names the locks the result does not contain, which is what a caller + shows an operator when their lock could not be honoured. + """ + + locks: tuple[PlacementLock, ...] + result: Mapping[str, Any] + preserved: bool + missing: tuple[PlacementLock, ...] = () + + def __post_init__(self) -> None: + if self.preserved and self.missing: + raise ValueError("a preserved resolve cannot be missing a lock") + if not self.preserved and not self.missing: + raise ValueError("an unpreserved resolve must name the locks it could not honour") + + +def _placed(result: Mapping[str, Any]) -> set[tuple[int, str, str, tuple[int, int, int]]]: + """Every placement in the result, keyed the way a lock addresses one. + + `container_index` is the container's position in the emitted list, which is the same + order `packvium.execution` indexes and the order the plan's references carry. + """ + placed = set() + for index, container in enumerate(result.get("containers") or ()): + for placement in container.get("placements") or (): + position = placement.get("position") or {} + try: + ticks = tuple(int(position[axis]["ticks"]) for axis in ("x", "y", "z")) + except (KeyError, TypeError): + continue + placed.add((index, str(placement.get("item_type")), + str(placement.get("orientation")), ticks)) + return placed + + +def _refuse_a_contradictory_set(locks: Sequence[PlacementLock], + items: Mapping[str, Any]) -> None: + """Every way a lock set can be wrong without a request being consulted. + + Cheap and exhaustive: O(L^2) over a set an operator typed, against a solve that is the + expensive thing here. Locks in different containers are never compared -- they describe + different boxes and cannot contradict one another. + """ + boxes = [] + for lock in locks: + item = items.get(lock.item_type) + if item is None: + raise LockSetError(f"the request has no item type {lock.item_type!r} to lock") + dimensions = item.dimensions.rotated(Rotation[lock.orientation]) + boxes.append((lock, AxisAlignedBox(Point(*lock.position_ticks), dimensions))) + + for index, (lock, box) in enumerate(boxes): + for other, other_box in boxes[index + 1:]: + if lock.container_index != other.container_index: + continue + if lock.slot == other.slot and lock.item_type == other.item_type: + raise LockSetError(f"the same placement is locked twice: {lock.slot}") + if box.intersects(other_box): + raise LockSetError( + f"two locks claim overlapping space in container {lock.container_index}: " + f"{lock.item_type} at {lock.orientation}@{lock.position_ticks} and " + f"{other.item_type} at {other.orientation}@{other.position_ticks}") + + +def resolve_with_locks(request: Mapping[str, Any], locks: Sequence[PlacementLock], + *, base: Optional[ExtensionRegistry] = None) -> LockedResolve: + """Re-solve `request` with `locks` applied, through the ordinary packing path. + + Imported here rather than at module scope so that this module holds no import-time + dependency on the solver: the lock layer is a caller of the engine, not a part of it, + and `docs/EXECUTION-PLAN.md` makes the direction of that dependency a rule. + """ + from .serialization import _item, pack_from_dict + + locks = tuple(locks) + unit = (request.get("units") or {}).get("length", "mm") + _refuse_a_contradictory_set(locks, { + raw["id"]: _item(raw, unit) for raw in request.get("items") or () + }) + result = pack_from_dict(dict(request), extensions=lock_registry(locks, base)) + placed = _placed(result) + missing = tuple( + lock for lock in locks + if (lock.container_index, lock.item_type, lock.orientation, + tuple(lock.position_ticks)) not in placed + ) + return LockedResolve(locks=locks, result=result, preserved=not missing, missing=missing) diff --git a/src/packvium/outcomes.py b/src/packvium/outcomes.py new file mode 100644 index 0000000..eafcb84 --- /dev/null +++ b/src/packvium/outcomes.py @@ -0,0 +1,212 @@ +"""Domain model for first-party shipment-outcome and packing-exception events. + +Closing the operating loop -- which alternative was actually chosen, what carton was +really used, measured dimensions/weight, a repack, damage, a return, an operator override +-- needs its own append-only ledger, not a mutable "decision" row a later event can quietly +edit: + + * every event names the exact decision it is about (`decision_id` -- expected to be a + job's own idempotency key in the caller's system; this module never imports one, to + stay decoupled) and is itself keyed by its own `event_id`, so a duplicate + delivery of the same event is idempotent (`OutcomeLedger.record`) the same way + `JobRegistry.submit` dedups a duplicate job submission; + * a correction never rewrites an earlier event -- it is a new event that names the + event id it supersedes (`OutcomeEvent.supersedes`), so "late corrections preserve + history": the raw ledger (`events_for_decision`) always has everything that was ever + recorded, while `current_view` folds corrections in to answer "what do we believe + now" without discarding what was believed before; + * every event's payload is validated against a closed, named field list per event type + (`ALLOWED_FIELDS`) -- a free-text field this module does not recognize (the PII risk + this task's acceptance calls out: a customer name, address, phone number typed into + an unstructured note) is a structured rejection at construction, not something that + is silently stored; + * "validation remains a hard gate" is not a rule this module enforces at runtime -- it + is a structural guarantee: this module has no method that reads, re-runs or + overrides a packing decision's own feasibility validation. It only ever appends + structured facts *about* a decision that some other, already-validated system + produced. There is no code path here that could bypass validation, because none of + this module's code ever touches a placement or a validator. + +Scope: the same kind of in-memory, domain-layer reference implementation +`packvium.commerce.catalog` and `packvium.commerce.policy` already are. A production +deployment would back `OutcomeLedger` with durable, truly-append-only storage; this module +defines and proves the contract, not the store. +""" + +from __future__ import annotations + +from ._compat import dataclass +from enum import Enum +from typing import Any, Mapping, Optional + + +# --------------------------------------------------------------------------------- errors + +class OutcomeError(Exception): + """Base class for every outcome-domain error raised by this module.""" + + +class UnsupportedOutcomeFieldError(OutcomeError): + """A payload field is not in the closed, named vocabulary for its event type -- + refused at construction, the same "fail admission instead of being ignored" + discipline `packvium.commerce.policy` uses for predicates. This is the module's PII + guard: a field this module does not explicitly know about (e.g. a free-text note + that might contain a name or address) can never be silently stored.""" + + +class DuplicateEventMismatchError(OutcomeError): + """The same `event_id` was recorded again with different content -- an idempotent + duplicate delivery must repeat the identical event, never a different one under the + same id.""" + + +class OutcomeEventNotFoundError(OutcomeError): + """No event is recorded under the given event id.""" + + +# ---------------------------------------------------------------------------- event types + +class OutcomeEventType(str, Enum): + ALTERNATIVE_CHOSEN = "alternative_chosen" + ACTUAL_CARTON = "actual_carton" + MEASURED_DIMENSIONS = "measured_dimensions" + MEASURED_WEIGHT = "measured_weight" + REPACK = "repack" + DAMAGE = "damage" + RETURN = "return" + OPERATOR_OVERRIDE = "operator_override" + + +#: The closed, named payload vocabulary per event type. Deliberately narrow and +#: structured (ids, numeric measurements, enumerated reason codes) -- nothing here is a +#: free-text field a caller could use to smuggle in personally identifiable data. A field +#: outside this list is refused at construction (`UnsupportedOutcomeFieldError`), never +#: silently dropped or silently stored. +ALLOWED_FIELDS: dict[OutcomeEventType, frozenset[str]] = { + OutcomeEventType.ALTERNATIVE_CHOSEN: frozenset({"alternative_index", "objective_score"}), + OutcomeEventType.ACTUAL_CARTON: frozenset({"carton_id", "catalog_version"}), + OutcomeEventType.MEASURED_DIMENSIONS: frozenset({ + "length_mm", "width_mm", "height_mm", + }), + OutcomeEventType.MEASURED_WEIGHT: frozenset({"weight_g"}), + OutcomeEventType.REPACK: frozenset({"reason_code", "new_carton_id"}), + OutcomeEventType.DAMAGE: frozenset({"reason_code", "severity"}), + OutcomeEventType.RETURN: frozenset({"reason_code"}), + OutcomeEventType.OPERATOR_OVERRIDE: frozenset({"reason_code", "operator_id"}), +} + + +def _validate_payload(event_type: OutcomeEventType, payload: Mapping[str, Any]) -> None: + allowed = ALLOWED_FIELDS.get(event_type) + if allowed is None: + raise UnsupportedOutcomeFieldError(f"unsupported event type {event_type!r}") + unsupported = set(payload) - allowed + if unsupported: + raise UnsupportedOutcomeFieldError( + f"event type {event_type.value!r} does not support field(s) {sorted(unsupported)}; " + f"allowed: {sorted(allowed)}" + ) + + +# ---------------------------------------------------------------------------------- event + +@dataclass(frozen=True, slots=True) +class OutcomeEvent: + """One immutable fact about one decision. Never mutated once recorded; a correction + is a *new* `OutcomeEvent` whose `supersedes` names this one's `event_id`.""" + + event_id: str + decision_id: str + event_type: OutcomeEventType + payload: Mapping[str, Any] + recorded_at: int + supersedes: Optional[str] = None + + def __post_init__(self) -> None: + if not self.event_id: + raise ValueError("event_id is required") + if not self.decision_id: + raise ValueError("decision_id is required") + if self.recorded_at < 0: + raise ValueError("recorded_at cannot be negative") + if self.supersedes == self.event_id: + raise ValueError("an event cannot supersede itself") + _validate_payload(self.event_type, self.payload) + + +# --------------------------------------------------------------------------------- ledger + +class OutcomeLedger: + """Append-only store of `OutcomeEvent`s, indexed by `decision_id`. See the module + docstring for what this contract does and does not cover.""" + + def __init__(self) -> None: + self._by_id: dict[str, OutcomeEvent] = {} + self._by_decision: dict[str, list[str]] = {} + + def record(self, event: OutcomeEvent) -> OutcomeEvent: + """Append `event`, or return the existing one unchanged if this `event_id` was + already recorded with identical content (idempotent duplicate delivery). Raises + `DuplicateEventMismatchError` if the same `event_id` is recorded again with + different content -- a duplicate delivery must repeat the same event, not a + different one wearing its id.""" + existing = self._by_id.get(event.event_id) + if existing is not None: + if existing != event: + raise DuplicateEventMismatchError( + f"event id {event.event_id!r} was already recorded with different content" + ) + return existing + if event.supersedes is not None and event.supersedes not in self._by_id: + raise OutcomeEventNotFoundError( + f"event {event.event_id!r} supersedes unknown event {event.supersedes!r}" + ) + self._by_id[event.event_id] = event + self._by_decision.setdefault(event.decision_id, []).append(event.event_id) + return event + + def events_for_decision(self, decision_id: str) -> tuple[OutcomeEvent, ...]: + """The complete, unfolded history for one decision, in recorded order -- every + event ever recorded, including every one a later correction superseded.""" + return tuple(self._by_id[event_id] for event_id in self._by_decision.get(decision_id, ())) + + def view_as_of(self, decision_id: str, at: int) -> tuple[OutcomeEvent, ...]: + """What was believed about one decision at time `at` -- events recorded strictly + before it, with only the corrections that had also been recorded by then. + + `current_view` cannot answer this. It folds every correction ever recorded, so + using it to reconstruct a past belief pulls later knowledge backward through time: + a holdout evaluation built on it would train on evidence that did not exist when + the recommendation was formed, and report a backtest that nobody could have run. + + History is still never discarded. This is a narrower fold over the same immutable + events, not a second store, and `events_for_decision` continues to return + everything. + """ + if at < 0: + raise ValueError("as-of time cannot be negative") + known = tuple( + event for event in self.events_for_decision(decision_id) + if event.recorded_at < at + ) + # A correction only applies if it too was known by `at`; one recorded later leaves + # the event it supersedes standing, because that is what was believed at the time. + superseded = { + event.supersedes for event in known if event.supersedes is not None + } + return tuple(event for event in known if event.event_id not in superseded) + + def current_view(self, decision_id: str) -> tuple[OutcomeEvent, ...]: + """The current, corrected understanding of one decision: every recorded event + for it *except* those a later correction has superseded. History itself is + never discarded -- `events_for_decision` still returns every event -- this only + filters which ones currently apply.""" + superseded = { + event.supersedes + for event in self.events_for_decision(decision_id) + if event.supersedes is not None + } + return tuple( + event for event in self.events_for_decision(decision_id) + if event.event_id not in superseded + ) diff --git a/src/packvium/pareto.py b/src/packvium/pareto.py new file mode 100644 index 0000000..59e89ac --- /dev/null +++ b/src/packvium/pareto.py @@ -0,0 +1,175 @@ +"""A Pareto report over benchmark results, one per profile. + +"A single blended score hides the trade-off" (this task's own description) -- a report +that turns utilisation, runtime and container count into one weighted number can name a +"winner" that is worse on the one axis a caller actually cares about. This module never +computes such a number. A result **dominates** another only if it is no worse on every +named axis and strictly better on at least one; the report per profile is the set of +results nothing else dominates (the Pareto frontier), not a ranking. + +"The report names a winner per profile" is read literally but not force-fitted: when +exactly one result survives as non-dominated, that is the named winner. When more than +one survives -- a genuine trade-off, e.g. one engine faster and another denser, neither +strictly better than the other -- the report says so explicitly (`winner is None`, +`pareto_optimal` lists all of them) rather than breaking the tie with an arbitrary or +blended pick, which would be exactly the single-number collapse this task rejects. + +Scope: this module is pure report generation over caller-supplied metrics -- it does not +itself run competitors or collect their results (still TODO) or filter by +'s validator (a disqualified candidate must be excluded by the caller before it +ever reaches this module; `generate_report` assumes every candidate it receives already +passed validation). +""" + +from __future__ import annotations + +import math +from ._compat import dataclass +from typing import Mapping, Optional, Sequence + + +class ParetoReportError(Exception): + """Base class for every error raised by this module.""" + + +class InconsistentAxesError(ParetoReportError): + """Two candidates being compared do not report the same set of metric axes -- an + axis missing from one side would silently drop out of the dominance comparison + rather than being caught.""" + + +class NonFiniteMetricError(ParetoReportError): + """A metric is `NaN`, which cannot take part in a dominance comparison. + + Dominance is decided by `>` and `<`, and **both are false for `NaN`** -- so an axis + carrying one sets neither `better` nor `worse` and is silently counted as *equal*. + Measured before this refusal existed: a candidate whose metrics were all `NaN` came back + on the Pareto frontier beside a clean one, because nothing could dominate it. Presenting + garbage as an optimal trade-off is the one output this module exists to prevent. + + This is the same judgement `InconsistentAxesError` already makes -- a silent partial + comparison is worse than an explicit error -- applied to an axis that is present but + uncomparable rather than absent. + + **Infinities are allowed, deliberately.** `inf` and `-inf` are ends of the number line, + not holes in it: `inf > 5` is true, `5 > inf` is false, and `inf` against `inf` is + correctly neither better nor worse. A caller encoding "unpriceable" or "timed out" as an + infinite cost gets exactly the dominance answer they mean, so refusing those would break + a reasonable use for the sake of a tidier rule. `NaN` is refused because it is not a + value; infinities are kept because they are. + """ + + +def _refuse_nan(metrics: Mapping[str, float], where: str) -> None: + """Guard for one candidate's metrics. O(axes), which is the size of `higher_is_better`. + + Named `where` rather than positional, because the error a caller sees has to say which + candidate carried the bad value -- an evaluator producing `NaN` is a bug upstream of this + module, and a message that does not point at it just moves the search. + """ + for axis, value in metrics.items(): + if isinstance(value, float) and math.isnan(value): + raise NonFiniteMetricError(f"{where}: metric {axis!r} is NaN and cannot be compared") + + +@dataclass(frozen=True, slots=True) +class CandidateResult: + """One engine's result on one profile, already validated by the caller.""" + + profile: str + engine: str + metrics: Mapping[str, float] + + def __post_init__(self) -> None: + if not self.profile: + raise ValueError("profile is required") + if not self.engine: + raise ValueError("engine is required") + if not self.metrics: + raise ValueError("metrics cannot be empty") + # At construction rather than at comparison: an invalid candidate then cannot exist, + # so `generate_report` and `_pareto_frontier` inherit the guarantee without paying + # for it once per pair. + _refuse_nan(self.metrics, f"{self.engine} on profile {self.profile}") + + +@dataclass(frozen=True, slots=True) +class ProfileReport: + """One profile's Pareto frontier. `winner` is set only when exactly one candidate + survived as non-dominated; a genuine multi-way trade-off leaves it `None` and lists + every surviving engine in `pareto_optimal` instead.""" + + profile: str + pareto_optimal: tuple[str, ...] + dominated: tuple[str, ...] + winner: Optional[str] = None + + def __post_init__(self) -> None: + if not self.profile: + raise ValueError("profile is required") + if self.winner is not None and self.winner not in self.pareto_optimal: + raise ValueError("winner, when set, must be one of the Pareto-optimal engines") + if self.winner is not None and len(self.pareto_optimal) != 1: + raise ValueError("winner can only be set when exactly one candidate is Pareto-optimal") + + +def dominates(a: Mapping[str, float], b: Mapping[str, float], higher_is_better: Mapping[str, bool]) -> bool: + """Whether `a` dominates `b`: no worse than `b` on every axis in `higher_is_better`, + and strictly better on at least one. Raises `InconsistentAxesError` if either side + is missing an axis `higher_is_better` names, and `NonFiniteMetricError` if either + carries a `NaN` -- in both cases because a silent partial comparison is worse than an + explicit error. Infinities are accepted and compare as the ends of the number line.""" + missing = set(higher_is_better) - set(a) | set(higher_is_better) - set(b) + if missing: + raise InconsistentAxesError(f"missing metric axis/axes: {sorted(missing)}") + # `dominates` is public and takes bare mappings, so it cannot rely on `CandidateResult` + # having already validated them. + _refuse_nan(a, "left candidate") + _refuse_nan(b, "right candidate") + at_least_as_good = True + strictly_better_somewhere = False + for axis, higher_wins in higher_is_better.items(): + a_value, b_value = a[axis], b[axis] + better = a_value > b_value if higher_wins else a_value < b_value + worse = a_value < b_value if higher_wins else a_value > b_value + if worse: + at_least_as_good = False + if better: + strictly_better_somewhere = True + return at_least_as_good and strictly_better_somewhere + + +def _pareto_frontier(candidates: Sequence[CandidateResult], higher_is_better: Mapping[str, bool]) -> tuple[tuple[str, ...], tuple[str, ...]]: + optimal: list[str] = [] + dominated: list[str] = [] + for candidate in candidates: + is_dominated = any( + other is not candidate and dominates(other.metrics, candidate.metrics, higher_is_better) + for other in candidates + ) + (dominated if is_dominated else optimal).append(candidate.engine) + return tuple(sorted(optimal)), tuple(sorted(dominated)) + + +def generate_report(candidates: Sequence[CandidateResult], higher_is_better: Mapping[str, bool]) -> tuple[ProfileReport, ...]: + """One `ProfileReport` per distinct `profile` among `candidates`, sorted by profile + name for a deterministic report order. `higher_is_better` must be supplied + explicitly for every metric axis used -- there is no default direction, since + guessing wrong (e.g. treating runtime as "higher is better") would silently invert + the whole report.""" + if not higher_is_better: + raise ValueError("higher_is_better must name at least one metric axis") + by_profile: dict[str, list[CandidateResult]] = {} + for candidate in candidates: + by_profile.setdefault(candidate.profile, []).append(candidate) + + reports = [] + for profile in sorted(by_profile): + group = by_profile[profile] + engine_names = [c.engine for c in group] + if len(set(engine_names)) != len(engine_names): + raise ParetoReportError(f"profile {profile!r} has more than one result for the same engine") + optimal, dominated = _pareto_frontier(group, higher_is_better) + winner = optimal[0] if len(optimal) == 1 else None + reports.append(ProfileReport(profile=profile, pareto_optimal=optimal, dominated=dominated, winner=winner)) + return tuple(reports) diff --git a/src/packvium/recommendations.py b/src/packvium/recommendations.py new file mode 100644 index 0000000..f580fe9 --- /dev/null +++ b/src/packvium/recommendations.py @@ -0,0 +1,219 @@ +"""Domain model for carton-catalog and rule-change recommendations. + +A recommendation proposes a carton or rule change, backed by a `packvium.simulation` +baseline-vs-treatment comparison over a real order cohort: it names exactly +which orders support it, what changed and by how much per metric, how confident that +evidence is, what constraints bound it, and how to undo it if approved. It is never an +automatic mutation of the production catalog -- `propose_recommendation` has no method +that could write to any registry; only a separate, explicit `approve()` call can, and +only through a caller-injected `publish` callback, the same dependency-injection shape +`packvium.simulation`'s `OrderEvaluator` uses. + +"Recommendations with insufficient evidence are suppressed": `propose_recommendation` +returns `None` rather than a `Recommendation` when the paired successful cohort is too +small or too unreliable (its share of all attempted orders falls below an explicit +caller-supplied threshold) -- suppression +is a `None` return, not an exception a caller might mishandle, and not a `Recommendation` +object with a misleadingly low confidence value quietly attached. +""" + +from __future__ import annotations + +from ._compat import dataclass +from typing import Callable, Mapping, Optional, Sequence + +from .commerce.catalog import CatalogRegistry, CatalogSnapshot +from .commerce.policy import ( + PolicyAction, PolicyPredicate, PolicyRegistry, PolicyRule, PolicyScope, +) +from .simulation import MismatchedOrderCorpusError, ScenarioResult, compare_scenarios + + +class RecommendationError(Exception): + """Base class for every recommendation-domain error raised by this module.""" + + +@dataclass(frozen=True, slots=True) +class ExpectedDelta: + """One metric's change from baseline to treatment, averaged over every order both + scenarios actually succeeded on (a failed order contributes to neither side's + average, the same exclusion `packvium.simulation`'s `compare_scenarios` already + applies per order).""" + + metric: str + baseline_mean: float + treatment_mean: float + + def __post_init__(self) -> None: + if not self.metric: + raise ValueError("metric is required") + + @property + def delta(self) -> float: + return self.treatment_mean - self.baseline_mean + + +@dataclass(frozen=True, slots=True) +class Recommendation: + """One reviewable proposal. Never mutates anything by existing -- see the module + docstring.""" + + recommendation_id: str + proposal: str + supporting_order_ids: tuple[str, ...] + expected_deltas: tuple[ExpectedDelta, ...] + confidence: float + constraints: tuple[str, ...] + rollback_plan: str + + def __post_init__(self) -> None: + if not self.recommendation_id: + raise ValueError("recommendation_id is required") + if not self.proposal: + raise ValueError("proposal is required") + if not self.supporting_order_ids: + raise ValueError("a recommendation must cite at least one supporting order") + if not self.expected_deltas: + raise ValueError("a recommendation must report at least one expected delta") + if not 0.0 <= self.confidence <= 1.0: + raise ValueError("confidence must be between 0 and 1") + if not self.constraints or any(not constraint for constraint in self.constraints): + raise ValueError("a recommendation must state at least one non-empty constraint") + if not self.rollback_plan: + raise ValueError("a recommendation must state its rollback plan") + + +@dataclass(frozen=True, slots=True) +class ApprovalRecord: + """The immutable record of one approval: which recommendation, when, and which new + registry version publishing it actually produced.""" + + recommendation_id: str + approved_at: int + published_version: int + + def __post_init__(self) -> None: + if not self.recommendation_id: + raise ValueError("recommendation_id is required") + if self.approved_at < 0: + raise ValueError("approved_at cannot be negative") + if self.published_version <= 0: + raise ValueError("published_version must be positive") + + +def propose_recommendation( + recommendation_id: str, proposal: str, baseline: ScenarioResult, treatment: ScenarioResult, + *, constraints: tuple[str, ...], rollback_plan: str, minimum_cohort_size: int, minimum_confidence: float, +) -> Optional[Recommendation]: + """Build a `Recommendation` from a baseline/treatment comparison, or return `None` + if the evidence behind it is too thin. `minimum_cohort_size` and + `minimum_confidence` are required explicitly -- there is no built-in default, since + what counts as "enough evidence" is a product decision this module does not make on + a caller's behalf. + """ + if baseline.order_ids != treatment.order_ids: + raise MismatchedOrderCorpusError( + "baseline and treatment must have run the identical order corpus, in the same order" + ) + if minimum_cohort_size <= 0: + raise ValueError("minimum_cohort_size must be positive") + if not 0.0 <= minimum_confidence <= 1.0: + raise ValueError("minimum_confidence must be between 0 and 1") + # A delta is a paired comparison, not the difference between two independently + # filtered populations. If the arms fail on different orders, averaging each arm's + # surviving orders separately can manufacture an improvement from cohort mix alone. + # Build each metric from the exact same successful order pairs. This is O(o * k) time + # and O(o * k) retained values for o orders and at most k common metrics per order. + paired: dict[str, tuple[list[float], list[float]]] = {} + supporting_order_ids: list[str] = [] + for baseline_run, treatment_run in zip(baseline.runs, treatment.runs): + if not baseline_run.succeeded or not treatment_run.succeeded: + continue + supporting_order_ids.append(baseline_run.order_id) + for metric in baseline_run.metrics.keys() & treatment_run.metrics.keys(): + baseline_values, treatment_values = paired.setdefault(metric, ([], [])) + baseline_values.append(baseline_run.metrics[metric]) + treatment_values.append(treatment_run.metrics[metric]) + + # Evidence size is the paired cohort, not the number of orders attempted in either + # arm. Otherwise two mostly-disjoint survivor sets could advertise a large cohort and + # high confidence while a delta rested on one shared order. + comparable_count = len(supporting_order_ids) + confidence = comparable_count / len(baseline.order_ids) + if comparable_count < minimum_cohort_size or confidence < minimum_confidence: + return None + + deltas = [] + for metric in sorted(paired): + baseline_values, treatment_values = paired[metric] + deltas.append(ExpectedDelta( + metric=metric, + baseline_mean=sum(baseline_values) / len(baseline_values), + treatment_mean=sum(treatment_values) / len(treatment_values), + )) + if not deltas: + return None # no metric both sides actually produced -- nothing to recommend from + + return Recommendation( + recommendation_id=recommendation_id, + proposal=proposal, + supporting_order_ids=tuple(supporting_order_ids), + expected_deltas=tuple(deltas), + confidence=confidence, + constraints=constraints, + rollback_plan=rollback_plan, + ) + + +def approve(recommendation: Recommendation, *, at: int, publish: Callable[[], int]) -> ApprovalRecord: + """The only path from a `Recommendation` to an actual registry change: calls the + caller-injected `publish` exactly once and records the version it returns. Nothing + in `propose_recommendation` above can reach this -- a recommendation existing is + never itself a mutation.""" + published_version = publish() + return ApprovalRecord( + recommendation_id=recommendation.recommendation_id, approved_at=at, published_version=published_version, + ) + + +def approve_catalog( + recommendation: Recommendation, + registry: CatalogRegistry, + snapshot: CatalogSnapshot, + *, + approved_at: int, + effective_at: int, +) -> ApprovalRecord: + """Publish an approved catalog proposal through the real append-only registry.""" + version = registry.publish( + snapshot, + published_at=approved_at, + effective_at=effective_at, + note=f"approved recommendation {recommendation.recommendation_id}", + ) + return ApprovalRecord(recommendation.recommendation_id, approved_at, version.number) + + +def approve_policy( + recommendation: Recommendation, + registry: PolicyRegistry, + *, + rule_id: str, + scope: PolicyScope, + action: PolicyAction, + predicates: Sequence[PolicyPredicate], + priority: int, + approved_at: int, + effective_at: int, +) -> ApprovalRecord: + """Publish an approved policy proposal through the real append-only registry.""" + rule: PolicyRule = registry.publish( + rule_id, + scope=scope, + action=action, + predicates=predicates, + priority=priority, + effective_at=effective_at, + reason=f"approved recommendation {recommendation.recommendation_id}", + ) + return ApprovalRecord(recommendation.recommendation_id, approved_at, rule.version) diff --git a/src/packvium/serialization.py b/src/packvium/serialization.py index 657dccb..2c2aeda 100644 --- a/src/packvium/serialization.py +++ b/src/packvium/serialization.py @@ -198,7 +198,8 @@ def reject_unsupported( ) -def pack_from_dict(data: dict) -> dict: +def pack_from_dict(data: dict, *, + extensions: ExtensionRegistry | None = None) -> dict: reject_unsupported(data) unit = data.get("units", {}).get("length", "mm"); cfg = data.get("configuration", {}) profile = SolverProfile(cfg.get("solver_profile", "balanced")) @@ -224,8 +225,18 @@ def pack_from_dict(data: dict) -> dict: # Rules compile into this engine's own constraint pipeline rather than post-filtering # a chosen answer: an illegal candidate is rejected during search, so the packing that # wins was never allowed to be illegal in the first place. + # A caller's extensions are *added to* the compiled policy rules, never substituted for + # them. Replacing would let an operator lock silently drop a policy rule the + # request asked for, which is the one thing a lock must not be able to do. + supplied = extensions or ExtensionRegistry() extensions = ExtensionRegistry( - placement_constraints=PolicyRuleSet.from_dict(data.get("policy")).constraints() + placement_constraints=( + *PolicyRuleSet.from_dict(data.get("policy")).constraints(), + *supplied.placement_constraints, + ), + item_order_strategies=supplied.item_order_strategies, + solvers=supplied.solvers, + container_selector=supplied.container_selector, ) result = Packer(config, extensions).pack([_item(i, unit) for i in data["items"]], [_container(c, unit) for c in data["containers"]]) result = replace(result, catalog_versions_used=references) diff --git a/src/packvium/simulation.py b/src/packvium/simulation.py new file mode 100644 index 0000000..eab765e --- /dev/null +++ b/src/packvium/simulation.py @@ -0,0 +1,187 @@ +"""Domain model for scenario simulation and what-if comparison. + +A scenario runs a fixed order corpus against one *version pin* -- an explicit, +recorded set of catalog/tariff/policy versions (the same append-only version numbers +`packvium.commerce.catalog`, `packvium.commerce.rating` and `packvium.commerce.policy` +already hand out) -- and records what happened, order by order, without mutating +anything those registries hold. Two scenarios (baseline and treatment) compare only when +they ran the *identical* order corpus; the comparison itself is a Pareto report +(`packvium.pareto`), never a single blended number, for the same reason that module +rejects one. + +This module deliberately does not itself price, validate, or pack anything: `run_scenario` +takes an injected `OrderEvaluator` callable -- the caller supplies a function that +actually calls the catalog/rating/policy registries and the solver, keeping simulation +orchestration decoupled from what it orchestrates. Given a deterministic evaluator, a +scenario is byte-for-byte reproducible purely from its own recorded `ScenarioVersionPin` +and `order_ids` -- proven directly by re-running the same evaluator twice, not assumed. +""" + +from __future__ import annotations + +from ._compat import dataclass +from math import isfinite +from types import MappingProxyType +from typing import Any, Callable, Mapping, Optional, Sequence + +from .pareto import CandidateResult, ProfileReport, generate_report + + +class ScenarioError(Exception): + """Base class for every scenario-domain error raised by this module.""" + + +class MismatchedOrderCorpusError(ScenarioError): + """Baseline and treatment did not run the identical order corpus -- comparing them + would not isolate the effect of the version pin, so this is refused rather than + silently compared anyway.""" + + +@dataclass(frozen=True, slots=True) +class ScenarioVersionPin: + """The exact, recorded configuration one scenario ran against. Every field is an + explicit version number (or `None` if that registry was not consulted) -- never + "current", so a stored `ScenarioResult` can be replayed byte-for-byte regardless of + what any registry's history has grown to since (the same reproducibility guarantee + `commerce/rating/model.py`'s `rate_with_version` gives one rate lookup).""" + + catalog_version: Optional[int] = None + tariff_version: Optional[int] = None + policy_version: Optional[int] = None + policy_versions: tuple[tuple[str, int], ...] = () + solver_version: Optional[str] = None + + def __post_init__(self) -> None: + if all(v is None for v in (self.catalog_version, self.tariff_version, self.policy_version, self.solver_version)) \ + and not self.policy_versions: + raise ValueError("a version pin must set at least one of its fields") + numeric = (self.catalog_version, self.tariff_version, self.policy_version) + if any(value is not None and value <= 0 for value in numeric): + raise ValueError("version numbers must be positive") + if any(not rule_id or version <= 0 for rule_id, version in self.policy_versions): + raise ValueError("policy version pins require a rule id and positive version") + if len({rule_id for rule_id, _ in self.policy_versions}) != len(self.policy_versions): + raise ValueError("a version pin cannot name the same policy rule twice") + object.__setattr__(self, "policy_versions", tuple(sorted(self.policy_versions))) + if self.solver_version is not None and not self.solver_version: + raise ValueError("solver_version must be non-empty when supplied") + + +@dataclass(frozen=True, slots=True) +class OrderRunResult: + """One order's outcome under one version pin. `metrics` are Pareto axes (material + cost, shipping cost, utilisation, a damage-risk proxy, runtime, ...) supplied by the + caller's evaluator -- this module never computes or invents them.""" + + order_id: str + succeeded: bool + metrics: Mapping[str, float] = () + failure_reason: Optional[str] = None + artifact: Any = None + + def __post_init__(self) -> None: + if not self.order_id: + raise ValueError("order_id is required") + if self.succeeded and not self.metrics: + raise ValueError("a succeeded run must report at least one metric") + if not self.succeeded and not self.failure_reason: + raise ValueError("a failed run must record a failure_reason") + metrics = dict(self.metrics) + if any(not name for name in metrics): + raise ValueError("metric names must be non-empty") + if any( + isinstance(value, bool) + or not isinstance(value, (int, float)) + or not isfinite(value) + for value in metrics.values() + ): + raise ValueError("metric values must be finite") + object.__setattr__(self, "metrics", MappingProxyType(metrics)) + + +#: `(order_id, version_pin) -> OrderRunResult`. Supplied by the caller -- keeps pricing, +#: policy evaluation and packing out of this orchestration layer. +OrderEvaluator = Callable[[str, ScenarioVersionPin], OrderRunResult] + + +@dataclass(frozen=True, slots=True) +class ScenarioResult: + """The complete, immutable record of one scenario run: which version pin, which + orders (the corpus itself, recorded -- not just its length), and every order's raw + outcome, retained in full (acceptance: "raw artifacts are retained").""" + + scenario_id: str + version_pin: ScenarioVersionPin + order_ids: tuple[str, ...] + runs: tuple[OrderRunResult, ...] + + def __post_init__(self) -> None: + object.__setattr__(self, "order_ids", tuple(self.order_ids)) + object.__setattr__(self, "runs", tuple(self.runs)) + if not self.scenario_id: + raise ValueError("scenario_id is required") + if not self.order_ids: + raise ValueError("order_ids cannot be empty") + if tuple(run.order_id for run in self.runs) != self.order_ids: + raise ValueError("runs must correspond 1:1 to order_ids, in the same order") + + @property + def succeeded_count(self) -> int: + return sum(1 for run in self.runs if run.succeeded) + + @property + def failed_count(self) -> int: + return sum(1 for run in self.runs if not run.succeeded) + + @property + def confidence(self) -> float: + """The fraction of orders that produced a usable outcome -- 's + "confidence and invalid-run counts are visible", made a first-class, + always-computed property rather than something a caller must derive.""" + return self.succeeded_count / len(self.runs) + + def raw_artifact(self, order_id: str) -> OrderRunResult: + for run in self.runs: + if run.order_id == order_id: + return run + raise ScenarioError(f"no run recorded for order {order_id!r}") + + +def run_scenario( + scenario_id: str, version_pin: ScenarioVersionPin, order_ids: Sequence[str], evaluator: OrderEvaluator, +) -> ScenarioResult: + """Run every order in `order_ids` through `evaluator` under `version_pin`, in + order. Given a deterministic `evaluator`, calling this twice with the same + arguments reproduces a byte-identical `ScenarioResult` -- 's "a scenario is + fully reproducible from stored versions" is this function's own determinism, not a + separate mechanism bolted on afterward.""" + runs = tuple(evaluator(order_id, version_pin) for order_id in order_ids) + return ScenarioResult(scenario_id=scenario_id, version_pin=version_pin, order_ids=tuple(order_ids), runs=runs) + + +def compare_scenarios( + baseline: ScenarioResult, treatment: ScenarioResult, higher_is_better: Mapping[str, bool], +) -> tuple[ProfileReport, ...]: + """Compare two scenarios order by order via 's Pareto report -- never a + single blended delta. Each order becomes its own "profile" (so a caller sees exactly + which orders improved, regressed, or traded off, not just an aggregate), and + "baseline"/"treatment" are the two "engines" compared within it. Only orders both + scenarios actually succeeded on are compared; a failed run has no metrics to compare + with and is excluded from that order's report rather than crashing the comparison. + """ + if baseline.order_ids != treatment.order_ids: + raise MismatchedOrderCorpusError( + "baseline and treatment must have run the identical order corpus, in the same order" + ) + candidates: list[CandidateResult] = [] + # ScenarioResult already proves that runs correspond 1:1 to order_ids in order, and + # the corpus equality check above proves both arms have the same order. Pair them in + # one pass instead of calling raw_artifact twice per order (which made this O(o^2)). + # The orchestration outside generate_report is now O(o) time and O(o) candidates. + for base_run, treat_run in zip(baseline.runs, treatment.runs): + order_id = base_run.order_id + if base_run.succeeded: + candidates.append(CandidateResult(profile=order_id, engine="baseline", metrics=base_run.metrics)) + if treat_run.succeeded: + candidates.append(CandidateResult(profile=order_id, engine="treatment", metrics=treat_run.metrics)) + return generate_report(candidates, higher_is_better) diff --git a/src/packvium/solvers.py b/src/packvium/solvers.py index bf9a2cd..7ddd569 100644 --- a/src/packvium/solvers.py +++ b/src/packvium/solvers.py @@ -6,7 +6,7 @@ from ._compat import dataclass from time import monotonic_ns -from typing import Callable, Iterable, Protocol, Sequence +from typing import Callable, Iterable, Protocol, Sequence, cast from .axle_load import axle_balanced_origins from .config import PackingConfig @@ -575,6 +575,8 @@ def find_candidates(state: ContainerState, item: ItemInstance, config: PackingCo ordered = sorted(points, key=lambda p: (p.z, p.y, p.x)) candidates: list[Candidate] = [] best: Candidate | None = None + retained: list[tuple[tuple[int, ...], int, Candidate]] = [] + bounded = max_candidates is not None and max_candidates > 1 for point in ordered: deadline.check() stats.candidate_points_considered += 1 @@ -625,10 +627,11 @@ def find_candidates(state: ContainerState, item: ItemInstance, config: PackingCo rejected = True break if rejected: continue - position = (tentative.position if tentative is not None - else Point(x1 + clearance, y1 + clearance, z1 + clearance)) - candidate = Candidate(point, position, rotation, physical, envelope, _candidate_score(state, point, envelope)) + score = _candidate_score(state, point, envelope) + position = tentative.position if tentative is not None else None if reserve_check: + if position is None: + position = Point(x1 + clearance, y1 + clearance, z1 + clearance) reserve_placement = tentative or Placement( item, position, rotation, physical, point, envelope ) @@ -652,20 +655,31 @@ def find_candidates(state: ContainerState, item: ItemInstance, config: PackingCo continue stats.candidates_evaluated += 1 if tracing: - trace.emit({"type": "score", "item_id": item.id, "point": {"x": x1, "y": y1, "z": z1}, "rotation": rotation.value, "score": list(candidate.score)}) + trace.emit({"type": "score", "item_id": item.id, "point": {"x": x1, "y": y1, "z": z1}, "rotation": rotation.value, "score": list(score)}) + # Keep every check, counter and trace event, but materialize only candidates + # that survive selection. Equal scores keep the earlier enumeration entry. + if max_candidates == 1 and best is not None and score >= best.score: + continue + if bounded and len(retained) == max_candidates and score >= retained[0][2].score: + continue + if position is None: + position = Point(x1 + clearance, y1 + clearance, z1 + clearance) + candidate = Candidate(point, position, rotation, physical, envelope, score) if max_candidates == 1: - if best is None or candidate.score < best.score: best = candidate + best = candidate + elif bounded: + # Negate the full integer key and ordinal so heapq's root is the worst + # retained entry, including the latest entry in a stable-sort tie. + entry = (tuple(-value for value in score), -stats.candidates_evaluated, candidate) + if len(retained) < max_candidates: + heapq.heappush(retained, entry) + else: + heapq.heapreplace(retained, entry) else: candidates.append(candidate) if max_candidates == 1: return [] if best is None else [best] - if max_candidates is not None and len(candidates) > max_candidates: - # Top-k selection avoids an O(f log f) full sort when the beam only consumes - # k candidates. Enumerated insertion order is the deterministic tie-break. - selected = heapq.nsmallest( - max_candidates, - enumerate(candidates), - key=lambda entry: (entry[1].score, entry[0]), - ) - return [candidate for _, candidate in selected] + if bounded: + return [entry[2] for entry in sorted(retained, reverse=True)] + if max_candidates is not None: return [] candidates.sort(key=lambda c: c.score) return candidates @@ -690,17 +704,28 @@ def group_batches(items: Sequence[ItemInstance]) -> list[tuple[ItemInstance, ... A batch that does not fit is rejected as a whole and leaves the rest of the order untouched -- an impossible group must never strand unrelated items. """ - batches: list[tuple[ItemInstance, ...]] = [] - seen: set[str] = set() + batches: list[tuple[ItemInstance, ...] | list[ItemInstance]] = [] + groups: dict[str, list[ItemInstance]] = {} for item in items: group = item.item.group if group is None: batches.append((item,)) continue - if group in seen: continue - seen.add(group) - batches.append(tuple(other for other in items if other.item.group == group)) - return batches + members = groups.get(group) + if members is None: + members = [item] + groups[group] = members + batches.append(members) + else: + members.append(item) + # Output slots follow first appearance, and members follow input order. Freeze + # grouped buckets in place, releasing the lookup before allocating the tuples. + if groups: + groups.clear() + for position, batch in enumerate(batches): + if isinstance(batch, list): + batches[position] = tuple(batch) + return cast(list[tuple[ItemInstance, ...]], batches) def _place_batch(state: ContainerState, batch: Sequence[ItemInstance], config: PackingConfig, constraints: Sequence[PlacementConstraint], stats: SearchStats, deadline: Deadline, width: int | None): @@ -995,7 +1020,18 @@ class GridSolver: order_insensitive = True def supports(self, items: Sequence[ItemInstance]) -> bool: - return bool(items) and len({_lattice_profile(i.item) for i in items}) == 1 + if not items: return False + prototype = items[0].item + profile = _lattice_profile(prototype) + previous = prototype + for instance in items: + item = instance.item + # Quantities reuse an immutable Item; only a different object can bring + # a different profile. Keep one profile instead of materializing a set. + if item is not prototype and item is not previous and _lattice_profile(item) != profile: + return False + previous = item + return True def pack_one(self, container, sequence, items, config, stats, deadline): if container.obstacles or not items or not self.supports(items): diff --git a/tests/test_constraints.py b/tests/test_constraints.py index 90fa1a3..ce54147 100644 --- a/tests/test_constraints.py +++ b/tests/test_constraints.py @@ -659,6 +659,24 @@ def test_appending_a_unit_matches_a_rebuild_across_scene_shapes(coordinates, ext for unit in units[1:]: graph = graph.with_unit(unit, cell_hint=hint) assert _edges(graph, count) == _edges(LoadSupportGraph(units), count) + expected_order = tuple(sorted( + range(count), key=lambda i: (-units[i].box.z2, -units[i].box.origin.z, i) + )) + assert graph._descending_indices() == expected_order + + +def test_candidate_load_order_preserves_height_ties_and_sibling_independence(): + units = [unit(0, 0, 9, 1, 1, 1), unit(10, 0, 0, 1, 1, 10), unit(20, 0, 8, 1, 1, 2)] + base = LoadSupportGraph(units) + assert base._descending_indices() == (0, 2, 1) + above = base.with_unit(unit(30, 0, 10, 1, 1, 1)) + below = base.with_unit(unit(30, 0, 0, 1, 1, 1)) + tied = base.with_unit(unit(30, 0, 8, 1, 1, 2)) + assert above._descending_indices() == (3, 0, 2, 1) + assert below._descending_indices() == (0, 2, 1, 3) + assert tied._descending_indices() == (0, 2, 3, 1) + assert base._descending_indices() == (0, 2, 1) + assert LoadSupportGraph([])._descending_indices() == () def test_a_nesting_unit_is_met_with_a_full_rebuild(monkeypatch): @@ -679,6 +697,7 @@ def test_a_nesting_unit_is_met_with_a_full_rebuild(monkeypatch): assert builds == [1, 2, 3] assert _edges(graph, 3) == _edges(LoadSupportGraph((*stack, arriving)), 3) + assert graph._descending_indices() == (2, 1, 0) # --------------------------------------------------------- stacked-item counting diff --git a/tests/test_execution_plan.py b/tests/test_execution_plan.py new file mode 100644 index 0000000..0de5bda --- /dev/null +++ b/tests/test_execution_plan.py @@ -0,0 +1,396 @@ +"""The execution-plan adapter. + +`docs/EXECUTION-PLAN.md` is the contract. What is checked here is what the design says the +adapter must not do, because those are the properties that decay quietly: + + * it must not invent a step order when it was not given one; + * it must not reference a placement by an identifier the cross-language projection drops; + * it must not blend a score vector into one number, or claim a cause the solver never + recorded; + * presentation text must cite the authoritative fields it came from. + +The alternatives assertions run on a constructed result, and the reason is not that +nothing produces them: all four engines do, and 165 of the 399 fixtures carry a non-empty +list on a live solve. It is that `conformance/golden/` stores the *projection*, which has no +`alternatives` key at all -- so the corpus this suite is built around cannot exercise these +rules, however many of its requests would produce one. `TestARealEngineProducesAlternatives` +below solves for real instead. +""" + +from __future__ import annotations + +import pytest + +from packvium.execution import ( + FORMAT, + ExecutionPlanError, + build_execution_plan, + canonical_plan_json, + placement_reference, +) + + +def _placement(item_type: str, x: int, y: int, z: int, orientation: str = "LWH") -> dict: + def scalar(ticks: int) -> dict: + # Both spellings, as a real result carries them: `value` is a rendering and + # `ticks` is the number. A test that supplied only one could not catch a reference + # built on the wrong one. + return {"ticks": ticks, "value": str(ticks // 16000), "unit": "mm"} + return { + "item_id": f"{item_type}#{x}{y}{z}", + "item_type": item_type, + "orientation": orientation, + "position": {"x": scalar(x), "y": scalar(y), "z": scalar(z)}, + "dimensions": {"length": scalar(1600000), "width": scalar(1600000), + "height": scalar(1600000)}, + "support_ratio": 1.0, + "top_load": 0, + } + + +def _result(**overrides) -> dict: + base = { + "status": "feasible", + "objective": "default", + "score": [0, 1, 0, 0, 1000000], + "feasibility": {"code": "feasible"}, + "optimality": {"code": "not_proven"}, + "containers": [{ + "id": "box#1", + "container_type": "box", + "volume_utilization": 0.5, + "placements": [_placement("cube", 0, 0, 0), _placement("cube", 1600000, 0, 0)], + }], + "unpacked_items": [], + "alternatives": [], + } + base.update(overrides) + return base + + +class TestTheStepOrderIsNeverInvented: + def test_without_an_injected_order_the_plan_says_so(self): + plan = build_execution_plan({}, _result()) + container = plan["containers"][0] + assert container["order"] == "unavailable" + # Every placement is still listed, so nothing is hidden -- but no step numbers are + # attached, because array position is an artifact of candidate iteration. + assert len(container["steps"]) == 2 + assert all("sequence" not in step for step in container["steps"]) + + def test_an_injected_order_is_used_verbatim(self): + plan = build_execution_plan({}, _result(), loading_orders={0: [1, 0]}) + container = plan["containers"][0] + assert container["order"] == "loading" + assert [step["sequence"] for step in container["steps"]] == [1, 2] + # Step 1 is the placement the caller put first, not the one the result listed first. + assert container["steps"][0]["placement"]["position_ticks"]["x"] == 1600000 + + def test_an_order_that_is_not_a_permutation_is_refused(self): + with pytest.raises(ExecutionPlanError, match="permutation"): + build_execution_plan({}, _result(), loading_orders={0: [0, 0]}) + + +class TestThePlacementReferenceIsCrossLanguageSafe: + def test_it_does_not_use_item_id(self): + """`item_id` exists and is dropped by conformance/canonical.py as an instance count. + + Referencing it would make a plan that two correct engines disagree about. + """ + reference = placement_reference(0, _placement("cube", 0, 0, 0)) + assert "item_id" not in reference + assert set(reference) == {"container_index", "item_type", "orientation", "position_ticks"} + + def test_it_reads_ticks_and_not_the_rendered_value(self): + placement = _placement("cube", 12345, 0, 0) + placement["position"]["x"]["value"] = "wrong" + assert placement_reference(0, placement)["position_ticks"]["x"] == 12345 + + def test_it_refuses_a_placement_it_cannot_reference(self): + placement = _placement("cube", 0, 0, 0) + del placement["orientation"] + with pytest.raises(ExecutionPlanError, match="missing a field"): + placement_reference(0, placement) + + +class TestAnAlternativeIsExplainedWithoutInventingOne: + """Constructed results: the golden corpus has no ranked runners-up to use.""" + + def test_the_loss_is_the_first_differing_index_and_never_a_blend(self): + loser = {"status": "feasible", "score": [0, 2, 0, 0, 900000]} + plan = build_execution_plan({}, _result(alternatives=[loser])) + facts = plan["alternatives"][0]["facts"] + assert facts["first_difference"] == { + "index": 1, "winner": 1, "alternative": 2, "difference": 1, + } + # Nothing anywhere sums or weights the vector. + assert "total" not in facts and "weighted" not in facts + + def test_the_sentence_names_an_axis_and_claims_no_cause(self): + loser = {"status": "feasible", "score": [0, 2, 0, 0, 900000]} + plan = build_execution_plan({}, _result(alternatives=[loser])) + presentation = plan["alternatives"][0]["presentation"] + assert "axis 1" in presentation["summary"] + assert presentation["cites"] == ["score", "alternatives[].score"] + for causal in ("because", "due to", "caused"): + assert causal not in presentation["summary"].lower() + + def test_an_identical_score_is_reported_as_undetermined(self): + twin = {"status": "feasible", "score": [0, 1, 0, 0, 1000000]} + plan = build_execution_plan({}, _result(alternatives=[twin])) + assert plan["alternatives"][0]["facts"]["first_difference"] is None + assert "does not record why" in plan["alternatives"][0]["presentation"]["summary"] + + def test_score_vectors_of_different_length_are_refused(self): + with pytest.raises(ExecutionPlanError, match="different length"): + build_execution_plan({}, _result(alternatives=[{"score": [0, 1]}])) + + def test_a_result_with_no_alternatives_is_well_formed(self): + # The common case, and today the only one. + plan = build_execution_plan({}, _result()) + assert plan["alternatives"] == [] + assert plan["format"] == FORMAT + + +class TestARealEngineProducesAlternatives: + """Evidence for the claim the constructed tests above stand in for. + + Worth a real solve rather than another fixture, because the golden corpus stores the + projection and cannot answer a question about this field -- reading it is what produced + two wrong claims about `alternatives` in a row. + + Live, 165 of 399 fixtures carry a non-empty list. The empty ones have four causes, and + the ones this class pins are the two a reader is most likely to misread: a request the + grid lattice completes has no runners-up, and `configuration.alternatives` + counts the winner, so the schema's own minimum of `1` returns nothing. + + The last two tests are 's decision made enforceable. No part of this field's + content is in the cross-language contract, but three invariants hold of every engine + independently, and two of them are checkable here. + """ + + @staticmethod + def _pack(items): + from packvium import pack_from_dict + + return pack_from_dict({ + "configuration": {"solver_profile": "quality", "alternatives": 4, + "time_limit_ms": 30000}, + "items": items, + "containers": [{"id": "box", + "inner_dimensions": {"length": "400", "width": "300", + "height": "300"}}], + }) + + def test_a_lattice_the_grid_completes_has_no_runners_up(self): + tiling = [{"id": "cube", "quantity": 8, + "dimensions": {"length": "100", "width": "100", "height": "100"}}] + assert self._pack(tiling).get("alternatives") == [] + + def test_a_request_the_lattice_cannot_complete_ranks_several(self): + mixed = [ + {"id": "a", "quantity": 3, + "dimensions": {"length": "170", "width": "110", "height": "90"}}, + {"id": "b", "quantity": 4, + "dimensions": {"length": "130", "width": "70", "height": "50"}}, + {"id": "c", "quantity": 2, + "dimensions": {"length": "90", "width": "90", "height": "210"}}, + ] + result = self._pack(mixed) + alternatives = result.get("alternatives") or [] + assert alternatives, "the engine ranked no runners-up on a mixed-size request" + # Each is a full result with its own integer score vector -- which is what the + # adapter's ranking rules read, and why they are implementable at all. + for alternative in alternatives: + assert alternative["score"] and all( + isinstance(value, int) for value in alternative["score"]) + + def test_no_alternative_outranks_the_winner(self): + """'s second invariant. If an alternative scored better it would *be* the + winner, so a violation means the ranking and the reported result disagree.""" + mixed = [ + {"id": "a", "quantity": 3, + "dimensions": {"length": "170", "width": "110", "height": "90"}}, + {"id": "b", "quantity": 4, + "dimensions": {"length": "130", "width": "70", "height": "50"}}, + ] + result = self._pack(mixed) + assert result.get("alternatives"), "no runners-up to check the invariant against" + for alternative in result["alternatives"]: + assert alternative["score"] >= result["score"], alternative["score"] + + def test_the_configured_count_includes_the_winner(self): + """The caller-visible surprise found, pinned so it cannot drift. + + `configuration.alternatives` is the size of the ranked set the portfolio keeps, not + the number of runners-up returned. The schema types it `minimum: 1`, and `1` yields + an empty list -- so a request asking for alternatives can be named for them and + still never carry one. Documented in `docs/PUBLIC-API.md`; changing the meaning + would be a contract break, so the documentation is the fix. + """ + from packvium import pack_from_dict + + mixed = [ + {"id": "a", "quantity": 3, + "dimensions": {"length": "170", "width": "110", "height": "90"}}, + {"id": "b", "quantity": 4, + "dimensions": {"length": "130", "width": "70", "height": "50"}}, + ] + + def pack(cap): + return pack_from_dict({ + "configuration": {"solver_profile": "quality", "alternatives": cap, + "time_limit_ms": 30000}, + "items": mixed, + "containers": [{"id": "box", "inner_dimensions": { + "length": "400", "width": "300", "height": "300"}}], + }) + + assert pack(1).get("alternatives") == [] + for cap in (2, 3, 4): + assert len(pack(cap).get("alternatives") or []) <= cap - 1 + + def test_the_adapter_explains_a_real_alternative(self): + """The ranking rules against engine output rather than a constructed dict.""" + mixed = [ + {"id": "a", "quantity": 3, + "dimensions": {"length": "170", "width": "110", "height": "90"}}, + {"id": "b", "quantity": 4, + "dimensions": {"length": "130", "width": "70", "height": "50"}}, + {"id": "c", "quantity": 2, + "dimensions": {"length": "90", "width": "90", "height": "210"}}, + ] + result = self._pack(mixed) + plan = build_execution_plan({}, result) + assert plan["alternatives"], "no alternative reached the plan" + for entry in plan["alternatives"]: + difference = entry["facts"]["first_difference"] + # Either they differ at some index, or they tie and the plan says the score + # does not record why one was taken. There is no third answer. + assert difference is None or difference["index"] >= 0 + assert entry["presentation"]["cites"] + + +class TestFactsAndPresentationStaySeparate: + def test_an_unpacked_item_keeps_its_proof_level_unsoftened(self): + result = _result(unpacked_items=[{ + "item_id": "ladder#1", "item_type": "ladder", "reason": "no_container_fits", + "details": ["longest dimension exceeds every container"], + "proof": {"level": "observed", "observations": [{"code": "too_long"}]}, + }]) + entry = build_execution_plan({}, result)["unplaced"][0] + assert entry["facts"]["proof_level"] == "observed" + # The level appears in the sentence too: a reader must not be told "cannot fit" + # when the engine only observed that it did not. + assert "observed" in entry["presentation"]["summary"] + assert entry["presentation"]["cites"] == [ + "unpacked_items[].reason", "unpacked_items[].proof.level", + ] + + def test_every_presentation_block_cites_at_least_one_field(self): + result = _result( + alternatives=[{"status": "feasible", "score": [0, 2, 0, 0, 900000]}], + unpacked_items=[{ + "item_id": "x#1", "item_type": "x", "reason": "no_container_fits", + "details": [], "proof": {"level": "proven", "observations": [{"code": "c"}]}, + }], + ) + plan = build_execution_plan({}, result) + blocks = [entry["presentation"] for entry in plan["alternatives"] + plan["unplaced"]] + assert blocks, "the fixture must produce at least one presentation block" + for block in blocks: + assert block["cites"], block + + +class TestThePlanIsDeterministicAndByteComparable: + def test_the_same_inputs_produce_the_same_bytes(self): + first = canonical_plan_json(build_execution_plan({}, _result())) + second = canonical_plan_json(build_execution_plan({}, _result())) + assert first == second + # Sorted keys and no incidental whitespace, so PHP can be diffed against this. + assert first.startswith('{"alternatives":') and ", " not in first + + def test_a_result_without_a_status_is_not_a_validated_result(self): + with pytest.raises(ExecutionPlanError, match="validated result"): + build_execution_plan({}, {"containers": []}) + + +class TestPythonAndPhpAgreeToTheByte: + """'s actual bar, under test rather than observed once. + + A committed expected string would only prove PHP still agrees with a string. Running + both adapters over the same result is what proves the two implementations agree with + each other, which is the claim being made. + """ + + @staticmethod + def _php_plan(result: dict, orders: dict | None = None) -> str: + import json + import shutil + import subprocess + from pathlib import Path + + if shutil.which("php") is None: + pytest.skip("php is not available in this environment") + root = Path(__file__).resolve().parents[2] + # The PHP engine sits beside this package in the workspace; a published copy does + # not carry it, and the cross-language equality is proven where both exist. + if not (root / "packvium-php" / "autoload.php").is_file(): + pytest.skip("the PHP engine is not part of this package") + script = ( + 'require $argv[1] . "/autoload.php";' + '$r = json_decode($argv[2], true, 512, JSON_THROW_ON_ERROR);' + '$o = json_decode($argv[3], true, 512, JSON_THROW_ON_ERROR);' + 'echo Packvium\\Execution\\Plan::canonicalJson(' + 'Packvium\\Execution\\Plan::build([], $r, $o));' + ) + finished = subprocess.run( + ["php", "-r", script, str(root / "packvium-php"), + json.dumps(result), json.dumps({str(k): v for k, v in (orders or {}).items()})], + capture_output=True, text=True, check=False, + ) + assert finished.returncode == 0, finished.stderr[-2000:] + return finished.stdout + + def test_a_plain_result_agrees(self): + plan = canonical_plan_json(build_execution_plan({}, _result())) + assert plan == self._php_plan(_result()) + + def test_floats_unpacked_items_and_alternatives_agree(self): + """The case most likely to diverge: a repeating float, a proof level and a ranking. + + `volume_utilization` is a double, and two languages rendering it differently is + the classic way a byte-identity claim quietly stops being true. + """ + result = _result( + containers=[{ + "id": "box#1", "container_type": "box", + "volume_utilization": 0.3333333333333333, + "placements": [_placement("crate", 0, 0, 0)], + }], + unpacked_items=[{ + "item_id": "ladder#1", "item_type": "ladder", "reason": "no_container_fits", + "details": ["longest dimension exceeds every container"], + "proof": {"level": "observed", "observations": [{"code": "too_long"}]}, + }], + alternatives=[{"status": "feasible", "score": [0, 3, 0, 250000, 900000]}], + ) + plan = canonical_plan_json(build_execution_plan({}, result, loading_orders={0: [0]})) + assert plan == self._php_plan(result, {0: [0]}) + + +class TestTheAdapterCannotBecomeABypass: + def test_it_imports_no_solver_and_no_validator(self): + """The dependency direction from docs/EXECUTION-PLAN.md, as the cheapest test of it. + + The adapter has no way to produce a placement, because it has no solver, and no way + to bless one, because it has no validator. It can only describe what a validated + result already says. + """ + from pathlib import Path + + source = (Path(__file__).resolve().parents[1] + / "src/packvium/execution.py").read_text() + for forbidden in ("from .solvers", "from .packer", "from .validation", + "import solvers", "import packer", "import validation"): + assert forbidden not in source, forbidden diff --git a/tests/test_holdout.py b/tests/test_holdout.py new file mode 100644 index 0000000..7c87afb --- /dev/null +++ b/tests/test_holdout.py @@ -0,0 +1,243 @@ +"""Replaying a recommendation against history it has not seen. + +`tests/` ships inside the wheel, so these import `packvium.holdout` the way a consumer +does rather than through the workspace shim. + +Two rules are the reason this module exists rather than a spreadsheet. A packing the +injected validator rejects is that arm's failure **at any cost** — its metrics never reach +the comparison. And a recommendation may not be scored on the evidence that produced it, +which is refused rather than warned about, because a holdout score is worth exactly as +much as that separation. +""" + +from __future__ import annotations + +import pytest + +from packvium.holdout import ( + IMPROVED, + REGRESSED, + TRADED_OFF, + UNPACKABLE, + HoldoutError, + OrderEvaluationArtifact, + SupportingEvidenceInHoldoutError, + ValidationVerdict, + evaluate_on_history, +) +from packvium.outcomes import OutcomeEvent, OutcomeEventType, OutcomeLedger +from packvium.recommendations import ExpectedDelta, Recommendation +from packvium.simulation import OrderRunResult, ScenarioVersionPin + +BASELINE_PIN = ScenarioVersionPin(catalog_version=1) +TREATMENT_PIN = ScenarioVersionPin(catalog_version=2) +DIRECTIONS = {"cost": False, "utilisation": True} + + +def recommendation(supporting=("trained-1",)): + return Recommendation( + recommendation_id="rec-1", proposal="cheaper cartons", + supporting_order_ids=tuple(supporting), + expected_deltas=(ExpectedDelta(metric="cost", baseline_mean=10.0, + treatment_mean=8.0),), + confidence=1.0, constraints=("no rejected placement",), + rollback_plan="republish version 1") + + +def ledger_with(*events): + """`events` are `(decision_id, recorded_at)` shipments, plus optional event types.""" + ledger = OutcomeLedger() + for index, event in enumerate(events, start=1): + decision_id, at = event[0], event[1] + kind = event[2] if len(event) > 2 else OutcomeEventType.ACTUAL_CARTON + payload = ({"carton_id": "box-a", "catalog_version": 1} + if kind is OutcomeEventType.ACTUAL_CARTON + else {"reason_code": "crushed_corner", "severity": "minor"}) + ledger.record(OutcomeEvent(event_id=f"e{index}", decision_id=decision_id, + event_type=kind, payload=payload, recorded_at=at)) + return ledger + + +def replay(table, invalid=()): + """An evaluator over a `{decision_id: {arm: (cost, utilisation)}}` table. + + `invalid` names `(decision_id, arm)` pairs the validator should reject. + """ + def evaluate(decision_id: str, pin: ScenarioVersionPin) -> OrderEvaluationArtifact: + arm = "baseline" if pin == BASELINE_PIN else "treatment" + cost, utilisation = table[decision_id][arm] + run = OrderRunResult(order_id=decision_id, succeeded=True, + metrics={"cost": cost, "utilisation": utilisation}) + return OrderEvaluationArtifact(run=run, request={"decision_id": decision_id}, + result={"arm": arm}) + + def validator(request, result) -> ValidationVerdict: + if (request["decision_id"], result["arm"]) in invalid: + return ValidationVerdict(valid=False, codes=("unsupported_item",)) + return ValidationVerdict(valid=True) + + return evaluate, validator + + +def evaluate(table, *, ledger, invalid=(), decision_ids, split_at=500, rec=None): + evaluator, validator = replay(table, invalid) + return evaluate_on_history( + rec or recommendation(), ledger, decision_ids=decision_ids, + baseline_pin=BASELINE_PIN, treatment_pin=TREATMENT_PIN, + evaluator=evaluator, validator=validator, higher_is_better=DIRECTIONS, + split_at=split_at) + + +class TestTheSplitIsByRecordedTime: + def test_decisions_recorded_before_the_split_are_training(self): + result = evaluate({"trained-1": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.8)}, + "held-1": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.8)}}, + ledger=ledger_with(("trained-1", 100), ("held-1", 900)), + decision_ids=("trained-1", "held-1")) + assert result.training_decision_ids == ("trained-1",) + assert result.holdout_decision_ids == ("held-1",) + + def test_only_held_out_decisions_are_scored(self): + result = evaluate({"trained-1": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.8)}, + "held-1": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.8)}}, + ledger=ledger_with(("trained-1", 100), ("held-1", 900)), + decision_ids=("trained-1", "held-1")) + assert [d.decision_id for d in result.decisions] == ["held-1"] + + def test_the_split_uses_the_earliest_event_for_a_decision(self): + # A later correction must not drag a decision across the split it was recorded on + # the other side of. + ledger = ledger_with(("held-1", 100), ("held-1", 900, OutcomeEventType.DAMAGE)) + result = evaluate({"held-1": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.8)}}, + ledger=ledger, decision_ids=("held-1",), split_at=500, + rec=recommendation(supporting=("other",))) + assert result.training_decision_ids == ("held-1",) + assert result.decisions == () + + def test_a_decision_with_no_recorded_events_is_refused(self): + """It has no timestamp, so it cannot be placed on either side. Defaulting it + would be this module guessing.""" + with pytest.raises(ValueError, match="no recorded events"): + evaluate({}, ledger=OutcomeLedger(), decision_ids=("never-shipped",)) + + def test_a_negative_split_is_refused(self): + with pytest.raises(ValueError, match="split_at cannot be negative"): + evaluate({}, ledger=ledger_with(("held-1", 900)), decision_ids=("held-1",), + split_at=-1) + + def test_an_empty_decision_list_is_refused(self): + # An empty holdout would report zero regressions and read as a clean bill. + with pytest.raises(ValueError, match="at least one decision id"): + evaluate({}, ledger=OutcomeLedger(), decision_ids=()) + + +class TestAProposalCannotBeScoredOnItsOwnEvidence: + def test_a_cited_decision_in_the_holdout_is_refused_by_name(self): + with pytest.raises(SupportingEvidenceInHoldoutError, match="trained-1"): + evaluate({"trained-1": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.8)}}, + ledger=ledger_with(("trained-1", 900)), + decision_ids=("trained-1",), split_at=500) + + def test_the_same_decision_on_the_training_side_is_fine(self): + result = evaluate({"trained-1": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.8)}}, + ledger=ledger_with(("trained-1", 100)), + decision_ids=("trained-1",), split_at=500) + assert result.training_decision_ids == ("trained-1",) + + def test_the_refusal_is_a_holdout_error(self): + assert issubclass(SupportingEvidenceInHoldoutError, HoldoutError) + + +class TestTheVerdictPerDecision: + def _verdict(self, baseline, treatment, invalid=()): + result = evaluate({"held-1": {"baseline": baseline, "treatment": treatment}}, + ledger=ledger_with(("held-1", 900)), invalid=invalid, + decision_ids=("held-1",)) + return result.decisions[0] + + def test_better_on_every_axis_is_an_improvement(self): + assert self._verdict((10.0, 0.7), (8.0, 0.9)).verdict == IMPROVED + + def test_worse_on_every_axis_is_a_regression(self): + assert self._verdict((8.0, 0.9), (10.0, 0.7)).verdict == REGRESSED + + def test_cheaper_but_sparser_is_a_trade_off_rather_than_a_win(self): + assert self._verdict((10.0, 0.9), (8.0, 0.7)).verdict == TRADED_OFF + + def test_identical_metrics_are_a_trade_off_rather_than_a_win(self): + # Neither dominates, so neither is named; that is the report saying "no difference". + assert self._verdict((10.0, 0.8), (10.0, 0.8)).verdict == TRADED_OFF + + def test_a_rejected_treatment_is_a_regression_however_cheap_it_was(self): + """The rule the module exists to enforce. The treatment here is cheaper *and* + denser, and the validator refused the placement, so its metrics are discarded + rather than discounted.""" + outcome = self._verdict((16.0, 0.6), (10.0, 0.9), invalid=(("held-1", "treatment"),)) + assert outcome.verdict == REGRESSED + assert outcome.treatment_valid is False + assert outcome.treatment_codes == ("unsupported_item",) + + def test_a_rejected_baseline_leaves_a_valid_treatment_an_improvement(self): + outcome = self._verdict((8.0, 0.9), (10.0, 0.7), invalid=(("held-1", "baseline"),)) + assert outcome.verdict == IMPROVED + assert outcome.baseline_valid is False + + def test_both_arms_rejected_is_unpackable_rather_than_a_tie(self): + outcome = self._verdict((10.0, 0.7), (8.0, 0.9), + invalid=(("held-1", "baseline"), ("held-1", "treatment"))) + assert outcome.verdict == UNPACKABLE + + +class TestRealisedRiskIsReportedAgainstTheArmThatShipped: + def test_recorded_damage_is_attached_to_the_baseline(self): + """Damage happened under the carton that actually shipped. Crediting the + treatment with avoiding it would score a counterfactual nobody ran.""" + ledger = ledger_with(("held-1", 900), ("held-1", 950, OutcomeEventType.DAMAGE)) + result = evaluate({"held-1": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.9)}}, + ledger=ledger, decision_ids=("held-1",)) + assert result.decisions[0].baseline_realised_risk == ("damage",) + + def test_a_clean_history_carries_no_realised_risk(self): + result = evaluate({"held-1": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.9)}}, + ledger=ledger_with(("held-1", 900)), decision_ids=("held-1",)) + assert result.decisions[0].baseline_realised_risk == () + + +class TestTheTally: + def test_each_verdict_is_counted_once(self): + table = { + "up": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.9)}, + "down": {"baseline": (8.0, 0.9), "treatment": (10.0, 0.7)}, + "sideways": {"baseline": (10.0, 0.9), "treatment": (8.0, 0.7)}, + } + ledger = ledger_with(("up", 900), ("down", 910), ("sideways", 920)) + result = evaluate(table, ledger=ledger, + decision_ids=("up", "down", "sideways")) + assert (result.improved, result.regressed, + result.traded_off, result.unpackable) == (1, 1, 1, 0) + + def test_the_evaluation_records_both_pins_it_compared(self): + result = evaluate({"held-1": {"baseline": (10.0, 0.7), "treatment": (8.0, 0.9)}}, + ledger=ledger_with(("held-1", 900)), decision_ids=("held-1",)) + assert result.baseline_pin == BASELINE_PIN + assert result.treatment_pin == TREATMENT_PIN + assert result.split_at == 500 + + +class TestTheArtifactCarriesWhatTheValidatorReads: + def _run(self): + return OrderRunResult(order_id="d1", succeeded=True, metrics={"cost": 1.0}) + + def test_a_complete_artifact_is_accepted(self): + artifact = OrderEvaluationArtifact(run=self._run(), request={"a": 1}, + result={"b": 2}) + assert artifact.run.order_id == "d1" + + @pytest.mark.parametrize("missing", ["request", "result"]) + def test_an_artifact_without_the_pair_is_refused(self, missing): + """A run summary alone cannot be independently validated: the validator is handed + the request/result pair precisely so that no score can influence it.""" + fields = {"run": self._run(), "request": {"a": 1}, "result": {"b": 2}} + fields[missing] = None + with pytest.raises(ValueError, match="request/result pair"): + OrderEvaluationArtifact(**fields) diff --git a/tests/test_intelligence_api.py b/tests/test_intelligence_api.py new file mode 100644 index 0000000..47798dd --- /dev/null +++ b/tests/test_intelligence_api.py @@ -0,0 +1,177 @@ +"""The exported simulation and recommendations API. + +`docs/INTELLIGENCE-API.md` is the contract. This task is packaging, not behaviour, so +what is checked here is packaging: + + * the modules are importable under their public names, and the surface the design + document names is actually present; + * the workspace paths resolve to the *same objects*. `simulation/scenario.py`, + `recommendations/engine.py` and `benchmarks/comparator/pareto_report.py` are + re-export shims after the move; the identity assertions fail the moment one starts + carrying its own copy. The design document's objection to a duplicate public + spelling -- "a second definition free to drift from the first" -- is what these + guard. + +Behaviour is not re-tested here. `simulation/tests/` and `recommendations/tests/` own +it and now exercise the package through the shims, so a second copy of those assertions +would be the very duplication this file exists to prevent. +""" + +from __future__ import annotations + +import sys +from pathlib import Path + +import pytest + +WORKSPACE_ROOT = Path(__file__).resolve().parents[2] + + +def _require_workspace_file(relative: str) -> None: + # The shims live in the workspace around this package; a published copy has no + # workspace, so there is nothing whose identity could drift and nothing to check. + if not (WORKSPACE_ROOT / relative).is_file(): + pytest.skip("the workspace re-export shims are not part of this package") + + +def _workspace_module(dotted: str): + _require_workspace_file(dotted.replace(".", "/") + ".py") + if str(WORKSPACE_ROOT) not in sys.path: + sys.path.insert(0, str(WORKSPACE_ROOT)) + return __import__(dotted, fromlist=["*"]) + + +class TestTheDesignedSurfaceIsExported: + """Every name docs/INTELLIGENCE-API.md tables as exported is reachable.""" + + def test_simulation_exports_the_scenario_surface(self): + import packvium.simulation as simulation + + for name in ("ScenarioVersionPin", "OrderRunResult", "ScenarioResult", + "run_scenario", "compare_scenarios", + "ScenarioError", "MismatchedOrderCorpusError"): + assert hasattr(simulation, name), name + + def test_recommendations_exports_the_proposal_surface(self): + import packvium.recommendations as recommendations + + for name in ("ExpectedDelta", "Recommendation", "ApprovalRecord", + "propose_recommendation", "approve", + "approve_catalog", "approve_policy", "RecommendationError"): + assert hasattr(recommendations, name), name + + def test_the_comparison_type_a_caller_receives_is_importable(self): + # `compare_scenarios` returns `tuple[ProfileReport, ...]`, so the type is part of + # a public signature. It could not stay in `benchmarks/`, which is never exported. + from packvium.pareto import ProfileReport + from packvium.simulation import ProfileReport as re_exported + + assert re_exported is ProfileReport + + +class TestWorkspaceShimsReExportTheSameObjects: + def test_scenario_shim_is_not_a_second_implementation(self): + import packvium.simulation as canonical + + shim = _workspace_module("simulation.scenario") + assert shim.run_scenario is canonical.run_scenario + assert shim.compare_scenarios is canonical.compare_scenarios + assert shim.ScenarioResult is canonical.ScenarioResult + + def test_recommendations_shim_is_not_a_second_implementation(self): + import packvium.recommendations as canonical + + shim = _workspace_module("recommendations.engine") + assert shim.propose_recommendation is canonical.propose_recommendation + assert shim.approve_catalog is canonical.approve_catalog + assert shim.Recommendation is canonical.Recommendation + + def test_pareto_shim_is_not_a_second_implementation(self): + import packvium.pareto as canonical + + # The benchmark comparators keep importing this path and have no reason to know + # where the implementation moved; what they must not get is a second dominance + # rule, which would make a benchmark score against something a caller never sees. + _require_workspace_file("benchmarks/comparator/pareto_report.py") + sys.path.insert(0, str(WORKSPACE_ROOT / "benchmarks" / "comparator")) + import pareto_report as shim + + assert shim.generate_report is canonical.generate_report + assert shim.dominates is canonical.dominates + assert shim.ProfileReport is canonical.ProfileReport + + +class TestHistoricalReplayIsExportedWithItsLedger: + """. `evaluate_on_history` could not ship until the ledger did: its central + argument is an `OutcomeLedger`, and a function whose main argument a consumer cannot + construct is worse than an unexported one.""" + + def test_the_ledger_and_the_replay_are_both_importable(self): + from packvium.holdout import HoldoutEvaluation, evaluate_on_history + from packvium.outcomes import OutcomeEvent, OutcomeEventType, OutcomeLedger + + assert callable(evaluate_on_history) + assert hasattr(HoldoutEvaluation, "improved") + assert callable(OutcomeLedger().record) + assert OutcomeEvent and OutcomeEventType + + def test_the_ledger_shim_is_not_a_second_implementation(self): + import packvium.outcomes as canonical + + shim = _workspace_module("domain.outcomes.model") + assert shim.OutcomeLedger is canonical.OutcomeLedger + assert shim.OutcomeEvent is canonical.OutcomeEvent + + def test_the_replay_shim_is_not_a_second_implementation(self): + import packvium.holdout as canonical + + shim = _workspace_module("recommendations.holdout") + assert shim.evaluate_on_history is canonical.evaluate_on_history + assert shim.ValidationVerdict is canonical.ValidationVerdict + + def test_the_ledger_public_surface_stays_closed(self): + """The guard `domain/outcomes/tests` holds, re-asserted where the code now lives. + + A new method on the ledger is a decision to widen a published surface, and has to + be made deliberately rather than by arriving. + """ + from packvium.outcomes import OutcomeLedger + + assert {n for n in dir(OutcomeLedger) if not n.startswith("_")} == { + "record", "events_for_decision", "current_view", "view_as_of", + } + + def test_the_validator_gate_survives_the_move(self): + """A rejected packing can never be recorded as an improvement, in the package too.""" + import pytest + + from packvium.holdout import IMPROVED, DecisionOutcome + + with pytest.raises(ValueError, match="never be recorded as an improvement"): + DecisionOutcome(decision_id="d1", verdict=IMPROVED, + baseline_valid=True, treatment_valid=False, + treatment_codes=("unsupported_overhang",)) + + +class TestTheRegistryRouteStaysStructural: + def test_a_proposal_is_handed_no_registry(self): + """`propose_recommendation` has no path to a registry, and that is structural. + + docs/INTELLIGENCE-API.md makes this the API's whole obligation: nothing exported + may write to a registry except through `approve*`. A proposal that could reach + one would make that a documented promise rather than an enforced one. + """ + import inspect + + from packvium.recommendations import propose_recommendation + + parameters = inspect.signature(propose_recommendation).parameters + assert not [p for p in parameters if "registry" in p.lower()], sorted(parameters) + + def test_a_recommendation_cannot_exist_without_a_rollback_plan(self): + import inspect + + from packvium.recommendations import propose_recommendation + + rollback = inspect.signature(propose_recommendation).parameters["rollback_plan"] + assert rollback.default is inspect.Parameter.empty diff --git a/tests/test_intelligence_guards.py b/tests/test_intelligence_guards.py new file mode 100644 index 0000000..b9b61aa --- /dev/null +++ b/tests/test_intelligence_guards.py @@ -0,0 +1,472 @@ +"""Every refusal the intelligence surface makes, fired at least once. + +Measured before this file existed: the six modules 1.2.0 exports sat at 93% statement +coverage, and **39 of the 41 uncovered statements were validation guards** -- the `raise` +inside a `__post_init__` or at the head of a function. Nothing anywhere fired them. + +That is a worse gap than the number suggests. A guard is the one kind of code whose +absence looks exactly like its presence: delete the check, reorder `__post_init__` so it +runs before the field is set, invert a comparison, and every existing test still passes, +because every existing test supplies valid input. The construction that should have been +refused is simply accepted, and the first thing to notice is a caller holding a +`Recommendation` with a confidence of 4.0 or a ledger event stamped before the epoch. + +So each test here builds one object that is wrong in exactly one way and asserts the +refusal. They are cheap, they are boring, and they are the reason a future edit to any of +these classes cannot quietly stop validating. +""" + +from __future__ import annotations + +import pytest + +from packvium.holdout import ( + IMPROVED, + REGRESSED, + TRADED_OFF, + UNPACKABLE, + DecisionOutcome, + HoldoutEvaluation, + ValidationVerdict, + evaluate_on_history, +) +from packvium.locks import _placed +from packvium.outcomes import ( + ALLOWED_FIELDS, + OutcomeEvent, + OutcomeEventType, + OutcomeLedger, + UnsupportedOutcomeFieldError, +) +from packvium.pareto import CandidateResult, ProfileReport +from packvium.recommendations import ( + ApprovalRecord, + ExpectedDelta, + MismatchedOrderCorpusError, + Recommendation, + propose_recommendation, +) +from packvium.simulation import ( + OrderRunResult, + ScenarioResult, + ScenarioVersionPin, +) + +PIN = ScenarioVersionPin(catalog_version=1) + + +def _run(order_id: str, cost: float = 10.0, **overrides) -> OrderRunResult: + fields = {"order_id": order_id, "succeeded": True, "metrics": {"cost": cost}} + fields.update(overrides) + return OrderRunResult(**fields) + + +def _scenario(scenario_id: str, runs: tuple[OrderRunResult, ...]) -> ScenarioResult: + return ScenarioResult(scenario_id=scenario_id, version_pin=PIN, + order_ids=tuple(r.order_id for r in runs), runs=runs) + + +class TestAVersionPinCannotBeAmbiguous: + """A pin exists so a scenario replays to the same answer. Every refusal here is a + pin that would have replayed to a different one, or to none.""" + + def test_a_version_number_below_one_is_refused(self): + # Versions are positions in a published history starting at 1, so 0 is not an + # "unset" sentinel -- `None` is. Accepting it would silently pin nothing. + with pytest.raises(ValueError, match="version numbers must be positive"): + ScenarioVersionPin(catalog_version=0) + + def test_a_tariff_version_below_one_is_refused(self): + with pytest.raises(ValueError, match="version numbers must be positive"): + ScenarioVersionPin(tariff_version=-1) + + def test_a_policy_pin_without_a_rule_id_is_refused(self): + with pytest.raises(ValueError, match="policy version pins require"): + ScenarioVersionPin(policy_versions=(("", 2),)) + + def test_a_policy_pin_with_a_non_positive_version_is_refused(self): + with pytest.raises(ValueError, match="policy version pins require"): + ScenarioVersionPin(policy_versions=(("carrier-rules", 0),)) + + def test_the_same_policy_rule_cannot_be_pinned_twice(self): + # Two pins for one rule is not a merge conflict to resolve by last-wins: it is a + # caller who does not know which version they meant. + with pytest.raises(ValueError, match="same policy rule twice"): + ScenarioVersionPin(policy_versions=(("carrier-rules", 1), ("carrier-rules", 2))) + + def test_an_empty_solver_version_is_refused_but_an_absent_one_is_not(self): + with pytest.raises(ValueError, match="solver_version must be non-empty"): + ScenarioVersionPin(catalog_version=1, solver_version="") + assert ScenarioVersionPin(catalog_version=1, + solver_version=None).solver_version is None + + def test_a_pin_that_pins_nothing_is_refused(self): + """Not a coverage gap -- this one was already exercised -- but it is the guard + that makes every other field optional without making the type meaningless.""" + with pytest.raises(ValueError, match="at least one of its fields"): + ScenarioVersionPin() + + +class TestARunCannotCarryAnUnnamedNumber: + def test_a_run_without_an_order_id_is_refused(self): + with pytest.raises(ValueError, match="order_id is required"): + OrderRunResult(order_id="", succeeded=True, metrics={"cost": 1.0}) + + def test_a_metric_with_an_empty_name_is_refused(self): + # An unnamed axis cannot be compared against the same axis in the other arm, so + # it would drop out of every dominance check while still looking like evidence. + with pytest.raises(ValueError, match="metric names must be non-empty"): + OrderRunResult(order_id="order-1", succeeded=True, metrics={"": 1.0}) + + +class TestAScenarioMustDescribeTheCorpusItRan: + def test_a_scenario_without_an_id_is_refused(self): + with pytest.raises(ValueError, match="scenario_id is required"): + ScenarioResult(scenario_id="", version_pin=PIN, order_ids=("order-1",), + runs=(_run("order-1"),)) + + def test_a_scenario_over_no_orders_is_refused(self): + with pytest.raises(ValueError, match="order_ids cannot be empty"): + ScenarioResult(scenario_id="s", version_pin=PIN, order_ids=(), runs=()) + + def test_runs_that_do_not_line_up_with_the_orders_are_refused(self): + # The pairing is positional everywhere downstream. A mismatch here would pair one + # order's baseline against another order's treatment and report the difference. + with pytest.raises(ValueError, match="1:1"): + ScenarioResult(scenario_id="s", version_pin=PIN, + order_ids=("order-1", "order-2"), runs=(_run("order-1"),)) + + def test_runs_in_a_different_order_than_the_ids_are_refused(self): + with pytest.raises(ValueError, match="1:1"): + ScenarioResult(scenario_id="s", version_pin=PIN, + order_ids=("order-1", "order-2"), + runs=(_run("order-2"), _run("order-1"))) + + +class TestAParetoCandidateCannotBeAnonymous: + def test_a_candidate_without_a_profile_is_refused(self): + with pytest.raises(ValueError, match="profile is required"): + CandidateResult(profile="", engine="python", metrics={"cost": 1.0}) + + def test_a_candidate_without_an_engine_is_refused(self): + with pytest.raises(ValueError, match="engine is required"): + CandidateResult(profile="fast", engine="", metrics={"cost": 1.0}) + + def test_a_candidate_with_no_metrics_is_refused(self): + # Nothing dominates a candidate with no axes, so an empty one would arrive on the + # frontier of every profile it was added to. + with pytest.raises(ValueError, match="metrics cannot be empty"): + CandidateResult(profile="fast", engine="python", metrics={}) + + +class TestAProfileReportCannotContradictItself: + def test_a_report_without_a_profile_is_refused(self): + with pytest.raises(ValueError, match="profile is required"): + ProfileReport(profile="", pareto_optimal=("python",), dominated=()) + + def test_a_winner_that_is_not_on_the_frontier_is_refused(self): + with pytest.raises(ValueError, match="must be one of the Pareto-optimal"): + ProfileReport(profile="fast", pareto_optimal=("python",), dominated=("php",), + winner="php") + + def test_a_winner_alongside_a_genuine_trade_off_is_refused(self): + # The whole point of the type: when two candidates survive, naming one of them + # winner is the single-number collapse the report exists to refuse. + with pytest.raises(ValueError, match="exactly one candidate is Pareto-optimal"): + ProfileReport(profile="fast", pareto_optimal=("php", "python"), dominated=(), + winner="python") + + +class TestARecommendationCannotBeUnaccountable: + def _valid(self, **overrides): + fields = { + "recommendation_id": "rec-1", "proposal": "cheaper cartons", + "supporting_order_ids": ("order-1",), + "expected_deltas": (ExpectedDelta(metric="cost", baseline_mean=10.0, + treatment_mean=8.0),), + "confidence": 0.8, "constraints": ("no placement the validator rejects",), + "rollback_plan": "republish v1", + } + fields.update(overrides) + return Recommendation(**fields) + + def test_the_valid_shape_is_actually_valid(self): + """Guards the negatives below: if the baseline were itself refused, every test in + this class would pass for the wrong reason.""" + assert self._valid().recommendation_id == "rec-1" + + def test_a_delta_on_an_unnamed_metric_is_refused(self): + with pytest.raises(ValueError, match="metric is required"): + ExpectedDelta(metric="", baseline_mean=1.0, treatment_mean=2.0) + + def test_a_recommendation_without_an_id_is_refused(self): + with pytest.raises(ValueError, match="recommendation_id is required"): + self._valid(recommendation_id="") + + def test_a_recommendation_without_a_proposal_is_refused(self): + with pytest.raises(ValueError, match="proposal is required"): + self._valid(proposal="") + + def test_a_recommendation_citing_no_orders_is_refused(self): + # Evidence-free advice is the one output this module must not be able to produce. + with pytest.raises(ValueError, match="at least one supporting order"): + self._valid(supporting_order_ids=()) + + @pytest.mark.parametrize("confidence", [-0.1, 1.1, 4.0]) + def test_a_confidence_outside_zero_to_one_is_refused(self, confidence): + with pytest.raises(ValueError, match="confidence must be between 0 and 1"): + self._valid(confidence=confidence) + + def test_a_recommendation_stating_no_constraint_is_refused(self): + """Found by the canary above rather than by reading: the shared fixture passed an + empty tuple and every negative test in this class was passing on the wrong + exception.""" + with pytest.raises(ValueError, match="at least one non-empty constraint"): + self._valid(constraints=()) + + def test_a_recommendation_whose_only_constraint_is_blank_is_refused(self): + with pytest.raises(ValueError, match="at least one non-empty constraint"): + self._valid(constraints=("",)) + + def test_a_recommendation_without_a_rollback_plan_is_refused(self): + with pytest.raises(ValueError, match="must state its rollback plan"): + self._valid(rollback_plan="") + + +class TestAnApprovalRecordCannotBeBackdatedOrUnversioned: + def test_an_approval_without_a_recommendation_is_refused(self): + with pytest.raises(ValueError, match="recommendation_id is required"): + ApprovalRecord(recommendation_id="", approved_at=1, published_version=1) + + def test_an_approval_before_the_epoch_is_refused(self): + with pytest.raises(ValueError, match="approved_at cannot be negative"): + ApprovalRecord(recommendation_id="rec-1", approved_at=-1, published_version=1) + + def test_an_approval_naming_version_zero_is_refused(self): + # Published versions are positions in a history starting at 1, so 0 names nothing. + with pytest.raises(ValueError, match="published_version must be positive"): + ApprovalRecord(recommendation_id="rec-1", approved_at=1, published_version=0) + + +class TestProposingFromEvidenceThatCannotSupportIt: + def test_two_different_corpora_are_refused_rather_than_intersected(self): + # Silently comparing the overlap would let cohort mix manufacture an improvement. + with pytest.raises(MismatchedOrderCorpusError): + propose_recommendation( + "rec-1", "p", _scenario("b", (_run("order-1"),)), + _scenario("t", (_run("order-2"),)), + constraints=(), rollback_plan="back", minimum_cohort_size=1, + minimum_confidence=0.0) + + def test_a_non_positive_cohort_floor_is_refused(self): + with pytest.raises(ValueError, match="minimum_cohort_size must be positive"): + propose_recommendation( + "rec-1", "p", _scenario("b", (_run("order-1"),)), + _scenario("t", (_run("order-1"),)), + constraints=(), rollback_plan="back", minimum_cohort_size=0, + minimum_confidence=0.0) + + @pytest.mark.parametrize("confidence", [-0.5, 1.5]) + def test_a_confidence_floor_outside_zero_to_one_is_refused(self, confidence): + with pytest.raises(ValueError, match="minimum_confidence must be between 0 and 1"): + propose_recommendation( + "rec-1", "p", _scenario("b", (_run("order-1"),)), + _scenario("t", (_run("order-1"),)), + constraints=(), rollback_plan="back", minimum_cohort_size=1, + minimum_confidence=confidence) + + def test_arms_that_share_no_metric_produce_no_recommendation(self): + """Both arms succeeded on every order, so the cohort is full and confidence is + 1.0 -- and there is still nothing to recommend, because no axis appears on both + sides. `None` rather than a `Recommendation` carrying an empty delta list.""" + baseline = _scenario("b", (OrderRunResult(order_id="order-1", succeeded=True, + metrics={"cost": 10.0}),)) + treatment = _scenario("t", (OrderRunResult(order_id="order-1", succeeded=True, + metrics={"utilisation": 0.8}),)) + assert propose_recommendation( + "rec-1", "p", baseline, treatment, constraints=(), rollback_plan="back", + minimum_cohort_size=1, minimum_confidence=0.0) is None + + +class TestALedgerEventCannotBeMalformed: + def test_an_event_without_an_id_is_refused(self): + with pytest.raises(ValueError, match="event_id is required"): + OutcomeEvent(event_id="", decision_id="d1", + event_type=OutcomeEventType.MEASURED_WEIGHT, + payload={"weight_g": 100}, recorded_at=1) + + def test_an_event_recorded_before_the_epoch_is_refused(self): + # Timestamps are what a holdout split is made of; a negative one would sort into + # the training side of every split that could ever be chosen. + with pytest.raises(ValueError, match="recorded_at cannot be negative"): + OutcomeEvent(event_id="e1", decision_id="d1", + event_type=OutcomeEventType.MEASURED_WEIGHT, + payload={"weight_g": 100}, recorded_at=-1) + + def test_a_view_at_a_negative_instant_is_refused(self): + with pytest.raises(ValueError, match="as-of time cannot be negative"): + OutcomeLedger().view_as_of("d1", -1) + + @pytest.mark.parametrize("event_type,payload,rejected", [ + (OutcomeEventType.DAMAGE, {"carton_id": "box-a"}, "carton_id"), + (OutcomeEventType.MEASURED_WEIGHT, {"weight_g": 1, "severity": "high"}, "severity"), + (OutcomeEventType.RETURN, {"operator_id": "op-7"}, "operator_id"), + ]) + def test_a_field_the_event_type_does_not_define_is_refused(self, event_type, payload, + rejected): + """The closed set is what lets a replay rely on what it finds. An open payload + would make the ledger a log.""" + with pytest.raises(UnsupportedOutcomeFieldError, match=rejected): + OutcomeEvent(event_id="e1", decision_id="d1", event_type=event_type, + payload=payload, recorded_at=1) + + +class TestEveryEventTypeDeclaresItsFields: + """The guard behind these two tests cannot fire through any legitimate call, and that + is exactly why it is worth holding: it exists to catch a *future* edit that adds an + `OutcomeEventType` member and forgets to say which fields it carries. Without a test, + the first sign would be an event accepted with a payload nobody validated.""" + + def test_no_event_type_is_missing_from_the_allowed_set(self): + undeclared = [event_type.name for event_type in OutcomeEventType + if event_type not in ALLOWED_FIELDS] + assert not undeclared, f"event type(s) with no declared fields: {undeclared}" + + def test_the_allowed_set_declares_nothing_that_is_not_an_event_type(self): + # The other direction: a stale entry left behind by a removed member would make + # the check above pass while the table quietly described a type that is gone. + assert set(ALLOWED_FIELDS) == set(OutcomeEventType) + + def test_an_undeclared_event_type_is_refused_rather_than_waved_through(self): + """`event_type` is not type-checked at runtime, so the refusal has to be real + rather than implied by the annotation.""" + with pytest.raises(UnsupportedOutcomeFieldError, match="unsupported event type"): + OutcomeEvent(event_id="e1", decision_id="d1", event_type="delivered", + payload={}, recorded_at=1) + + +class TestAValidationVerdictCannotBeSelfContradictory: + def test_a_valid_verdict_carrying_rejection_codes_is_refused(self): + with pytest.raises(ValueError, match="valid verdict cannot carry rejection codes"): + ValidationVerdict(valid=True, codes=("unsupported_item",)) + + def test_a_rejection_naming_no_rule_is_refused(self): + # "Rejected, reason unavailable" is the shape an operator cannot act on. + with pytest.raises(ValueError, match="must say which rules it failed"): + ValidationVerdict(valid=False, codes=()) + + +class TestADecisionOutcomeCannotMisreportItsArm: + def test_a_decision_without_an_id_is_refused(self): + with pytest.raises(ValueError, match="decision_id is required"): + DecisionOutcome(decision_id="", verdict=IMPROVED, baseline_valid=True, + treatment_valid=True) + + def test_a_verdict_outside_the_four_is_refused(self): + with pytest.raises(ValueError, match="unknown verdict"): + DecisionOutcome(decision_id="d1", verdict="better", baseline_valid=True, + treatment_valid=True) + + def test_an_improvement_whose_treatment_was_rejected_is_refused(self): + """The gate that matters most, asserted at the type rather than trusted to the + control flow that also enforces it: there is no cost at which a packing the + validator refused becomes an improvement.""" + with pytest.raises(ValueError): + DecisionOutcome(decision_id="d1", verdict=IMPROVED, baseline_valid=True, + treatment_valid=False, treatment_codes=("unstable",)) + + @pytest.mark.parametrize("verdict", [IMPROVED, REGRESSED, TRADED_OFF, UNPACKABLE]) + def test_each_of_the_four_verdicts_is_accepted(self, verdict): + outcome = DecisionOutcome(decision_id="d1", verdict=verdict, baseline_valid=True, + treatment_valid=True) + assert outcome.verdict == verdict + + +class TestTheHoldoutTallyCountsEveryVerdict: + def test_all_four_counters_report_their_own_verdict(self): + """`unpackable` had no caller anywhere. A counter nothing reads is a counter that + can be wrong for a whole release.""" + evaluation = HoldoutEvaluation( + recommendation_id="rec-1", split_at=10, baseline_pin=PIN, treatment_pin=PIN, + training_decision_ids=(), holdout_decision_ids=("a", "b", "c", "d"), + decisions=tuple( + DecisionOutcome(decision_id=decision_id, verdict=verdict, + baseline_valid=True, treatment_valid=True) + for decision_id, verdict in ( + ("a", IMPROVED), ("b", REGRESSED), ("c", TRADED_OFF), ("d", UNPACKABLE)) + ), + ) + assert (evaluation.improved, evaluation.regressed, + evaluation.traded_off, evaluation.unpackable) == (1, 1, 1, 1) + + +class TestReplayRefusesAnUnanswerableRequest: + def _recommendation(self): + return Recommendation( + recommendation_id="rec-1", proposal="p", supporting_order_ids=("order-1",), + expected_deltas=(ExpectedDelta(metric="cost", baseline_mean=10.0, + treatment_mean=8.0),), + confidence=1.0, constraints=("no placement the validator rejects",), + rollback_plan="back") + + def _evaluate(self, **overrides): + fields = { + "recommendation": self._recommendation(), "ledger": OutcomeLedger(), + "decision_ids": ("d1",), "baseline_pin": PIN, "treatment_pin": PIN, + "evaluator": lambda decision_id, pin: None, + "validator": lambda request, result: ValidationVerdict(valid=True), + "higher_is_better": {"cost": False}, "split_at": 10, + } + fields.update(overrides) + return evaluate_on_history(**fields) + + def test_a_negative_split_is_refused(self): + with pytest.raises(ValueError, match="split_at cannot be negative"): + self._evaluate(split_at=-1) + + def test_an_empty_decision_list_is_refused(self): + # Vacuous success is the failure mode: an empty holdout would report zero + # regressions and read as a clean bill of health. + with pytest.raises(ValueError, match="at least one decision id is required"): + self._evaluate(decision_ids=()) + + def test_a_decision_with_no_recorded_events_is_refused(self): + """An unrecorded decision has no timestamp, so it cannot be placed on either side + of a split by time -- and defaulting it to one side would be this library + guessing.""" + with pytest.raises(ValueError, match="no recorded events"): + self._evaluate(decision_ids=("never-shipped",)) + + +class TestAMalformedPlacementIsSkippedRatherThanCrashing: + """`_placed` is private and reached through `resolve_with_locks`, but the branch it + guards cannot be provoked through that path: the solver does not emit a placement + without tick coordinates. Tested directly, because a defensive branch nothing + exercises is a defensive branch that may already be broken.""" + + def _container(self, *placements): + return {"containers": [{"placements": list(placements)}]} + + def _placement(self, position): + return {"item_type": "box", "orientation": "LWH", "position": position} + + def test_a_well_formed_placement_is_collected(self): + found = _placed(self._container(self._placement( + {"x": {"ticks": 0}, "y": {"ticks": 16000}, "z": {"ticks": 32000}}))) + assert found == {(0, "box", "LWH", (0, 16000, 32000))} + + def test_a_placement_missing_an_axis_is_skipped(self): + assert _placed(self._container(self._placement( + {"x": {"ticks": 0}, "y": {"ticks": 0}}))) == set() + + def test_a_placement_whose_axis_is_not_a_mapping_is_skipped(self): + assert _placed(self._container(self._placement( + {"x": 0, "y": 0, "z": 0}))) == set() + + def test_a_skipped_placement_does_not_hide_a_good_one(self): + # The `continue` must skip one placement, not abandon the container. + found = _placed(self._container( + self._placement({"x": {"ticks": 0}}), + self._placement({"x": {"ticks": 1}, "y": {"ticks": 2}, "z": {"ticks": 3}}), + )) + assert found == {(0, "box", "LWH", (1, 2, 3))} diff --git a/tests/test_locks.py b/tests/test_locks.py new file mode 100644 index 0000000..f4431cf --- /dev/null +++ b/tests/test_locks.py @@ -0,0 +1,509 @@ +"""Operator locks. + +`docs/EXECUTION-PLAN.md` names four properties, and each one is a thing the layer must +*not* do. So this suite is written against the failure modes rather than the happy path: + + * a lock must not mutate the approved plan; + * a lock must not place anything the engine would refuse, in any language of the word -- + it can forbid alternatives to itself and nothing else; + * an unsatisfiable lock must come back as a result with a status, not an exception; + * preservation must be *measured*, because forbidding alternatives does not make the + solver try the locked point. + +A fifth property is not in the design document because it was found here: **a lock must not +cost the load anything it did not ask for.** Two implementations failed it, each by emptying +a container that had been full -- one refused by `item_type`, so eight cubes became one; the +other demanded the locked slot be filled first, so a lock on the far corner produced nothing +at all, because an empty container offers only the origin as a candidate point. Both are the +accommodation this layer exists to prevent, wearing the opposite costume, and both are +regression tests here. +""" + +from __future__ import annotations + +import copy +import json +import pathlib + +import pytest + +from packvium.constraints import ConstraintContext +from packvium.extensions import ExtensionRegistry +from packvium.geometry import Dimensions, Point, Rotation +from packvium.locks import ( + LOCK_VIOLATED, + LockSetError, + LockedResolve, + PlacementLock, + _LockConstraint, + lock_registry, + locks_from_plan, + resolve_with_locks, +) +from packvium.models import Container, Item, ItemInstance +from packvium.serialization import pack_from_dict + +FIXTURES = pathlib.Path(__file__).resolve().parents[2] / "conformance" / "fixtures" + +#: Millimetres in the fixed-point ticks the schema's `exactScalar` carries. +MM = 16_000 +MM100 = 100 * MM + + +def eight_cubes() -> dict: + """Eight 100mm cubes that exactly fill a 200mm box -- the corpus's simplest tiling.""" + # A cross-language fixture kept one level above this package; a published copy does not + # carry it, and the lock tests that build their own scenes still run. + fixture = FIXTURES / "exact-fit.json" + if not fixture.is_file(): + pytest.skip("the shared cross-language fixture corpus is not part of this package") + return json.loads(fixture.read_text()) + + +def placed(result: dict) -> set[tuple[int, str, str, tuple[int, int, int]]]: + return { + (index, placement["item_type"], placement["orientation"], + tuple(int(placement["position"][axis]["ticks"]) for axis in ("x", "y", "z"))) + for index, container in enumerate(result["containers"]) + for placement in container["placements"] + } + + +def count(result: dict) -> int: + return sum(len(container["placements"]) for container in result["containers"]) + + +# ----------------------------------------------------- a lock binds one instance + + +def test_locking_one_cube_leaves_the_other_seven_free(): + """The regression this suite exists for. + + A lock addresses a placement -- `(container_index, item_type, position, orientation)`. + Reading it as a rule about the *type* turns "keep this cube here" into "no cube may be + anywhere else", which silently drops seven items from a feasible load. + """ + request = eight_cubes() + baseline = pack_from_dict(request) + assert count(baseline) == 8, "the fixture no longer tiles; the rest of this test is void" + + lock = PlacementLock(0, "cube", "LWH", (0, 0, 0)) + resolve = resolve_with_locks(request, [lock]) + + assert resolve.preserved + assert count(resolve.result) == 8 + + +@pytest.mark.parametrize("position", [ + (0, 0, 0), + (MM100, 0, 0), + (MM100, MM100, MM100), +]) +def test_the_locked_slot_is_the_one_the_operator_named(position): + """The far corner is the second regression. + + Candidate points are extreme points derived from what is already placed, so an empty + container offers only the origin. An implementation that required the locked slot to be + filled before any other instance could not reach `(100, 100, 100)` at all and returned an + empty container -- a lock the search has not reached yet is not an infeasible lock. + """ + request = eight_cubes() + resolve = resolve_with_locks(request, [PlacementLock(0, "cube", "LWH", position)]) + + assert resolve.preserved + assert (0, "cube", "LWH", position) in placed(resolve.result) + assert count(resolve.result) == 8, "the lock cost the load items it did not ask for" + + +def test_two_locks_on_one_type_are_both_honoured(): + request = eight_cubes() + locks = [PlacementLock(0, "cube", "LWH", (0, 0, 0)), + PlacementLock(0, "cube", "LWH", (MM100, MM100, 0))] + resolve = resolve_with_locks(request, locks) + + assert resolve.preserved, resolve.missing + assert count(resolve.result) == 8 + + +# ---------------------------------------------------- the approved plan survives + + +def test_the_request_is_not_mutated(): + request = eight_cubes() + before = copy.deepcopy(request) + resolve_with_locks(request, [PlacementLock(0, "cube", "LWH", (0, 0, 0))]) + assert request == before + + +def test_the_original_result_is_a_separate_artifact(): + """Property 1. The lock produces a candidate beside the approved plan, never over it.""" + request = eight_cubes() + approved = pack_from_dict(request) + snapshot = copy.deepcopy(approved) + + resolve = resolve_with_locks(request, [PlacementLock(0, "cube", "LWH", (0, 0, 0))]) + + assert approved == snapshot + assert resolve.result is not approved + + +# ------------------------------- a self-contradictory set is refused before any solve + + +def test_two_locks_claiming_the_same_space_are_refused_without_solving(): + """The line between the two diagnostics this layer produces. + + Whether a lock *fits* is a question about a request, and its answer is `preserved=False` + with the lock named. Whether a lock set contradicts *itself* is not a question about a + request at all: two overlapping boxes cannot both hold under any request, container or + solver. Solving it anyway returns an emptied container -- measured, before this guard + existed: the two reservations blocked every cell of a lattice that had packed eight + cubes -- and calling that a result would be the accommodation in reverse. + """ + with pytest.raises(LockSetError, match="overlapping space"): + resolve_with_locks(eight_cubes(), [ + PlacementLock(0, "cube", "LWH", (0, 0, 0)), + PlacementLock(0, "cube", "LWH", (50 * MM, 0, 0)), + ]) + + +def test_the_same_placement_locked_twice_is_refused(): + with pytest.raises(LockSetError, match="locked twice"): + resolve_with_locks(eight_cubes(), [PlacementLock(0, "cube", "LWH", (0, 0, 0))] * 2) + + +def test_a_lock_on_an_item_the_request_does_not_contain_is_refused(): + """An operator naming an item that is not in the shipment is a typo, not an + infeasibility. Reported as one, it would arrive as a quietly unpreserved lock.""" + with pytest.raises(LockSetError, match="no item type 'wedge'"): + resolve_with_locks(eight_cubes(), [PlacementLock(0, "wedge", "LWH", (0, 0, 0))]) + + +def test_locks_in_different_containers_never_contradict_each_other(): + """They describe different boxes, so identical coordinates in each are ordinary.""" + request = eight_cubes() + request["containers"][0]["quantity"] = 2 + resolve_with_locks(request, [PlacementLock(0, "cube", "LWH", (0, 0, 0)), + PlacementLock(1, "cube", "LWH", (0, 0, 0))]) + + +# ------------------------------------------- an unsatisfiable lock is an answer + + +def test_a_lock_the_solver_cannot_satisfy_returns_a_result_and_not_an_exception(): + """Property 3, and the reason it matters: an operator gets diagnostics, not a stack trace. + + The lock reserves a box straddling the centre of a container that tiles exactly, so the + reserved volume meets every cell of the lattice and no cube has anywhere legal to go. + Nothing in the lock layer decides that: the reservation refuses the overlapping + candidates and the ordinary search reports what it could not place, in its own + vocabulary. + """ + request = eight_cubes() + off_grid = (50 * MM, 50 * MM, 50 * MM) + + resolve = resolve_with_locks(request, [PlacementLock(0, "cube", "LWH", off_grid)]) + + assert not resolve.preserved + assert resolve.missing[0].position_ticks == off_grid + assert resolve.result["status"] in {"best_found", "infeasible"} + assert resolve.result["unpacked_items"], "an unhonoured lock must leave a trail" + assert resolve.result["unpacked_items"][0]["proof"]["level"] in { + "proven", "observed", "inferred", "unknown_due_to_limit" + } + + +def test_an_unhonoured_lock_is_never_softened_into_a_success(): + request = eight_cubes() + resolve = resolve_with_locks( + request, [PlacementLock(0, "cube", "LWH", (50 * MM, 50 * MM, 50 * MM))]) + + with pytest.raises(ValueError, match="cannot be missing a lock"): + LockedResolve(locks=resolve.locks, result=resolve.result, preserved=True, + missing=resolve.missing) + + +def test_an_unpreserved_resolve_must_say_which_lock_failed(): + with pytest.raises(ValueError, match="must name the locks"): + LockedResolve(locks=(), result={}, preserved=False, missing=()) + + +# ------------------------------------- a lock outside the container is inert, not fatal + + +def test_a_lock_that_cannot_apply_to_this_container_does_not_empty_it(): + """The narrowing in `_applicable`, from the operator's side. + + A constraint cannot know which container index it is packing, so a lock read from one + container is offered to every container in the solve. Without the reach test, a lock + whose box does not fit here would refuse every candidate and the container would come + back empty. It is reported as missing instead -- the load is untouched and the operator + is told their lock did not hold. + """ + request = eight_cubes() + resolve = resolve_with_locks(request, [PlacementLock(0, "cube", "LWH", (2 * MM100, 0, 0))]) + + assert not resolve.preserved + assert count(resolve.result) == 8 + assert resolve.result["status"] == "feasible" + + +# -------------------------------------------------------- the constraint itself + + +CUBE = Item.create("cube", Dimensions.mm(100, 100, 100), 0) +BOX = Container.create("box", Dimensions.mm(200, 200, 200)) + + +def context(x: int, y: int, z: int, rotation: Rotation = Rotation.LWH, + placements=()) -> ConstraintContext: + dimensions = CUBE.dimensions.rotated(rotation) + return ConstraintContext(BOX, tuple(placements), ItemInstance(CUBE, 1), + Point(x, y, z), rotation, dimensions, dimensions) + + +def test_the_constraint_can_only_ever_refuse(): + """Property 4, as the cheapest possible test of it. + + `ConstraintResult` has two shapes and a chain is an AND, so a constraint that returns + `allow()` has said nothing -- it cannot overrule the rules that refuse. Whatever this + constraint decides, it cannot be the reason a placement exists. + """ + constraint = _LockConstraint((PlacementLock(0, "cube", "LWH", (0, 0, 0)),)) + verdicts = {constraint.evaluate(context(x, 0, 0)).allowed for x in (50 * MM, MM100)} + assert verdicts == {True, False} + + +def test_a_candidate_that_misses_the_reserved_volume_is_untouched(): + """What leaves the seven unlocked cubes free: a reservation is about one box, not about + an item type and not about the order the search fills things in.""" + constraint = _LockConstraint((PlacementLock(0, "cube", "LWH", (0, 0, 0)),)) + for position in ((MM100, 0, 0), (0, MM100, 0), (MM100, MM100, MM100)): + assert constraint.evaluate(context(*position)).allowed, position + + +def test_a_candidate_overlapping_the_reserved_volume_is_refused(): + constraint = _LockConstraint((PlacementLock(0, "cube", "LWH", (0, 0, 0)),)) + assert not constraint.evaluate(context(50 * MM, 0, 0)).allowed + + +def test_the_candidate_that_is_the_lock_is_admitted(): + constraint = _LockConstraint((PlacementLock(0, "cube", "LWH", (0, 0, 0)),)) + assert constraint.evaluate(context(0, 0, 0)).allowed + + +def test_the_refusal_names_the_lock_and_carries_the_operator_code(): + constraint = _LockConstraint((PlacementLock(0, "cube", "LWH", (0, 0, 0)),)) + verdict = constraint.evaluate(context(50 * MM, 0, 0)) + + assert not verdict.allowed + assert verdict.code == LOCK_VIOLATED + assert "LWH@(0, 0, 0)" in verdict.detail + + +def test_an_unlocked_item_type_is_untouched(): + constraint = _LockConstraint((PlacementLock(0, "wedge", "LWH", (0, 0, 0)),)) + assert constraint.evaluate(context(0, 0, 0)).allowed + + +def test_a_satisfied_lock_stops_reserving(): + """The reservation lasts exactly as long as the lock is outstanding. + + Once the container holds the locked slot, the volume is defended by the ordinary overlap + rule and this constraint has nothing left to say -- which is what stops a lock set from + accumulating cost as the container fills. + """ + from packvium.models import Placement + + constraint = _LockConstraint((PlacementLock(0, "cube", "LWH", (0, 0, 0)),)) + assert not constraint.evaluate(context(50 * MM, 0, 0)).allowed + + occupied = Placement(ItemInstance(CUBE, 1), Point(0, 0, 0), Rotation.LWH, + CUBE.dimensions, Point(0, 0, 0), CUBE.dimensions) + assert constraint.evaluate(context(50 * MM, 0, 0, placements=(occupied,))).allowed + + +def test_the_orientation_is_part_of_the_lock_and_not_decoration(): + """Same origin, different rotation: not the locked placement, and it takes the reserved + volume -- so it is refused rather than accepted as close enough.""" + constraint = _LockConstraint((PlacementLock(0, "cube", "LWH", (0, 0, 0)),)) + assert not constraint.evaluate(context(0, 0, 0, Rotation.WLH)).allowed + + +# ------------------------------------------------------------ addressing a lock + + +@pytest.mark.parametrize("kwargs, message", [ + (dict(container_index=-1), "container index"), + (dict(item_type=""), "item type"), + (dict(orientation="sideways"), "orientation this engine emits"), + (dict(position_ticks=(0, 0)), "three axes"), + (dict(position_ticks=(0, 0, 1.5)), "exact integer ticks"), + (dict(position_ticks=(0, 0, True)), "exact integer ticks"), + (dict(position_ticks=(0, 0, -1)), "cannot be negative"), +]) +def test_an_address_the_engine_could_not_have_emitted_is_refused(kwargs, message): + address = dict(container_index=0, item_type="cube", orientation="LWH", + position_ticks=(0, 0, 0)) + with pytest.raises(ValueError, match=message): + PlacementLock(**{**address, **kwargs}) + + +def test_a_lock_is_read_from_the_plans_own_reference(): + """The plan is where an operator sees a placement, so it is where they point at one. + + Reading the plan's emitted reference rather than a second address format is the only + reason the two can be trusted to name the same box. + """ + from packvium.execution import build_execution_plan + + request = eight_cubes() + result = pack_from_dict(request) + plan = build_execution_plan(request, result) + + locks = locks_from_plan(plan, container_index=0, item_types=["cube"]) + + assert len(locks) == 8 + assert {lock.slot for lock in locks} == { + (placement["orientation"], + tuple(int(placement["position"][axis]["ticks"]) for axis in ("x", "y", "z"))) + for placement in result["containers"][0]["placements"] + } + + +def test_locking_a_whole_container_reproduces_it(): + """Every placement locked is the strongest form of the first property: the load the + operator approved is the load that comes back.""" + from packvium.execution import build_execution_plan + + request = eight_cubes() + approved = pack_from_dict(request) + locks = locks_from_plan(build_execution_plan(request, approved), + container_index=0, item_types=["cube"]) + + resolve = resolve_with_locks(request, locks) + + assert resolve.preserved, resolve.missing + assert placed(resolve.result) == placed(approved) + + +def test_a_lock_read_from_another_container_index_is_not_read(): + from packvium.execution import build_execution_plan + + request = eight_cubes() + plan = build_execution_plan(request, pack_from_dict(request)) + assert locks_from_plan(plan, container_index=1, item_types=["cube"]) == () + + +def test_only_the_named_item_types_become_locks(): + from packvium.execution import build_execution_plan + + request = eight_cubes() + plan = build_execution_plan(request, pack_from_dict(request)) + assert locks_from_plan(plan, container_index=0, item_types=["wedge"]) == () + + +# ---------------------------------------------------------------- the registry + + +def test_an_empty_lock_set_returns_the_callers_registry_unchanged(): + base = ExtensionRegistry() + assert lock_registry([], base) is base + + +def test_locking_does_not_drop_the_callers_own_constraints(): + """An application that locks a placement must not lose its own rules by doing so.""" + class Refuses: + def evaluate(self, context): + raise AssertionError("not evaluated in this test") + + mine = Refuses() + registry = lock_registry([PlacementLock(0, "cube", "LWH", (0, 0, 0))], + ExtensionRegistry(placement_constraints=(mine,))) + + assert registry.placement_constraints[0] is mine + assert isinstance(registry.placement_constraints[1], _LockConstraint) + + +#: The instant the policy is evaluated at. A policy carries one because reading a clock +#: here would make the same request pack differently on different days. +AS_OF = 1_704_067_200_000 + +SEGREGATED = { + "configuration": {"solver_profile": "fast", "time_limit_ms": 300000}, + "policy": { + "as_of": AS_OF, + "rules": [{ + "id": "hazmat-food-segregation", "version": 1, "effective_at": AS_OF, + "priority": 100, + "separate_tags": {"tag": "hazmat", "from_tag": "food"}, + }], + }, + "items": [ + {"id": "drum", "quantity": 1, "tags": ["hazmat"], + "dimensions": {"length": "100", "width": "100", "height": "100"}}, + {"id": "crate", "quantity": 1, "tags": ["food"], + "dimensions": {"length": "100", "width": "100", "height": "100"}}, + ], + "containers": [{"id": "box", "quantity": 2, + "inner_dimensions": {"length": "200", "width": "200", "height": "200"}}], +} + + +def test_locking_does_not_switch_off_the_policy_the_request_carries(): + """`pack_from_dict` compiles `policy` into placement constraints, and a lock arrives + through the same door. Replacing that set instead of adding to it would make locking + anything a way to turn a segregation rule off -- a lock silently loosening a rule is the + exact failure property 4 forbids, reached through the serialization layer instead of the + solver.""" + request = copy.deepcopy(SEGREGATED) + unlocked = pack_from_dict(request) + assert len(unlocked["containers"]) == 2, "the rule no longer separates; the test is void" + + resolve = resolve_with_locks(request, [PlacementLock(0, "drum", "LWH", (0, 0, 0))]) + + assert len(resolve.result["containers"]) == 2 + for container in resolve.result["containers"]: + tags = {placement["item_type"] for placement in container["placements"]} + assert tags != {"drum", "crate"}, "the lock let the segregation rule lapse" + + +# ------------------------------------------------------- the dependency direction + + +def test_the_lock_layer_is_a_caller_of_the_engine_and_not_a_part_of_it(): + """`docs/EXECUTION-PLAN.md` makes this a rule, and a grep is the cheapest test of it. + + A solver that imported the lock layer could grow a lock-aware path, which is exactly the + special case property 4 forbids. + """ + package = pathlib.Path(__file__).resolve().parents[1] / "src" / "packvium" + for module in ("packer.py", "solvers.py", "constraints.py", "validation.py"): + source = (package / module).read_text() + assert "locks" not in source.replace("blocks", "").replace("interlock", ""), module + + +def test_a_locked_result_is_still_a_validated_result(): + """The acceptance asks that independent validation cover lock preservation *and* every + existing physical constraint. It does so by construction -- `Packer` runs + `IndependentSolutionValidator` over every solve and the locked solve is an ordinary one + -- and this re-derives the cheapest of those guarantees from the placements alone, so + the claim is not resting entirely on the call graph. + """ + request = eight_cubes() + resolve = resolve_with_locks(request, [PlacementLock(0, "cube", "LWH", (MM100, 0, 0))]) + + assert resolve.result["feasibility"]["code"] == "feasible" + boxes = [ + (tuple(int(p["position"][a]["ticks"]) for a in ("x", "y", "z")), + tuple(int(p["dimensions"][d]["ticks"]) for d in ("length", "width", "height"))) + for c in resolve.result["containers"] for p in c["placements"] + ] + for index, (origin, size) in enumerate(boxes): + for other_origin, other_size in boxes[index + 1:]: + overlap = all( + origin[axis] < other_origin[axis] + other_size[axis] + and other_origin[axis] < origin[axis] + size[axis] + for axis in range(3) + ) + assert not overlap, "a locked solve returned overlapping placements" diff --git a/tests/test_outcomes.py b/tests/test_outcomes.py new file mode 100644 index 0000000..918117c --- /dev/null +++ b/tests/test_outcomes.py @@ -0,0 +1,156 @@ +"""The append-only outcome ledger, through the package's own import path. + +The ledger is the argument `evaluate_on_history` is built around, so a consumer who cannot +construct one cannot use historical replay at all. `tests/` ships inside the wheel; these +assert the ledger a consumer gets rather than the workspace shim in front of it. + +The distinction worth reading twice is `view_as_of` against `current_view`. Both fold the +same immutable events; only one of them is safe to reconstruct a past belief with. +""" + +from __future__ import annotations + +import pytest + +from packvium.outcomes import ( + DuplicateEventMismatchError, + OutcomeEvent, + OutcomeEventNotFoundError, + OutcomeEventType, + OutcomeLedger, +) + +SHIPPED = OutcomeEventType.ACTUAL_CARTON +DAMAGE = OutcomeEventType.DAMAGE + + +def shipped(event_id: str, decision_id: str, at: int, carton: str = "box-a", **extra): + return OutcomeEvent(event_id=event_id, decision_id=decision_id, event_type=SHIPPED, + payload={"carton_id": carton, "catalog_version": 1}, + recorded_at=at, **extra) + + +class TestRecordingIsAppendOnlyAndIdempotent: + def test_a_recorded_event_comes_back_for_its_decision(self): + ledger = OutcomeLedger() + ledger.record(shipped("e1", "d1", 100)) + assert [e.event_id for e in ledger.events_for_decision("d1")] == ["e1"] + + def test_an_unknown_decision_has_an_empty_history_rather_than_raising(self): + assert OutcomeLedger().events_for_decision("never-shipped") == () + + def test_recording_the_identical_event_twice_is_a_no_op(self): + """Duplicate delivery is a fact of every queue. Repeating the same event must not + double it, and must not be an error either.""" + ledger = OutcomeLedger() + first = ledger.record(shipped("e1", "d1", 100)) + again = ledger.record(shipped("e1", "d1", 100)) + assert again is first + assert len(ledger.events_for_decision("d1")) == 1 + + def test_reusing_an_event_id_for_different_content_is_refused(self): + # A duplicate delivery repeats an event; it does not carry a different one wearing + # the same id. Accepting that would make the ledger's history unreadable. + ledger = OutcomeLedger() + ledger.record(shipped("e1", "d1", 100, carton="box-a")) + with pytest.raises(DuplicateEventMismatchError, match="different content"): + ledger.record(shipped("e1", "d1", 100, carton="box-b")) + + def test_events_come_back_in_recorded_order(self): + ledger = OutcomeLedger() + for index, at in enumerate((300, 100, 200), start=1): + ledger.record(shipped(f"e{index}", "d1", at)) + assert [e.event_id for e in ledger.events_for_decision("d1")] == ["e1", "e2", "e3"] + + def test_decisions_are_kept_apart(self): + ledger = OutcomeLedger() + ledger.record(shipped("e1", "d1", 100)) + ledger.record(shipped("e2", "d2", 100)) + assert [e.event_id for e in ledger.events_for_decision("d1")] == ["e1"] + assert [e.event_id for e in ledger.events_for_decision("d2")] == ["e2"] + + +class TestACorrectionSupersedesRatherThanOverwrites: + def _corrected(self): + ledger = OutcomeLedger() + ledger.record(shipped("e1", "d1", 100, carton="box-a")) + ledger.record(shipped("e2", "d1", 200, carton="box-b", supersedes="e1")) + return ledger + + def test_the_correction_replaces_the_original_in_the_current_view(self): + view = self._corrected().current_view("d1") + assert [e.event_id for e in view] == ["e2"] + + def test_the_original_is_still_in_the_unfolded_history(self): + """"Append-only" is the whole claim. A correction that erased what it corrected + would make the ledger a mutable store wearing an immutable name.""" + assert [e.event_id for e in self._corrected().events_for_decision("d1")] == ["e1", "e2"] + + def test_superseding_an_event_the_ledger_has_never_seen_is_refused(self): + ledger = OutcomeLedger() + with pytest.raises(OutcomeEventNotFoundError, match="unknown event"): + ledger.record(shipped("e2", "d1", 200, supersedes="ghost")) + + def test_an_event_cannot_supersede_itself(self): + with pytest.raises(ValueError, match="cannot supersede itself"): + shipped("e1", "d1", 100, supersedes="e1") + + +class TestTimeTravelIsNarrowerThanTheCurrentView: + """`view_as_of` exists because `current_view` folds every correction ever recorded. + Using the latter to reconstruct a past belief pulls later knowledge backward through + time, and a holdout score built that way reports a backtest nobody could have run.""" + + def _ledger(self): + ledger = OutcomeLedger() + ledger.record(shipped("e1", "d1", 100, carton="box-a")) + ledger.record(shipped("e2", "d1", 300, carton="box-b", supersedes="e1")) + return ledger + + def test_before_anything_was_recorded_the_view_is_empty(self): + assert self._ledger().view_as_of("d1", 50) == () + + def test_between_the_event_and_its_correction_the_original_still_stands(self): + view = self._ledger().view_as_of("d1", 200) + assert [e.event_id for e in view] == ["e1"] + assert view[0].payload["carton_id"] == "box-a" + + def test_after_the_correction_the_correction_applies(self): + view = self._ledger().view_as_of("d1", 400) + assert [e.event_id for e in view] == ["e2"] + + def test_the_boundary_is_strictly_before(self): + # `at` is the instant the question is asked; an event recorded at that same + # instant is not yet part of what was believed when it was asked. + assert self._ledger().view_as_of("d1", 100) == () + + def test_the_current_view_disagrees_with_the_past_one_on_purpose(self): + ledger = self._ledger() + assert [e.event_id for e in ledger.view_as_of("d1", 200)] == ["e1"] + assert [e.event_id for e in ledger.current_view("d1")] == ["e2"] + + def test_a_negative_instant_is_refused(self): + with pytest.raises(ValueError, match="as-of time cannot be negative"): + self._ledger().view_as_of("d1", -1) + + +class TestSeveralEventKindsCoexistOnOneDecision: + def test_a_shipment_and_its_damage_both_stand(self): + ledger = OutcomeLedger() + ledger.record(shipped("e1", "d1", 100)) + ledger.record(OutcomeEvent(event_id="e2", decision_id="d1", event_type=DAMAGE, + payload={"reason_code": "crushed_corner", + "severity": "minor"}, + recorded_at=200)) + kinds = sorted(e.event_type.value for e in ledger.current_view("d1")) + assert kinds == ["actual_carton", "damage"] + + +class TestAnEventMustNameItsDecision: + def test_an_event_without_a_decision_id_is_refused(self): + # The decision id is the only index the ledger has; an event without one could + # never be read back. + with pytest.raises(ValueError, match="decision_id is required"): + OutcomeEvent(event_id="e1", decision_id="", event_type=SHIPPED, + payload={"carton_id": "box-a", "catalog_version": 1}, + recorded_at=100) diff --git a/tests/test_pareto.py b/tests/test_pareto.py new file mode 100644 index 0000000..d9789af --- /dev/null +++ b/tests/test_pareto.py @@ -0,0 +1,182 @@ +"""Dominance and the report built from it, through the package's own import path. + +`packvium.pareto` is exercised in the workspace by the benchmark comparators, which reach +it through a re-export shim. That proves the shim works; it says nothing about the module +a consumer imports, and `tests/` ships inside the wheel — so a consumer running the +shipped suite exercised none of this until these tests existed. + +What is asserted here is the module's one job: **never collapse a trade-off into a single +number.** A result dominates another only when it is no worse on every named axis and +strictly better on at least one, and when two results survive that, the report says so +instead of picking. +""" + +from __future__ import annotations + +import pytest + +from packvium.pareto import ( + CandidateResult, + InconsistentAxesError, + NonFiniteMetricError, + ParetoReportError, + ProfileReport, + dominates, + generate_report, +) + +#: Cost is cheaper-is-better, utilisation is higher-is-better. Stating both directions is +#: mandatory in this module, so every test carries them. +DIRECTIONS = {"cost": False, "utilisation": True} + + +def candidate(engine: str, cost: float, utilisation: float, profile: str = "balanced"): + return CandidateResult(profile=profile, engine=engine, + metrics={"cost": cost, "utilisation": utilisation}) + + +class TestDominance: + def test_better_on_both_axes_dominates(self): + assert dominates({"cost": 8.0, "utilisation": 0.9}, + {"cost": 10.0, "utilisation": 0.7}, DIRECTIONS) + + def test_worse_on_both_axes_does_not(self): + assert not dominates({"cost": 12.0, "utilisation": 0.6}, + {"cost": 10.0, "utilisation": 0.7}, DIRECTIONS) + + def test_a_genuine_trade_off_dominates_in_neither_direction(self): + """Cheaper but sparser. This is the case the whole module exists for: a blended + score would rank these, and ranking them is a claim about the caller's priorities + that nothing in the metrics supports.""" + cheap = {"cost": 8.0, "utilisation": 0.6} + dense = {"cost": 12.0, "utilisation": 0.9} + assert not dominates(cheap, dense, DIRECTIONS) + assert not dominates(dense, cheap, DIRECTIONS) + + def test_identical_metrics_dominate_in_neither_direction(self): + # "No worse on every axis" is satisfied; "strictly better somewhere" is not. + same = {"cost": 10.0, "utilisation": 0.8} + assert not dominates(same, dict(same), DIRECTIONS) + + def test_equal_on_one_axis_and_better_on_the_other_dominates(self): + assert dominates({"cost": 10.0, "utilisation": 0.9}, + {"cost": 10.0, "utilisation": 0.8}, DIRECTIONS) + + def test_direction_is_read_from_the_map_rather_than_guessed(self): + """The same two candidates, with the meaning of `cost` inverted. Nothing about the + numbers says which way is better, which is why the map has no default.""" + low, high = {"cost": 8.0}, {"cost": 12.0} + assert dominates(low, high, {"cost": False}) + assert dominates(high, low, {"cost": True}) + + def test_an_axis_missing_from_either_side_is_refused(self): + with pytest.raises(InconsistentAxesError, match="utilisation"): + dominates({"cost": 8.0}, {"cost": 10.0, "utilisation": 0.7}, DIRECTIONS) + with pytest.raises(InconsistentAxesError, match="utilisation"): + dominates({"cost": 8.0, "utilisation": 0.9}, {"cost": 10.0}, DIRECTIONS) + + def test_an_axis_absent_from_the_direction_map_is_simply_not_compared(self): + # Extra metrics are not an error; they are data the caller did not ask to rank on. + assert dominates({"cost": 8.0, "weight": 99.0}, {"cost": 10.0, "weight": 1.0}, + {"cost": False}) + + def test_a_nan_on_either_side_is_refused_rather_than_counted_equal(self): + """`NaN > x` and `NaN < x` are both false, so an unguarded comparison silently + reports the axis as a tie and lets garbage onto the frontier.""" + with pytest.raises(NonFiniteMetricError, match="left candidate.*cost"): + dominates({"cost": float("nan")}, {"cost": 1.0}, {"cost": False}) + with pytest.raises(NonFiniteMetricError, match="right candidate.*cost"): + dominates({"cost": 1.0}, {"cost": float("nan")}, {"cost": False}) + + def test_infinities_compare_as_the_ends_of_the_number_line(self): + """Deliberately allowed: a caller encoding "unpriceable" as an infinite cost gets + the answer they mean.""" + assert dominates({"cost": 5.0}, {"cost": float("inf")}, {"cost": False}) + assert not dominates({"cost": float("inf")}, {"cost": 5.0}, {"cost": False}) + assert not dominates({"cost": float("inf")}, {"cost": float("inf")}, {"cost": False}) + + +class TestTheReport: + def test_one_dominant_candidate_becomes_the_named_winner(self): + reports = generate_report( + [candidate("python", 10.0, 0.7), candidate("php", 8.0, 0.9)], DIRECTIONS) + assert len(reports) == 1 + assert reports[0].winner == "php" + assert reports[0].pareto_optimal == ("php",) + assert reports[0].dominated == ("python",) + + def test_a_trade_off_leaves_no_winner_and_lists_both(self): + reports = generate_report( + [candidate("python", 8.0, 0.6), candidate("php", 12.0, 0.9)], DIRECTIONS) + assert reports[0].winner is None + assert reports[0].pareto_optimal == ("php", "python") + assert reports[0].dominated == () + + def test_a_single_candidate_is_optimal_by_default(self): + """Nothing can dominate it, so it is on the frontier. Worth pinning because it is + easy to misread as evidence that it beat something.""" + reports = generate_report([candidate("python", 10.0, 0.7)], DIRECTIONS) + assert reports[0].pareto_optimal == ("python",) + assert reports[0].winner == "python" + + def test_three_candidates_split_into_frontier_and_dominated(self): + reports = generate_report([ + candidate("cheap", 8.0, 0.6), + candidate("dense", 12.0, 0.9), + candidate("worse", 14.0, 0.5), + ], DIRECTIONS) + assert reports[0].pareto_optimal == ("cheap", "dense") + assert reports[0].dominated == ("worse",) + assert reports[0].winner is None + + def test_each_profile_gets_its_own_report_sorted_by_name(self): + reports = generate_report([ + candidate("python", 10.0, 0.7, profile="quality"), + candidate("php", 8.0, 0.9, profile="fast"), + ], DIRECTIONS) + assert [r.profile for r in reports] == ["fast", "quality"] + + def test_frontier_and_dominated_are_each_sorted(self): + """Report order cannot depend on input order, or two runs of the same comparison + would print differently.""" + forward = generate_report( + [candidate("zeta", 8.0, 0.6), candidate("alpha", 12.0, 0.9)], DIRECTIONS) + reverse = generate_report( + [candidate("alpha", 12.0, 0.9), candidate("zeta", 8.0, 0.6)], DIRECTIONS) + assert forward[0].pareto_optimal == reverse[0].pareto_optimal == ("alpha", "zeta") + + def test_no_candidates_produces_no_reports(self): + assert generate_report([], DIRECTIONS) == () + + def test_two_results_for_one_engine_in_one_profile_are_refused(self): + with pytest.raises(ParetoReportError, match="more than one result"): + generate_report([candidate("python", 10.0, 0.7), candidate("python", 8.0, 0.9)], + DIRECTIONS) + + def test_an_empty_direction_map_is_refused(self): + # With no axes there is no dominance, so every candidate would be "optimal". + with pytest.raises(ValueError, match="at least one metric axis"): + generate_report([candidate("python", 10.0, 0.7)], {}) + + def test_a_nan_candidate_cannot_reach_the_frontier(self): + """Measured before the refusal existed: an all-`NaN` candidate came back optimal + beside a clean one, because nothing could dominate it.""" + with pytest.raises(NonFiniteMetricError): + generate_report([candidate("python", 10.0, 0.7), + candidate("broken", float("nan"), float("nan"))], DIRECTIONS) + + +class TestTheReportTypeGuardsItsOwnClaims: + def test_a_winner_must_be_on_the_frontier(self): + with pytest.raises(ValueError, match="must be one of the Pareto-optimal"): + ProfileReport(profile="fast", pareto_optimal=("php",), dominated=("python",), + winner="python") + + def test_a_winner_cannot_be_named_when_two_survive(self): + with pytest.raises(ValueError, match="exactly one candidate"): + ProfileReport(profile="fast", pareto_optimal=("php", "python"), dominated=(), + winner="php") + + def test_no_winner_alongside_a_frontier_of_two_is_well_formed(self): + report = ProfileReport(profile="fast", pareto_optimal=("php", "python"), dominated=()) + assert report.winner is None diff --git a/tests/test_recommendations.py b/tests/test_recommendations.py new file mode 100644 index 0000000..653a27f --- /dev/null +++ b/tests/test_recommendations.py @@ -0,0 +1,204 @@ +"""Proposing a change, and the single route from a proposal to a published one. + +`tests/` ships inside the wheel, so these import `packvium.recommendations` the way a +consumer does. + +Two properties carry the module. A proposal is built from a **paired** comparison, so an +order only one arm could pack contributes to neither mean and cannot manufacture an +improvement out of cohort mix. And a `Recommendation` is never itself a mutation: it is +handed no registry, and the only path to one is an explicit approval call. +""" + +from __future__ import annotations + +import pytest + +from packvium.recommendations import ( + ApprovalRecord, + CatalogRegistry, + CatalogSnapshot, + ExpectedDelta, + MismatchedOrderCorpusError, + PolicyAction, + PolicyPredicate, + PolicyRegistry, + PolicyScope, + Recommendation, + approve, + approve_catalog, + approve_policy, + propose_recommendation, +) +from packvium.commerce.policy import PolicyOperator +from packvium.simulation import OrderRunResult, ScenarioResult, ScenarioVersionPin + +PIN = ScenarioVersionPin(catalog_version=1) +CONSTRAINTS = ("no placement the validator rejects",) +ROLLBACK = "republish catalog version 1" + + +def scenario(name: str, table: dict[str, tuple[float, float] | None]) -> ScenarioResult: + runs = [] + for order_id, measured in table.items(): + if measured is None: + runs.append(OrderRunResult(order_id=order_id, succeeded=False, + failure_reason="no carton fits")) + else: + cost, utilisation = measured + runs.append(OrderRunResult(order_id=order_id, succeeded=True, + metrics={"cost": cost, "utilisation": utilisation})) + return ScenarioResult(scenario_id=name, version_pin=PIN, + order_ids=tuple(table), runs=tuple(runs)) + + +def propose(baseline, treatment, **overrides): + fields = {"constraints": CONSTRAINTS, "rollback_plan": ROLLBACK, + "minimum_cohort_size": 1, "minimum_confidence": 0.0} + fields.update(overrides) + return propose_recommendation("rec-1", "cheaper cartons", baseline, treatment, **fields) + + +class TestAProposalIsBuiltFromPairedEvidence: + def test_the_deltas_are_the_means_of_the_paired_orders(self): + proposal = propose(scenario("b", {"a": (10.0, 0.6), "b": (20.0, 0.8)}), + scenario("t", {"a": (8.0, 0.7), "b": (16.0, 0.9)})) + assert proposal is not None + deltas = {d.metric: (d.baseline_mean, d.treatment_mean) + for d in proposal.expected_deltas} + assert deltas["cost"] == (15.0, 12.0) + assert deltas["utilisation"] == (0.7, 0.8) + + def test_deltas_are_sorted_by_metric_name(self): + # Report order cannot depend on dict insertion order, or two identical + # comparisons would print differently. + proposal = propose(scenario("b", {"a": (10.0, 0.6)}), scenario("t", {"a": (8.0, 0.7)})) + assert [d.metric for d in proposal.expected_deltas] == ["cost", "utilisation"] + + def test_an_order_only_one_arm_packed_is_excluded_from_both_means(self): + """The paired rule. Averaging each arm's survivors separately would let the + difference in *who survived* look like a difference in performance.""" + proposal = propose(scenario("b", {"a": (10.0, 0.6), "b": (100.0, 0.1)}), + scenario("t", {"a": (8.0, 0.7), "b": None})) + assert proposal.supporting_order_ids == ("a",) + deltas = {d.metric: d.baseline_mean for d in proposal.expected_deltas} + assert deltas["cost"] == 10.0 + + def test_confidence_is_the_paired_share_of_the_whole_corpus(self): + proposal = propose( + scenario("b", {"a": (10.0, 0.6), "b": (10.0, 0.6), "c": (10.0, 0.6), + "d": (10.0, 0.6)}), + scenario("t", {"a": (8.0, 0.7), "b": (8.0, 0.7), "c": (8.0, 0.7), "d": None})) + assert proposal.confidence == 0.75 + + def test_only_metrics_both_arms_reported_become_deltas(self): + baseline = ScenarioResult( + scenario_id="b", version_pin=PIN, order_ids=("a",), + runs=(OrderRunResult(order_id="a", succeeded=True, + metrics={"cost": 10.0, "weight": 5.0}),)) + treatment = ScenarioResult( + scenario_id="t", version_pin=PIN, order_ids=("a",), + runs=(OrderRunResult(order_id="a", succeeded=True, + metrics={"cost": 8.0, "volume": 3.0}),)) + proposal = propose(baseline, treatment) + assert [d.metric for d in proposal.expected_deltas] == ["cost"] + + +class TestAProposalIsWithheldRatherThanHedged: + def test_a_cohort_below_the_floor_produces_nothing(self): + assert propose(scenario("b", {"a": (10.0, 0.6)}), scenario("t", {"a": (8.0, 0.7)}), + minimum_cohort_size=2) is None + + def test_a_confidence_below_the_floor_produces_nothing(self): + assert propose(scenario("b", {"a": (10.0, 0.6), "b": (10.0, 0.6)}), + scenario("t", {"a": (8.0, 0.7), "b": None}), + minimum_confidence=0.9) is None + + def test_arms_that_share_no_metric_produce_nothing(self): + """Full cohort, full confidence, and still no axis both sides measured. `None` + rather than a recommendation carrying an empty delta list.""" + baseline = ScenarioResult( + scenario_id="b", version_pin=PIN, order_ids=("a",), + runs=(OrderRunResult(order_id="a", succeeded=True, metrics={"cost": 10.0}),)) + treatment = ScenarioResult( + scenario_id="t", version_pin=PIN, order_ids=("a",), + runs=(OrderRunResult(order_id="a", succeeded=True, + metrics={"utilisation": 0.9}),)) + assert propose(baseline, treatment) is None + + def test_two_different_corpora_are_refused(self): + with pytest.raises(MismatchedOrderCorpusError): + propose(scenario("b", {"a": (10.0, 0.6)}), scenario("t", {"z": (8.0, 0.7)})) + + +class TestPublishingIsASeparateExplicitAct: + def _recommendation(self): + return Recommendation( + recommendation_id="rec-1", proposal="cheaper cartons", + supporting_order_ids=("a",), + expected_deltas=(ExpectedDelta(metric="cost", baseline_mean=10.0, + treatment_mean=8.0),), + confidence=1.0, constraints=CONSTRAINTS, rollback_plan=ROLLBACK) + + def test_approve_calls_the_injected_publisher_exactly_once(self): + calls = [] + + def publish() -> int: + calls.append(1) + return 7 + + record = approve(self._recommendation(), at=1_000, publish=publish) + assert calls == [1] + assert record == ApprovalRecord(recommendation_id="rec-1", approved_at=1_000, + published_version=7) + + def test_a_catalog_approval_publishes_through_the_real_registry(self): + registry = CatalogRegistry(catalog_id="cartons") + record = approve_catalog(self._recommendation(), registry, CatalogSnapshot(), + approved_at=1_000, effective_at=2_000) + assert record.published_version == 1 + assert record.recommendation_id == "rec-1" + + def test_a_second_catalog_approval_appends_a_new_version(self): + # Append-only: publishing again never replaces what was published before. + registry = CatalogRegistry(catalog_id="cartons") + first = approve_catalog(self._recommendation(), registry, CatalogSnapshot(), + approved_at=1_000, effective_at=2_000) + second = approve_catalog(self._recommendation(), registry, CatalogSnapshot(), + approved_at=3_000, effective_at=4_000) + assert (first.published_version, second.published_version) == (1, 2) + + def test_a_policy_approval_publishes_a_versioned_rule(self): + registry = PolicyRegistry() + record = approve_policy( + self._recommendation(), registry, rule_id="no-hazmat-air", + scope=PolicyScope.HAZMAT, action=PolicyAction.REJECT, + predicates=(PolicyPredicate(scope=PolicyScope.HAZMAT, field="class", + operator=PolicyOperator.EQUALS, value="1.4"),), + priority=10, approved_at=1_000, effective_at=2_000) + assert record.published_version == 1 + assert record.recommendation_id == "rec-1" + + def test_a_proposal_is_handed_no_registry_of_its_own(self): + """Structural, not a convention: `propose_recommendation` has no registry + parameter, so the proposal path cannot reach a mutation even by mistake.""" + import inspect + parameters = inspect.signature(propose_recommendation).parameters + assert not [name for name in parameters if "registry" in name.lower()] + + +class TestTheDeltaItself: + def test_delta_is_treatment_minus_baseline(self): + """Signed on purpose: whether a negative delta is good depends on the axis, and + this type deliberately does not know.""" + cheaper = ExpectedDelta(metric="cost", baseline_mean=10.0, treatment_mean=8.0) + denser = ExpectedDelta(metric="utilisation", baseline_mean=0.7, treatment_mean=0.9) + assert cheaper.delta == -2.0 + assert denser.delta == pytest.approx(0.2) + + def test_a_recommendation_must_report_at_least_one_delta(self): + # A proposal with nothing measured is advice without evidence. + with pytest.raises(ValueError, match="at least one expected delta"): + Recommendation(recommendation_id="rec-1", proposal="p", + supporting_order_ids=("a",), expected_deltas=(), + confidence=1.0, constraints=CONSTRAINTS, + rollback_plan=ROLLBACK) diff --git a/tests/test_simulation.py b/tests/test_simulation.py new file mode 100644 index 0000000..80e0b6f --- /dev/null +++ b/tests/test_simulation.py @@ -0,0 +1,208 @@ +"""Scenario runs and the order-by-order comparison built from them. + +`tests/` ships inside the wheel, so these import `packvium.simulation` the way a consumer +does rather than reaching it through the workspace shim. + +The comparison's one rule: a scenario is compared **per order**, never as a blended delta. +An aggregate would hide which orders improved and which paid for the improvement, and that +is exactly the information a caller needs before publishing a catalog change. +""" + +from __future__ import annotations + +import pytest + +from packvium.simulation import ( + MismatchedOrderCorpusError, + ScenarioError, + OrderRunResult, + ScenarioResult, + ScenarioVersionPin, + compare_scenarios, + run_scenario, +) + +PIN = ScenarioVersionPin(catalog_version=1) +DIRECTIONS = {"cost": False, "utilisation": True} + + +def arm(**by_order): + """An evaluator that replays a table. `None` means the arm could not pack that order.""" + def evaluate(order_id: str, version_pin: ScenarioVersionPin) -> OrderRunResult: + measured = by_order[order_id] + if measured is None: + return OrderRunResult(order_id=order_id, succeeded=False, + failure_reason="no carton fits") + cost, utilisation = measured + return OrderRunResult(order_id=order_id, succeeded=True, + metrics={"cost": cost, "utilisation": utilisation}) + return evaluate + + +class TestRunningAScenario: + def test_every_order_is_evaluated_in_the_given_sequence(self): + seen = [] + + def evaluate(order_id, version_pin): + seen.append(order_id) + return OrderRunResult(order_id=order_id, succeeded=True, metrics={"cost": 1.0}) + + result = run_scenario("s", PIN, ("b", "a", "c"), evaluate) + assert seen == ["b", "a", "c"] + assert result.order_ids == ("b", "a", "c") + + def test_the_pin_reaches_the_evaluator_unchanged(self): + """A scenario replays from stored versions, so the pin is what the evaluator is + expected to resolve its catalog against.""" + seen = [] + + def evaluate(order_id, version_pin): + seen.append(version_pin) + return OrderRunResult(order_id=order_id, succeeded=True, metrics={"cost": 1.0}) + + run_scenario("s", PIN, ("a",), evaluate) + assert seen == [PIN] + + def test_the_same_arguments_reproduce_an_equal_result(self): + # Reproducibility is this function's own determinism, not a separate mechanism. + first = run_scenario("s", PIN, ("a", "b"), arm(a=(10.0, 0.7), b=(12.0, 0.8))) + second = run_scenario("s", PIN, ("a", "b"), arm(a=(10.0, 0.7), b=(12.0, 0.8))) + assert first == second + + def test_a_failed_order_is_carried_rather_than_dropped(self): + result = run_scenario("s", PIN, ("a", "b"), arm(a=(10.0, 0.7), b=None)) + assert [r.succeeded for r in result.runs] == [True, False] + assert result.runs[1].failure_reason == "no carton fits" + + def test_metrics_are_read_only_once_the_run_exists(self): + run = OrderRunResult(order_id="a", succeeded=True, metrics={"cost": 1.0}) + with pytest.raises(TypeError): + run.metrics["cost"] = 2.0 + + +class TestARunGuardsItsOwnClaims: + def test_a_succeeded_run_must_report_a_metric(self): + # "It worked" with nothing measured is not a comparable result. + with pytest.raises(ValueError, match="at least one metric"): + OrderRunResult(order_id="a", succeeded=True, metrics={}) + + def test_a_failed_run_must_say_why(self): + with pytest.raises(ValueError, match="failure_reason"): + OrderRunResult(order_id="a", succeeded=False) + + @pytest.mark.parametrize("value", [float("nan"), float("inf"), float("-inf")]) + def test_a_non_finite_metric_is_refused_at_the_run(self, value): + """Stricter than the Pareto module, which accepts infinities: a *measurement* of + infinite cost is a broken evaluator, not a caller's encoding of "unpriceable".""" + with pytest.raises(ValueError, match="finite"): + OrderRunResult(order_id="a", succeeded=True, metrics={"cost": value}) + + def test_a_boolean_is_not_a_metric(self): + # `True` is an `int` in Python, which is exactly why this needs its own guard. + with pytest.raises(ValueError, match="finite"): + OrderRunResult(order_id="a", succeeded=True, metrics={"cost": True}) + + +class TestComparingTwoScenarios: + def _pair(self, baseline_table, treatment_table, orders=("a", "b")): + return (run_scenario("base", PIN, orders, arm(**baseline_table)), + run_scenario("treat", PIN, orders, arm(**treatment_table))) + + def test_one_report_per_order(self): + base, treat = self._pair({"a": (10.0, 0.7), "b": (12.0, 0.8)}, + {"a": (8.0, 0.9), "b": (11.0, 0.85)}) + reports = compare_scenarios(base, treat, DIRECTIONS) + assert [r.profile for r in reports] == ["a", "b"] + + def test_a_dominant_arm_is_named_per_order(self): + base, treat = self._pair({"a": (10.0, 0.7), "b": (12.0, 0.8)}, + {"a": (8.0, 0.9), "b": (14.0, 0.6)}) + winners = {r.profile: r.winner for r in compare_scenarios(base, treat, DIRECTIONS)} + assert winners == {"a": "treatment", "b": "baseline"} + + def test_a_real_trade_off_on_one_order_leaves_that_order_without_a_winner(self): + """The reason there is no aggregate: order `b` genuinely trades cost for density, + and no single number can report that without deciding it.""" + base, treat = self._pair({"a": (10.0, 0.7), "b": (12.0, 0.8)}, + {"a": (8.0, 0.9), "b": (14.0, 0.9)}) + reports = {r.profile: r for r in compare_scenarios(base, treat, DIRECTIONS)} + assert reports["a"].winner == "treatment" + assert reports["b"].winner is None + assert reports["b"].pareto_optimal == ("baseline", "treatment") + + def test_an_order_only_one_arm_could_pack_reports_that_arm_alone(self): + base, treat = self._pair({"a": (10.0, 0.7), "b": (12.0, 0.8)}, + {"a": (8.0, 0.9), "b": None}) + reports = {r.profile: r for r in compare_scenarios(base, treat, DIRECTIONS)} + assert reports["b"].pareto_optimal == ("baseline",) + # Named winner, and not a win: there was only one candidate to be optimal. + assert reports["b"].winner == "baseline" + + def test_an_order_neither_arm_could_pack_produces_no_report(self): + base, treat = self._pair({"a": (10.0, 0.7), "b": None}, {"a": (8.0, 0.9), "b": None}) + assert [r.profile for r in compare_scenarios(base, treat, DIRECTIONS)] == ["a"] + + def test_two_different_corpora_are_refused_rather_than_intersected(self): + """Comparing the overlap would let cohort mix manufacture a difference that + neither arm actually produced.""" + base = run_scenario("base", PIN, ("a", "b"), arm(a=(10.0, 0.7), b=(12.0, 0.8))) + treat = run_scenario("treat", PIN, ("a", "c"), arm(a=(8.0, 0.9), c=(9.0, 0.9))) + with pytest.raises(MismatchedOrderCorpusError, match="identical order corpus"): + compare_scenarios(base, treat, DIRECTIONS) + + def test_the_same_orders_in_a_different_sequence_are_refused(self): + base = run_scenario("base", PIN, ("a", "b"), arm(a=(10.0, 0.7), b=(12.0, 0.8))) + treat = run_scenario("treat", PIN, ("b", "a"), arm(a=(8.0, 0.9), b=(11.0, 0.9))) + with pytest.raises(MismatchedOrderCorpusError): + compare_scenarios(base, treat, DIRECTIONS) + + def test_the_arms_may_be_pinned_to_different_catalog_versions(self): + # That is the point of a comparison: one pin per arm, the same orders. + treatment_pin = ScenarioVersionPin(catalog_version=2) + base = run_scenario("base", PIN, ("a",), arm(a=(10.0, 0.7))) + treat = ScenarioResult(scenario_id="treat", version_pin=treatment_pin, + order_ids=("a",), + runs=(OrderRunResult(order_id="a", succeeded=True, + metrics={"cost": 8.0, + "utilisation": 0.9}),)) + assert compare_scenarios(base, treat, DIRECTIONS)[0].winner == "treatment" + + +class TestAScenarioReportsItsOwnConfidence: + """asks for confidence and invalid-run counts to be *visible*. They are + properties of the result rather than something each caller re-derives, which is what + keeps two callers from computing them differently.""" + + def _mixed(self): + return run_scenario("s", PIN, ("a", "b", "c", "d"), + arm(a=(10.0, 0.7), b=None, c=(12.0, 0.8), d=(9.0, 0.9))) + + def test_the_counts_partition_the_runs(self): + result = self._mixed() + assert (result.succeeded_count, result.failed_count) == (3, 1) + assert result.succeeded_count + result.failed_count == len(result.runs) + + def test_confidence_is_the_usable_share(self): + assert self._mixed().confidence == 0.75 + + def test_a_wholly_successful_scenario_has_full_confidence(self): + result = run_scenario("s", PIN, ("a",), arm(a=(10.0, 0.7))) + assert result.confidence == 1.0 + assert result.failed_count == 0 + + def test_a_wholly_failed_scenario_has_no_confidence(self): + result = run_scenario("s", PIN, ("a",), arm(a=None)) + assert result.confidence == 0.0 + assert result.succeeded_count == 0 + + +class TestReachingOneOrdersRawRun: + def test_the_run_for_a_known_order_comes_back(self): + result = run_scenario("s", PIN, ("a", "b"), arm(a=(10.0, 0.7), b=(12.0, 0.8))) + assert result.raw_artifact("b").metrics["cost"] == 12.0 + + def test_an_unknown_order_is_refused_by_name(self): + # Returning `None` would push the mistake downstream into a metrics lookup. + result = run_scenario("s", PIN, ("a",), arm(a=(10.0, 0.7))) + with pytest.raises(ScenarioError, match="no run recorded for order 'zzz'"): + result.raw_artifact("zzz") diff --git a/tests/test_solvers.py b/tests/test_solvers.py index 5c393f8..4e4fa9a 100644 --- a/tests/test_solvers.py +++ b/tests/test_solvers.py @@ -538,6 +538,18 @@ def test_two_groups_stay_separate(): assert group_batches([*left, *right]) == [left, right] +def test_interleaved_groups_keep_first_occurrence_and_member_order(): + first = instances("a", 10, 10, 10, quantity=2, group="00") + second = instances("b", 10, 10, 10, quantity=2, group="0") + third = instances("c", 10, 10, 10, quantity=2, group="1") + loose = instances("loose", 10, 10, 10, quantity=2) + ordered = [second[1], loose[0], third[0], first[1], second[0], first[0], loose[1], third[1]] + assert group_batches(ordered) == [ + (second[1], second[0]), (loose[0],), third, + (first[1], first[0]), (loose[1],), + ] + + # --------------------------------------------------------------- maximal spaces def space(x, y, z, length, width, height) -> Space: @@ -719,6 +731,45 @@ def test_a_wider_beam_returns_a_sorted_prefix(): assert [c.score for c in top] == sorted(c.score for c in top) +@pytest.mark.parametrize("width", [1, 2, 3, 7, 128]) +@pytest.mark.parametrize("clearance", [0, 1]) +def test_bounded_candidates_preserve_full_sort_ties_work_and_trace(width, clearance): + from packvium.constraints import ConstraintResult + from packvium.trace import use_trace + + class RecordingConstraint: + def __init__(self): + self.calls = [] + + def evaluate(self, context): + self.calls.append((context.point, context.rotation)) + return ConstraintResult.allow() + + config, constraints = config_and_constraints(clearance=Length.mm(clearance)) + state = ContainerState(Container.create("c", Dimensions.mm(100, 100, 100)), 1) + item, = instances("a", 30, 20, 10) + # On a square base, swapped horizontal rotations have exactly equal scores. + # Later x origins improve the score, so the bounded selection must also evict. + points = [Point(Length.mm(x).ticks, 0, 0) for x in (95, 40, 20, 0)] + rule = RecordingConstraint() + full_stats, bounded_stats = SearchStats(), SearchStats() + full_trace, bounded_trace = [], [] + with use_trace(full_trace.append): + every = find_candidates(state, item, config, (*constraints, rule), full_stats, + generous(), None, points=points) + full_calls = rule.calls[:] + rule.calls.clear() + with use_trace(bounded_trace.append): + selected = find_candidates(state, item, config, (*constraints, rule), bounded_stats, + generous(), width, points=points) + + assert len({candidate.score for candidate in every}) < len(every) + assert selected == every[:width] + assert bounded_stats == full_stats + assert bounded_trace == full_trace + assert rule.calls == full_calls + + def test_an_item_that_cannot_fit_yields_no_candidate(): config, constraints = config_and_constraints() state = ContainerState(Container.create("c", Dimensions.mm(10, 10, 10)), 1) @@ -794,6 +845,20 @@ def test_the_lattice_admits_a_declared_type_that_is_a_rotation_of_another(): assert GridSolver().supports([*instances("a", 6, 12, 20, quantity=11), *swapped]) +def test_lattice_profile_checks_survive_interleaved_aliases_and_reuse(): + first = instances("same-id", 6, 12, 20, quantity=2) + alias = instances("alias", 12, 6, 20, quantity=2) + # Direct solver callers can supply different immutable objects sharing an id. + # Identity reuse must never become an id-based assumption about their geometry. + different = instances("same-id", 7, 12, 20) + ordered = [first[0], alias[0], first[1], alias[1]] + solver = GridSolver() + assert solver.supports(ordered) + assert not solver.supports([*ordered, *different]) + assert solver.supports(alias) + assert not solver.supports([]) + + def test_the_lattice_keeps_different_declared_nesting_types_out_of_one_column(): """Nesting permits physical overlap only between instances of the same item type.