Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions reviews/plat-1658-code-review.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
id: SR-1620
title: "code-review review — PLAT-1658 spec-objects-security"
type: SpecReview
analysis: code-review
scope: "agent-ix/spec-objects-security; typespec/main.tsp; spec_objects_security/manifest.yaml; spec_objects_security/schemas/ProcessingActivity*.json; tests/test_processing_activity.py; tests/test_regulated_categories.py"
review_set: subset
---

## Summary

Reviewed the ProcessingActivity schemas, manifest relation, skeleton and tests against FR-028. The required-coverage rule is advisory, so an activity without a covers edge still exits validation successfully; the record schema also permits a controller role to carry a controller list contrary to the role contract.

## Verdict

**FAIL** — one high-severity requirement mismatch and one schema contract defect remain.

## Scope

- Examined: FR-028-AC-1 through FR-028-AC-8; ProcessingActivity TypeSpec and generated schemas; manifest relation; skeleton; new focused tests and the regulated-category test adjustment.
- Ticket: PLAT-1658; PR #69; reviewed published head `84b182f01a6db11c1c72c42f4b94442afe365268`.
- Applicable AssuranceProfile: none found under `spec/`.
- Checks: schema drift passed (67 schemas match); focused tests had 19 passed and 13 failed in pre-existing `test_regulated_categories.py` cases because the activated Python Quire wheel rejects baseline `where` fields in `manifest.yaml`; Ruff and Black passed. The ProcessingActivity-focused test file passed; the combined focused run had 19 passes and 13 pre-existing regulated-category failures because the activated Python Quire wheel rejects baseline `where` fields in `manifest.yaml`.

## Findings

| ID | Severity | Summary | Refs |
| --- | --- | --- | --- |
| FND-001 | high | The required `covers` relation only emits an advisory warning: a valid-looking processing activity with no covers edge produces `processing-activity-covers-nothing` but `quire validate --okf` exits 0, violating the explicit fail behavior in AC-5. The new tests check declaration shape and only a valid edge, never a no-edge failure. | spec/functional/FR-028-processing-activity.md:61; spec_objects_security/manifest.yaml:844-851; tests/test_processing_activity.py:69-87,98-123 |
| FND-002 | medium | `ProcessingActivity.json` allows `controller` to be a string or array at the base schema, while its controller-role conditional only adds required fields and never narrows that property to a string. A controller record with a controller list therefore validates, although FR-028 requires one name-and-contact string; the frontmatter schema correctly narrows it, so the published record schema disagrees. | spec/functional/FR-028-processing-activity.md:35; spec_objects_security/schemas/ProcessingActivity.json:9-20,72-90; spec_objects_security/schemas/ProcessingActivityFrontmatter.json:98-109 |

## Evidence

For FND-001, running Quire CLI 0.46.0 / engine 0.61.0 against a bundle containing an activity with no `covers` edge returned exit 0 and emitted: `warning: ... 'PA-EMPTY' has no 'covers' edge to any connection/data_sink/part ... [processing-activity-covers-nothing]`. Quire FR-058-AC-7 and `quire-rs/src/corpus/required_relations.rs` route required-relation findings at grammar severity, advisory by default. The PR test only asserts that the manifest declares `covers`; its bundle test exercises only a positive connection edge.

For FND-002, Draft 2020-12 schema inspection shows `controller` has an `anyOf` string/array at the top level, while the controller conditional requires only `purposes`, `data_subject_categories`, and `retention`. The corresponding controller conditional in `ProcessingActivityFrontmatter.json` explicitly sets `controller.type: string`; that restriction is absent from the record schema.

## Examined Units

- `FR-028-AC-1` (examined): “A controller activity with controller details, purposes, data-subject categories, retention, home regions and a valid `covers` edge validates.”
- `FR-028-AC-2` (examined): “A processor activity with a processor name and contact, a non-empty controller list and home regions validates without `purposes`, `data_subject_categories` or `retention`.”
- `FR-028-AC-3` (examined): “A controller activity without `retention`, `purposes` or `data_subject_categories`, and a processor activity without its own `processor` identity/contact or with an empty controller list, each fail with a diagnostic naming the missing or invalid field.”
- `FR-028-AC-4` (examined): “An unknown role, an empty home-region list, or a home-region value outside the stated vocabulary fails validation.”
- `FR-028-AC-5` (examined): “A `processing_activity` with no `covers` edge fails validation; an edge to a `connection`, `data_sink` or `part` validates.”
- `FR-028-AC-6` (examined): “A `covers` edge to a `queue` fails with a wrong-kind diagnostic, and an edge to an absent target fails with a dangling-target diagnostic.”
- `FR-028-AC-7` (examined): “The module exports `processing_activity`; its generated schema and skeleton pass the module schema-drift and `quire validate` checks.”
- `FR-028-AC-8` (examined): “Omitting any optional authored field leaves it absent in the extracted record; no omitted value is populated from another field, artifact, code or runtime source.”


## Dispositions

Round 1, published head 7e27cf4934f5c0e6dfc5dde0f7adfcd6c87d66b3.

| FND | outcome | reason |
| --- | --- | --- |
| FND-001 | fixed | FR-028 and AC-5 now specify the located advisory warning and the warning's non-failing effect, matching the generic required-relation behavior. The test asserts the warning's location, declared check, and message; a direct run of the same fixture returned 0. |
| FND-002 | fixed | The controller-role branch of `ProcessingActivity.json` now applies the `NonBlankText` schema to `controller`, rejecting an array in the controller role while the processor role still accepts its list. `node scripts/generate-schemas.mjs --check` reports 67 schemas match. |

Round-1 verification note: `tests/test_processing_activity.py` yielded 9 passed, 1 failed after npm dependencies were installed. The remaining extraction assertion used the globally installed Python Quire 0.53.0, while the repository requires `quire >=0.61.0`; it returned an empty record for the processor fixture. The rest of the focused tests ran against npm quire-cli 0.46.0 / engine 0.61.0.
51 changes: 51 additions & 0 deletions reviews/plat-1658-gap-analysis.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
id: SR-1621
title: "gap-analysis review — PLAT-1658 spec-objects-security"
type: SpecReview
analysis: gap-analysis
scope: "agent-ix/spec-objects-security; spec/functional/FR-028-processing-activity.md; typespec/main.tsp; spec_objects_security/manifest.yaml; spec_objects_security/schemas/ProcessingActivity*.json; tests/test_processing_activity.py; tests/test_regulated_categories.py"
review_set: subset
---

## Summary

The computed matrix has no test binders for any FR-028 acceptance criterion. The newly added `pytest.mark.trace` annotations do not back criteria because the module has no FR-028 rows in its trace matrix; the source tags consequently produce `criterion-tagged-outside-row` warnings. Plan completion was not assessed. Optional semantic review was skipped because it was not requested.

## Verdict

**FAIL** — all eight examined acceptance criteria are untagged in the computed Test Matrix.

## Scope

- Examined: FR-028-AC-1 through FR-028-AC-8, their test annotations, generated schemas, TypeSpec model, manifest relation and skeleton.
- Ticket: PLAT-1658; PR #69; reviewed published head `84b182f01a6db11c1c72c42f4b94442afe365268`.
- Plan completion: not assessed.
- Semantic review: skipped (not requested).

## Findings

| ID | Severity | Summary | Refs |
| --- | --- | --- | --- |
| FND-001 | high | The computed Test Matrix reports FR-028-AC-1 through AC-8 as untagged with zero binders. `tests/test_processing_activity.py` contains trace markers, but `quire matrix` reports `criterion-tagged-outside-row` for these tags because no matrix row declares the new criterion ids; the tags do not establish coverage. | spec/functional/FR-028-processing-activity.md:57-64; tests/test_processing_activity.py:21-27,59-70,98-99 |

## Coverage

`quoin matrix --repo . --json` exited 0 and reported `static_status=untagged` and no binders for each FR-028-AC-1 through FR-028-AC-8. `quire matrix --scope . --format json` exited 0 and reported the added tags as `criterion-tagged-outside-row`; the report identifies the expected same-test tag in a declared matrix row, but none exists for these criteria. The module has no `spec/tests.md` row changes in this PR. Reverse-gap and stub scan found no other new unowned behavior or stubs in the reviewed change. The PR's schema generation check passed (67 schemas match). The focused test run produced 19 passes; 13 unrelated regulated-category tests failed because the active Python Quire wheel rejects a pre-existing `where` manifest field.


## New findings (round 1)

| ID | Severity | Summary | Refs |
| --- | --- | --- | --- |
| FND-002 | medium | FR-028's changed coverage behavior explicitly requires an advisory warning that does not make `quire validate` exit non-zero. The tagged no-covers test checks the warning location and text but never asserts `result.returncode == 0`, so it cannot catch a regression that emits the warning and still fails validation. A direct run currently returns 0. | spec/functional/FR-028-processing-activity.md:44; tests/test_processing_activity.py:160-174 |

## Dispositions

Round 1, published head 7e27cf4934f5c0e6dfc5dde0f7adfcd6c87d66b3.

| FND | outcome | reason |
| --- | --- | --- |
| FND-001 | fixed | `spec/tests.md` now contains TC-248 through TC-255 rows for FR-028-AC-1 through AC-8. Quoin 0.35.0 and Quire 0.46.0 computed matrices both report all eight criteria tagged with their expected `tests/test_processing_activity.py` binders. |
| FND-002 | still-open | The warning test still checks only output text. Add an assertion that `result.returncode == 0` to enforce the explicit advisory/non-failing rule in FR-028 line 44. |

Round-1 verification note: schema generation check passed (67 schemas match). Focused Python tests: 9 passed, 1 failed. The remaining extraction assertion ran against globally installed Python Quire 0.53.0 while the repository requires `quire >=0.61.0`; it returned an empty record for the processor fixture. Matrix evidence came from Quoin 0.35.0 and npm Quire CLI 0.46.0 / engine 0.61.0.
31 changes: 27 additions & 4 deletions scripts/generate-schemas.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -114,10 +114,10 @@ function normalize(schemas, base, moduleFiles, coreBase) {

/**
* Frontmatter validators compile without the module schema registry (FR-069).
* Keep their closed enum members inline so validation stays offline while the
* regular data schemas can continue to reference the shared enum schemas.
* Keep enum and string-scalar constraints inline so validation stays offline
* while regular data schemas can continue to reference shared schema files.
*/
function inlineFrontmatterEnums(schemas) {
function inlineFrontmatterConstraints(schemas) {
const byId = new Map(
schemas.map(([, schema]) => [schema.$id, schema]).filter(([id]) => id),
);
Expand All @@ -135,6 +135,29 @@ function inlineFrontmatterEnums(schemas) {
delete node.$ref;
node.type = target.type;
node.enum = [...target.enum];
} else if (
target &&
typeof target.type === "string" &&
!["properties", "allOf", "oneOf", "anyOf", "$defs"].some(
(key) => key in target,
)
) {
delete node.$ref;
for (const key of [
"type",
"const",
"enum",
"pattern",
"minLength",
"maxLength",
"format",
"minItems",
"maxItems",
"uniqueItems",
"items",
]) {
if (key in target) node[key] = target[key];
}
}
}
for (const value of Object.values(node)) visit(value);
Expand Down Expand Up @@ -169,7 +192,7 @@ function emit() {
}
const moduleFiles = new Set(mine.map(([name]) => name));
normalize(mine, base, moduleFiles, semanticCoreBase());
inlineFrontmatterEnums(mine);
inlineFrontmatterConstraints(mine);
const rendered = new Map(mine.map(([name, schema]) => [name, render(schema)]));

return rendered;
Expand Down
2 changes: 1 addition & 1 deletion spec/functional/FR-026-threat-model-object.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ not a testable obligation of this requirement.
| FR-026-AC-7 | `ThreatModel.json` carries no `default` for `method` or `reviewed_by`, and an extracted record has no `method` or `reviewed_by` key the author did not write. | Test |
| FR-026-AC-8 | No manifest entry, rule text, skeleton or README added by this requirement names or quotes a paywalled standard; the only sources named are STRIDE, OWASP's free threat-modelling guidance and NIST SP 800-154. | Inspection |
| FR-026-AC-9 | The module manifest declares `considers` and `covers` in `edge_types`; the module loads, and removing either declaration makes it fail to load naming the verb. | Test |
| FR-026-AC-10 | `semantic.exports` names `threat_model` and has twenty-seven entries; `ThreatModel.json` is emitted without `default` keys; the `threat_model` skeleton validates; the empty record `{}` fails `ThreatModel.json`. | Test |
| FR-026-AC-10 | `semantic.exports` names `threat_model`; `ThreatModel.json` is emitted without `default` keys; the `threat_model` skeleton validates; the empty record `{}` fails `ThreatModel.json`. | Test |

## Dependencies

Expand Down
4 changes: 2 additions & 2 deletions spec/functional/FR-028-processing-activity.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ The security module SHALL let a person describe one processing activity and iden
### Coverage edges

- The module manifest SHALL declare the `processing_activity` object type and allow `covers` targets of `connection`, `data_sink` and `part`.
- Each `processing_activity` SHALL have at least one outgoing `covers` edge to one of those types.
- The manifest SHALL declare a required-relation check for outgoing `covers` edges to those types. When an activity has no such edge, Quire SHALL emit a located warning naming the activity and the declared check; the warning is advisory and does not make `quire validate` exit non-zero.
- A `covers` edge to any other object type or to a missing target SHALL fail validation with a located edge diagnostic.
- A `covers` edge to a `queue` SHALL fail as a wrong-kind edge.

Expand All @@ -58,7 +58,7 @@ The security module SHALL let a person describe one processing activity and iden
| FR-028-AC-2 | A processor activity with a processor name and contact, a non-empty controller list and home regions validates without `purposes`, `data_subject_categories` or `retention`. | Test |
| FR-028-AC-3 | A controller activity without `retention`, `purposes` or `data_subject_categories`, and a processor activity without its own `processor` identity/contact or with an empty controller list, each fail with a diagnostic naming the missing or invalid field. | Test |
| FR-028-AC-4 | An unknown role, an empty home-region list, or a home-region value outside the stated vocabulary fails validation. | Test |
| FR-028-AC-5 | A `processing_activity` with no `covers` edge fails validation; an edge to a `connection`, `data_sink` or `part` validates. | Test |
| FR-028-AC-5 | A `processing_activity` with no `covers` edge emits a located warning naming the activity and declared check; an edge to a `connection`, `data_sink` or `part` validates. | Test |
| FR-028-AC-6 | A `covers` edge to a `queue` fails with a wrong-kind diagnostic, and an edge to an absent target fails with a dangling-target diagnostic. | Test |
| FR-028-AC-7 | The module exports `processing_activity`; its generated schema and skeleton pass the module schema-drift and `quire validate` checks. | Test |
| FR-028-AC-8 | Omitting any optional authored field leaves it absent in the extracted record; no omitted value is populated from another field, artifact, code or runtime source. | Test |
Expand Down
Loading
Loading