From 941312d300088dd215c14cdf229897a60e0c65bb Mon Sep 17 00:00:00 2001 From: Said Boudjelda Date: Fri, 30 Oct 2026 21:12:16 +0100 Subject: [PATCH] feat(compliance): the check's audit row holds its report hash, so an edit shows A stored compliance check could be edited in the database without a trace: its audit row recorded only the verdict, and the audit log's hash chain covers the audit rows, not the compliance_checks table. A database admin could turn a failed rule into a passed one and keep the verdict. The check's audit row now records "verdict X, report sha256 Y, reference sha256 Z", and GET /api/conformity/checks/{id} answers reportSha256, the SHA-256 of the report as stored, computed when asked. An edited report no longer matches the hash its chained audit row holds; a rewrite of the chain itself changes the rowHash of an anchor kept outside the database. The compliance chapter gains "Proving the Results Were Not Changed": the three records behind a check, how to keep the anchor in a pipeline, a script that verifies a stored check, what each failure means, and the limits. The API reference describes the new fields. --- docs/book/11-api-reference.md | 4 +- docs/book/compliance.md | 81 +++++++++++++++++++ .../bmscomp/kates/api/ComplianceResource.java | 17 ++-- .../compliance/ComplianceCheckRepository.java | 24 ++++-- .../kates/api/ComplianceResourceTest.java | 54 +++++++++++++ .../bmscomp/kates/it/ComplianceChecksIT.java | 3 +- 6 files changed, 168 insertions(+), 15 deletions(-) diff --git a/docs/book/11-api-reference.md b/docs/book/11-api-reference.md index 30d331cb..d8500824 100644 --- a/docs/book/11-api-reference.md +++ b/docs/book/11-api-reference.md @@ -679,7 +679,7 @@ The response is 200 whatever the verdict, with one result per declared rule. `ve A reference the check cannot run is a 400 whose `fieldErrors` name each path, as `spec.brokers.cuont: unknown field`. A reference for another cluster than the one Kates watches, or a cluster without its Kafka resource, is a 409; a body over 256 KiB is a 413; an unreachable Kubernetes API, or one that refuses the Kates API the Kafka resource, is a 503. -The Kates API stores each report it answers with, and its `Location` header names it, as `/api/conformity/checks/1f0c…`. A refused reference stores nothing. If the database fails, the caller still gets its report, without a `Location`, and the Kates API logs an error. Each stored check also leaves an audit row (`RUN`, type `compliance`) whose target is the check's id. Nothing deletes stored checks. +The Kates API stores each report it answers with, and its `Location` header names it, as `/api/conformity/checks/1f0c…`. A refused reference stores nothing. If the database fails, the caller still gets its report, without a `Location`, and the Kates API logs an error. Each stored check also leaves an audit row (`RUN`, type `compliance`) whose target is the check's id and whose details hold the verdict and two hashes, as `verdict FAIL, report sha256 5d41…, reference sha256 44136f…`. The audit log is a hash chain, so those hashes are what a later reader compares the stored check against ([Proving the Results Were Not Changed](compliance.md#proving-the-results-were-not-changed)). Nothing deletes stored checks. #### GET /api/conformity/checks @@ -698,7 +698,7 @@ List the stored checks, newest first (scope `read`). `since` (inclusive) and `un #### GET /api/conformity/checks/{id} -One stored check (scope `read`): the fields the list shows, `referenceSha256`, the SHA-256 of the body the Kates API was sent (the JSON `kates compliance check` made from the file, not the YAML file itself), and `report`, the full report as the check answered it. An unknown id is a 404. +One stored check (scope `read`): the fields the list shows, `referenceSha256`, the SHA-256 of the body the Kates API was sent (the JSON `kates compliance check` made from the file, not the YAML file itself), `report`, the full report as the check answered it, and `reportSha256`, the SHA-256 of the report as it is stored now, computed when you ask. It equals the `report sha256` in the check's audit row unless the stored report was changed after the check. An unknown id is a 404. ### Disruption Testing diff --git a/docs/book/compliance.md b/docs/book/compliance.md index 75621ea7..71ccb7ed 100644 --- a/docs/book/compliance.md +++ b/docs/book/compliance.md @@ -607,6 +607,86 @@ WAIVED monitoring.kafkaExporter `kates compliance` belongs to the `topology` and `deploy` [command profiles](10-cli-reference.md#command-profiles). A context whose profiles leave both out refuses all three subcommands, `lint` and `fmt` included, so a pipeline's context needs one of them or none. The [Compliance](10-cli-reference.md#compliance) entries in the CLI Reference list every flag, and [POST /api/conformity/check](11-api-reference.md#post-apiconformitycheck) documents the report's JSON for callers that skip the CLI. +## Proving the Results Were Not Changed + +An auditor who reads a stored check months later needs to know it still says what the check said. Whoever administers the Kates API's database can edit any row by hand. They can turn a failed rule into a passed one, change who ran the check, or delete it. Kates can't stop that, but it makes the change show, as long as you keep one value outside the database on the day the check runs. + +Three records stand behind each check, and each one guards the record above it: + +| Record | Where it lives | What it holds | +|:----------|:--------------|:--------------------------------------------| +| The stored check | The database, read with `GET /api/conformity/checks/{id}` | The full report, the reference's SHA-256, and `reportSha256`, computed from the stored report when you ask | +| The check's audit row | The database, in the audit log's hash chain | Who ran the check and when, the verdict, the report's SHA-256 and the reference's SHA-256 | +| The anchor | Outside the database, where its administrators can't write | The audit row's `id` and `rowHash`, kept on the day of the check | + +An edited stored report doesn't match the hash in its audit row. An edit to the audit row breaks the chain at that row. A rewrite of the whole chain, every hash recomputed so that it verifies, changes the anchored row's `rowHash`. [Audit Logging](17-security.md#audit-logging) explains how the chain is built. + +### Keep the Anchor When the Check Runs + +The anchor is the check's own audit row, read right after the check. The command-line check doesn't print the stored check's id, so the step below finds the newest `compliance` row of the pipeline's own principal. Give each gate its own API key, so that its newest row is its own check. Run the step whether the check passed or not, because a failed check is evidence too: + +```bash +curl -s -H "X-API-Key: $KATES_API_KEY" \ + "http://localhost:30083/api/audit?type=compliance&actor=ci-gate&size=1" \ + | jq '.items[0] | {check: .target, auditId: .id, rowHash, details}' \ + > compliance-anchor.json +``` + +The file names the stored check, its audit row and that row's hash, and repeats the hashes the row holds: + +```json +{ + "check": "1f0c6a52-…", + "auditId": 4529, + "rowHash": "9c1e…", + "details": "verdict FAIL, report sha256 5d41…, reference sha256 44136f…" +} +``` + +Keep it where the database's administrators can't change it. Your pipeline's artifacts, a ticket, or a file you sign with your own key all qualify. An anchor inside the same database proves nothing. + +### Verify a Stored Check + +Verifying a check later takes three answers from the Kates API, and the check holds only if all three agree. The first is the chain: `GET /api/audit/verify` walks the whole audit log and must answer `"intact": true`. The second is the anchor: the audit row named by `auditId` must still have the `rowHash` you kept. The third is the report: the check's `reportSha256` must equal the `report sha256` in that row's details. + +This script asks all three for the anchor kept above, and prints the three results: + +```bash +H="X-API-Key: $KATES_API_KEY"; API=http://localhost:30083 +a=compliance-anchor.json +curl -s -H "$H" "$API/api/audit/verify" | jq '{intact, brokenAt, problem}' +curl -s -H "$H" "$API/api/audit?type=compliance&size=200" \ + | jq --argjson a "$(cat $a)" \ + '.items[] | select(.id == $a.auditId) | {sameRow: (.rowHash == $a.rowHash)}' +curl -s -H "$H" "$API/api/conformity/checks/$(jq -r .check $a)" \ + | jq --argjson a "$(cat $a)" \ + '.reportSha256 as $r | {sameReport: ($a.details | contains("report sha256 " + $r))}' +``` + +A clean check prints `"intact": true`, `"sameRow": true` and `"sameReport": true`. Each other answer names a different change: + +| What you see | What changed | +|:------------------------|:------------------------------------------------| +| `"intact": false` | A row of the audit log was changed, removed or added; `brokenAt` names the first one | +| No `sameRow` line | The anchored audit row is missing from the newest 200 `compliance` rows, or was deleted | +| `"sameRow": false` | The audit log was rewritten up to that row, every hash recomputed | +| `"sameReport": false` | The stored report was changed after the check | +| A 404 for the check | The stored check was deleted; its audit row still names it | + +A deleted audit row breaks the chain as surely as an edited one, so `"intact": false` covers it. + +### Limits + +The proof covers what the check stored, not what it read. It shows that a report is the one the Kates API wrote, not that the cluster was read truthfully. + +An anchor protects the rows up to its own. Rows written after it are covered by the next anchor you keep. + +Audit rows written before the Kates API chained its log aren't covered; `GET /api/audit/verify` counts them as `unchained`. + +The script reads the newest 200 `compliance` rows. Add `page=1` and `page=2` to reach 500, the most the audit listing returns, newest first. An older check's audit row can't be read back this way, though `GET /api/audit/verify` still covers it. + +Someone who can change both the database and the place you keep anchors can hide a change. Keep the anchors where the database's administrators have no access. + ## The Language at a Glance A reference holds two kinds of field. Settings say which cluster to check and how to read it; they aren't rules, and a waiver can't name one. Every other field is a rule, named by its path under `spec`. The tables below are a summary; the [Compliance Language Reference](appendix-compliance-language.md) gives every field's values, what the check reads for it, and when it's UNKNOWN. @@ -661,5 +741,6 @@ The check reports what it can read, and some things are deliberately outside it: - Lint catches typos and impossible combinations on your machine, before any cluster is involved. - A failed rule means fixing the cluster, fixing the file, or recording the gap with an owner and an end date. - The exit code makes the check a pipeline step that fails when the cluster drifts from its design. +- A hash kept outside the database on the day of a check shows whether its stored report was changed since. The reference checks that every listener authenticates its clients, but not who those clients are or what they may do once connected. [Security & Compliance](17-security.md) covers authentication, authorization, network policies and certificates. diff --git a/kates/src/main/java/com/bmscomp/kates/api/ComplianceResource.java b/kates/src/main/java/com/bmscomp/kates/api/ComplianceResource.java index 87e7cf30..b1d46584 100644 --- a/kates/src/main/java/com/bmscomp/kates/api/ComplianceResource.java +++ b/kates/src/main/java/com/bmscomp/kates/api/ComplianceResource.java @@ -100,18 +100,23 @@ public Response check(byte[] body) { .entity(e.getError()) .build(); } - String id; + ComplianceCheckEntity stored; try { - id = checks.store(report, body, actors.current()); + stored = checks.store(report, body, actors.current()); } catch (RuntimeException e) { // The caller, often a CI gate, still gets its verdict; the missing // Location and this line say the evidence was not kept. LOG.errorf(e, "Compliance report of %s not stored", report.reference()); return Response.ok(report).build(); } - auditService.record("RUN", "compliance", id, "verdict " + report.verdict()); + auditService.record( + "RUN", + "compliance", + stored.getId(), + "verdict " + report.verdict() + ", report sha256 " + ComplianceCheckRepository.reportSha256(stored) + + ", reference sha256 " + stored.getReferenceSha256()); return Response.ok(report) - .location(URI.create("/api/conformity/checks/" + id)) + .location(URI.create("/api/conformity/checks/" + stored.getId())) .build(); } @@ -153,7 +158,8 @@ public Response list( @Path("/checks/{id}") @Operation( summary = "Get a stored compliance check", - description = "The check's id, who ran it, the SHA-256 of the reference it was sent, and its full report") + description = "The check's id, who ran it, the SHA-256 of the reference it was sent, its full report," + + " and the SHA-256 of the report as stored now, to compare with the one its audit row recorded") @APIResponse(responseCode = "200", description = "The stored check") @APIResponse(responseCode = "404", description = "No check has this id") public Response get(@Parameter(description = "Check ID") @PathParam("id") String id) { @@ -165,6 +171,7 @@ public Response get(@Parameter(description = "Check ID") @PathParam("id") String } Map answer = summary(entity); answer.put("referenceSha256", entity.getReferenceSha256()); + answer.put("reportSha256", ComplianceCheckRepository.reportSha256(entity)); try { answer.put("report", objectMapper.readTree(entity.getReportJson())); } catch (JsonProcessingException e) { diff --git a/kates/src/main/java/com/bmscomp/kates/compliance/ComplianceCheckRepository.java b/kates/src/main/java/com/bmscomp/kates/compliance/ComplianceCheckRepository.java index 85de3ee8..f16734d1 100644 --- a/kates/src/main/java/com/bmscomp/kates/compliance/ComplianceCheckRepository.java +++ b/kates/src/main/java/com/bmscomp/kates/compliance/ComplianceCheckRepository.java @@ -1,5 +1,6 @@ package com.bmscomp.kates.compliance; +import java.nio.charset.StandardCharsets; import java.security.MessageDigest; import java.security.NoSuchAlgorithmException; import java.time.Instant; @@ -30,19 +31,18 @@ public class ComplianceCheckRepository { /** * Stores report, checked against the reference body, as run by actor - * (null when no one can be named), and returns the new row's id. + * (null when no one can be named), and returns the new row. */ @Transactional - public String store(ComplianceReport report, byte[] body, Actor actor) { + public ComplianceCheckEntity store(ComplianceReport report, byte[] body, Actor actor) { String json; try { json = objectMapper.writeValueAsString(report); } catch (JsonProcessingException e) { throw new IllegalStateException("cannot serialize the compliance report", e); } - String id = UUID.randomUUID().toString(); - em.persist(new ComplianceCheckEntity( - id, + ComplianceCheckEntity entity = new ComplianceCheckEntity( + UUID.randomUUID().toString(), report.checkedAt(), report.reference(), sha256(body), @@ -50,8 +50,18 @@ public String store(ComplianceReport report, byte[] body, Actor actor) { report.verdict().name(), actor == null ? null : actor.name(), actor == null ? null : actor.type(), - json)); - return id; + json); + em.persist(entity); + return entity; + } + + /** + * The SHA-256 of the report as stored. The check's audit row records it + * when the check runs, and the audit log is a hash chain, so a report + * edited in the database no longer matches its row there. + */ + public static String reportSha256(ComplianceCheckEntity entity) { + return sha256(entity.getReportJson().getBytes(StandardCharsets.UTF_8)); } public Optional find(String id) { diff --git a/kates/src/test/java/com/bmscomp/kates/api/ComplianceResourceTest.java b/kates/src/test/java/com/bmscomp/kates/api/ComplianceResourceTest.java index fc12b917..ee63d4e6 100644 --- a/kates/src/test/java/com/bmscomp/kates/api/ComplianceResourceTest.java +++ b/kates/src/test/java/com/bmscomp/kates/api/ComplianceResourceTest.java @@ -4,6 +4,9 @@ import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.hasItem; import static org.hamcrest.Matchers.nullValue; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.ArgumentMatchers.any; import java.nio.charset.StandardCharsets; @@ -11,8 +14,11 @@ import java.time.Instant; import java.util.HexFormat; import java.util.List; +import jakarta.inject.Inject; +import jakarta.persistence.EntityManager; import jakarta.ws.rs.ClientErrorException; +import io.quarkus.narayana.jta.QuarkusTransaction; import io.quarkus.test.InjectMock; import io.quarkus.test.junit.QuarkusTest; import io.quarkus.test.junit.TestProfile; @@ -36,6 +42,9 @@ class ComplianceResourceTest { @InjectMock ComplianceCheck check; + @Inject + EntityManager em; + private static ComplianceReport report(String checkedAt, ComplianceReport.Verdict verdict) { return new ComplianceReport( "conformity.kates.io/v1alpha1", @@ -119,6 +128,51 @@ void storesEachReportWithWhoRanItAndWhatItWasSent() throws Exception { RestAssured.get("/api/audit?type=compliance").then().statusCode(200).body("items.target", hasItem(id)); } + @Test + void theAuditRowHoldsTheReportHashSoAnEditInTheDatabaseShows() { + Mockito.when(check.check(any())).thenReturn(report("2026-10-19T12:00:00Z", ComplianceReport.Verdict.FAIL)); + String location = RestAssured.given() + .contentType(ContentType.JSON) + .body(REFERENCE) + .post("/api/conformity/check") + .then() + .statusCode(200) + .extract() + .header("Location"); + String id = location.substring(location.lastIndexOf('/') + 1); + String recorded = RestAssured.given() + .queryParam("type", "compliance") + .queryParam("size", 200) + .get("/api/audit") + .then() + .statusCode(200) + .extract() + .jsonPath() + .getString("items.find { it.target == '" + id + "' }.details"); + + String stored = RestAssured.get("/api/conformity/checks/" + id) + .then() + .statusCode(200) + .extract() + .path("reportSha256"); + String referenceSha256 = + RestAssured.get("/api/conformity/checks/" + id).then().extract().path("referenceSha256"); + assertEquals("verdict FAIL, report sha256 " + stored + ", reference sha256 " + referenceSha256, recorded); + + // What a database admin could do by hand: turn the failed rule count to 0. + QuarkusTransaction.requiringNew() + .run(() -> em.createNativeQuery( + "update compliance_checks set report_json = replace(report_json, '\"fail\":1', '\"fail\":0') where id = ?1") + .setParameter(1, id) + .executeUpdate()); + + RestAssured.get("/api/conformity/checks/" + id).then().statusCode(200).body("report.summary.fail", equalTo(0)); + String after = + RestAssured.get("/api/conformity/checks/" + id).then().extract().path("reportSha256"); + assertNotEquals(stored, after); + assertTrue(recorded.contains(stored) && !recorded.contains(after)); + } + @Test void listsTheChecksOfAPeriodNewestFirst() { Mockito.when(check.check(any())) diff --git a/kates/src/test/java/com/bmscomp/kates/it/ComplianceChecksIT.java b/kates/src/test/java/com/bmscomp/kates/it/ComplianceChecksIT.java index babee604..1c882836 100644 --- a/kates/src/test/java/com/bmscomp/kates/it/ComplianceChecksIT.java +++ b/kates/src/test/java/com/bmscomp/kates/it/ComplianceChecksIT.java @@ -42,7 +42,8 @@ private String store(String checkedAt, ComplianceReport.Verdict verdict) { verdict, new ComplianceReport.Summary(1, 0, 0, 0, 0), List.of()); - return checks.store(report, "{}".getBytes(StandardCharsets.UTF_8), new Actor("ci-gate", "agent")); + return checks.store(report, "{}".getBytes(StandardCharsets.UTF_8), new Actor("ci-gate", "agent")) + .getId(); } @Test