From 471cfecad17adf8876bad2a3d1c95cd8dd811875 Mon Sep 17 00:00:00 2001 From: Matt <47545907+SoundMatt@users.noreply.github.com> Date: Wed, 29 Jul 2026 09:55:13 -0700 Subject: [PATCH] =?UTF-8?q?fix:=20fusa-asil-b=20CI=20gate=20=E2=80=94=20pi?= =?UTF-8?q?n=20cpp-FuSa=20v0.17.1,=20stop=20wiping=20requirements,=20enfor?= =?UTF-8?q?ce=20real=20gap=20thresholds?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three compounding problems in the fusa-asil-b (and sarif) CI jobs: 1. cpp-FuSa was pinned to v0.15.0, missing conformance fixes through v0.17.1 that affect this repo's own generated evidence (sbom.json component hashes, safety-case completeness — see cpp-FuSa's CHANGELOG v0.17.0 entry). Both jobs now pin v0.17.1. 2. `cpfusa init --force` ran as the first step in both jobs, before the traceability step, unconditionally overwriting the real, populated `.fusa-reqs.json` (50+ KB of hand-maintained REQ-* entries) with an empty scaffold. CI traceability coverage was always computed against zero requirements regardless of the repo's actual state. Removed the step entirely — `.fusa.json`/`.fusa-reqs.json` are already checked in and match what `init` would have generated; the step is only needed for first-time local setup. 3. The ISO 26262 / IEC 61508 gap-analysis steps were wrapped in `|| true`, so they could never fail the build regardless of gap count (15/20 and 12/18 gaps at last real measurement). Two changes: - Reordered the job so boundary/tara/fmea/safety-case/sas/sci run BEFORE the gap analyses: cpfusa grades several objectives (e.g. §9-2.1 Safety case, §10.4 SCI) on whether safety-case.json/ sci.json already exist, so gap-checking first was undercounting what the repo actually addresses. This alone closed 2 gaps. - Replaced `|| true` with an explicit, intentional gate: fail the build if the gap count regresses past the current, documented baseline (13 for ISO 26262, 11 for IEC 61508). A "0 gaps" target is not reachable with cpfusa v0.17.1 regardless of this repo's actual documentation completeness — several objectives (e.g. §6.1 Software architectural design, §8-6.2 Safety manual) have no evidence-detection logic implemented in the tool at all and report Gap unconditionally (cpp-FuSa src/iso26262/iso26262.cpp's detect_status()). The gate at least makes future regressions visible instead of being structurally unable to fail. Also fixed independently while investigating (2), a real bug the fix exposed: cpp-FuSa v0.17.0 started actually enforcing `sourceDirs` (previously ignored per its own CHANGELOG), and `.fusa.json`'s sourceDirs (["src", "include"]) omitted "tests" and "cli" — so every `fusa:test`/`fusa:req` annotation in those directories silently stopped being seen by `trace`, dropping real test-coverage traceability from 93.5% to 0% the moment the pin bumped. Added both directories to sourceDirs, and registered 6 real, already-annotated CLI requirements (REQ-CLI-001..006, both impl- and test-annotated in cli/ and tests/test_cli.cpp already) that this surfaced as dangling references in requirements/requirements.json and .fusa-reqs.json. README's ASIL-B badge/description is reworded to "ASIL-B target (SEooC)" with a pointer to the real gap-report artifacts, per this repo's own SAFETY_PLAN.md framing — the prior flat "ISO 26262 ASIL-B" claim overstated completeness relative to the tool's own (now enforced) gap reports. Verified locally end-to-end against a clean checkout with a freshly built cpfusa v0.17.1: check/lint/trace/cyber/qualify/boundary/tara/ fmea/safety-case/sas/sci/badge/vuln/metrics/report all exit 0; trace reports 135/144 (93.8%) annotated and tested with zero dangling references; iso26262 gate passes at 13 gaps (baseline); iec61508 gate passes at 10 gaps (under the 11 baseline). Closes #19, #20, #21 Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com> --- .fusa-reqs.json | 48 ++++++++++++++++++++ .fusa.json | 4 +- .github/workflows/ci.yml | 82 ++++++++++++++++++++++++---------- README.md | 12 ++++- requirements/requirements.json | 6 +++ 5 files changed, 125 insertions(+), 27 deletions(-) diff --git a/.fusa-reqs.json b/.fusa-reqs.json index 533cf5a..0241124 100644 --- a/.fusa-reqs.json +++ b/.fusa-reqs.json @@ -609,6 +609,54 @@ "rationale": "Slaves must be able to update responses (e.g., refreshed sensor values) without removing and re-registering.", "tags": ["slave", "update"] }, + { + "id": "REQ-CLI-001", + "title": "version command reports tool/protocol/spec_version fields", + "description": "cmd_version's json and text renderings shall include tool, protocol, version, spec_version, language, and runtime fields per RELAY spec §12.1.", + "asil": "ASIL-B", + "rationale": "Integrators must be able to programmatically confirm which cpp-lin build and RELAY spec version a binary implements.", + "tags": ["cli"] + }, + { + "id": "REQ-CLI-002", + "title": "capabilities command reports transports/features/interfaces", + "description": "capabilities_json() shall report kind, tool, protocol, version, spec_version, commands, transports, features, interfaces, optional_interfaces, and adapt per RELAY spec §12.2.", + "asil": "ASIL-B", + "rationale": "Downstream tooling (e.g. relay conform) discovers a binary's supported feature set from this document.", + "tags": ["cli"] + }, + { + "id": "REQ-CLI-003", + "title": "status command reports health/connection fields", + "description": "cmd_status's json and text renderings shall include protocol, tool, version, healthy, connected, endpoint, and details fields per RELAY spec §12.3.", + "asil": "ASIL-B", + "rationale": "Operators and orchestration tooling need a machine-readable liveness/health signal.", + "tags": ["cli"] + }, + { + "id": "REQ-CLI-004", + "title": "parse_frame_json parses a LIN frame from JSON", + "description": "parse_frame_json shall extract id, data, checksum_type, and checksum from a JSON object and throw on a missing or invalid id field.", + "asil": "ASIL-B", + "rationale": "The CLI's frame-publish path depends on correctly reconstructing a lin::Frame from user-supplied JSON.", + "tags": ["cli", "json"] + }, + { + "id": "REQ-CLI-005", + "title": "message_to_json serialises a RELAY Message to JSON", + "description": "message_to_json shall serialise protocol, version, id, base64 payload, timestamp, and (when present) seq/meta fields.", + "asil": "ASIL-B", + "rationale": "relay conform and downstream consumers depend on a spec-conformant JSON rendering of RELAY messages.", + "tags": ["cli", "json"] + }, + { + "id": "REQ-CLI-006", + "title": "convert command rejects RELAY error-vector inputs per spec", + "description": "cmd_convert (and the underlying conversion path) shall reject inputs matching RELAY's published error vectors (e.g. ID overflow, wrong checksum) rather than silently accepting them.", + "asil": "ASIL-B", + "rationale": "RELAY spec §20 continuous conformance requires a conformant implementation to exercise the reject path, not only the happy path.", + "tags": ["cli", "conformance"] + }, { "id": "REQ-SAFETY-001", "title": "DataID embedded in header bytes 0-1", diff --git a/.fusa.json b/.fusa.json index ff5328d..e12b912 100644 --- a/.fusa.json +++ b/.fusa.json @@ -7,7 +7,9 @@ }, "sourceDirs": [ "src", - "include" + "include", + "tests", + "cli" ], "standard": "iso26262", "strict": false diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9d15da1..be38769 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -234,7 +234,7 @@ jobs: with: repository: SoundMatt/cpp-FuSa path: cpp-FuSa - ref: v0.15.0 + ref: v0.17.1 - name: Install tools run: sudo apt-get update -qq && sudo apt-get install -y cmake ninja-build zip @@ -247,9 +247,13 @@ jobs: -G Ninja cmake --build cpp-FuSa/build --parallel - - name: cpfusa init - working-directory: cpp-LIN - run: ../cpp-FuSa/build/cpfusa init --name cpp-LIN --standard iso26262 --asil ASIL-B --project-version 0.4.0 --force || true + # No `cpfusa init` step: .fusa.json and .fusa-reqs.json are real, + # populated, checked-in files (the latter is 50+ KB of hand-maintained + # REQ-* entries). `init --force` unconditionally overwrote both with + # an empty scaffold before the traceability step ever ran, so CI's + # requirements coverage was always computed against zero requirements + # regardless of the repo's real state (cpp-LIN#19). `init` is only + # needed for first-time local setup, where these files don't exist yet. - name: cpfusa check working-directory: cpp-LIN @@ -275,22 +279,12 @@ jobs: working-directory: cpp-LIN run: ../cpp-FuSa/build/cpfusa hara init --project cpp-LIN --dir . || true - - name: cpfusa iso26262 (ASIL-B) - working-directory: cpp-LIN - run: | - ../cpp-FuSa/build/cpfusa iso26262 \ - --asil ASIL-B \ - --output iso26262-gap-report.json \ - --dir . || true - - - name: cpfusa iec61508 (SIL-2) - working-directory: cpp-LIN - run: | - ../cpp-FuSa/build/cpfusa iec61508 \ - --sil SIL-2 \ - --output iec61508-gap-report.json \ - --dir . || true - + # boundary/tara/fmea/safety-case/sas/sci run BEFORE the iso26262/ + # iec61508 gap analyses below: several ISO 26262/IEC 61508 objectives + # (e.g. §9-2.1 Safety case, §10.4 SCI) are graded by cpfusa on whether + # safety-case.json/sci.json etc. already exist on disk. Gap-checking + # before generating them undercounted objectives this repo actually + # addresses. - name: cpfusa boundary working-directory: cpp-LIN run: ../cpp-FuSa/build/cpfusa boundary --dir . @@ -315,6 +309,47 @@ jobs: working-directory: cpp-LIN run: ../cpp-FuSa/build/cpfusa sci --dir . + # These gates enforce a documented, currently-achieved baseline gap + # count rather than requiring zero gaps outright: cpfusa v0.17.1 has + # no evidence-detection logic at all for several objectives (e.g. §6.1 + # Software architectural design, §8-6.2 Safety manual — see cpp-FuSa + # src/iso26262/iso26262.cpp's detect_status(), which falls through to + # Status::Gap unconditionally for those IDs regardless of what evidence + # exists), so a "0 gaps" target is not achievable with this tool + # version no matter how complete this repo's safety documentation is. + # The gate below still turns this into a real, enforced check: it + # fails the build if the gap count *regresses* past the current + # baseline, rather than being structurally unable to fail at all + # (cpp-LIN#20). See SAFETY_PLAN.md for this repo's ASIL-B *target* + # framing (SEooC, not a completed-certification claim). + - name: cpfusa iso26262 (ASIL-B) + working-directory: cpp-LIN + run: | + ../cpp-FuSa/build/cpfusa iso26262 \ + --asil ASIL-B \ + --output iso26262-gap-report.json \ + --dir . + GAPS=$(jq '.summary.gaps' iso26262-gap-report.json) + echo "ISO 26262 gaps: ${GAPS} (baseline: 13)" + if [ "${GAPS}" -gt 13 ]; then + echo "::error::ISO 26262 gap count ${GAPS} exceeds baseline of 13 — see iso26262-gap-report.json" + exit 1 + fi + + - name: cpfusa iec61508 (SIL-2) + working-directory: cpp-LIN + run: | + ../cpp-FuSa/build/cpfusa iec61508 \ + --sil SIL-2 \ + --output iec61508-gap-report.json \ + --dir . + GAPS=$(jq '.summary.gaps' iec61508-gap-report.json) + echo "IEC 61508 gaps: ${GAPS} (baseline: 11)" + if [ "${GAPS}" -gt 11 ]; then + echo "::error::IEC 61508 gap count ${GAPS} exceeds baseline of 11 — see iec61508-gap-report.json" + exit 1 + fi + - name: cpfusa badge working-directory: cpp-LIN run: ../cpp-FuSa/build/cpfusa badge --dir . @@ -435,7 +470,7 @@ jobs: with: repository: SoundMatt/cpp-FuSa path: cpp-FuSa - ref: v0.15.0 + ref: v0.17.1 - name: Install tools run: sudo apt-get update -qq && sudo apt-get install -y cmake ninja-build @@ -448,9 +483,8 @@ jobs: -G Ninja cmake --build cpp-FuSa/build --parallel - - name: cpfusa init - working-directory: cpp-LIN - run: ../cpp-FuSa/build/cpfusa init --name cpp-LIN --standard iso26262 --asil ASIL-B --project-version 0.4.0 --force || true + # No `cpfusa init` step here either — see the fusa-asil-b job's note + # (cpp-LIN#19): .fusa.json/.fusa-reqs.json are already checked in. - name: Generate SARIF report working-directory: cpp-LIN diff --git a/README.md b/README.md index c02b0f9..aa59b59 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,20 @@ # cpp-LIN -C++ LIN bus library — RELAY-conformant, ISO 26262 ASIL-B. +C++ LIN bus library — RELAY-conformant, ISO 26262 ASIL-B target (SEooC). [![CI](https://github.com/SoundMatt/cpp-LIN/actions/workflows/ci.yml/badge.svg)](https://github.com/SoundMatt/cpp-LIN/actions/workflows/ci.yml) [![RELAY Conformant](https://img.shields.io/badge/RELAY-v1.11%20conformant-blue)](https://github.com/SoundMatt/RELAY) -[![ASIL-B](https://img.shields.io/badge/ISO%2026262-ASIL--B-orange)](SAFETY_PLAN.md) +[![ASIL-B target](https://img.shields.io/badge/ISO%2026262-ASIL--B%20target-yellow)](SAFETY_PLAN.md) [![License: MPL 2.0](https://img.shields.io/badge/License-MPL%202.0-brightgreen.svg)](LICENSE) +cpp-LIN is a Safety Element out of Context (SEooC, ISO 26262-10 §9) targeting +ASIL-B, not a completed third-party certification. CI runs `cpfusa iso26262`/ +`cpfusa iec61508` gap analyses on every build and gates on non-regression +against a documented baseline (`.github/workflows/ci.yml`'s `fusa-asil-b` +job); see `iso26262-gap-report.json`/`iec61508-gap-report.json` in that job's +uploaded evidence artifacts for the current, honest gap count rather than +treating the badge above as a completeness claim. + ## Overview cpp-LIN is a production-quality C++ implementation of the LIN (Local Interconnect diff --git a/requirements/requirements.json b/requirements/requirements.json index 03121d6..51de1eb 100644 --- a/requirements/requirements.json +++ b/requirements/requirements.json @@ -84,6 +84,12 @@ {"id": "REQ-SLAVE-006", "title": "subscribe delegates to bus", "asil": "ASIL-B", "tags": ["slave"]}, {"id": "REQ-SLAVE-007", "title": "registered_ids returns empty slice when none", "asil": "ASIL-B", "tags": ["slave", "boundary"]}, {"id": "REQ-SLAVE-008", "title": "set_response overwrites previous registration", "asil": "ASIL-B", "tags": ["slave", "update"]}, + {"id": "REQ-CLI-001", "title": "version command reports tool/protocol/spec_version fields", "asil": "ASIL-B", "tags": ["cli"]}, + {"id": "REQ-CLI-002", "title": "capabilities command reports transports/features/interfaces", "asil": "ASIL-B", "tags": ["cli"]}, + {"id": "REQ-CLI-003", "title": "status command reports health/connection fields", "asil": "ASIL-B", "tags": ["cli"]}, + {"id": "REQ-CLI-004", "title": "parse_frame_json parses a LIN frame from JSON", "asil": "ASIL-B", "tags": ["cli", "json"]}, + {"id": "REQ-CLI-005", "title": "message_to_json serialises a RELAY Message to JSON", "asil": "ASIL-B", "tags": ["cli", "json"]}, + {"id": "REQ-CLI-006", "title": "convert command rejects RELAY error-vector inputs per spec", "asil": "ASIL-B", "tags": ["cli", "conformance"]}, {"id": "REQ-SAFETY-001", "title": "DataID embedded in header bytes 0-1", "asil": "ASIL-B", "tags": ["safety", "header"]}, {"id": "REQ-SAFETY-002", "title": "SourceID embedded in header bytes 2-3", "asil": "ASIL-B", "tags": ["safety", "header"]}, {"id": "REQ-SAFETY-003", "title": "SequenceCounter starts at 0 and increments", "asil": "ASIL-B", "tags": ["safety", "counter"]},