Skip to content

feat: add verification report need type - #660

Merged
AlexanderLanin merged 4 commits into
mainfrom
verification_needs
Jul 30, 2026
Merged

feat: add verification report need type#660
AlexanderLanin merged 4 commits into
mainfrom
verification_needs

Conversation

@FScholPer

Copy link
Copy Markdown
Contributor

Relates to #611

Split out of #612 (verification part). The inspection-record part is in a separate PR.

📌 Description

Adds a machine-readable module verification report need type to the metamodel.

  • metamodel.yaml: new mod_ver_report need type (prefix: mod_vrep__) classifying a report by safety, security, status and verification_method, with optional coverage/verdict/version attributes; links belongs_to a mod and optionally contains/covers/evidence/realizes. Adds the evidence extra link (evidence / evidence_for) it depends on.
  • requirements.rst: new "Verification Evidence" section with tool_req__docs_verification_report_need, plus the Verif row in the requirements overview table.
  • Tests: rst/options/test_options_verification_report.rst covering a valid report and an invalid verdict value.

🚨 Impact Analysis

  • This change does not violate any tool requirements and is covered by existing tool requirements
  • This change does not violate any design decisions
  • Otherwise I have created a ticket for new tool qualification

✅ Checklist

  • Added/updated documentation for new or changed features
  • Added/updated tests to cover the changes
  • Followed project coding standards and guidelines

Frank Scholter Peres frank.scholter_peres@mercedes-benz.com, Mercedes-Benz Tech Innovation GmbH
Provider Information

Adds the mod_ver_report need type to the metamodel with an evidence extra
link, documents the tool requirement, and adds option tests.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run --lockfile_mode=error //src:license-check

Status: ⚠️ Needs Review

Click to expand output
[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.6.0) and connecting to it...
INFO: Invocation ID: 95cdfb2e-c7a1-4a93-9e63-190cac893a43
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=ppc64le-unknown-linux-gnu. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=riscv64-unknown-linux-gnu. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=s390x-unknown-linux-gnu. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=x86_64-unknown-linux-musl. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
Loading: 0 packages loaded
    currently loading: src
Loading: 0 packages loaded
    currently loading: src
WARNING: Target pattern parsing failed.
ERROR: Skipping '//src:license-check': no such target '//src:license-check': target 'license-check' not declared in package 'src' defined by /home/runner/work/docs-as-code/docs-as-code/src/BUILD
ERROR: no such target '//src:license-check': target 'license-check' not declared in package 'src' defined by /home/runner/work/docs-as-code/docs-as-code/src/BUILD
INFO: Elapsed time: 7.845s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

@github-actions

Copy link
Copy Markdown

The created documentation from the pull request is available at: docu-html

@FScholPer FScholPer changed the title Add machine-readable module verification report need type verification report need type Jul 23, 2026

@aschemmel-tech aschemmel-tech left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made some remarks, not crucial for the PoC

Comment thread src/extensions/score_metamodel/metamodel.yaml Outdated
Comment thread src/extensions/score_metamodel/metamodel.yaml Outdated
Comment thread src/extensions/score_metamodel/metamodel.yaml Outdated
Comment thread src/extensions/score_metamodel/metamodel.yaml Outdated
Comment thread src/extensions/score_metamodel/metamodel.yaml Outdated
verification_method: ^.*$
optional_options:
requirements_coverage_percent: ^(100|[1-9]?[0-9])$
structural_coverage_percent: ^(100|[1-9]?[0-9])$

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An overall value I do not think helpful

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped the ambiguous overall structural_coverage_percent entirely; now only the concrete line_coverage_percent and branch_coverage_percent remain.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You misunderstood my comment, what I meant was that a 90% gathered over all the units in a module is maybe not really interesting. For example you could have 90% coverage in all units or 100% in only 9 out of 10 - and 0% in one unit - which would be quite a different meaning but both would result in the same "overall" value. But as the PLs will be the main users of this report and you think it helpful ...

FScholPer and others added 2 commits July 28, 2026 08:53
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@FScholPer

Copy link
Copy Markdown
Contributor Author

@aschemmel-tech fixed. Can you check again?

@aschemmel-tech aschemmel-tech left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good for now

@AlexanderLanin AlexanderLanin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is 100% backwards compatible as far as I can tell. Merging.

@AlexanderLanin AlexanderLanin changed the title verification report need type feat: add verification report need type Jul 30, 2026
@AlexanderLanin
AlexanderLanin merged commit 1563b9d into main Jul 30, 2026
15 of 17 checks passed
@AlexanderLanin
AlexanderLanin deleted the verification_needs branch July 30, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

3 participants