Repository navigation
PROF-driven SHACL + schema validation for CGMES 3.0 + NCP 2.5 with code scanning integration - #397
Merged
HarisVranaj merged 2 commits intoSep 11, 2026
Conversation
added 2 commits
September 7, 2026 11:32
…ode scanning) Port of the cgmes-3.0_ncp-2.4_tc-1.1 pipeline (entsoe#387) to this branch, adapted to the application-profiles-library main line: - buildScripts/prof_map.py: besides the role/constraints shape files, resolve the NCP 2.5 role/validation aggregates — owl:imports closures resolved by basename in the checkout (SHACL/** incl. Validation/) — and expose the shapes they mark sh:deactivated. - buildScripts/validate_instances.py: one APL checkout (main) serves both releases; NC shape sets are the validation aggregates plus the AllComplex aggregate for the union pass; APL deactivations honoured by dropping those shapes' findings (triplets 0.2.0 evaluates them); per-dataset findings are the counting constraints, data-graph counts (ClassCount, PrefixDeclaration) and class membership of shapes the per-file pass re-runs; superseded dataset versions (Dataset_version_dependency/) never join a union frame; boundary dcat:Dataset headers (no profile declared) validated as CoreEquipment-EU/3.0 (BOUNDARY_PROFILE_FIX, documented hardcode); Jotunheim/GridSituation/cimxml skipped as duplicate of Jotunheim/NetworkCode. - Dependency-closure comparison run for NC: reference checks scoped to each file + its transitive dcterms:requires closure, per-rule diff against the type split in summary.md, closure-ncp-2.5.sarif as run artifact only. - buildScripts/schemas/: vendored NCP 2.5-dev export schema (triplets format, provenance in SOURCE.json) passed in as rdf_map until a triplets release ships it; triplets pinned to 0.2.0, pandas 2.3.3 excluded. - .github/workflows/shacl_validation.yml: single APL checkout, categories shacl-/schema-{cgmes-3.0,ncp-2.5}. Local run: 127 files mapped, ~2.5 min.
Collaborator
Author
HarisVranaj
approved these changes
Sep 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Counterpart of the 2.4-branch PR #387 (
cgmes-3.0_ncp-2.4_tc-1.1, CGMES 3.0 + NCP 2.4 — stays as is) forcgmes-3.0_ncp-2.5_tc-2.0, adapted to the NCP 2.5 line of the application-profiles-librarymain. Implements #381 on this branch. Fork CI run: https://github.com/Haigutus/relicapgrid/actions/runs/34101124661 (green, four SARIF categories uploaded).What it adds:
buildScripts/prof_map.py— maps each instance file's declared profile (md:Model.profile/dcterms:conformsTo) to its SHACL files via the DX-PROF descriptors. For NCP 2.5 it also resolves therole/validationaggregates: theirowl:importsclosure is resolved by basename in the checkout, and the shapes they marksh:deactivatedare honoured.buildScripts/validate_instances.py— validates everyInstance/**/*.xml(127 mapped, ~2.5 min): SHACL in cross-file-resolving groups (per area + boundary + CGM assembly; NC per area with Grid context) with per-dataset constraints (sh:minCount/maxCount/sh:closed, ClassCount/PrefixDeclaration, AllowedClasses) evaluated per instance file; a shapes-independent schema-conformance layer; aModel.DependentOn/requiresresolvability check. NC additionally runs a dependency-closure comparison (each file over itself + its transitivedcterms:requires), reported as a per-rule diff in the run summary — informational, not uploaded..github/workflows/shacl_validation.yml— one APL checkout (maincarries CGMES 3.0 and NCP 2.5), four code-scanning categories (shacl-/schema-×cgmes-3.0/ncp-2.5), full sh:ValidationReports + summary as run artifact.buildScripts/schemas/— the NCP 2.5-dev export schema (triplets format, provenance inSOURCE.json), vendored until a triplets release ships it; triplets pinned to 0.2.0.Differences from #387: single APL checkout; NC shape sets come from the PROF
role/validationaggregates (2.4 had none); the closure comparison run; the vendored schema; and the branch-specific policies below.Policies encoded (each documented at its constant in
validate_instances.py):CoreEquipment-EU/3.0until Boundary dataset headers declare no application profile (cgmes-3.0_ncp-2.5_tc-2.0) #395 is fixed (BOUNDARY_PROFILE_FIX).Instance/Jotunheim/GridSituation/cimxml/duplicatesInstance/Jotunheim/NetworkCode/(same dataset UUIDs) → skipped;Dataset_version_dependency/files validated alone, never in a union frame.Svedala_FAP_PV.xml(Provenance/1.0has no PROF).Issues found by this run:
conformsToEQ.GridStateAlterationRemedialActionin Espheim RA; IAM 01:30 ↔AssessedElementWithContingencyin Belgovia AE); surfaced by the closure comparison.dcatcim:prefix, imports pinned tomain).#SubstationisDescription+concretewith no attributes NC-AP#23 — RA#SubstationisDescription+concretewith no attributes (empty class in the RA RDFS; schema generation emits an empty class).SvStatus.ConductingEquipment-valueType/sh:inbulk = SV Simple SHACL: SvStatus.ConductingEquipment-valueType sh:in lists only CsConverter/VsConverter — every other ConductingEquipment reference fails CGMES-AP#11.Notes for review:
ContingencyEquipment.Equipment-valueType/sh:in(314) andAssessedElement.ConductingEquipment-valueType/sh:in(316): SSH continues EQ objects as<cim:Equipment rdf:about>, so on the area union the target carries types {ACLineSegment, Equipment} and the APL( assoc rdf:type ) sh:incheck fails per SHACL semantics; under the declaredrequiresclosure it does not. Left in the code-scanning result for now; the summary table shows the diff.security-events: write(in the workflow); alerts appear under Security → Code scanning after the first run on the branch, tooltriplets-shacl.