Skip to content

SHACL CI: check out CGMES-AP and NC-AP directly #404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BenedekFodor wants to merge 1 commit into
base: cgmes-3.0_ncp-2.5_tc-2.1
Choose a base branch
from
Open

SHACL CI: check out CGMES-AP and NC-AP directly #404

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions .github/workflows/shacl_validation.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,19 +21,25 @@ jobs:
steps:
- uses: actions/checkout@v4

# main carries both CGMES 3.0 and NCP 2.5 (PROF + SHACL)
- name: Checkout application profiles library (main)
# application-profiles-library only links CGMES/ and NCP/ as submodules
# since 2026-10-15; check out the two AP repos (PROF + SHACL) directly
- name: Checkout CGMES application profiles (CGMES 3.0)
uses: actions/checkout@v4
with:
repository: entsoe/application-profiles-library
ref: main
path: .apl-main
repository: entsoe/CGMES-AP
path: .apl-cgmes

- name: Checkout NC application profiles (NCP 2.5)
uses: actions/checkout@v4
with:
repository: entsoe/NC-AP
path: .apl-ncp

- name: Install uv
uses: astral-sh/setup-uv@v6

- name: Validate instances, export SARIF + SHACL reports
run: uv run buildScripts/validate_instances.py --apl cgmes-3.0=.apl-main --apl ncp-2.5=.apl-main
run: uv run buildScripts/validate_instances.py --apl cgmes-3.0=.apl-cgmes --apl ncp-2.5=.apl-ncp

# one upload per SARIF run — code scanning does not combine
# multiple runs uploaded under the same category
Expand Down
4 changes: 3 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,11 @@ validation_report/
__pycache__/
*.pyc

# APL checkout + caches for SHACL validation #
# AP checkouts + caches for SHACL validation #
############
.apl-main/
.apl-cgmes/
.apl-ncp/
.shacl_cache/

# generated validation reports (CI run artifacts) #
Expand Down
10 changes: 6 additions & 4 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,12 +131,14 @@ the workflow's `Run workflow` button, which opens `auto/manifests-manual-<run-id
### Running Validation

SHACL + schema conformance of every instance file (what CI runs, see
`.github/workflows/shacl_validation.yml`; needs one application-profiles-library
checkout of `main`, which carries CGMES 3.0 and NCP 2.5):
`.github/workflows/shacl_validation.yml`; needs checkouts of `entsoe/CGMES-AP` for CGMES 3.0
and `entsoe/NC-AP` for NCP 2.5 — the application-profiles-library only links them as
submodules, with an extra folder level, so it cannot be used directly):

```bash
git clone --depth 1 https://github.com/entsoe/application-profiles-library .apl-main
uv run buildScripts/validate_instances.py --apl cgmes-3.0=.apl-main --apl ncp-2.5=.apl-main
git clone --depth 1 https://github.com/entsoe/CGMES-AP .apl-cgmes
git clone --depth 1 https://github.com/entsoe/NC-AP .apl-ncp
uv run buildScripts/validate_instances.py --apl cgmes-3.0=.apl-cgmes --apl ncp-2.5=.apl-ncp
# SARIF + sh:ValidationReports + summary.md written to reports/
```

Expand Down
11 changes: 6 additions & 5 deletions buildScripts/validate_instances.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
gets until the closure variant is adopted.

Run:
uv run buildScripts/validate_instances.py --apl cgmes-3.0=.apl-main --apl ncp-2.5=.apl-main
uv run buildScripts/validate_instances.py --apl cgmes-3.0=.apl-cgmes --apl ncp-2.5=.apl-ncp
"""
import argparse
import json
Expand All @@ -40,12 +40,13 @@
REPORTS = REPO_ROOT / "reports"
LEVEL_ICONS = {"error": "🔴", "warning": "🟠", "note": "🔵"}

# One APL checkout (main) carries both CGMES 3.0 and NCP 2.5 PROF + SHACL.
# PROF + SHACL come from the AP repos the APL links as submodules: entsoe/CGMES-AP
# (CGMES 3.0) and entsoe/NC-AP (NCP 2.5), each checked out on its own.
# The NCP 2.5-dev export schema is vendored (buildScripts/schemas/, provenance
# in SOURCE.json) until a triplets release ships it.
RELEASES = {
"cgmes-3.0": {"apl": ".apl-main", "kind": "cgmes", "rdf_map": schemas.ENTSOE_CGMES_3_0_0_552_ED1},
"ncp-2.5": {"apl": ".apl-main", "kind": "nc",
"cgmes-3.0": {"apl": ".apl-cgmes", "kind": "cgmes", "rdf_map": schemas.ENTSOE_CGMES_3_0_0_552_ED1},
"ncp-2.5": {"apl": ".apl-ncp", "kind": "nc",
"rdf_map": str(REPO_ROOT / "buildScripts/schemas/ENTSOE_NC_2.5-dev_552_ED1.json")},
}

Expand Down Expand Up @@ -480,7 +481,7 @@ def main():
os.chdir(REPO_ROOT) # relative source paths => repo-relative SARIF artifact URIs
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--apl", action="append", default=[], metavar="RELEASE=PATH",
help="APL checkout per release, e.g. cgmes-3.0=.apl-main (repeatable)")
help="APL checkout per release, e.g. cgmes-3.0=.apl-cgmes (repeatable)")
parser.add_argument("--only", choices=sorted(RELEASES), help="run a single release")
parser.add_argument("--no-closure", action="store_true", help="skip the dependency-closure comparison run")
args = parser.parse_args()
Expand Down
Loading