Merge upstream NVIDIA/SkillSpector keshavp/oss-release-2026-07-21 (v2.4.2) by will-exaforce · Pull Request #12 · ExaForce/SkillSpector

will-exaforce · 2026-08-03T15:18:10Z

Summary

Merge 51 upstream commits from NVIDIA/SkillSpector:keshavp/oss-release-2026-07-21 into the fork, covering the 2026-07-13, 2026-07-14 and 2026-07-20 OSS release snapshots.
Bumps skillspector 2.3.11 → 2.4.2, and syncs benchmark/uv.lock to match the editable dep.

Details

Last sync point was c2d09df (v2.3.11, PR #9). No file-level conflicts — upstream touched none of the paths this fork carries (benchmark/, src/skillspector/exaforce/, tests/exaforce/); the changed-file sets intersect at zero files. The contrib/multilingual/ → contrib/batch_scan/ move came through as renames rather than add/delete.

Notable upstream content: reasoning-effort pass-through for Anthropic and OpenAI-compatible providers (NVIDIA#283), documentation-false-positive gating for PE3/RA1/TM1/AR2 (NVIDIA#251), full finding metadata preserved in SARIF output (NVIDIA#229), scoped LLM provider injection (NVIDIA#243), YARA packaging/unicode fixes (NVIDIA#236), MCP stdio initialize compatibility (NVIDIA#199), SC7 untrusted-container-image detection, and event-loop-reuse fixes.

Fork-patch drift check. Upstream rewrote meta_analyzer.py, llm_utils.py, providers/* and chat_models.py — all targets of this fork's exaforce schema-pruning monkeypatch. A clean textual merge does not imply the patch still binds, so this matters: _patchlib.py raises PatchDriftError at import time on a missing field or changed prompt text, and apply_patches() runs from src/skillspector/__init__.py, so any drift fails every test that imports skillspector. The suite passing is therefore real evidence the patch still applies to the new upstream code.

NOTE

No notable improvements were observed when comparing to the current version but efficacy did not degrade. This is primarily an effort to keep drift from upstream manageable. Important observations between runs below.

	base · `Ver 2.3.11`	head · `Bump to keshavp/oss-release-2026-07-21`
run id	`a1d8b9cd9a4b`	`2994ce3de3dc`
description	Ver 2.3.11	Bump to keshavp/oss-release-2026-07-21
model	nvidia.nemotron-super-3-120b	nvidia.nemotron-super-3-120b
LLM	on	on
units (total)	900	900
started	2026-07-13 04:00	2026-08-03 08:36
workers	8	8

	base	head	Δ
✅ correct (TP + TN)	807	816	▲ +9
true positives · malware caught	523	531	+8
true negatives · benign passed	284	285	+1
🔴 incorrect (FP + FN + errors)	93	84	▲ -9
false positives · benign flagged	16	13	-3
false negatives · malware missed	64	64	0
errors / timeouts	13	7	-6
total scanned	900	900	0

Overall correctness (correct ÷ scanned, errors counted as wrong): 0.897 → 0.907 (▲ +0.010).

	scans	avg	median	p95	max	total scan time	~est. wall
base	900	51.34s	10.70s	245.85s	300.01s	12h50m	1h36m
head	900	49.46s	10.18s	201.72s	300s	12h21m	1h32m

Verified locally:

Suite	Result
`uv run pytest tests/exaforce -q`	10 passed
`uv run pytest -q` (full)	1378 passed, 4 failed, 14 skipped, 6 xfailed (was 1261 passing pre-merge)
`cd benchmark && uv run pytest -q`	4 passed

The 4 failures are the pre-existing, documented ones in docs/superpowers/EXPECTED_TEST_FAILURES.md — upstream tests deliberately kept at upstream parity that assert the un-pruned schema. They fail the documented way (AssertionError on a pruned explanation/intent key), not as import/collection errors.

Known pre-existing CI failure, not introduced here: make lint / make format-check (scoped to src/ tests/) is red on two fork-owned files — E402/import-order plus a formatting diff in src/skillspector/__init__.py (the exaforce activation import must come after the module body), and UP037 at src/skillspector/exaforce/_schema_patches.py:18. Confirmed against pre-merge a57ac2a with the same ruff version, so this merge did not cause it. Left alone here to keep the merge reviewable; fixing __init__.py also pushes an upstream-parity file further from upstream.

No structured plan — mechanical upstream merge; no fork code authored.

Summary: This fix allows SkillSpector to run in environments that already have a running event loop, preventing RuntimeError when asyncio.run() is called from within an existing loop. Problem: When running SkillSpector in environments like: - Jupyter Notebooks - LangGraph Studio - FastAPI applications - Any programmatic usage within async code The call to asyncio.run() throws a RuntimeError: This event loop is already running and falls back to unfiltered static findings, silently disabling LLM analysis. The previous approach of detecting this state via error message substring matching is fragile and locale-dependent. Solution: 1. Add utility function in that properly detects running loops using 2. When no running loop exists, fall back to directly 3. When a loop is already running, offload execution to a separate thread with its own event loop via 4. Replace all calls across all analyzer nodes with the new helper 5. Remove unused asyncio imports from analyzer files Test: Add comprehensive unit tests for run_async covering: - Normal execution without existing running loop - Nested execution inside an already running loop - Exception propagation from async coroutines - Correct handling of async functions with await calls Signed-off-by: zhenliemao <494822673@qq.com>

Signed-off-by: zhenliemao <494822673@qq.com>

supply_chain (SC1-SC6) covers package dependencies but not the container-image supply chain. A skill pulling images with verification disabled (--disable-content-trust, DOCKER_CONTENT_TRUST=0, --insecure-registry) accepts tampered images but scored 9/SAFE (NVIDIA#223). Add SC7_PATTERNS to the supply_chain analyzer (is_code_example filter) with pattern_defaults entries and 5 tests. --tls-verify=false is excluded since TM3's verify=False already covers it. Signed-off-by: CharmingGroot <ohyes9711@gmail.com>