Skip to content

[AAASM-6359] 🔧 (ci): Run the native core runtime tests in CI - #364

Merged
Chisanan232 merged 3 commits into
mainfrom
v0.0.1/AAASM-6359/config/native_core_ci
Oct 20, 2026
Merged

Chisanan232 merged 3 commits into
mainfrom
v0.0.1/AAASM-6359/config/native_core_ci

Conversation

@Chisanan232

Copy link
Copy Markdown
Contributor

Description

test/integration/test_native_core_runtime.py skips itself unless AAASM_RUN_NATIVE_CORE_TESTS=1 and no workflow set it, so the native-core runtime tests never ran in CI (AAASM-6359; the two stale tests were already fixed in #363). This wires them into the existing native-core-build job, which already builds the PyO3 core with maturin.

  • 🔧 (ci): the job sets the gate variable and runs only that file. -o addopts="" drops pytest.ini's --reruns 1 (a retry would hide a flaky timing assertion). A skip exits 0, so the job also fails if the junit report shows any skipped test; otherwise a removed or typo'd variable would be a silent pass. The file is added to the PR path filter.
  • ✅ (test): test_send_event_is_non_blocking flaked while validating this: 2 of 14 runs under CPU contention measured 255-286 ms against its 250 ms budget (steady about 54 ms idle). Budget is now 1 s, with the comment updated. This is a weak oracle for "does not block" (it is a wall-clock bound); a structural test with a deliberately slow runtime would be stronger and is out of scope here.

CI-definition change: minimal diff, existing action pins and permissions: contents: read untouched, no new secrets.

Type of Change

  • ✨ New feature
  • 🔧 Bug fix
  • ♻️ Refactoring
  • 🍀 Performance improvement
  • 📚 Documentation update
  • 🚀 Release

(Configuration / CI change; the template has no such box.)

Breaking Changes

  • No
  • Yes (please describe below)

Related Issues

Testing

  • Unit tests added/updated
  • Integration tests added/updated
  • Manual testing performed
  • No tests required (explain why)

Local, with the exact job steps against a freshly maturin-built core (venv and target dir under /private/tmp): 13 passed, junit skipped="0" grep exit 0. Gate off: 13 skipped and the grep exits 1, so the job would fail rather than pass silently. Mutation: changing the expected decision in test_query_policy_returns_deny_when_runtime_denies from deny to allow fails that test (1 failed, 12 passed); restored byte-identical. After the budget change, 10 of 10 gated runs passed. Not verified: that the new CI step is green on a GitHub runner (checked after this PR is up), and that uv run --with pytest keeps the maturin-installed _core in the CI venv.

Checklist

  • Code follows project style guidelines
  • Self-review completed
  • Comments added for complex logic
  • Documentation updated if needed
  • All tests passing

🤖 Generated with Claude Code

https://claude.ai/code/session_01JvXQ9VcS72E7DREhsgjyHV

Chisanan232 and others added 2 commits October 19, 2026 11:08
The 250 ms budget measured 255-286 ms in 2 of 14 runs under CPU contention.
1 s still separates enqueueing from a per-event runtime wait.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvXQ9VcS72E7DREhsgjyHV
The env-gated test file never ran in CI. The job sets the gate variable, drops
the rerun plugin, and fails if any test was skipped.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvXQ9VcS72E7DREhsgjyHV
@codecov

codecov Bot commented Oct 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

The native runtime test drives runtime_interceptor, runtime_audit and assembly,
which import _core, so editing only those files skipped the job that tests them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvXQ9VcS72E7DREhsgjyHV
@sonarqubecloud

Copy link
Copy Markdown

@Chisanan232

Copy link
Copy Markdown
Contributor Author

Review record (independent AI review, author-account posting) — NOT a GitHub approval.

Reviewed head: e591122b5d9dfda644d41a4de40534b63f827d03 (AAASM-6359). Independent review of 74752f6 = APPROVE-WITH-NOTES; I read the one later commit (e591122b5, +2 lines: agent_assembly/core/** added to the pull_request paths), which addresses the review's trigger-gap note.

Verified (independent reviewer, from CI log and a local pytest 9.1.1 run):

  • Permissions stay contents: read; no secrets; the new run block has no ${{ }}; action pins unchanged.
  • The skipped-count guard fails closed: a missing junit attribute, a removed gate variable, a missing _core, a collection error and zero collected tests all fail the job. (An xfail would false-fail; none exist in the file.)
  • uv run --with pytest did not disturb the maturin-installed _core: CI log shows collected 13 items, 13 passed, --reruns dropped.
  • On my side: gate off gives 13 skipped and the grep exits 1; a mutated deny→allow assertion fails that test; 10 of 10 gated runs pass after the budget change.

CI on this head: build-native-core = success (new step included), CI Success, CodeQL, unit/integration, pip-audit, SonarCloud, codecov/patch all success; no failing check-run.

Notes (none blocking): (1) no push to main trigger, so nothing re-runs this after merge; (2) the 1 s budget only catches a regression to blocking if a blocking send costs more than about 2 ms per event, and the PR says so; (3) --with pytest is redundant (harmless); (4) the 2-of-14 flake statistics are my local measurements.
Not verified: behaviour on a runner under heavy load.
Merge: author is the owner account, so no formal self-approval; needs the owner's merge (head-pinned merge commit).

@Chisanan232
Chisanan232 merged commit 64c169c into main Oct 20, 2026
25 checks passed
@Chisanan232
Chisanan232 deleted the v0.0.1/AAASM-6359/config/native_core_ci branch October 20, 2026 02:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant