Skip to content

Add clean-install check script and CI job - #259

Merged
HanjiangWu313 merged 2 commits into
mlcommons:mainfrom
Keysight:ci/issue-227-clean-install-check
Nov 11, 2026
Merged

HanjiangWu313 merged 2 commits into
mlcommons:mainfrom
Keysight:ci/issue-227-clean-install-check

Conversation

@razvcris

@razvcris razvcris commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds a script and CI job that prove a fresh install of Chakra works on every supported Python. Addresses #227.

scripts/check_clean_install.sh:

  1. Builds the wheel (python -m build) from a copy of the git-tracked files.
  2. Installs the wheel and PARAM et_replay in a fresh venv.
  3. Runs the six tools with --help.
  4. Imports chakra.schema.protobuf.et_def_pb2 and storage_pb2 in an isolated interpreter and checks that both .proto files are packaged.
  5. Runs chakra_trace_link and chakra_converter on the fixture. For each tool it checks: exit code 0, the output exists and decodes (linked JSON parses; the .et reads with protolib and has more than 0 nodes), and no ERROR/CRITICAL line or Traceback in stdout, stderr or debug.log. It also fails on "Critical path analysis encountered an invalid graph structure". Warning lines are counted and reported, not fatal.
  6. If a run fails, the script copies its logs, trace outputs and inputs (linked JSON, .et, debug.json and the input traces) to FAILURE_ARTIFACTS_DIR, and the workflow uploads them as an artifact (clean-install-<python version>), so a failure on one Python version can be investigated without rerunning it.

The script runs standalone (e.g. PYTHON=python3.12 scripts/check_clean_install.sh) and in CI (.github/workflows/clean_install_tests.yml).

Fixture

tests/data/simple_onerank_traces.tar.gz contains host.0.json and device.0.json generated by trace_collection/post_execution/simple_onerank.py on a CUDA machine.

Python versions

The workflow runs the script on Python 3.11, 3.12 and 3.13 (requires-python >= 3.11).

Test Plan

Run locally in a clean environment, as the workflow does (PYTHON=python scripts/check_clean_install.sh):

  • Python 3.11.18 - passed (592 nodes in the .et, 10 WARNING lines)
  • Python 3.12.16 - passed (592 nodes in the .et, 10 WARNING lines)
  • Python 3.13.18 - passed (592 nodes in the .et, 10 WARNING lines)

The first real run on ubuntu-latest is this PR's CI.

Additional Notes

  • GPU-collected traces only. I also tried a trace set collected on a CPU-only run. chakra_trace_link fails on it: load_sync_dependencies calls HTA/traceinsight critical_path_analysis, which expects GPU kernels and fails on a trace with none.
  • The fixture has no cuda_sync events (the profiler disables them by default).
  • closes clean-install check #227

@razvcris
razvcris requested a review from theodorbadea November 9, 2026 14:17
@razvcris razvcris self-assigned this Nov 9, 2026
@razvcris razvcris added enhancement New feature or request packaging labels Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅


- name: Check Clean Install
run: |
PYTHON=python scripts/check_clean_install.sh

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A failed run discards all logs.

  • Why it matters: The script deletes ${WORK} on exit through the trap. A CI failure on one Python version leaves only the console replay. There is no debug.log, linked trace or .et output to inspect.
  • Suggested Fix: Keep the work directory on failure and upload it.

on:
pull_request:
workflow_dispatch:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow does not restrict the GITHUB_TOKEN.

  • Why it matters: The job runs third-party code from a pinned git dependency. Default token scopes can be broader than the job needs. CodeQL flags a missing permissions block.

@HanjiangWu313
HanjiangWu313 self-requested a review November 10, 2026 22:34

@HanjiangWu313 HanjiangWu313 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally on both standalone and with current main; all clean-install checks passed—looks good to me.

@theodorbadea

Copy link
Copy Markdown
Contributor

This seems to be missing versions 3.14 and 3.15, any particular reason?

strategy:
fail-fast: false
matrix:
python-version: ['3.11', '3.12', '3.13']

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should add 3.14 and 3.15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request packaging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

clean-install check

4 participants