Repository navigation
Conversation
setuptools-grpc already adds build_grpc to the build command through an entry point. The copy in setup.py made every editable install fail with "command 'build_grpc' has no such option 'proto_files'". So `uv sync` and `pip install -e .` failed. The wheel does not change: same files, same generated modules. Tests: a check of setup.py, and a slow test (CHAKRA_TEST_SLOW=1) that builds an editable install and imports a generated module. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add [dependency-groups] to pyproject.toml, at the end of the file. The dev group has the same pins as requirements-dev.txt. The param group installs PARAM (et_replay) from git at the commit that CI uses. It is temporary. Ignore .venv/ and uv.lock. There is no .python-version file, on purpose. pyenv, mise and asdf read it, so it would reach people who do not use uv. A test guards this. uv rebuilds the package when a .proto file changes (cache-keys). Without this, the generated *_pb2.py files stay old after a .proto edit. requirements-dev.txt and the workflows do not change. Tests check that the two pin lists stay equal and that the PARAM commit matches CI. These tests read files that other branches change, so they run only when CHAKRA_TOOLING_TESTS=1 (`just test` sets it). The existing CI job does not set it, so an open branch does not fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Shell logic goes in scripts/, so it can be linted and tested alone: - run_on_git_files.sh runs a command on tracked and new files, not on ignored or deleted ones. It stops on an empty list, so a check cannot pass on nothing. A file that is not added to git yet is checked too. - proto_smoke.sh is the CI "Protobuf Gencode/Runtime Smoke Test": it imports the generated protobuf modules with the newest protobuf, then with the lowest one that pyproject.toml allows. - wheel_check.sh checks the package the way a user and CI get it. It builds a wheel from a clean copy, installs it (not editable) into a new environment, and tests it there. The other checks use the editable install that `uv sync` makes, and an editable install can hide a file that the wheel does not hold. A slow test shows this: with the .proto files left out of the package data, the editable install still works and the wheel check fails. .gitignore ignores *.pb.cc and *.pb.h, the protoc output of the C++ feeder build, so a check of new files does not see generated code. tests/test_scripts.py runs each script with fake programs first on PATH. It uses real git repositories, isolated from the git settings of the caller (a git hook sets GIT_INDEX_FILE). It needs no network, except the slow wheel test. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
`just gate` runs the checks that CI runs: ruff, vulture, pyright, the unit tests, the protobuf smoke test and clang-format 16. It also checks the Python format (`ruff format --check`), the shell scripts (shellcheck), and the built wheel (`just wheel-check`), so that both ways to install the package are tested. CI runs `ruff format .` without --check, so a format error never failed there. The justfile sets UV_PYTHON to 3.11, the version that CI uses, so uv needs no .python-version file. Also: `just test-slow`, `just fmt`, `just setup` and `just ci-github-lint` (actionlint in Docker; not part of the gate). Tests check the recipes, that CI and the justfile run the same commands, that UV_PYTHON matches the Python version of CI, and that a comment does not point to a missing file. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
`just act [workflow]` runs one workflow in Docker with act (scripts/act.sh). The script finds the Docker socket and the CPU type itself, so it works with Docker Desktop and OrbStack, on arm64 and x86_64. It tells act not to mount the Docker socket into the job. The workflows do not need it, and it is a likely fix for Docker Desktop on macOS. Before each run the script removes the shared act toolcache volume and the act containers that use it. Without this, a package that one run installed is visible to the next run, and a failed job leaves a container that holds the volume. The script refuses to start while another act run is active on this computer, and it says what it removes. It also stops if pgrep, which it needs for that check, is missing. .actrc maps ubuntu-latest to catthehacker/ubuntu:act-24.04. Ignore .secrets, .env and .vars, which act reads. Tests run the script with fake docker, act, uname and pgrep programs. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
README.md gets a short Development section: two commands, a line that the pip setup still works, and a link. doc/DEVELOPMENT.md is the developer guide: what to install, the daily steps, the recipes, the shell scripts, how to run the CI workflows with act (and which ones work), and the classic pip setup, which still works. USER_GUIDE.md gets an "install with uv" alternative for end users. Tests check that the README section stays short, that links resolve, that the docs name only recipes that exist and list every recipe, that sentences have at most 25 words (a simple Simplified Technical English check), that the docs use the PARAM commit of pyproject.toml, that the classic setup installs PARAM before it runs the tests, and that the README says the pip setup still works. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Brings in PR mlcommons#259 (clean-install check). No conflicts.
CONTRIBUTING.md now tells contributors where the development guide is and to run `just gate` before a pull request. Tests check that the link is there and that it resolves. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The recipes and the scripts need bash. The development guide now says to use WSL2 on Windows, that native Windows is not supported, and that the steps were tested only on macOS. A test checks that the section exists and names WSL2 and bash. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A tester ran `just setup` and `just gate` in a WSL instance of Debian 13 (trixie). The Windows section now says what was tested, and no longer names Ubuntu, which nobody tested. The test for the Windows section checks the new wording. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
MLCommons CLA bot: |
Contributor
|
recheck |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #261. Part of Goal 0, #260.
What this adds
A modern dev environment next to the pip setup. Nobody has to move to uv.
just setupinstalls Python 3.11, the dev tools and PARAM (uv).just gateruns the full local check: ruff, vulture, shellcheck, pyright, tests, a protobuf smoke test, a wheel check and the C++ format check.just act <workflow>runs a GitHub workflow in Docker on your computer.doc/DEVELOPMENT.mdexplains it all. README and CONTRIBUTING link to it. USER_GUIDE has an "install with uv" option.Start here: doc/DEVELOPMENT.md.
What does not change
.github/(CI workflows)requirements-dev.txt,setup.cfgpython -m pytest testsCHAKRA_TOOLING_TESTS=1is set (just testsets it).just wheel-check)pyproject.tomlgets new sections at the end only. The pip dev path is removed in a later PR (#264), and only when the maintainers agree.One code change to know
setup.pybecomessetup(). The custombuild_grpcclass in it repeats the one thatsetuptools-grpcalready adds, and it brokeuv sync(editable install). #254 makes the same change.Evidence
just gatein a fresh clone of this branch (macOS arm64)python -m pytest testspython_lint,python_tests,end_to_end_tests,cpp_lint) underact(OrbStack)just setupandjust gatein a WSL Debian 13 (trixie) instancemainissue-254-linterjust testneeds 2 new pins (see below). pyright reports 4 errors in its new tests.Three independent reviews found no blockers. We took their fixes.
If you have an open branch
just gateis local and optional. Run it before your PR if you use the new setup.[dependency-groups] devinpyproject.toml.just testchecks that it matchesrequirements-dev.txt. Addchakra_linterto validate Chakra execution traces against a rule set #254 addsjsonschema==4.21.1andmarkdown==3.11.2. Whoever merges second adds these two pins.chakra_linterto validate Chakra execution traces against a rule set #254: pyright 1.1.359 reports 4 errors intests/linter/test_attribute_rules.py(lines 17, 84, 94) andtest_timing_rules.py(line 163). The tests pass. Only the type check fails.Not tested
just act(it works on OrbStack).clean_install_tests.ymlworkflow from Add clean-install check script and CI job #259 underact.feeder_tests.ymlworkflow fails underact(a Dockeraptprompt). It is not part ofjust gate. It is planned in Run CI through the just recipes #263.Next steps (not in this PR)
uv.lockafter Handle et_replay installation #237 and Addchakra_linterto validate Chakra execution traces against a rule set #254 merge (Pin the dev tools and add uv.lock #262).justrecipes (Run CI through the just recipes #263).🤖 Generated with Claude Code