Introduce pytest integration out of the box

Currently users need to know about pytest-bazel or other ways to integrate rules_python with bazel.

Ideally the discovery and onboarding of new users should be as easy as we can make it to be without making rules_python unclude everything and the kitchen sink.

Some ideas how we could do this:

Implement the default pypi hub idea Support a default hub repo concept in bzlmod #2094
pre-declare the pytest and pytest-bazel dependencies, where we would generate entries in the hub repo with aliases. Maybe we could define a particular default version for everything to work out of the box.
create a pytest_test rule where we are using the declared deps.
the users can override the pytest and other included deps.
use the main_module attribute to launch pytest-bazel. If pytest is not there users would get an error that it could not be imported.

Open questions:

can we default to pytest-bazel in all py_test invocations if there is no main or main_module defined?
Is "pypi" a good name to reserve as the default hub repo? How does the predeclaring work? We just accept the same args as whl_library but all optional?
Who can pre-declare deps and if multiple versions are declared, which one do we chose? We have version marker parsing in rules_python and we can even do resolution based on version ranges, e.g. users declare that they need pytest 7.1 or above and we get the lowest available version that satisfies the conditions. We can also do this by python version since we have marker expression evaluation support.
How to handle sharding related optional dependencies?
How to handle other features that may require extra plugins?

Design goals

no overhead in the number of targets, hence usage of main_module.
little maintenance in rules_python.
better out of the box experience. How would the users discover the feature?

@rickeylev

@rickeylev anything to add? @keith, any requests as a user that we should think about?

default py_test to use pytest if there's no main/main_module

Oh, that's clever. Maybe? Eh. Well. I don't see how to do so without it being considered a breaking change. Today, you have to put if name == main: call_your_main(), so if we invoke it in a way where it isn't name==main, then its breaking.

I was more thinking, as step one, we introduce a pytest_test rule, then figure out how to transition.

is pypi a good name to reserve as the default hub repo

That's what I was thinking, yes. I'm pretty sure we can do so in a non-intrusive enough way.

Who can pre-declare deps

Anyone. The basic design I had in mind was: you call pip.dep(name), which makes @pypi//name available. If no name is seen, then it maps to a target that gives an error at execution phase saying "no implementation registered"

if multiple versions are declared, which one do we chose?

Yeah, it gets murky here. I was thinking something simple, e.g. order of pip.parse calls, after accounting for the more statically set platform settings (e.g. python_verison).

default py_test to use pytest if there's no main/main_module

Oh, that's clever. Maybe? Eh. Well. I don't see how to do so without it being considered a breaking change. Today, you have to put if name == main: call_your_main(), so if we invoke it in a way where it isn't name==main, then its breaking.

I was more thinking, as step one, we introduce a pytest_test rule, then figure out how to transition.

Another option would be to also control the default main_module value through a string flag. Have it set to nothing, but users could set it to whatever. Either unittest.main, pytest-bazel equivalent or similar.

Having default test deps is also something that would have to be done.

We could flip the default with some major version bump (doesn't have to be 2.0).

That said, stepping back a little, we want thebest OOB experience and the least surprising system behaviour. Adding flags may not make it easier to configure/discover features when setting up a prject from scratch.

FWIW I don't think it's as important that we pre-supply the requirement("pytest") part. I think the primary value is from having a runner that does what users expect for pytest. Since python users are so used to providing requirements files, I think that part of the overhead, to reduce the risk of being too magical, is fine

It seems that Aspect is not doing anything super magical here: https://github.com/aspect-build/rules_py/blob/b87227504d6ec8f8fdde3dff3f467380e8198971/py/defs.bzl#L126

Just an idea before I logout:

If the user has pytest in their requirements.txt, then we can also add pytest-bazel package which can be configured via pip.defaults.
Make the main_module for py_test a string_flag that can be set to anything.
The user can easily set it in the .bazelrc to be --@rules_python//python/config_settings:py_test_main_module=pytest_bazel
Default pytest behaviour out of the box with very minimal effort and not too much magic?

Make the main_module for py_test a string_flag that can be set to anything.

How would this work? It's currently a plain string attribute.

OK, so the latest thinking is that #3837 and #3850 will help with this. We declare via pip.dep a dependency on pytest-bazel, then we create a pytest_test rule based on aignas/pytest-bazel#27

load("//python:py_test.bzl", "py_test")

def pytest_test(*, deps = [], data = [], **kwargs):
    """Use pytest-bazel for running pytest tests

    Args:
        deps: Deps to pass to the test.
        data: The data to pass to the test rule.
        **kwargs: Extra args passed to the py_test.
    """

    py_test(
        deps = (deps or []) + [
            "@pypi//pytest_bazel", # this is what is declared via the unified hub
        ],
        main_module = "pytest_bazel",
        data = (data or []) + [
            "//:pyproject.toml",
        ],
        **kwargs
    )

Should be a reasonably easy lift, I'll do that later this week.

For reference, this is what I do for pytest integration:

macro wraps all py_test targets
Gazelle uses that macro instead via map_kind directive
macro injects pytest dep, sets pytest env, adjusts main to be a shim file instead of the test file

Shim file is basically:

if __name__ == __main__:
    exit_code = pytest.main(sys.argv[1:])
    sys.exit(exit_code)

This works quite well for us because we actually have a lot more in the pytest shim (force junit-xml output for BTX, support pytest's --test_filter, skip dirs when running bazel coverage, etc.) and we do more in our wrapper macro.

... more in the pytest shim (force junit-xml output for BTX, support pytest's --test_filter, skip dirs when running bazel coverage, etc.) ...

I think Ignas's pytest_bazel package does this, too, but I'll let him comment (I see such code in it, but haven't used it myself). I'm sure he'd love feedback on how well it works for you

use main_module = "pytest_bazel" [which auto-discovers what files to run as tests]

How would this work? I don't see how it could reliably identify the file that is suppose to be the test. The main executable name and intended test file don't have to have any relationship to each other, e.g, i can create name=foo_test and have it run other/entirely.py as the test.

Another issue I see with auto-discovery is how the lack of sandboxing (e.g. with a local execution without sandboxing enabled) would cause it to see different files, potentially from different or past invocations. How would it know which of the sibling files a_test.py and b_test.py are supposed to be the test to run?

Yeah, the auto-discovery works in sandboxed cases well, but otherwise users need to pass the list of files via args or another mechanism.

I'm -1 on having it auto-discover by default, here's my rationale:

A target already knows what files should be run as tests, so discovery is unnecessary overhead.
The default test discovery logic doesn't match the typical bazel test target behavior. By which I mean: In bazel, we generally expect that a test target with source X will use X as the file that is tested, regardless of X's name, location, etc. Pytest discovery, in comparison, looks for certain file name patterns (*_test.py and test_*.py i think? But this is also customizable via plugins).
I'm just generally concerned discovery will act surprising when under the bazel runfiles structure. IIRC, pytest discovery logic is somewhat subtle and has various heuristics. e.g. the "find all tests under CWD" behavior sounds prone to finding tests in sub-directories that belong to other targets, or finding defunct files that no longer exist. Within Google, we ran into a weird behavior where tests had to be moved to directories with a hyphen in the name to thwart some pytest hueristics that mis-identified tests.

(2) does raise some interesting design questions about how discovery should/shouldn't fit into things. I can easily imagine some people just wanting to do e.g pytest_test(glob(**)) and have a single pytest invocation that runs everything.

tests: implement pytest_test rule for internal test usage (bazel-cont…

feat(rules): expose pytest_test rule for public use (bazel-contrib#4244)

Navigation Menu

Repository navigation

Introduce pytest integration out of the box #3594

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Introduce pytest integration out of the box #3594

Description

Activity

aignas commented on Feb 13, 2026

rickeylev commented on Feb 14, 2026

aignas commented on Feb 16, 2026

keith commented on Mar 2, 2026

aignas commented on Apr 26, 2026

rickeylev commented on Apr 26, 2026

aignas commented on Jul 1, 2026

dougthor42 commented on Jul 17, 2026

rickeylev commented on Jul 17, 2026

rickeylev commented on Jul 18, 2026

aignas commented on Jul 18, 2026

rickeylev commented on Jul 18, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions