Skip to content

Let conftest.py skip its directory only with pytest.importorskip - #15143

Open
RonnyPfannschmidt wants to merge 4 commits into
mainfrom
claude/project-thread-zeq5mi
Open

RonnyPfannschmidt wants to merge 4 commits into
mainfrom
claude/project-thread-zeq5mi

Conversation

@RonnyPfannschmidt

@RonnyPfannschmidt RonnyPfannschmidt commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Requested by Ronny · project thread

🤖 Written by Claude Opus 5.5 via Claude Code for the pytest maintainers; I prompted it, it did the work, I read it.

Closes #15142.

Before: a conftest.py with tables = pytest.importorskip("tables") skipped its directory under pytest, but pytest tests/io/pytables crashed with a raw Skipped traceback. A plain pytest.skip() in a conftest silently skipped the whole package, and pytest.fail() crashed the same way.

After: pytest.importorskip() in a conftest skips its directory however pytest is invoked, including node ids below it. pytest.skip() in a conftest, with or without allow_module_level=True, still skips but warns (PytestRemovedIn10Warning) and becomes an error in pytest 10. pytest.fail()/pytest.xfail() are reported as a conftest import error.

How

  • New internal SkippedModule(Skipped) in _pytest.outcomes, raised by pytest.skip(allow_module_level=True), and its subclass SkippedImport, raised by pytest.importorskip() (including the minversion skip). Both set allow_module_level=True, so python.importtestmodule and anything reading the flag behave as before; their __qualname__ is Skipped, so reports are unchanged.
  • _importconftest: SkippedImport propagates; any other Skipped warns first, then propagates; any other OutcomeException becomes ConftestImportFailure like other exceptions (cause is typed Exception | OutcomeException). The skip is cached per conftest, so the conftest runs once even when it is both an initial conftest and reached by collection.
  • _set_initial_conftests leaves an anchor alone when a conftest on the way skips; the Dir collector re-raises the cached skip and reports it.
  • Session.collect: a skipped collector on the way to an argument counts as matching it, so pytest test_mod.py::test_a into a module-level skip reports the skip instead of ERROR: found no collectors (applies to test modules too).
  • --import-mode=importlib: _import_module_using_spec drops the module from sys.modules when executing it raises, like importlib does.

allow_module_level / SkippedModule assessment

Scan of 53k test files and conftests in 177 popular repos:

usage calls repos
importorskip at module top level 1297 76
skip(..., allow_module_level=True) at module top level 166 41
same, in a helper function called at import (e.g. numpy f2py) 6 5
same, in a package __init__.py 5 2
importorskip at conftest top level 10 6
skip(..., allow_module_level=True) in a conftest 0 0
explicit allow_module_level=False 0 0
reading .allow_module_level on an exception 0 0 (only pytest itself)
constructing Skipped(..., allow_module_level=...) directly 0 0

So the flag is only ever used as pytest.skip(..., allow_module_level=True), and nobody inspects it. That makes it a type in all but name; the subclasses let the conftest loader and importtestmodule ask isinstance() instead of reading an attribute. The allow_module_level parameter and attribute stay for compatibility.

Checklist

  • Tests: TestConftestImportOutcome (test_conftest.py), node-id and SkippedModule/SkippedImport tests (test_skipping.py), importlib sys.modules test (test_pathlib.py). The new tests fail on main.
  • Docs: how-to/skipping.rst "Skipping a whole directory", deprecations.rst entry.
  • Changelog: 15142.bugfix.rst, 15142.deprecation.rst.
  • AI agent credited in Co-authored-by trailer.

Full test suite passes locally (Python 3.13) apart from test_pdb_unittest_postmortem, a pexpect timing test that failed once and passed on 3 reruns; pre-commit run --all-files clean except zizmor, which needs GitHub API access not available here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GHmGPdXhVMcboqUeE6q62M

@RonnyPfannschmidt RonnyPfannschmidt self-assigned this Oct 6, 2026
@pyba-chronographer pyba-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Oct 6, 2026
@RonnyPfannschmidt
RonnyPfannschmidt marked this pull request as ready for review October 7, 2026 05:52
@RonnyPfannschmidt RonnyPfannschmidt changed the title Let conftest.py skip its directory only with a module-level skip Let conftest.py skip its directory only with pytest.importorskip Oct 7, 2026
Ronny Pfannschmidt and others added 4 commits October 27, 2026 09:06
A conftest.py that raised an outcome exception while being imported got
different results depending on how pytest was invoked: a skip during
collection skipped the directory (even a plain pytest.skip()), while the
same conftest loaded as an initial conftest crashed with a raw Skipped
traceback, and pytest.fail()/xfail() escaped the conftest import error
handling.

- Add SkippedModule, an internal Skipped subclass raised by
  pytest.skip(allow_module_level=True) and pytest.importorskip().
- A conftest raising SkippedModule skips its directory, also for
  initial conftests; the skip is cached so the conftest runs once.
- A plain pytest.skip() in a conftest still skips but issues a
  PytestRemovedIn10Warning.
- pytest.fail()/xfail() in a conftest become a ConftestImportFailure.
- A node id below a skipped collector reports the skip instead of
  "found no collectors".
- --import-mode=importlib no longer leaves a module that raised while
  executing in sys.modules.

Closes #15142.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GHmGPdXhVMcboqUeE6q62M
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GHmGPdXhVMcboqUeE6q62M
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GHmGPdXhVMcboqUeE6q62M
Only pytest.importorskip() stays supported for skipping a directory from
its conftest.py: it now raises SkippedImport, a SkippedModule subclass,
and any other skip at conftest import warns with PytestRemovedIn10Warning.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GHmGPdXhVMcboqUeE6q62M

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

conftest.py: support importorskip at import, make every other outcome exception a usage error

1 participant