Repository navigation
pytest.importorskip(..., defer=True): gate on a missing dependency but still collect every test #15267
Description
🤖 Written by Claude Opus 5.5 via Claude Code for the pytest maintainers; I prompted it, it did the work, I read it.
Problem
pytest.importorskip("tables") at module level stops collection of the whole module, and in a conftest.py (#15142, #15143) of the whole directory. The tests behind the gate never show up: no node ids, no parametrization, nothing in --collect-only or junit, one skip line per module instead of per test.
Projects that want full collection build it by hand. A scan of 177 popular repos:
| pattern | repos |
|---|---|
shared requires_x = pytest.mark.skipif(not has_x, ...) (e.g. xarray _importorskip → (has_x, requires_x), ~1000 uses) |
81 |
skipif(not HAS_X) with a try/except import flag |
57 |
pytestmark = pytest.mark.skipif(...) |
41 |
custom marker + pytest_collection_modifyitems / pytest_runtest_setup (gdal require_driver, keras, salt) |
~12 |
per-param gates (pandas td.skip_if_no) |
13 |
None of them binds the module for use in fixtures, which is why x = pytest.importorskip("x") stays the most common gate: module-level importorskip in 939 test modules across 76 repos, and in 10 conftests across 6 repos.
Hard limit
pytest learns tests by executing the module. A module that uses the dependency at import time (import X, from X import Y, @parametrize("f", X.FILTERS), X.__version__ in a decorator) cannot be collected without X, whatever the API. Any solution only helps code that uses the dependency lazily, and must fall back to today's module skip for the rest.
Classifying the 939 gated test modules by what follows the gate:
| after the gate | modules |
|---|---|
| bound name used only inside functions | 339 (36%) |
| bare gate, nothing bound | 238 (25%) |
| bound name used at module scope | 167 (18%) |
hard import X / from X import Y |
195 (21%) |
So roughly 40-55% of existing gates would collect fully with no edits (some of the first two groups pull X in transitively through later imports).
Proposal
Opt-in defer=True on pytest.importorskip:
tables = pytest.importorskip("tables", defer=True)- X present: returns the real module; nothing changes.
- X missing, called while pytest imports a test module or conftest: record the gate on that node (the
Module, or the conftest's directory) and return a stand-in object. Import continues, every test in scope is collected, and each is skipped at setup with the original reason and the gate's location. - Fallback: if the import then fails for any reason (the stand-in used at import time,
from X import Y, project code importing X), raise the normal module-level skip. That is exactly today's result, because today nothing after the gate runs. - Called inside a test or fixture: unchanged.
Possible companion: a pytest.mark.requires_import("x", minversion=...) marker evaluated with the skip marks, replacing the hand-rolled requires_x / skip_if_no helpers for per-test and per-param cases.
Implementation sketch
- A context variable for "node currently being imported", set around
importtestmoduleand conftest import;importorskip(defer=True)checks it. Without it,defer=Truebehaves like today. - Gates stored in the node stash;
pytest_runtest_setupskips an item when any node initem.listchain()has a gate. - The stand-in must raise
AttributeErrorfor private/dunder names (pytest's own introspection:__test__,pytestmark, fixture markers) and raise the module skip for any other attribute access during import, soX.__version__at module scope degrades to today's behaviour instead of a collection error. -rsshould fold item skips by gate so the summary stays one line per gate.
Costs
- Skip counts go from 1 per module to 1 per test (junit, dashboards);
-rsneeds folding. - The rest of the module body runs when X is missing (side effects, import time). Import errors after a failed gate are absorbed as today, but side effects are new.
- The stand-in is not
Noneand not aModuleType;if tables is Nonestyle checks will mislead. - Only about half of existing gates benefit without edits; the rest need rewrites (
from X import Y→X.Yin test bodies, module-scope version checks → stringskipifconditions) or stay module skips.
Prototype
A ~110-line plugin implementing the above (frame-walking instead of a context variable) collects all tests of lazily-gated modules and of a conftest-gated directory with a fixture using the module, skips each at setup, falls back to module skips for eager use and hard imports, and still reports a real ImportError without a gate as a collection error.
Activity