Skip to content

pytest.importorskip(..., defer=True): gate on a missing dependency but still collect every test #15267

Open

Description

@RonnyPfannschmidt

🤖 Written by Claude Opus 5.5 via Claude Code for the pytest maintainers; I prompted it, it did the work, I read it.

Problem

pytest.importorskip("tables") at module level stops collection of the whole module, and in a conftest.py (#15142, #15143) of the whole directory. The tests behind the gate never show up: no node ids, no parametrization, nothing in --collect-only or junit, one skip line per module instead of per test.

Projects that want full collection build it by hand. A scan of 177 popular repos:

pattern repos
shared requires_x = pytest.mark.skipif(not has_x, ...) (e.g. xarray _importorskip → (has_x, requires_x), ~1000 uses) 81
skipif(not HAS_X) with a try/except import flag 57
pytestmark = pytest.mark.skipif(...) 41
custom marker + pytest_collection_modifyitems / pytest_runtest_setup (gdal require_driver, keras, salt) ~12
per-param gates (pandas td.skip_if_no) 13

None of them binds the module for use in fixtures, which is why x = pytest.importorskip("x") stays the most common gate: module-level importorskip in 939 test modules across 76 repos, and in 10 conftests across 6 repos.

Hard limit

pytest learns tests by executing the module. A module that uses the dependency at import time (import X, from X import Y, @parametrize("f", X.FILTERS), X.__version__ in a decorator) cannot be collected without X, whatever the API. Any solution only helps code that uses the dependency lazily, and must fall back to today's module skip for the rest.

Classifying the 939 gated test modules by what follows the gate:

after the gate modules
bound name used only inside functions 339 (36%)
bare gate, nothing bound 238 (25%)
bound name used at module scope 167 (18%)
hard import X / from X import Y 195 (21%)

So roughly 40-55% of existing gates would collect fully with no edits (some of the first two groups pull X in transitively through later imports).

Proposal

Opt-in defer=True on pytest.importorskip:

tables = pytest.importorskip("tables", defer=True)
  • X present: returns the real module; nothing changes.
  • X missing, called while pytest imports a test module or conftest: record the gate on that node (the Module, or the conftest's directory) and return a stand-in object. Import continues, every test in scope is collected, and each is skipped at setup with the original reason and the gate's location.
  • Fallback: if the import then fails for any reason (the stand-in used at import time, from X import Y, project code importing X), raise the normal module-level skip. That is exactly today's result, because today nothing after the gate runs.
  • Called inside a test or fixture: unchanged.

Possible companion: a pytest.mark.requires_import("x", minversion=...) marker evaluated with the skip marks, replacing the hand-rolled requires_x / skip_if_no helpers for per-test and per-param cases.

Implementation sketch

  • A context variable for "node currently being imported", set around importtestmodule and conftest import; importorskip(defer=True) checks it. Without it, defer=True behaves like today.
  • Gates stored in the node stash; pytest_runtest_setup skips an item when any node in item.listchain() has a gate.
  • The stand-in must raise AttributeError for private/dunder names (pytest's own introspection: __test__, pytestmark, fixture markers) and raise the module skip for any other attribute access during import, so X.__version__ at module scope degrades to today's behaviour instead of a collection error.
  • -rs should fold item skips by gate so the summary stays one line per gate.

Costs

  1. Skip counts go from 1 per module to 1 per test (junit, dashboards); -rs needs folding.
  2. The rest of the module body runs when X is missing (side effects, import time). Import errors after a failed gate are absorbed as today, but side effects are new.
  3. The stand-in is not None and not a ModuleType; if tables is None style checks will mislead.
  4. Only about half of existing gates benefit without edits; the rest need rewrites (from X import Y → X.Y in test bodies, module-scope version checks → string skipif conditions) or stay module skips.

Prototype

A ~110-line plugin implementing the above (frame-walking instead of a context variable) collects all tests of lazily-gated modules and of a conftest-gated directory with a fixture using the module, skips each at setup, falls back to module skips for eager use and hard imports, and still reports a real ImportError without a gate as a collection error.

Related: #15142, #15143, #4479.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions