Skip to content

fix(benchmarks): skip benchmark tests without pytest-codspeed - #63

Merged
cbueth merged 1 commit into
mainfrom
fix/benchmark-conftest-codspeed-optional
Aug 17, 2026
Merged

fix(benchmarks): skip benchmark tests without pytest-codspeed#63
cbueth merged 1 commit into
mainfrom
fix/benchmark-conftest-codspeed-optional

Conversation

@cbueth

@cbueth cbueth commented Aug 15, 2026

Copy link
Copy Markdown
Owner

This PR fixes the benchmark suite failing when pytest-codspeed is not installed, e.g. when conda-forge runs pytest tests/ against the sdist.

tests/benchmarks/ was added in 0.3.3. Its tests request the benchmark fixture (provided only by pytest-codspeed) and use @pytest.mark.benchmark, a marker the plugin registers. The old skip logic in tests/benchmarks/conftest.py only ran when the plugin was importable and silently did nothing otherwise. Without the plugin, pytest tests/ aborted with PytestUnknownMarkWarning for the unregistered benchmark mark (escalated to an error by filterwarnings = ["error"]) and fixture 'benchmark' not found.

register benchmark / codspeed_benchmark markers in pytest_configure when not already present.
identify benchmark tests by the benchmark fixture in item.fixturenames (same check the plugin uses) instead of importing the plugin, so skipping works with or without pytest-codspeed

@cbueth cbueth self-assigned this Aug 15, 2026
@cbueth cbueth added the dependencies Pull requests that update a dependency file label Aug 15, 2026
@codecov

codecov Bot commented Aug 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (da07a73) to head (fd93231).

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #63   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           29        29           
  Lines          866       866           
  Branches       150       150           
=========================================
  Hits           866       866           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@codspeed-hq

codspeed-hq Bot commented Aug 15, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 28 untouched benchmarks


Comparing fix/benchmark-conftest-codspeed-optional (fd93231) with main (3eee1cc)1

Open in CodSpeed

Footnotes

  1. No successful run was found on main (da07a73) during the generation of this report, so 3eee1cc was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@cbueth
cbueth merged commit a127b18 into main Aug 17, 2026
13 checks passed
@cbueth
cbueth deleted the fix/benchmark-conftest-codspeed-optional branch August 17, 2026 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant