Skip to content

Test using pytest-run-parallel and related fixups in the tests#2194

Open
seberg wants to merge 16 commits into
NVIDIA:mainfrom
seberg:ft-testing
Open

Test using pytest-run-parallel and related fixups in the tests#2194
seberg wants to merge 16 commits into
NVIDIA:mainfrom
seberg:ft-testing

Conversation

@seberg

@seberg seberg commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Description

This is the full follow up to gh-2162 for a full picture. I tried to split commits up roughly and could split them into individual PRs as well.

I am planning to have another look through myself once (see if I can think of a nicer pattern than the current mini plugins).

The buffer closing/sync is an upstream issue I think that I have opened a bug for.

And another reason to split things up and get started: rebased and of course there are new issues :).

Checklist

  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@copy-pr-bot

copy-pr-bot Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added cuda.bindings Everything related to the cuda.bindings module cuda.core Everything related to the cuda.core module cuda.pathfinder Everything related to the cuda.pathfinder module labels Jun 10, 2026
@seberg seberg changed the title Ft testing Test using pytest-run-parallel and related fixups in the tests Jun 10, 2026
@leofang leofang self-requested a review June 11, 2026 02:55
@seberg

seberg commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test ed40f60

@github-actions

Copy link
Copy Markdown

@github-actions github-actions Bot added the CI/CD CI/CD infrastructure label Jun 11, 2026
@seberg

seberg commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

Fun, the refactor made the cufile xfail-strict tests pass on CI, but I didn't set up the parallel run correctly... one more try:

/ok to test eb6a2ff

@seberg

seberg commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test eb6a2ff

@seberg

seberg commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 7b59bff

@copy-pr-bot

copy-pr-bot Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

seberg added 7 commits July 8, 2026 13:44
… 3.15t

Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
- thread_unsafe: nvml init ref-count, graphMem attr, mock-based tests,
  OpenGL, peer-access pool state, multiprocessing warning, program-cache
  race reproduction, and functools.cache mutation tests
- parallel_threads_limit: IPC / worker-pool tests that spawn subprocesses
  or open file descriptors (limit 4), example tests (limit 8), and the
  event-registration test whose timeouts are slow

Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
…unsafe always

Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
…empool

Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
seberg added 4 commits July 8, 2026 13:44
After my first AI try was a crazy mess, the second run actually found
a neat solution...
These objects can be created in the main thread, but we can't create
them on the fly in many threads as it was...

Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
For some reason the latch kernel helper test started failing now
(it did not before my update from CUDA 13.2 to 13.3?).

The reason isn't that it is not thread-safe, but that something
(presumably module loading/unloading) causes synchronizations which
in turn cause threads having to wait on their LatchKernel to finish.

And of course the test itself really needs that not to happen.
Making sure there is only one LatchKernel compiled and loaded exactly
once seems to avoid this problem.

Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
@seberg seberg marked this pull request as ready for review July 8, 2026 11:53
@seberg seberg added the P1 Medium priority - Should do label Jul 8, 2026
@seberg

seberg commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Still quite a few changes in this now (and let's see if CI passes) including gh-2321. But maybe it's still fine to review.

The most annoying one is probably the test_object_protocols.py changes. IIRC the problem was that the design with the .getfixturevalue() deferred things into threads, which might even have been fine with initialized.
But none of the tests actually test things deep enough to require cuda set up in the test itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module cuda.core Everything related to the cuda.core module cuda.pathfinder Everything related to the cuda.pathfinder module P1 Medium priority - Should do

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant