Skip to content

fix(roles): ask the driver before writing a vGPU profile - #76

Open
Aleksei Sviridkin (lexfrei) wants to merge 1 commit into
mainfrom
fix/nvidia-vgpu-profile-stage-decline
Open

Aleksei Sviridkin (lexfrei) wants to merge 1 commit into
mainfrom
fix/nvidia-vgpu-profile-stage-decline

Conversation

@lexfrei

@lexfrei Aleksei Sviridkin (lexfrei) commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

The boot unit wrote current_vgpu_type on cards whose driver does not report Host VGPU Mode: SR-IOV. A card that reports Non SR-IOV runs vGPU through the legacy mdev path, and a write to one of its functions can disturb the vGPUs that path already placed there. Only the virtual-function stage asked the driver for that mode. The profile stage did not, so it wrote to a card the function stage had refused, and to a card declared with a profile alone, which that stage never sees at all.

Changes

  • Both stages ask the driver for the card's mode and leave the card alone unless it reports SR-IOV. They call one helper, so the check and the line it logs exist once, and the flag that carried the function stage's decision into the profile stage is gone.
  • The stages test covers both paths: a card declared with sriov and a profile, and a card declared with a profile and no sriov. Each keeps the functions it exposes, and each asserts the profile node is unchanged after the run.
  • README says the skip covers such a card whatever was declared for it.

Test plan

  • ansible-lint passes
  • ansible-test sanity passes
  • Tested on a live cluster (describe environment)
  • Idempotency verified (second run: changed=0)

No vGPU hardware here, so the stages ran against the fake nvidia-smi, sriov-manage and PCI tree under tests/, on Ubuntu 24.04 with systemd.

Summary by CodeRabbit

  • Bug Fixes

    • NVIDIA vGPU restore now safely skips virtual-function and profile configuration when the driver reports a non-SR-IOV mode.
    • Skipped cards no longer cause the restore operation to fail.
    • Unsupported profile-only configurations are handled as successful skips with clear log messages.
  • Documentation

    • Updated NVIDIA vGPU documentation to describe non-SR-IOV handling and skip behavior.
  • Tests

    • Expanded coverage for declined cards, profile-only configurations, and multi-GPU restore scenarios.

A card whose driver does not report Host VGPU Mode SR-IOV runs vGPU
through the legacy mdev path, and writing current_vgpu_type on one of
its functions can disturb the vGPUs that path already placed there. Only
the virtual-function stage asked the driver for that mode, so a card
reached through a profile declaration alone, or one the function stage
had already refused, had its profile written anyway.

Both stages now ask for themselves and leave the card alone unless it
reports SR-IOV.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
@coderabbitai

coderabbitai Bot commented Sep 18, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 7a523d3f-4d75-4905-9bce-8f5376ed3765

📥 Commits

Reviewing files that changed from the base of the PR and between 89ef8dd and ccbdcd8.

📒 Files selected for processing (3)
  • README.md
  • roles/nvidia_vgpu_host/templates/cozystack-nvidia-vgpu-restore.sh.j2
  • tests/test-nvidia-vgpu-host-stages.yml

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The restore script now uses one helper to check host VGPU mode before VF and profile stages. Non-SR-IOV cards are logged and skipped successfully. Tests cover declined cards and profile-only declarations, and the README documents the behavior.

Changes

SR-IOV mode handling

Layer / File(s) Summary
Shared SR-IOV refusal flow
roles/nvidia_vgpu_host/templates/cozystack-nvidia-vgpu-restore.sh.j2
The script adds refuse_unless_sriov and uses it in both VF and profile stages. Declined cards are no longer tracked between stages.
Skip behavior validation and documentation
tests/test-nvidia-vgpu-host-stages.yml, README.md
Tests verify successful skips, unchanged profile state, mode logging, and profile-only declarations. The README documents the non-SR-IOV path.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant VFProfileStage
  participant refuse_unless_sriov
  participant NVIDIADriver
  VFProfileStage->>refuse_unless_sriov: validate declared card mode
  refuse_unless_sriov->>NVIDIADriver: query host_vgpu_mode
  NVIDIADriver-->>refuse_unless_sriov: return observed mode
  refuse_unless_sriov-->>VFProfileStage: allow SR-IOV or decline stage
Loading

Merge Risk: ⚪ Minimal · up to ccbdc

Non-SR-IOV cards are skipped without changing virtual functions or profiles, while supported cards continue through the existing stages. No actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: checking the driver before writing a vGPU profile. It is concise and specific.
Description check ✅ Passed The description includes all required sections, explains the change, lists the main updates, and documents the test results and environment. It clearly states that live-cluster testing was not perform…
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lexfrei
Aleksei Sviridkin (lexfrei) marked this pull request as ready for review September 18, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant