Skip to content

Add the OV renderer lease and teleop-owned OVXR adapter - #8787

Open
hougantc-nvda wants to merge 2 commits into
isaac-sim:mainfrom
hougantc-nvda:hougantc/kitless-borrowed-renderer
Open

hougantc-nvda wants to merge 2 commits into
isaac-sim:mainfrom
hougantc-nvda:hougantc/kitless-borrowed-renderer

Conversation

@hougantc-nvda

@hougantc-nvda hougantc-nvda commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Description

Review stack · 1/5: #8787 → #8788 → #8789 → #8790 → #8791

Review only this layer. These PRs target main; the comparison excludes predecessors.

Connect kitless Isaac Lab teleoperation to Isaac Capture's OVXR session using Lab's existing OVRTX renderer and OVStage. Capture borrows the resources through an explicit lease; Lab keeps their ownership and lifetime.

  • isaaclab_ov owns the renderer, stage, scene publication and retained write buffers. The lease checks render-thread ownership and rejects nested operations; the application publication guard supplies cross-thread exclusion.
  • isaaclab_teleop implements the OVXR host adapter, bootstrap, scene/anchor publication and coordinated shutdown.
  • Isaac Capture owns OVXR presentation, the render worker and borrowed OpenXR handles for input.

Shutdown retires input, XR and the lease before destroying Lab resources. Failed retirement retains resources for their owning thread. Ordinary rendering remains available without a lease. Publication buffers stay separate from rendering because they own pending-write storage and stream lifetime.

Dynamic anchor smoothing uses elapsed XR-frame time, with synchronous reset seeding. Scale and bootstrap bounds reject invalid input before allocation. Public signatures and configuration fields document frames, units and ownership.

Export behavior: stage export preserves an explicitly authored showAllPartitionsByDefault=False, including ordinary exports. Missing opinions still default to True; callers requiring all partitions visible should explicitly author True.

See adapter ownership and lifecycle.

Installation: use published isaaccapture==1.7.93a1 from NVIDIA’s Python index with the scoped OVXR installation. The guide creates the optional ovxr selection; project-wide Kit/provider defaults are unchanged.

Type of change

  • New feature
  • Bug fix
  • Documentation update

Testing

The combined stack passed 877 tests across 33 affected test files, with no failures or skips. This includes the native Newton graph-capture/replay/reset tests, renderer/lease contracts, recorded-control gating, camera cleanup failures, registered task selection and CPU collision checks, plus the inherited upstream noise-reset cases. Formatting, all five changelog boundaries and the warning-as-error documentation build pass. These are cumulative integration results; individual PR boundaries received static/API and changelog checks. Runtime and performance scope is detailed in #8791.

Remaining qualification: native Kit execution and MCAP-file playback were not rerun; control/replay owner tests substitute external I/O. Native consumer-destruction failure is not established by normal shutdown. Cold renderer attachment, intermittent GR1T2 first-Play recovery and SO101 cube retention remain open. Existing USD, render-product GPU-assignment and native runtime warnings are retained; this is not a warning-free qualification.

Checklist

  • Contribution guidelines read
  • Changed-file formatting/lint and Python 3.12 changelog checks pass
  • Public documentation and release fragments updated
  • Affected behavior and cumulative IWER regression checks complete within the scope above
  • Native Kit and physical-headset qualification at this revision
  • No new warnings

@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Oct 23, 2026
@hougantc-nvda
hougantc-nvda force-pushed the hougantc/kitless-borrowed-renderer branch from 9e86c66 to f63b4c2 Compare October 26, 2026 17:12
@hougantc-nvda hougantc-nvda changed the title Enable OVXR to use Isaac Lab's renderer and scene Expose the OV renderer and scene through a borrowed-renderer lease Oct 26, 2026
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 26, 2026
@hougantc-nvda hougantc-nvda changed the title Expose the OV renderer and scene through a borrowed-renderer lease Add the OV renderer lease and teleop-owned OVXR adapter Oct 26, 2026
@hougantc-nvda
hougantc-nvda force-pushed the hougantc/kitless-borrowed-renderer branch 3 times, most recently from d6d62fe to 5acf5b9 Compare October 29, 2026 16:05
@hougantc-nvda

Copy link
Copy Markdown
Contributor Author

@greptile Please review the latest changes in this PR.

@greptile-apps

greptile-apps Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The PR affects OVXR teleoperation and the OVRTX renderer paths it borrows.

Update the renderer double in the failed-seal tests before merging.

Findings

  1. P1 Failed-seal tests now raise ▶

Summary

Adds an explicit renderer lease and a teleop-owned OVXR adapter.

  • Applications can borrow the renderer and stage without taking ownership of them.
  • Isaac Lab connects teleoperation to Isaac Capture’s OVXR session and render worker.
  • CPU-backed renderer data can now use host paths without a CUDA stream.
  • Material-only changes now trigger scene publication and wait for native writes to finish.
  • USD export keeps an authored visibility setting instead of replacing it.

Diagram

sequenceDiagram
    participant Lab
    participant Lease
    participant Adapter
    participant Capture
    Lab->>Lease: Reserve renderer and stage
    Adapter->>Capture: Start render worker
    Capture->>Lease: Claim render ownership
    Lab->>Lease: Write scene and dirty materials
    Lease-->>Adapter: Complete native writes
    Adapter->>Capture: Supply sealed scene ordinal
    Capture->>Lease: Retire borrowed access
    Lab->>Lab: Destroy owned resources
Loading

Reviews (4) · Last reviewed commit: "Publish dirty visual materials in leased..." · Reviewed by Greptile

@hougantc-nvda

Copy link
Copy Markdown
Contributor Author

@greptile Please review the current head (1ebf0ab) again. Recheck the fixes and remaining issues against the published code, including the actual interfaces and callers at this PR boundary.

@hougantc-nvda
hougantc-nvda force-pushed the hougantc/kitless-borrowed-renderer branch from 1ebf0ab to 91d4de5 Compare October 30, 2026 01:00
@hougantc-nvda

Copy link
Copy Markdown
Contributor Author

@greptile please review this PR again at the latest head (91d4de5610eee269ac922d78f9d8494dc95674f7), including the changes since your previous review.

@hougantc-nvda
hougantc-nvda marked this pull request as ready for review October 30, 2026 01:48
@hougantc-nvda
hougantc-nvda requested a review from a team October 30, 2026 01:48

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

Needs attention: Design and architecture (2), Scope and over-engineering (2), Implementation (2), Style consistency (3), Test quality (4). Other reviewed categories look good.

  • PR description: Update the body to include CPU publication and authored partition preservation, narrow the scene-write claim, and either validate a83a3e3 or label earlier results; trim stacked-PR dependency prose; no benchmark is needed without a performance claim.
  • Design and architecture: Lease acquisition depends on the unset ISAAC_LAB_OVRTX_USE_OVSTAGE global, so the documented session fails by default; expose private-OVStage selection in configuration and export the lease directly from render_lease.
  • Scope and over-engineering: Minimal implementation: keep render_lease.py for lease lifecycle, publication_buffers.py for native-write retention, ovxr_session.py for SimulationContext bootstrap, and ov_libraries_xr.py for the Capture adapter; delete _lane_lease and inline _import_ovxr. Scope is cohesive; no split recommended.
  • API: API changes look good.
  • Compatibility and deprecation: No breaking changes identified.
  • Implementation: log_mesh still mutates renderer scene state during an active lease, violating exclusivity; apply reject_ordinary_operation before creating or updating runtime meshes.
  • Style consistency: stage.py still claims perspective cameras see every partition although authored false settings now win; update the comment to describe the preserved-opinion behavior.
  • Test quality: Added tests include self-derived, assertion-free, and branch-identical cases that cannot catch distinct regressions; remove those probes and keep explicit observable contract assertions.

Significant concerns. Posted 13 inline findings.

Automated review; human maintainers own approval decisions.

@hougantc-nvda

Copy link
Copy Markdown
Contributor Author

@isaaclab-review-bot review

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

Needs attention: Design and architecture (1), Scope and over-engineering (1), Compatibility and deprecation (1), Implementation (1), Style consistency (3), Test quality (1). Other reviewed categories look good.

  • PR description: The body separates boundary and cumulative validation, but its streaming claim uses a later stack revision and omits baseline revision, command, and hardware; add these and list CPU-device and export changes.
  • Design and architecture: The owner lane excludes application-thread scene publication, so it does not serialize all OVStage writers; use one shared lock or narrow the stated contract.
  • Scope and over-engineering: Minimal implementation: keep render_lease.py, ov_libraries_xr.py, and ovxr_session.py; merge publication_buffers.py back into ovrtx_renderer.py, preserving lease acquisition, scene publication, OVXR delegation, and cleanup. Scope is cohesive; no split recommended.
  • API: API changes look good.
  • Compatibility and deprecation: Breaking changes: export_stage_to_string now preserves authored false partition visibility without a deprecation bridge; document the behavior and migration or restrict it to the lease path.
  • Implementation: OvLibrariesXrConfig accepts NaN meters_per_unit and passes it to the native session; require math.isfinite before the positive-value check.
  • Style consistency: The exported OvLibrariesXrConfig diverges from adjacent public configuration conventions; rename it to OvLibrariesXrCfg and implement it with configclass.
  • Test quality: The OVXR test fakes expose several unused methods and fields that preserve no tested contract; delete that dead scaffolding.

Significant concerns. Posted 8 inline findings.

Automated review; human maintainers own approval decisions.

Comment thread source/isaaclab_ov/isaaclab_ov/stage.py Resolved
@hougantc-nvda
hougantc-nvda force-pushed the hougantc/kitless-borrowed-renderer branch from cf2477a to 3795b51 Compare October 30, 2026 22:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant