[Docs] Merge and clean up RL docs

[StafaH]

Description

Merge the reinforcement learning overview, framework comparison, existing-script reference, and training guide into one task-oriented page updated for the Isaac Lab 3.0 CLI and preset conventions.

The consolidated guide recommends RSL-RL as the default and explains the distinct reasons to choose RL-Games, SKRL, or Stable-Baselines3. It also adds a reproducible OVRTX-rendered ANYmal-D learning-progression GIF and a media-generation workflow modeled after the quickstart scripts.

No additional dependencies are required.

Type of change

• Documentation update

Release backport

• Backport this pull request to the active release branch after it merges into develop

Screenshots

The merged RL page includes the generated ANYmal-D learning-progression GIF, recorded with OVRTX at iterations 0, 100, and 299.

Validation

• uv run --isolated --extra test -- make -C docs current-docs (passed with no warnings)
• uv run --no-project python tools/skills/cli.py check (validated 21 skills)
• uvx ruff check tools/docs/media/capture_reinforcement_learning.py
• uvx ruff format --check tools/docs/media/capture_reinforcement_learning.py
• bash -n tools/docs/media/generate_reinforcement_learning.sh
• git diff --check
• uv run isaaclab -f (all applicable formatting, RST, codespell, executable, large-file, and LFS checks passed; the repository-wide changelog hook reports unrelated baseline fragment divergence in source packages not touched by this documentation-only PR)
• Ran tools/docs/media/generate_reinforcement_learning.sh end-to-end with OVRTX and inspected the generated GIF.

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with uv run isaaclab -f
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works (documentation-only change; validated with a warning-as-error Sphinx build)
• I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (no source packages touched)
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Summary

The PR consolidates the reinforcement-learning documentation around the Isaac Lab 3.0 CLI and adds a reproducible ANYmal-D learning-progression media workflow.

• Replaces three RL documentation pages with a task-oriented consolidated guide.
• Updates documentation and skill references to the consolidated page.
• Adds OVRTX playback configuration and a shell pipeline for training, recording, and assembling the progression GIF.

Confidence Score: 3/5

The PR should not merge until the media generator selects an emitted final checkpoint and the removed documentation URLs are preserved or redirected.

A fresh media-generation run stops when it looks for model_299.pt after a 300-iteration training run, and deleting the legacy source pages leaves existing repository links pointing to missing generated pages.

Files Needing Attention: tools/docs/media/generate_reinforcement_learning.sh; docs/source/overview/reinforcement-learning/rl_existing_scripts.rst; docs/source/overview/reinforcement-learning/rl_frameworks.rst

Important Files Changed

Filename	Overview
docs/source/overview/reinforcement-learning/index.rst	Consolidates library selection, training, monitoring, checkpoint, and troubleshooting guidance into one current-CLI page.
tools/docs/media/generate_reinforcement_learning.sh	Adds the end-to-end media pipeline, but its model_299.pt expectation does not match the configured run's final checkpoint.
tools/docs/media/capture_reinforcement_learning.py	Registers a fixed ANYmal-D capture configuration and preserves the custom visualizer-backed recorder through playback setup.
docs/source/overview/reinforcement-learning/rl_existing_scripts.rst	Removes a published page that still has inbound repository links, without a redirect stub.
docs/source/overview/reinforcement-learning/rl_frameworks.rst	Removes a published page still referenced by maintained source and release notes, without preserving its URL.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Train[Train ANYmal-D for 300 iterations] --> Saves[RSL-RL saves checkpoints]
  Saves --> Select[Select iterations 0, 100, and final]
  Select --> Play[Play each checkpoint with OVRTX]
  Play --> Clips[Write three MP4 clips]
  Clips --> FFmpeg[Stack and encode clips]
  FFmpeg --> GIF[Documentation GIF]

Loading

Comments Outside Diff (1)

1. docs/source/overview/reinforcement-learning/rl_existing_scripts.rst

   Deleted pages break inbound links

   Deleting the legacy RL pages without compatibility stubs leaves unchanged links in README.md, maintained agent configurations, and release notes pointing to missing generated pages instead of the consolidated guide.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "Merge and clean up RL documentation" | Re-trigger Greptile}

[isaaclab-review-bot]

Isaac Lab Review Bot

The RL documentation consolidation is coherent and references to the removed pages are retargeted, but two actionable issues remain: the media generator uses a physics selector inconsistent with its installed backend extra, and the merged source-of-truth page drops the maintenance link to its synchronized RL skills.

• Design and architecture: The task-oriented consolidated page is a suitable replacement for the deleted RL overview, comparison, workflow, and training-guide pages. However, it should preserve the former training guide’s explicit instruction to synchronize the train-rl-agents and debug-rl-training skills, since those skills now identify this merged page as their maintained documentation source.
• API: The supplied documentation targets and inbound references are consistently migrated: _rl-frameworks and _pretrained-checkpoints remain available, _choose-an-rl-library supports the new quickstart reference, and references in the changed documentation and skills point to the consolidated page.
• Implementation: The capture helper and generator follow the existing documentation-media workflow, but the generator installs the ovphysx extra while passing physics=physx during both training and playback. Those commands should use the documented physics=ovphysx selector to match the selected integration and make the workflow reproducible.

Minor fixes needed. Posted 2 actionable findings inline.

Automated review; human maintainers own approval decisions.

[StafaH]

Applied the review follow-up in 8f9411331:

• restored the RL skill synchronization note
• added orphaned compatibility stubs for all three legacy RL page URLs
• updated the README to link directly to the consolidated guide
• documented the verified zero-based RSL-RL checkpoint numbering and automatic PhysX selector behavior

Validation: bash -n passed, all 21 repository skills validated, and the full current-docs Sphinx build completed successfully without warnings.

[StafaH]

run-ci

[kellyguo11]

we try to keep gif files uploaded on S3 instead so that we can avoid bloating the repo size. generally, we only keep .jpg media files in the repo.

[StafaH]

Done

[kellyguo11]

maybe we don't use kit here since it requires isaac sim?

[StafaH]

Done

[kellyguo11]

what is the camera/ovrtx setup used for this? the environment doesn't come with a camera by default right?

[StafaH]

Fixed, this section was reworked so it no longer discuesses ovrtx

[kellyguo11]

maybe also a different visualizer here?

[StafaH]

Fixed (ended up removing this part of the docs)