Skip to content

Fix zero agent for semantic action spaces - #7439

Open
StafaH wants to merge 4 commits into
isaac-sim:developfrom
StafaH:fix/zero-agent-actions
Open

Fix zero agent for semantic action spaces#7439
StafaH wants to merge 4 commits into
isaac-sim:developfrom
StafaH:fix/zero-agent-actions

Conversation

@StafaH

@StafaH StafaH commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes zero_agent.py for tasks where literal zero actions are invalid or where the action space is not a flat Box.

  • Adds a semantic neutral_actions contract to manager-based action terms.
  • Makes absolute differential IK, Pink IK, RMPFlow, and operational-space controllers hold their current end-effector pose.
  • Supports composite direct-workflow and direct multi-agent action spaces.
  • Validates task configurations before launching Kit or initializing native physics, so unsupported Newton/surface-gripper combinations fail cleanly.

No new dependencies are required.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Release backport

  • Backport this pull request to the active release branch after it merges into develop

Screenshots

Not applicable.

Validation

  • uv run pytest -q source/isaaclab/test/envs/test_neutral_actions.py source/isaaclab_rl/test/test_entrypoints.py — 20 passed in 2.33 seconds.
  • uv run isaaclab -f — passed, including changelog validation.
  • Ran the reported IsaacContrib-PickPlace-FixedBaseUpperBodyIK-G1-Abs zero-agent reproduction for two steps without IK NaNs.
  • Ran Isaac-Reach-Franka-OSC for two steps to validate absolute OSC neutral actions.
  • Verified IsaacContrib-Stack-Cube-UR10-Long-Suction-IK-Rel physics=newton_mjwarp exits before simulator launch with the PhysX-only surface-gripper guidance.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with uv run isaaclab -f
  • Documentation changes are not required; the user-visible behavior is covered by package changelog fragments
  • My changes generate no new warnings
  • I have added focused, lightweight tests that prove the fix is effective
  • I have added a changelog fragment for every touched package
  • My name already exists in CONTRIBUTORS.md

@StafaH
StafaH requested a review from a team August 30, 2026 21:15
@github-actions github-actions Bot added bug Something isn't working isaac-lab Related to Isaac Lab team labels Aug 30, 2026
@greptile-apps

greptile-apps Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR introduces semantic neutral actions for manager-based controllers and updates checkpoint-free zero-agent playback to support composite and multi-agent action spaces. It also validates resolved task configurations before simulator startup.

  • Adds neutral-action aggregation to the action manager.
  • Generates current-pose commands for absolute differential IK, Pink IK, RMPFlow, and operational-space control.
  • Tensorizes zero actions for direct composite and multi-agent Gymnasium spaces.
  • Adds focused tests and changelog fragments for the new behavior.

Confidence Score: 5/5

The PR appears safe to merge with no concrete blocking or independently actionable non-blocking issue identified.

The new neutral-action paths preserve the configured batched action structure, and the investigated controller commands round-trip through their current preprocessing for repository configurations.

Important Files Changed

Filename Overview
source/isaaclab/isaaclab/managers/action_manager.py Adds a zero-default neutral-action contract for terms and concatenates term-specific commands in manager order.
source/isaaclab/isaaclab/envs/mdp/actions/task_space_actions.py Adds current-pose neutral commands for absolute differential IK and operational-space controllers.
source/isaaclab/isaaclab/envs/mdp/actions/pink_task_space_actions.py Resolves Pink target bodies, validates frame-task counts, and emits current frame and hand state as neutral actions.
source/isaaclab/isaaclab/envs/mdp/actions/rmpflow_task_space_actions.py Adds current end-effector pose commands for absolute RMPFlow playback while retaining zeros for relative mode.
source/isaaclab_rl/isaaclab_rl/entrypoints/simple_agents.py Selects semantic manager actions or recursively tensorized direct-space zeros and validates configuration before simulator launch.
source/isaaclab/test/envs/test_neutral_actions.py Verifies Pink neutral actions preserve target ordering, environment-relative positions, valid quaternions, and hand state.
source/isaaclab_rl/test/test_entrypoints.py Covers manager semantic actions, composite and multi-agent direct spaces, and pre-launch validation failure.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Z[Zero-agent loop] --> K{Environment type}
  K -->|Manager-based| M[ActionManager.neutral_actions]
  M --> T[Concatenate term-specific neutral commands]
  T --> E[Environment step]
  K -->|Direct single-agent| S[Tensorize single action space with zero]
  S --> E
  K -->|Direct multi-agent| A[Tensorize each agent action space with zero]
  A --> E
Loading

Reviews (1): Last reviewed commit: "Make zero agent use neutral actions" | Re-trigger Greptile

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The additive neutral_actions contract and zero-agent routing are coherent, but Pink IK constructs its neutral pose using the asset’s quaternion convention rather than the action term’s documented quaternion layout. This prevents the Pink neutral command from reliably holding the current orientation.

  • Design and architecture: The base zero-valued default with controller-specific overrides provides a clean extension point, and _get_neutral_actions appropriately centralizes manager-based and direct-workflow behavior. The accepted concern is localized to Pink IK pose encoding rather than the overall design.
  • API: ActionTerm.neutral_actions and ActionManager.neutral_actions are additive properties with backward-compatible defaults. However, Pink IK documents action orientations as (x, y, z, w) while its override directly flattens body_link_pose_w, whose quaternion uses (w, x, y, z), violating the new semantic contract.
  • Implementation: The direct and multi-agent fallbacks, pre-launch validation, and other controller overrides follow the intended paths. Pink IK must reorder the current body quaternion into the action term’s expected layout before flattening; the added test currently checks only quaternion magnitude and mirrors the implementation, so it does not detect this convention mismatch.

Minor fixes needed. Posted 1 actionable finding inline.

Automated review; human maintainers own approval decisions.

@property
def neutral_actions(self) -> torch.Tensor:
"""Raw actions that hold the controlled frames and hand joints at their current state."""
frame_poses = self._asset.data.body_link_pose_w.torch[:, self._controlled_frame_ids].clone()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Warning · Api — Pink neutral pose uses wrong quaternion order

body_link_pose_w stores orientation as (w, x, y, z), but this term documents its pose slots as position plus orientation (x, y, z, w) (see orientation_dim). Concatenating the raw body pose therefore hands process_actions a reordered quaternion, so the "hold current frames" command requests a different orientation than the current one. Convert the quaternion to the action term's documented layout before flattening.

@StafaH
StafaH requested a review from hujc7 as a code owner August 31, 2026 21:57
@kellyguo11

Copy link
Copy Markdown
Contributor

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants