Skip to content

Articulation atom action axis auto generation. #573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
matafela wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Articulation atom action axis auto generation. #573

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
afc7e38
update
Sep 1, 2026
f963be6
fix pour tutorial
Sep 1, 2026
1df12a9
update
Sep 2, 2026
5dc39b3
update twist axis_origin
Sep 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 39 additions & 11 deletions docs/source/overview/sim/atomic_actions/builtin_actions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -677,10 +677,13 @@ migration.
## `Press`

Plans **close hand -> approach target -> contact -> press along axis -> return
to the approach pose**. `PressAffordance` is entity-free and stores an explicit
target-local surface `press_position` and `press_axis`. `PressGoal.target_pose`
is either a pose snapshot or `SceneEntityPose`, which resolves through the
current `PlanningContext.scene` and participates in dynamic-goal recovery.
to the approach pose**. For an articulation link,
`Articulation.sample_initial_point_clouds()` stores target-link-local target and
whole-articulation clouds in `ObjectSemantics.geometry`. `PressAffordance`
derives `press_axis` from that geometry and can derive the outer-surface
`press_position` when it is omitted. `PressGoal.target_pose` is either a pose
snapshot or `SceneEntityPose`, which resolves through the current
`PlanningContext.scene` and participates in dynamic-goal recovery.

The contact, press, and retract segments use axis-aligned Cartesian keyframes;
each output sample is grounded with IK instead of being interpolated only in
Expand All @@ -698,7 +701,7 @@ right-handed orthonormal rotation even for vertical or oblique press axes.

`PressOptions` controls hand-close interpolation, approach distance,
press distance, and an optional target-local `press_position`. An options-level
position overrides the affordance's explicit surface point. The bound
position overrides the affordance's resolved surface point. The bound
`primary.grasp` endpoint must provide `grasp`; both endpoints come from the
generic `ActionBinding`, and the action keeps the gripper closed for all arm
motion segments. Applications that require force/contact confirmation must
Expand All @@ -711,13 +714,25 @@ verify it externally.
## `Slide`

Plans a grasped linear interaction for one articulation link. The entity-free
`SlideAffordance` stores the link-local grasp mesh, `translation_axis`, and
`SlideAffordance` stores the link-local grasp mesh and resolves its
`translation_axis` from initial articulation point-cloud geometry, plus
optional joint name/limits. `SlideGoal.target_pose` supplies the link pose as a
snapshot or `SceneEntityPose`. The positive axis direction means approach and
push/close; pull/open uses its negative direction. The affordance inherits
`AntipodalAffordance` and selects a grasp with `get_best_grasp_poses()`. The grasp
approach direction is the link-frame translation axis transformed by the
current link rotation.
`AntipodalAffordance` and selects a grasp with `get_best_grasp_poses()`. The
grasp approach direction is the resolved link-frame translation axis
transformed by the current link rotation.

Axis inference samples the target link and the merged articulation surface at
`ArticulationCfg.init_qpos`, expressed in the target link's initial local frame.
Both clouds use Open3D uniform surface sampling. Sampling the whole articulation
as one merged mesh preserves triangle-area weighting instead of giving every
link an equal point budget, which would over-represent tiny decorative links.
The target cloud center and twice its distribution radius define a spherical
neighborhood in the full cloud. The largest signed component of the
neighborhood-center offset selects exactly one of `+X`, `-X`, `+Y`, `-Y`, `+Z`,
or `-Z`. A complete point-cloud geometry snapshot overrides any legacy explicit
axis fallback.

With `direction="pull"`, the sequence is **approach -> reach -> close -> pull ->
open**. With `direction="push"`, it is **approach -> reach -> close -> push -> open
Expand Down Expand Up @@ -806,8 +821,21 @@ normalized against the resolved hinge limits and passed as the goal's

Plans **approach -> reach -> close -> twist -> open -> retract** for an
articulation link or a rigid object. The entity-free `TwistAffordance` stores an
explicit local `grasp_position`, `twist_axis`, and `axis_origin`, plus optional
joint name/limits. `TwistGoal.target_pose` supplies the grounded target pose.
explicit local `grasp_position`, plus optional joint name/limits. For an
articulation link, the same initial target-neighborhood geometry used by
`Slide` and `Press` resolves `twist_axis`. The independent
`target_link_revolute_axis_origin` geometry entry sets `axis_origin` to the
nearest parent revolute joint's initial origin, expressed in the target link's
initial local frame. Resolution walks through fixed parent joints. The sampled
target-link centroid is only a neighborhood/contact reference; it is never used
as the rotation origin.

Complete point-cloud geometry overrides the legacy `twist_axis` fallback but
does not overwrite an explicit `axis_origin` when revolute-joint metadata is
absent. If neither source supplies an origin, planning reports the missing
rotation-axis point. A rigid object without articulation context may still
provide explicit compatibility values. `TwistGoal.target_pose` supplies the
grounded target pose.

The grasp frame's z-axis follows the world-transformed twist axis; an adaptive
reference completes a right-handed orthonormal frame. Twist keyframes rotate
Expand Down
40 changes: 40 additions & 0 deletions docs/source/overview/sim/sim_articulation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ State data is accessed via getter methods that return batched tensors (`N` envir
| :--- | :--- | :--- |
| `get_local_pose(to_matrix=False)` | `(N, 7)` or `(N, 4, 4)` | Root link pose `[x, y, z, qw, qx, qy, qz]` or a 4x4 matrix. |
| `get_link_pose(link_name, to_matrix=False)` | `(N, 7)` or `(N, 4, 4)` | Specific link pose `[x, y, z, qw, qx, qy, qz]` or a 4x4 matrix. |
| `sample_initial_point_clouds(target_link_name)` | `Dict[str, Tensor]` | Uniformly sample the target link and merged articulation surface at `init_qpos` in the target link's initial local frame, and include the nearest parent revolute-joint origin when available. |
| `get_qpos(target=False)` | `(N, dof)` | Current joint positions (or joint targets if `target=True`). |
| `get_qvel(target=False)` | `(N, dof)` | Current joint velocities (or velocity targets if `target=True`). |
| `get_joint_drive()` | `Tuple[Tensor, ...]` | Returns `(stiffness, damping, max_effort, max_velocity, friction, armature)`, each shaped `(N, dof)`. |
Expand All @@ -193,6 +194,45 @@ print(f"Current Joint Positions: {articulation.get_qpos()}")
print(f"End Effector Pose: {articulation.get_link_pose('ee_link')}")
```

### Initial Link-Local Point Clouds

`sample_initial_point_clouds()` uses the configured initial joint positions and
forward kinematics, rather than the articulation's mutable runtime state. It
transforms every link mesh into the requested target link's initial frame,
merges the meshes, and uses Open3D uniform surface sampling on the combined
triangle mesh. Sampling the merged mesh makes each surface's representation
proportional to triangle area instead of assigning the same point count to
every link. The returned float32 tensors are moved back to the articulation
device and are ready to store in an atomic action's `ObjectSemantics.geometry`:

```python
geometry = articulation.sample_initial_point_clouds(
"button_cap",
articulation_point_count=100_000,
target_point_count=5_000,
)
target_points = geometry["target_link_point_cloud"]
articulation_points = geometry["articulation_point_cloud"]
revolute_origin = geometry.get("target_link_revolute_axis_origin")
```

The shown point counts are the defaults. Open3D draws random uniform samples
from the target mesh and merged articulation mesh independently, so repeated
calls are not expected to be bitwise identical and consumers should rely on
the spatial distribution rather than point-for-point correspondence. The
method requires a built kinematic chain and currently supports unit
`body_scale`.

When the target link has a revolute ancestor, the returned
`target_link_revolute_axis_origin` tensor has shape `(3,)`. It is the first
revolute joint found while walking upward from the target link (through any
fixed parent joints), evaluated at the initial joint configuration and
expressed in the target link's initial local frame. If no revolute ancestor
exists, the key is omitted without preventing point-cloud sampling. Atomic
`Twist` uses this metadata as its rotation origin. The target-cloud centroid is
used only to center neighborhood/contact calculations and is not a substitute
for a physical joint origin.

### Visual Appearance

Asset materials are wrapped automatically during articulation construction. Materials are organized by environment and link:
Expand Down
Loading
Loading