Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ jobs:
- run: source-leap --demo --frames 3 --json
- run: source-leap --demo --latest --health --frames 3 --json
- run: source-leap teleop --demo --headless --auto-arm --frames 3 --json
- run: source-leap teleop --demo --interpolate --headless --auto-arm --frames 3 --json
- run: python examples/receiver_expiry.py
build:
runs-on: ubuntu-latest
steps:
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ A Python package for the **Original Leap Motion Controller**. Read hand skeleton
record and replay motion, and map hand movement to a simulator or your own robot
controller through configurable adapters.

**Status:** [0.1.0b3 is published on PyPI](https://pypi.org/project/source-robotics-leap-motion/).
**Status:** [Published Windows beta on PyPI](https://pypi.org/project/source-robotics-leap-motion/).
Windows tracking and the simulated robot workflow have passed live-controller tests.
Clean-PC driver installation and Linux/macOS controller hardware tests are still pending.

Expand Down Expand Up @@ -123,7 +123,8 @@ health counters, configurable Cartesian control, and callback adapters for robot
- [User guides and documentation index](https://github.com/Source-Robotics/Source-Robotics-Leap-Motion/blob/main/docs/README.md)
- [Python API, recording, and mapping](https://github.com/Source-Robotics/Source-Robotics-Leap-Motion/blob/main/docs/USAGE.md)
- [Robot adapters and teleoperation](https://github.com/Source-Robotics/Source-Robotics-Leap-Motion/blob/main/docs/TELEOPERATION.md)
- [Compatibility and measured validation](https://github.com/Source-Robotics/Source-Robotics-Leap-Motion/blob/main/docs/B3_VALIDATION.md)
- [Interpolated receiver and gripper filtering (b4)](https://github.com/Source-Robotics/Source-Robotics-Leap-Motion/blob/main/docs/RECEIVER.md)
- [Compatibility and measured validation](https://github.com/Source-Robotics/Source-Robotics-Leap-Motion/blob/main/docs/B4_VALIDATION.md)
- [Contributing and development setup](https://github.com/Source-Robotics/Source-Robotics-Leap-Motion/blob/main/CONTRIBUTING.md)

## Repository layout
Expand Down
4 changes: 4 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ flowchart LR
| `mapping.py`, `_geometry.py` | Coordinate transforms and quaternion operations |
| `control.py`, `control_config.py`, `control_models.py` | Pure control state machine, strict configuration, poses, commands, and feedback |
| `teleoperation.py`, `adapters/` | Synchronous control/adapter integration, callback adapter, and simulated arm |
| `receiver.py`, `adapters/receiver.py` | Transport-independent interpolation/expiry state machine and optional simulated receiver worker |
| `runtime*.py`, `delivery.py`, `setup.py`, `_broker.py` | Discovery, pinned delivery/cache, explicit device setup, and shared runtime ownership |
| `cli.py`, `lab.py`, `lab.html`, `lab.js`, `lab.css` | Console entry point and packaged loopback browser lab |

Expand Down Expand Up @@ -57,6 +58,9 @@ drivers, or an SDK-specific network receiver. A controller integration provides
where needed and enforces command expiry/hold at the receiving controller.

See [teleoperation](TELEOPERATION.md) for states, ownership, and adapter callbacks.
The optional [receiver](RECEIVER.md) advances independently of incoming commands,
rejects old-session/duplicate packets, and latches command expiry using its own clock.
It provides bounded simulated feedback, not a physical robot watchdog or network transport.

## Runtime ownership

Expand Down
81 changes: 81 additions & 0 deletions docs/B4_VALIDATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# b4 validation — 6 November 2026

This report covers independent gripper filtering/cadence and the optional simulated
receiver in 0.1.0b4. The Leap tracking engine, runtime pin, and base dependencies are
unchanged. The implementation uses the package's existing geometry/contracts and
standard library; UltraGrasp supplied design ideas, with no external source bundled.

## Installed Windows wheel

The candidate wheel passed **233 tests on each** of CPython 3.10.11, 3.11.9,
3.12.10, 3.13.17, 3.14.10, and 3.15.2 in isolated Windows x64 venvs. Imports resolved
from site-packages, the import/metadata versions matched, and pip check passed.

Each environment also received 30 live sequential frames and 30 latest-mode frames,
round-tripped the 1,047-frame recording containing 50 complete hand samples, ran
30 interpolated demo frames, and executed the deterministic expiry example.
No hands were visible during the new live probes; full-skeleton input came from
the previously recorded real hand. These checks do not establish a new camera FPS,
tracking-accuracy, or end-to-end latency improvement.

## Combined recorded-hand run

The real 1,047-frame recording drove the control engine and deterministic receiver
together using its recorded intervals. The run generated **18 pose commands
and 9 gripper updates**, with a 0.12-second independent gripper time constant
and 30 Hz gripper cap. Receiver Cartesian workspace, linear/angular speed, and
normalized gripper speed checks passed. Clutch held output, explicit resume used a
fresh re-anchor and sender sequence watermark, and tracking loss stayed latched.
There were **zero commands after loss and zero receiver rejections** in this valid run.

## One-minute simulated receiver soak

A 60.0-second synthetic-hand run processed 3,457 control steps and
3,440 simulated targets. Five deliberate 0.22-second sender pauses caused
receiver-owned expiry before the sender called it again. Tracking loss emitted no
additional command; explicit recalibration created six distinct sessions in total.
The receiver worker and tracking reader both closed cleanly.

Python CPU averaged 2.55% of one core for this synthetic, headless
process, including its tracking and receiver workers. This excludes the vendor engine
and browser and is not a hardware or prior-version performance comparison.

## Receiver and lifecycle regressions

Behavioral tests verify interpolation continuity under replacement/jitter, per-channel
speed/workspace enforcement, local timeout caps, expiry at the correct instant after
a delayed tick, and expiry while the sender is not calling the simulated adapter.
Duplicates, wrong-session commands, invalid fields, and out-of-order packets do not
refresh deadlines. Pose-only packets do not extend prior gripper/rotation deadlines.

Clutch resume supplies the sender's last produced sequence, so even previously unseen
packets delayed from before clutching are rejected. Expired streams require a new
session; invalid receiver clocks latch faults. Construction starts no worker, and
close joins it even after an injected worker/close failure. Existing immediate
simulation and omitted gripper settings retain their previous behavior.

## Platform CI and packaging

Code commit: `693c666192ba9846ccbc09ebc8e913f9670a59d1`.
Platform CI results are available at the linked run; completion is still pending.
[CI run](https://github.com/Source-Robotics/Source-Robotics-Leap-Motion/actions/runs/42253682504).

Hosted CI covers Python 3.10–3.15, software behavior, installed CLI demos, and package
builds. Local checks include Ruff, JavaScript syntax, wheel-from-sdist builds, strict
Twine metadata, source-archive contents, documentation links, and candidate installs.
Versioned runtime and vendor notice contents are retained; no native code, numerical
toolbox, robot SDK, recording, or private runtime is added to the Python wheel.

## Scope

Output was simulated. No physical robot command, service-state change, or USB driver
installation occurred. The receiver worker is a software scheduling demonstration,
not a real-time guarantee or physical robot watchdog. An actual integration must
enforce expiry/hold where it can still act after the sender or network disappears.

The earlier live calibration/clutch/hand-loss acceptance and 20-minute camera soak
remain documented in [b3 validation](B3_VALIDATION.md). Clean-Windows driver-only
installation, Linux/macOS controller hardware, and actual robot integration remain
pending.

[Receiver guide](RECEIVER.md) · [Documentation index](README.md)
7 changes: 5 additions & 2 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Documentation

Start with [installation and the two Windows setup paths](../README.md#install-and-run-on-windows).
The current public package is **0.1.0b3**. Windows tracking and simulated output have
Package versions are listed on [PyPI](https://pypi.org/project/source-robotics-leap-motion/).
Windows tracking and simulated output have
live hardware evidence; clean-PC driver setup and Linux/macOS hardware remain pending.

## Using the package
Expand All @@ -10,6 +11,7 @@ live hardware evidence; clean-PC driver setup and Linux/macOS hardware remain pe
| --- | --- |
| [Python API and CLI](USAGE.md) | Hand data, units, recording/replay, mapping, and diagnostics |
| [Teleoperation and adapters](TELEOPERATION.md) | Browser lab, calibration, clutch, limits, TOML, and controller callbacks |
| [Interpolated receiver](RECEIVER.md) | Receiver-side timing/expiry, bounded interpolation, and independent gripper filtering in b4 |
| [Latest frames and health](LATEST_FRAMES.md) | Slow consumers, freshness, skipped frames, and health semantics |
| [Installation and platforms](INSTALLATION_PLATFORMS.md) | Supported setup paths and platform limitations |
| [Runtime setup](RUNTIME.md) | Automatic download/cache, driver prompts, offline setup, and private engines |
Expand All @@ -33,7 +35,8 @@ CI does not establish USB/controller support on an untested operating system.

| Report | Scope |
| --- | --- |
| [b3 acceptance](B3_VALIDATION.md) | Current live-hand behavior, 20-minute soak, Python matrix, and remaining checks |
| [b4 receiver validation](B4_VALIDATION.md) | Installed-wheel matrix, independent gripper filtering, interpolation/expiry, and recorded-hand receiver replay |
| [b3 acceptance](B3_VALIDATION.md) | Live-hand behavior, 20-minute camera soak, Python matrix, and remaining hardware checks |
| [Clean Windows checklist](CLEAN_MACHINE_CHECK.md) | Procedure for a PC without an existing Leap installation; still pending |
| [b1 runtime validation](BETA_VALIDATION.md) | Earlier delivery, caching, shared leases, restart, and crash cleanup |
| [Compatibility and performance](COMPATIBILITY_PERFORMANCE.md) | Earlier Python-version checks and old-SDK comparison measurements |
Expand Down
151 changes: 151 additions & 0 deletions docs/RECEIVER.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# Interpolated simulated receiver

> Requires 0.1.0b4 or newer. If that version is not yet available from PyPI, use a
> [source checkout](../CONTRIBUTING.md#set-up-a-source-checkout).

Version **0.1.0b4** adds an optional simulated command receiver and independent gripper
filtering. The base install remains dependency-light. These components use the
standard library and the package's own geometry; no robot SDK or numerical toolbox
is required.

## Try it

```powershell
source-leap teleop --demo --interpolate
```

Remove `--demo` to use the connected Leap controller with simulated output. The
viewer starts disarmed. Arm and calibrate normally; its receiver indicator reports
the receiver state and accepted-command count. The viewport uses the interpolated
simulated feedback. Omitting `--interpolate` keeps immediate simulated feedback.

For a bounded run without opening a browser:

```powershell
source-leap teleop --demo --interpolate --headless --auto-arm --frames 60 --json
```

JSON adds a `receiver` status object; it is `null` for the immediate simulator.
The lab requests a 200 Hz receiver worker and a 40 ms interpolation horizon, with
workspace/speed bounds taken from the teleoperation configuration. This is a software
simulation, not a robot dynamics model or a real-time scheduling guarantee.

## Independent gripper filtering

The existing `[gripper]` TOML table accepts two optional settings:

```toml
[gripper]
source = "grab"
open_value = 1.0
closed_value = 0.0
max_speed = 1.0
deadband = 0.01
smoothing_time_constant = 0.12
max_output_hz = 30.0
```

Omit the new settings to retain the previous behavior: gripper smoothing inherits
`teleoperation.smoothing_time_constant`, and every emitted pose command can carry a
gripper value. A separate time constant changes gripper responsiveness independently
of position/orientation smoothing. A separate rate cap reduces gripper updates while
pose updates continue at their own cap. It must not exceed the total teleoperation
command-rate cap. Zero time constant disables smoothing; both frequency caps remain
upper bounds dependent on fresh frames, not fixed-rate guarantees.

Between gripper updates, `PoseTarget.gripper` is `None`, meaning no new gripper
setpoint. Speed limits use the elapsed time since the last gripper output. Calibration
and clutch re-anchoring reset both its reference and timing. The supplied simulated
receivers preserve existing gripper motion until its own command deadline; pose-only
packets do not extend that deadline. Choose a gripper cadence appropriate to command
lifetime; slower updates can intentionally leave held intervals between valid commands.

## Receiver contract

`CartesianReceiver` is a deterministic, transport-independent state machine. Call
`begin_session(session_id)` explicitly, then `accept(PoseTarget)` for incoming commands
and `advance()` from the receiving side's own loop. It retains one bounded interpolation
plan per channel rather than a FIFO of old targets. All time-dependent methods accept
an optional `now=` value from one monotonic clock for deterministic tests/replay.

- Session IDs identify new calibrations. Packets from other sessions are rejected.
- Sequence numbers must increase; rejected duplicates/out-of-order packets do not
refresh expiry.
- Expiry uses local receipt time plus the smaller of `valid_for_s` and the configured
`command_timeout`. Camera timestamps and wall clocks never extend that deadline.
- Interpolation starts from the current simulated pose, preserving continuity when
replacing a target. Receiver workspace, linear/angular speed, and gripper speed
bounds are enforced independently of sender-side checks.
- A delayed `advance()` samples motion at the expiry instant before holding; it does
not move to the overdue target at the eventual tick time.
- Missing orientation/gripper fields leave their previous plans in place without
refreshing those channels' own expiry. Position packets cannot keep an old gripper
or rotation command valid indefinitely.
- Stream expiry latches. Ordinary packets cannot rearm it. A new explicit session is
required. A deliberate clutch hold/resume can retain its session and sequence
watermark; the resume handshake supplies the sender's last produced sequence,
rejecting even unseen packets delayed from before the clutch. Other holds require
a new session.
- Invalid/regressed receiver time latches a fault; close/reopen is required.

The [expiry example](../examples/receiver_expiry.py) demonstrates this using explicit
clock values, without sleeping, starting threads, or opening devices:

```powershell
python examples/receiver_expiry.py
```

## Use the receiver with Teleoperator

`SimulatedReceiver` implements the robot adapter contract and owns one interpolation/
expiry worker. Construction/import starts no threads. `open()` starts its worker;
`close()` wakes and joins it. `read_feedback()` returns owned simulated feedback.
The worker continues to expire commands while the sending application is idle.

```python
from source_robotics_leap_motion import (
Pose, ReceiverConfig, TeleoperationConfig, Teleoperator, Tracker,
)
from source_robotics_leap_motion.adapters import SimulatedReceiver

config = TeleoperationConfig.from_toml("examples/teleoperation.toml")
center = (config.workspace.minimum + config.workspace.maximum) * 0.5
robot = SimulatedReceiver(
Pose(center),
config=ReceiverConfig(
workspace=config.workspace,
max_linear_speed=config.max_linear_speed,
max_angular_speed=config.max_angular_speed,
max_gripper_speed=config.gripper.max_speed,
),
)

with Tracker.demo(mode="latest", max_frame_age=config.max_frame_age) as tracker:
with Teleoperator(tracker, robot=robot, config=config) as teleop:
teleop.arm()
for _ in range(120):
teleop.step()
```

Session-aware adapters declare `RobotCapabilities.command_sessions=True` and implement
`begin_session(session_id, after_sequence)`. `Teleoperator` calls it after explicit arm calibration
and after a requested clutch resume has received a fresh re-anchor frame. Existing
adapters default to `False` and keep their previous lifecycle.
`CallbackAdapter(..., begin_session=callback)` wraps this optional handshake.
The pure receiver accepts the watermark as the keyword `after_sequence=`; initial
calibration defaults to zero. A resumed sender must supply its current sequence,
which must not be lower than the receiver's accepted sequence.

This receiver supplies simulation and an integration boundary, not a network protocol
or hardware controller. A physical integration must run expiry/hold where it can
still act after the sending process or network disappears. The simulated worker
inside the sender process cannot supply that physical guarantee.

## Design reference

The separate robot-loop and gripper-filter ideas were informed by
[UltraGrasp](https://github.com/kvasios/ultragrasp/tree/3495255321b036b9bcaca99d06f4f7882991d8cf).
This implementation uses the existing package contracts and geometry; no UltraGrasp,
UMI, or Ultraleap binding code is bundled by this addition.

[Documentation index](README.md)
4 changes: 2 additions & 2 deletions docs/RELEASING.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Releasing

Package and runtime releases have separate versions. The published Python package
is **0.1.0b3**; it reuses the pinned Windows 3.2.1 runtime hosted with **v0.1.0b1**.
Package and runtime releases have separate versions. Python package versions b3 and
b4 reuse the pinned Windows 3.2.1 runtime hosted with **v0.1.0b1**.
A package or documentation change does not require regenerating the native archive.
Published wheel/sdist/runtime files are immutable: use a new version for changed bytes.

Expand Down
10 changes: 8 additions & 2 deletions docs/TELEOPERATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ and needs no web framework, numerical toolbox, robot SDK, or JavaScript build st
```

The first command uses a synthetic hand. The second uses the Leap controller. Both
send targets exclusively to `SimulatedArm`, which supplies immediate Cartesian
feedback. The viewer shows a workspace, tool position/orientation, recent path,
send targets to simulated output. `SimulatedArm` supplies immediate Cartesian
feedback. Add `--interpolate` for the optional [simulated receiver](RECEIVER.md),
with its own interpolation and command-expiry loop. The viewer shows a workspace, tool position/orientation, recent path,
normalized gripper openness, frame rate, and local tracking age. It is a target
visualizer, with no robot joint geometry, inverse kinematics, collisions, or dynamics.

Expand Down Expand Up @@ -55,6 +56,11 @@ reference to `arm(reference)` and `release_clutch(reference)`. Orientation/gripp
capabilities default to false and must be declared when implemented. Unsupported
configured commands are rejected before opening the adapter.

Gripper smoothing and its output-rate cap can be set independently in `[gripper]`;
see [gripper filtering and timing](RECEIVER.md#independent-gripper-filtering).
Session-aware receivers add an explicit `begin_session` handshake on arm and after
fresh clutch re-anchoring; existing adapters keep their previous lifecycle.

The runnable [callback example](../examples/custom_robot.py) wraps the simulated
controller through these exact callbacks. Replacing that boundary is the integration
work for an arm. Cartesian controllers can accept mapped poses directly after unit
Expand Down
Loading