Skip to content

feat: add RTC asynchronous execution - #5

Merged
pppppppppper merged 1 commit into
mainfrom
feat/async-rtc-vlash-execution
Oct 14, 2026
Merged

pppppppppper merged 1 commit into
mainfrom
feat/async-rtc-vlash-execution

Conversation

@pppppppppper

@pppppppppper pppppppppper commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds RTC asynchronous execution across EmbodiRun's simulation and physical-control paths.

  • provides sync, naive, and rtc runtime modes with explicit overlap and modeled-delay settings
  • emits the versioned metadata.async.rtc contract to EmbodiInfer and the SGLang π0.5 integration
  • validates SGLang RTC endpoint capabilities before execution
  • supports RTC in LIBERO and SO101 control, including camera and serial recovery needed by the real-robot loop
  • keeps optional RTC numerical dependencies lazy so minimal host/recipe installations still start

The existing synchronous path remains the default when runtime_options is empty.

Companion inference PR: BUAA-CI-LAB/EmbodiInfer#2.

End-to-end LIBERO success rate

libero_spatial, 10 tasks × 5 trials, modeled delay Δ = 4 steps, identical seeds, and the standard LIBERO checkpoint:

Mode Inference behavior Trials Success rate Mean environment steps
sync synchronous upper bound 50 1.000 101.9
naive async asynchronous, no compensation 50 0.620 325.2
RTC asynchronous with prefix guidance 50 0.960 113.4

RTC recovers 34 percentage points over the naive asynchronous baseline and finishes within 4 percentage points of synchronous execution.

Verification

ruff check and ruff format --check pass for the changed source and tests.

PYTHONPATH=src pytest tests/test_async_algorithms.py tests/test_async_simulation_runtime.py tests/test_sglang_async_wire.py tests/test_sglang_http.py tests/test_control_services.py -q

Result: 85 passed, 21 skipped in the non-SGLang environment.

PYTHONPATH=src:integrations/sglang_pi05/src pytest tests/test_sglang_http.py tests/test_sglang_async_wire.py -q

Result: 33 passed with SGLang 0.5.18 and Torch 2.13.

The previously failing shared-device/runtime compatibility case now passes.

Historical real-robot benchmark recordings remain local and are intentionally excluded from this PR.

ZhouAo-ZA
ZhouAo-ZA previously approved these changes Oct 3, 2026
@pppppppppper
pppppppppper force-pushed the feat/async-rtc-vlash-execution branch 2 times, most recently from 2f8753a to fd11c15 Compare October 13, 2026 16:59
Signed-off-by: 于乐 <yule@yuledeMacBook-Air.local>
@pppppppppper
pppppppppper force-pushed the feat/async-rtc-vlash-execution branch from fd11c15 to 925367d Compare October 13, 2026 17:03
@pppppppppper pppppppppper changed the title feat: RTC and VLASH execution side (async runtime, LIBERO binding) feat: add RTC asynchronous execution Oct 13, 2026
@anHappyDog
anHappyDog self-requested a review October 13, 2026 17:20
@pppppppppper
pppppppppper requested review from ZhouAo-ZA and hootandy321 and removed request for ZhouAo-ZA October 14, 2026 01:17
@pppppppppper
pppppppppper merged commit 0636d6d into main Oct 14, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants