[yufoo1]

Summary

Bring ActiveVLN and StreamVLN inference optimizations onto current main: speculative/action-tree decoding, GPU image/history processing, CUDA graphs, resident/incremental KV state, packed prefill, and shared fused operators. StreamVLN also gains bounded-memory INT8 execution and tensor-batch HTTP benchmark services.

Publish portable measured configurations and verified summaries for the same first 128 episodes of each fixed 256-episode selection under benchmarks/{activevln,streamvln}-benchmark/{rtx-4090,orin-nx}. RTX 4090 runs inference and rendering on one card; Orin NX uses local weights and the LAN 4090 for rendering. The NX ActiveVLN INT8 source completed only 164/256; its published subset is explicitly 128 episodes. NX StreamVLN retains the older v9 result. These are measurements of the recorded frozen sources, not a new GPU validation of this integrated tree.

The final diff contains model code, benchmark services, tests, profiles and documentation. Go2 deployment/navigation and asynchronous execution-prefix support remain on their separate branches. Raw logs, report archives and weights are excluded from the final tree. Habitat orchestration and SR/SPL calculation live in the companion EmbodiRun change.

Type of change

• Kernel, layer, or numerical change
• Documentation
• Engine, scheduler, session, or rollout change (policy-local batching/serving only)

Verification

The selected ActiveVLN test changes remain excluded: 10 added files are removed and 3 existing files retain their main-branch versions. Residual grammar/batching cases that imported the removed test helper are also removed; independent grammar helpers and upload-layout cases remain. Inference code, benchmark implementations, profiles and results are unchanged.

Validated with the same CPU environment as CI: Python 3.12, Torch 2.10.0+cpu and Transformers 5.3.0.

CUDA_VISIBLE_DEVICES='' PYTHONPATH=. NO_PROXY=localhost,127.0.0.1 no_proxy=localhost,127.0.0.1 python -m pytest tests -q
ruff check embodiinfer tests
ruff format --check embodiinfer tests
python3 scripts/check_commit_policy.py --base upstream/main --head HEAD --title 'perf(vln): optimize ActiveVLN and StreamVLN inference'
git diff --check upstream/main HEAD

1141 passed, 311 skipped; full lint/format, public entrypoint and commit-policy checks passed. No new GPU, model-checkpoint or navigation workloads were started. Device tables retain the previously recorded frozen-source measurements.

Engine invariants

• The engine remains model-neutral; model behavior stays in policies/models and reusable kernels in backends.
• No deployment-runtime imports were added to the inference package.
• Optional paths preserve explicit dtype, shape, state and numerical contracts; INT8 is documented as a separate precision experiment.
• Dependency declarations and the lockfile are unchanged.

Checklist

• Conventional Commit subjects and existing authors' DCO sign-offs are preserved; new commits use the configured human author and sign-off.
• Proposals and benchmark/model documentation describe the implementation and validation limits.
• No weights, datasets, raw benchmark outputs, recordings, credentials, private host addresses or private paths are added to the final diff.
• Contribution follows the repository's Apache-2.0 license.