docs: E11 on the home page, and the pi0 examples that could not run

docs/env/libero_env.md

@@ -24,6 +24,41 @@ Run a few episodes.
 plugrl-run-env-client libero-v1 --num-episodes 10 --server-host 127.0.0.1 --server-port 8000
 ```
 
+## As E11 ran it
+
+E11 evaluated a `pi05_libero` checkpoint on LIBERO through a PlugRL server: one
+client process per task, ten tasks at once, with each task's initial states
+taken in order rather than sampled.
+
+```bash
+plugrl-run-env-client libero-v1 \
+  --server-host 127.0.0.1 --server-port 8000 \
+  --num-envs 1 --num-procs 10 --num-episodes 10 \
+  --env.task-suite-name libero_spatial \
+  --env.no-randomize-initial-state \
+  --runner.pass-proc-id \
+  --runner.replan-steps 5 \
+  --runner.seed 7 \
+  --recorder.no-thread0-only \
+  --exp-name my_eval
+```
+
+- `--runner.pass-proc-id` gives process *i* task *i*, so ten processes cover a
+  ten-task suite and `--num-episodes` counts per task rather than in total. To
+  hold one task fixed instead - which is what the fine-tuning runs did - drop
+  that flag and pass `--env.task-id 8 --env.randomize-initial-state`.
+- `--recorder.no-thread0-only` makes every process write its own
+  `summary.json`. Without it only process 0 reports, and per-task success
+  rates cannot be recovered afterwards.
+- Rendering goes through EGL, so the client needs `MUJOCO_GL=egl` and
+  `PYOPENGL_PLATFORM=egl` in its environment. Which GPU it renders on follows
+  EGL's own device order, which need not agree with `CUDA_VISIBLE_DEVICES`.
+- LIBERO writes its own config file, so point `LIBERO_CONFIG_PATH` somewhere
+  writable before the first run.
+
+Protocol, results and the recorded environment of both processes:
+[E11](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e11-vla-rl-libero).
+
 ## Verify
 
 - Env client can create the Libero env.

docs/env/libero_env.zh.md

@@ -24,6 +24,37 @@ plugrl-run-env-client libero-v1 --help
 plugrl-run-env-client libero-v1 --num-episodes 10 --server-host 127.0.0.1 --server-port 8000
 ```
 
+## E11 是怎么跑的
+
+E11 通过一个 PlugRL server 在 LIBERO 上评测 `pi05_libero` checkpoint：一个任务一个
+客户端进程，十个任务同时跑，每个任务的初始状态按顺序取，而不是随机采样。
+
+```bash
+plugrl-run-env-client libero-v1 \
+  --server-host 127.0.0.1 --server-port 8000 \
+  --num-envs 1 --num-procs 10 --num-episodes 10 \
+  --env.task-suite-name libero_spatial \
+  --env.no-randomize-initial-state \
+  --runner.pass-proc-id \
+  --runner.replan-steps 5 \
+  --runner.seed 7 \
+  --recorder.no-thread0-only \
+  --exp-name my_eval
+```
+
+- `--runner.pass-proc-id` 让第 *i* 个进程跑第 *i* 个任务，十个进程正好覆盖十任务
+  套件；此时 `--num-episodes` 是**每个任务**的回合数，不是总数。若想固定跑单个
+  任务——微调那几次就是这么做的——去掉该 flag，改传
+  `--env.task-id 8 --env.randomize-initial-state`。
+- `--recorder.no-thread0-only` 让每个进程都写出自己的 `summary.json`。不加它就只有
+  0 号进程上报，事后无法还原逐任务成功率。
+- 渲染走 EGL，客户端环境里需要 `MUJOCO_GL=egl` 与 `PYOPENGL_PLATFORM=egl`。它渲染
+  在哪块 GPU 上由 EGL 自己的设备顺序决定，不一定与 `CUDA_VISIBLE_DEVICES` 一致。
+- LIBERO 会写自己的配置文件，首次运行前把 `LIBERO_CONFIG_PATH` 指到可写目录。
+
+协议、结果，以及两个进程各自被记录下来的环境：
+[E11](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e11-vla-rl-libero)。
+
 ## 验证
 
 - env client 能创建 Libero 环境

docs/index.md

@@ -57,6 +57,25 @@ confirm the two sides talk to each other, not to train anything.
   and starts stepping episodes.
 - With `fpo`, the server prints a metrics table whose `rollout/reward` rises.
 
+## A real VLA, end to end
+
+The same two processes carry a full-size pi0.5. The env client steps LIBERO,
+the server answers with actions, and FPO trains on the feedback that comes
+back. Nothing about the boundary changes; only the policy does.
+
+As a control, the unmodified checkpoint scored 99 of 100 on `libero_spatial`
+and 185 of 200 on `libero_10`, against openpi's published 98.8 and 92.4 - and
+the server's episode and step counts matched the clients' exactly, which is
+what says the transport dropped nothing.
+
+**The reinforcement learning result is negative.** One FPO iteration on the
+hardest task took its success rate from 26 of 50 to 0 of 50, and the run is
+incomplete at one iteration of ten: a second learn step does not fit beside
+the optimizer state the first one allocates on a 24 GB card. The predictions
+were pre-registered, and one of them is falsified. The numbers, the recorded
+environment of both processes, and what none of it supports:
+[E11](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e11-vla-rl-libero).
+
 ## Components
 
 - `plugrl-server`: training server, runs algorithm, policy, checkpoints, tracking

docs/index.zh.md

@@ -51,6 +51,21 @@ dummy 算法的 `learn` 是一个 sleep，不会移动任何权重。它用来
 - env client 打印 server 元信息（策略名、动作形状）并开始跑 episode
 - 用 `fpo` 时，server 的指标表里 `rollout/reward` 会上升
 
+## 真实 VLA，端到端
+
+同样的两个进程也能承载一个完整尺寸的 pi0.5：环境端步进 LIBERO，服务端返回动作，
+FPO 用回传的反馈训练。边界本身没有任何改动，变的只是策略。
+
+作为对照，未经微调的 checkpoint 在 `libero_spatial` 上 99/100、在 `libero_10` 上
+185/200，与 openpi 公布的 98.8 与 92.4 一致；而且服务端记录的回合数与步数和客户端
+逐一相等——正是这一条说明传输层没有悄悄丢掉任何东西。
+
+**强化学习的结果是负面的。** 在最难的那个任务上，一轮 FPO 把成功率从 26/50 打到
+0/50；而且这次训练并不完整，10 轮只跑完 1 轮——第二次 learn 装不下，它要和第一次
+分配的优化器状态挤在同一张 24 GB 卡上。预测是预注册的，其中一条被证伪。数字、两个
+进程各自被记录下来的环境，以及这些数据**不能**支持的结论，见
+[E11](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e11-vla-rl-libero)。
+
 ## 组件
 
 - `plugrl-server`：训练端，负责算法、策略、checkpoint、指标追踪

docs/policy/dppo_policy.md

@@ -13,15 +13,28 @@ This page covers:
 DPPO policy.
 
 ```bash
-plugrl-run-server dppo-policy default dppo hopper --exp_name my_dppo_exp
+plugrl-run-server dppo-policy default dppo hopper --exp-name my_dppo_exp
 ```
 
-OpenPI PI0 policy (requires a checkpoint directory).
+OpenPI PI0 policy (requires a checkpoint directory). It is a flow policy: it
+pairs with `fpo` or `eval`, never with `dppo`. These are the two invocations
+[E11](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e11-vla-rl-libero) ran.
 
 ```bash
-plugrl-run-server pi0-policy default dppo hopper \
-  --policy.checkpoint_path /path/to/pi0_checkpoint \
-  --policy.name pi05_tiny_libero
+# evaluation - no learning
+plugrl-run-server pi0-policy default eval default \
+  --policy.name pi05_libero \
+  --policy.checkpoint-path /path/to/pi0_checkpoint \
+  --policy.device cuda
+
+# FPO fine-tuning
+plugrl-run-server pi0-policy default fpo default \
+  --policy.name pi05_libero \
+  --policy.checkpoint-path /path/to/pi0_checkpoint \
+  --policy.device cuda \
+  --algo.learning-rate 1e-5 --algo.batch-size 8 \
+  --algo.n-samples-per-action 4 --algo.buffer-size 4096 \
+  --algo.global-steps 40960
 ```
 
 ## Verify
@@ -52,26 +65,26 @@ python -c "import my_pkg.plugrl_policies; from plugrl_server.cli import main; ma
 
 Common flags.
 
-- `--policy.env_type gym`
-- `--policy.env_name hopper-medium-v2`
-- `--policy.checkpoint_path /path/to/checkpoint.pt`
+- `--policy.env-type gym`
+- `--policy.env-name hopper-medium-v2`
+- `--policy.checkpoint-path /path/to/checkpoint.pt`
 - `--policy.critic.*`
 
 ## Built-in: `pi0-policy` (OpenPI)
 
 - UID: `pi0-policy`
 - Code: `plugrl-server/src/plugrl_server/policy/openpi/openpi_policy.py`
-- `--policy.checkpoint_path` is required and must point to a directory with:
+- `--policy.checkpoint-path` is required and must point to a directory with:
   - `model.safetensors`
   - `assets/` with normalization stats
 - Setup notes: `plugrl-server/src/plugrl_server/policy/openpi/README.md`.
 
 Common flags.
 
-- `--policy.name pi05_tiny_libero`
-- `--policy.denoising_steps 5`
-- `--policy.train_expert_only true`
-- `--policy.default_prompt "..."`
+- `--policy.name pi05_libero`
+- `--policy.denoising-steps 5`
+- `--policy.train-expert-only true`
+- `--policy.default-prompt "..."`
 
 ## Implement a diffusion-style policy
 

docs/policy/dppo_policy.zh.md

@@ -13,15 +13,27 @@
 DPPO 策略。
 
 ```bash
-plugrl-run-server dppo-policy default dppo hopper --exp_name my_dppo_exp
+plugrl-run-server dppo-policy default dppo hopper --exp-name my_dppo_exp
 ```
 
-OpenPI PI0 策略（需要 checkpoint 目录）。
+OpenPI PI0 策略（需要 checkpoint 目录）。它是流策略：与 `fpo` 或 `eval` 搭配，
+**绝不**与 `dppo` 搭配。下面两条就是 [E11](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e11-vla-rl-libero) 实际跑的命令。
 
 ```bash
-plugrl-run-server pi0-policy default dppo hopper \
-  --policy.checkpoint_path /path/to/pi0_checkpoint \
-  --policy.name pi05_tiny_libero
+# 评测 —— 不训练
+plugrl-run-server pi0-policy default eval default \
+  --policy.name pi05_libero \
+  --policy.checkpoint-path /path/to/pi0_checkpoint \
+  --policy.device cuda
+
+# FPO 微调
+plugrl-run-server pi0-policy default fpo default \
+  --policy.name pi05_libero \
+  --policy.checkpoint-path /path/to/pi0_checkpoint \
+  --policy.device cuda \
+  --algo.learning-rate 1e-5 --algo.batch-size 8 \
+  --algo.n-samples-per-action 4 --algo.buffer-size 4096 \
+  --algo.global-steps 40960
 ```
 
 ## 验证
@@ -52,26 +64,26 @@ python -c "import my_pkg.plugrl_policies; from plugrl_server.cli import main; ma
 
 常用参数。
 
-- `--policy.env_type gym`
-- `--policy.env_name hopper-medium-v2`
-- `--policy.checkpoint_path /path/to/checkpoint.pt`
+- `--policy.env-type gym`
+- `--policy.env-name hopper-medium-v2`
+- `--policy.checkpoint-path /path/to/checkpoint.pt`
 - `--policy.critic.*`
 
 ## 内置：`pi0-policy`（OpenPI）
 
 - UID：`pi0-policy`
 - 代码：`plugrl-server/src/plugrl_server/policy/openpi/openpi_policy.py`
-- `--policy.checkpoint_path` 必填，目录内需要：
+- `--policy.checkpoint-path` 必填，目录内需要：
   - `model.safetensors`
   - `assets/`（归一化统计）
 - 本地安装/替换步骤见：`plugrl-server/src/plugrl_server/policy/openpi/README.md`。
 
 常用参数。
 
-- `--policy.name pi05_tiny_libero`
-- `--policy.denoising_steps 5`
-- `--policy.train_expert_only true`
-- `--policy.default_prompt "..."`
+- `--policy.name pi05_libero`
+- `--policy.denoising-steps 5`
+- `--policy.train-expert-only true`
+- `--policy.default-prompt "..."`
 
 ## 自定义 diffusion policy
 

docs/policy/index.md

@@ -24,10 +24,14 @@ plugrl-run-server --help
 - `fpo-policy`: FPO flow-matching policy, used by the get-started run
 - `pi0-policy`: OpenPI policy, requires a checkpoint path
 
-OpenPI example.
+OpenPI example. `pi0-policy` is a flow policy, so it pairs with `fpo` or with
+`eval` - not with `dppo`, which expects a diffusion policy.
 
 ```bash
-plugrl-run-server pi0-policy default dppo hopper --policy.checkpoint_path /path/to/checkpoint
+plugrl-run-server pi0-policy default eval default \
+  --policy.name pi05_libero \
+  --policy.checkpoint-path /path/to/checkpoint \
+  --policy.device cuda
 ```
 
 ## Troubleshooting

docs/policy/index.zh.md

@@ -24,10 +24,14 @@ plugrl-run-server --help
 - `fpo-policy`：FPO flow matching 策略，快速上手那条命令用的就是它
 - `pi0-policy`：OpenPI 策略，需要 checkpoint 路径
 
-OpenPI 示例。
+OpenPI 示例。`pi0-policy` 是流策略，因此与 `fpo` 或 `eval` 搭配，
+**不能**与 `dppo` 搭配——后者要的是 diffusion 策略。
 
 ```bash
-plugrl-run-server pi0-policy default dppo hopper --policy.checkpoint_path /path/to/checkpoint
+plugrl-run-server pi0-policy default eval default \
+  --policy.name pi05_libero \
+  --policy.checkpoint-path /path/to/checkpoint \
+  --policy.device cuda
 ```
 
 ## 常见问题