From 6c5b4bb0f9376546bfa8b55c678b6256879ad510 Mon Sep 17 00:00:00 2001 From: Gotham-Zolio <18781106300@163.com> Date: Fri, 11 Sep 2026 14:44:34 -0400 Subject: [PATCH] docs: replace "within a few minutes" with the curve E6 measured The quickstart said the return climbs out of the -300s within a few minutes. It is true in the weakest sense and useless in every other: the curve is noisy enough early that a seed is still dipping into the -300s at step 20k, and someone who watches for three minutes and sees noise concludes the thing does not learn. Now it says what the three seeds did - across zero at about 60k steps, 1928 +/- 224 at 500k, about a hundred minutes on the CPU-only machine that measured it - tells the reader the first few minutes are noise, and links the run. Both languages. Co-Authored-By: Claude Opus 5 (1M context) --- docs/index.md | 14 ++++++++++---- docs/index.zh.md | 10 +++++++--- 2 files changed, 17 insertions(+), 7 deletions(-) diff --git a/docs/index.md b/docs/index.md index 3841d05..e5f0c83 100644 --- a/docs/index.md +++ b/docs/index.md @@ -22,10 +22,16 @@ plugrl-run-env-client mujoco-v1 \ --num-envs 1 --num-episodes 600 --runner.replan-steps 1 --runner.seed 0 ``` -Episode return climbs out of the -300s within a few minutes. `HalfCheetah-v5` -has a 17-dimensional observation and a 6-dimensional action, which are -exactly `fpo-policy`'s defaults, so nothing needs configuring. The -environment needs `plugrl-env-client[mujoco]`. +`HalfCheetah-v5` has a 17-dimensional observation and a 6-dimensional +action, which are exactly `fpo-policy`'s defaults, so nothing needs +configuring. The environment needs `plugrl-env-client[mujoco]`. + +Episode return starts near -300. Across three seeds it is still dipping back +into the -300s at step 20k, the mean crosses zero at about 60k, and by 500k +steps it reaches **1928 ± 224** — roughly a hundred minutes on the CPU-only +machine that measured it. The first few minutes are noise, so judge it over +tens of thousands of steps rather than the first screenful. Curve, seeds and +logs: [E6](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e6-first-learning-curve). !!! warning "`--algo.buffer-size` is not decoration" diff --git a/docs/index.zh.md b/docs/index.zh.md index 4661db3..4ed2462 100644 --- a/docs/index.zh.md +++ b/docs/index.zh.md @@ -21,9 +21,13 @@ plugrl-run-env-client mujoco-v1 \ --num-envs 1 --num-episodes 600 --runner.replan-steps 1 --runner.seed 0 ``` -几分钟内 episode 回报就会从 -300 附近爬上来。`HalfCheetah-v5` 的观测是 17 维、 -动作是 6 维,**正好是 `fpo-policy` 的默认值**,所以不需要任何配置。 -环境端需要 `plugrl-env-client[mujoco]`。 +`HalfCheetah-v5` 的观测是 17 维、动作是 6 维,**正好是 `fpo-policy` 的默认值**, +所以不需要任何配置。环境端需要 `plugrl-env-client[mujoco]`。 + +episode 回报从 -300 附近起步。三个随机种子里,到第 2 万步仍有种子会掉回 -300 区间, +均值在约 6 万步处越过零点,到 50 万步达到 **1928 ± 224** —— 在做这次测量的纯 CPU +机器上大约一百分钟。**最初几分钟全是噪声**,要按几万步的尺度看,而不是看屏幕上 +最先出现的那几行。曲线、种子与日志:[E6](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e6-first-learning-curve)。 !!! warning "`--algo.buffer-size` 不是装饰"