Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,16 @@ plugrl-run-env-client mujoco-v1 \
--num-envs 1 --num-episodes 600 --runner.replan-steps 1 --runner.seed 0
```

Episode return climbs out of the -300s within a few minutes. `HalfCheetah-v5`
has a 17-dimensional observation and a 6-dimensional action, which are
exactly `fpo-policy`'s defaults, so nothing needs configuring. The
environment needs `plugrl-env-client[mujoco]`.
`HalfCheetah-v5` has a 17-dimensional observation and a 6-dimensional
action, which are exactly `fpo-policy`'s defaults, so nothing needs
configuring. The environment needs `plugrl-env-client[mujoco]`.

Episode return starts near -300. Across three seeds it is still dipping back
into the -300s at step 20k, the mean crosses zero at about 60k, and by 500k
steps it reaches **1928 ± 224** — roughly a hundred minutes on the CPU-only
machine that measured it. The first few minutes are noise, so judge it over
tens of thousands of steps rather than the first screenful. Curve, seeds and
logs: [E6](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e6-first-learning-curve).

!!! warning "`--algo.buffer-size` is not decoration"

Expand Down
10 changes: 7 additions & 3 deletions docs/index.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,13 @@ plugrl-run-env-client mujoco-v1 \
--num-envs 1 --num-episodes 600 --runner.replan-steps 1 --runner.seed 0
```

几分钟内 episode 回报就会从 -300 附近爬上来。`HalfCheetah-v5` 的观测是 17 维、
动作是 6 维,**正好是 `fpo-policy` 的默认值**,所以不需要任何配置。
环境端需要 `plugrl-env-client[mujoco]`。
`HalfCheetah-v5` 的观测是 17 维、动作是 6 维,**正好是 `fpo-policy` 的默认值**,
所以不需要任何配置。环境端需要 `plugrl-env-client[mujoco]`。

episode 回报从 -300 附近起步。三个随机种子里,到第 2 万步仍有种子会掉回 -300 区间,
均值在约 6 万步处越过零点,到 50 万步达到 **1928 ± 224** —— 在做这次测量的纯 CPU
机器上大约一百分钟。**最初几分钟全是噪声**,要按几万步的尺度看,而不是看屏幕上
最先出现的那几行。曲线、种子与日志:[E6](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e6-first-learning-curve)。

!!! warning "`--algo.buffer-size` 不是装饰"

Expand Down
Loading