Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,26 @@ were pre-registered, and one of them is falsified. The numbers, the recorded
environment of both processes, and what none of it supports:
[E11](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e11-vla-rl-libero).

## The environment side needs neither CUDA nor a GPU

The training server is 6.5G and wants a GPU. The machine running environments
does not have to be either. A LIBERO env client installs at **3.4G with no
nvidia wheels** instead of 7.8G with sixteen, sending byte-identical
observations, and renders on the **CPU** - ten clients at once, 30 of 30
episodes successful, at **1.91x** the wall clock of the same run on a GPU.

The catch is that 1.91x, and the numbers that make it: stepping is 10x slower
in software, and most but not all of that hides behind the queue of clients
waiting on one policy.

This overturns a conclusion the project had already published. E1 measured a
robomimic-class env client at 7.2G with CUDA and said the environment side does
need a GPU; that was true of a default install, where `torch` brings CUDA along
whether or not anything uses it. [E12](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e12-cuda-free-rollout)
reproduced E1's row exactly before changing one pin, and
[E13](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e13-gpu-free-rendering)
measured what rendering without a GPU costs.

## Components

- `plugrl-server`: training server, runs algorithm, policy, checkpoints, tracking
Expand Down
16 changes: 16 additions & 0 deletions docs/index.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,22 @@ FPO 用回传的反馈训练。边界本身没有任何改动,变的只是策
进程各自被记录下来的环境,以及这些数据**不能**支持的结论,见
[E11](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e11-vla-rl-libero)。

## 环境端既不需要 CUDA,也不需要 GPU

训练端有 6.5G,而且要一张显卡;跑环境的那台机器不必如此。LIBERO 环境端可以装成
**3.4G、零个 nvidia wheel**(原本是 7.8G 带十六个),发出的观测逐字节相同,并且能在
**CPU 上渲染**——十个客户端同时跑,30 个回合全部成功,代价是 **1.91 倍**墙钟。

代价就在这 1.91 倍上,构成它的数字是:软件渲染的单步慢 10 倍,其中大部分、但不是
全部,被"多个客户端排队等同一个策略"的等待所掩盖。

这推翻了本项目自己已经写下的结论。E1 曾测得 robomimic 类环境端 7.2G 且带 CUDA,并
断言这类环境的环境端**确实**需要 GPU——那对**默认安装**是成立的,因为 Linux 上
`torch` 无论用不用都会把 CUDA 一起带来。[E12](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e12-cuda-free-rollout)
先原样复现了 E1 那一行,再只改一个版本钉定;
[E13](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e13-gpu-free-rendering)
则量出了无 GPU 渲染的代价。

## 组件

- `plugrl-server`:训练端,负责算法、策略、checkpoint、指标追踪
Expand Down
Loading