Skip to content

feat(inference): allow running a policy ref that isn't tied to a LeLab job - #114

Open
ravediamond wants to merge 1 commit into
huggingface:mainfrom
ravediamond:custom-policy-inference
Open

ravediamond wants to merge 1 commit into
huggingface:mainfrom
ravediamond:custom-policy-inference

Conversation

@ravediamond

Copy link
Copy Markdown
Contributor

Closes #111.

lerobot already supports third-party policy plugins (a pip-installed lerobot_policy_<name> package registers into PreTrainedConfig's choice registry, and lerobot-rollout already calls register_third_party_plugins() at startup) — and _resolve_policy_path in lelab/rollout.py already accepts an arbitrary local checkpoint dir or Hub ref. The actual gap was the UI: the inference modal only ever built policy_ref from a job's own checkpoints, so there was no way to point at a checkpoint that isn't tied to a LeLab training run.

Added a "Use a custom policy path or ref" toggle to the inference modal. When on, you type a local checkpoint directory or a Hub ref (same user/repo@root / user/repo@checkpoints/000050 shape as job checkpoints) instead of picking from the dropdown.

New GET /policy-config?policy_ref= endpoint reads that ref's config.json and returns the same summary (policy type, expected cameras, whether it needs a task string) the job-checkpoint route already returns — pulled the shared extraction into jobs.summarize_policy_config() so both routes use the same logic.

Known limitation, not fixed here: requires_task is driven by a hardcoded list of language-conditioned policy types (smolvla, pi0, pi0_fast, pi05). A plugin policy that needs a task string won't get the task input shown automatically — you can still type one, the field just won't appear on its own. Didn't want to guess at a convention for plugins to declare this.

Tested: pytest (273 passed, 6 new tests), ruff check/format, tsc, eslint, prod build all clean. Not tested against a real third-party plugin package — don't have one installed to verify with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: run inference with a policy from a third-party lerobot plugin

1 participant