Skip to content

Propose backend-independent reinforcement learning framework#704

Open
promotesd wants to merge 1 commit into
turtlebot:jazzyfrom
promotesd:agent/rl-framework-rfc
Open

Propose backend-independent reinforcement learning framework#704
promotesd wants to merge 1 commit into
turtlebot:jazzyfrom
promotesd:agent/rl-framework-rfc

Conversation

@promotesd

Copy link
Copy Markdown

Summary

This Draft PR proposes a backend-independent reinforcement-learning architecture for
TurtleBot 4 and asks maintainers to decide package ownership before any implementation
is proposed upstream.

The RFC separates:

  • a Python/NumPy task and environment core;
  • typed robot and world backend ports;
  • deterministic Mock contract tests;
  • standard ROS robot I/O and safety behavior;
  • Gazebo Harmonic world control; and
  • optional reference algorithms.

Why

RL examples tend to couple reward logic, ROS callbacks, simulator resets, and learning
frameworks. That makes deterministic testing difficult, adds dependencies to unrelated
users, and makes simulator-to-robot transitions unsafe. The proposed boundaries keep
existing TurtleBot 4 packages unchanged and make environment-dependent validation
explicit.

Requested maintainer direction

In particular, I would appreciate guidance on:

  1. whether the core abstractions and optional ROS interfaces belong in this repository
    or in a dedicated TurtleBot 4 RL repository;
  2. whether Gazebo integration should live exclusively in turtlebot4_simulator; and
  3. whether a small reference DQN is appropriate upstream or should remain a community
    example.

Scope and impact

This PR adds one RFC document. It does not add packages, dependencies, ROS interfaces,
runtime behavior, or trained models. Follow-up implementation PRs would be split only
after the repository ownership questions are resolved.

Prototype evidence

A full prototype is available for review in
promotesd/turtlebot4#1. Its automated
ROS build, Python/static checks, deterministic Mock integration, documentation, and
dependency review pass. Local environment-dependent validation also completed 100
timestamp-valid Gazebo Harmonic resets. These results demonstrate feasibility; they do
not claim upstream acceptance or physical-robot validation.

Validation

  • Diff against jazzy: one Markdown file only
  • git diff --check: passed
  • No production code or dependency changes

This remains a Draft while the architecture and repository split are discussed.

@promotesd
promotesd marked this pull request as ready for review July 16, 2026 09:38
@promotesd
promotesd requested a review from hilary-luo as a code owner July 16, 2026 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant