Add R2E-Gym patch execution reward - #12
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
R2E-Gym currently rewards generated issue text; model outputs are never applied as patches or evaluated by task tests. This adds an opt-in patch mode that asks for a unified diff, applies it in an isolated task checkout or Singularity sandbox, runs tests before and after the patch, and rewards a candidate only when the baseline fails and the patched tests pass. Execution feedback is returned to the next rollout turn. The existing issue-generation mode remains the default.
The Docker path uses a local repository checkout and the dataset image. The Singularity path supports R2E task SIF images with
/testbedand/r2e_tests. Both require an explicit test command; configuration and prerequisites are documented indocs/r2e_patch_execution.md.Validation: the R2E workflow and harness tests passed in the remote PyTorch environment. A real SymPy R2E SIF task failed before the known patch and passed after it, and a
R2EGymWorkflow.run_episodesmoke returned reward 1.0 from that container execution. The new harness and workflow tests are included in the CPU CI suite.