Add explicit cost-aware expansion and global scaling - #2
KataDavidXD wants to merge 2 commits into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 110919dc2d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| transition = transition_model.transition(current, action, context) | ||
| if not isinstance(transition, TransitionResult): | ||
| raise TypeError("transition_model must return TransitionResult") |
There was a problem hiding this comment.
When an environment-backed rollout has multiple actions and a later transition() raises, this helper unwinds before returning the effects already collected for earlier successful transitions. The caller's recovery block only surrounds the subsequent evaluator call, so costs and materialized keys from the expansion and completed rollout steps are neither merged into the context nor checkpointed, corrupting recovery accounting for partially executed trajectories.
Useful? React with 👍 / 👎.
| reward = float(result.reward) | ||
| evaluation_cost = float(result.total_cost) | ||
| terminal_evaluation_cost = float(result.total_cost) | ||
| if not math.isfinite(reward): |
There was a problem hiding this comment.
When transitions have already realized effects but the evaluator returns a malformed reward or cost (for example NaN, a negative cost, or a value that cannot be converted to float), these conversions and validations raise outside the evaluator recovery block. Consequently the known transition cost and reuse keys are discarded and no recovery checkpoint is emitted, unlike when evaluate() raises directly; result conversion and validation should use the same failure-accounting path.
Useful? React with 👍 / 👎.
1 participant
Summary
This PR also includes the existing observable-transition and search-signal prerequisite commit that was not yet on main.
Validation