Repository navigation
dream(evaluation-adapters): New bench test pins darwin evaluator to… - #27
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft PR opened by the dream engine on a GATED ACCEPT night (2026-10-07-dream-machine, run
98fb7cbb0bccc2f2).Candidate tree
5246ec1703ef7da3b42a5df46de50164723f3c6ewas applied in isolation and the required evaluators re-run against it: bench=PASSED, darwin=PASSED. Witnessfd7a41d5a3c1.A human decides the merge — evaluation is not promotion.
Finding: Given dream.config.json routes the darwin evaluator through
scripts/darwin-entrypoint.sh(A: tonight's compiled prompt embeds./scripts/darwin-entrypoint.sh npx @metaharness/darwin@0.10.2 evolve . --sandbox mock …; B:compileembedsevaluatorEntrypointsverbatim — shown fixture test assertsprompt.toContain('experiments/*/run.mjs'),packages/compile/src/index.test.ts), when bench asserts the compiled self-hosted prompt contains the entrypoint-wrapped pinned darwin command, then the new test passes (224/224) and any future config edit that unwraps the pinned npx from the script fails bench.Gate: ACCEPT upheld: 2 required evaluator(s) passed on candidate tree 5246ec1703ef