Give a refused spec a second try with in2lambda's refusal in the prompt - #30
Open
peterbjohnson wants to merge 2 commits into
Open
peterbjohnson wants to merge 2 commits into
peterbjohnson wants to merge 2 commits into
Conversation
…in the prompt (t34)
…in the prompt (t34)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three documents in the sweep ended with
spec rejectedbecause in2lambda refused the spec the model wrote (a comma inside a quoted pattern;Header, Tableon one line).iterate_specraises SpecRejected and the run ends, though the loop has tries left and the refusal names the line and the fault.Treat a refused spec as a failed try: record it as a SpecTry with
rejected: <message>and the worst score, show the refusal to the next call in the revision prompt underin2lambda refused the spec:with the spec text, and go on to the next try. Where every try is refused, the run ends as today with the last refusal. The spec prompt also says, in the list of things about selectors: several selectors for one role go under the role as a YAML list, never on one line with commas, and a comma inside a pattern must be inside its quotes. Tests in tests/test_spec.py: a fake backend whose first reply is a spec in2lambda refuses and whose second is good - the run builds, the second prompt holds the refusal, the record holds both tries. Done when the three shapes from the sweep get a second try.Workbench ticket t34.