Report unprocessable submissions as invalid-submission errors (422)

[m-messer]

Summary

Lets an evaluation function tell the caller that it can't process a submission (e.g. an unparseable expression), so shimmy can return 422 instead of a 200 or a 500.

• Handler._call_user_handler turns a ValueError raised by a user handler into InvalidSubmissionError. Other exceptions are handled as before.
• RPC transport: JsonRpcHandler returns InvalidSubmissionError as a JSON-RPC error with code 422 (INVALID_SUBMISSION_CODE).
• File transport: FileHandler returns {"error": {"message": ..., "code": "INVALID_SUBMISSION"}}. Other errors are now {"error": {"message": ...}} instead of a bare string, which is the shape shimmy reads.
• Adds tests for the RPC and file handlers.

⚠️ Behaviour change for all evaluation functions

Any ValueError that escapes an evaluation function, preview or chat handler is now reported as an invalid submission (422), not a generic error. Functions that raise ValueError for anything other than bad input should raise a different exception type.

Related

• shimmy: maps the RPC 422 code and the file INVALID_SUBMISSION code to an HTTP 422 response: Return 422 when the evaluation function rejects a submission shimmy#38.
• compareSets: Raise on unparseable submissions so they surface as 422 compareSets#13 uses this to return 422 for unparseable sets.

🤖 Generated with Claude Code