Skip to content

Classify validator-rejected dialog answers as model output - #296

Merged
iamwavecut merged 2 commits into
mainfrom
fix/dialog-validator-failure-class
Nov 13, 2026
Merged

iamwavecut merged 2 commits into
mainfrom
fix/dialog-validator-failure-class

Conversation

@iamwavecut

Copy link
Copy Markdown
Owner

Problem

Two dialog session-engine exits reported provider_protocol_error when the provider answered and our own validation rejected the answer:

  • the answer became empty after sanitization, the provider returned no text and no tool material, or the model skipped an explicitly requested image tool (dialog_turn/session.rs, sanitize branch);
  • the outbound deliverability validator rejected the answer (dialog_turn/session.rs, outbound branch).

The job-retry exhaustion event then repeated that class, so the admin report and the maintenance controller blamed a healthy provider. Production attribution for #277: all 11 job exhaustions labelled provider_protocol_error since 2026-10-20 carry the ledger reason sanitized_empty_exhausted (9 × "dialog answer became empty after sanitization", 2 × "did not execute the explicitly requested image tool"); none came from an endpoint decode failure.

The Gemini adapter had the same disagreement: it typed "empty final text" as a protocol error, while the shared message classifier maps that phrase to model_output_rejected.

Change

  • Both session-engine exits pass FailureReason::ModelOutputRejected.
  • The Gemini adapter's two "empty final text" sites use ModelOutputRejected. Undecodable bodies, missing candidates and missing content stay ProviderProtocolError.

Behaviour is otherwise unchanged. These exits never reach the routed walker, so they never charged a circuit breaker; the whole turn is still requeued with the same attempt cap and turn budget. Only the reported class changes, and is_admin_actionable_event does not depend on it.

Tests

  • dialog_worker_fails_when_answer_sanitizes_to_empty_after_attempts_exhausted, the raw-empty and the undeliverable-answer exhaustion tests now also assert that the terminal all_attempts_exhausted routing event carries last_retryable_reason = model_output_rejected.
  • gemini_empty_text_is_a_model_output_rejection_and_a_bad_body_a_protocol_fault covers both Gemini empty-text paths and keeps an undecodable body as provider_protocol_error.

Local checks: cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test -p openplotva-llm, cargo test -p openplotva-app.

Fixes #277

…col faults

The session engine typed answers that our sanitizer emptied, that skipped a
requested image tool, or that failed outbound validation as
provider_protocol_error. The terminal routing event then blamed a healthy
provider. Report these exits, and the Gemini adapter's empty final text, as
model_output_rejected, matching the shared message classifier. Undecodable
bodies stay protocol faults.

Fixes #277
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Messages
📖 Danger checked 4 changed file(s).

Generated by 🚫 dangerJS against a589f0a

@iamwavecut
iamwavecut merged commit fb95a81 into main Nov 13, 2026
12 checks passed
@iamwavecut
iamwavecut deleted the fix/dialog-validator-failure-class branch November 13, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant