Fix #2356: l3.abstraction: misleading "title must be a non-empty string" error — output tru - #2357
Conversation
…a error When an LLM output is truncated at maxTokens, two cooperating bugs caused a misleading 'title must be a non-empty string' validation error that pointed blame at the model output rather than the real cause. Bug 1 — extractFirstJsonBlock (json-mode.ts): the old loop continued scanning for a nested balanced sub-block when the root opener returned null from walkToClose. This allowed a fully-balanced inner value (e.g. a domain_tags array) to pass JSON.parse successfully, after which the caller's validate() received that nested value and threw a field-level schema error. Fix: stop at the first opener. If walkToClose returns null (root block is truncated), return null immediately rather than hunting for a balanced child. Bug 2 — completeJson (client.ts): finishReason=length was set on LlmCompletion by providers when the token budget was exhausted, but completeJson passed the truncated text straight to parseLlmJson without inspecting it. Fix: check finishReason === 'length' before attempting to parse. Throw LLM_OUTPUT_MALFORMED immediately with finishReason and rawLen in details so the log clearly names the cause. Also adds rawLen to all existing malformed warn rows so token-budget diagnosis is easier without a separate log query. Closes MemTensor#2356
🤖 Open Code ReviewTarget: PR #2357 🔍 OpenCodeReview found 1 issue(s) in this PR. 1.
|
🔧 Open Code Review requested Agent fixOpen Code Review found 1 issue(s). I have resumed the development Agent to fix them.
The Agent will push a new commit to this PR branch. OCR will recheck after the commit is pushed. |
Description
Fixed the misleading "title must be a non-empty string" error in L3 world-model generation caused by two cooperating bugs.
Bug 1 —
extractFirstJsonBlock(json-mode.ts): The old loop continued scanning for a nested balanced sub-block when the root opener returnednullfromwalkToClose. A truncated root object like{"title":"...","domain_tags":["network","http"],...left the root{unbalanced, causing the loop to continue and find the innerdomain_tagsarray, which was fully balanced.JSON.parsethen succeeded on that array, and the caller'svalidate()received["network","http","local-services","cron"]instead of the full object — correctly reporting'title' must be a non-empty stringbut pointing blame at the model rather than at token truncation. Fix: stop at the first opener; ifwalkToClosereturnsnull, returnnullimmediately.Bug 2 —
completeJson(client.ts):finishReason === "length"was already present onLlmCompletionbutcompleteJsonpassed the truncated text straight toparseLlmJsonwithout inspecting it. Fix: checkfinishReason === "length"before parsing; throwLLM_OUTPUT_MALFORMEDimmediately withfinishReason: "length"andrawLenin the error details. Also addedrawLento all existing malformed warn log rows for easier token-budget diagnosis.Tests: 5 new test cases added (2 in
json-mode.test.ts, 3 inclient.test.ts). All 43 LLM unit tests pass. Pre-existing failures insubscriber.test.tsare unrelated (SQLite native binding absent in this environment).Related Issue (Required): Fixes #2356
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Automated tests are pending.
Checklist
@whipser030, @hijzy please review this PR.
Reviewer Checklist