Summary
With @memtensor/memos-local-plugin v2.0.18 (Hermes local adapter, full self-evolution mode), L3 world-model generation fails almost always with a misleading validation error:
[core.memory.l3.abstract] abstract.llm_failed err="l3.abstraction: 'title' must be a non-empty string"
[llm.json] malformed op="l3.abstraction.v2"
The model's answer is actually valid JSON with a non-empty title at the top level. The real failure chain is:
- The L3 answer is truncated at
maxTokens (long answer + a reasoning model burning 5–12k tokens on thoughts).
JSON.parse on the full text fails (Unterminated string).parseLlmJson falls back to extractFirstJsonBlock, which returns the first balanced {...} / [...] — a nested block (in our case the domain_tags array).validate then runs against that nested value and reports 'title' must be a non-empty string — pointing the debugging effort at the model, not at the truncation.
Related: #1755 lists these schema failures as a secondary symptom; the root cause was not analyzed there.
Environment
- Plugin:
@memtensor/memos-local-plugin v2.0.18 (Hermes local adapter, Windows)
- Memory mode: full self-evolution (
algorithm.lightweightMemory.enabled: false)
- LLM: OpenAI-compatible endpoint,
deepseek-v4-flash (reasoning model), temperature: 0, response_format: {"type":"json_object"}
- Config before the fix:
llm.maxTokens: 8192, llm.timeoutMs: 120000
- Volume:
world_model_generate — 6 successes out of 456 calls (1/22 today).
Evidence (raw answers captured from completeJson)
attempt 1: len 13082, JSON.parse error: Unterminated string starting at: line 1 column 11856 (char 11855)
raw starts with: {"title":"...","domain_tags":["network","http","local-services","cron"],...
attempt 2: len 0 -> "LLM output not valid JSON: empty response"
parsedShape (what parseLlmJson actually returned): array[4] <- the `domain_tags` array
Model-side limit probe against the same endpoint:
| max_tokens |
finish_reason |
completion_tokens |
reasoning_tokens |
| 8192 |
length |
8192 |
5004 |
| 16384 |
length |
16384 |
5081 |
| 32768 |
stop |
17374 |
11827 |
The prompt asks for a body field (full markdown rendering that duplicates environment/inference/constraints), so an L3 answer is ~13k chars — reasoning + answer do not fit in the old 8192 budget.
Fix that works locally
llm:
maxTokens: 32768
timeoutMs: 300000
After restarting the bridge/daemon, a manual L3 run over the live DB:
run.done: clusters=6 created=0 merged=6 skipped=0
5 world models updated, confidence ~1.0, no further malformed warnings.
Suggested improvements
- Don't let the parse fallback masquerade as a schema error. When the top-level text fails to parse, either restrict the extracted-block fallback to a value that satisfies the caller's shape, or surface the original parse error (with
rawPreview) instead of validating a nested value.
- Log the raw answer on malformed output.
logs/llm.jsonl stays empty and memos.log only carries "msg":"malformed" without the payload — which makes this whole bug class invisible.
- Handle
finishReason === "length" explicitly: retry with a "shorter answer" instruction (or raise the budget) instead of a blind retry; the current retry sometimes returns an empty response.
- Trim the L3 schema:
body duplicates the three structured sections; a short body (or a derived one) would roughly halve the answer size.
Happy to attach the small local diagnostic patch (adds rawPreview / rawLen / parsedShape to the malformed warning) if it helps.
Summary
With
@memtensor/memos-local-pluginv2.0.18 (Hermes local adapter, full self-evolution mode), L3 world-model generation fails almost always with a misleading validation error:The model's answer is actually valid JSON with a non-empty
titleat the top level. The real failure chain is:maxTokens(long answer + a reasoning model burning 5–12k tokens on thoughts).JSON.parseon the full text fails (Unterminated string).parseLlmJsonfalls back toextractFirstJsonBlock, which returns the first balanced{...}/[...]— a nested block (in our case thedomain_tagsarray).validatethen runs against that nested value and reports'title' must be a non-empty string— pointing the debugging effort at the model, not at the truncation.Related: #1755 lists these schema failures as a secondary symptom; the root cause was not analyzed there.
Environment
@memtensor/memos-local-pluginv2.0.18 (Hermes local adapter, Windows)algorithm.lightweightMemory.enabled: false)deepseek-v4-flash(reasoning model),temperature: 0,response_format: {"type":"json_object"}llm.maxTokens: 8192,llm.timeoutMs: 120000world_model_generate— 6 successes out of 456 calls (1/22 today).Evidence (raw answers captured from
completeJson)Model-side limit probe against the same endpoint:
The prompt asks for a
bodyfield (full markdown rendering that duplicatesenvironment/inference/constraints), so an L3 answer is ~13k chars — reasoning + answer do not fit in the old 8192 budget.Fix that works locally
After restarting the bridge/daemon, a manual L3 run over the live DB:
5 world models updated, confidence ~1.0, no further malformed warnings.
Suggested improvements
rawPreview) instead of validating a nested value.logs/llm.jsonlstays empty andmemos.logonly carries"msg":"malformed"without the payload — which makes this whole bug class invisible.finishReason === "length"explicitly: retry with a "shorter answer" instruction (or raise the budget) instead of a blind retry; the current retry sometimes returns an empty response.bodyduplicates the three structured sections; a short body (or a derived one) would roughly halve the answer size.Happy to attach the small local diagnostic patch (adds
rawPreview/rawLen/parsedShapeto the malformed warning) if it helps.