Pre-submission checklist
Bug Description
Summary
Skill graduation (candidate → active) requires trialsAttempted >= cfg.candidateTrials (core/skill/lifecycle.ts), and trialsAttempted only increments via applyTrial, which only fires from recordSkillTrial (core/pipeline/memory-core.ts), which only runs when getSkill() is called with recordTrial: true. But the primary path by which an agent actually consumes a skill — turn-start retrieval (core/retrieval/tier1-skill.ts → SkillCandidate) — reads repos.skills directly and injects the full invocationGuide into context without ever calling getSkill(). recordUse has the same single gate.
Net effect: the routine, load-bearing path that surfaces and uses skills generates zero lifecycle signal. The only path that generates signal is a separate, optional tool call (skill.get via e.g. memos_skill_get) that an agent has little reason to make once turn-start injection has already given it the full invocation guide. Skill candidates starve for trials not because any adapter is broken, but because MemOS's own retrieval architecture bypasses its own instrumentation point.
Evidence
Ruled out the two obvious "just misconfigured" explanations before concluding this is structural:
- The Hermes adapter's tool wiring is correct.
adapters/hermes/memos_provider/__init__.py registers memos_skill_get in get_tool_schemas() and calls skill.get with recordTrial: True in handle_tool_call(). Not a wiring bug.
- The tool isn't disabled.
_disabled_retrieval_tools() is gated by MEMOS_RETRIEVAL_TOOLS_DISABLED; verified unset (commented out, not exported) on all three of our production hosts.
SkillCandidate (core/retrieval/types.ts) carries the full invocationGuide: string, not a pointer/id requiring a follow-up call — confirmed the field is populated end-to-end from repos.skills through to the injected context.recordUse and recordTrial have exactly one call site between them: getSkill() in core/pipeline/memory-core.ts. Grepped the full file; no other caller of either.
Fleet impact (three production hosts, deployed commit)
906 of 922 candidate skills (98.3%) have never had a single trial recorded, over roughly three months of active operation with regular turn-start skill retrieval happening continuously. Skills are being surfaced and used; almost none of that generates a trial.
Suggested Fix
A couple of directions, would appreciate maintainer input on which fits the intended design:
- Have turn-start skill injection call something equivalent to
getSkill(..., { recordUse: true }) (or a lighter dedicated counter) when a SkillCandidate is actually surfaced into context — at minimum tracking exposure, ideally feeding into trial counting too.
- Alternatively, if
trialsAttempted is meant to track deliberate invocation specifically (as distinct from passive context exposure), that's a reasonable design choice — but it means candidateTrials as currently tuned assumes a usage pattern (explicit skill lookup calls) that doesn't match how skills are actually consumed via turn-start retrieval, and the gate should be reconsidered against that reality.
Happy to contribute data or a PR once maintainers weigh in on intended design.
Pre-submission checklist
Bug Description
Summary
Skill graduation (
candidate → active) requirestrialsAttempted >= cfg.candidateTrials(core/skill/lifecycle.ts), andtrialsAttemptedonly increments viaapplyTrial, which only fires fromrecordSkillTrial(core/pipeline/memory-core.ts), which only runs whengetSkill()is called withrecordTrial: true. But the primary path by which an agent actually consumes a skill — turn-start retrieval (core/retrieval/tier1-skill.ts→SkillCandidate) — readsrepos.skillsdirectly and injects the fullinvocationGuideinto context without ever callinggetSkill().recordUsehas the same single gate.Net effect: the routine, load-bearing path that surfaces and uses skills generates zero lifecycle signal. The only path that generates signal is a separate, optional tool call (
skill.getvia e.g.memos_skill_get) that an agent has little reason to make once turn-start injection has already given it the full invocation guide. Skill candidates starve for trials not because any adapter is broken, but because MemOS's own retrieval architecture bypasses its own instrumentation point.Evidence
Ruled out the two obvious "just misconfigured" explanations before concluding this is structural:
adapters/hermes/memos_provider/__init__.pyregistersmemos_skill_getinget_tool_schemas()and callsskill.getwithrecordTrial: Trueinhandle_tool_call(). Not a wiring bug._disabled_retrieval_tools()is gated byMEMOS_RETRIEVAL_TOOLS_DISABLED; verified unset (commented out, not exported) on all three of our production hosts.SkillCandidate(core/retrieval/types.ts) carries the fullinvocationGuide: string, not a pointer/id requiring a follow-up call — confirmed the field is populated end-to-end fromrepos.skillsthrough to the injected context.recordUseandrecordTrialhave exactly one call site between them:getSkill()incore/pipeline/memory-core.ts. Grepped the full file; no other caller of either.Fleet impact (three production hosts, deployed commit)
906 of 922 candidate skills (98.3%) have never had a single trial recorded, over roughly three months of active operation with regular turn-start skill retrieval happening continuously. Skills are being surfaced and used; almost none of that generates a trial.
Suggested Fix
A couple of directions, would appreciate maintainer input on which fits the intended design:
getSkill(..., { recordUse: true })(or a lighter dedicated counter) when aSkillCandidateis actually surfaced into context — at minimum tracking exposure, ideally feeding into trial counting too.trialsAttemptedis meant to track deliberate invocation specifically (as distinct from passive context exposure), that's a reasonable design choice — but it meanscandidateTrialsas currently tuned assumes a usage pattern (explicit skill lookup calls) that doesn't match how skills are actually consumed via turn-start retrieval, and the gate should be reconsidered against that reality.Happy to contribute data or a PR once maintainers weigh in on intended design.