feat(ai): support project-defined skills (skills/*.md) for AI agents - #9851
feat(ai): support project-defined skills (skills/*.md) for AI agents#9851nishantmonu51 wants to merge 7 commits into
skills/*.md) for AI agents#9851Conversation
Skills are markdown files at skills/<name>.md or skills/<name>/SKILL.md with YAML front matter (description required; optional name, metrics_views, agents, always_apply) that teach Rill's AI agents project-specific practices such as analysis playbooks and business glossaries. Runtime: - New session-scoped loader in runtime/ai reads skills from the repo; malformed files are reported as issues and logged, never failing the session. - New list_skills and load_skill tools gated on UseAI only, so cloud viewers without repo access can use them; they are exposed on the MCP server automatically for external clients. - The analyst agent's prompt gains an index of relevant skills (scope-filtered by the dashboard's metrics views) and inlines always_apply skill bodies after ai_instructions, capped at 32kb. - The MCP server instructions tell clients to discover and load skills. - instructions.ParseFrontMatter is exported and shared with the embedded instructions. Frontend: - load_skill/list_skills render in the chat thinking trace with a skill icon. - Add > More gains an "AI Skill" entry that creates skills/my_skill.md from a starter template; skill files get a dedicated icon in the file explorer. Claude-Session: https://claude.ai/code/session_017udazBgdXmdTXMTq7sTh2L
…ctions for external clients Piggybacks always_apply skill bodies on the ai_instructions field that list_metrics_views returns to external MCP clients (restored on main), so clients receive glossary-style skills on their first call without relying on the server instructions to load them. Internal rill sessions are excluded since their prompts already inline these skills. Claude-Session: https://claude.ai/code/session_017udazBgdXmdTXMTq7sTh2L
The soft "check whether any skill matches" wording let the model skip load_skill on questions a skill clearly covered; instruct it to load a matching skill before running any queries. Claude-Session: https://claude.ai/code/session_017udazBgdXmdTXMTq7sTh2L
The analyst prompt already injects nothing when a project defines no skills; apply the same to the per-session MCP server, which advertised a static skills section regardless. The exported MCPInstructions keeps the full text for the unified admin MCP server, which cannot tailor per project. Claude-Session: https://claude.ai/code/session_017udazBgdXmdTXMTq7sTh2L
| - `name:` (optional) overrides the name derived from the file path | ||
| - `metrics_views:` (optional) list of metrics view names; the skill is only offered when the analysis involves one of them | ||
| - `agents:` (optional) list of agents the skill applies to, `analyst` and/or `developer`; defaults to `[analyst]` | ||
| - `always_apply:` (optional) if `true`, the skill's full body is always injected into the agent's context instead of being loaded on demand; use for short, broadly applicable guidance such as glossaries |
There was a problem hiding this comment.
This seems like its slightly different from the agent skills standard. Consider making it fully compatible with the standard: https://agentskills.io/specification
Also, would it make sense to load from the generic .agents/skills instead of skills? For compatibility with other chat clients like Claude.
There was a problem hiding this comment.
Done in 0db9c94: skills now follow the Agent Skills spec (<name>/SKILL.md, name must match the directory, standard optional fields accepted; Rill's metrics_views/agents/always_apply are extension fields). Also loaded from .agents/skills/.
| func loadSkills(ctx context.Context, rt *runtime.Runtime, instanceID string) ([]*Skill, []SkillIssue, error) { | ||
| repo, release, err := rt.Repo(ctx, instanceID) | ||
| if err != nil { | ||
| return nil, nil, fmt.Errorf("failed to open repo: %w", err) | ||
| } | ||
| defer release() | ||
|
|
||
| entries, err := repo.ListGlob(ctx, skillsGlob, true) | ||
| if err != nil { | ||
| return nil, nil, fmt.Errorf("failed to list skill files: %w", err) | ||
| } |
There was a problem hiding this comment.
It's best to avoid reading files directly from server code. The repo may be unavailable (e.g. Github outage, disconnected repo, slow clone due to large files, etc.), and we don't want it to cause APIs to time out or error. The only code that normally reads from the repo directly is the reconcilers, which then write state into the catalog; then API code normally only hits the catalog to stay reliable.
So a better option may be to parse skills in runtime/parser, and write them into the catalog in e.g. runtime/reconcilers/project_parser.go, and then read from the catalog here.
There was a problem hiding this comment.
Done in 0db9c94: skills are parsed in runtime/parser into a new Skill resource, written to the catalog by the project parser reconciler, and the AI code reads only from the catalog.
| } | ||
|
|
||
| var fm skillFrontMatter | ||
| body, err := instructions.ParseFrontMatter([]byte(content), &fm) |
There was a problem hiding this comment.
The instructions directly is specifically made for instructions in runtime/ai/instructions/data, maybe consider having separate/dedicated code for the user-facing skills parsing so the two use cases don't get too coupled.
There was a problem hiding this comment.
Done in 0db9c94: skill parsing lives in runtime/parser/parse_skill.go with no dependency on runtime/ai/instructions; the ParseFrontMatter export is reverted.
… Skills format Addresses PR review feedback: - Skills are now parsed in runtime/parser into a new Skill resource kind, inserted into the catalog by the project parser, and read from the catalog by the AI session, so API code no longer reads the repo directly. - The file format follows the Agent Skills standard (https://agentskills.io): a skill is a directory with a SKILL.md file, the name must match the directory, and the standard optional fields are accepted. Rill's metrics_views/agents/always_apply remain as extension fields that other clients ignore. Skills are also loaded from .agents/skills/ for cross-client compatibility. - Skill parsing no longer depends on the runtime/ai/instructions package; its ParseFrontMatter export is reverted. - Invalid skill files now surface as parse errors on the file, and a skill scoped to a nonexistent metrics view gets a reconcile error. Claude-Session: https://claude.ai/code/session_017udazBgdXmdTXMTq7sTh2L
- Fix `TestAnalystSkills` to use the `skills/<name>/SKILL.md` layout, which the flat `skills/<name>.md` paths stopped matching after the Agent Skills refactor. - Forward `list_skills` and `load_skill` on the Cloud unified MCP server, which advertised the skills instructions without exposing the tools they reference. - Generate unique skill directory names with hyphens instead of post-processing `getName`'s `_N` suffix, which could produce an existing name and silently fail the create. - List skills with a dedicated `**/SKILL.md` glob so unrelated markdown files don't count against `drivers.RepoListLimit` and fail the whole parse. - Wire skills into the developer agent, so `agents: [developer]` is no longer inert, and share the prompt-building logic with the analyst agent. - Filter skills by agent in `list_metrics_views`, so developer-only skills don't leak into the `ai_instructions` served to external clients. - Require `UseAI` to access a skill resource, and note in the docs that this includes anonymous visitors on public projects. - Report empty skill front matter as a missing `description` instead of an unclosed delimiter. - Resolve the skills instructions during the MCP initialization handshake instead of on every stateless request. Claude-Session: https://claude.ai/code/session_01WTWACZUSP88ypUfVb8FbJr
nishantmonu51
added
Type:Feature
2 participants
Adds skills: instruction files that teach Rill's AI agents project-specific practices, such as root-cause-analysis playbooks and business glossaries. This is the inbound counterpart of the SKILL.md format
runtime/ai/instructionsalready emits for Claude/Cursor.SKILL.mdfile atskills/<name>/SKILL.md(also loaded from.agents/skills/for cross-client compatibility), with front mattername(must match the directory) anddescription(drives selection). Rill adds extension fieldsmetrics_views(relevance scoping),agents, andalways_apply, which other clients ignore.runtime/parserinto a newSkillresource kind and inserted into the catalog by the project parser, so the AI session reads them from the catalog rather than the repo. Invalid skill files surface as parse errors on the file, and a skill scoped to a nonexistent metrics view gets a reconcile error.load_skilltool on demand;always_applybodies are inlined afterai_instructions(32kb cap).list_skills/load_skilltools are gated onUseAIonly, so cloud viewers withoutReadRepocan use them; both are exposed on the MCP server automatically, and the server instructions tell external clients to discover and load skills.load_skillin the thinking trace ("Loading skill..." / "Loaded skill" labels ship server-side via tool meta); Add → More gains an "AI Skill" entry with a starter template, and skill files get a dedicated icon in the file explorer.ai-configuration.md(format, scoping,always_applyvsai_instructions, and a warning that skill content is visible to all AI-enabled members) plus MCP guide updates.TestAnalystSkillseval assertingload_skillis called and the playbook is followed.Checklist:
https://claude.ai/code/session_017udazBgdXmdTXMTq7sTh2L