Skip to content

Report first-party skill names on skill_used, bucket the rest as custom - #959

Merged
TheGreatAxios merged 3 commits into
mainfrom
cl-7326-skill-used-allowlist
Sep 13, 2026
Merged

TheGreatAxios merged 3 commits into
mainfrom
cl-7326-skill-used-allowlist

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Collaborator

Summary

  • skill_used carries skill_name: a first-party corbits-skills name from a closed 16-name allowlist, or custom for unknown, project-local, and plugin-authored skills
  • TELEMETRY.md documents the allowlist and drops the no-name claim; the event allowlist carries skill_name and nothing else identifying

Verification

  • bun run typecheck, bun run build, and bun run check pass (7134 pass, 0 fail)
  • Classification unit tests pin the 16-name set and cover the employer-named-skill leak case via the serialized wire payload

Fixes CL-7326

@linear-code

linear-code Bot commented Sep 13, 2026

Copy link
Copy Markdown

CL-7326

@TheGreatAxios

Copy link
Copy Markdown
Collaborator Author

Audit: the data structure is fine — closed allowlist + custom bucket, skill_name on the event, captureSkillUsed at the use_skill boundary. The rationale is wrong and must be rewritten before merge:

The classify.ts comment and TELEMETRY.md both claim the allowlist covers "user-invocable" skills and that bundled skills marked user-invocable: false (native-integration, native-runtime, ponytail, idiot-proof — 4 names) are "deliberately excluded". Ground truth on main: 11 skills carry user-invocable: false, and 7 of the names in this allowlist (git-rebase, git-worktrees, linear-issue-workflow, opsh, philosophy, style, typescript) carry that flag. So the comment describes a set that does not match the code. Correct framing: first-party corbits-skills names are reportable by name because they are our own strings (nothing user-authored leaks), and everything else buckets to custom. Please also document why slash-opt-out is not telemetry-opt-out: user-invocable: false means "no slash command", not "must not be named in telemetry" — the privacy property comes from first-party authorship, not from invocability.

@TheGreatAxios

Copy link
Copy Markdown
Collaborator Author

Audit (fleet builder): rationale rewrite — data structure stands, words were wrong.

Ground truth confirmed: 11 bundled skills carry user-invocable: false. Seven of them (git-rebase, git-worktrees, linear-issue-workflow, opsh, philosophy, style, typescript) are real use_skill recipes that merely opt out of slash synthesis (PRODUCT.md: they stay use_skill only), so they belong on the telemetry allowlist — slash-opt-out is not telemetry-opt-out. Only the four bake-only background skills (idiot-proof, native-integration, native-runtime, ponytail) are excluded, because they are baked into agent prompts rather than invoked as skills. Fix pushed: classify.ts comment + TELEMETRY.md now say first-party-skills-reportable-by-name with the 7/4 reconciliation spelled out (was: wrong user-invocable:false-exclusion story covering only 4). Test pins all four bake-only names to custom. Allowlist contents unchanged.

user-invocable false opts out of slash synthesis, not name reporting: seven flagged skills stay allowlisted as use_skill recipes; only the four bake-only background skills are excluded.
@TheGreatAxios
TheGreatAxios merged commit 862fdc8 into main Sep 13, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant