anthropics/skills · skills/skill-creator/SKILL.md
skill-creator
“Anthropic's skill-maker: a real test loop with baselines and benchmarks, inside a chatty 486-line pep talk”
What it does
Guides the agent through building or improving a skill: interview the user, draft SKILL.md, run test prompts with and without the skill, grade and benchmark the runs, collect feedback in a browser viewer, and iterate. It can also tune the description for triggering and package the result.
- When the agent uses it
- When a user wants to create, edit or improve a skill, run evals or benchmarks on it, or optimize its description so it triggers more reliably.
- Who it is for
- For anyone writing skills in Claude Code or Cowork who wants evidence they work. Skip it if you just want a quick one-off SKILL.md with no testing.
- What it costs
- 327 lines, about 8.9k tokens once the agent loads it, plus 17 files beside it.
Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.
What it tells the agent to do
- Work out where the user is; capture what the skill does, when it triggers, its output format, and whether to test it
- Interview about edge cases and dependencies, then draft SKILL.md with a deliberately pushy description
- Write 2-3 realistic test prompts to evals/evals.json and spawn with-skill and baseline runs in the same turn
- Draft assertions while runs go, save timing.json, grade each run, and aggregate with scripts.aggregate_benchmark
- Open the generate_review.py viewer, read feedback.json, generalize fixes, and rerun into a new iteration
- Optionally build 20 trigger queries and run scripts.run_loop to pick the best description by held-out score
- Package with scripts.package_skill if the present_files tool is available
Lines worth reading
Straight from the file, with why each one matters.
line 67please make the skill descriptions a little bit "pushy
Anthropic admits Claude tends to undertrigger skills and tells you to oversell the description to compensate.
line 219This is the only opportunity to capture this data
A sharp, practical rule: token and timing numbers arrive only in the task notification, so save them right away.
line 451Sorry in advance but I'm gonna go all caps here
The skill tells you to avoid ALWAYS/NEVER in caps (line 302), then shouts in caps itself to get the viewer generated.
line 306we are trying to create billions a year in economic value here!
A motivational aside in the middle of the instructions. It may or may not work on the model, and it costs context either way.
line 30Cool? Cool.
Sets the tone for the whole file: friendly and conversational, and more narrative than a skill needs.
Watch out for
- Medium
Testing spawns two subagents per test case, and description optimization runs many claude -p calls. That can use a lot of tokens with no stated cost estimate.
line 394running each query 3 times to get a reliable trigger rate
- Low
The viewer starts as a detached background server. It keeps running if the agent forgets the later kill step.
line 242> /dev/null 2>&1 &
- Low
The skill picks the newest eval_set file from ~/Downloads by name and date, so it could read the wrong file.
line 371check the Downloads folder for the most recent version
The scorecard
- TriggerSays when to use it9/10
Clear what and when, with request words like create, edit, evals, benchmark, description
- FocusOne clear job7/10
One domain, skill development, but it spans drafting, evals, benchmarking, tuning and packaging
- LeanLight to load5/10
About 8.9k tokens. Asides and per-platform sections could move to reference files
- ConcreteSteps you can follow8/10
Exact commands, JSON schemas, directory layout and clear stop conditions
- SafeSafe to run8/10
Local scripts only. Watch token spend and the background server
What it does well
- Runs baselines (no skill, or the old version) alongside the skill so improvements are measured, not assumed
- The description optimizer splits queries into train and test sets and picks the winner by held-out score
- Tells the agent to bundle helper scripts that every test run ended up rewriting
How it could be better
Move platform sections to a reference file
The Claude.ai and Cowork sections load in every session even though only one platform applies.
Add## Platform notes On Claude.ai or Cowork, read `references/platforms.md` before running tests. It lists which steps to skip and how to collect feedback without a browser.
Cut the narration
Asides about plumbers, billions in value and 'Cool? Cool.' use context and do not change what the agent does.
AddMatch jargon to the user. Explain terms like JSON and assertion unless the user clearly knows them.
Warn about cost before the optimization loop
run_loop makes many claude -p calls, and the user should agree to the spend before it starts.
AddBefore running, tell the user this makes about <N> `claude -p` calls (20 queries × 3 runs × up to 5 iterations) and ask whether to proceed.
In the skill’s folder
Listed, not read: the agent opens these only when SKILL.md sends it there.
LICENSE.txtagents/analyzer.mdagents/comparator.mdagents/grader.mdassets/eval_review.htmleval-viewer/generate_review.pyeval-viewer/viewer.htmlreferences/schemas.mdscripts/__init__.pyscripts/aggregate_benchmark.pyscripts/generate_report.pyscripts/improve_description.pyscripts/package_skill.pyscripts/quick_validate.pyscripts/run_eval.pyscripts/run_loop.pyscripts/utils.py
Kin reviews against Anthropic’s Extend Claude with skills and agentskills.io’s the Agent Skills standard. It reads SKILL.md only, so a script beside it can still do things the file does not say.