Preface¶

Agent Skills are rapidly gaining popularity: AI programming tools like Cursor, Claude Code, and Codex CLI all support packaging domain knowledge into reusable capabilities using SKILL.md. However, many first-time developers get stuck on a few common pain points: not knowing how to write a description that will trigger the Skill correctly, lacking testing methods for finished Skills, and being unsure if revised versions are actually better after multiple iterations.

Anthropic has published a Skill in their open-source repository anthropics/skills that specifically addresses these issues: skill-creator. It is itself a Skill, but its role is to “teach you how to write Skills, test Skills, and iterate on Skills”. If you plan to get started with Agent Skills systematically, starting here is more efficient than jumping straight into the official documentation.

What is this?¶

skill-creator is an Agent Skill “meta-skill” developed by Anthropic—it does not directly help you write business code, but instead guides you through the complete closed-loop process of Skill development from ideation, drafting, evaluation to optimization.

Its core positioning can be summarized by its official description:

Create new Skills, modify and optimize existing Skills, and measure Skill performance through eval evaluations and benchmark tests. Suitable for creating Skills from scratch, editing and optimizing existing Skills, running eval tests, conducting variance analysis benchmark tests, or optimizing descriptions to improve trigger accuracy.

Agent Skills use the universal SKILL.md format: YAML frontmatter declares metadata, the Markdown body contains operational instructions, and optional resource directories like scripts/, references/, and assets/ can be included. Tools like Claude Code and Cursor load each Skill’s name and description when starting a session, and read the full SKILL.md on demand when a task matches—this is what official documentation calls “Progressive Disclosure”. skill-creator is built around this mechanism to help you write Skills “correctly”, test them “accurately”, and refine them “stably”.

Core Features and Highlights¶

1. Structured Creation Workflow¶

skill-creator breaks Skill development into actionable steps:
1. Capture Intent: Clarify what the Skill does, when it should trigger, what output format it should use, and whether test cases are needed.
2. Interview and Research: Proactively ask about edge cases, dependencies, and success criteria; you can use MCP or web search to reference similar Skills and best practices.
3. Write the SKILL.md: Fill in the name, description, and body instructions according to the specifications.

The description is the key to the triggering mechanism—official recommendations suggest including both “what it does” and “when to use it” in the description, rather than in the body; and make it appropriately “proactive” to counteract the model’s tendency to “undertrigger” (fail to use when it should).

2. Skill Directory Specifications and Writing Guidelines¶

skill-creator includes built-in Skill anatomy and writing patterns. The standard directory structure is as follows:

skill-name/
├── SKILL.md          # Required: frontmatter + instruction body
├── scripts/          # Optional: Executable scripts
├── references/       # Optional: Reference documents loaded on demand
└── assets/           # Optional: Templates, icons, and other output resources

Key principles include:
- Progressive Disclosure: Metadata resides in the context permanently (around 100 tokens), the body loads only when triggered (it is recommended to keep the SKILL.md body under 500 lines), and resource files are read on demand.
- Split references by domain: For multi-framework or multi-cloud scenarios, organize them using forms like references/aws.md to avoid overwhelming the context all at once.
- Explain “why”: Instead of listing rigid MUST/NEVER rules, it is better to explain the reasoning so the model understands the intent and executes flexibly.
- No Surprises Principle: Skill content must not contain malicious code or behavior inconsistent with its description.

3. Eval Evaluation and Benchmark Testing¶

This is the biggest feature that sets skill-creator apart from ordinary documentation. It provides a complete evaluation workflow:

Test Cases: Saved to evals/evals.json, each entry contains a prompt, expected output description, and optional input files:

{
  "skill_name": "example-skill",
  "evals": [
    {
      "id": 1,
      "prompt": "The user's task prompt",
      "expected_output": "Description of the desired result",
      "files": []
    }
  ]
}

Parallel Comparative Runs: For each test case, launch two runs simultaneously—one “with Skill” and one “without Skill (baseline)”. When creating a new Skill, the baseline is no Skill; when improving an existing Skill, the baseline is a snapshot of the old version. Results are organized by iteration directory:

<skill-name>-workspace/
├── iteration-1/
│   ├── eval-0/
│   │   ├── with_skill/outputs/
│   │   └── without_skill/outputs/
│   └── benchmark.json
└── iteration-2/
    ...

Quantifiable Assertions: Write assertions for objectively verifiable outputs (file formats, data extraction, fixed workflow steps); for subjective Skills (writing style, design aesthetics), focus on human review instead.

Benchmark Aggregation: Run the aggregation script to generate pass rate, time consumption, token usage, and mean ± standard deviation:

python -m scripts.aggregate_benchmark <workspace>/iteration-N --skill-name <name>

Visual Review: Launch the browser review interface via eval-viewer/generate_review.py (with two tabs: Outputs and Benchmark) so users can view outputs one by one and submit feedback; without a graphical interface, use the --static flag to generate a standalone HTML file.

4. Description Trigger Optimization¶

Whether a Skill is invoked largely depends on the description in the frontmatter. skill-creator provides a dedicated optimization loop:
1. Generate approximately 20 trigger test queries, including both positive trigger samples and “near-miss distraction” samples that should not trigger.
2. The user reviews the eval set using the assets/eval_review.html template.
3. Run the optimization script (up to 5 iterations, 60% training / 40% held-out test):

python -m scripts.run_loop \
  --eval-set <path-to-trigger-eval.json> \
  --skill-path <path-to-skill> \
  --model <model-id> \
  --max-iterations 5 \
  --verbose

The script will evaluate the trigger rate of the current description, ask Claude to propose improvements, and finally select the best_description with the highest score on the test set.

5. Packaging and Distribution¶

After the Skill is finalized, it can be packaged as a .skill file for easy installation:

python -m scripts.package_skill <path/to/skill-folder>

Installation and Activation¶

skill-creator comes from Anthropic’s open-source repository, so you first need to clone or download the corresponding directory:

git clone https://github.com/anthropics/skills.git
cp -r skills/skills/skill-creator ~/.claude/skills/skill-creator

The installation paths vary slightly across AI programming tools (all take the “Skill directory + SKILL.md” as the minimal unit):

Skills in Claude Code will load automatically when the description matches, or you can call them directly via /skill-creator. In Cursor, you can mention intentions like “create a Skill” or “optimize Skill description” in an Agent conversation, and the Agent will read the corresponding SKILL.md and guide you through the process.

Dependency Notes: The full evaluation workflow (parallel subagents, benchmark aggregation, description optimization loops) offers the most complete experience in the Claude Code environment; Claude.ai does not support subagents, so you will need to run tests serially and skip baseline comparisons; description optimization relies on the claude -p CLI, which is not available on Claude.ai.

Typical Usage Examples¶

Scenario 1: Create a New Skill from Scratch¶

Suppose you want to固化 the stylistic requirements you repeatedly use when writing technical WeChat official account articles into a Skill:
1. Tell the Agent: “I want to create a Skill for writing technical articles for WeChat official accounts, referencing the style in data/style_reference.md.”
2. skill-creator will first conduct an interview: What are the trigger terms? What is the output format? Do you need eval tests?
3. Draft the SKILL.md and generate 2–3 real test prompts to write into evals/evals.json.
4. Run parallel with_skill / without_skill tests, aggregate the benchmark, and open the eval viewer for you to review.
5. Modify the Skill based on feedback.json, enter iteration-2, and repeat until you are satisfied.
6. Optional: Run the description optimization to improve the trigger rate for phrases like “write WeChat official account” or “technical article”.
7. Use package_skill to bundle the Skill and distribute it to your team.

Scenario 2: Optimize the Description of an Existing Skill¶

An existing Skill often fails to trigger when it should? You can focus solely on trigger optimization:
1. Generate 20 should-trigger / should-not-trigger test queries.
2. Have the user review them using the assets/eval_review.html template.
3. Run scripts.run_loop to compare the trigger rates before and after optimization.
4. Write the best_description back to the SKILL.md frontmatter.

Scenario 3: Improve the Body of an Existing Skill¶

If a Skill triggers correctly but has unstable output quality:
1. Take a snapshot of the existing Skill as the baseline.
2. Rerun the eval after modifying SKILL.md.
3. Review the delta in pass rate, tokens, and time consumption in the benchmark.
4. Read the run transcripts: if multiple evals repeatedly write the same helper script, consider moving the script into the scripts/ directory—this is the pattern skill-creator explicitly recommends: “refining resources from repetitive work”.

Applicable Scenarios and Notes¶

Who this is for:
- Developers writing their first Agent Skill who need a standardized starting point
- Teams that want to unify Skill quality and have eval and benchmark capabilities
- Maintainers of existing Skills with inaccurate triggering or unstable output
- AI programming practitioners who want to turn one-off prompts into reusable, testable capabilities

Notes:
1. Evaluation Cost: Full evals will launch multiple parallel subagents, consuming tokens and time; for simple Skills or rapid iterations with users, you can skip some quantifiable processes.
2. Environment Differences: Parallel subagents, benchmark comparisons, and description optimization are most complete in Claude Code; other environments will need to adapt the workflow per the Claude.ai / Cowork adaptation instructions in the skill-creator documentation.
3. Description Design: Test queries should be specific and multi-step; overly simple requests like “read a PDF” may not trigger the Skill—because the model can complete them directly using basic tools.
4. Security Audit: Skills can include scripts and external references, so you should conduct a security review of SKILL.md and scripts/ before installation, and the official documentation also emphasizes only using Skills from trusted sources.
5. Cross-platform Incompatibility: Claude Code file system Skills, API-uploaded Skills, and Claude.ai uploaded Skills do not sync automatically, and you need to manage them separately in each environment.

Summary¶

skill-creator turns Agent Skill development from “writing a Markdown and crossing your fingers” into an engineering practice with standardized processes, testing, benchmarks, and trigger optimization. It is both the “meta-skill” of the Anthropic Skill ecosystem and the best starting point for getting started with Agent Skills—first learn to use it to create and evaluate Skills, then expand to custom Skills for your business domain, and you will avoid many pitfalls.

Official repository: https://github.com/anthropics/skills/tree/main/skills/skill-creator
Agent Skills Overview Documentation: https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview