Preface¶

Semgrep is a very common static analysis tool in modern security engineering. It uses YAML rules to match code patterns, capable of scanning for injection flaws, dangerous APIs, and also embedding internal team coding standards into CI pipelines. The problem lies in the fact that writing these rules themselves is not easy.

Writing only the half that “catches vulnerabilities” will usually flag safe coding practices as well; writing rules too strictly will miss cases when the calling style changes slightly. Taint mode can track whether untrusted data flows into dangerous functions, but documentation for configuring sources, sinks, and sanitizers, as well as marking test cases, is scattered. If an Agent only generates a YAML file based on impression, it is easy to run into issues like excessive false positives, incomplete test cases, or cramming multiple rules into a single file.

Trail of Bits has turned this process into an Agent Skill: semgrep-rule-creator. It does not run existing rule sets for you, but instead constrains Agents to follow the workflow of “write tests first, inspect the AST, then write rules, optimize after passing tests” to produce rules that can be validated with semgrep --test. Below is an explanation of what it is, how to install it, and how to use it, based on the official repository and the original SKILL.md.

What is this¶

semgrep-rule-creator is a plugin in Trail of Bits’ Security Skill Marketplace (trailofbits/skills), used to create custom Semgrep rules that can detect security vulnerabilities, bug patterns, and code patterns. The plugin author is Maciej Domanski, and the current version in the plugin metadata is 1.2.2. The Skill itself is located at:

https://github.com/trailofbits/skills/tree/main/plugins/semgrep-rule-creator/skills/semgrep-rule-creator

It is based on the universal SKILL.md format, and can be loaded by AI programming tools such as Claude Code, Codex CLI, and Cursor. The official repository positions it as an offering in the Claude Code plugin marketplace; Codex can load the same marketplace via the Claude marketplace compatibility method. The repository is licensed under Creative Commons Attribution-ShareAlike 4.0.

The official README lists the intended use cases:
- Writing Semgrep rules for specific bug patterns
- Writing detection rules for security vulnerabilities in a codebase
- Writing taint mode rules for dataflow analysis
- Writing pattern matching rules for code quality or coding standard checks

Explicitly unsupported scenarios include: running existing Semgrep rule sets, and general-purpose static analysis without custom rules. For the latter, the official recommends the static-analysis Skill from the same marketplace.

The prerequisite is straightforward: Semgrep must be installed locally first.

pip install semgrep
# or
brew install semgrep

Core Capabilities¶

This Skill truly constrains how to write rules rather than wrapping the Semgrep CLI a second time. Comparing the SKILL.md and the plugin README, its capabilities can be summarized as follows:

Mandatory test-first workflow. Write test files with # ruleid: / # ok: annotations first, then write the YAML rules. Only covering vulnerability test cases and omitting safe use cases will be explicitly flagged as an anti-pattern: catching vulnerabilities only completes half the job; safe coding practices must not produce false positives, otherwise the rule will be difficult to trust in production environments.
Inspect the AST before writing patterns. Semgrep matches against abstract syntax trees, not source code strings. foo.bar() and foo().bar look similar, but their parsed AST structures can be completely different. The Skill requires using semgrep --dump-ast to inspect the structure before writing patterns.
Prefer taint mode for dataflow issues. A pure syntax match like eval($X) will catch both eval(user_input) and eval("safe_literal"). Taint mode tracks whether untrusted data actually flows to the sink, which usually drastically reduces false positives for injection-related issues. The official also allows switching between the two approaches: switch back to pattern matching when taint propagation does not work as expected, or switch to taint mode when pattern matching produces too many false positives for safe use cases. The goal is a usable rule, not locking into a single writing style.
One rule per YAML file. The output will always be “a directory named after the rule ID + one YAML file + one test file”, and will not cram multiple rules into the same file.
Require the Agent to pull Semgrep official documentation first before writing rules: rule syntax, pattern syntax, rule testing, taint analysis overview and advanced topics, constant propagation, as well as the Semgrep chapter in the Trail of Bits Testing Handbook. The Skill also includes two local references: references/quick-reference.md (commands, operators, taint syntax) and references/workflow.md (full workflow and examples).

There is also a Claude Code command trailofbits:semgrep-rule built into the plugin, which will call the full Skill workflow based on the vulnerability pattern, target language, and suitability for taint mode in the current conversation; it will ask for clarification if there is insufficient context about what to detect.

Installation and Activation¶

The official installation method from Trail of Bits focuses on the Claude Code plugin.

First add the marketplace, then install this plugin:

/plugin marketplace add trailofbits/skills
/plugin install trailofbits/skills/plugins/semgrep-rule-creator

You can also install by browsing the marketplace via /plugin menu. The plugin name is semgrep-rule-creator.

Codex’s official documentation states that it supports direct loading of the Claude plugin marketplace without additional sidecar metadata:

codex plugin marketplace add trailofbits/skills
codex plugin list
codex plugin add semgrep-rule-creator@trailofbits

If using the universal skills CLI (installation instructions on officialskills.sh), you can use:

npx skills add https://github.com/trailofbits/skills --skill semgrep-rule-creator

After installation, you can directly describe the pattern you want to detect in the conversation, for example “Write a taint rule for Python that captures request.args flowing into eval()”, or use the /semgrep-rule command mentioned above. The Skill declares callable tools as Bash, Read, Write, Edit, Glob, Grep, WebFetch, which allows the Agent to read documentation, modify files, and run Semgrep commands.

Rule Writing Workflow¶

The Skill formalizes the process into a mandatory checklist, marked as strict by the official, and steps cannot be skipped.

Semgrep Rule Progress:
- [ ] Step 1: Analyze the Problem
- [ ] Step 2: Write Tests First
- [ ] Step 3: Analyze AST structure
- [ ] Step 4: Write the rule
- [ ] Step 5: Iterate until all tests pass (semgrep --test)
- [ ] Step 6: Optimize the rule (remove redundancies, re-test)
- [ ] Step 7: Final Run

The requirements for each step are as follows in workflow.md:

Step 1: Analyze the Problem. First read the documentation, then explain the vulnerability or pattern to be detected in a way that can be understood by junior developers, confirm the target language, and then decide whether to use pattern matching or taint mode. Taint mode is suitable for tracking untrusted data across variables and functions, especially for issues like SQL injection, command injection, XSS, and is less likely to be broken by constructs like if statements, loops, etc.

Step 2: Write Tests First. The directory structure must be:

<rule-id>/
├── <rule-id>.yaml     # Semgrep rule
└── <rule-id>.<ext>    # Test file with ruleid/ok annotations

Only ruleid: and ok: annotations are allowed in test cases. todoruleid: and todook: are prohibited, and multi-line comments should not be used for marking. Annotations must be on a separate line immediately above the target code – Semgrep will flag the line immediately following the annotation. Tests must cover: explicit vulnerability cases, explicit safe use cases, edge cases and different writing styles, sanitized/validated inputs, completely unrelated normal code, and nested cases inside if statements, loops, try/catch blocks, and callbacks.

Step 3: Analyze the AST.

semgrep --dump-ast --lang <language> <rule-id>.<ext>

Inspect how function calls are represented, how variables are bound, and how control flow is expanded, to avoid writing patterns that “look correct to the human eye but do not match the AST structure”.

Step 4: Write and validate the rule. The required fields for a rule are clearly listed in the quick-reference: id, languages, severity (LOW / MEDIUM / HIGH / CRITICAL), message, and one of pattern / patterns / pattern-either / mode: taint. After writing the rule:

semgrep --validate --config <rule-id>.yaml
cd <rule-directory>
semgrep --test --config <rule-id>.yaml <rule-id>.<ext>

The expected output is 1/1: ✓ All tests passed. For debugging taint rules, you can use:

semgrep --dataflow-traces --config <rule-id>.yaml <rule-id>.<ext>

This will print the source, sink, dataflow path, and the reason why taint did not propagate.

Step 5: Iterate until all tests pass. “Most tests passing” does not count as completion. Missed detections usually come from patterns that are too strict, requiring pattern-either; false positives usually come from patterns that are too broad, requiring pattern-not or tightening sanitizer rules.

Step 6: Optimize only after all tests pass. Remove redundancies after all tests pass: quote variants, subsets covered by ..., similar calls that can be merged with metavariable-regex. You must re-run the tests after each change, because some patterns that look redundant actually correspond to different AST structures.

Step 7: Final Verification. Run the rule for real with semgrep --config. The message should be short and explain what was matched, and must not leave un-interpolated metavariables (such as $OP, $VAR) – any metavariable mentioned in the message must be captured by the pattern.

Official Example: eval Taint Rule¶

The Quick Start in SKILL.md provides a Python rule that detects user input entering eval(). The rule file can be named insecure-eval.yaml:

rules:
  - id: insecure-eval
    languages: [python]
    severity: HIGH
    message: User input passed to eval() allows code execution
    mode: taint
    pattern-sources:
      - pattern: request.args.get(...)
    pattern-sinks:
      - pattern: eval(...)

The corresponding test file insecure-eval.py:

# ruleid: insecure-eval
eval(request.args.get('code'))

# ok: insecure-eval
eval("print('safe')")

Run semgrep --test in the rule directory. This rule uses taint mode instead of pattern: eval(...), so literal calls can be marked as ok and will not be flagged as vulnerabilities.

The official also lists several writing patterns to avoid, which align with the workflow above.

Overly broad patterns provide no actual detection capability:

# BAD: matches any function call
pattern: $FUNC(...)

# GOOD: targets specific dangerous functions
pattern: eval(...)

Tests only cover vulnerabilities and omit safe use cases:

# BAD: only vulnerability test cases
# ruleid: my-rule
dangerous(user_input)

# GOOD: include safe test cases to block false positives
# ruleid: my-rule
dangerous(user_input)

# ok: my-rule
dangerous(sanitize(user_input))

# ok: my-rule
dangerous("hardcoded_safe_value")

Overly strict patterns will miss cases when the splicing style changes; dataflow issues should use taint mode instead:

# BAD: only matches this specific string concatenation pattern
pattern: os.system("rm " + $VAR)

# GOOD: track whether input flows into os.system
mode: taint
pattern-sources:
  - pattern: input(...)
pattern-sinks:
  - pattern: os.system(...)

Taint rules can also add pattern-sanitizers to mark functions like sanitize(...) and escape(...) as sanitization points. The quick-reference also lists options like exact and by-side-effect to control whether “the entire call counts as a source / only a specific parameter counts as a source”. These details are subject to the Semgrep documentation and the Skill’s built-in quick-reference; let the Agent read these materials when writing rules, do not rely on memory.

Use Cases and Notes¶

Typical use cases listed on the official and Skill index pages include:
- Writing taint rules to detect user-controlled request parameters flowing into SQL execution points
- Catching eval() / exec() receiving untrusted input in Python code
- Marking deprecated APIs in large repositories for coding compliance
- Adding a regression detection rule after discovering a specific vulnerability during a security audit
- Turning repository-specific bug patterns into custom rules for CI pipelines

There are two other Skills in the same marketplace that are often used together: semgrep-rule-variant-creator is used to port existing rules to another language; variant-analysis is used to find similar vulnerabilities in a codebase. If you only need to run scans and view SARIF results instead of writing new rules, you should use the static-analysis Skill instead.

There are several hard constraints for usage; violating them will produce rules that cannot be used in production:
- Semgrep must be installed first; the Skill itself does not replace the scanner
- Tests must pass 100%; optimization can only happen after all tests pass
- Do not use languages: generic for generalized matching for target languages
- One YAML file only contains one rule
- Test annotations must not contain other text, and do not use todoruleid / todook as placeholders for “fix later”
- Metavariables must be uppercase, such as $X, $FUNC, and must not be written as $x

In addition, this Skill will not help you select existing rule sets like p/security-audit or p/trailofbits, nor will it configure CI pipelines for you. Its output is “a tested custom rule”; actual repository scanning and CI integration still require using Semgrep’s own commands and configurations.

Summary¶

The most common failures when writing Semgrep rules manually are not syntax errors in YAML, but patterns that are too broad or too strict, and not using safe use cases to block false positives. semgrep-rule-creator formalizes Trail of Bits’ rule writing habits into a checklist that Agents must follow: write tests first, inspect the AST, prioritize taint mode for dataflow issues, optimize only after passing tests, and one rule per file.

Official links:
- Skill directory: https://github.com/trailofbits/skills/tree/main/plugins/semgrep-rule-creator/skills/semgrep-rule-creator
- Plugin documentation: https://github.com/trailofbits/skills/tree/main/plugins/semgrep-rule-creator
- Skill marketplace: https://github.com/trailofbits/skills
- Index page: https://officialskills.sh/trailofbits/skills/semgrep-rule-creator

Semgrep-Rule-Creator: Using Agent to Write Testable Semgrep Security Rules