Preface¶

When connecting web data to applications, it’s easy to conflate “searching” and “scraping” into a single step. A user might ask, “What pricing changes has my competitor made recently?” or “How do I configure retries for this library now?” and they don’t have a ready-made URL. If you directly call the scraping interface at this point, the workflow will get stuck on “no address yet yet”; conversely, if you generate an answer using only search summaries, the citations often won’t match the actual content.

Searching and scraping are two distinct data pipelines. The former starts from a query and is responsible for discovery, ranking, and sourcing; the latter starts from a known URL and is responsible for converting pages into Markdown or structured fields. Firecrawl’s official documentation formalizes this distinction into a set of Agent Skills: the main entry point is firecrawl-build, while the narrow skill dedicated to “query-first” scenarios is firecrawl-build-search.

What it is¶

One-sentence positioning: firecrawl-build-search guides AI coding assistants to connect Firecrawl’s /search endpoint into production code — the functionality starts with a query, not a URL; you can supplement the results with page content in the same call (the official documentation calls this scraping search results via scrapeOptions, which corresponds to “optional hydrate” in the skill description).

It is maintained by Firecrawl, and its source code is in the skills/firecrawl-build-search directory of the firecrawl/skills repository. The frontmatter of SKILL.md specifies:
- name: firecrawl-build-search
- version: 0.1.0
- license: ISC
- author: firecrawl
- homepage: https://www.firecrawl.dev

Currently, the directory only contains this single SKILL.md file, with no attached scripts or references/ folder. It does not replace the API manual; instead, it tells the Agent when to use /search, when not to use it, and which language-specific documentation to read during integration.

You need to distinguish between the different skills in the same ecosystem to avoid installing or using them incorrectly:

The official README states it clearly: use the CLI skills for tasks like “help me search/scrape this now” in a one-off session; use the build skills for tasks like “add Firecrawl to this codebase”. firecrawl-build-search falls into the latter category.

Core Features and Highlights¶

Based on the official SKILL.md, the repository README, and the authoritative sources of truth: the Search documentation and Node.js source documentation, the capabilities can be summarized as follows.

1. Trigger Condition: Start with a Query, Not a URL¶

The skill requires using /search in these scenarios:
- The user asks a question, and the product must first discover the source material
- The feature requires up-to-date web results
- You need to turn a search term into a shortlist of pages that can be scraped later

In short: when URL discovery is part of the product behavior, use /search first. If you already have a specific address, skip this step and use firecrawl-build-scrape instead.

2. Search and Extraction Are Separated by Default, Hydration is Optional¶

There are three default recommendations:
1. When URL discovery is part of the product behavior, call /search first
2. Separate search and extraction conceptually, unless you explicitly need to scrape the search result pages themselves
3. When cost and latency are sensitive, prioritize “pick URLs first, then selectively extract” instead of broadly hydrating all search results

Corresponding to the API: when you do not include scrapeOptions, /search returns titles, descriptions, and URLs; only when you add scrapeOptions (for example, formats: ["markdown"]) will the response include the page content in the same call. The official Search documentation describes these two approaches as:
- One-step: Include scrapeOptions in the search request, suitable for scenarios where you need full content for every result
- Two-step: First run a search, filter the results, then call /scrape on the selected URLs, suitable for scenarios where you need filtering, sorting, and cost control

The skill explicitly favors the two-step approach unless the product truly requires full page content.

3. Treat /search as Discovery, Ranking, and Source Selection¶

The implementation notes require the Agent to:
- Treat /search as discovery, ranking, and source selection
- Clarify whether the product needs snippets, a list of URLs, or full page content
- Maintain a stable query contract so that subsequent scraping logic can be predictable

Common product patterns (all from the skill, not custom examples) include: generating answers with citations, discovering companies/competitors/topics, research workflows that first generate a shortlist of web pages before diving deeper, and pipelines that take a query to a URL and then pass it to /scrape or /interact. The “research workflow” here refers to discovering web pages; searching for academic papers is a separate use case, see the escalation rules below.

4. Hardcoded Escalation Rules to Prevent Misuse of Indexes¶

The skill outlines several easy-to-confuse escalation paths:
- If you already have a URL → use firecrawl-build-scrape
- If you need to click or fill out forms on the result page → use firecrawl-build-interact
- If you are searching for published academic papers (biomedical/clinical/life sciences literature, PubMed, bioRxiv, medRxiv, or arXiv preprints) → use firecrawl-research-index. Passing categories: ["research"] to /search will not target the academic paper index: it only restricts regular web searches to research-focused sites (including PubMed, bioRxiv, medRxiv, arXiv, and publisher websites), and returns page results without abstract search, related paper expansion, or full-text paragraphs
- If you need to answer development questions from Issues, PRs, READMEs, or documentation pages → use firecrawl-developer-index. The categories: ["developer"] tag has the same limitation: it is not a dedicated index

The Search feature documentation aligns with this rule: research is a website filter, not an academic paper repository.

5. Specific API Call Details Are Defined by Language-Specific Documentation¶

The skill does not embed an SDK parameter list, but instead requires you to read the corresponding language source-of-truth page before writing integration code:
- Node.js / TypeScript: https://docs.firecrawl.dev/agent-source-of-truth/node
- Python: https://docs.firecrawl.dev/agent-source-of-truth/python
- Rust / Java / Elixir / cURL: Replace the language name in the above path to access the corresponding page

These pages are the authoritative source for method names, parameters, and return values. The skill is responsible for explaining “when and why”; the “how to call” details are defined by the official documentation.

Installation and Activation¶

The repository README states that these build skills follow the Agent Skills format and are provided as plugins for Claude Code (.claude-plugin/), Cursor (.cursor-plugin/), and OpenAI Codex (.codex-plugin/). In the universal SKILL.md format, other coding assistants that can discover skill directories can also use them; the specific installation paths for each tool depend on the installer’s output, and will not be guessed here.

The official installation methods are listed from broad to narrow.

1. Install both CLI skills and build skills (including this skill) in one go

npx -y firecrawl-cli@latest init --all --browser

The --all flag will install both the CLI and build skills; the --browser flag will open a browser to complete Firecrawl login. After installation, you need to restart the Agent for it to detect the new skill.

2. Install only the build skills repository

npx skills add firecrawl/skills

3. Install only the firecrawl-build-search skill

The command from officialskills.sh is:

npx skills add https://github.com/firecrawl/skills --skill firecrawl-build-search

You can also paste the GitHub directory link directly into the coding assistant, which will follow the Agent Skills installation process.

For product-side authentication, the skill’s inputs lists two required/optional items:
- FIRECRAWL_API_KEY (required): For requests to the managed service. You can obtain it at firecrawl.dev/app, and store it in a .env file or runtime environment variables
- FIRECRAWL_API_URL (optional): The base URL for self-hosted Firecrawl instances; only set this if you are not using the managed api.firecrawl.dev service

Without an API key, the official recommendation is to first use the firecrawl-build-onboarding skill from the same repository, which includes a browser-based authentication flow. It is worth noting that the Search documentation mentions “you can try it without an API key, adding the key will increase rate limits”; however, the build skill marks the API key as a required item for production integration, and you should follow the skill’s requirements for production code scenarios.

SDK installation details are defined by the language-specific documentation, for example:

npm install firecrawl

pip install firecrawl-py

The authentication code from the Node.js documentation:

import { Firecrawl } from "firecrawl";

const client = new Firecrawl({
  apiKey: process.env.FIRECRAWL_API_KEY,
  // apiUrl: "https://api.firecrawl.dev" // Optional; can also read from FIRECRAWL_API_URL
});

The corresponding Python code is Firecrawl(api_key=os.environ.get("FIRECRAWL_API_KEY")), and pass api_url when using a self-hosted instance.

Typical Usage Examples¶

The following prompts and code snippets are all sourced from the official skill or the source-of-truth/Search documentation, and can be reproduced according to your project’s language. First, ask the Agent to read the correct documentation before writing integration code.

1. Trigger this skill in Cursor / Claude Code / Codex

You can use phrasing close to the official description, for example:

This feature starts with the user's natural language question and does not have a ready-made URL.
Please use firecrawl-build-search to connect Firecrawl's /search endpoint to the existing backend:
First discover sources based on the query and generate a shortlist of pages;
Do not hydrate all results with full text upfront, as we are sensitive to cost and latency. Pick the URLs first and then selectively scrape them.

If you already have a specific URL, you should instead say “use firecrawl-build-scrape to scrape this page” to prevent the Agent from incorrectly using the search flow.

2. Only discover sources: titles, summaries, URLs

Node.js (source-of-truth):

const results = await client.search("site:docs.firecrawl.dev webhook retries");
for (const item of results.web ?? []) {
  console.log(item.url, item.title);
}

Python:

results = client.search("site:docs.firecrawl.dev webhook retries")
for item in results.web or []:
    print(getattr(item, "url", None), getattr(item, "title", None))

REST (from the Search documentation, POST /v2/search):

curl -s -X POST "https://api.firecrawl.dev/v2/search" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -d '{
    "query": "firecrawl",
    "limit": 3
  }'

The official repeatedly reminds: The SDK’s search() method does not return { data: [...] }. Web results are in result.web, news results in result.news, and image results in result.images. Do not attempt to read result.data. In the full cURL JSON response, when not hydrated, data is an object grouped by web/news/images.

3. Supplement results with full content in the same call (hydration)

Suitable for scenarios where you need Markdown for every search hit. Python Search documentation example:

results = firecrawl.search(
    "firecrawl web scraping",
    limit=3,
    scrape_options={
        "formats": ["markdown", "links"]
    }
)

Node.js:

const results = await firecrawl.search("firecrawl", {
  limit: 3,
  scrapeOptions: { formats: ["markdown"] }
});

When you include scrapeOptions, search hits will be converted from lightweight result objects to Document objects with full content. Most options from the /scrape endpoint can be passed to /search via scrapeOptions (the Search documentation states that except for a few capabilities like Change-Tracking, scrape options work for search requests).

4. Two-step flow: first generate a shortlist, then selectively scrape (the skill’s default recommended approach)

Two-step implementation from the Search documentation:

results = firecrawl.search("firecrawl web scraping", limit=5)

for item in results.web or []:
    page = firecrawl.scrape(item.url, formats=["markdown"])
    print(page.markdown[:200])

In production code, you typically add filtering between the two steps: filter by domain, title, or use ignoreInvalidURLs: true to discard addresses that subsequent scraping endpoints cannot handle. Complex Node.js examples may also include parameters like sources: ["web", "news"], time filtering via tbs, and localization via location — always refer to the current language documentation, and do not copy fields from outdated blog posts.

The limit parameter works per source type when multiple sources are specified: with limit: 5 and sources: ["web", "news"], you will get a maximum of 5 web results + 5 news results. If you need different limits or different scrapeOptions for different sources, the official requirement is to split the calls into multiple separate requests.

Applicable Scenarios and Notes¶

Recommended for
- Integrating “web search” as a single tool call in an Agent workflow, starting from a user’s question
- Research, competitor tracking, and citation-augmented question answering: first get a sorted list of pages, then decide which ones to scrape
- Pipelines that take a query to a URL and pass it downstream to /scrape or /interact
- Scenarios where you explicitly need “search results with full text”, use scrapeOptions to retrieve Markdown/HTML/links in one call

Not recommended, or should use other skills
- One-time temporary searches or scrapes in a current session: use firecrawl/cli instead of writing integration code in your business repository
- You already have a URL: use firecrawl-build-scrape
- You need to click or fill out forms to access page content: first scrape and then use firecrawl-build-interact
- You are searching for academic paper records instead of web pages: use firecrawl-research-index, do not rely on categories: ["research"]
- Answering development questions from repository Issues/PRs/READMEs: use firecrawl-developer-index

Usage restrictions
- This is an integration guide, not a complete copy of the API. Parameters and return values are subject to the language-specific pages on docs.firecrawl.dev; the skill version is still 0.1.0, and the repository README notes that evaluations were intentionally delayed in the first round
- Broad hydration will increase credit usage and latency; the skill’s default recommendation is to selectively follow up with extraction
- Keep query strings stable (including constraints like site:) so that downstream scraping logic can be tested and cached reliably
- The Node.js SDK documentation specifies a minimum requirement of Node.js >= 22; package names are subject to the current source-of-truth documentation (Node.js uses firecrawl, Python uses firecrawl-py)
- Installation counts and security scan scores on third-party directory