Web Tools Strategy Guide
Paste the following prompt into your AI chat to install this skill:
Please follow the official guide at https://skillhub.cn/install/skillhub.md to install @user_ec205dbb/web-tools-guide.
About this skill
Problem Background
In automated Web data acquisition tasks, developers often face tool selection dilemmas: when to use search engines, direct fetching, command-line tools, or browsers? Incorrect choices can lead to failed requests, missing content, or inefficiency. This skill provides a scenario-based decision strategy to reliably handle Web content in knowledge management.
Core Capabilities and Key Steps
This skill follows the ReAct paradigm, treating four tools as branch decision points rather than a hierarchical structure:
web_search: Used when no clear URL is available, for searching information like news or resources. Failures are handled per the flow.web_fetch: Used for known URLs with static content, such as articles or API endpoints. If errors occur, downgrade to other tools.opencli: First fallback afterweb_searchorweb_fetchfailure, covering 70+ mainstream websites and providing structured data.browser: Last resort for JS-rendered pages, login states, or interactive operations, but should be used cautiously.
Decision Flow:
1. Choose web_search or web_fetch based on the scenario.
2. On failure, first try opencli.
3. If still failing, escalate to browser.
4. Inform the user of the reason for each tool switch, avoiding silent degradation.
Key Operational Guidelines:
- Login Operations: Must obtain user authorization; use well-known-sites.json to find login page URLs.
- Page Interactions: Prefer CSS selectors over XPath, with smart waits for element appearance, avoiding fixed sleep.
- Failure Handling: For web_search failures, guide users to configure API keys or degrade to alternative solutions.
Applicability Boundaries and Considerations
This skill is suitable for standard Web data acquisition but has limitations:
- Tool Limitations: opencli requires prior installation and covers mainstream websites but isn't universal; the browser tool is heavyweight and should be the last choice.
- Security Considerations: Logins and sensitive operations need secondary confirmation; CAPTCHAs may not be automated.
- Performance and State: Reuse browser contexts in multi-step operations to avoid reopening; set reasonable thresholds for page load timeouts.
By following this strategy, developers can systematically handle Web data tasks, reducing errors and manual interventions.
Use Cases
- When batch downloading technical documentation from multiple static blog sites and parsing content, use web_fetch tool to fetch by URL directly, switching to opencli on failure.
- While monitoring social media hot topics, use web_search to search keywords for related URLs, then follow the decision flow to select subsequent tools for deeper data acquisition.
- When accessing dynamic admin dashboards (like CMS systems) requiring login, use the browser tool to navigate, fill credentials, and perform data export operations.
- For competitive analysis projects, extract product pricing information from e-commerce websites, first trying web_fetch for page fetching, and downgrading to browser tool if encountering JS rendering issues.
Best For
- Data Analysts: Need to regularly collect market data from web sources and ensure the fetching process is automated to support analysis reports.
- Content Editors: Responsible for monitoring industry news websites, promptly extracting article content to update internal knowledge bases.
- Researchers: Batch extract literature materials on specific topics from academic sites or blogs for project research.
- Operations Specialists: Need to access multiple platform backends to perform data exports, but require automated handling when encountering dynamic pages or login restrictions.
Related Skills
A unified real-time search service supporting general web search, vertical domain queries, parallel batch search, and web page content extraction.
A core ETL engine for RAG/knowledge base pipelines that decomposes long documents into semantically complete, retrieval-ready atomic slices.
A smart converter that transforms various document formats into Markdown, with automatic dependency management, fallback conversion, and output cleaning.
A RAG-oriented assistant for bidding data, supporting retrieval, summarization, and structuring of bid announcements across all industries nationwide.