Introduction¶
When performing web searches in DeepSeek Harness (DSH), common practices involve temporarily connecting a search API or having the model directly fetch pages. The former often covers only a single engine with no fallback; the latter is unstable for dynamic pages and login-required sites, and lacks caching for repeated queries. If you wish to consolidate “multi-engine search, platform retrieval, content extraction, site-specific rules, and Playwright rendering” into a toolset that the model can call directly, along with SQLite persistence and history replay, you may want to look at the community plugin dsh-web-search-pro (maintainer anweat, GitHub stars ~41, category: web tools).
What Is It¶
dsh-web-search-pro is a DSH bundle plugin: it packages multi-engine web search, 20 platform searches, readable fetching, persistent caching, site-specific extraction rules, and Playwright rendering into 11 tools. The routing control plane borrows ideas from Agent-Reach’s backend probing, sequential selection, and failure cooldown, with the core logic implemented natively in TypeScript for this project. Licensed under MIT.
It is used in tandem with @anweat/dsh-browser: browser automation, OpenCLI, approval policies, and crawl budgets are handled by dsh-browser; Web Search Pro manages the search backend and cache layer. Both plugins must be direct dependencies of the profile—DSH only activates bundle layers for direct dependencies, and standard profiles may set autoInstallPeers: false. Do not rely on peer dependencies being automatically installed if you only install the search plugin.
Core Features¶
Search and Fetching¶
| Tool | Purpose |
|---|---|
web_search_pro |
Multi-engine search + RRF fusion + in-memory/SQLite dual-layer caching + history |
web_exa_contents |
Native Exa /contents batch content fetching (1–100 URLs) |
web_fetch_pro |
Readable fetching (Jina → HTTP+rule extraction → Playwright fallback) + snapshot caching |
web_platform_search |
20 platforms: GitHub/Bilibili/YouTube/V2EX/Xiaohongshu/Twitter/Reddit/IG/FB/RSS, plus Zhihu/Weibo/Douban/Tieba/Douyin/Kuaishou, etc. |
The default engine order is ddg, bing, exa, seam, jina (free engines first), with automatic fallback and short cooldown on failure; you can also specify a single engine or multi for parallel fusion. When an Exa API Key is available, it uses the native client; if only an Exa MCP connection exists, it falls back via mcporter (advanced filtering and web_exa_contents still require EXA_API_KEY).
Caching, History, and Rules¶
| Tool | Purpose |
|---|---|
web_history / web_cache_clear / web_search_stats |
Persistent history, cache clearing, storage statistics |
web_rule |
Persistent site-specific extraction rules (list/upsert/remove/import/export) |
web_backend_status |
Backend probing, failure/cooldown diagnostics, and CLI status |
web_deps |
Detect/install external dependencies for search backends |
web_snapshot |
Playwright HTML + text persistence; PNG not generated when screenshot=false |
Caching uses SQLite + LRU; different queries, engine orders, and single/multi-engine modes use distinct cache fingerprints. web_history supports filtering by kind/query/engine/platform, replay, and JSON export.
Platforms and Chinese Community Login States¶
Sites like Zhihu, Weibo, Douban, Tieba, Douyin, and Kuaishou often have anti-scraping measures on their public APIs without login; the plugin uses a Playwright-driven browser with login state for these platforms (an independent implementation inspired by MediaCrawler). After logging in, storageState is saved per domain via authProfiles and browserBindings; selectors can be adjusted with platformRules or dsh-browser’s rulePacks when sites change.
OpenCLI platforms like Reddit, Xiaohongshu, and Twitter require the Chrome extension to be online; platforms like Bilibili and YouTube require installing CLI tools such as bili-cli or yt-dlp as needed, which can be detected and installed via web_deps.
Installation and Enabling¶
Below are the installation methods from the official README. dsh-web-search-pro >= 0.1.8 requires @anweat/dsh-browser >= 0.1.8; when upgrading, both packages should be updated together. After upgrading, completely stop and restart the Web profile (merely refreshing the webpage will not rescan the plugin’s client.js).
dsh plugin --profile web add @anweat/dsh-browser@^0.1.8 dsh-web-search-pro@^0.1.8
# Or from a local directory / tarball:
dsh plugin --profile web add ../dsh-browser ./dsh-web-search-pro
# Restart (HMR is disabled in the web profile):
dsh --profile web
After installation, check in the following order:
browser_status: Confirm OpenCLI, Playwright/patchright runtime,automationMode, andusagePolicy.web_backend_status: Confirm if search, CLI, Agent Reach, and browser backends are ready.- Open
Settings → Plugins → Plugin Configuration: Ensure both the “Web Search Pro” and “Browser Automation” cards are loaded.
Dependencies on @deepseek-ai/* are published to npm (^0.1.0-rc.6). If the harness is a local source checkout, version numbers may vary; use dsh plugin --profile web add ./<path> and align versions in the profile’s pnpm-workspace.yaml before reinstalling.
Typical Usage¶
After installation and restarting, simply request the model to call the tools directly in a DSH session. Here is a reproducible example:
Please call web_backend_status to check the backend, then use web_search_pro to search for "DeepSeek Harness community feedback", specifying exa, fresh=true, and returning 8 sources.
Choose the entry point based on the scenario:
| Scenario | Recommended Entry | Note |
|---|---|---|
| Daily web search | web_search_pro |
Fallbacks according to configured order by default; pass fresh=true to force refresh |
| Semantic research, community opinions | web_search_pro + exa |
Uses native Exa API when API Key is available |
| Batch content for known URLs | web_exa_contents |
Requires EXA_API_KEY to be configured |
| Platforms like GitHub/Bilibili/Reddit | web_platform_search |
Reddit etc. require Chrome extension online |
| Post-login pages or private forums | browserBindings + AuthProfile |
Cookies saved in local storageState, read-only by default |
| Page redesign, lazy loading | platformRules or RulePack |
Prefer adjusting selectors; use bounded RulePack for waiting/clicking/scrolling |
Run web_backend_status first to determine if the backend is ready. When a single engine is specified, failures are returned as-is; when unspecified, automatic fallback occurs according to the engines order.
Configuration Tips¶
For daily configuration, prefer the DSH visual panel: Settings → Plugins → Plugin Configuration → Web Search Pro. Exa, Jina, and GitHub keys are written via DSH Credentials; the panel only shows “configured/not configured” and does not read plaintext keys back into the browser.
You can also hot-reload in $DSH_HOME/settings.yaml under the web-search-pro: section:
web-search-pro:
exaApiKeyEnv: EXA_API_KEY
jinaApiKeyEnv: JINA_API_KEY
engines: [ddg, bing, exa, seam, jina]
parallelEngines: false
ttlSeconds: 3600
searchMaxResults: 8
browserBindings:
zhihu:
authProfile: china-community
rulePack: zhihu-enhanced
The approval flexibility for browser tools is governed by dsh-browser.automationMode: read-only rejects write operations; standard (default) approves interactions, recipe writes, external scripts, etc.; unrestricted is recommended only for isolated test profiles. All modes retain domain, parameter, size, and step limit validations.
Use Cases and Caveats¶
Who is it for: Agent developers who perform daily web searches, semantic searches, or platform content fetching in the DSH Web profile, or need persistent caching and search history.
Permissions and Security: The plugin runs with the current dsh process permissions and can access local network, credentials, and browser login states. Before installation, read the source code and MIT license to confirm that automationMode, usagePolicy, and domain bindings meet your security expectations. External UserScripts run in the page’s main world, not a security sandbox; production profiles are advised to use standard mode.
Ecosystem Note: DSH’s philosophy is “everything is a plugin.” SkillHub (skillhub.cn) is a community plugin directory, with no official affiliation to DeepSeek / High-Flyer.
Links¶
- Community directory page: anweat/dsh-web-search-pro
- Source code and documentation: github.com/anweat/dsh-web-search-pro