Introduction

The web_fetch tool in DeepSeek Harness (DSH) fetches the raw HTML of a web page by default. For modern single-page applications (SPAs) or content that requires client-side JavaScript rendering, this direct fetching often yields only an empty shell. The dsh-web-fetch-playwright plugin provides a backend for web_fetch based on Playwright and Chrome DevTools Protocol (CDP), allowing pages to be rendered in a real browser and then cleaned and converted to Markdown.

Plugin Introduction

This plugin is maintained by the user chendefine, belongs to the online tools category, and is released under the MIT open-source license. Its core role is to solve the content-missing issue encountered by the native DSH fetching tool when handling dynamic web pages, while providing a standardized Markdown output pipeline.

Core Features

Real Browser Rendering

The plugin launches a real browser instance (local Playwright or a remote CDP connection) and loads the page from the perspective of a user’s browser. This can capture content generated dynamically by JavaScript, resolving the issue that the native fetching tool cannot obtain SPA content.

Content Denoising and Conversion

The rendered content goes through a standardized denoising pipeline:
1. Readability: Extracts the main body content and ignores navigation, sidebars, and ads.
2. DOMPurify: Removes layout-interfering tags and scripts.
3. Turndown + GFM: Converts HTML to Markdown and applies GitHub-style formatting.
4. Size limit: Automatically replaces inline data: images (such as screenshots embedded by build tools) with placeholders to prevent the main content from becoming too large.

Dual Backend Support

The plugin provides two runtime modes:
* Local mode: Launches Chromium directly on the local machine. This can be done by configuring a path, using the playwright CLI in $PATH, or using the built-in playwright-core.
* CDP mode: Connects to a browser instance that is already running and has a debugging port enabled (--remote-debugging-port).

Session Management

In CDP mode, two context modes are supported:
* Shared mode: Uses the real profile of the remote browser and retains Cookies and login state.
* Isolated mode: Creates a new context for each fetch and retains no state.

Live Configuration and Budget Control

The plugin includes a built-in configuration card that can be adjusted without restarting:
* Concurrency control: Limits the number of concurrent rendering tasks (default: 4 for local mode and 50 for CDP mode). Requests that exceed the limit fail fast and prompt a retry.
* Resource limits: Sets the request deadline, the character limit for the main text, and the limit for image subrequests.
* Challenge waiting: Automatically identifies verification pages such as Cloudflare and waits for browser verification to pass within the same tab, avoiding returning the verification page directly as content.

Installation and Enabling

The plugin is prebuilt, so manual compilation is not required during installation. Ensure the environment meets the requirements (DSH web profile, Node.js ≥ 20).

  1. Run the installation command:
    dsh plugin --profile web add dsh-web-fetch-playwright
  1. After installation, you must restart dsh web to load the plugin.

Configuration and Usage

After restarting, find “Plugin configuration” in the DSH settings interface, where you can see the “Playwright Web Scraping” card. Here you can:
* Switch the backend (local or CDP).
* Set the browser path or CDP endpoint.
* Enable/disable denoising.
* Adjust the concurrency limit and Cloudflare wait time.

Notes

  • Environment dependencies: Local mode requires Playwright to have installed browser binaries or an available browser path.
  • CDP dependency: Using CDP mode requires ensuring that a browser process is running and listening on the debugging port.
  • Permissions and security: The plugin runs with the permissions of the current DSH process. It is recommended to review the source code before installation to confirm that its behavior meets expectations.

More Information

  • Listing page: https://www.skillhub.cn/plugins/chendefine/dsh-web-fetch-playwright
  • GitHub: https://github.com/chendefine/dsh-web-fetch-playwright