In DeepSeek Harness (DSH), the web_fetch tool is responsible for fetching web page content. When handling complex single-page applications (SPAs) or micro-frontends, traditional browser engines consume excessive resources. The dsh-web-fetch-moli plugin provides a Rust-based lightweight solution that retains dynamic rendering capability while keeping memory usage to ~50-60MB.

What This Is

This is a DeepSeek Harness (DSH) plugin that provides a lightweight Moli backend for rendering SPAs and micro-frontends. It is maintained by GreyRaphael and follows the MIT license.

Core Features

  • Ultra-light memory footprint: A single instance uses about 50-60MB of memory, a reduction of more than 90% compared with standard Chromium/Playwright (~1GB). It is suitable for resource-constrained servers.
  • SPA and micro-frontend rendering: Supports complex micro-frontend architectures (such as qiankun and single-spa), achieving native layout calculation through CDP and CSP bypass techniques.
  • Dual-backend mode:
    • local (default): Automatically manages a local moli serve daemon.
    • cdp: Connects to a remote Moli or Chromium CDP service.
  • Content denoising: Integrates the LinkeDOM + Readability + mdream pipeline to extract body content and generate LLM-optimized GitHub Flavored Markdown.
  • Cloudflare challenge handling: Can detect and wait for Cloudflare verification to pass without manual intervention.

Installation and Enabling

The plugin supports automatic downloading of the Moli binary.

  1. Install the plugin:
    dsh plugin --profile web add github:GreyRaphael/dsh-web-fetch-moli
  1. After installation is complete, restart dsh web to load the plugin.

Typical Usage

In DSH configuration or code, specify the moli provider and set the backend and denoising parameters.

// 在 DSH 上下文中配置
ctx.web.fetchProvider = 'moli';

// 配置选项
// backend: 'local' 或 'cdp'
// denoise: true (启用内容提取)

Use Cases and Notes

  • Environment requirements: Node.js >= 22 is required.
  • Automatic download: On first use, the plugin automatically downloads the Moli binary matching the system architecture into the cache directory.
  • Runtime permissions: The plugin runs with the permissions of the current DSH process. Before installation, it is recommended to review the plugin source code and license.

Summary

The plugin achieves a balance between memory efficiency and dynamic rendering capability, making it suitable for high-concurrency scraping of dynamic pages. For more information, refer to the GitHub repository.