Introduction¶

DeepSeek Harness (DSH) agents can search and obtain links with summaries, but to read a URL into clean, directly reason-able text still requires a dedicated reading step. The official tool-web’s web_fetch converts the entire HTML page to Markdown, preserving navigation, ads, and sidebars, with a default limit of 200,000 characters, which can easily fill the context. There are also several community web scraping plugins, but each has trade-offs in aspects like text cleaning, Chinese encoding, and default output length.

Here we introduce dsh-read-url (maintainer 2672243194, GitHub 14 stars, category: Web Tools). It is designed for DSH’s URL reading scenario: it scrapes web pages, JSON APIs, and RSS/Atom feeds, automatically detects encoding, extracts main content, and returns it as compact plain text or Markdown, with a default limit of 6,000 characters, including caching and offset-based continuation.

What is this¶

dsh-read-url is a URL reader plugin for DeepSeek Harness, with zero runtime dependencies (Node 20+ has built-in fetch/decoding/extraction), requiring no API Key and no self-hosted server. After installing it into DSH, agents can use tools like read_url to read any http(s) link into cleaned text, rather than piling up the entire HTML page.

The plugin follows DSH’s official capability seam design: network requests are prioritized through ctx.web.fetch(), session caching is registered under ctx.effect and cleaned up on uninstall, tool timeout is coordinated via timeoutMs and exec.signal, and the model-side output is compact plain text without needing JSON parsing again.

Core Features¶

Main Text Extraction and Token Control¶

The plugin locates article/main content containers in HTML, strips noise like navigation and footers, and by default returns up to 6,000 characters, with support for truncation aligned by paragraphs. For long articles, you can use offset to continue reading from the cache, avoiding re-sending already-read paragraphs. Within the same session, a 5-minute TTL cache is used; repeated readings of the same URL will be marked (cached).

Multi-format URLs¶

Besides regular web pages, the plugin natively handles JSON APIs (compact rendering, long values trimmed) and RSS/Atom (item lists + feedCount). Paginated articles (novels, news, forums, etc.) are automatically concatenated by default for up to 3 pages.

Chinese and Multi-language Encoding¶

It automatically detects and normalizes encodings such as UTF-16 BOM, GBK/GB2312, UTF-8, Big5, Shift-JIS, with mojibake fallback. Compared to some similar plugins, its README notes more complete handling of GB2312 and Chinese sites.

Page Metadata and Redirects¶

Beyond the main text, the plugin collects fields like published and author via a three-level fallback from meta → schema.org JSON-LD → byline (v1.3.0 onwards allows JSON-LD to supplement news site publication times). When encountering shell pages with <meta http-equiv="refresh">, anti-hotlinking redirects, or JS-SPA entry points without JS, it automatically follows to the real page (up to 3 hops to prevent loops); pages declaring <base href> will resolve relative links accordingly.

Parallel and Batch Operations¶

From v1.1.0 onwards, all four tools declare isConcurrencySafe, allowing agents to initiate multiple read_url calls in parallel. read_url_batch reads 1–10 URLs at once with concurrency of 4; failure on a single page does not affect other results.

Network Layer¶

Requests are prioritized through the ctx.web capability seam; when the seam is unavailable, it falls back to the global fetch. When a proxy is configured, direct connection and proxy curl race, with the faster one winning. README tests mention: using a full browser UA, some sites (like Baidu) can return complete static content, while the official web_fetch in the same environment might get a degraded page due to TLS/UA characteristics.

Installation and Enabling¶

Install from GitHub (recommended in README for easy updates):

npx @deepseek-ai/dsh plugin --profile web add github:2672243194/dsh-read-url

For local development, you can point to a directory:

npx @deepseek-ai/dsh plugin --profile web add ./dsh-read-url

After installation, restart DSH (Web or TUI), and confirm dsh-read-url is enabled in Settings → Plugins. The current package.json version is 1.3.1, license MIT, requires Node >= 20.

Typical Usage¶

After installation, simply use natural language to have the agent read a URL, for example:

Read https://example.com/article and summarize the key points
Read https://docs.example.org/guide in markdown mode
Read these URLs together and compare their viewpoints: <url1> <url2> <url3>

Single Page Reading read_url¶

Output example (from README tests):

title: 新闻中心首页_新浪网
charset utf-8
(chars 800/12398 — 截断，offset 续读)

Batch Reading read_url_batch¶

Pass in 1–10 URLs at once, clean them in parallel, and merge into one compact report. Failed URLs are marked separately, for example:

读取 2/4 页成功，2 页失败
--- 阮一峰的网络日志 (491 字符) ---
--- Example Domain (127 字符) ---
[失败] https://zh.wikipedia.org/... — Fetch failed: HTTP 403 ...

Long Article Continuation¶

For articles exceeding maxChars, first read the initial segment with default parameters, then increase offset to continue reading subsequent segments; cached hits avoid re-fetching.

Use Cases and Notes¶

Suitable for: DSH agent scenarios that need to read web pages, API responses, or RSS entries into the context for summarization, comparison, and research; Chinese sites, old encoding pages, paginated long articles; scenarios where token consumption needs control and avoiding full-page Markdown.

Notes:

1. The plugin initiates network requests with the current DSH process permissions. Before installing, please read the GitHub source code and MIT license to ensure it meets your security policy.
2. Failures like anti-scraping, 403, timeout, etc., are returned as-is; in batch reading, a single URL failure does not block other URLs.
3. The community directory SkillHub Plugin Page is an independent community site with no official affiliation to DeepSeek / High-Flyer; plugin lists and star counts are based on GitHub.

Summary¶

dsh-read-url fills the DSH agent’s step of “reading links into clean text”: zero dependencies, no API Key, default 6,000-character compact output, supporting encoding detection, pagination concatenation, cache continuation, and batch parallelism. If you are already using DSH for web research, you can install and try it using the commands above.

• Community directory: https://www.skillhub.cn/plugins/2672243194/dsh-read-url
• GitHub: https://github.com/2672243194/dsh-read-url