Preface¶

The default dialogue model of DeepSeek Harness (DSH) does not possess visual capabilities. To “see” images or screens in a session, the common practice is to manually feed screenshots into other image recognition services and then copy the recognition results back to DSH—a disjointed process where the model also cannot autonomously decide when it needs to view an image.

dsh-vision is a community plugin maintained by linenxi-ctrl, which attaches a customizable external vision model to DSH, covering webpage image selection and recognition, automatic return of recognition results, and agent’s autonomous screenshot and recognition tools. The following explains how to implement this through installation, configuration, and usage instructions.

What This Is¶

dsh-vision (npm package name @linenxi-ctrl/dsh-vision, current version v0.2.6) adds the ability of an “external image recognition model” to DSH: enabling dialogue models without visual capabilities to understand images and screen content through external vision APIs.

The plugin is categorized as a “networking tool” in the SkillHub directory, with a GitHub repository linenxi-ctrl/dsh-vision having approximately 12 stars, licensed under MIT, targeting DSH 0.1.0-rc.6, and supporting Windows, macOS, and Linux.

Core Features¶

Webpage Configuration and Image Recognition¶

A draggable DeepSeek whale circular button appears at the bottom right corner of the page. Clicking it opens the configuration panel, where you can set the API address, key, model name, image recognition prompt (skill), proxy, and timeout for the external recognition model.

After selecting an image by clicking “Send Image” in the panel, the plugin first sends the image to the external recognition model; once recognition is complete, the recognized text is automatically injected as a message into the current session without manual copy-pasting, and DSH then responds based on this text. During recognition, a notification “External model is recognizing images” appears in the top right corner.

Model Autonomous Screenshot and Recognition¶

The plugin injects two tools for the agent: screenshot (for taking screenshots) and recognize_image (for recognizing images), along with corresponding prompts. The model can autonomously execute the “screenshot → recognize image → wait for result” process. For example, when a user says “Look at the error on my screen now,” the model calls the tools to complete the task.

Multi-Protocol Auto-Adaptation¶

It includes four built-in protocols: OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google Gemini. The protocol defaults to auto, which automatically detects based on apiBase; there is also a custom template protocol that adapts to long-tail interfaces via requestTemplate and responsePath.

Brief Principle of Operation¶

Image recognition requests are initiated on the host (Node) side, unaffected by browser CORS restrictions; client-side image selection uses the same-origin POST /api/vision/recognize, also without cross-origin issues.

[User clicks whale button to select image]                [Model calls tool]
      │                               │
      ▼                               ▼
  client converts to base64 and sends    screenshot tool captures screen
      │                               │
      ▼                               ▼
  POST /api/vision/recognize           recognize_image tool
      │                               │
      ▼                               ▼
  host plugin ctx.vision service ──► protocol auto-adaptation then calls external recognition API
      │                               │
      ▼                               ▼
  recognized text → auto-injects into current session    recognized text returns to model

Installation and Enablement¶

The DSH ecosystem follows the principle of “everything is a plugin”; SkillHub (skillhub.cn) is a community directory site, with no official affiliation with DeepSeek /幻方. It is recommended to browse the repository source code and confirm the MIT license before installation; plugins run with the current dsh process permissions, involving screenshot and network access.

Method 1: npm Installation (Recommended)¶

Requires Node.js 18+ and pnpm installed on the system. Install in DSH’s web profile:

dsh plugin --profile web add @linenxi-ctrl/dsh-vision

After installation, DSH will automatically reconcile cordis.patch.yml into the profile’s bundle layer, generally requiring no manual configuration file modification.

To enable the model’s autonomous screenshot and recognition, execute the agent tool plane configuration:

node ~/.dsh/profiles/web/node_modules/@linenxi-ctrl/dsh-vision/install.mjs

Method 2: Manual / Offline Installation¶

Download the zip from Releases and extract:

1. For Windows, double-click install.bat; for macOS/Linux, run bash install.sh;
2. If the script does not detect Node.js, it will automatically download a portable version from domestic mirrors (npmmirror / Huawei Cloud / Tencent Cloud) without requiring administrator privileges;
3. The script automatically copies the plugin, updates cordis.patch.yml, creates an agent preset vision, and sets it as default;
4. Restart DSH (close and re-run dsh web).

Update and Uninstall¶

Update: First uninstall the old version, then install the new version (cordis.patch.yml and preset will be automatically rebuilt).

Uninstall:

# For npm method, first remove the package
dsh plugin --profile web remove @linenxi-ctrl/dsh-vision

# Then clean up preset and settings (choose one)
node uninstall.mjs
# Or for Windows double-click uninstall.bat, for macOS/Linux run bash uninstall.sh

Configuration¶

Click the whale button at the bottom right corner of the page, or directly edit the vision section in $DSH_HOME/settings.yaml:

When protocol is auto, it identifies the protocol based on apiBase; you can also specify manually. Under the custom protocol, placeholders in requestTemplate must be written without quotes and support {{model}}, {{prompt}}, {{image}}, {{dataUrl}}, and {{mime}}.

Typical Usage¶

Send Image for Recognition: Open a session, click the whale button at the bottom right corner → click “Send Image” in the panel to select an image. After recognition, the text is automatically sent back to the current session.

Model Autonomous Recognition: Directly tell the model “Look at the error on my screen now,” and the model will call the screenshot tool to capture the screen, then call the recognize_image tool to recognize the image and continue reasoning.

Use Cases and Notes¶

Suitable for scenarios requiring image viewing, screen viewing within DSH, and allowing the agent to autonomously decide when to recognize images; the external model address and key are configured by the user, and can connect to OpenAI, Anthropic, Gemini, or interfaces adapted via custom.

Notes:

• When the model does not call the image recognition tool, confirm that tool.js has been added to the preset’s agent.cordis.yml, and the session uses that preset.
• Screenshot relies on system capabilities: Windows requires PowerShell (System.Drawing); macOS uses screencapture; Linux needs ImageMagick import.
• Common reasons for image recognition failure: incorrect apiKey (401/403), mismatch between apiBase and protocol (404), inaccurate protocol identification or incorrect responsePath in custom; if external network is inaccessible, fill in a proxy address in proxy.

Links¶

• SkillHub Directory Page: linenxi-ctrl/dsh-vision
• GitHub Repository: github.com/linenxi-ctrl/dsh-vision