Introduction

DSH (DeepSeek Harness) extends capabilities using a “everything is a plugin” approach. The community directory is an independent site with no official affiliation with DeepSeek / Huanfang, nor is it equivalent to the official app store.

For agents using only pure text models, images cannot typically be directly entered into the context. If you want the model to see images, read text within images, and locate interface elements, you often need to additionally connect a vision backend. dsh-img is a DeepSeek Harness plugin aimed at providing analyze_image (image Q&A, OCR, and layout understanding) and vision_ground (target element localization) for pure text models; simultaneously, it provides a set of key-free local pixel tools.

The following introduces its core capabilities, installation methods, and usage boundaries.

What is it

  • Plugin ID: gmleong/dsh-img
  • Positioning: DeepSeek Harness vision plugin, adding image understanding capabilities to pure text models
  • License: MIT
  • Prerequisites: Node.js >= 20, and dsh core is installed

This plugin can process images via a vision backend or call local pixel tools to complete partial image processing. The vision backend defaults to Zhipu GLM-4V-Flash, with Tongyi Qwen-VL as an alternative, and also supports Ollama local vision models and any OpenAI-compatible vision endpoint.

Core Capabilities

Vision Backend Tools

If using a vision backend that requires an API key, the key must be injected into the dsh process.

  • analyze_image: Used for image Q&A, OCR, and layout understanding
  • vision_ground: Used for target element localization

Key-Free Local Pixel Tools

The following local pixel tools require no key:

  • vision_crop
  • vision_pixel_diff
  • vision_colors
  • vision_ocr
  • vision_trace
  • vision_extract_foreground

Backends, Caching, and Runtime Environment

  • Supports preset single backend configuration
  • Supports backends multi-backend automatic fallback
  • Recognition results are cached to ~/.dsh/dsh-img-cache/ based on the hash of image content and the question
  • The web settings page allows visual modification of preset and entry of apiKey
  • Supports using tools under dsh --profile headless
  • Supports .png .jpg .jpeg .webp .gif .bmp

Installation and Enablement

First, confirm that the current environment meets Node.js >= 20 and that the dsh core is available. The plugin will run with the permissions of the current dsh process; it is recommended to check the source code and license before installing.

Installation to web profile

The command below installs dsh-img to the corresponding profile in ~/.dsh/profiles/web:

cd ~/.dsh/profiles/web && pnpm add dsh-img --registry https://registry.npmjs.org

Configure Vision Backend Key

If only using local pixel tools, you can omit configuring the key. If using vision backend tools that require an API key, you need to prepare the corresponding API key. The following uses the default Zhipu GLM-4V-Flash ZHIPU_API_KEY as an example:

export ZHIPU_API_KEY=your-key-here

This key must be injected into the dsh process. Exporting it only in the current shell may not take effect for already running dsh processes.

Restart dsh web

After restarting, start the dsh process carrying the key:

pkill -f "dsh web"
ZHIPU_API_KEY=your-key-here dsh web

headless profile

If you want to use it under dsh --profile headless, install it once in that profile:

dsh plugin --profile headless add dsh-img

After that, tools can be used under dsh --profile headless. Local pixel tools require no key; image recognition tools can use Ollama local vision models or API keys as needed.

Backends and Configuration

You can configure backends in cordis.patch.yml. Use preset for a single backend, and the backends list for multi-backend fallback.

  • preset: ollama: Use local Ollama vision model
  • backends: Configure multi-backend fallback chain

If using an API key backend, the key must be injected into the dsh process; the web settings page also allows filling in apiKey.

Typical Usage

Image Q&A

After the installation steps above, after creating a new session, you can directly paste the image into the chat box, or tell the agent:

用 analyze_image 看一下 ./screenshot.png 里写了什么

Local Pixel Tools

Local pixel tools require no key. After the steps above, you can call vision_crop, vision_pixel_diff, vision_colors, vision_ocr, vision_trace, vision_extract_foreground as needed in the session.

Target Element Localization

If you need the agent to find a specific target element in the image, you can let it call vision_ground.

Settings Page

The web settings page allows visual modification of preset and entry of apiKey.

Applicable Boundaries and Notes

The following boundaries come from the plugin documentation; check if they match before selection:

  • vision_trace: Not suitable for colored illustrations, photos, or images with gradients
  • vision_extract_foreground: Suitable for uniform backgrounds, not suitable for complex or textured backgrounds
  • vision_pixel_diff: Suitable for UI screenshots of the same size, and comparing design draft fidelity
  • vision_ocr: Suitable for printed text and screenshot text; recognition rates may drop for handwriting and artistic fonts

Other notes:

  • API key must be injected into the dsh process; local pixel tools require no key
  • Recognition results will be cached to ~/.dsh/dsh-img-cache/
  • Supports .png .jpg .jpeg .webp .gif .bmp
  • The README troubleshooting mentions that the old version <=0.2.3 has a dual-instance bug; you can upgrade using the command below:
pnpm add dsh-img@latest
  • Old sessions may be affected by the old bug; just create a new session

References

  • GitHub: https://github.com/gmleong/dsh-img
  • Directory Page: https://www.skillhub.cn/plugins/gmleong/dsh-img

The directory page is a community directory page and is not equivalent to the official app store.